Title: Extragradient Method for $(L_0, L_1)$-Lipschitz Root-finding Problems
Venue: NeurIPS 2025
Accepted: yes
Keywords: optimization

extragradient

min-max

variational inequality problem
Abstract: Introduced by Korpelevich in 1976, the extragradient method (EG) has become a cornerstone technique for solving min-max optimization, root-finding problems, and variational inequalities (VIs). Despite its longstanding presence and significant attention within the optimization community, most works focusing on understanding its convergence guarantees assume the strong $L$-Lipschitz condition. In this work, building on the proposed assumptions by Zhang et al. [2019] for minimization and  Vankov et al. [2024a] for VIs, we focus on the more relaxed $\alpha$-symmetric $(L_0, L_1)$-Lipschitz condition. This condition generalizes the standard Lipschitz assumption by allowing the Lipschitz constant to scale with the operator norm, providing a more refined characterization of problem structures in modern machine learning. Under the $\alpha$-symmetric $(L_0, L_1)$-Lipschitz condition, we propose a novel step size strategy for EG to solve root-finding problems and establish sublinear convergence rates for monotone operators and linear convergence rates for strongly monotone operators. Additionally, we prove local convergence guarantees for weak Minty operators. We supplement our analysis with experiments validating our theory and demonstrating the effectiveness and robustness of the proposed step sizes for EG.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper extends extragradient methods to variational inequality problems with relaxed assumptions. Most existing work assumes an L-Lipschitz condition on the operator. This paper relaxes this to the more general $\\alpha$-symmetric $(L_0,L_1)$-Lipschitz condition. This condition has been studied in the past under a wide range of contexts and captures a number of basic loss functions that the standard L-Lipschitzness cannot capture. \n\nThe paper has a few main contributions. The first is that they tighten existing analysis based on extragradient methods. Concretely, consider the function $f(x) = \\log (1+\\exp(-a^{\\top}x))$. This function is $\\|a\\|^2$ Lipschitz under the standard definition but it is also $\\alpha$-symmetric $(L_0,L_1)$-Lipschitz with $\\alpha = 1$, $L_0 = 1$, $L_1 = \\|a\\|$. When $\\|a\\|>>1$ then these new methods significantly improve upon existing ones. The second contribution is that they give the first convergence analysis of EG methods for weak minty variational inequality problems under this generalized smoothness. The third contribution is that the step size considered for the EG algorithm is new and is adaptive. The rates of convergence in all cases improve upon previous works.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nThe paper provides a new EG algorithm and the only one which works for VIPs under the generalized smoothness condition. Their algorithm improves upon previous works. The key improvement is via their new adaptive step size.\n\nWeaknesses:\n\nI am not entirely sure of the exact algorithm used in implementations and also the one analyzed. Maybe an algorithm box would help. Also the paper only talks about EG methods. It is not clear if there are other existing works based on other methods that show convergences for these kinds of problems.\n\nQUESTIONS:\n1. Can the authors give some more information on the methods used to solve these generalized smooth problems? It seems like they are focussed entirely on EG methods, which I am not sure give the best rates of convergence compared to other prox point methods.\n2. A complete algorithm box would certainly help.\n3. It is not clear what the main challenge is with generalizing the EG methods to this setting. Maybe a bit more description on this would help.\n\nLIMITATIONS:\nI am not sure I see where these are addressed explicitly.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the extragradient method (EG) for variational inequality problems under the assumption that the operator in question satisfies the recently introduced $\\alpha$-symmetric $(L_0,L_1)$-Lipschitz condition. Under this assumption, a new step-size rule for EG is proposed. Convergence of EG under various additional assumptions on F (strongly monotone, monotone, and weak Minty condition) are shown. The paper is rounded out by a short experimental section.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n - Writing is clear and easy to follow\n - Theorems are presented in an easy to digest manner. The proofs are well-structured.\n - Variational inequalities are a topic of perennial importance to the optimization/ML community, so this work is likely to be of interest.\n - I like the extension to \"weak Minty\" operators, which is not something I've seen before. It's good to see some results under assumptions weaker than monotonicity.\n\n### Weaknesses\n - In my opinion, the biggest weakness of this paper pertains to its scope/significance. Specifically:\n\t - The authors focus on unconstrained variational inequalities, which might more appropriately be called root-finding problems. This makes comparisons with prior work, particularly [Vankov 2024] somewhat misleading. \n\t - The authors focus only on extragradient. It would be nice to see if this analysis applies to similar methods, e.g. KM iteration, optimistic gradient method.\n - It would also be good to see some discussion of, and comparison to Halpern-style methods [Park & Ryu].\n - Although the new step-size rule is nice, it requires knowing three parameters ($L_0,L_1,\\alpha$) of $F$. Step-sizes based on the Lipschitz assumption just require one parameter $L$. This should probably be discussed somewhere.\n\n### Typos etc\n - line 674: \"maximum eigenvalue $\\sqrt{2}$ \" should be $2$. \n - line 690--691: \"We add the details of this example to 691 Appendix ??.\" is probably a typo.\n - lines 707--708 duplicate said in lines 705--706.\n - line 714: Can you change the label \"Proposition C.1.\" --> \"Proposition 3.1\" ? Same for all Theorem/Proposition statements in the appendix.\n - In the proof of Theorem 3.2: the (7) over the first inequality should be (8). Similarly for the proof of Theorem 3.4\n - In final line of proof of Theorem 3.2 suggest using square brackets $[\\ldots ]$ in addition to parentheses $(\\ldots)$ as the final factorization is a bit challenging to parse.\n - line 729: I think you want $x=x_{*}$ so that $F(x_{*}) = 0$.  Same for line 761\n\n### References\n[Vankov 2024]: _Generalized Smooth Variational Inequalities: Methods with Adaptive Stepsizes_ Vankov _et al_ (ICML2024)\n\n[Park & Ryu]: _Exact Optimal Accelerated Complexity for Fixed-Point Iterations_ Park & Ryu (ICML2022)\n\n\n-----\n\nAfter author-reviewer discussion, I raised my score 3 --> 4.\n\nQUESTIONS:\n- How easily do your results extend to the true VI problem, where there is a non-trivial constraint set?\n - Could you apply your analysis to, say, the optimistic gradient method?\n - The definition of $\\alpha$-symmetric is new to me and very interesting. It feels somehow 'dual' to Holder continuity. Is there a reason for choosing to focus on $\\alpha$-symmetry as opposed to Holder continuity in this setting?\n - I'm not familiar with the weak Minty condition. Could you provide some intuition for why this is interesting? Is there an analogue of this condition in the convex optimization world? (like the analogue between (strong) monotonicity and (strong) convexity)?\n - In the proof of Theorem 3.4, in the ninth line, I believe there is a sign error. I think $-\\gamma^2_k\\|F(x_k) - F(\\hat{x}_k)\\|^2$ should be $+\\gamma^2_k\\|F(x_k) - F(\\hat{x}_k)\\|^2$. It seems this will affect some of the constants in the statement of the theorem. Relatedly, the first 9 lines of the proofs of Theorem 3.4 and Theorem 3.2 are the same, so I suggest you package them into a lemma.\n - Similarly, there is duplication in the first several lines of the proofs of Theorems 3.5 and 3.7 which might benefit from being packed into a lemma.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the Extragradient (EG) method under the $(L_0, L_1)$-Lipschitz condition. This is a generalized and perhaps more practical condition compared to the traditional $L$-Lipschitz assumption commonly used when studying the EG method. The paper provides sublinear convergence under the monotone assumption, linear convergence under the strongly monotone assumption, and local convergence under the weak Minty (non-monotone) assumption. Together with the theoretical analysis, the paper presents numerical experiments that demonstrate the effectiveness of the proposed step sizes.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n- This paper starts by introducing the motivation for the $(L_0, L_1)$-Lipschitz condition they are considering, and they did a good job on this point. To the best of this reviewer's understanding, they needed to come up with new observations related to EG for this purpose, and the reviewer believes these are meaningful observations worth sharing. They also provided persuasive examples.\n\n- They provide theoretical convergence analyses under popular assumptions—monotone, strongly monotone, and weak Minty—commonly considered for the EG method. In the context that these results are generalizations of results already established in function minimization problems or setups with only an $L$-Lipschitz condition, they could be seen as somewhat expected. Nonetheless, the reviewer believes they still contribute meaningfully and represent work that needed to be done for the field.\n\n- The paper provides meaningful equivalent formulations of the core assumptions. They seem useful in carrying out the analyses and are also meaningful on their own.\n\n\n**Weakness**\n\n- **W1.** To the best of this reviewer's understanding, most of the techniques used in the theoretical analyses are extended versions of the techniques used for function minimization or merely the $L$-Lipschitz setup. The extension may not be trivial, but it seems like something that could be expected. However, this reviewer does not consider this a weakness, though it may have contributed to a slightly more moderate overall assessment.\n\nQUESTIONS:\n- **Q1.** In lines 185–187, the authors state that the results in their paper recover the result in [Gorbunov et al., 2022b]. However, as the title of the cited paper suggests, it provides 'last-iterate' convergence, rather than an ergodic convergence rate. The reviewer is confused, as the results provided in the submitted paper are ergodic convergence rates (Theorem 3.5). Can the authors clarify this point?\n\n- **Q2.**  Is there a significant technical difficulty compared to the function minimization problem or the standard $L$-Lipschitz setup that the authors would like to highlight, which the reviewer might be overlooking?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work analyzes the convergence rate of extragradient method under the α-symmetric (L0,L1)-Lipschitz continuity, a smoothness condition that generalizes both the classical Lipschitz assumption and the (L0,L1)- Lipschitz framework (where α = 1) introduced by [1]. The authors claim that the classical global L-smooth condition is often too restrictive for certain optimization problems, and instead propose an adaptive method that performs effectively under the α-symmetric assumption. In Section 3, they establish convergence guarantees under three different assumptions on the monotone operator F, which matches known upper bounds in certain extreme cases. Specifically, their Lyapunov-style analysis results in an adaptive stepsize that ensures the distance to the solution decreases across iterations. Finally, the authors present experimental results that support and validate their theoretical findings.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n - The results are both strong and straightforward. The authors establish convergence guarantees across all six scenarios by varying the parameter α and the monotonicity assumptions on the operator. They successfully extend the analysis of prior works (e.g., [2]) to the extragradient method. Most of the convergence results are novel and contribute meaningfully to the theory of the extragradient method.\n - In my opinion, this paper is written in a very clear and reader-friendly manner. Six different cases in Section 3 as well as their proof in Appendix are presented in a nice parallel structure, making both the theory and its proof easy to follow. This also makes the paper a valuable reference for future research upon these findings.\n- The class of operators considered in this work is broad enough to encompass many practical problems. Combined with the significant role of the extragradient method in optimization, these results broaden its applicability to a wide range of optimization problems. The broad nature of this work opens up future research directions, such as extensions to other momentum-based methods or the development of alternative stepsize schemes.\n\n\nWeaknesses\n -  The proposed method is not fully “adaptive,” as the step sizes still depend on global problem parameters (L0,L1, α). I understand that the primary goal of this work is not to develop an adaptive method, but to establish convergence guarantees under a new problem assumption. However, the contribution could have been stronger if the method had incorporated local estimates of global parameters. For example, [3] proposes an approach that estimates L0 locally. In fact, the importance of choosing proper (L0,L1) is also evident in the experiments, as the performance appears to vary depending on their choice. \n - As previously mentioned, the core content is well-structured. However, the paper lacks a dedicated prior work section, either in the main text or the appendix. As a result, it is difficult to clearly understand how this work relates to existing literature or how it compares to previous methods developed under similar assumptions. It seems that most of related works cited in the main text appear to focus on the (L0,L1)-smooth and α-symmetric assumption, rather than on method-level analyses.\n - I found the experimental section to be somewhat limited. While the paper is primarily theoretical—so this may not be a major concern—it seems that most experiments were conducted on synthetic data. Including experiments on more practical problems or scaling up the current ones could help further demonstrate the method’s effectiveness in realistic settings.\n\n\nTypos\n• Line 3: Variational inequalities problems → Variational inequality problems\n• Line 45 : Memorys → Memories\n• For theorems 3.2,3.4,3.5,3.6,3.7,and 3.9: satisfy → satisfies\n• x∗ not defined in Assumption 1.3.\n\nQUESTIONS:\nIn addition to the mentioned weaknesses, I have some questions:\n• What are the problem parameters (L0,L1, α) of equation (19)? Does it matches the parameteres that lead to good performance in Figure 3c?\n• The choice of stepsize differs significantly depending on whether α = 1 or not. However, in practice, α is typically unknown. Given only a problem instance, how should one select an appropriate stepsize? Would it be necessary to sweep over possible α values or estimate α in advance? Is it possible to design an algorithm that works reliably for all values of α without prior knowledge?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper analyzes the extragradient method for min-max and variational inequality problems under relaxed assumptions. In particular, the common (and often strong) L-Lipschitz continuous assumption is relaxed to an $\\alpha$-symmetric $(L_0, L_1)$-Lipschitz continuity assumption, which allows the Lipschitz constant to scale with the operator norm. Under this more general condition, convergence analysis is given for strongly monotone and monotone variational inequalities. In addition, a local convergence rate is given for variational inequalities which satisfy the weak Minty variational inequality condition. Experimental results are provided that validate the effectiveness of a newly proposed adaptive stepsize based on $(L_0, L_1)$-Lipschitz continuity.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: This paper analyzes the highly popular extragradient method for variational inequalities under a relaxed set of assumptions. The convergence results improve upon known results in the strongly monotone case and are the first convergence results under the $(L_0, L_1)$-Lipschitz continuity condition for monotone operators. In addition, the work provides the first convergence results for weak Minty variational inequalities, though the convergence is only local. These results are proven when a novel adaptive stepsize is used. This stepsize strategy is validated numerically on a number of simulated experiments.\n\nWeaknesses: In the strongly monotone case, the improved convergence rate comes from an argument that shows that the exponential dependence on the distance of the initial point to the solution can be moved into a constant factor, as opposed to multiplying a $\\mathcal{O}(\\log(\\epsilon^{-1}))$ factor (where $\\epsilon$ is the optimization tolerance), as it is in prior work. While this is an improvement, it is a relatively minor change that is only relevant for extremely small values of $\\epsilon$. In addition, the convergence rate for weak Minty variational inequalities requires that $\\rho$ is sufficiently small relative to $L_1 e^{\\|x_0-x_*\\|}$, which essentially requires that either $\\|x_0-x_*\\|$ is extremely small for any non-monotone problem or $L_1 = 0$, in which case the generalization to $(L_0, L_1)$ smoothness is irrelevant.\n\nQuality: The focus on extending convergence results for extragradient methods beyond is a reasonable focus, as current results rely on restrictive assumptions that may not hold in practical applications. The relaxed assumptions provide a nice generalization and the results appear to be technically sound. However, a number of the results are only minor improvements over previous works or require extremely strong assumptions that limit their applicability.\n\nClarity: The paper is well written and well organized. There are a number of examples given to motivate the modified assumptions and the relationship with prior work is clearly stated. The limitations of the work could be stated more clearly, as they are spread throughout the main convergence results. It would also be nice if there was a brief discussion or conclusion section added to this work discussing potential future directions.\n\nSignificance: This paper provides some of the first results for extragradient methods under the $(L_0, L_1)$ smoothness assumption. However, the strongly monotone result is only a mild improvement on prior work and the weak Minty variational case only applies to an extremely tight neighborhood of the solution. Therefore, the main contributions are mostly limited to the monotone case.\n\nNovelty: The main change from prior work is the use of the $(L_0, L_1)$ smoothness assumption and applying this to the monotone and weak Minty variational inequality cases. While this is novel, it feels more incremental than a major new direction of research.\n\nQUESTIONS:\nIs the exponential dependence on $\\|x-y\\|$ tight in Proposition 3.1? Are there any conditions under which this exponential dependence may vanish (other than $L_1 = 0$)? My curiosity stems from the fact that this appears to cause major issues throughout the paper, such as for the strongly monotone case (where this only shows up in a constant now, but still persists), but especially in the weak Minty variational inequality case, where the neighborhood of convergence is directly dependent on this exponential dependence, which seriously restricts the applicability of the result. If it was possible to find reasonable conditions under which this exponential dependence vanished or was more controlled, it could significantly improve my evaluation score, as it would significantly improve the result in the weak Minty case.\n\nFor the weak Minty experiment, do the methods struggle when started far from the solution? I am curious if the authors observe the issue of only obtaining local convergence using their adaptive stepsize or if this is just an artifact of the analysis.\n\nIn the theoretical results, the adaptive stepsize is dependent on the constant $\\alpha$ in the $\\alpha$-symmetric assumption, however in the numerical experiments, $\\alpha = 1$ is used throughout. Do the experimental results change much when $\\alpha$ varies away from 1? In particular, for the monotone case, the introduction suggests that this problem should use an $\\alpha < 1$, which partially motivates the assumption, but this is not used in the experiments, which raises questions about the value behind this relaxation of $\\alpha$.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}