Original: Solving Constrained Variational Inequalities via a First-order Interior Point-based Method
Revised: Solving Constrained Variational Inequalities via a First-order Interior Point-based Method

Original: author names withheld

Original: Under Review for OPT

Original: Abstract
Revised: Abstract

Original: We focus on the open problem to develop a first-order method that can solve constrained variational inequality (cVI) problems when given general constraints.
Revised: We focus on the open problem to develop a first-order method that can solve constrained variational inequality (cVI) problems when given general constraints.

Original: We generalize the alternating direction method of multipliers (ADMM) method and combine it with interior-point approaches, yielding a first-order method that we refer to as A DMM-based interior-point method for cVI s (ACVI).
Revised: We generalize the alternating direction method of multipliers (ADMM) method and combine it with interior-point approaches, yielding a first-order method that we refer to as A DMM-based interior-point method for cVI s (ACVI).

Original: We provide convergence guarantees for ACVI in two general classes of problems: (i) when the operator is ξ -monotone, and (ii) when it is monotone, some constraints are active and the game is not purely rotational.
Revised: We provide convergence guarantees for ACVI in two general classes of problems: (i) when the operator is ξ -monotone, and (ii) when it is monotone, some constraints are active and the game is not purely rotational.

Original: When the operator is, in addition, L -Lipschitz for the latter case, we match known lower bounds on rates for the gap function of O (1 / √ K ) and O (1 /K ) for the last and average iterate, respectively.
Revised: When the operator is, in addition, L -Lipschitz for the latter case, we match known lower bounds on rates for the gap function of O (1 / √ K ) and O (1 /K ) for the last and average iterate, respectively.

Original: To our knowledge, this is the first first -order interior-point method for the general cVI problem that has a global convergence guarantee.
Revised: To our knowledge, this is the first first -order interior-point method for the general cVI problem that has a global convergence guarantee.

Original: Empirical analyses demonstrate clear advantages of ACVI over common first-order methods.
Revised: Empirical analyses demonstrate clear advantages of ACVI over common first-order methods.

Original: In particular, (i) cyclical behavior is notably reduced as our method approaches the solution from the analytic center, and (ii) unlike projection-based methods that zigzag when near a constraint, ACVI efficiently handles the constraints.
Revised: In particular, (i) cyclical behavior is notably reduced as our method approaches the solution from the analytic center, and (ii) unlike projection-based methods that zigzag when near a constraint, ACVI efficiently handles the constraints.

Original: 1. Introduction
Revised: 1. Introduction

Original: We are interested in the constrained variational inequality problem [75]:
Revised: We are interested in the constrained variational inequality problem [75]:

Original: [Equation]
Revised: [Equation]

Original: where X is a subset of the Euclidean n -dimensional space R n, and where F : X (cid:55)→ R n is a continuous map.
Revised: where X is a subset of the Euclidean n -dimensional space R n and where F : X (cid:55)→ R n is a continuous map.

Original: Finding (an element of) the solution set S ⋆ X ,F of cVI is a key problem in multiple fields such as economics and game theory.
Revised: Finding (an element of) the solution set S ⋆ X ,F of cVI is a key problem in multiple fields such as economics and game theory.

Original: More pertinent to machine learning, CVIs generalize standard single-objective optimization, complementarity problems [17], zero-sum games [70, 82] and multi-player games.
Revised: More pertinent to machine learning, CVIs generalize standard single-objective optimization, complementarity problems [17], zero-sum games [70, 82] and multi-player games.

Original: For example, solving cVI is the optimization problem underlying reinforcement learning [e.g., 62]—and generative adversarial networks [37].
Revised: For example, solving cVI is the optimization problem underlying reinforcement learning [e.g., 62]—and generative adversarial networks [37].

Original: Moreover, even when training one set of parameters with one loss f , that is F ( x ) ≡ ∇ x f ( x ) , a natural way to improve the model’s robustness in some regard is to introduce an adversary to perturb the objective or the input, or to consider the worst sample distribution of the empirical objective.
Revised: Moreover, even when training one set of parameters with one loss f , that is F ( x ) ≡ ∇ x f ( x ) , a natural way to improve the model’s robustness in some regard is to introduce an adversary to perturb the objective or the input, or to consider the worst sample distribution of the empirical objective.

Original: As has been noted in many problem domains, including robust classification [54], adversarial training [76], causal inference [16], and robust objectives [e.g., 71], this leads to a min-max structure, which is an instance of the cVI problem.
Revised: As has been noted in many problem domains, including robust classification [54], adversarial training [76], causal inference [16], and robust objectives [e.g., 71], this leads to a min-max structure, which is an instance of the cVI problem.

Original: Solving cVIs is significantly more challenging than solving classical single-objective optimization problems, due to the fact that F is a general vector field, leading to possible “rotational” trajectories in parameter space.

Original: In response, the development of efficient algorithms with provable convergence has recently been the focus of interest in machine learning and optimization, particularly in the unconstrained setting, where X ≡ R n [e.g., 3, 8, 13, 19, 36, 38, 56, 57, 78].

Original: © .

Original: [Figure]
Revised: [Figure]

Original: The constraints are depicted with dashed lines and the iterates with circles.
Revised: The constraints are depicted with dashed lines and the iterates with circles.

Original: ACVI gets close to the Nash Equilibrium ( ⋆ ) in a single step, whereas EG zigzags when hitting a constraint.
Revised: ACVI gets close to the Nash Equilibrium ( ⋆ ) in a single step, whereas EG zigzags when hitting a constraint.

Original: The remaining methods perform similarly to EG, see App.
Revised: The remaining methods perform similarly to EG, see App.

Original: G.
Revised: G.

Revised: Solving cVIs is significantly more challenging than solving classical single-objective optimization problems, due to the fact that F is a general vector field, leading to possible “rotational” trajectories in parameter space.

Revised: In response, the development of efficient algorithms with provable convergence has recently been the focus of interest in machine learning and optimization, particularly in the unconstrained setting, where X ≡ R n [e.g., 3, 8, 13, 19, 36, 38, 56, 57, 78].

Original: In many applications, however, we have constraints on (part of) the decision variable x , that is, X is often a strict subset of R n .
Revised: In many applications, however, we have constraints on (part of) the decision variable x , that is, X is often a strict subset of R n .

Original: As an example, let us revisit the aforementioned distributionally robust prediction problem: consider a linear setting [cf.
Revised: As an example, let us revisit the aforementioned distributionally robust prediction problem: consider a linear setting [cf.

Original: Eq.
Revised: Eq.

Original: 1 in 71] and class of parametrized distributions △ ≡ { w ∈ R d
Revised: 1 in 71] and class of parametrized distributions △ ≡ { w ∈ R d

Original: | w ≥ 0 , e ⊺ w = 1 } , where e ∈ R
Revised: | w ≥ 0 , e ⊺ w = 1 } , where e ∈ R

Original: d is a vector of all ones.
Revised: d is a vector of all ones.

Original: Thus, the robust problem is: min x ∈ R n max w ∈ R d w ⊺ ( y − Dx ) , subject to w ≥ 0 , e ⊺ w = 1 , where D ∈ R d × n contains d samples of an n -dimensional covariate vector, and
Revised: Thus, the robust problem is: min x ∈ R n max w ∈ R d w ⊺ ( y − Dx ) , subject to w ≥ 0 , e ⊺ w = 1 , where D ∈ R d × n contains d samples of an n -dimensional covariate vector, and

Original: y ∈ R
Revised: y ∈ R

Original: d is the vector of target variables (the constraint w ≤ 1 is implied), leading to an instance of the cVI problem.
Revised: d is the vector of target variables (the constraint w ≤ 1 is implied), leading to an instance of the cVI problem.

Original: Additional example applications include (i) applications in business, finance, and economics where often the sum of the decision variables—representing for example resources —cannot exceed a specific value, (ii) contract theory (e.g. § 2 . 3 .
Revised: Additional example applications include (i) applications in business, finance, and economics where often the sum of the decision variables—representing for example resources —cannot exceed a specific value, (ii) contract theory (e.g. § 2 . 3 .

Original: 2 in [5]) and (iii) solving optimal control problems numerically, among others.
Revised: 2 in [5]) and (iii) solving optimal control problems numerically, among others.

Original: Significantly fewer works address the convergence of first-order optimization methods in the constrained setting; see § A for an overview.
Revised: Significantly fewer works address the convergence of first-order optimization methods in the constrained setting; see § A for an overview.

Original: Recently, Cai et al.
Revised: Recently, Cai et al.

Original: [11] established a convergence rate for the projected extragradient method [46], when F is monotone and Lipschitz (see § 2 for definitions).
Revised: [11] established a convergence rate for the projected extragradient method [46], when F is monotone and Lipschitz (see § 2 for definitions).

Original: However, (i) the presented proof is computer-assisted making it hard to interpret, and (ii) the considered setting assumes the projection is fast to compute and thus ignores the projection in the rate.
Revised: However, (i) the presented proof is computer-assisted making it hard to interpret, and (ii) the considered setting assumes the projection is fast to compute and thus ignores the projection in the rate.

Original: The latter assumption only holds in rare cases when the constraints are simple, so that operations such as clipping suffice.
Revised: The latter assumption only holds in rare cases when the constraints are simple, so that operations such as clipping suffice.

Original: However, when given general constraints, each EG update requires two projections (see App. B.5).
Revised: However, when given general constraints, each EG update requires two projections (see App. B.5).

Original: Each projection requires solving a separate constrained optimization problem, which if given general constraints requires using a second-order method as explained next.
Revised: Each projection requires solving a separate constrained optimization problem, which if given general constraints requires using a second-order method as explained next.

Original: Interior point (IP) methods are the de facto family of iterative algorithms for constrained optimization.
Revised: Interior point (IP) methods are the de facto family of iterative algorithms for constrained optimization.

Original: These methods enjoy well established guarantees and theoretical understanding in the context of single-objective optimization [see e.g., 9, 55, 84] and have extensions to a wide range of problem settings [e.g., 59, 60, 69, 77, 85].
Revised: These methods enjoy well established guarantees and theoretical understanding in the context of single-objective optimization [see e.g., 9, 55, 84] and have extensions to a wide range of problem settings [e.g., 59, 60, 69, 77, 85].

Original: Several works extend IP methods to the cVI problem, by applying the second-order Newton method to a modified Karush-Kuhn-Tucker (KKT) system appropriate for cVI [15, 27, 58, 65, 68].
Revised: Several works extend IP methods to the cVI problem, by applying the second-order Newton method to a modified Karush-Kuhn-Tucker (KKT) system appropriate for cVI [15, 27, 58, 65, 68].

Original: Many of these approaches, however, rely on strong assumptions—see App.
Revised: Many of these approaches, however, rely on strong assumptions—see App.

Original: A for more details.
Revised: A for more details.

Original: Moreover, although these methods enjoy fast convergence in terms of iterations, each iteration involves the computation of the Jacobian of F which quickly becomes prohibitive for large dimension n .
Revised: Moreover, although these methods enjoy fast convergence in terms of iterations, each iteration involves the computation of the Jacobian of F which quickly becomes prohibitive for large dimension n .

Original: Hence first-order methods are preferred in practice.
Revised: Hence first-order methods are preferred in practice.

Original: We are currently missing a first -order optimization method for solving cVI with general constraints.
Revised: We are currently missing a first -order optimization method for solving cVI with general constraints.

Original: Accordingly, in this paper, we focus on the following open question:
Revised: Accordingly, in this paper, we focus on the following open question:

Original: Can we derive first-order algorithms for the cVI problem that (i) can be applied when general constraints are given, and that (ii) have global convergence guarantees?
Revised: Can we derive first-order algorithms for the cVI problem that (i) can be applied when general constraints are given, and that (ii) have global convergence guarantees?

Original: In this paper, to mitigate the computational burden of second-derivative computation we replace the Newton step of the traditional IP methods with the alternating direction method of multipliers (ADMM) method.
Revised: In this paper, to mitigate the computational burden of second-derivative computation we replace the Newton step of the traditional IP methods with the alternating direction method of multipliers (ADMM) method.

Original: ADMM was designed with a different purpose: it is applicable only when the objective is separable into two or more different functions—see § B.3—and can be seen as equivalent to Douglas–Rachford operator splitting [24] applied in the dual space [see e.g. 51].
Revised: ADMM was designed with a different purpose: it is applicable only when the objective is separable into two or more different functions—see § B.3—and can be seen as equivalent to Douglas–Rachford operator splitting [24] applied in the dual space [see e.g. 51].

Original: ADMM owes its popularity primarily to its computational efficiency [10] for large-scale problems and fast convergence in some settings [e.g., 61].
Revised: ADMM owes its popularity primarily to its computational efficiency [10] for large-scale problems and fast convergence in some settings [e.g., 61].

Original: The core idea of our approach is to reformulate the original cVI problem with equality and inequality constraints via the KKT conditions, so as to apply ADMM in a way that the subproblems of the resulting algorithm have desirable properties (see § C).
Revised: The core idea of our approach is to reformulate the original cVI problem with equality and inequality constraints via the KKT conditions, so as to apply ADMM in a way that the subproblems of the resulting algorithm have desirable properties (see § C).

Original: This framework can also be used to design novel algorithms for solving cVIs; see App.
Revised: This framework can also be used to design novel algorithms for solving cVIs; see App.

Original: E.
Revised: E.

Original: Our contributions include:
Revised: Our contributions include:

Original: • Based on the KKT system for cVIs and the ADMM technique, we derive an algorithm that we refer 
Revised: • Based on the KKT system for cVIs and the ADMM technique, we derive an algorithm that we refer to as the A DMM-based IP Method for cVI s (ACVI)—see § C and Alg.

Original: to as the A DMM-based IP Method for cVI s (ACVI)—see § C and Alg. 1. • We derive parameter-free global convergence guarantees of ACVI given two sets of assumptions: (i) when F is ξ -monotone, and (ii) when it is monotone, the constraints are active at the solution, and the game is not purely rotational.
Revised: • We derive parameter-free global convergence guarantees of ACVI given two sets of assumptions: (i) when F is ξ -monotone, and (ii) when it is monotone, the constraints are active at the solution, and the game is not purely rotational.

Original: By further assuming F is a Lipschitz operator, we upper bound the rate of decrease of the gap function and we match the known lower bound for the gap function of O (1 / √ K ) for the last iterate—see § 3.
Revised: By further assuming F is a Lipschitz operator, we upper bound the rate of decrease of the gap function and we match the known lower bound for the gap function of O (1 / √ K ) for the last iterate—see § 3.

Original: Empirically, we document two notable advantages of ACVI over projection-based methods: (i) the ACVI iterates exhibit significantly reduced rotations, as they approach the solution from the analytic center, and (ii) while projection-based methods show extensive zigzagging when hitting a constraint, ACVI avoids this, resulting in more efficient updates—see § 4.
Revised: Empirically, we document two notable advantages of ACVI over projection-based methods: (i) the ACVI iterates exhibit significantly reduced rotations, as they approach the solution from the analytic center, and (ii) while projection-based methods show extensive zigzagging when hitting a constraint, ACVI avoids this, resulting in more efficient updates—see § 4.

Original: 2. Preliminaries
Revised: 2. Preliminaries

Original: The constraint set is defined as an intersection of finitely many inequalities and linear equalities:C = {
Revised: The constraint set is defined as an intersection of finitely many inequalities and linear equalities:

Original: [Equation]
Revised: [Equation]

Original: ∈
Revised: and

Original: [Equation]
Revised: [Equation]

Original: n | φ i ( x ) ≤ 0 , i ∈ [ m ] , Cx = d } , and C ⊆ X , (CS) where each φ i : R n (cid:55)→ R , C ∈ R p × n , d ∈ R p , with rank ( C ) = p .
Revised: where each φ i : R n (cid:55)→ R , C ∈ R p × n , d ∈ R p , with rank ( C ) = p .

Original: With φ we denote the concatenated φ i ( · ) , i ∈ [ m ] , and each φ i ∈ C 1 ( R n ) , i ∈ [ m ] and is convex.
Revised: With φ we denote the concatenated φ i ( · ) , i ∈ [ m ] , and each φ i ∈ C 1 ( R n ) , i ∈ [ m ] and is convex.

Original: Let: C ≤ ≜ { x ∈ R n | φ ( x ) ≤ 0 } , C < ≜ { x ∈ R n | φ ( x ) < 0 } , and C = ≜ { y ∈ R n | Cy = d } ; thus the relative interior of C is int C ≜ C < ∩ C = , and we consider int C ̸ = ∅ and C is compact.
Revised: Let: C ≤ ≜ { x ∈ R n | φ ( x ) ≤ 0 } , C < ≜ { x ∈ R n | φ ( x ) < 0 } , and C = ≜ { y ∈ R n | Cy = d } ; thus the relative interior of C is int C ≜ C < ∩ C = , and we consider int C ̸ = ∅ and C is compact.

Original: Definition 1 ((strong/ ξ ) monotonicity)
Revised: Definition 1 ((strong/ ξ ) monotonicity)

Original: An operator F : X ⊇ S → R n is monotone on S if: ⟨ x − x ′ , F ( x ) − F ( x ′ ) ⟩ ≥ 0 , ∀ x , x ′ ∈ S .
Revised: An operator F : X ⊇ S → R n is monotone on S if: ⟨ x − x ′ , F ( x ) − F ( x ′ ) ⟩ ≥ 0 , ∀ x , x ′ ∈ S .

Original: F is said to be ξ –monotone on S iff there exist c >and ξ > 1 such that ⟨ x − x ′ , F ( x ) − F ( x ′ )
Revised: F is said to be ξ –monotone on S iff there exist c >and ξ > 1 such that ⟨ x − x ′ , F ( x ) − F ( x ′ )

Original: ⟩ ≥ c
Revised: ⟩ ≥ c

Original: ∥ x − x ′ ∥ ξ , for all x , x ′ ∈ S .
Revised: ∥ x − x ′ ∥ ξ , for all x , x ′ ∈ S .

Original: Finally, F is µ -strongly monotone on S if there exists µ > 0 , such that ⟨ x − x ′ , F ( x ) − F ( x ′ ) ⟩ ≥ µ ∥ x − x ′ ∥ 2 , for all x , x ′ ∈ S .
Revised: Finally, F is µ -strongly monotone on S if there exists µ > 0 , such that ⟨ x − x ′ , F ( x ) − F ( x ′ ) ⟩ ≥ µ ∥ x − x ′ ∥ 2 , for all x , x ′ ∈ S .

Original: Moreover, we say that an operator F is star–monotone , star– ξ -monotone or star–strongly-monotone (on S ) if the respective definition holds for x ′ ≡ x ⋆ , where x ⋆ ∈ S ⋆ S ,F .
Revised: Moreover, we say that an operator F is star–monotone , star– ξ -monotone or star–strongly-monotone (on S ) if the respective definition holds for x ′ ≡ x ⋆ , where x ⋆ ∈ S ⋆ S ,F .

Original: Assumption 1 (first-order smoothness) Let F : X ⊇ S → R n be an operator, we say that F satisfies L -first-order smoothness on S , or L -smoothness, if F is an L -Lipschitz map; that is, there exists L > [-
-] such that ∥ F ( x ) − F ( x ′ ) ∥ ≤ L ∥ x − x ′ ∥ , for all x , x ′ ∈ S .
Revised: Assumption 1 (first-order smoothness) Let F : X ⊇ S → R n be an operator, we say that F satisfies L -first-order smoothness on S , or L -smoothness, if F is an L -Lipschitz map; that is, there exists L > [-
-]0 such that [Equation]

Original: App.
Revised: App.

Original: B.3 provides background on path-following IP methods and describes the ADMM method [30, 32, 33, 52]; whereas App.
Revised: B.3 provides background on path-following IP methods and describes the ADMM method [30, 32, 33, 52]; whereas App.

Original: B.2 summarizes the solution existence guarantees for cVI.
Revised: B.2 summarizes the solution existence guarantees for cVI.

Original: 3. ACVI : first-order ADMM-based IP method for constrained VIs

Original: Algorithm 1 describes the ACVI method, derived in App.

Original: C. See App.

Original: D.5 for further discussion.

Original: In the remaining discussion, where clear from context, we drop the superscript from the iterate x ( t ) k .

Original: We consider two broad problem classes, see App D.1 for a discussion.

Original: For r, s > 0 , let ˆ C r ≜ { x ∈ R n | Cx = d , φ ( x ) ≤ r e } , and let ˜ C s ≜ { x ∈ R n | ∥

Original: Cx − d ∥ ≤ s, φ ( x ) ≤ 0 } .

Original: 2: Initialize: y
Revised: 2: Initialize: y

Original: ∈ R n , λ (
Revised: ∈ R n , λ (

Original: ∈ R n 3: P c ≜ I − C ⊺ ( CC ⊺ ) − 1 C ; d c ≜ C ⊺ ( CC ⊺ ) − 1 d where P c ∈ R n × n , d c ∈ R n 4: for t = 
Revised: ∈ R n 3: P c ≜ I − C ⊺ ( CC ⊺ ) − 1 C ; d c ≜ C ⊺ ( CC ⊺ ) − 1 d where P c ∈ R n × n , d c ∈ R n 4: for t = 0 , . .

Original: , T − 1 do 5: µ t =
Revised: , T − 1 do 5: µ t =

Original: δµ t −6: for k = 
Revised: δµ t −6: for k = 0 , . . .

Original:  K −  do 7:
Revised: , K − 1 do 7:

Original: Set x ( t ) k
Revised: Set x ( t ) k

Original: +1 to be the solution of: x + 1 β P c F ( x ) − P c y ( t ) k
Revised: +1 to be the solution of: x + 1 β P c F ( x ) − P c y ( t ) k

Original: + 1 β P c λ ( t ) k − d c = 

Revised: + 1 β P c λ ( t ) k − d c =

Original: (w.r.t. x ) 8: y k +1 = argmin
Revised: (w.r.t. x ) 8: y k +1 = argmin

Original: y − µ (cid:80) mi =1 log (cid:0) − φ i ( y ) (cid:1)
Revised: y − µ (cid:80) mi =1 log (cid:0) − φ i ( y ) (cid:1)

Original: + β 2 (cid:13)(cid:13)(cid:13)
Revised: + β 2 (cid:13)(cid:13)(cid:13)

Original: y − x k +1 − 1 β λ k (cid:13)(cid:13)(cid:13)9: λ k +1 =
Revised: y − x k +1 − 1 β λ k (cid:13)(cid:13)(cid:13)9: λ k +1 =

Original: λ k + β
Revised: λ k + β

Original: ( x k +1 − y k +1 ) 10: end for 11: ( y ( t +1),
Revised: ( x k +1 − y k +1 ) 10: end for 11: ( y ( t +1),

Original: λ ( t +1)) ≜
Revised: λ ( t +1)) ≜

Original: ( y ( t ) K , λ ( t ) K ) 12: end for
Revised: ( y ( t ) K , λ ( t ) K ) 12: end for

Revised: 3. ACVI : first-order ADMM-based IP method for constrained VIs

Revised: Algorithm 1 describes the ACVI method, derived in App.

Revised: C. See App.

Revised: D.5 for further discussion.

Revised: In the remaining discussion, where clear from context, we drop the superscript from the iterate x ( t ) k .

Original:  Theorem 1 (Last and average iterate convergence for star- ξ -monotone operator)
Revised: We consider two broad problem classes, see App D.1 for a discussion. For r, s > 0 , let ˆ C r ≜ { x ∈ R n | Cx = d , φ ( x ) ≤ r e } , and let ˜ C s ≜ { x ∈ R n | ∥ Cx − d ∥ ≤ s, φ ( x ) ≤ 0 } . Theorem 1 (Last and average iterate convergence for star- ξ -monotone operator)

Original: Given an operator F : X → R n monotone on C
Revised: Given an operator F : X → R n monotone on C

Original: =
Revised: =

Original: (Def. 1), assume that either F is strictly monotone on C or one of φ i is strictly convex.
Revised: (Def. 1), assume that either F is strictly monotone on C or one of φ i is strictly convex.

Original: Assume there exists r > 0 or s > 0 such that F is star-
Revised: Assume there exists r > 0 or s > 0 such that F is star-

Original: ξ -monotone on either ˆ C r or ˜ C s .
Revised: ξ -monotone on either ˆ C r or ˜ C s .

Original: Let x ( t ) K and ˆ x ( t ) K ≜ 1 K (cid:80)
Revised: Let x ( t ) K and ˆ x ( t ) K ≜ 1 K (cid:80)

Original: Kk
Revised: Kk

Original: =1 x ( t ) k denote the last and average iterate of Algorithm 1, respectively, run with sufficiently small µ − 1 .
Revised: =1 x ( t ) k denote the last and average iterate of Algorithm 1, respectively, run with sufficiently small µ − 1 .

Original: Then for all t ∈ [ T ] , we have: (cid:13)(cid:13)(cid:13) x
Revised: Then for all t ∈ [ T ] , we have: (cid:13)(cid:13)(cid:13) x

Original: ( t ) K − x ⋆ (cid:13)(cid:13)(cid:13) ≤ O (  K 
Revised: ( t ) K − x ⋆ (cid:13)(cid:13)(cid:13) ≤ O ( 1 K 1 / (2 ξ ) ) .

Original: ξ If in addition F is ξ -monotone on C = , we have (cid:13)(cid:13)(cid:13) ˆ x ( t ) K −
Revised: If in addition F is ξ -monotone on C = , we have (cid:13)(cid:13)(cid:13) ˆ x ( t ) K −

Original: x ⋆ (cid:13)(cid:13)(cid:13) ≤ O ( 1 K 1 /ξ ) .
Revised: x ⋆ (cid:13)(cid:13)(cid:13) ≤ O ( 1 K 1 /ξ ) .

Original: Moreover, if F is L-Lipschitz on C = we have: G ( x ( t ) K , C ) ≤O ( [Equation] ; G  ( [Equation]
Revised: Moreover, if F is L-Lipschitz on C = we have: G ( x ( t ) K , C ) ≤O ( L K 1 / (2 ξ ) ) ; G (ˆ x ( t ) K , C ) ≤O ( LK 1 /ξ ) .

Original: x (

Original: [Equation]

Original: [Equation]

Original: [Equation]

Original: ) ≤O ( LK

Original: [Equation]

Original: ξ ) .

Original: Theorem 2 (Last and average iterate convergence for monotone operator)
Revised: Theorem 2 (Last and average iterate convergence for monotone operator)

Original: Given an operator
Revised: Given an operator

Original: F : X → R n , assume (i) F is monotone on C = , and (ii) either F is strictly monotone on C or one of φ i is strictly convex, and (iii) inf x ∈ S \{ x ⋆ }
Revised: F : X → R n , assume (i) F is monotone on C = , and (ii) either F is strictly monotone on C or one of φ i is strictly convex, and (iii) inf [Equation] F [Equation]

Revised: x

Original: F ( x ) ⊺ x − x ⋆ ∥ x − x ⋆ ∥ > 0 , where S ≡ ˆ C r or ˜ C s .
Revised: [Equation] x − x [Equation] ∥ > 0 , where S ≡ ˆ C r or ˜ C s .

Original: Let x ( t ) K and ˆ x ( t ) K ≜ 1 K (cid:80)
Revised: Let x ( t ) K and ˆ x ( t ) K ≜ 1 K (cid:80)

Original: Kk
Revised: Kk

Original: =1 x ( t ) k denote the last and average iterate of Algorithm 1, respectively, run with sufficiently small µ − 1 .
Revised: =1 x ( t ) k denote the last and average iterate of Algorithm 1, respectively, run with sufficiently small µ − 1 .

Original: Then for all t ∈ [ T ] , we have that: (cid:13)(cid:13)(cid:13) x
Revised: Then for all t ∈ [ T ] , we have that: (cid:13)(cid:13)(cid:13) x

Original: ( t ) K − x ⋆ (cid:13)(cid:13)(cid:13) ≤ O ( 1 √ K ) .
Revised: ( t ) K − x ⋆ (cid:13)(cid:13)(cid:13) ≤ O ( 1 √ K ) .

Original: If in addition inf [Equation] ⋆ ∥ > 0 (with S ≡ ˆ C r or ˜ C s ), then (cid:13)(cid:13)(cid:13) ˆ x ( t ) K − x ⋆ (cid:13)(cid:13)(cid:13) ≤ O ( 1 K ) .
Revised: If in addition inf x ∈ S \{ x ⋆ } F ( x ⋆ ) ⊺ x − x ⋆ ∥ x − x ⋆ ∥ > 0 (with S ≡ ˆ C r or ˜ C s ), then (cid:13)(cid:13)(cid:13) ˆ x ( t ) K − x ⋆ (cid:13)(cid:13)(cid:13) ≤ O ( 1 K ) .

Original: Moreover, if F is L-Lipschitz on C = we have: G ( x ( t ) K , C ) ≤
Revised: Moreover, if F is L-Lipschitz on C = we have: G ( x ( t ) K , C ) ≤

Original: O ( L √ K ) and G (ˆ x ( t ) K , C ) ≤ O ( LK ) .
Revised: O ( L √ K ) and G (ˆ x ( t ) K , C ) ≤ O ( LK ) .

Original: 4. Experiments
Revised: 4. Experiments

Original: Problems.
Revised: Problems.

Original: We use the following 2D problems: (i) cBG : the bilinear game, constrained on R + , stated in Fig.
Revised: We use the following 2D problems: (i) cBG : the bilinear game, constrained on R + , stated in Fig.

Original: 1, (ii) Von Neumann’s ratio game [20, 23, 81], (iii)
Revised: 1, (ii) Von Neumann’s ratio game [20, 23, 81], (iii)

Original: Forsaken game [43]–which exhibits limit cycles , as well as (iv) toy GAN —used in [1, 19].
Revised: Forsaken game [43]–which exhibits limit cycles , as well as (iv) toy GAN —used in [1, 19].

Original: We also consider a constrained BG on the probability simplex, with n = 1000 , given in (HBG).
Revised: We also consider a constrained BG on the probability simplex, with n = 1000 , given in (HBG).

Original: Finally, we use GANs on MNIST [49] with added linear inequalities, see App.
Revised: Finally, we use GANs on MNIST [49] with added linear inequalities, see App.

Original: F.3.
Revised: F.3.

Original: App.
Revised: App.

Original: G lists additional experiments, including on Fashion-MNIST [86].
Revised: G lists additional experiments, including on Fashion-MNIST [86].

Revised: [Figure]

Original: Methods.
Revised: Methods.

Original: We compare with the projection-based methods (described in App.
Revised: We compare with the projection-based methods (described in App.

Original: B.5)
Revised: B.5)

Original: (i) GDA , (ii)
Revised: (i) GDA , (ii)

Original: EG [46], (iii) OGDA [64], and (iv) LA ˜ k
Revised: EG [46], (iii) OGDA [64], and (iv) LA ˜ k

Original: -GDA [14, 87], where ˜ k is the hyperparameter of LA.
Revised: -GDA [14, 87], where ˜ k is the hyperparameter of LA.

Original: For
Revised: For

Original: ACVI on MNIST, l denotes the number of steps to solve the subproblems; see Alg. 3.[Figure] [Figure] [Figure]
Revised: ACVI on MNIST, l denotes the number of steps to solve the subproblems; see Alg. 3.

Original: over wall-clock time; averaged over seeds.

Original: See also § 4 and

Original: Results.
Revised: Results.

Original: From Fig.
Revised: From Fig.

Original: 1 we observe that projection-based algorithms may zigzag when hitting a constraint, due to the rotational nature of F , behavior that ACVI avoids because it incorporates the constraints in its update rule; see Fig.
Revised: 1 we observe that projection-based algorithms may zigzag when hitting a constraint, due to the rotational nature of F , behavior that ACVI avoids because it incorporates the constraints in its update rule; see Fig.

Original: 5 for the remaining baselines.
Revised: 5 for the remaining baselines.

Original: Fig.
Revised: Fig.

Original: 2 shows that even with problems that go beyond our theoretical assumptions a single step of ACVI significantly reduces the distance to the solution.
Revised: 2 shows that even with problems that go beyond our theoretical assumptions a single step of ACVI significantly reduces the distance to the solution.

Original: Moreover, from Fig.
Revised: Moreover, from Fig.

Original: 2( b )subfigure we observe that ACVI escapes the limit cycles; see also Fig. 6.
Revised: 2( b )subfigure we observe that ACVI escapes the limit cycles; see also Fig. 6.

Original: Fig.
Revised: Fig.

Original: 3 shows results for (HBG), indicating that ACVI is time efficient, and that ACVI performs well relative to projection-based methods for varying rotational intensity ( 1 − η ).
Revised: 3 shows results for (HBG), indicating that ACVI is time efficient, and that ACVI performs well relative to projection-based methods for varying rotational intensity ( 1 − η ).

Original: Fig.
Revised: Fig.

Original: 4 summarizes the experiments on MNIST with linear inequality constraints; we observe that
Revised: 4 summarizes the experiments on MNIST with linear inequality constraints; we observe that

Original: ACVI converges significantly faster than the corresponding baseline. 
Revised: ACVI converges significantly faster than the corresponding baseline. [Figure] i m [Figure]

Revised: [Figure]

Original: 5. Conclusion
Revised: 5. Conclusion

Original: Motivated by the lack of a first -order method to solve cVIs with general constraints, we proposed a framework that combines (i) interior-point methods—needed to be able to handle general constraints— with (ii) the ADMM method—designed to deal with separable objectives.
Revised: Motivated by the lack of a first -order method to solve cVIs with general constraints we proposed a framework that combines (i) interior-point methods—needed to be able to handle general constraints— with (ii) the ADMM method—designed to deal with separable objectives.

Original: The combination yields
Revised: The combination yields

Original: ACVI —a first-order ADMM-based interior point method for cVIs.
Revised: ACVI —a first-order ADMM-based interior point method for cVIs.

Original: We proved convergence for two broad classes of problems, and derived the corresponding convergence rates.
Revised: We proved convergence for two broad classes of problems, and derived the corresponding convergence rates.

Original: Numerical experiments showed that while projection-based methods zigzag when hitting a constraint due to the rotational vector field, ACVI avoids this by incorporating the constraints in the update rule.
Revised: Numerical experiments showed that while projection-based methods zigzag when hitting a constraint due to the rotational vector field, ACVI avoids this by incorporating the constraints in the update rule.

Original: Appendix A. Related Work
Revised: Appendix A. Related Work

Original: Unconstrained VIs: methods and guarantees.
Revised: Unconstrained VIs: methods and guarantees.

Original: Apart from the standard gradient descent ascent (GDA) method, among the most commonly used methods for VI optimization are the extragradient method [EG, 46], optimistic GDA [OGDA, 64], and the lookahead method [LA, 14, 87].
Revised: Apart from the standard gradient descent ascent (GDA) method, among the most commonly used methods for VI optimization are the extragradient method [EG, 46], optimistic GDA [OGDA, 64], and the lookahead method [LA, 14, 87].

Original: (See App.
Revised: (See App.

Original: B for a full description).
Revised: B for a full description).

Original: In contrast to gradient fields (as in a single-objective setting), when F is a general vector field, the last iterate can be far from the solution even though the average iterate converges to it [12, 19].
Revised: In contrast to gradient fields (as in a single-objective setting), when F is a general vector field, the last iterate can be far from the solution even though the average iterate converges to it [12, 19].

Original: This is problematic since it implies that the average convergence guarantee is weaker in the sense that it may not extend to more general setups where we can no longer rely on the convexity of X .
Revised: This is problematic since it implies that the average convergence guarantee is weaker in the sense that it may not extend to more general setups where we can no longer rely on the convexity of X .

Original: Golowich et al.
Revised: Golowich et al.

Original: [35, 36] provided a last-iterate lower bound of O ( 1 ˜ p √ K ) for the broad class of ˜ p - stationary canonical linear iterative ( ˜ p -SCLI) first-order methods [2].
Revised: [35, 36] provided a last-iterate lower bound of O ( 1 ˜ p √ K ) for the broad class of ˜ p - stationary canonical linear iterative ( ˜ p -SCLI) first-order methods [2].

Original: An extensive line of further work has provided guarantees for the last iterate for other problem classes.
Revised: An extensive line of further work has provided guarantees for the last iterate for other problem classes.

Original: For the general MVI class, the following ˜ p
Revised: For the general MVI class, the following ˜ p

Original: -SCLI
Revised: -SCLI

Original: methods come with guarantees that match the lower bound: (i) Golowich et al.
Revised: methods come with guarantees that match the lower bound: (i) Golowich et al.

Original: [36] obtained a rate in terms of the gap function relying on first- and second-order smoothness of F , and Gorbunov et al.
Revised: [36] obtained a rate in terms of the gap function relying on first- and second-order smoothness of F , and Gorbunov et al.

Original: [38] obtained a rate of O ( 1 K ) in terms of reducing the squared norm of the operator relying on first-order smoothness of F (Assumption 1), using a computer-assisted proof, and (ii) Golowich et al.
Revised: [38] obtained a rate of O ( 1 K ) in terms of reducing the squared norm of the operator relying on first-order smoothness of F (Assumption 1), using a computer-assisted proof, and (ii) Golowich et al.

Original: [36] and Chavdarova et al.
Revised: [36] and Chavdarova et al.

Original: [13] provided the best iterate rate for OGDA.
Revised: [13] provided the best iterate rate for OGDA.

Original: Constrained zero-sum and VI classes of problems.
Revised: Constrained zero-sum and VI classes of problems.

Original: Gidel et al.
Revised: Gidel et al.

Original: [31] extended the Frank-Wolfe [28, 45, 48] method—also known as the conditional gradient [79]—to solve a subclass of cVI, specifically constrained zero-sum problems.
Revised: [31] extended the Frank-Wolfe [28, 45, 48] method—also known as the conditional gradient [79]—to solve a subclass of cVI, specifically constrained zero-sum problems.

Original: This extension was carried out under a strong convex-concavity assumption and also under the assumption that the constraint set is strongly convex; that is, it has sublevel sets that are strongly convex functions [80].
Revised: This extension was carried out under a strong convex-concavity assumption and also under the assumption that the constraint set is strongly convex; that is, it has sublevel sets that are strongly convex functions [80].

Original: Daskalakis and Panageas [18] provided an asymptotic proof for the last iterate for zero-sum convex-concave constrained problems for the optimistic multiplicative weights update (OMWU) method.
Revised: Daskalakis and Panageas [18] provided an asymptotic proof for the last iterate for zero-sum convex-concave constrained problems for the optimistic multiplicative weights update (OMWU) method.

Original: Wei et al.
Revised: Wei et al.

Original: [83] focused on OGDA and OMWU in the constrained setting and provided convergence rates for bilinear games over the simplex.
Revised: [83] focused on OGDA and OMWU in the constrained setting and provided convergence rates for bilinear games over the simplex.

Original: In her seminal work, Korpelevich [46] proposed the classical (projected) extragradient method (EG)—see App.
Revised: In her seminal work, Korpelevich [46] proposed the classical (projected) extragradient method (EG)—see App.

Original: B—and proved its convergence for monotone (c)VIs with an L -Lipschitz operator, and, as mentioned above, Cai et al.
Revised: B—and proved its convergence for monotone (c)VIs with an L -Lipschitz operator, and, as mentioned above, Cai et al.

Original: [11] established a rate with respect to the gap function using a computer-aided proof.
Revised: [11] established a rate with respect to the gap function using a computer-aided proof.

Original: For the constrained setting, Tseng [78] built on [63] and provided a linear convergence rate for EG in the setting of strongly monotone F , whereas Malitsky [53] focused on the same setting but on the projected reflected gradient method.
Revised: For the constrained setting, Tseng [78] built on [63] and provided a linear convergence rate for EG in the setting of strongly monotone F , whereas Malitsky [53] focused on the same setting but on the projected reflected gradient method.

Original: Diakonikolas [22] obtained parameter-free guarantee for Halpern iteration [39] for cocoercive operators.
Revised: Diakonikolas [22] obtained parameter-free guarantee for Halpern iteration [39] for cocoercive operators.

Original: Goffin et al.
Revised: Goffin et al.

Original: [34] described a second-order cutting-plane method for solving pseudomonotone VIs with linear inequalities.
Revised: [34] described a second-order cutting-plane method for solving pseudomonotone VIs with linear inequalities.

Original: Interior point (IP) methods in single-objective and VI settings.
Revised: Interior point (IP) methods in single-objective and VI settings.

Original: Traditionally IP methods primarily express the inequality constraints by augmenting the objective with a log -barrier penalty (see § B.3), and then use Newton’s method to solve the subproblem [9].
Revised: Traditionally IP methods primarily express the inequality constraints by augmenting the objective with a log -barrier penalty (see § B.3), and then use Newton’s method to solve the subproblem [9].

Original: The latter involves computing either the inverse of a large matrix or a Cholesky decomposition, and yet it can be highly efficient in low dimensions as it requires only a few iterations to converge.
Revised: The latter involves computing either the inverse of a large matrix or a Cholesky decomposition, and yet it can be highly efficient in low dimensions as it requires only a few iterations to converge.

Original: When the dimensionality of the variable is large, however, the computation becomes infeasible.
Revised: When the dimensionality of the variable is large, however, the computation becomes infeasible.

Original: Among other IP variants that address this issue, Lin et al.
Revised: Among other IP variants that address this issue, Lin et al.

Original: [50] replaced the Newton step with the ADMM method, which is known to be highly scalable in terms of the dimension [10].
Revised: [50] replaced the Newton step with the ADMM method, which is known to be highly scalable in terms of the dimension [10].

Original: In the context of cVIs, a few works apply IP methods, mostly
Revised: In the context of cVIs, a few works apply IP methods, mostly

Original: Newton-based [e.g., 59, Chapter 7 ].
Revised: Newton-based [e.g., 59, Chapter 7 ].

Original: Monteiro and Pang [58] analyze path-following IP methods for complementarity problems, which are a subclass of cVI, using local homeomorphic maps.
Revised: Monteiro and Pang [58] analyze path-following IP methods for complementarity problems, which are a subclass of cVI, using local homeomorphic maps.

Original: Chen et al.
Revised: Chen et al.

Original: [15] provided a superlinear global convergence rate of the smoothing Newton method when F is semi-smooth for box constrained VIs.
Revised: [15] provided a superlinear global convergence rate of the smoothing Newton method when F is semi-smooth for box constrained VIs.

Original: Similarly, Qi and Sun [65], Qi et al.
Revised: Similarly, Qi and Sun [65], Qi et al.

Original: [66] focused on the smoothing Newton method and provided the rate for the outer loop.
Revised: [66] focused on the smoothing Newton method and provided the rate for the outer loop.

Original: Ralph and Wright [68] showed superlinear convergence for MVI problems under inequality constraints, under the following set of assumptions: (i) existence of a strictly complementary solution, (ii) full rank of the Jacobian of the active constraints at the solution, and (iii) twice differentiable constraints.
Revised: Ralph and Wright [68] showed superlinear convergence for MVI problems under inequality constraints, under the following set of assumptions: (i) existence of a strictly complementary solution, (ii) full rank of the Jacobian of the active constraints at the solution, and (iii) twice differentiable constraints.

Original: They provided a local convergence rate.
Revised: They provided a local convergence rate.

Original: Fan and Yan [27] considered inequality constraints and proposed a second-order
Revised: Fan and Yan [27] considered inequality constraints and proposed a second-order

Original: Newton-based method that has global convergence guarantees under certain conditions.
Revised: Newton-based method that has global convergence guarantees under certain conditions.

Original: A rate was not provided.
Revised: A rate was not provided.

Original: Appendix B. Background: Additional Details
Revised: Appendix B. Background: Additional Details

Original: This section lists additional background such as omitted definitions, and a description of ADMM and the used baseline methods.
Revised: This section lists additional background such as omitted definitions, and a description of ADMM and the used baseline methods.

Original: B.1. Additional VI definitions & equivalent formulations
Revised: B.1. Additional VI definitions & equivalent formulations

Original: The analog for cVI of the function values used as a performance measure for convergence rates in convex optimization is the gap function (a.k.a., the optimality gap or primal gap), defined next.
Revised: The analog for cVI of the function values used as a performance measure for convergence rates in convex optimization is the gap function (a.k.a., the optimality gap or primal gap), defined next.

Original: Definition 2 (gap function)
Revised: Definition 2 (gap function)

Original: Given a candidate point x ′ ∈ X and a map F : X ⊇ S → R n where
Revised: Given a candidate point x ′ ∈ X and a map F : X ⊇ S → R n where

Original: S is compact, the gap function G : R n → R is defined as G ( x ′ , S ) =
Revised: S is compact, the gap function G : R n → R is defined as G ( x ′ , S ) =

Original: max x ∈S ⟨ F ( x ′ ) , x ′ − x ⟩ .
Revised: max x ∈S ⟨ F ( x ′ ) , x ′ − x ⟩ .

Original: Note that the gap function requires S to be compact in order to be defined (as otherwise, it can be infinite).
Revised: Note that the gap function requires S to be compact in order to be defined (as otherwise, it can be infinite).

Original: Seeing an operator F : X → R n as the graph Gr F = { ( x , y ) | x ∈ X , y = F ( x ) } , its inverse
Revised: Seeing an operator F : X → R n as the graph Gr F = { ( x , y ) | x ∈ X , y = F ( x ) } , its inverse

Original: F − 1 is defined as Gr F − 1 = { ( y , x ) | ( x
Revised: F − 1 is defined as Gr F − 1 = { ( y , x ) | ( x

Original: , y ) ∈
Revised: , y ) ∈

Original: Gr F } .
Revised: Gr F } .

Original: See for example [72] for further discussion.
Revised: See for example [72] for further discussion.

Original: We denote the projection to the set X with Π X .
Revised: We denote the projection to the set X with Π X .

Original: Definition 3 ( 1 µ -cocoercive operator)
Revised: Definition 3 ( 1 µ -cocoercive operator)

Original: An operator F : X ⊇ S → R n is 1 µ - cocoercive (or 1 µinverse strongly monotone ) on S if its inverse (graph) F − 1 is µ -strongly monotone on S , that is,
Revised: An operator F : X ⊇ S → R n is 1 µ - cocoercive (or 1 µinverse strongly monotone ) on S if its inverse (graph) F − 1 is µ -strongly monotone on S , that is,

Original: [Equation]
Revised: [Equation]

Original: It is star 1 µ - cocoercive if the above holds when setting x ′ ≡ x ⋆ where x ⋆ denotes a solution, that is: ∃ µ > 0 , s.t.
Revised: It is star 1 µ - cocoercive if the above holds when setting x ′ ≡ x ⋆ where x ⋆ denotes a solution, that is: ∃ µ > 0 , s.t.

Original: [Equation]
Revised: [Equation]

Original: x ⋆ ∈ S ⋆ X ,F .
Revised: x ⋆ ∈ S ⋆ X ,F .

Original: Note from Def. 3 that cocoercivity is a strict subclass of monotone and L-Lipschitz operators, thus is it is a stronger assumption.
Revised: Note from Def. 3 that cocoercivity is a strict subclass of monotone and L-Lipschitz operators, thus is it is a stronger assumption.

Original: See Chapter 4.2 of [6] for further relations of cocoercivity with other properties of operators.
Revised: See Chapter 4.2 of [6] for further relations of cocoercivity with other properties of operators.

Original: In the following, we will make use of the natural and normal mappings of an operator F : X →
Revised: In the following, we will make use of the natural and normal mappings of an operator F : X →

Original: R n , where X ⊂ R n .
Revised: R n , where X ⊂ R n .

Original: Following the notation of [26], the natural map F NAT X : X → R n is:
Revised: Following the notation of [26], the natural map F NAT X : X → R n is:

Original: [Equation]
Revised: [Equation]

Original: whereas the normal map F NOR X : R n → R n is:
Revised: whereas the normal map F NOR X : R n → R n is:

Original: [Equation]
Revised: [Equation]

Original: Moreover, we have the following solution characterizations:
Revised: Moreover, we have the following solution characterizations:

Original: 1. x ⋆ ∈ S ⋆ X ,F iff F NAT X ( x ⋆ ) = 0 , and 2.
Revised: 1. x ⋆ ∈ S ⋆ X ,F iff F NAT X ( x ⋆ ) = 0 , and 2.

Original: x ⋆ ∈ S ⋆ X ,F iff ∃ x ′ ∈ R n s.t. x
Revised: x ⋆ ∈ S ⋆ X ,F iff ∃ x ′ ∈ R n s.t. x

Original: ⋆ = Π X ( x ′ ) and F NOR X
Revised: ⋆ = Π X ( x ′ ) and F NOR X

Original: [Equation]
Revised: [Equation]

Original: x ′ )
Revised: x ′ )

Original: = 0 .
Revised: = 0 .

Original: B.2. Existence of solution
Revised: B.2. Existence of solution

Original: We provide brief informal summary of some sufficient conditions for solution existence, that
Revised: We provide brief informal summary of some sufficient conditions for solution existence, that

Original: S ⋆ · ,F ̸ = ∅ .
Revised: S ⋆ · ,F ̸ = ∅ .

Original: See Chapter 2 of [26] for a full treatment of the topic.
Revised: See Chapter 2 of [26] for a full treatment of the topic.

Original: The common underlying tool to establish that a solution to the VI( X , F ) problem exists is using topological degree .
Revised: The common underlying tool to establish that a solution to the VI( X , F ) problem exists is using topological degree .

Original: The topological degree tool is designed so as to satisfy the so-called homotopy invariance axiom, which in turn allows for reducing a solution existence question of a complicated map to a simpler one (which is homotopy-invariant to the original one) for which we can more easily show that it has a solution (e.g., the identity map on a closed domain).
Revised: The topological degree tool is designed so as to satisfy the so-called homotopy invariance axiom, which in turn allows for reducing a solution existence question of a complicated map to a simpler one (which is homotopy-invariant to the original one) for which we can more easily show that it has a solution (e.g., the identity map on a closed domain).

Original: It can be used (as one way) to prove the celebrated Brouwer fixed-point theorem , which states that any continuous map Φ : S → S , where S is nonempty convex compact set, has a fixed point in S .
Revised: It can be used (as one way) to prove the celebrated Brouwer fixed-point theorem , which states that any continuous map Φ : S → S , where S is nonempty convex compact set, has a fixed point in S .

Original: We have the following sufficient condition, see [Cor. 2.2.5, 26]
Revised: We have the following sufficient condition, see [Cor. 2.2.5, 26]

Original: Theorem 3 (sufficient condition for existence of the solution, Cor. 2.2.5, [26])
Revised: Theorem 3 (sufficient condition for existence of the solution, Cor. 2.2.5, [26])

Original: If X ⊆ R n is compact and convex, and F : X → R n is continuous, then the solution set is nonempty and compact.
Revised: If X ⊆ R n is compact and convex, and F : X → R n is continuous, then the solution set is nonempty and compact.

Original: It can also be shown that when X is closed convex (and F continuous), if one can find x ′ ∈
Revised: It can also be shown that when X is closed convex (and F continuous), if one can find x ′ ∈

Original: X s.t.
Revised: X s.t.

Original: ⟨ F ( x ) , x − x ′ ⟩ ≥ 0 , ∀ x ∈ X , then S ⋆ X ,F ̸ = ∅ .
Revised: ⟨ F ( x ) , x − x ′ ⟩ ≥ 0 , ∀ x ∈ X , then S ⋆ X ,F ̸ = ∅ .

Original: The same conclusion follows if one can show that ∃ x ′ ∈ X and the set { x ∈ X
Revised: The same conclusion follows if one can show that ∃ x ′ ∈ X and the set { x ∈ X

Original: |⟨ F ( x ) , x − x ′ ⟩ < 0 } is bounded (possibly empty, see Prop. 2.2.
Revised: |⟨ F ( x ) , x − x ′ ⟩ < 0 } is bounded (possibly empty, see Prop. 2.2.

Original: in [26]).
Revised: in [26]).

Original: Sufficient conditions can also be established via the natural and the normal map due to the above solution characterizations.
Revised: Sufficient conditions can also be established via the natural and the normal map due to the above solution characterizations.

Original: In this case, we require that F is continuous on an open set S and we are interested if S ⋆ X ,F ̸ = ∅ , where X is assumed closed and convex and subset of S , X ⊆ S .
Revised: In this case, we require that F is continuous on an open set S and we are interested if S ⋆ X ,F ̸ = ∅ , where X is assumed closed and convex and subset of S , X ⊆ S .

Original: If one establishes that a solution exists for F NAT X on a bounded open set U , and if cl U ⊆ S , then it follows that S ⋆ X ,F ̸ = ∅ .
Revised: If one establishes that a solution exists for F NAT X on a bounded open set U , and if cl U ⊆ S , then it follows that S ⋆ X ,F ̸ = ∅ .

Original: A similar implication holds when we have such a guarantee for F NOR X .
Revised: A similar implication holds when we have such a guarantee for F NOR X .

Original: See Theorem 2.2.1 of [26].
Revised: See Theorem 2.2.1 of [26].

Original: In summary, the solution existence guarantee follows from the boundness of some set which includes X , the boundness of the set of potential solutions (if we can construct such set), or the compactness of X itself.
Revised: In summary, the solution existence guarantee follows from the boundness of some set which includes X , the boundness of the set of potential solutions (if we can construct such set), or the compactness of X itself.

Original: B.3. Relevant path-following interior-point methods and ADMM
Revised: B.3. Relevant path-following interior-point methods and ADMM

Original: In this section, we overview the interior-point approach to single-objective optimization, focusing on aspects that are most relevant to our proposed method.
Revised: In this section, we overview the interior-point approach to single-objective optimization, focusing on aspects that are most relevant to our proposed method.

Original: Consider the following problem:
Revised: Consider the following problem:

Original: [Equation]
Revised: [Equation]

Original: where f, φ i : R n → R are convex and continuously differentiable, x ∈ R n , C ∈ R p × n , and d ∈ R p .
Revised: where f, φ i : R n → R are convex and continuously differentiable, x ∈ R n , C ∈ R p × n , and d ∈ R p .

Original: IP methods solve problem (cCVX) by reducing it to a sequence of linear equality-constrained problems via a logarithmic barrier [see, e.g., 9, Chapter 11]:
Revised: IP methods solve problem (cCVX) by reducing it to a sequence of linear equality-constrained problems via a logarithmic barrier [see, e.g., 9, Chapter 11]:

Original: [Equation]
Revised: [Equation]

Original: Assume that (l-cCVX) has a solution for each µ > 0 , and let x µ denote the solution of (l-cCVX) for a given µ .
Revised: Assume that (l-cCVX) has a solution for each µ > 0 , and let x µ denote the solution of (l-cCVX) for a given µ .

Original: The central path of (l-cCVX) is defined as the set of points x µ , µ > 0 .
Revised: The central path of (l-cCVX) is defined as the set of points x µ , µ > 0 .

Original: Note that x µ ∈ R n is a strictly feasible point of (cCVX) as it satisfies φ ( x µ ) < 0 and Cx µ = d .
Revised: Note that x µ ∈ R n is a strictly feasible point of (cCVX) as it satisfies φ ( x µ ) < 0 and Cx µ = d .

Original: Alternating direction method of multipliers (ADMM) method.
Revised: Alternating direction method of multipliers (ADMM) method.

Original: ADMM [30, 32, 33, 52] is a gradient-based algorithm for convex optimization problems that splits the objective into subproblems each of which is easier to solve.
Revised: ADMM [30, 32, 33, 52] is a gradient-based algorithm for convex optimization problems that splits the objective into subproblems each of which is easier to solve.

Original: Its popularity is due to its computational scalability [10].
Revised: Its popularity is due to its computational scalability [10].

Original: Consider a problem of the following form:
Revised: Consider a problem of the following form:

Original: [Equation]
Revised: [Equation]

Original: where f, g : R n → R are convex, x , y ∈ R n , A , B ∈ R n ′ × n , and b ∈ R n ′ .
Revised: where f, g : R n → R are convex, x , y ∈ R n , A , B ∈ R n ′ × n , and b ∈ R n ′ .

Original: The augmented
Revised: The augmented

Original: Lagrangian function, L β ( · ) , of the (ADMM-Pr) problem is:
Revised: Lagrangian function, L β ( · ) , of the (ADMM-Pr) problem is:

Original: [Equation]
Revised: [Equation]

Original: where β > 0 is referred to as the penalty parameter .
Revised: where β > 0 is referred to as the penalty parameter .

Original: If the augmented Lagrangian method is used to solve (AL-CVX), at each step k we have: x k +1 ,
Revised: If the augmented Lagrangian method is used to solve (AL-CVX), at each step k we have: x k +1 ,

Original: y k +1 =
Revised: y k +1 =

Original: argmin x , y L β ( x , y , λ k ) and λ k
Revised: argmin x , y L β ( x , y , λ k ) and λ k

Original: +

Revised: +

Original: =
Revised: =

Original: λ k + β
Revised: λ k + β

Original: ( Ax k +1 + By k +1 − b ) , where the latter step is gradient ascent on the dual.
Revised: ( Ax k +1 + By k +1 − b ) , where the latter step is gradient ascent on the dual.

Original: In contrast,
Revised: In contrast,

Original: ADMM updates x and y in an alternating way as follows:
Revised: ADMM updates x and y in an alternating way as follows:

Original: [Equation]
Revised: [Equation]

Original: B.4. Existence of Central path
Revised: B.4. Existence of Central path

Original: In this section, we discuss the results that establish guarantees of the existence of the central path.
Revised: In this section, we discuss the results that establish guarantees of the existence of the central path.

Original: Let L ( x , λ , ν ) ≜ F ( x ) + ∇ φ ⊺ ( x ) λ + C ⊺ ν , h ( x ) =
Revised: Let L ( x , λ , ν ) ≜ F ( x ) + ∇ φ ⊺ ( x ) λ + C ⊺ ν , h ( x ) =

Original: C ⊺ x − d .
Revised: C ⊺ x − d .

Original: For ( λ , w , x , ν ) ∈ R 2 m + n + p , let
Revised: For ( λ , w , x , ν ) ∈ R 2 m + n + p , let

Original: G ( λ , w , x , ν ) ≜   w ◦ λ w + φ ( x )
Revised: G ( λ , w , x , ν ) ≜   w ◦ λ w + φ ( x )

Original: L ( x , λ , ν ) h ( x )   ∈ R 2 m + n + p , and
Revised: L ( x , λ , ν ) h ( x )   ∈ R 2 m + n + p , and

Original: H ( λ , w , x , ν ) ≜   w + φ ( x )
Revised: H ( λ , w , x , ν ) ≜   w + φ ( x )

Original: L ( x , λ , ν ) h ( x ) 
Revised: L ( x , λ , ν ) h ( x ) 

Original:  ∈
Revised:  ∈

Original: R m + n + p .
Revised: R m + n + p .

Original: Let H ++ ≜ H
Revised: Let H ++ ≜ H

Original: ( R 2 m ++ × R n × R p ) .
Revised: ( R 2 m ++ × R n × R p ) .

Original: By [Corollary 11.4.24, 26] we have the following proposition.
Revised: By [Corollary 11.4.24, 26] we have the following proposition.

Original: Proposition 1 (sufficient condition for the existence of the central path.)
Revised: Proposition 1 (sufficient condition for the existence of the central path.)

Original: If F is monotone, either F is strictly monotone or one of φ i is strictly convex, and C is bounded.
Revised: If F is monotone, either F is strictly monotone or one of φ i is strictly convex, and C is bounded.

Original: The following four statements hold for the functions G and H :
Revised: The following four statements hold for the functions G and H :

Original: G maps R 2 m ++ × R n + p homeomorphically onto R m ++ × H
Revised: G maps R 2 m ++ × R n + p homeomorphically onto R m ++ × H

Original: ++ ; 2.
Revised: ++ ; 2.

Original: R m ++ × H ++ ⊆ G
Revised: R m ++ × H ++ ⊆ G

Original: ( R 2 m + × R n + p ) ;
Revised: ( R 2 m + × R n + p ) ;

Original: for every vector a ∈ R m + , the system
Revised: for every vector a ∈ R m + , the system

Original: [Equation]
Revised: [Equation]

Original: has a solution ( λ , w , x , ν ) ∈ R 2 m + × R n + p ; and 4.
Revised: has a solution ( λ , w , x , ν ) ∈ R 2 m + × R n + p ; and 4.

Original: the set H ++ is convex.
Revised: the set H ++ is convex.

Original: B.5. Saddle-point optimization methods
Revised: B.5. Saddle-point optimization methods

Original: In this section, we describe in detail the saddle point methods that we compare with in the main paper in § 4.
Revised: In this section, we describe in detail the saddle point methods that we compare with in the main paper in § 4.

Original: We denote the projection to the set X with Π X , and when the method is applied in the unconstrained setting Π X ≡ I .
Revised: We denote the projection to the set X with Π X , and when the method is applied in the unconstrained setting Π X ≡ I .

Original: For an example of the associated vector field and its Jacobian, consider the following constrained zero-sum game:
Revised: For an example of the associated vector field and its Jacobian, consider the following constrained zero-sum game:

Original: [Equation]
Revised: [Equation]

Original: where f : X 1 × X 2 → R is smooth and convex in x 1 and concave in x 2 .
Revised: where f : X 1 × X 2 → R is smooth and convex in x 1 and concave in x 2 .

Original: As in the main paper, we write x ≜ ( x 1 , x 2 ) ∈ R n .
Revised: As in the main paper, we write x ≜ ( x 1 , x 2 ) ∈ R n .

Original: The vector field F : X → R n and its Jacobian J are defined as:
Revised: The vector field F : X → R n and its Jacobian J are defined as:

Original: [Equation]
Revised: [Equation]

Original: In the remaining of this section, we will only refer to the joint variable x , and (with abuse of notation) the subscript will denote the step.
Revised: In the remaining of this section, we will only refer to the joint variable x , and (with abuse of notation) the subscript will denote the step.

Original: Let γ ∈ [0 , 1] denote the step size.
Revised: Let γ ∈ [0 , 1] denote the step size.

Original: (Projected)
Revised: (Projected)

Original: Gradient Descent Ascent (GDA).
Revised: Gradient Descent Ascent (GDA).

Original: The extension of gradient descent for the cVI problem is gradient descent ascent (GDA).
Revised: The extension of gradient descent for the cVI problem is gradient descent ascent (GDA).

Original: The GDA update at step k is then:
Revised: The GDA update at step k is then:

Original: [Equation]
Revised: [Equation]

Original: (Projected) Extragradient (EG).
Revised: (Projected) Extragradient (EG).

Original: EG [46] uses a “prediction” step to obtain an extrapolated point x k + 12 using GDA: x k + 

Revised: EG [46] uses a “prediction” step to obtain an extrapolated point x k + 12 using GDA: x k +

Original: =
Revised: =

Original: Π X (cid:0) x k − γF ( x k ) (cid:1) , and the gradients at the extrapolated point are then applied to the current iterate x t :
Revised: Π X (cid:0) x k − γF ( x k ) (cid:1) , and the gradients at the extrapolated point are then applied to the current iterate x t :

Original: [Equation]
Revised: [Equation]

Original: In the original EG paper, [46] proved that the EG method (with a fixed step size) converges for monotone VIs, as follows.
Revised: In the original EG paper, [46] proved that the EG method (with a fixed step size) converges for monotone VIs, as follows.

Original: Theorem 4 (Korpelevich [46]) Given a map F : X (cid:55)→ R n , if the following is satisfied:
Revised: Theorem 4 (Korpelevich [46]) Given a map F : X (cid:55)→ R n , if the following is satisfied:

Original: the set X is closed and convex, 2.
Revised: the set X is closed and convex, 2.

Original: F is single-valued, definite, and monotone on X –as per Def. 1, 3.
Revised: F is single-valued, definite, and monotone on X –as per Def. 1, 3.

Original: F is L-Lipschitz–as per Asm. 1.
Revised: F is L-Lipschitz–as per Asm. 1.

Original: then there exists a solution x ⋆ ∈ X , such that the iterates x k produced by the EG update rule with a fixed step size γ ∈ (0 , 1 L ) converge to it, that is x k → x ⋆ , as k → ∞ .
Revised: then there exists a solution x ⋆ ∈ X , such that the iterates x k produced by the EG update rule with a fixed step size γ ∈ (0 , 1 L ) converge to it, that is x k → x ⋆ , as k → ∞ .

Original: Facchinei and Pang [26] also show that for any convex-concave function f and any closed convex sets x 1 ∈ X 1 and x 2 ∈ X 2 , the EG method converges [26, Theorem 12.1.11].
Revised: Facchinei and Pang [26] also show that for any convex-concave function f and any closed convex sets x 1 ∈ X 1 and x 2 ∈ X 2 , the EG method converges [26, Theorem 12.1.11].

Original: (Projected)
Revised: (Projected)

Original: Optimistic Gradient Descent Ascent (OGDA).
Revised: Optimistic Gradient Descent Ascent (OGDA).

Original: The update rule of Optimistic Gradient
Revised: The update rule of Optimistic Gradient

Original: Descent Ascent OGDA [(OGDA) 64] is:
Revised: Descent Ascent OGDA [(OGDA) 64] is:

Original: [Equation]
Revised: [Equation]

Original: (Projected) Lookahead–Minmax (LA).
Revised: (Projected) Lookahead–Minmax (LA).

Original: The LA algorithm for min-max optimization [14], originally proposed for minimization by [87], is a general wrapper of a “base” optimizer where at every step t : (i) a copy of the current iterate ˜ x n is made: ˜ x n ← x n , (ii) ˜ x n is updated k ≥ 1 times, yielding ˜ ω
Revised: The LA algorithm for min-max optimization [14], originally proposed for minimization by [87], is a general wrapper of a “base” optimizer where at every step t : (i) a copy of the current iterate ˜ x n is made: ˜ x n ← x n , (ii) ˜ x n is updated k ≥ 1 times, yielding ˜ ω

Original: n + k , and finally (iii) the actual update x n +1 is obtained as a point that lies on a line between the current x n iterate and the predicted one ˜ x n + k :
Revised: n + k , and finally (iii) the actual update x n +1 is obtained as a point that lies on a line between the current x n iterate and the predicted one ˜ x n + k :

Original: [Equation]
Revised: [Equation]

Original: In this work, we use solely GDA as a base optimizer for LA, and denote it with LA k -GDA .
Revised: In this work, we use solely GDA as a base optimizer for LA, and denote it with LA k -GDA .

Original: Appendix C. Deriving the ACVI algorithm
Revised: Appendix C. Deriving the ACVI algorithm

Original: In this section, we derive an interior-point method for the cVI problem that we refer to as ACVI ( A DMM-based i nterior problem for constrained VI s).
Revised: In this section, we derive an interior-point method for the cVI problem that we refer to as ACVI ( A DMM-based i nterior problem for constrained VI s).

Original: We first restate the cVI problem in a form that will allow us to derive an interior-point procedure.
Revised: We first restate the cVI problem in a form that will allow us to derive an interior-point procedure.

Original: By the definition of cVI it follows [see §1.
Revised: By the definition of cVI it follows [see §1.

Original: in 26] that:
Revised: in 26] that:

Original: [Equation]
Revised: [Equation]

Original: where λ ∈ R m and ν ∈ R p are dual variables.
Revised: where λ ∈ R m and ν ∈ R p are dual variables.

Original: Recall that we assume that int C ̸ = ∅ , thus, by the
Revised: Recall that we assume that int C ̸ = ∅ , thus, by the

Original: Slater condition (using the fact that φ i ( x ) , i ∈ [ m ] are convex) and the KKT conditions, the second equivalence holds, yielding the KKT system of cVI.
Revised: Slater condition (using the fact that φ i ( x ) , i ∈ [ m ] are convex) and the KKT conditions, the second equivalence holds, yielding the KKT system of cVI.

Original: Note that the above equivalence also guarantees the two solutions coincide; see Facchinei and Pang [26, Prop. 1.3.4 (b)].
Revised: Note that the above equivalence also guarantees the two solutions coincide; see Facchinei and Pang [26, Prop. 1.3.4 (b)].

Original: Analogous to the method described in § 2, we add a log -barrier term to the objective to remove the inequality constraints and obtain the following modified version of (KKT):
Revised: Analogous to the method described in § 2, we add a log -barrier term to the objective to remove the inequality constraints and obtain the following modified version of (KKT):

Original: [Equation]
Revised: [Equation]

Original: with µ > 0 , e ≜ [1 , . .
Revised: with µ > 0 , e ≜ [1 , . .

Original: . , 1] ⊺ ∈ R m .
Revised: . , 1] ⊺ ∈ R m .

Original: Again, the equivalence holds by the KKT and the Slater condition.
Revised: Again, the equivalence holds by the KKT and the Slater condition.

Original: We derive the update rule at step k via the following subproblem: min x F ( w k ) ⊺ x − µ
Revised: We derive the update rule at step k via the following subproblem: min x F ( w k ) ⊺ x − µ

Original: [Equation]
Revised: [Equation]

Original: i =
Revised: i =

Original: [Equation]
Revised: [Equation]

Original: log (cid:0) − φ i ( x ) (cid:1) , s.t.
Revised: log (cid:0) − φ i ( x ) (cid:1) , s.t.

Original: Cx = d , where we fix w = w k .
Revised: Cx = d , where we fix w = w k .

Original: Notice that (i) w k is a constant vector in this subproblem, and (ii) the objective is split, making ADMM a natural choice to solve the subproblem.
Revised: Notice that (i) w k is a constant vector in this subproblem, and (ii) the objective is split, making ADMM a natural choice to solve the subproblem.

Original: To apply an ADMM-type method, we introduce a new variable y ∈ R n yielding:
Revised: To apply an ADMM-type method, we introduce a new variable y ∈ R n yielding:

Original: [Equation]
Revised: [Equation]

Original: Note that 1 [ Cx = d ] is a generalized real-valued convex function of x .
Revised: Note that 1 [ Cx = d ] is a generalized real-valued convex function of x .

Original: We introduce the following:
Revised: We introduce the following:

Original: [Equation]
Revised: [Equation]

Original: where P c ∈ R n × n and d c ∈ R n .
Revised: where P c ∈ R n × n and d c ∈ R n .

Original: The augmented Lagrangian of (1) is thus:
Revised: The augmented Lagrangian of (1) is thus:

Original: [Equation]
Revised: [Equation]

Original: where β > 0 is the penalty parameter.
Revised: where β > 0 is the penalty parameter.

Original: Finally, using ADMM, we have the following update rule for x at step k :
Revised: Finally, using ADMM, we have the following update rule for x at step k :

Original: [Equation]
Revised: [Equation]

Original: This yields the following update for x :
Revised: This yields the following update for x :

Original: [Equation]
Revised: [Equation]

Original: For y and the dual variable λ , we have:
Revised: For y and the dual variable λ , we have:

Original: [Equation]
Revised: [Equation]

Original: Next, we derive the update rule for w .
Revised: Next, we derive the update rule for w .

Original: We set w k to be the solution of the following equation:
Revised: We set w k to be the solution of the following equation:

Original: [Equation]
Revised: [Equation]

Original: The following theorem ensures the solution of (W-EQ) exists and is unique, see App.
Revised: The following theorem ensures the solution of (W-EQ) exists and is unique, see App.

Original: D.2 for proof.
Revised: D.2 for proof.

Original: Theorem 5 (W-EQ: solution uniqueness)
Revised: Theorem 5 (W-EQ: solution uniqueness)

Original: If F is monotone on C = , the following statements hold true for the solution of (W-EQ) :
Revised: If F is monotone on C = , the following statements hold true for the solution of (W-EQ) :

Original: (i) it always exists, (ii) it is unique, and (iii) it is contained in C
Revised: (i) it always exists, (ii) it is unique, and (iii) it is contained in C

Original: = .
Revised: = .

Original: Remark 1 Note that when there are no equality constraints, C = becomes the entire space R n .
Revised: Remark 1 Note that when there are no equality constraints, C = becomes the entire space R n .

Original: Further notice that w k = x k +1 , thus it is redundant to state it in the algorithm, and we remove w .
Revised: Further notice that w k = x k +1 , thus it is redundant to state it in the algorithm, and we remove w .

Original: Appendix D. Omitted Proofs and Discussions concerning Algorithm
Revised: Appendix D. Omitted Proofs and Discussions concerning Algorithm

Original: This section first discusses the theoretical settings–that is, the assumptions of the main theorems, and then provides the proofs of the theoretical results in § 3.
Revised: This section first discusses the theoretical settings–that is, the assumptions of the main theorems, and then provides the proofs of the theoretical results in § 3.

Original: D.1. Discussion on the assumptions of the main theorems
Revised: D.1. Discussion on the assumptions of the main theorems

Original: We consider two broad classes of problems.
Revised: We consider two broad classes of problems.

Original: The first class assumes that F is ξ -monotone on
Revised: The first class assumes that F is ξ -monotone on

Original: C = —a stronger assumption than monotonicity, yet weaker than strong monotonicity.
Revised: C = —a stronger assumption than monotonicity, yet weaker than strong monotonicity.

Original: The second setup requires that (i) F is monotone, (ii) the constraints are active at the solution, and (iii) F is not purely rotational.
Revised: The second setup requires that (i) F is monotone, (ii) the constraints are active at the solution, and (iii) F is not purely rotational.

Original: Note that (iii) is weaker than requiring that the active constraints at the solution form an acute angle with the operator; in other words, given the latter, the former holds due to monotonicity of F .
Revised: Note that (iii) is weaker than requiring that the active constraints at the solution form an acute angle with the operator; in other words, given the latter, the former holds due to monotonicity of F .

Original: (See App. D).
Revised: (See App. D).

Original: Note that (iii) is not strong, as purely rotational games occur “almost never” in a Baire category sense [4, 43, 47, 74].
Revised: Note that (iii) is not strong, as purely rotational games occur “almost never” in a Baire category sense [4, 43, 47, 74].

Original: The proofs of the main theorems use the following lemma.
Revised: The proofs of the main theorems use the following lemma.

Original: Lemma 1 (Upper bound for G ( · ) )
Revised: Lemma 1 (Upper bound for G ( · ) )

Original: When F is L-Lipschitz on C = —as per Assumption 1—we have that any iterate x k produced by Algorithm 1 satisfies G ( x k , C ) ≤ M 

Revised: When F is L-Lipschitz on C = —as per Assumption 1—we have that any iterate x k produced by Algorithm 1 satisfies G ( x k , C ) ≤ M

Original: ∥ x
Revised: ∥ x

Original: k − x ⋆ ∥ , where M 0 >depends linearly on L , and x ⋆ ∈ S ⋆ C ,F .
Revised: k − x ⋆ ∥ , where M 0 >depends linearly on L , and x ⋆ ∈ S ⋆ C ,F .

Original: Assumption (iii) in Theorem 2 requires the angle of F ( x ) and x − x ⋆ to be acute on S\ { x ⋆ } , where S =
Revised: Assumption (iii) in Theorem 2 requires the angle of F ( x ) and x − x ⋆ to be acute on S\ { x ⋆ } , where S =

Original: ˆ C r or ˜ C s .
Revised: ˆ C r or ˜ C s .

Original: For example, when there are no equality constraints, Assumption (iii) becomes inf x ∈C\{ x ⋆ }
Revised: For example, when there are no equality constraints, Assumption (iii) becomes inf x ∈C\{ x ⋆ }

Original: F ( x ) ⊺ x − x ⋆ ∥ x − x ⋆ ∥ > 0 .
Revised: F ( x ) ⊺ x − x ⋆ ∥ x − x ⋆ ∥ > 0 .

Original: From (cVI) and by the monotonicity of F , we can see that for any point x ∈ C\ { x ⋆ } , the angle between F ( x ) and x − x ⋆ is always less than or equal to π/ 2 .
Revised: From (cVI) and by the monotonicity of F , we can see that for any point x ∈ C\ { x ⋆ } , the angle between F ( x ) and x − x ⋆ is always less than or equal to π/ 2 .

Original: And assumption (iii) requires that F ( x ⋆ ) ̸ = 0 , which means some constraints are active at x ⋆ , and ∃ θ ∈ (0 , π/ 2) s.t.
Revised: And assumption (iii) requires that F ( x ⋆ ) ̸ = 0 , which means some constraints are active at x ⋆ , and ∃ θ ∈ (0 , π/ 2) s.t.

Original: for any x ∈ C\ { x ⋆ } , the angle between F ( x ) and x − x ⋆ is upper bounded by θ .
Revised: for any x ∈ C\ { x ⋆ } , the angle between F ( x ) and x − x ⋆ is upper bounded by θ .

Original: Remark 2 Note that the convergence guarantee does not rely on Assumption 1, and it is solely used to relate the rate to the gap function.
Revised: Remark 2 Note that the convergence guarantee does not rely on Assumption 1, and it is solely used to relate the rate to the gap function.

Original: Also, note that µ − 1 does not impact the convergence rate.
Revised: Also, note that µ − 1 does not impact the convergence rate.

Original: Moreover, for simplicity we state the result with sufficiently small µ − 1 , however, the proof extends to any µ − 1 > 0 .
Revised: Moreover, for simplicity we state the result with sufficiently small µ − 1 , however, the proof extends to any µ − 1 > 0 .

Original: That is, the above result can be made parameter-free; see App.
Revised: That is, the above result can be made parameter-free; see App.

Original: D.
Revised: D.

Original: Remark 3 Our proofs rely on the existence of the central path—see Appendix B. Note that since C is compact, it suffices that either: (i) F is strictly monotone on C , or that (ii) one of the inequality constraints φ
Revised: Remark 3 Our proofs rely on the existence of the central path—see Appendix B. Note that since C is compact, it suffices that either: (i) F is strictly monotone on C , or that (ii) one of the inequality constraints φ

Original: i is strictly convex for the central path to exist [26, Corollary 11.4.24].
Revised: i is strictly convex for the central path to exist [26, Corollary 11.4.24].

Original: Thus, if F is ξ -monotone on C , then the central path exists.
Revised: Thus, if F is ξ -monotone on C , then the central path exists.

Original: However, to relax the former assumption, notice that— by the compactness of C —there exists a sufficiently large M such that for any x ∈ C , x ⊺ x ≤ M .
Revised: However, to relax the former assumption, notice that— by the compactness of C —there exists a sufficiently large M such that for any x ∈ C , x ⊺ x ≤ M .

Original: Thus, one can add a strictly convex inequality constraint φ m +1 ( x ) —e.g., x ⊺ x − M ≤ 0 —and the solution set remains intact.
Revised: Thus, one can add a strictly convex inequality constraint φ m +1 ( x ) —e.g., x ⊺ x − M ≤ 0 —and the solution set remains intact.

Original: That is, as µ tends to 0 the original problem is recovered.
Revised: That is, as µ tends to 0 the original problem is recovered.

Original: This ensures the existence of the central path without changing the original problem.
Revised: This ensures the existence of the central path without changing the original problem.

Original: D.2. Proof of Theorem 5: uniqueness of the solution of Eq. W-EQ
Revised: D.2. Proof of Theorem 5: uniqueness of the solution of Eq. W-EQ

Original: Recall first that Eq.
Revised: Recall first that Eq.

Original: W-EQ is as follows:
Revised: W-EQ is as follows:

Original: [Equation]
Revised: [Equation]

Original: since w = x .
Revised: since w = x .

Original: Proof [Proof of Theorem 5: uniqueness of the solution of (W-EQ)] Let G ( x ) denote the LHS of (W-EQ), that is:
Revised: Proof [Proof of Theorem 5: uniqueness of the solution of (W-EQ)] Let G ( x ) denote the LHS of (W-EQ), that is:

Original: [Equation]
Revised: [Equation]

Original: We claim that G ( x ) is strongly monotone on C = .
Revised: We claim that G ( x ) is strongly monotone on C = .

Original: In fact, ∀ x , y ∈ C = , P c ( x − y )
Revised: In fact, ∀ x , y ∈ C = , P c ( x − y )

Original: = x − y .
Revised: = x − y .

Original: Note that P c is symmetric, thus we have:
Revised: Note that P c is symmetric, thus we have:

Original: [Equation]
Revised: [Equation]

Original: Therefore, according to Theorem 2.3.3 (b) in [26], S ⋆ C = ,G has a unique solution ˜ x ∈ C = .
Revised: Therefore, according to Theorem 2.3.3 (b) in [26], S ⋆ C = ,G has a unique solution ˜ x ∈ C = .

Original: Thus, we have:
Revised: Thus, we have:

Original: [Equation]
Revised: [Equation]

Original: From the above, we deduce that G ( x ) ∈ Span { c 1 , · · · , c p } , where c i is the row vectors of C , i ∈
Revised: From the above, we deduce that G ( x ) ∈ Span { c 1 , · · · , c p } , where c i is the row vectors of C , i ∈

Original: [ p ] .
Revised: [ p ] .

Original: Suppose that G (˜ x ) = (cid:80) pi
Revised: Suppose that G (˜ x ) = (cid:80) pi

Original: =

Revised: =

Original: α i c i .
Revised: α i c i .

Original: Notice that C G ( x ) = 0 , ∀ x ∈ C = .
Revised: Notice that C G ( x ) = 0 , ∀ x ∈ C = .

Original: Thus, we have that:
Revised: Thus, we have that:

Original: [Equation]
Revised: [Equation]

Original: p ] .
Revised: p ] .

Original: Hence,
Revised: Hence,

Original: [Equation]
Revised: [Equation]

Original: which indicates that G (˜ x ) = 0 .
Revised: which indicates that G (˜ x ) = 0 .

Original: Hence, ˜ x is a solution of (W-EQ) in C = .
Revised: Hence, ˜ x is a solution of (W-EQ) in C = .

Original: On the other hand, ∀ x ∈ R n , if x is a solution of (W-EQ), i.e. G ( x ) = 0 , then x ∈ C = .
Revised: On the other hand, ∀ x ∈ R n , if x is a solution of (W-EQ), i.e. G ( x ) = 0 , then x ∈ C = .

Original: By the uniqueness of ˜ x in C = we have that x = ˜ x , which means ˜ x is unique in R n .
Revised: By the uniqueness of ˜ x in C = we have that x = ˜ x , which means ˜ x is unique in R n .

Original: D.3. Proof of Lemma 1: Upper bound on the gap function
Revised: D.3. Proof of Lemma 1: Upper bound on the gap function

Original: Proof [Proof of Lemma 1: Upper bound on the gap function] Let x k denote an iterate produced by
Revised: Proof [Proof of Lemma 1: Upper bound on the gap function] Let x k denote an iterate produced by

Original: Algorithm 1, and let x ∈ C .
Revised: Algorithm 1, and let x ∈ C .

Original: Note that we always have x k ∈ C = .
Revised: Note that we always have x k ∈ C = .

Original: We have that:
Revised: We have that:

Original: [Equation]
Revised: [Equation]

Original: From the proof of Theorem 6 we know that x k is bounded, which gives: ⟨ F ( x k ) , x k − x ⋆ ⟩ ≤
Revised: From the proof of Theorem 6 we know that x k is bounded, which gives: ⟨ F ( x k ) , x k − x ⋆ ⟩ ≤

Original: M ∥ x k − x ⋆ ∥ , with M > 0 , as well as that ⟨ F ( x ⋆ ) , x ⋆ − x ⟩ ≤ 0 .
Revised: M ∥ x k − x ⋆ ∥ , with M > 0 , as well as that ⟨ F ( x ⋆ ) , x ⋆ − x ⟩ ≤ 0 .

Original: Thus, for the above we get:
Revised: Thus, for the above we get:

Original: [Equation]
Revised: [Equation]

Original: where for the second row we used x ⋆ − x ≤ D where D ≜ max x ′ ∈C ∥ x ⋆ − x ′ ∥ is the largest distance between any point in C and x ⋆ .
Revised: where for the second row we used x ⋆ − x ≤ D where D ≜ max x ′ ∈C ∥ x ⋆ − x ′ ∥ is the largest distance between any point in C and x ⋆ .

Original: For the last row we used that F is L-Lipschitz—Assumption 1—which concludes the proof.
Revised: For the last row we used that F is L-Lipschitz—Assumption 1—which concludes the proof.

Original: The proof is analogous for the y k ∈ C ≤ iterates produced by Algorithm 1.
Revised: The proof is analogous for the y k ∈ C ≤ iterates produced by Algorithm 1.

Original: D.4. Proofs of the convergence rate: Theorems 1 and
Revised: D.4. Proofs of the convergence rate: Theorems 1 and

Original: Let
Revised: Let

Original: [Equation]
Revised: [Equation]

Original: where x µ t is a solution of (KKT-2) when µ = µ t .
Revised: where x µ t is a solution of (KKT-2) when µ = µ t .

Original: Note that the existence of x µ t is guaranteed by the existence of the central path-see App. B, and that f ( t ) , f ( t ) k and g ( t ) are all convex.
Revised: Note that the existence of x µ t is guaranteed by the existence of the central path-see App. B, and that f ( t ) , f ( t ) k and g ( t ) are all convex.

Original: In the following proofs, unless causing confusion, we drop the subscript t to simplify notations.
Revised: In the following proofs, unless causing confusion, we drop the subscript t to simplify notations.

Original: Let y µ = x µ , from (KKT-2) we can see that ( x µ , y µ ) is an optimal solution of
Revised: Let y µ = x µ , from (KKT-2) we can see that ( x µ , y µ ) is an optimal solution of

Original: [Equation]
Revised: [Equation]

Original: There exists λ µ ∈ R n such that ( x µ , y µ , λ µ ) is a KKT point of (4).
Revised: There exists λ µ ∈ R n such that ( x µ , y µ , λ µ ) is a KKT point of (4).

Original: We give the following proposition which we will repeatedly use in the proofs:
Revised: We give the following proposition which we will repeatedly use in the proofs:

Original: Proposition 2 If F is monotone, then ∀ k ∈ N ,
Revised: Proposition 2 If F is monotone, then ∀ k ∈ N ,

Original: [Equation]
Revised: [Equation]

Original: Furthermore, if F is ξ -monotone, as per Def.
Revised: Furthermore, if F is ξ -monotone, as per Def.

Original: [Equation]
Revised: [Equation]

Original: Proof [Proof of Proposition 2] It suffices to note that:
Revised: Proof [Proof of Proposition 2] It suffices to note that:

Original: [Equation]
Revised: [Equation]

Original: Some of our proofs that follow are inspired by some convergence proofs in ADMM [21, 25, 29, 40, 41, 51].
Revised: Some of our proofs that follow are inspired by some convergence proofs in ADMM [21, 25, 29, 40, 41, 51].

Original: However, although Algorithm 1 adopts the high level idea of ADMM, we can not directly refer to the convergence proofs of ADMM, but need to substantially modify these.
Revised: However, although Algorithm 1 adopts the high level idea of ADMM, we can not directly refer to the convergence proofs of ADMM, but need to substantially modify these.

Original: We will use the following lemma.
Revised: We will use the following lemma.

Original: [Equation]
Revised: [Equation]

Original: Proof The Lagrange function of (4) is
Revised: Proof The Lagrange function of (4) is

Original: [Equation]
Revised: [Equation]

Original: And by the property of KKT point we have
Revised: And by the property of KKT point we have

Original: [Equation]
Revised: [Equation]

Original: from which the conclusion follows.
Revised: from which the conclusion follows.

Original: The following lemma is straightforward to verify:
Revised: The following lemma is straightforward to verify:

Original: Lemma 

Revised: Lemma

Original: If
Revised: If

Original: [Equation]
Revised: [Equation]

Original: then we have
Revised: then we have

Original: [Equation]
Revised: [Equation]

Original: The following lemma lists some simple but useful facts that we will use in the following proofs.
Revised: The following lemma lists some simple but useful facts that we will use in the following proofs.

Original: Lemma 

Revised: Lemma

Original: For (4) and Algorithm 1, we have
Revised: For (4) and Algorithm 1, we have

Original: [Equation]
Revised: [Equation]

Original: We define:
Revised: We define:

Original: [Equation]
Revised: [Equation]

Original: Then from (5) and (6) we can see that
Revised: Then from (5) and (6) we can see that

Original: [Equation]
Revised: [Equation]

Original: x k +

Revised: x k +

Original: ) ∈
Revised: ) ∈

Original: ∂f k ( x k +1 ) and ˆ ∇ g ( y k +1 ) ∈ ∂g
Revised: ∂f k ( x k +1 ) and ˆ ∇ g ( y k +1 ) ∈ ∂g

Original: ( y k +1 ) .
Revised: ( y k +1 ) .

Original: [Equation]
Revised: [Equation]

Original: Lemma 

Revised: Lemma

Original: For Algorithm 1, we have
Revised: For Algorithm 1, we have

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: Proof [Proof of Lemma 5] From (7), (11) and (12) we have:
Revised: Proof [Proof of Lemma 5] From (7), (11) and (12) we have:

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: Adding these together yields (15).
Revised: Adding these together yields (15).

Original: Lemma 6 For Algorithm 1, we have
Revised: Lemma 6 For Algorithm 1, we have

Original: [Equation]
Revised: [Equation]

Original: Proof [Proof of Lemma 6] Letting ( x , y , λ ) = ( x µ , y µ , λ µ ) in(15), adding ⟨ λ µ , x k +1 − y k +1 ⟩ to both sides, and using (7) and (10), we have: ⟨ ˆ
Revised: Proof [Proof of Lemma 6] Letting ( x , y , λ ) = ( x µ , y µ , λ µ ) in(15), adding ⟨ λ µ , x k +1 − y k +1 ⟩ to both sides, and using (7) and (10), we have: ⟨ ˆ

Original: [Equation]
Revised: [Equation]

Original: f k
Revised: f k

Original: [Equation]
Revised: [Equation]

Original: +1 ) , x
Revised: +1 ) , x

Original: [Equation]
Revised: [Equation]

Original: +1 −
Revised: +1 −

Original: [Equation]
Revised: [Equation]

Original: µ ⟩
Revised: µ ⟩

Original: [Equation]
Revised: [Equation]

Original: g
Revised: g

Original: [Equation]
Revised: [Equation]

Original: y k +1 )
Revised: y k +1 )

Original: [Equation]
Revised: [Equation]

Original: y k +1 − y µ ⟩ + ⟨ λ µ , x k +1 −
Revised: y k +1 − y µ ⟩ + ⟨ λ µ , x k +1 −

Original: y k +1 ⟩
Revised: y k +1 ⟩

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: ⟩ =
Revised: ⟩ =

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: k
Revised: k

Original: [Equation]
Revised: [Equation]

Original: ⟨−
Revised: ⟨−

Original: [Equation]
Revised: [Equation]

Original: 1 +
Revised: 1 +

Original: [Equation]
Revised: [Equation]

Original: On the other hand, (14) gives
Revised: On the other hand, (14) gives

Original: [Equation]
Revised: [Equation]

Original: Letting y
Revised: Letting y

Original: =
Revised: =

Original: y k in (14) and y =
Revised: y k in (14) and y =

Original: y k +1 in (18), and adding them together, we have:
Revised: y k +1 in (18), and adding them together, we have:

Original: [Equation]
Revised: [Equation]

Original: By the monotonicity of ∂g we know that the first term of the above equality is non-negative.
Revised: By the monotonicity of ∂g we know that the first term of the above equality is non-negative.

Original: Thus, we have:
Revised: Thus, we have:

Original: [Equation]
Revised: [Equation]

Original: Plugging it into (17), we have the conclusion.
Revised: Plugging it into (17), we have the conclusion.

Original: Lemma 7 For Algorithm 1, we have
Revised: Lemma 7 For Algorithm 1, we have

Original: [Equation]
Revised: [Equation]

Original: Furthermore, if F is ξ -monotone on C =
Revised: Furthermore, if F is ξ -monotone on C =

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Proof From the convexity of f k ( x ) and g ( y ) and using Proposition 2 and (13)
Revised: Proof From the convexity of f k ( x ) and g ( y ) and using Proposition 2 and (13)

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: If F is ξ -monotone on C = , again by Proposition 2, we can add the term c
Revised: If F is ξ -monotone on C = , again by Proposition 2, we can add the term c

Original: ∥ x k +1 − x µ ∥ ξ 2 to the first line and the inequality still holds.
Revised: ∥ x k +1 − x µ ∥ ξ 2 to the first line and the inequality still holds.

Original: Theorem 6 For Algorithm 1, we have
Revised: Theorem 6 For Algorithm 1, we have

Original: [Equation]
Revised: [Equation]

Original: as k → ∞ .
Revised: as k → ∞ .

Original: Furthermore, if F is ξ -monotone on C = , we have
Revised: Furthermore, if F is ξ -monotone on C = , we have

Original: [Equation]
Revised: [Equation]

Original: Proof [Proof of Theorem 6] Proof From Lemma 2 and (20), we have
Revised: Proof [Proof of Theorem 6] Proof From Lemma 2 and (20), we have

Original: [Equation]
Revised: [Equation]

Original: Summing over k = 0 , · · · , ∞ , we have
Revised: Summing over k = 0 , · · · , ∞ , we have

Original: [Equation]
Revised: [Equation]

Original: from which we deduce that λ
Revised: from which we deduce that λ

Original: k +1 − λ k
Revised: k +1 − λ k

Original: → 0 and − y
Revised: → 0 and − y

Original: k +

Revised: k +

Original: + y k
Revised: + y k

Original: → 0 .
Revised: → 0 .

Original: Moreover, ∥ λ k − λ µ ∥ 2 and ∥− y k + y µ ∥ 2 are bounded for all k , as well as ∥ λ k ∥ .
Revised: Moreover, ∥ λ k − λ µ ∥ 2 and ∥− y k + y µ ∥ 2 are bounded for all k , as well as ∥ λ k ∥ .

Original: Since
Revised: Since

Original: [Equation]
Revised: [Equation]

Original: we deduce that x k +1 − y k +

Revised: we deduce that x k +1 − y k +

Original: → 0 and x k +1 − x µ is also bounded.
Revised: → 0 and x k +1 − x µ is also bounded.

Original: From (15) and the convexity of f and g , and using Proposition 2, we have:
Revised: From (15) and the convexity of f and g , and using Proposition 2, we have:

Original: [Equation]
Revised: [Equation]

Original: On the other hand, from (8), (9), and (10), we have:
Revised: On the other hand, from (8), (9), and (10), we have:

Original: [Equation]
Revised: [Equation]

Original: ≥
Revised: ≥

Original: [Equation]
Revised: [Equation]

Original: Thus, we have f ( x k +1 )
Revised: Thus, we have f ( x k +1 )

Original: − f ( x µ )+ g ( y k +1 )
Revised: − f ( x µ )+ g ( y k +1 )

Original: − g ( y µ ) → 0 and f k ( x k +1 )
Revised: − g ( y µ ) → 0 and f k ( x k +1 )

Original: − f k ( x µ )+ g ( y k +1 )
Revised: − f k ( x µ )+ g ( y k +1 )

Original: − g ( y µ ) → 0 , k → ∞ .
Revised: − g ( y µ ) → 0 , k → ∞ .

Original: If F is ξ -monotone on C = , from Lemma 2 and (21) we have
Revised: If F is ξ -monotone on C = , from Lemma 2 and (21) we have

Original: [Equation]
Revised: [Equation]

Original: From which we deduce that:
Revised: From which we deduce that:

Original: [Equation]
Revised: [Equation]

Original: Therefore, ∥ x k +1 − x µ ∥ 2 → 0 , k → ∞ .
Revised: Therefore, ∥ x k +1 − x µ ∥ 2 → 0 , k → ∞ .

Original: Lemma 8 For Algorithm 1, we have
Revised: Lemma 8 For Algorithm 1, we have

Original: [Equation]
Revised: [Equation]

Original: Furthermore, if F is ξ -monotone on C = , we have
Revised: Furthermore, if F is ξ -monotone on C = , we have

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Proof [Proof of Lemma 8] (15) gives:
Revised: Proof [Proof of Lemma 8] (15) gives:

Original: [Equation]
Revised: [Equation]

Original: Letting ( x , y , λ ) =
Revised: Letting ( x , y , λ ) =

Original: ( x k , y k , λ k ) in (15) and ( x , y , λ ) =
Revised: ( x k , y k , λ k ) in (15) and ( x , y , λ ) =

Original: ( x k +1 , y k
Revised: ( x k +1 , y k

Original: +1 , λ k +1 ) in (27), and adding them together, and using (7), we have
Revised: +1 , λ k +1 ) in (27), and adding them together, and using (7), we have

Original: [Equation]
Revised: [Equation]

Original: Note that
Revised: Note that

Original: [Equation]
Revised: [Equation]

Original: Using the monotonicity of ∂f k , ∂g and F
Revised: Using the monotonicity of ∂f k , ∂g and F

Original: [Equation]
Revised: [Equation]

Original: we draw the conclusion
Revised: we draw the conclusion

Original: [Equation]
Revised: [Equation]

Original: Theorem 7 If F is monotone on C = , then for Algorithm 1, we have
Revised: Theorem 7 If F is monotone on C = , then for Algorithm 1, we have

Original: [Equation]
Revised: [Equation]

Original: where ∆ ≜ 1 β ∥ λ 0 − λ µ ∥ 2 + β ∥ y 0 − y µ ∥ 2 .
Revised: where ∆ ≜ 1 β ∥ λ 0 − λ µ ∥ 2 + β ∥ y 0 − y µ ∥ 2 .

Original: Furthermore, if F is ξ -monotone on C =
Revised: Furthermore, if F is ξ -monotone on C =

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: 31) where ˆ x K +1 =
Revised: 31) where ˆ x K +1 =

Original: [Equation]
Revised: [Equation]

Original: Proof [Proof of Theorem 7.]
Revised: Proof [Proof of Theorem 7.]

Original: Summing (23) over k = 0 , 1 , . .
Revised: Summing (23) over k = 0 , 1 , . .

Original: , K and using the monotonicity of 1 2 β ∥ λ k
Revised: , K and using the monotonicity of 1 2 β ∥ λ k

Original: +

Revised: +

Original: − λ k
Revised: − λ k

Original: ∥ 2 + β 2 ∥−
Revised: ∥ 2 + β 2 ∥−

Original: y k +

Revised: y k +

Original: + y k
Revised: + y k

Original: ∥ 2 from Lemma 8, we have:
Revised: ∥ 2 from Lemma 8, we have:

Original: [Equation]
Revised: [Equation]

Original: From which we deduce that
Revised: From which we deduce that

Original: [Equation]
Revised: [Equation]

Original: On the other hand, (23) gives:
Revised: On the other hand, (23) gives:

Original: [Equation]
Revised: [Equation]

Original: Hence, we have:
Revised: Hence, we have:

Original: [Equation]
Revised: [Equation]

Original: Then from (16) and the convexity of f and g
Revised: Then from (16) and the convexity of f and g

Original: [Equation]
Revised: [Equation]

Original: we have:
Revised: we have:

Original: [Equation]
Revised: [Equation]

Original: From Lemma 3, we have (28).
Revised: From Lemma 3, we have (28).

Original: If in addition F is ξ -monotone on C = , using (24), we can obtain the following inequality similar to (32):
Revised: If in addition F is ξ -monotone on C = , using (24), we can obtain the following inequality similar to (32):

Original: [Equation]
Revised: [Equation]

Original: By the convexity of ∥·∥ ξ 2 we have c ∥
Revised: By the convexity of ∥·∥ ξ 2 we have c ∥

Original: ˆ x K +1 − x µ ∥ ξ 2 ≤ ∆ K +1 .
Revised: ˆ x K +1 − x µ ∥ ξ 2 ≤ ∆ K +1 .

Original: And from (34) we can see that
Revised: And from (34) we can see that

Original: [Equation]
Revised: [Equation]

Original: Theorem 8 If F is monotone on C = , then for Algorithm 1, we have
Revised: Theorem 8 If F is monotone on C = , then for Algorithm 1, we have

Original: [Equation]
Revised: [Equation]

Original: where ˆ x K +1 =
Revised: where ˆ x K +1 =

Original: [Equation]
Revised: [Equation]

Original: +
Revised: +

Original: [Equation]
Revised: [Equation]

Original: = 1 K +
Revised: = 1 K +

Original: [Equation]
Revised: [Equation]

Original: (cid:80) K +1 k
Revised: (cid:80) K +1 k

Original: =1 y k , and ∆ ≜
Revised: =1 y k , and ∆ ≜

Original: 1 β ∥ λ 0 − λ µ ∥ 2 + β ∥
Revised: 1 β ∥ λ 0 − λ µ ∥ 2 + β ∥

Original: y 0 − y µ ∥ 2 .
Revised: y 0 − y µ ∥ 2 .

Original: Proof [Proof of Theorem 8] Summing (20) over k = 0 , 1 , . . .
Revised: Proof [Proof of Theorem 8] Summing (20) over k = 0 , 1 , . . .

Original: , K , dividing both sides with K + 1 , and using the definitions of ˆ x K +1 and ˆ y K +1 and the convexity of f and g , we have
Revised: , K , dividing both sides with K + 1 , and using the definitions of ˆ x K +1 and ˆ y K +1 and the convexity of f and g , we have

Original: [Equation]
Revised: [Equation]

Original: From (7) and (33), we have:
Revised: From (7) and (33), we have:

Original: [Equation]
Revised: [Equation]

Original: Finally, from Lemma 3, the conclusion follows.
Revised: Finally, from Lemma 3, the conclusion follows.

Original: From Proposition 1 and the fact that C is compact we can see that lim µ → 0 x µ and lim µ → 0 λ µ exist and are finite.
Revised: From Proposition 1 and the fact that C is compact we can see that lim µ → 0 x µ and lim µ → 0 λ µ exist and are finite.

Original: Let x ⋆ = lim µ → 0 x µ and λ ⋆ = lim µ → 0 λ µ , then x ⋆ ∈ S ⋆ C ,F .
Revised: Let x ⋆ = lim µ → 0 x µ and λ ⋆ = lim µ → 0 λ µ , then x ⋆ ∈ S ⋆ C ,F .

Original: Theorem∃ ˜ µ > 0 , s.t.
Revised: Theorem∃ ˜ µ > 0 , s.t.

Original: if µ t < ˜ µ , then (cid:12)(cid:12)(cid:12) F ( x ( t ) K +1 ) ⊺ ( x ( t
Revised: if µ t < ˜ µ , then (cid:12)(cid:12)(cid:12) F ( x ( t ) K +1 ) ⊺ ( x ( t

Original: [Equation]
Revised: [Equation]

Original: 1 − x
Revised: 1 − x

Original: [Equation]
Revised: [Equation]

Original: ≤
Revised: ≤

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: [Equation]
Revised: [Equation]

Original: ∥ λ ⋆ ∥ (cid:113) ∆ β
Revised: ∥ λ ⋆ ∥ (cid:113) ∆ β

Original: [Equation]
Revised: [Equation]

Original: F
Revised: F

Original: [Equation]
Revised: [Equation]

Original: ⊺
Revised: ⊺

Original: [Equation]
Revised: [Equation]

Original: − x
Revised: − x

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: +
Revised: +

Original: [Equation]
Revised: [Equation]

Original: √
Revised: √

Original: [Equation]
Revised: [Equation]

Original: + ∥ λ ⋆ ∥ (cid:113)
Revised: + ∥ λ ⋆ ∥ (cid:113)

Original: ∆ β ( K +1) ) and (cid:12)(cid:12)(cid:12) F ( x ⋆ ) ⊺ (ˆ x ( t ) K +1 − x ⋆ ) (cid:12)(cid:12)(cid:12) ≤ 2( ∆ 2( K
Revised: ∆ β ( K +1) ) and (cid:12)(cid:12)(cid:12) F ( x ⋆ ) ⊺ (ˆ x ( t ) K +1 − x ⋆ ) (cid:12)(cid:12)(cid:12) ≤ 2( ∆ 2( K

Original: [Equation]
Revised: [Equation]

Original: +
Revised: +

Original: [Equation]
Revised: [Equation]

Original: λ ⋆ ∥
Revised: λ ⋆ ∥

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Proof [Proof of Theorem 9.]
Revised: Proof [Proof of Theorem 9.]

Original: For simplicity we let B ( x ) denote the log -barrier term − (cid:80) mi =1 log ( − φ i ( x )) .
Revised: For simplicity we let B ( x ) denote the log -barrier term − (cid:80) mi =1 log ( − φ i ( x )) .

Original: From Theoremand 8 we have
Revised: From Theoremand 8 we have

Original: [Equation]
Revised: [Equation]

Original: From Proposition 1 in App.
Revised: From Proposition 1 in App.

Original: B we know that when µ → 0 , x µ → x ⋆ , λ µ → λ ⋆ and µ ( B ( y ( t ) K +1 ) −
Revised: B we know that when µ → 0 , x µ → x ⋆ , λ µ → λ ⋆ and µ ( B ( y ( t ) K +1 ) −

Original: B ( y µ ))
Revised: B ( y µ ))

Original: → 0 , so ∃ ˜ µ > 0 , s
Revised: → 0 , so ∃ ˜ µ > 0 , s

Original: [Equation]
Revised: [Equation]

Original: t. if µ t < ˜ µ , then we have
Revised: t. if µ t < ˜ µ , then we have

Original: [Equation]
Revised: [Equation]

Original: To make the dependencies of the constants clear, here we restate Theorem 1 and Theorem 2 and provide their proofs.
Revised: To make the dependencies of the constants clear, here we restate Theorem 1 and Theorem 2 and provide their proofs.

Original: Theorem 10 (restatement of Theorem 1) Given an operator F : X → R n monotone on C =
Revised: Theorem 10 (restatement of Theorem 1) Given an operator F : X → R n monotone on C =

Original: (Def. 1), and either F is strictly monotone on C or one of φ i is strictly convex.
Revised: (Def. 1), and either F is strictly monotone on C or one of φ i is strictly convex.

Original: Assume there exists r > 0 or s > 0 such that F is star-
Revised: Assume there exists r > 0 or s > 0 such that F is star-

Original: ξ -monotone—as per Def. 1—on either ˆ C r or ˜ C s , resp.
Revised: ξ -monotone—as per Def. 1—on either ˆ C r or ˜ C s , resp.

Original: Let x ( t ) K and ˆ x ( t ) K ≜ 1 K (cid:80)
Revised: Let x ( t ) K and ˆ x ( t ) K ≜ 1 K (cid:80)

Original: Kk
Revised: Kk

Original: =1 x ( t ) k denote the last and average iterate of Algorithm 1, respectively, run with sufficiently small µ − 1 .
Revised: =1 x ( t ) k denote the last and average iterate of Algorithm 1, respectively, run with sufficiently small µ − 1 .

Original: Then there exists K 0 ∈ N , K 0 depends on r or s ,
Revised: Then there exists K 0 ∈ N , K 0 depends on r or s ,

Original: s.t. ∀ K > K 0 , for all t ∈ [ T ] , we have that:
Revised: s.t. ∀ K > K 0 , for all t ∈ [ T ] , we have that:

Original: [Equation]
Revised: [Equation]

Original: If in addition F is ξ -monotone on C = , we have ∥ ˆ x ( t ) K − x ⋆ ∥ ≤ ( 2∆ cK ) 1 /ξ . 3.
Revised: If in addition F is ξ -monotone on C = , we have ∥ ˆ x ( t ) K − x ⋆ ∥ ≤ ( 2∆ cK ) 1 /ξ . 3.

Original: Moreover, if F is L-Lipschitz on C = —as per Assumption 1— then G ( x ( t ) K , C ) ≤ M 0 ( 4 c ( ∆ K + 2∆ √ K + ∥ λ ⋆ ∥ (cid:113) ∆ βK )) 1 /ξ and G (ˆ x ( t ) K , C ) ≤ M 0 ( 4 c ( ∆2 K + 2 √ β ∆ ∥ λ ⋆ ∥ βK
Revised: Moreover, if F is L-Lipschitz on C = —as per Assumption 1— then G ( x ( t ) K , C ) ≤ M 0 ( 4 c ( ∆ K + 2∆ √ K + ∥ λ ⋆ ∥ (cid:113) ∆ βK )) 1 /ξ and G (ˆ x ( t ) K , C ) ≤ M 0 ( 4 c ( ∆2 K + 2 √ β ∆ ∥ λ ⋆ ∥ βK

Original: [Equation]
Revised: [Equation]

Original: ) 1 /ξ ,
Revised: ) 1 /ξ ,

Original: Proof [Proof of Theorem 10.]
Revised: Proof [Proof of Theorem 10.]

Original: Without loss of generality, we suppose that F is star-
Revised: Without loss of generality, we suppose that F is star-

Original: ξ -monotone on ˆ C r .
Revised: ξ -monotone on ˆ C r .

Original: Since you ϕ ( y k ) ≤ 0 , from (29) we can see that ∃ K 0 ∈ N , K 0 depends on r , s.t.
Revised: Since you ϕ ( y k ) ≤ 0 , from (29) we can see that ∃ K 0 ∈ N , K 0 depends on r , s.t.

Original: ∀ K ≥
Revised: ∀ K ≥

Original: K 0 , x k ∈ ˆ C r , so if µ t < ˜ µ ( ˜ µ is defined in Theorem 9), by Theorem 9 we have
Revised: K 0 , x k ∈ ˆ C r , so if µ t < ˜ µ ( ˜ µ is defined in Theorem 9), by Theorem 9 we have

Original: [Equation]
Revised: [Equation]

Original: So we have
Revised: So we have

Original: [Equation]
Revised: [Equation]

Original: If in addition F is ξ -monotone on C = , then from (30) we know that when ˆ µ is small enough, we have
Revised: If in addition F is ξ -monotone on C = , then from (30) we know that when ˆ µ is small enough, we have

Original: [Equation]
Revised: [Equation]

Original: If F is L -Lipschitz on C = , then from Lemma 1 we can see that
Revised: If F is L -Lipschitz on C = , then from Lemma 1 we can see that

Original: [Equation]
Revised: [Equation]

Original: where M 0 is a linear function of L , see the proof of Lemma 1 in App.
Revised: where M 0 is a linear function of L , see the proof of Lemma 1 in App.

Original: B.2.
Revised: B.2.

Original: Theorem 11 (restatement of Theorem 2) Given an operator F : X → R n , assume: (i) F is monotone on C = , as per Def. 1; (ii) either F is strictly monotone on C or one of φ i is strictly convex; and (iii) inf
Revised: Theorem 11 (restatement of Theorem 2) Given an operator F : X → R n , assume: (i) F is monotone on C = , as per Def. 1; (ii) either F is strictly monotone on C or one of φ i is strictly convex; and (iii) inf

Original: [Equation]
Revised: [Equation]

Original: F
Revised: F

Original: [Equation]
Revised: [Equation]

Original: x ⋆ ∥ = a > 0 , where S ≡ ˆ C r or ˜ C s .
Revised: x ⋆ ∥ = a > 0 , where S ≡ ˆ C r or ˜ C s .

Original: Let x ( t ) K and ˆ x ( t
Revised: Let x ( t ) K and ˆ x ( t

Original: [Equation]
Revised: [Equation]

Original: K ≜
Revised: K ≜

Original: [Equation]
Revised: [Equation]

Original: denote the last and average iterate of Algorithm 1, respectively, run with sufficiently small µ − 1 .
Revised: denote the last and average iterate of Algorithm 1, respectively, run with sufficiently small µ − 1 .

Original: Then there exists K 0 ∈ N , K 0 depends on r or s ,
Revised: Then there exists K 0 ∈ N , K 0 depends on r or s ,

Original: s.t. ∀ t ∈
Revised: s.t. ∀ t ∈

Original: [ T ] , ∀ K > K 0 , we have that:
Revised: [ T ] , ∀ K > K 0 , we have that:

Original: (cid:13)(cid:13)(cid:13) x
Revised: (cid:13)(cid:13)(cid:13) x

Original: ( t ) K − x ⋆ (cid:13)(cid:13)(cid:13) ≤ 2 a (cid:16) ∆ K + 2∆ √ K + ∥
Revised: ( t ) K − x ⋆ (cid:13)(cid:13)(cid:13) ≤ 2 a (cid:16) ∆ K + 2∆ √ K + ∥

Original: λ ⋆ ∥ (cid:113) ∆ βK (cid:17) .
Revised: λ ⋆ ∥ (cid:113) ∆ βK (cid:17) .

Original: If in addition inf x ∈ S \{ x ⋆ } F ( x ⋆ ) ⊺ x − x ⋆ ∥ x − x ⋆ ∥ = b > 0 (with S ≡ ˆ C r or ˜ C s ), then (cid:13)(cid:13)(cid:13) ˆ x ( t ) K − x ⋆ (cid:13)(cid:13)(cid:13) ≤ 2 b (cid:16) ∆2 K + 2 √ β ∆ ∥ λ ⋆ ∥ βK (cid:17) .
Revised: If in addition inf x ∈ S \{ x ⋆ } F ( x ⋆ ) ⊺ x − x ⋆ ∥ x − x ⋆ ∥ = b > 0 (with S ≡ ˆ C r or ˜ C s ), then (cid:13)(cid:13)(cid:13) ˆ x ( t ) K − x ⋆ (cid:13)(cid:13)(cid:13) ≤ 2 b (cid:16) ∆2 K + 2 √ β ∆ ∥ λ ⋆ ∥ βK (cid:17) .

Original: Moreover, if F is L-Lipschitz on C = —as per Assumption 1—then G ( x ( t ) K , C ) ≤ 2 M 0 a (cid:16) ∆ K + 2∆ √ K + ∥
Revised: Moreover, if F is L-Lipschitz on C = —as per Assumption 1—then G ( x ( t ) K , C ) ≤ 2 M 0 a (cid:16) ∆ K + 2∆ √ K + ∥

Original: λ ⋆ ∥ (cid:113) ∆ βK (cid:17) and
Revised: λ ⋆ ∥ (cid:113) ∆ βK (cid:17) and

Original: [Equation]
Revised: [Equation]

Original: Proof [Proof of Theorem 11] Without loss of generality
Revised: Proof [Proof of Theorem 11] Without loss of generality

Original: [Equation]
Revised: [Equation]

Original: we suppose
Revised: we suppose

Original: [Equation]
Revised: [Equation]

Original: a > 0 .
Revised: a > 0 .

Original: When K ≥ K 0 ( K 0 and ˜ µ are defined in the proof of Theorem 10 and in Theorem
Revised: When K ≥ K 0 ( K 0 and ˜ µ are defined in the proof of Theorem 10 and in Theorem

Original: [Equation]
Revised: [Equation]

Original: resp.)
Revised: resp.)

Original: [Equation]
Revised: [Equation]

Original: by Theorem
Revised: by Theorem

Original: [Equation]
Revised: [Equation]

Original: we have that
Revised: we have that

Original: [Equation]
Revised: [Equation]

Original: Similarly, if inf
Revised: Similarly, if inf

Original: [Equation]
Revised: [Equation]

Original: 0 , we have that
Revised: 0 , we have that

Original: [Equation]
Revised: [Equation]

Original: If F is L -Lipschitz on C = , then from Lemma 1 we can see that
Revised: If F is L -Lipschitz on C = , then from Lemma 1 we can see that

Original: [Equation]
Revised: [Equation]

Original: From the above proofs we can see that by setting µ − 1 small enough, Theorem 1 and 2 hold true.
Revised: From the above proofs we can see that by setting µ − 1 small enough, Theorem 1 and 2 hold true.

Original: But since we do not know exactly how small µ − 1 should be, in practice, we can set a small µ − 1 , and µ t could be smaller than any fixed positive number after a very small number of outer loops, as µ t decays exponentially.
Revised: But since we do not know exactly how small µ − 1 should be, in practice, we can set a small µ − 1 , and µ t could be smaller than any fixed positive number after a very small number of outer loops, as µ t decays exponentially.

Original: Thus, Algorithm 1 is actually parameter-free.
Revised: Thus, Algorithm 1 is actually parameter-free.

Original: Note that the assumption (iii) in Theorem 2 is the weakening of the assumption
Revised: Note that the assumption (iii) in Theorem 2 is the weakening of the assumption

Original: [Equation]
Revised: [Equation]

Original: 0 , where S = ˆ C r or ˜ C s .
Revised: 0 , where S = ˆ C r or ˜ C s .

Original: In fact, by the monotonicity of F , we have F ( x ) ⊺ ( x − x ⋆ ) ≥ F ( x ⋆ ) ⊺ ( x − x ⋆ ) , so if inf
Revised: In fact, by the monotonicity of F , we have F ( x ) ⊺ ( x − x ⋆ ) ≥ F ( x ⋆ ) ⊺ ( x − x ⋆ ) , so if inf

Original: [Equation]
Revised: [Equation]

Original: , there must be
Revised: , there must be

Original: [Equation]
Revised: [Equation]

Original: D.5. Discussion on Algorithm
Revised: D.5. Discussion on Algorithm

Original: Advantages and disadvantages of Algorithm 1.
Revised: Advantages and disadvantages of Algorithm 1.

Original: In Algorithm 1, the update of x (step 7) requires solving (W-EQ).
Revised: In Algorithm 1, the update of x (step 7) requires solving (W-EQ).

Original: Our method is especially suitable for problems where (W-EQ) is easy to solve analytically.
Revised: Our method is especially suitable for problems where (W-EQ) is easy to solve analytically.

Original: This includes the class of affine variational inequalities, low-dimensional problems, and when optimization variables represent probabilities, for example.
Revised: This includes the class of affine variational inequalities, low-dimensional problems, and when optimization variables represent probabilities, for example.

Original: For problems where (W-EQ) is hard to solve—for example, min-max optimization problems in
Revised: For problems where (W-EQ) is hard to solve—for example, min-max optimization problems in

Original: GANs—one could use other unconstrained methods like GDA or EG methods ( without projection) so as to solve V I ( R n , G ) , where G is defined in (3).
Revised: GANs—one could use other unconstrained methods like GDA or EG methods ( without projection) so as to solve V I ( R n , G ) , where G is defined in (3).

Original: Algorithm 3 describes ACVI when using an unconstrained solver for the inner problems.
Revised: Algorithm 3 describes ACVI when using an unconstrained solver for the inner problems.

Original: We observe that Algorithm 3 outperforms the projection-based baseline algorithms, see for example Fig. 4.
Revised: We observe that Algorithm 3 outperforms the projection-based baseline algorithms, see for example Fig. 4.

Original: Note that when there are constraints, the projection-based methods such as GDA, EG, OGDA etc. require solving a quadratic programming problem in each iteration (or twice per iteration for EG).
Revised: Note that when there are constraints, the projection-based methods such as GDA, EG, OGDA etc. require solving a quadratic programming problem in each iteration (or twice per iteration for EG).

Original: This problem is often nontrivial and solving it may require using an interior point method or variants (such as the Frank-Wolfe algorithm).
Revised: This problem is often nontrivial and solving it may require using an interior point method or variants (such as the Frank-Wolfe algorithm).

Original: Because of this, Algorithm 1 can be seen as an orthogonal approach to projection-based methods, or in other words, as a complementary tool to solve (cVI) and particularly relevant when the constraints are non-trivial.
Revised: Because of this, Algorithm 1 can be seen as an orthogonal approach to projection-based methods, or in other words, as a complementary tool to solve (cVI) and particularly relevant when the constraints are non-trivial.

Original: ACVI with only equality or inequality constraints.
Revised: ACVI with only equality or inequality constraints.

Original: If there are no equality constraints, then C = becomes R n .
Revised: If there are no equality constraints, then C = becomes R n .

Original: In this case, we have that x ( t ) k
Revised: In this case, we have that x ( t ) k

Original: +1 is the solution of:
Revised: +1 is the solution of:

Original: [Equation]
Revised: [Equation]

Original: When there are no inequality constraints, we let y k
Revised: When there are no inequality constraints, we let y k

Original: = x k and λ k = 0 for every k , and we can remove the outer loop.
Revised: = x k and λ k = 0 for every k , and we can remove the outer loop.

Original: Thus, Algorithm 1 can be simplified to update only one variable x each iteration with the following updating rule: x
Revised: Thus, Algorithm 1 can be simplified to update only one variable x each iteration with the following updating rule: x

Original: [Equation]
Revised: [Equation]

Original: is the unique solution of
Revised: is the unique solution of

Original: [Equation]
Revised: [Equation]

Original: Appendix E. Variant of the ACVI Algorithm (v-ACVI)
Revised: Appendix E. Variant of the ACVI Algorithm (v-ACVI)

Original: The presented approach of combining interior point methods with ADMM can be used as a framework to derive additional algorithms that may be more suitable for some specific problems.
Revised: The presented approach of combining interior point methods with ADMM can be used as a framework to derive additional algorithms that may be more suitable for some specific problems.

Original: More precisely, observe from Eq.
Revised: More precisely, observe from Eq.

Original: 1 that we could also consider a different splitting than that in § 3.
Revised: 1 that we could also consider a different splitting than that in § 3.

Original: Following this approach, we present a variant of Algorithm 1 and discuss its advantages and disadvantages relative to Algorithm 1.
Revised: Following this approach, we present a variant of Algorithm 1 and discuss its advantages and disadvantages relative to Algorithm 1.

Original: E.1. Introduction of the variant ACVI
Revised: E.1. Introduction of the variant ACVI

Original: Deriving the v-ACVI algorithm.
Revised: Deriving the v-ACVI algorithm.

Original: By considering a different splitting in (1) we get:
Revised: By considering a different splitting in (1) we get:

Original: [Equation]
Revised: [Equation]

Original: where:
Revised: where:

Original: [Equation]
Revised: [Equation]

Original: The augmented Lagrangian of (35) is thus:
Revised: The augmented Lagrangian of (35) is thus:

Original: [Equation]
Revised: [Equation]

Original: where β > 0 is the penalty parameter.
Revised: where β > 0 is the penalty parameter.

Original: We have that for x at step k + 1 :
Revised: We have that for x at step k + 1 :

Original: [Equation]
Revised: [Equation]

Original: The following proposition ensures the existence and uniqueness of x k +1 in C < .
Revised: The following proposition ensures the existence and uniqueness of x k +1 in C < .

Original: i.e. We show that x k +1 is the unique solution in C < of the following closed form equation (see App.
Revised: i.e. We show that x k +1 is the unique solution in C < of the following closed form equation (see App.

Original: E.2 for its proof):
Revised: E.2 for its proof):

Original: [Equation]
Revised: [Equation]

Original: Proposition 3 (unique solution)
Revised: Proposition 3 (unique solution)

Original: The problem (X-CF) has a solution in C < and the solution is unique.
Revised: The problem (X-CF) has a solution in C < and the solution is unique.

Original: For y , the updating rule is
Revised: For y , the updating rule is

Original: [Equation]
Revised: [Equation]

Original: And the updating rule for dual variable λ is
Revised: And the updating rule for dual variable λ is

Original: [Equation]
Revised: [Equation]

Original: As in § 3, we would like to choose w k so that w k = x k +1 .
Revised: As in § 3, we would like to choose w k so that w k = x k +1 .

Original: To this end, we need the following assumption:
Revised: To this end, we need the following assumption:

Original: Assumption
Revised: Assumption

Original: [Equation]
Revised: [Equation]

Original: If Assumption 2 holds true, we can let w
Revised: If Assumption 2 holds true, we can let w

Original: k
Revised: k

Original: be a solution of
Revised: be a solution of

Original: [Equation]
Revised: [Equation]

Original: in C < .
Revised: in C < .

Original: And by Proposition 3, we can let x k +1 be the unique solution of
Revised: And by Proposition 3, we can let x k +1 be the unique solution of

Original: [Equation]
Revised: [Equation]

Original: in C < .
Revised: in C < .

Original: Note that w k is also a solution of ( x ).
Revised: Note that w k is also a solution of ( x ).

Original: By the uniqueness of the solution of ( x ) shown in Prop. 3, we know that w k = x k +1 .
Revised: By the uniqueness of the solution of ( x ) shown in Prop. 3, we know that w k = x k +1 .

Original: So in the ( k + 1) -th step, we can compute x , y , λ and w by ( x ), ( y ), ( λ ) and (37), respectively.
Revised: So in the ( k + 1) -th step, we can compute x , y , λ and w by ( x ), ( y ), ( λ ) and (37), respectively.

Original: Since w k = x k +1 , we can simplify our algorithm by removing variable w and only keep x , y and λ , see Algorithm 2.
Revised: Since w k = x k +1 , we can simplify our algorithm by removing variable w and only keep x , y and λ , see Algorithm 2.

Original: 1: Input: operator F : X → R n , equality Cx = d and inequality constraints φ i ( x ) ≤ 0 , i = [ m ] , hyperparameters µ − 1 , β > 0 , δ ∈ (0 , 1) , number of outer and inner loop iterations T and K , resp.
Revised: 1: Input: operator F : X → R n , equality Cx = d and inequality constraints φ i ( x ) ≤ 0 , i = [ m ] , hyperparameters µ − 1 , β > 0 , δ ∈ (0 , 1) , number of outer and inner loop iterations T and K , resp.

Original: 2: Initialize y (0) ∈ R n , λ (0) ∈ R n 3: P c ≜
Revised: 2: Initialize y (0) ∈ R n , λ (0) ∈ R n 3: P c ≜

Original: I
Revised: I

Original: [Equation]
Revised: [Equation]

Original: C
Revised: C

Original: [Equation]
Revised: [Equation]

Original: ( CC
Revised: ( CC

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: C where P c ∈ R n × n 4: d c ≜ C ⊺ ( CC ⊺ ) − 1 d where d c ∈ R n 5: for t = 0 , . . .
Revised: C where P c ∈ R n × n 4: d c ≜ C ⊺ ( CC ⊺ ) − 1 d where d c ∈ R n 5: for t = 0 , . . .

Original: , T − 1 do 6: µ t =
Revised: , T − 1 do 6: µ t =

Original: δµ t 7:
Revised: δµ t 7:

Original: Denote ˆ φ ( λ , y ) as the solution of 1 β (cid:0) µ t (cid:80) mi =1 ∇ φ
Revised: Denote ˆ φ ( λ , y ) as the solution of 1 β (cid:0) µ t (cid:80) mi =1 ∇ φ

Original: i ( x ) φ i ( x ) − F ( x ) − λ (cid:1) + y − x = 0 with respect to x , where y , λ are variables 8: for k = 0 , . . .
Revised: i ( x ) φ i ( x ) − F ( x ) − λ (cid:1) + y − x = 0 with respect to x , where y , λ are variables 8: for k = 0 , . . .

Original: , K − 1 do 9:
Revised: , K − 1 do 9:

Original: [Equation]
Revised: [Equation]

Original: Ensures x k +1 ∈ X ≤ 10:
Revised: Ensures x k +1 ∈ X ≤ 10:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: 12: end for 13: ( y ( t +1), λ
Revised: 12: end for 13: ( y ( t +1), λ

Original: [Equation]
Revised: [Equation]

Original: t +1)
Revised: t +1)

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: 14: end for
Revised: 14: end for

Original: Discussion: ACVI Vs.
Revised: Discussion: ACVI Vs.

Original: v-ACVI.
Revised: v-ACVI.

Original: Relative to Algorithm 1, the subproblem for solving x in linein Algorithm 2 becomes more complex, whereas the subproblem for y becomes simpler.
Revised: Relative to Algorithm 1, the subproblem for solving x in linein Algorithm 2 becomes more complex, whereas the subproblem for y becomes simpler.

Original: Hence, in cases when the inequality constraints are simpler, or there are no inequality constraints Alg.
Revised: Hence, in cases when the inequality constraints are simpler, or there are no inequality constraints Alg.

Original: 2 may be more suitable, as that simplifies the x subproblem.
Revised: 2 may be more suitable, as that simplifies the x subproblem.

Original: However, Algorithm 1 balances better the complexities of the subproblems, hence it may be simpler to use for general problems.
Revised: However, Algorithm 1 balances better the complexities of the subproblems, hence it may be simpler to use for general problems.

Original: Convergence of v-ACVI.
Revised: Convergence of v-ACVI.

Original: By analogous proofs to those in App.
Revised: By analogous proofs to those in App.

Original: D, we can get similar convergence results as for Algorithm 1, that is Theorems 1 and 2.
Revised: D, we can get similar convergence results as for Algorithm 1, that is Theorems 1 and 2.

Original: Specifically, Theorem 1 and 2 hold for Algorithm 2, provided that we replace the assumption “ F is monotone on C = ” with “ F is monotone on C ≤ ” in
Revised: Specifically, Theorem 1 and 2 hold for Algorithm 2, provided that we replace the assumption “ F is monotone on C = ” with “ F is monotone on C ≤ ” in

Original: Theorems 1 and 2.
Revised: Theorems 1 and 2.

Original: E.2. Proof of Proposition
Revised: E.2. Proof of Proposition

Original: To prove proposition 3, we will use the following lemma.
Revised: To prove proposition 3, we will use the following lemma.

Original: [Equation]
Revised: [Equation]

Original: Proof [Proof of Lemma 9.]
Revised: Proof [Proof of Lemma 9.]

Original: We denote ϕ : C <
Revised: We denote ϕ : C <

Original: → R by
Revised: → R by

Original: [Equation]
Revised: [Equation]

Original: let B ( x ) =
Revised: let B ( x ) =

Original: − µβ (cid:80) mi =1 log( − φ i ( x )) .
Revised: − µβ (cid:80) mi =1 log( − φ i ( x )) .

Original: We choose an arbitrary x 0 ∈ C < .
Revised: We choose an arbitrary x 0 ∈ C < .

Original: Then by the convexity of
Revised: Then by the convexity of

Original: B ( x ) we deduce that
Revised: B ( x ) we deduce that

Original: [Equation]
Revised: [Equation]

Original: In the remaining, we prove Proposition 3 which guarantees that (X-CF) has a unique solution.
Revised: In the remaining, we prove Proposition 3 which guarantees that (X-CF) has a unique solution.

Original: Proof [Proof of Proposition 3: uniqueness of the solution of (X-CF).]
Revised: Proof [Proof of Proposition 3: uniqueness of the solution of (X-CF).]

Original: Let ϕ : C <
Revised: Let ϕ : C <

Original: → R denote:
Revised: → R denote:

Original: [Equation]
Revised: [Equation]

Original: We choose x 0 ∈ C < .
Revised: We choose x 0 ∈ C < .

Original: By Lemma 9, ∀ x ∈ C < , ϕ ( x ) → + ∞ ,
Revised: By Lemma 9, ∀ x ∈ C < , ϕ ( x ) → + ∞ ,

Original: ∥ x ∥ 2 → + ∞ .
Revised: ∥ x ∥ 2 → + ∞ .

Original: So there exists M > 0 such that x 0 ∈ B (0 , M ) and ∀ x ∈ S, ϕ ( x ) ≤ ϕ ( x 0 ) , x must belong to
Revised: So there exists M > 0 such that x 0 ∈ B (0 , M ) and ∀ x ∈ S, ϕ ( x ) ≤ ϕ ( x 0 ) , x must belong to

Original: B (0 , M ) ,where
Revised: B (0 , M ) ,where

Original: [Equation]
Revised: [Equation]

Original: It’s clear that there exists t > 0 such that for every x ∈ C < that satisfies ϕ ( x ) ≤ ϕ ( x 0 ) , x must belong to C t ,where
Revised: It’s clear that there exists t > 0 such that for every x ∈ C < that satisfies ϕ ( x ) ≤ ϕ ( x 0 ) , x must belong to C t ,where

Original: [Equation]
Revised: [Equation]

Original: And we can make t small enough so that x 0 ∈ C t .
Revised: And we can make t small enough so that x 0 ∈ C t .

Original: C t is a nonempty compact set and ϕ is continuous, so there exists x ⋆ ∈ C t such that ϕ ( x ⋆ ) ≤ ϕ ( x ) , ∀ x ∈ C < .
Revised: C t is a nonempty compact set and ϕ is continuous, so there exists x ⋆ ∈ C t such that ϕ ( x ⋆ ) ≤ ϕ ( x ) , ∀ x ∈ C < .

Original: Note that ∀ x ∈ C < \C t , ϕ ( x ) ≥ ϕ ( x 0 ) ≥ ϕ
Revised: Note that ∀ x ∈ C < \C t , ϕ ( x ) ≥ ϕ ( x 0 ) ≥ ϕ

Original: ( x ⋆ ) .
Revised: ( x ⋆ ) .

Original: Therefore, x ⋆ is a global minimizer of ϕ. ϕ is strongly-convex thus x k +

Revised: Therefore, x ⋆ is a global minimizer of ϕ. ϕ is strongly-convex thus x k +

Original: =
Revised: =

Original: x
Revised: x

Original: ⋆ is its unique minimizer.
Revised: ⋆ is its unique minimizer.

Original: So x = x k +1 if and only if ∇ ϕ ( x ) = 0 .
Revised: So x = x k +1 if and only if ∇ ϕ ( x ) = 0 .

Original: Therefore, x k +1 is the unique solution of x − y k
Revised: Therefore, x k +1 is the unique solution of x − y k

Original: + 1 β ( F ( w ) + λ k ) − µβ (cid:80) mi =1 ∇ φ
Revised: + 1 β ( F ( w ) + λ k ) − µβ (cid:80) mi =1 ∇ φ

Original: i ( x ) φ i ( x ) = 0 .
Revised: i ( x ) φ i ( x ) = 0 .

Original: Appendix F. Details on the implementation
Revised: Appendix F. Details on the implementation

Original: In this section, we provide the details on the implementation of the experiments shown in the main part in 2D and higher dimension bilinear game, see § F.1 and § F.2, respectively.
Revised: In this section, we provide the details on the implementation of the experiments shown in the main part in 2D and higher dimension bilinear game, see § F.1 and § F.2, respectively.

Original: We also provide here in § F.3 the details of the MNIST experiments presented later in App. G.
Revised: We also provide here in § F.3 the details of the MNIST experiments presented later in App. G.

Original: In addition, we provide the source code through the following link: https://anonymous.4open.science/r/ACVI .
Revised: In addition, we provide the source code through the following link: https://anonymous.4open.science/r/ACVI .

Original: F.1. Experiments in 2D
Revised: F.1. Experiments in 2D

Original: We first state the considered problem fully, then describe the setup what includes the hyperparameters.
Revised: We first state the considered problem fully, then describe the setup what includes the hyperparameters.

Original: Problems.
Revised: Problems.

Original: We consider the following constrained bilinear game (for the same experiment shown in
Revised: We consider the following constrained bilinear game (for the same experiment shown in

Original: Fig. 1 and 5):
Revised: Fig. 1 and 5):

Original: [Equation]
Revised: [Equation]

Original: The Von Neumann’s ratio game [81] (results in Fig.
Revised: The Von Neumann’s ratio game [81] (results in Fig.

Original: 2( a )subfigure) is as follows:
Revised: 2( a )subfigure) is as follows:

Original: [Equation]
Revised: [Equation]

Original: where ∆ 2 = (cid:8) z ∈ R 2 | z ≥ 0 , e ⊺ z = 1 (cid:9) , R = (cid:18) − 0 .
Revised: where ∆ 2 = (cid:8) z ∈ R 2 | z ≥ 0 , e ⊺ z = 1 (cid:9) , R = (cid:18) − 0 .

Original: 6 − 0 .0 .− 0 .(cid:19) and S = (cid:18) 0 .
Revised: 6 − 0 .0 .− 0 .(cid:19) and S = (cid:18) 0 .

Original: 8 0 .(cid:19) .
Revised: 8 0 .(cid:19) .

Original: The so called Forsaken [43] game—used in Fig.
Revised: The so called Forsaken [43] game—used in Fig.

Original: 2( b )subfigure and in App.
Revised: 2( b )subfigure and in App.

Original: G—is as follows:
Revised: G—is as follows:

Original: [Equation]
Revised: [Equation]

Original: (Forsaken) where h ( z ) =
Revised: (Forsaken) where h ( z ) =

Original: z 2 − 12 z 4 + 16 z 6 .
Revised: z 2 − 12 z 4 + 16 z 6 .

Original: The original version is unconstrained X ≡ R 2 .
Revised: The original version is unconstrained X ≡ R 2 .

Original: In Fig.
Revised: In Fig.

Original: 2( b )subfigure we use the constraint x 21 + x 22 ≤ 4 , and in App.
Revised: 2( b )subfigure we use the constraint x 21 + x 22 ≤ 4 , and in App.

Original: G we use two other constraints: x 1 ≥ 0 .
Revised: G we use two other constraints: x 1 ≥ 0 .

Original: 08 and x 2 ≥ 0 .
Revised: 08 and x 2 ≥ 0 .

Original: For the toy GAN experiments shown in Fig.
Revised: For the toy GAN experiments shown in Fig.

Original: 2( c )subfigure, the problem is as follows:
Revised: 2( c )subfigure, the problem is as follows:

Original: [Equation]
Revised: [Equation]

Original: In the experiment, we look at a finite sum (sample average) approximation, which we then solve deterministically in a full batch fashion.
Revised: In the experiment, we look at a finite sum (sample average) approximation, which we then solve deterministically in a full batch fashion.

Original: Setup.
Revised: Setup.

Original: For all the 2D problems, we set the step size of GDA, EG and OGDA to 0 .
Revised: For all the 2D problems, we set the step size of GDA, EG and OGDA to 0 .

Original: 1 , we use k =and α = 0 .
Revised: 1 , we use k =and α = 0 .

Original: 5 for LA-GDA, we set β = 0 .
Revised: 5 for LA-GDA, we set β = 0 .

Original: 08 , µ − 1 = 10 − 5 , δ = 0 .
Revised: 08 , µ − 1 = 10 − 5 , δ = 0 .

Original: 5 and λ 0 = 0 for ACVI; and run for 50 iterations.
Revised: 5 and λ 0 = 0 for ACVI; and run for 50 iterations.

Original: For ACVI, we set the number of outer loop T = 20 .
Revised: For ACVI, we set the number of outer loop T = 20 .

Original: In the first 19 outer loop iterations, we only run one inner loop iteration, and in the last outer loop iteration, we run 30 inner loop iterations (for total of 50 updates).
Revised: In the first 19 outer loop iterations, we only run one inner loop iteration, and in the last outer loop iteration, we run 30 inner loop iterations (for total of 50 updates).

Original: In Fig.
Revised: In Fig.

Original: 1 and Fig.
Revised: 1 and Fig.

Original: 2( c )subfigure, we set the starting point for all algorithms.
Revised: 2( c )subfigure, we set the starting point for all algorithms.

Original: In Fig.
Revised: In Fig.

Original: 2( a )subfigure and 2( b )subfigure, we set the starting point to be (0 . 5 , 0 . 5) ⊺ for all algorithms.
Revised: 2( a )subfigure and 2( b )subfigure, we set the starting point to be (0 . 5 , 0 . 5) ⊺ for all algorithms.

Original: F.2. High-dimension Bilinear Game
Revised: F.2. High-dimension Bilinear Game

Original: We also consider the following higher-dimension bilinear game on the probability simplex, with η ∈ (0 , 1) , n = 1000 :
Revised: We also consider the following higher-dimension bilinear game on the probability simplex, with η ∈ (0 , 1) , n = 1000 :

Original: [Equation]
Revised: [Equation]

Original: We set the step size of GDA, EG and OGDA to 0 .
Revised: We set the step size of GDA, EG and OGDA to 0 .

Original: 1 , use k = 4 and α = 0 .
Revised: 1 , use k = 4 and α = 0 .

Original: 5 for LA-GDA.
Revised: 5 for LA-GDA.

Original: For
Revised: For

Original: ACVI, we set β = 0 .
Revised: ACVI, we set β = 0 .

Original: 5 , µ − 1 = 10 − 6 , δ = 0 .
Revised: 5 , µ − 1 = 10 − 6 , δ = 0 .

Original: 5 and λ 0 = 0 for ACVI.
Revised: 5 and λ 0 = 0 for ACVI.

Original: The solution of (HBG) is x ⋆ = 1500 e , where e ∈ R 1000 .
Revised: The solution of (HBG) is x ⋆ = 1500 e , where e ∈ R 1000 .

Original: As a metric of the experiments on this problem we use the relative error: ε r ( x k ) =
Revised: As a metric of the experiments on this problem we use the relative error: ε r ( x k ) =

Original: ∥ x k − x ⋆ ∥ ∥ x ⋆ ∥ .
Revised: ∥ x k − x ⋆ ∥ ∥ x ⋆ ∥ .

Original: In Fig.3( b )subfigure, we set ε = 0 .
Revised: In Fig.3( b )subfigure, we set ε = 0 .

Original: 02 and compute number of iterations of ACVI needed to reach the relative error given different rotation “strength” 1 − η , η ∈ (0 , 1) .
Revised: 02 and compute number of iterations of ACVI needed to reach the relative error given different rotation “strength” 1 − η , η ∈ (0 , 1) .

Original: Here we set the maximum numbers of iterations to be 50 for all algorithms.
Revised: Here we set the maximum numbers of iterations to be 50 for all algorithms.

Original: In Fig.
Revised: In Fig.

Original: 3( a )subfigure, we set η = 0 . 05 and compute CPU times needed for ACVI, EG, OGDA and
Revised: 3( a )subfigure, we set η = 0 . 05 and compute CPU times needed for ACVI, EG, OGDA and

Original: LA4-GDA to reach different relative errors.
Revised: LA4-GDA to reach different relative errors.

Original: Here we set the maximum run time to be 1500 seconds for all algorithms.
Revised: Here we set the maximum run time to be 1500 seconds for all algorithms.

Original: In Fig.
Revised: In Fig.

Original: 7 in App.
Revised: 7 in App.

Original: G on the other hand, we fix η = 0 .
Revised: G on the other hand, we fix η = 0 .

Original: 05 , and for varying CPU time limits, we compute the relative error of the last iterates of ACVI, GDA, EG, OGDA and LA4-GDA.
Revised: 05 , and for varying CPU time limits, we compute the relative error of the last iterates of ACVI, GDA, EG, OGDA and LA4-GDA.

Original: More general HD-BG game (g-HBG).
Revised: More general HD-BG game (g-HBG).

Original: Since (HBG) has perfect conditioning (that is, the interactive term x ⊺ 1 Bx 2 , is with B ≡ I ), in App.
Revised: Since (HBG) has perfect conditioning (that is, the interactive term x ⊺ 1 Bx 2 , is with B ≡ I ), in App.

Original: G.2 we present results on the following more general high dimensional bilinear game:
Revised: G.2 we present results on the following more general high dimensional bilinear game:

Original: [Equation]
Revised: [Equation]

Original: Where A , B and C are randomly generated 500 × 500 matrices, and A , C are positive semi-definite.
Revised: Where A , B and C are randomly generated 500 × 500 matrices, and A , C are positive semi-definite.

Original: The solution of (g-HBG) is x ⋆ = 0 , where 0 ∈ R 1000 .
Revised: The solution of (g-HBG) is x ⋆ = 0 , where 0 ∈ R 1000 .

Original: As a metric of the experiments on this problem we use the error ε ( x k ) =
Revised: As a metric of the experiments on this problem we use the error ε ( x k ) =

Original: ∥ x k ∥ .
Revised: ∥ x k ∥ .

Original: The remaining settings are identical to those of (HBG), explained above.
Revised: The remaining settings are identical to those of (HBG), explained above.

Original: F.3. MNIST and Fashion-MNIST experiments
Revised: F.3. MNIST and Fashion-MNIST experiments

Original: For the experiments on the MNIST dataset 1 , we use the source code of Chavdarova et al.
Revised: For the experiments on the MNIST dataset 1 , we use the source code of Chavdarova et al.

Original: [14] for the baselines
Revised: [14] for the baselines

Original: and we build on it to implement ACVI.
Revised: and we build on it to implement ACVI.

Original: For completeness, we provide an overview of the implementation.
Revised: For completeness, we provide an overview of the implementation.

Original: Models.
Revised: Models.

Original: We used the DCGAN architectures [67], listed in Table 1, and the parameters of the models are initialized using PyTorch default initialization.
Revised: We used the DCGAN architectures [67], listed in Table 1, and the parameters of the models are initialized using PyTorch default initialization.

Original: For experiments on this dataset, we used the non-saturating GAN loss as proposed in [37]:
Revised: For experiments on this dataset, we used the non-saturating GAN loss as proposed in [37]:

Original: [Equation]
Revised: [Equation]

Original: where G ( · ) , D ( · ) denote the generator and discriminator, resp., and p d and p z denote the data and the latent distributions (the latter predefined as normal distribution).
Revised: where G ( · ) , D ( · ) denote the generator and discriminator, resp., and p d and p z denote the data and the latent distributions (the latter predefined as normal distribution).

Original: Generator
Revised: Generator

Original: Input: z ∈ R 128 ∼ N (0 , I ) transposed conv.
Revised: Input: z ∈ R 128 ∼ N (0 , I ) transposed conv.

Original: (ker: 3 × 3 , 128 → 512 ; stride: 1 )
Revised: (ker: 3 × 3 , 128 → 512 ; stride: 1 )

Original: Batch Normalization
Revised: Batch Normalization

Original: ReLU transposed conv.
Revised: ReLU transposed conv.

Original: (ker: 4 × 4 , 512 → 256 , stride: 2 )
Revised: (ker: 4 × 4 , 512 → 256 , stride: 2 )

Original: Batch Normalization
Revised: Batch Normalization

Original: ReLU transposed conv.
Revised: ReLU transposed conv.

Original: (ker: 4 × 4 , 256 → 128 , stride: 2 )
Revised: (ker: 4 × 4 , 256 → 128 , stride: 2 )

Original: Batch Normalization
Revised: Batch Normalization

Original: ReLU transposed conv.
Revised: ReLU transposed conv.

Original: (ker: 4 × 4 , 128 → 1 , stride: 2 , pad: 1)
Revised: (ker: 4 × 4 , 128 → 1 , stride: 2 , pad: 1)

Original: Tanh ( · )
Revised: Tanh ( · )

Original: Discriminator
Revised: Discriminator

Original: Input: x ∈ R 1 × 28 ×conv.
Revised: Input: x ∈ R 1 × 28 ×conv.

Original: (ker: 4 × 4 , 1 → 64 ; stride: 2 ; pad:1)
Revised: (ker: 4 × 4 , 1 → 64 ; stride: 2 ; pad:1)

Original: LeakyReLU (negative slope: 0 . 2 ) conv.
Revised: LeakyReLU (negative slope: 0 . 2 ) conv.

Original: (ker: 4 × 4 , 64 → 128 ; stride: 2 ; pad:1)
Revised: (ker: 4 × 4 , 64 → 128 ; stride: 2 ; pad:1)

Original: Batch Normalization
Revised: Batch Normalization

Original: LeakyReLU (negative slope: 0 . 2 ) conv.
Revised: LeakyReLU (negative slope: 0 . 2 ) conv.

Original: (ker: 4 × 4 , 128 → 256 ; stride: 2 ; pad:1)
Revised: (ker: 4 × 4 , 128 → 256 ; stride: 2 ; pad:1)

Original: Batch Normalization
Revised: Batch Normalization

Original: LeakyReLU (negative slope: 0 . 2 ) conv.
Revised: LeakyReLU (negative slope: 0 . 2 ) conv.

Original: (ker: 3 × 3 , 256 → 1 ; stride: 1 )
Revised: (ker: 3 × 3 , 256 → 1 ; stride: 1 )

Original: [Figure]
Revised: [Figure]

Original: Details on the ACVI implementation.
Revised: Details on the ACVI implementation.

Original: When implementing ACVI on MNIST, we “remove” the outer loop of Algorithm 1 (that is we set T = 1 ), and fix µ to be a small number, in particular, we select µ = 10 − 9 .
Revised: When implementing ACVI on MNIST, we “remove” the outer loop of Algorithm 1 (that is we set T = 1 ), and fix µ to be a small number, in particular, we select µ = 10 − 9 .

Original: We randomly initialize x and y and initialize λ to zero.
Revised: We randomly initialize x and y and initialize λ to zero.

Original: For line 8 and 9 of
Revised: For line 8 and 9 of

Original: Algorithm 1, we run l steps of (unconstrained) EG and GDA, respectively.
Revised: Algorithm 1, we run l steps of (unconstrained) EG and GDA, respectively.

Original: For the update of x (using EG), we use step-size η x = 0 .
Revised: For the update of x (using EG), we use step-size η x = 0 .

Original: 001 , whereas for y (using GDA), we use step-size η y = 0 . 2 .
Revised: 001 , whereas for y (using GDA), we use step-size η y = 0 . 2 .

Original: We present results when l ∈ { 1 , 10 } .
Revised: We present results when l ∈ { 1 , 10 } .

Original: At every iteration, we update λ using the expression in lineof Algorithm 1, with β = 0 . 5 .
Revised: At every iteration, we update λ using the expression in lineof Algorithm 1, with β = 0 . 5 .

Original: Because the problem in step 7 of Algorithm 1 does not change a lot over the iterations (as well as when computing y ), when we implement Algorithm 1 we do not reinitialize the variable x .
Revised: Because the problem in step 7 of Algorithm 1 does not change a lot over the iterations (as well as when computing y ), when we implement Algorithm 1 we do not reinitialize the variable x .

Original: We instead use the one from the previous iteration as an initialization and update it l times.
Revised: We instead use the one from the previous iteration as an initialization and update it l times.

Original: The full details of the training when using an inner optimizer for step 7 of Algorithm 1 are provided in
Revised: The full details of the training when using an inner optimizer for step 7 of Algorithm 1 are provided in

Original: Algorithm 3, where we recall that G ( x ) is defined as:
Revised: Algorithm 3, where we recall that G ( x ) is defined as:

Original: [Equation]
Revised: [Equation]

Original: Note that in the case of MNIST we consider only inequality constraints (and there are no equality constraints), therefore the matrices P c and d c are identity and zero, respectively.
Revised: Note that in the case of MNIST we consider only inequality constraints (and there are no equality constraints), therefore the matrices P c and d c are identity and zero, respectively.

Original: Therefore, there is no need for lines 3 and 4 in Algorithm 3.
Revised: Therefore, there is no need for lines 3 and 4 in Algorithm 3.

Original: 1: Input: operator F : X → R n , equality Cx = d and inequality constraints φ i ( x ) ≤ 0 , i = [ m ] , hyperparameters µ, β > 0 , δ ∈ (0 , 1) , inner optimizer A (e.g. EG, GDA, OGDA), l number of steps for the inner-optimizer, number of iterations K .
Revised: 1: Input: operator F : X → R n , equality Cx = d and inequality constraints φ i ( x ) ≤ 0 , i = [ m ] , hyperparameters µ, β > 0 , δ ∈ (0 , 1) , inner optimizer A (e.g. EG, GDA, OGDA), l number of steps for the inner-optimizer, number of iterations K .

Original: 2: Initialize: x 0 ∈ R n , y 0 ∈ R n , λ 0 ∈ R n 3: P c ≜ I − C ⊺ ( CC ⊺ ) − 1 C where P c ∈ R n × n 4: d c ≜ C ⊺ ( CC ⊺ ) − 1 d where d c ∈ R n 5: for k = 0 , . . .
Revised: 2: Initialize: x 0 ∈ R n , y 0 ∈ R n , λ 0 ∈ R n 3: P c ≜ I − C ⊺ ( CC ⊺ ) − 1 C where P c ∈ R n × n 4: d c ≜ C ⊺ ( CC ⊺ ) − 1 d where d c ∈ R n 5: for k = 0 , . . .

Original: , K − 1 do 6:
Revised: , K − 1 do 6:

Original: To obtain x k +1 : run l steps of A solving VI ( R n , G ) , where G is defined in (G-EQ) 7:
Revised: To obtain x k +1 : run l steps of A solving VI ( R n , G ) , where G is defined in (G-EQ) 7:

Original: To obtain y k +1 : run l steps of GD to find y k +1 that minimizes:
Revised: To obtain y k +1 : run l steps of GD to find y k +1 that minimizes:

Original: [Equation]
Revised: [Equation]

Original: 8: λ
Revised: 8: λ

Original: [Equation]
Revised: [Equation]

Original: y
Revised: y

Original: [Equation]
Revised: [Equation]

Original: +
Revised: +

Original: [Equation]
Revised: [Equation]

Original: ) 9: end for 10: Return: x K
Revised: ) 9: end for 10: Return: x K

Original: The implementation details for the Fashion-MNIST experiment are identical to those on the
Revised: The implementation details for the Fashion-MNIST experiment are identical to those on the

Original: MNIST experiment.
Revised: MNIST experiment.

Original: the Generator are where θ ∈ R n is the vector of all parameters of the Generator, A
Revised: the Generator are where θ ∈ R n is the vector of all parameters of the Generator, A

Original: = ( a ⊺ 1 , · · · , a ⊺ 100 ) ⊺ ∈ R 100 × n , b = ( b 1 , . . .
Revised: = ( a ⊺ 1 , · · · , a ⊺ 100 ) ⊺ ∈ R 100 × n , b = ( b 1 , . . .

Original: , b 100 ) ∈ R 100 .
Revised: , b 100 ) ∈ R 100 .

Original: We use the greedy projection algorithm described in [7].
Revised: We use the greedy projection algorithm described in [7].

Original: A greedy projection algorithm is essentially a projected gradient method, it is easy to implement in highdimension problems, and it has a convergence rate of O (1 / √ K ) .
Revised: A greedy projection algorithm is essentially a projected gradient method, it is easy to implement in highdimension problems, and it has a convergence rate of O (1 / √ K ) .

Original: See Chapter 8.2.3 in [7] for more details.
Revised: See Chapter 8.2.3 in [7] for more details.

Original: Since the dimension n is very large, at each step of the projection, one could only project θ to one hyperplane a ⊺ i x = b i for some
Revised: Since the dimension n is very large, at each step of the projection, one could only project θ to one hyperplane a ⊺ i x = b i for some

Original: i ∈
Revised: i ∈

Original: I
Revised: I

Original: ( θ ) , where
Revised: ( θ ) , where

Original: [Equation]
Revised: [Equation]

Original: For every j ∈ { 1 , 2 , . . .
Revised: For every j ∈ { 1 , 2 , . . .

Original: , 100 } , let
Revised: , 100 } , let

Original: [Equation]
Revised: [Equation]

Original: The greedy projection method chooses i so that i ∈ arg max { dist ( θ , S i ) } .
Revised: The greedy projection method chooses i so that i ∈ arg max { dist ( θ , S i ) } .

Original: Note that as long as θ is not in the constraint set C ≤ = { x | Ax ≤ b } , i would be in I ( θ ) .
Revised: Note that as long as θ is not in the constraint set C ≤ = { x | Ax ≤ b } , i would be in I ( θ ) .

Original: Algorithm 4 gives the details of the greedy projection method we use for the baseline, written for the Generator only for simplicity; the same projection method is used for the Discriminator as well.
Revised: Algorithm 4 gives the details of the greedy projection method we use for the baseline, written for the Generator only for simplicity; the same projection method is used for the Discriminator as well.

Original: Setup 2 : MNIST & Fashion-MNIST.
Revised: Setup 2 : MNIST & Fashion-MNIST.

Original: We add two constraints for the MNIST experiment: we set the squared sum of all parameters of the Generator and that of the Discriminator (separately) to be less than or equal to a hyperparameter M .
Revised: We add two constraints for the MNIST experiment: we set the squared sum of all parameters of the Generator and that of the Discriminator (separately) to be less than or equal to a hyperparameter M .

Original: We select some large number for M , in particular, we set
Revised: We select some large number for M , in particular, we set

Original: M = 50 .
Revised: M = 50 .

Original: Metrics.
Revised: Metrics.

Original: We describe the metrics for the MNIST experiments shown later in App.
Revised: We describe the metrics for the MNIST experiments shown later in App.

Original: G. We use the two standard GAN metrics, Inception Score (IS, 73) and Fr´echet Inception Distance (FID, 42).
Revised: G. We use the two standard GAN metrics, Inception Score (IS, 73) and Fr´echet Inception Distance (FID, 42).

Original: Both
Revised: Both

Original: FID and IS rely on a pre-trained classifier, and take finite set of ˜ m samples from the generator to compute these.
Revised: FID and IS rely on a pre-trained classifier, and take finite set of ˜ m samples from the generator to compute these.

Original: Since MNIST has greyscale images, we used a classifier trained on this dataset and used ˜ m
Revised: Since MNIST has greyscale images, we used a classifier trained on this dataset and used ˜ m

Original: = 5000 .
Revised: = 5000 .

Original: 1: Input: θ ∈ R n , A
Revised: 1: Input: θ ∈ R n , A

Original: = ( a ⊺ 1 , . .
Revised: = ( a ⊺ 1 , . .

Original: . , a ⊺ 100 ) ⊺ ∈ R 100 × n , b = ( b 1 , . . .
Revised: . , a ⊺ 100 ) ⊺ ∈ R 100 × n , b = ( b 1 , . . .

Original: , b 100 ) ∈ R 100 , ε >2: while True do 3:
Revised: , b 100 ) ∈ R 100 , ε >2: while True do 3:

Original: I ( θ ) ≜ { j | a ⊺ j θ >
Revised: I ( θ ) ≜ { j | a ⊺ j θ >

Original: b j } 4: if I ( θ ) = ∅ or max j ∈I ( θ ) | a ⊺ j θ − b
Revised: b j } 4: if I ( θ ) = ∅ or max j ∈I ( θ ) | a ⊺ j θ − b

Original: j | ∥ a j ∥ < ε then 5: break 6: end if 7: choose i ∈ arg max j ∈I ( θ ) | a ⊺ j θ − b
Revised: j | ∥ a j ∥ < ε then 5: break 6: end if 7: choose i ∈ arg max j ∈I ( θ ) | a ⊺ j θ − b

Original: j
Revised: j

Original: | ∥ a j ∥ 8: θ ← θ − | a ⊺ i θ − b i | ∥ a i ∥ 2 a i 9: end while 10: Return: θ
Revised: | ∥ a j ∥ 8: θ ← θ − | a ⊺ i θ − b i | ∥ a i ∥ 2 a i 9: end while 10: Return: θ

Original: Metrics: IS.
Revised: Metrics: IS.

Original: Given a sample from the generator ˜ x g ∼ p g —where p g denotes the data distribution of the generator—IS uses the softmax output of the pre-trained network p (˜ y | ˜ x g ) which represents the probability that ˜ x g is of class c
Revised: Given a sample from the generator ˜ x g ∼ p g —where p g denotes the data distribution of the generator—IS uses the softmax output of the pre-trained network p (˜ y | ˜ x g ) which represents the probability that ˜ x g is of class c

Original: i , i ∈ 1 . . .
Revised: i , i ∈ 1 . . .

Original: C , i.e., p (˜ y | ˜ x g ) ∈
Revised: C , i.e., p (˜ y | ˜ x g ) ∈

Original: [0 , 1] C .
Revised: [0 , 1] C .

Original: It then computes the marginal class distribution p (˜ y )
Revised: It then computes the marginal class distribution p (˜ y )

Original: = (cid:82) ˜ x p (˜ y |
Revised: = (cid:82) ˜ x p (˜ y |

Original: ˜ x g ) p g
Revised: ˜ x g ) p g

Original: (˜ x g ) .
Revised: (˜ x g ) .

Original: IS measures the Kullback–Leibler divergence D KL between the predicted conditional label distribution p (˜ y | ˜ x g ) and the marginal class distribution p (˜ y ) .
Revised: IS measures the Kullback–Leibler divergence D KL between the predicted conditional label distribution p (˜ y | ˜ x g ) and the marginal class distribution p (˜ y ) .

Original: More precisely, it is computed as follows:
Revised: More precisely, it is computed as follows:

Original: [Equation]
Revised: [Equation]

Original: IS
Revised: IS

Original: [Equation]
Revised: [Equation]

Original: It aims at estimating (i) if the samples look realistic i.e., p (˜ y | ˜ x g ) should have low entropy, and (ii) if the samples are diverse (from different ImageNet classes) i.e., p (˜ y ) should have high entropy.
Revised: It aims at estimating (i) if the samples look realistic i.e., p (˜ y | ˜ x g ) should have low entropy, and (ii) if the samples are diverse (from different ImageNet classes) i.e., p (˜ y ) should have high entropy.

Original: As these are combined using the Kullback–Leibler divergence, the higher the score is, the better the performance.
Revised: As these are combined using the Kullback–Leibler divergence, the higher the score is, the better the performance.

Original: Metrics: FID.
Revised: Metrics: FID.

Original: Contrary to IS, FID compares the synthetic samples ˜ x g ∼ p g with those of the training dataset ˜ x
Revised: Contrary to IS, FID compares the synthetic samples ˜ x g ∼ p g with those of the training dataset ˜ x

Original: d ∼ p d in a feature space.
Revised: d ∼ p d in a feature space.

Original: The samples are embedded using the first several layers of a pretrained classifier.
Revised: The samples are embedded using the first several layers of a pretrained classifier.

Original: It assumes p g and p d are multivariate normal distributions, and estimates the means m g and m d and covariances C g and C d , respectively for p g and p d in that feature space.
Revised: It assumes p g and p d are multivariate normal distributions, and estimates the means m g and m d and covariances C g and C d , respectively for p g and p d in that feature space.

Original: Finally, FID is computed as:
Revised: Finally, FID is computed as:

Original: [Equation]
Revised: [Equation]

Original: where D 2 denotes the Fr´echet Distance.
Revised: where D 2 denotes the Fr´echet Distance.

Original: Note that as this metric is a distance, the lower it is, the better the performance.
Revised: Note that as this metric is a distance, the lower it is, the better the performance.

Original: Hardware.
Revised: Hardware.

Original: We used the Colab platform ( https://colab.research.google.com/ ) and
Revised: We used the Colab platform ( https://colab.research.google.com/ ) and

Original: Tesla P100 GPUs.
Revised: Tesla P100 GPUs.

Original: The running times are reported in App.
Revised: The running times are reported in App.

Original: G.
Revised: G.

Original: Appendix G. Additional Empirical Analysis
Revised: Appendix G. Additional Empirical Analysis

Original: In this section, we provide some omitted plots/analyses of the results in the main paper as well as additional experiments.
Revised: In this section, we provide some omitted plots/analyses of the results in the main paper as well as additional experiments.

Original: In particular, (i) App.
Revised: In particular, (i) App.

Original: G.1 lists results in 2D, (ii) App.
Revised: G.1 lists results in 2D, (ii) App.

Original: G.2 on (HBG) and (g-HBG), whereas (iii) App.
Revised: G.2 on (HBG) and (g-HBG), whereas (iii) App.

Original: G.3 provides more detailed plots of those experiments summarized in Fig. 4, and presents additional results on other constraints on MNIST where we compare computationally-wise with unconstained baselines.
Revised: G.3 provides more detailed plots of those experiments summarized in Fig. 4, and presents additional results on other constraints on MNIST where we compare computationally-wise with unconstained baselines.

Original: G.1. Additional experiments in 2D, on HBG and on g-HBG
Revised: G.1. Additional experiments in 2D, on HBG and on g-HBG

Original: Depicting the omitted baselines of Fig.
Revised: Depicting the omitted baselines of Fig.

Original: While Fig.
Revised: While Fig.

Original: 1 lists solely EG and ACVI for clarity, Fig.depicts all the considered baselines in this paper on the cBG problem for completeness.
Revised: 1 lists solely EG and ACVI for clarity, Fig.depicts all the considered baselines in this paper on the cBG problem for completeness.

Original: [Figure]
Revised: [Figure]

Original: Additional experiments: varying constraints on the Forsaken problem.
Revised: Additional experiments: varying constraints on the Forsaken problem.

Original: The Forsaken game was first pointed out in [43] and is particularly relevant because it has limit cycles , despite that it is in 2D. Since we are missing a tool to detect if we are in a limit cycle when in higher dimensions, this example is a popular benchmark in many recent works.
Revised: The Forsaken game was first pointed out in [43] and is particularly relevant because it has limit cycles , despite that it is in 2D. Since we are missing a tool to detect if we are in a limit cycle when in higher dimensions, this example is a popular benchmark in many recent works.

Original: Interestingly, in Fig.
Revised: Interestingly, in Fig.

Original: 2( b )subfigure we observe that ACVI is the only method that escapes the limit cycle.
Revised: 2( b )subfigure we observe that ACVI is the only method that escapes the limit cycle.

Original: However, since in those simulations, given the initial point the constraints are not active throughout the training, in this section we run experiments with additional constraints.
Revised: However, since in those simulations, given the initial point the constraints are not active throughout the training, in this section we run experiments with additional constraints.

Original: Fig.
Revised: Fig.

Original: 6 depicts the baseline methods and
Revised: 6 depicts the baseline methods and

Original: ACVI on the Forsaken problem with two different constraints than that considered in Fig.
Revised: ACVI on the Forsaken problem with two different constraints than that considered in Fig.

Original: 2 (that x 21 + x 22 ≤ 4 ).
Revised: 2 (that x 21 + x 22 ≤ 4 ).

Original: Since this game is non-monotone, we observe that for some constraints the baseline methods—GDA, EG, OGDA, LA4-GDA—stay near the constraint (and do not converge).
Revised: Since this game is non-monotone, we observe that for some constraints the baseline methods—GDA, EG, OGDA, LA4-GDA—stay near the constraint (and do not converge).

Original: This may indicate that ACVI may have better chances of converging for broader problem classes than monotone VIs, relative to baseline methods whose convergence may depend on the constraints, and when hitting a constraint may be significantly slower (as Fig.
Revised: This may indicate that ACVI may have better chances of converging for broader problem classes than monotone VIs, relative to baseline methods whose convergence may depend on the constraints, and when hitting a constraint may be significantly slower (as Fig.

Original: 1 illustrates).
Revised: 1 illustrates).

Original: [Figure]
Revised: [Figure]

Original: − 0 .0 .0 .1 .1 .
Revised: − 0 .0 .0 .1 .1 .

Original: ACVI
Revised: ACVI

Original: GDA
Revised: GDA

Original: EG
Revised: EG

Original: OGDA
Revised: OGDA

Original: LA4-GDA
Revised: LA4-GDA

Original: [Figure]
Revised: [Figure]

Original: G.2. Additional experiments HBG and on g-HBG
Revised: G.2. Additional experiments HBG and on g-HBG

Original: Complementary analysis to those in Fig.
Revised: Complementary analysis to those in Fig.

Original: Similar to Fig.
Revised: Similar to Fig.

Original: 3, in Fig.
Revised: 3, in Fig.

Original: 7 we run experiments on the HBG problem.
Revised: 7 we run experiments on the HBG problem.

Original: However, here for a given fixed CPU time, we depict the relative error of the considered baselines and ACVI.
Revised: However, here for a given fixed CPU time, we depict the relative error of the considered baselines and ACVI.

Original: [Figure]
Revised: [Figure]

Original: GDA
Revised: GDA

Original: EG
Revised: EG

Original: [Figure]
Revised: [Figure]

Original: LA
Revised: LA

Original: [Figure]
Revised: [Figure]

Original: Additional experiments on (g-HBG) .
Revised: Additional experiments on (g-HBG) .

Original: In Fig.
Revised: In Fig.

Original: 8 we run experiments on the generalized HBG problem (g-HBG).
Revised: 8 we run experiments on the generalized HBG problem (g-HBG).

Original: In figure 8( a )subfigure, we compute number of iterations needed to reach ε - distance to solution for varying intensity of the rotational component (1 − η ) ; in figure 8( b )subfigure, we compute the error of the last iterate given fixed CPU time.
Revised: In figure 8( a )subfigure, we compute number of iterations needed to reach ε - distance to solution for varying intensity of the rotational component (1 − η ) ; in figure 8( b )subfigure, we compute the error of the last iterate given fixed CPU time.

Original: We observe that despite the highly rotational monotone vector field, ACVI converges significantly faster in terms of wall clock time in higher dimensions as well.
Revised: We observe that despite the highly rotational monotone vector field, ACVI converges significantly faster in terms of wall clock time in higher dimensions as well.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: η
Revised: η

Original: [Figure]
Revised: [Figure]

Original: n u m b e r o f i t e r a t i o n s
Revised: n u m b e r o f i t e r a t i o n s

Original: [Figure]
Revised: [Figure]

Original: G.3. Experiments on MNIST and Fashion-MNIST G.3.1. S ETUP 1: EXPERIMENTS ON MNIST WITH LINEAR INEQUALITIES
Revised: G.3. Experiments on MNIST and Fashion-MNIST G.3.1. S ETUP 1: EXPERIMENTS ON MNIST WITH LINEAR INEQUALITIES

Original: In this section, we present more detailed results of the summarizing plot in Fig.
Revised: In this section, we present more detailed results of the summarizing plot in Fig.

Original: 4 of the main paper.
Revised: 4 of the main paper.

Original: For this experiment, we used linear inequalities as described in § F.3.
Revised: For this experiment, we used linear inequalities as described in § F.3.

Original: Unlike in subsection G.3.2, here all the baselines are projected methods (that is, the same problem setting applies to ACVI and the baselines).
Revised: Unlike in subsection G.3.2, here all the baselines are projected methods (that is, the same problem setting applies to ACVI and the baselines).

Original: Fig.
Revised: Fig.

Original: 9 and 10 depict the comparisons with projected GDA and projected EG, respectively.
Revised: 9 and 10 depict the comparisons with projected GDA and projected EG, respectively.

Original: We observe that ACVI converges fast relative to the corresponding baseline.
Revised: We observe that ACVI converges fast relative to the corresponding baseline.

Original: When choosing a larger number of steps for the inner problem l = 10 (see Algorithm 3) the wall-clock time per iteration increases, and interestingly the ACVI steps compensate for that and overall converge as fast as when l = 1 .
Revised: When choosing a larger number of steps for the inner problem l = 10 (see Algorithm 3) the wall-clock time per iteration increases, and interestingly the ACVI steps compensate for that and overall converge as fast as when l = 1 .

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: G.3.2. S ETUP 2: EXPERIMENTS ON (F ASHION -)MNIST WITH QUADRATIC INEQUALITIES
Revised: G.3.2. S ETUP 2: EXPERIMENTS ON (F ASHION -)MNIST WITH QUADRATIC INEQUALITIES

Original: In this section
Revised: In this section

Original: [Figure]
Revised: [Figure]

Original: we consider the MNIST and Fashion-MNIST datasets, which are unconstrained problems so as to make use of the well-established performance metrics (which are otherwise unclear in the non-monotone settings, where we do not know the optimal solution apriori).
Revised: we consider the MNIST and Fashion-MNIST datasets, which are unconstrained problems so as to make use of the well-established performance metrics (which are otherwise unclear in the non-monotone settings, where we do not know the optimal solution apriori).

Original: We augment the problem with a mild constraint which requires that the norm of the per-player parameters does not exceed a certain value (see App.
Revised: We augment the problem with a mild constraint which requires that the norm of the per-player parameters does not exceed a certain value (see App.

Original: F.3).
Revised: F.3).

Original: We compare ACVI with unconstrained baselines, what sets
Revised: We compare ACVI with unconstrained baselines, what sets

Original: ACVI in a disadvantage as the projection requires additional computation.
Revised: ACVI in a disadvantage as the projection requires additional computation.

Original: However, the primary purpose of these experiments is to observe if Algorithm 1 is competitive computationally-wise when lines 8 and 9 are non-trivial and require an (unconstrained) solver.
Revised: However, the primary purpose of these experiments is to observe if Algorithm 1 is competitive computationally-wise when lines 8 and 9 are non-trivial and require an (unconstrained) solver.

Original: However note that since MNIST is a relatively easy problem, it may not answer the natural question if ACVI has advantages on problems augmented with constraints over standard unconstrained methods.
Revised: However note that since MNIST is a relatively easy problem, it may not answer the natural question if ACVI has advantages on problems augmented with constraints over standard unconstrained methods.

Original: We leave such analyses for future work.
Revised: We leave such analyses for future work.

Original: The implementation and the used metrics are described in App.
Revised: The implementation and the used metrics are described in App.

Original: F.3.
Revised: F.3.

Original: Fig.
Revised: Fig.

Original: 11 summarizes the experiments in terms of the obtained FID score over time.
Revised: 11 summarizes the experiments in terms of the obtained FID score over time.

Original: We observe that ACVI (although it uses two solvers at each iteration) is yet performing competitively to unconstrained GDA and EG.
Revised: We observe that ACVI (although it uses two solvers at each iteration) is yet performing competitively to unconstrained GDA and EG.

Original: Figures 12–17 provide in addition samples of the Generator and IS scores, separately for each method.
Revised: Figures 12–17 provide in addition samples of the Generator and IS scores, separately for each method.

Original: Figures 18 and 19 depict samples generated by the different methods, when trained on the Fashion-MNIST dataset.
Revised: Figures 18 and 19 depict samples generated by the different methods, when trained on the Fashion-MNIST dataset.

Original: We believe that further exploring the type of constraints to be added, or the implementation options (e.g., l , step-size) may be proven fruitful even for problems that are originally unconstrained–as such an approach may reduce the rotational component of the original vector field, what in turn causes faster convergence or may help in escaping limit cycles for problems beyond monotone ones.
Revised: We believe that further exploring the type of constraints to be added, or the implementation options (e.g., l , step-size) may be proven fruitful even for problems that are originally unconstrained–as such an approach may reduce the rotational component of the original vector field, what in turn causes faster convergence or may help in escaping limit cycles for problems beyond monotone ones.

Original: [Figure]
Revised: [Figure]

Original: GDA
Revised: GDA

Original: [Figure]
Revised: [Figure]

Original: ACVI
Revised: ACVI

Original: [Figure]
Revised: [Figure]

Original: GDA
Revised: GDA

Original: [Figure]
Revised: [Figure]

Original: Using step size of 0 .
Revised: Using step size of 0 .

Original: The depicted results are over multiple seeds.
Revised: The depicted results are over multiple seeds.

Original: See App.
Revised: See App.

Original: F.3 and G.3 for details on the implementation and discussion, resp.
Revised: F.3 and G.3 for details on the implementation and discussion, resp.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]