Paper: Weaker MVI Condition: Extragradient Methods with Multi-Step Exploration (ICLR-2024)

Reviewer comment (Experiments):
It seems problematic that the max distance algorithm (Algorithm 1) has worse complexity when the operator \(F\) has _more_ structure (e.g. when cocoercive such that the subroutine itself is sufficient for convergence). In other words, it seems that the scheme is trading off complexity in easier problem for a convergence in a larger class. What is the observed oracle complexity if you run the algorithm on a strongly monotone problem or cocoercive (for which the subroutine convergence) in comparison with e.g. EG+? (e.g. take \(f(x,y)=x^2 - y^2 + xy\))

Author response:
We agree on the point that Algorithm 1 has worse complexity if $F$ is more structured. We believe that it depends on how the subroutine converges to the stationary point. For example on Example 1, where the trajectory of ODEs would be a logarithmic spiral, the subroutine will possibly stop near the next point $v$ on the trajectory that makes $Fv$ parallel to $z^k-z^* $. 

- We have conducted several experiments on Example 1, with $a>0, b>0$. The stepsizes for all methods are fixed to $\gamma_k=1/(5L)$. The numerals shown in the sheet denote the operator evaluations the algorithm have taken until $\lVert F\bar{z}^k\rVert<1e-10$. ($\lVert Fz^k\rVert$ for GDA)

- We provide two cases for the selection of $\sigma_k$. In the next sheet, the parameter is set to $\sigma_k = 0$. 

  | a    | b    | GDA       | AdaptiveEG+ | 2-step EG | Algorithm 1 |
  | ---- | ---- | --------- | ----------- | --------- | ----------- |
  | 50   | 1    | divergent | 2321        | 931       | 242         |
  | 10   | 1    | divergent | 1289        | 628       | 137         |
  | 2    | 1    | 328       | 435         | 271       | 162         |
  | 1    | 1    | 174       | 289         | 199       | 211         |
  | 1    | 2    | 129       | 241         | 175       | 251         |
  | 1    | 10   | 118       | 233         | 175       | 338         |
  | 1    | 50   | 125       | 247         | 184       | 309         |

- The following sheet presents the results when $\sigma_k = \rho$.

  | a    | b    | GDA       | AdaptiveEG+ | 2-step EG | Algorithm 1 |
  | ---- | ---- | --------- | ----------- | --------- | ----------- |
  | 50   | 1    | divergent | 2299        | 928       | 252         |
  | 10   | 1    | divergent | 1137        | 598       | 137         |
  | 2    | 1    | 328       | 211         | 178       | 93          |
  | 1    | 1    | 174       | 91          | 91        | 68          |
  | 1    | 2    | 129       | 47          | 55        | 60          |
  | 1    | 10   | 118       | 21          | 28        | 32          |
  | 1    | 50   | 125       | 15          | 19        | 33          |

- It is observed that Algorithm 1 has relatively worse complexity when $\rho$ is close to $1/L$  and has relatively better complexity when $\rho$ is close to 0.

Reviewer comment (Theory):
I would state the minimal \(n\) that achieves the \(\rho_0\) in Thm. 3.4. This way you can also specify the oracle complexity.

Author response:
**On the $n$ that achieves optimal $\rho$** We have included the minimal $n$ in the revised Thm. 4.4 (ii).

Reviewer comment (Novelty):
Lemma 2.4 is very related to [Solodov and Svaiter 1999](https://www.emis.de/journals/JCA/vol.6_no.1/j149.pdf) Lemma 2.1 which might be worth mentioning.

Author response:
**On Lemma 2.4** Lemma 2.4 has been relocated to the appendix. We have included remarks in Appendix A to connect the two lemmas.

Reviewer comment (Theory):
Concerning \(\rho < -1/L\) threshold maybe mention [Bauschke et al. 2019, e.g. Table 1](https://arxiv.org/pdf/1902.09827.pdf) characterization for cohypomonotone problems.

Author response:
**The $\rho<-1/L$ threshold**  We have included a reference to the specified work in Section 4.

Reviewer comment (Theory):
The discussion after Thm. 3.4 is confusing to me:  
- The lower bound could still be tight for time-invariant stepsize (as Thm. 3.4 is concerned with)  
- The numerical evaluation only _suggests_ that improving the lower bound is possible if time varying stepsize is allowed. Stating that a counterexample proofs something in this setting can be misleading (especially in the context of the word "lower bound" being used in a slightly unconventional way).

Author response:
**The discussion after previous Thm. 3.4 (now Thm. 4.4)** 
  - Even when the stepsizes are time-invariant, there are still better results when $\sum_i \gamma_i>1/L$. In the 2-step case, the tight result is $\rho_0 L\approx-0.582565$ when $\gamma_i\approx0.579785/L$. Refer to the contour for a better understanding (now numbered as Fig. 6).
  - We have updated the expression to avoid confusion.

Reviewer comment (Writing):
Maybe use "range of \(\rho\)" instead of lower bound where possible to avoid conflation with negative results (e.g. after Thm. 3.4).

Author response:
**On contribution 1** We have updated our expression to avoid confusion.

Reviewer comment (Presentation):
A lot of whitespace. To compress page 6 you could consider writing the inner loop of (n-step EG) with \(\forall i \in [n]\) in one line.

Author response:
**The inner loop of (n-step EG)** We appreciate your suggestion and have implemented the suggested notation.

Reviewer comment (Writing):
Unspecified in places (e.g. section 2.1 should define \(\alpha_k\) and \(\gamma_k\), "recommended parameter" requires specification, etc).

Author response:
**Unspecified in places** Fixed.

Reviewer comment (Theory):
Eq. 3.6 (and consequently Thm 3.2) is tight for time invariant stepsize? Since this would be a stronger claim I would state it explicitly, and contrast it with e.g. Thm. 3.4.

Author response:
**Tight for time invariant stepsize?** No. There could be improvement after removing the constraint on the sum of stepsizes and we believe that this is true for every $n$.

Reviewer comment (Writing):
Why define \(z^0 = z^{init}\)?

Author response:
**Why define $z^0=z^{init}$ ?** It is unnecessary. We have removed the definition.

Reviewer comment (Theory):
Is it possible to prove anything for Algorithm 1?

Author response:
**Is it possible to prove anything for Algorithm 1?** We manage to prove that Algorithm 1 with infinitely small stepsize and information of $\rho$ converges on Example 1 with any $a>0$, $b<0$, i.e. $\rho\in(-1/L,0)$. The proof has been included in Appendix D.2. We believe that the result can be generalized. This could be related to the property of Example 1 that the local $\rho$ is same everywhere.

Reviewer comment (Theory):
Could the authors further explain the derivation for the inequality in (3.1)?

Author response:
Eq. 3.1, now renumbered as Eq. 4.1, originates from AdaptiveEG+. We have introduced this algorithm in Section 2.1. It may looks different from the presentation in the original work ([1], Alg. 1, 1.2). We would like to mention that it is helpful to replace the notation $H=id-γ_kF$ using $H\bar{z}^k-Hz^k=-\gamma_k F\bar{z}^k$ (which is true in unconstrained situations). 

**On the inequality in Eq. (4.1)**

   We provide the detailed proof below.
   $$
   \lVert Fz^k-F\bar{z}^k \rVert\leq \gamma_kL\lVert Fz^k \rVert $$

   $$
   (1-\gamma_k^2L^2)\lVert Fz^k\rVert^2+\lVert F\bar{z}^k\rVert^2-2\langle Fz^k, F\bar{z}^k\rangle \leq 0$$

   $$
   \lVert Fz^k\rVert^2-\frac{2}{1-\gamma_k^2L^2}\langle Fz^k, F\bar{z}^k\rangle +(\frac{1}{1-\gamma_k^2L^2})^2 \lVert F\bar{z}^k\rVert^2 \leq \frac{\gamma_k^2L^2}{(1-\gamma_k^2L^2)^2}\lVert F\bar{z}^k\rVert^2$$

   $$
   \lVert Fz^k-\frac{1}{1-\gamma_k^2L^2} F\bar{z}^k\rVert \leq \frac{\gamma_kL}{1-\gamma_k^2L^2}\lVert F\bar{z}^k\rVert$$

   $$
   \langle F\bar{z}^k, Fz^k-\frac{1}{1-\gamma_k^2L^2} F\bar{z}^k\rangle \geq-\frac{\gamma_kL}{1-\gamma_k^2L^2}\lVert F\bar{z}^k\rVert^2 $$
   $$
   \langle F\bar{z}^k, Fz^k\rangle \geq \frac{1}{1+\gamma_kL}\lVert F\bar{z}^k\rVert^2
   $$
   A similar process can be found in the proof of Lemma A.2, Eq. (A.17)~(A.22).

   We acknowledge that it would be somewhat hasty to summarize this as "follows from $\lVert F\bar{z}^k-Fz^k\rVert \leq \gamma_k L \lVert Fz^k\rVert$ and Cauchy-Schwarz inequality". We will refine the expression in the subsequent revision. (done)

Reviewer comment (Theory):
Besides the adaptive exploration proposed in Algorithm 1, I am curious for $n$-step EG whether it is expected to get better range on $\rho$ with larger $n$, or it is sufficient to only take $n = 3$ with proper intermediate stepsizes for better range on $\rho$? Based on the current (non-tight for $n \geq 3$) analysis in the paper, one can get better range on $\rho$ with 3 steps than taking $n$ steps with $n$ tends to infinity (in which case we get $\rho > -(1 - 1/e)/L$).

Author response:
We anticipate that a larger $n$ will yield an improved range for $\rho$. The constraint of $\sum_i \gamma_i L<1$ represents a compromise, given the complexity of the otherwise condition. Furthermore, stepsize choice $(\gamma_1,\gamma_2)$ in the 2-step scenario can be regarded as special cases $(\gamma_1,\gamma_2,0)$, $(\gamma_1,0,\gamma_2)$, $(0,\gamma_1,\gamma_2)$ in the 3-step scenario. Consequently, we expect that expanding the number of dimensions will enhance the overall results.

   We have conducted new numerical calculations to determine the stepsizes that minimize $\rho_0 L$ under the scenarios of $n=3$ and $n=4$ . We enumerate the results below, covering the $n=2$ case as well.

   |      | $\delta_1$ | $\delta_2$ | $\delta_3$ | $\delta_4$ | $-\rho_0 L$ |
   | ---- | ---------- | ---------- | ---------- | ---------- | ----------- |
   | n=2  | 0.52212    | 0.644793   |            |            | 0.583456    |
   | n=3  | 0.272899   | 0.512753   | 0.515522   |            | 0.632242    |
   | n=4  | 0.22       | 0.31       | 0.39       | 0.44       | 0.657724    |

   The result for $n=3$ is derived by the numerical maximize function *NMaximize[]* in Mathematica and is tight. The result for $n=4$ is achieved by manual tests and serves as a lower bound. These results confirm our expectations.

   The codes used for the calculations have been added to the supplementary materials. (`3stepEG.nb` and `4stepEG.nb`)

Reviewer comment (Writing):
What is the $S^c$ in equation 2.6?

Author response:
2. $\mathcal{S}^c$ denotes the complement of the set $\mathcal{S}$. We have included this notation in the definition.

Reviewer comment (Theory):
Proof of Lemma 2.3 is missing (even if it is trivial the authors should include it for completeness).

Author response:
3. We appreciate the reviewer's reminder to include the proof of Lemma 2.3, which has now been incorporated into the revised manuscript.

Reviewer comment (Theory):
The presentation of Lemma 2.4, its importance, and the discussion after the results should be improved as this is a vital part of the understanding of the paper. The authors should include more information and explain the motivation behind it. For example, what is the purpose of parameter $\lambda_k$ here (why is not simply equal to 1)? Also, how does algorithm 2.9 satisfy the assumptions of Lemma 2.4? More explanation is needed.

Author response:
4. We have moved the previous Alg. 2.9 and Thm. 2.5 to an independent section and elaborated on the motivation. We have relocated Lemma 2.4 to Appendix A, as it serves as an intermediary result in our analysis.

   $\lambda_k$ is a conventional scaling parameter in related algorithms. The idea lies in that scaling the projection distance with a factor in $(0,2)$ still progresses toward the target. This parameter provides flexibility, encompassing any small enough update stepsize into the algorithm. In the initial manuscript, we consistently set $\lambda_k$=1. In the revised paper we have incorporated $\lambda_k$ into algorithm presentation and theorems for completeness.

Reviewer comment (Theory):
In section 3 the authors mentioned, "The key is to calculate stepsize from projection and assure its positiveness." The reviewer understands the statement but questions how this is related to equation (3.1). Important connections are missing. Why is that projection?

Author response:
5. We have elaborated the idea in the newly added Section 3. The concept starts from (AdaptiveEG+) where the update are interpreted as a projection onto a hyperplane. We have as well introduced the link between calculation of $\alpha_k$ and the projection distance.

Reviewer comment (Presentation):
Sections 3.1 and 3.2 are focused on the algorithms and statements of the theorems. However, there is no explanation of what the results mean and how they are positioned in the literature. What are the benefits of the final statements? For example, it is unclear why Theorem 3.2 is useful and what it means. The whole section 3, and in particular pages 5 and 6, include several theorems one after the other without a proper presentation and without paragraphs explaining in more detail the main outcome.

Author response:
6. We have restructured the sections (now numbered 4.1 and 4.2), along with the theorems. Each theorem now is accompanied by explanatory remarks that elucidate their significance and utility. Guiding text has been added to both sections to ensure clarity and prevent any potential confusion among readers.

Reviewer comment (Presentation):
There is no clear explanation in the main paper of why by taking multiple extrapolation steps one can have a weaker MVI. This is the main selling point of the work, but all the discussion of this is in the appendix. By simply reading the main paper, this is not clear.

Author response:
8. It is challenging to explain the underlying idea. We have introduced the perspective in Section 3 that exploration provides new information and narrows down the possible region of $z^*$. Another related viewpoint has been supplemented in Appendix C.2.  We hope that these efforts will enhance the clarity and comprehension of our work.

Reviewer comment (Experiments):
The cited work of Choudhury et al. 2023 includes convergence of the deterministic optimistic methods for solving weak Minty inequalities with $\rho>-1/2L$. It might be worth comparing with the Optimistic method as well in experiments and in the comparison of theory in the main paper.

Author response:
10. We thank you for your suggestion regarding optimistic methods. We find this idea very interesting. Due to space limitation, we have included a concise analysis in Appendix C.3 that interpret the algorithm from the perspective of projection.

    Regarding the experiments, we believe that single-call algorithms represent a different direction in the performance-complexity tradeoff compared to our algorithm, which is why we chose not to include them in the comparison.

Reviewer comment (Theory):
The guarantee requires knowing $\rho$. To ensure convergence, it is necessary to know the $\rho$ problem parameter. However, this presents a challenge as the convergence guarantee depends on it. Although the authors suggest ways to estimate it, it remains uncertain if the proposed approach will converge when using a noisy estimate. This limitation also applies to Alg. 1, which also requires knowledge of $\rho$ to function correctly.

Author response:
**The guarantee requires knowing $\rho$** We would like to clarify that our methods maintains the same level of requirement for the knowledge of $\rho$ as previous methods. In the revised version, the estimation parameters are incorporated into the theorem and convergence has been proved. Furthermore, the presence of $\lambda_k \in (0,2)$ in the framework accommodates any sufficiently small update stepsize, reducing the dependence of the knowledge of $\rho$ for calculating $\alpha_k$. Alg. 1 has also been revised to include the estimation parameter and Thm. 4.4(i) works for Alg. 1 as well.

Reviewer comment (Writing):
Writing. The writeup needs further refinement. See the non-exhaustive examples listed under Minor.

Author response:
**Writing** We have fixed the mentioned issues and refined the remaining part of the paper.

Reviewer comment (Evaluation):
Additional practical concerns. The studied methods involve a variable number of inner steps, which makes it more complex than the methods discussed in this paper. As a result, it is uncertain whether it can be applied to a wider range of problem classes. However, it is reassuring that it offers guarantees for the specific problem class addressed in this paper.

Author response:
**Additional practical concerns** Concerning the variable exploration steps, we would like to mention that fixing $n=2$ provides guarantee for $\rho>-0.55/L$ and $n=6$ provides guarantee for $\rho>-0.6/L$. These are significant improvements, achieved with increasing computational cost by a constant factor.

Reviewer comment (Presentation):
Many of the figures are not referenced, making it unclear when the reader should consult them. Moreover, these figures are not explained; for example, it would be helpful to describe how Fig. 2 helps understand the theorems.

Author response:
**Figures not referenced** We have now ensured that each figure in the document is not only referenced but also accompanied by a detailed explanation. Regarding the previous Fig. 2, we've transferred it to Appendix C.1 and provided discussion there. Furthermore, we have reformulated the theorem statements in Section 4.1, enhancing the figure's understandability.

Reviewer comment (Theory):
The method should not be classified as a *more than local* method (page 1) because the information is still very much local, but it's applied to multiple points, which has obvious computational disadvantages.

Author response:
**On the expression "more than local"** We agree that for n-step EG the information is still very much local. However, we believe it is not applicable to Alg. 1. In the case of circular limit cycles or divergent GDA, the subroutine often explores through about 1/4 of the circumference. Fig. 1(c) and Fig. 8 (random exploration) may also provide support for this claim. Therefore we would like to stand by this expression.

Reviewer comment (Writing):
The abstract has numerous typos. Examples include:  
- Min-max problem -> The min-max problem  
- application in ML -> applications in ML  
- exhaustive researches -> exhaustive research  
- problem on nonconvex-nonconcave setting -> problem of nonconvex-nonconcave setting  
- is proved -> has proved  
- process exploit [...] tackle -> process exploits [...] tackles

Author response:
**Typos in abstract and introduction** We thank the reviewer for carefully pointing out these errors. We have fixed them in the revised text.

Reviewer comment (Writing):
In the Introduction, $\rho, L$ are mentioned without introducing them. It would be helpful if at least (2.3) is moved into the introduction and if it is mentioned that $L$ is the Lipschitz constant.

Author response:
**Missing introduction for parameters** The mentioned parameters are introduced in the revision.

Reviewer comment (Writing):
Typo in the Introduction: "algorithm that exploit" -> "exploits".

Author response:
**On contribution 1** We have updated our expression to avoid confusion.

Reviewer comment (Theory):
On page 4, you mentioned that the lower bound can be represented by parameters in $G_k$. However, it is unclear how convergence can be proven if $\rho$ is estimated as commented therein.

Author response:
**Representing range of $\rho$ by parameters in $G_k$** We have elaborated this issue in the revised paper. The parameters have now been included in Thm. 4.4 in a rigorous manner. The main idea of the proof lies in that parameters within this range fulfill the conditions in Thm 4.3, which in turn are pivotal for ensuring convergence. Additionally, we manage to recover the parameter range of (AdaptiveEG+) when n=1.

Reviewer comment (Theory):
In Theorem 2.5, it is unclear what the assumption on $\rho_0$ is.

Author response:
**Assumption on $\rho$ in the general framework** There is no explicit assumption on $\rho$. The addressable range of $\rho$ depends on the structure of the subroutine. We have covered this issue in the revised paper (Page 3, below Thm. 3.1) and guide readers towards Section 4 for details.

Reviewer comment (Presentation):
Description of Fig. 2 is lacking; please clarify how it was obtained (e.g., are $L,\rho$ from a specific problem?).

Author response:
**Derivation of Fig. 2** No specific example is involved. This figure is used to visually display the results of Theorems 4.1 and 4.2. The figure is a contour of a bivariate function, now presented in equation (C.1). The Matlab code used to create this figure have been included in the supplementary materials (`2stepEG_range.m`).