Paper: Communication-Efficient Gradient Descent-Accent Methods for Distributed Variational Inequalities: Unified Analysis and Local Updates (ICLR-2024)

Reviewer comment (Novelty):
The main concern is the motivation behind this paper is not clear. Why do we need to design communication-efficient federated learning algorithms suitable for multi-player game formulations

Author response:
Thank you for the question. Let us further explain the motivation of our study. 

As we mentioned in the introduction of our manuscript, in the last few years, more and more machine learning models have moved from the classical optimization formulation to a multi-player game perspective where the problems are formulated as the equilibrium of a game. Also, considering communication-efficient federated learning algorithms are natural with the surge of data-intensive applications and factors like privacy. On that end, several applications require solving min-max optimization problems and VIPs using federated learning algorithms. Recent studies have been conducted on the combination of federated learning approaches and multi-player games. For example, paper [1] (see below) focuses on federated generative adversarial networks (GANs), while the work [2] explains how federated learning algorithms can be used in adversarial learning. Both problems are classical applications formulated as min-max problems (a special case of VIPs) that can also be cast as special cases of our general framework. This clearly motivates the reasons behind working in this area. In addition, existing work on the convergence analysis cannot showcase the superiority of Local GDA-type algorithms, or they need some other strong assumptions like bounded heterogeneity. Regarding all the arguments above, we believe our study here is fully motivated, and our result is significant to the community.

[1] Rasouli, Mohammad, Tao Sun, and Ram Rajagopal. "Fedgan: Federated generative adversarial networks for distributed data." arXiv preprint arXiv:2006.07228 (2020).

[2] Zhang, Jie, et al. "Delving into the adversarial robustness of federated learning." arXiv preprint arXiv:2302.09479 (2023).

Reviewer comment (Experiments):
The experimental analysis is very limited.

Author response:
Our work primarily focuses on the theoretical analysis of the proposed novel algorithms. As we mentioned in our manuscript with our experiments, our goal was to corroborate our theoretical findings. That is, running experiments on very complicated problems is not the main focus of our work. The selected experiments are well-designed, and through them, we show the performance of our algorithms for solving **robust learning** problems on **real datasets**. This is a practical setting satisfying the assumptions of our setting. 

Per the suggestion of Reviewers bdRp and  HQew, we tested the proposed methods in different settings (not captured by our theory) and compared them with other recently proposed methods, and for all settings, our approach has favorable performance. 
For example, in the burglar problem, our proposed method outperforms Local GDA and Local EG. In addition, for all settings tested, our method is faster than the recently proposed FedGDA-GT. Please see the figures attached in our general response at the beginning of our rebuttal and the individual responses to Reviewers bdRp and  HQew. As we show, our algorithm performs better compared to other state-of-the-art algorithms in the problems we considered.

Reviewer comment (Presentation):
Figures with comparison of the methods have only convergence curves without shadows showing standard deviation of function values from run to run, which is required due to stochasticity of the algorithms.

Author response:
Thank you very much for the suggestion. In our experiments, each stochastic algorithm in the stochastic setting was run for 10 trials, and in the final figures we plotted the mean of the trajectories (still fair and standard comparison in the stochastic regime). We can easily include the suggestion of the reviewer in the camera-ready version of our work and revise the figures to include shadows showing the standard deviation of the stochastic algorithms. Example for the updated plots for  Figure 1(b) and 1(d) is inlcuded [(HERE)](https://ibb.co/Rcr8S3K) and [(HERE)](https://ibb.co/RbNqLTk), respectively. We plan to include such plots in the camera-ready version of the paper.

Reviewer comment (Presentation):
Is usage of colors in Appendix C motivated?

Author response:
We simply mark the key variables in the algorithm using colors aiming to facilitate the readability of the paper.

Reviewer comment (Theory):
Only strongly-monotone co-coercitive case is considered in the paper. Can authors report on the convergence guarantees in non-strongly-monotone case?

Author response:
Thank you for the question. Let us highlight that our theorems hold quasi-strongly monotone and star-cocoercive problems. This is a structured, non-monotone class of problems, which, as we explained in the paper, includes several non-monotone problems (non-convex, non-concave min-max problems) as special cases.

Extending the convergence guarantees of our proposed methods to different settings is an interesting and challenging future research direction. Let us also note that even in the much simpler minimization problems (single-player), no convergence guarantees with an accelerated communication complexity have been provided beyond the strong convex case.

Following the reviewer's suggestion in the first question, we have applied our algorithm in the Policemen Burglar Game, and as shown in the attached figure above, our algorithm still outperforms the popular methods of Local GDA and Local EG. This gives the motivation to dive into the theoretical understanding of the methods in different settings beyond the (quasi-)strongly-monotone case.

Reviewer comment (Writing):
$prox_{\gamma R}(v)$ in Equation (4) should be $prox_{\gamma R}(x)$.

Author response:
**Response 1**: Thank you for the pointer. We have fixed it in the updated version.

Reviewer comment (Theory):
This paper does not mention the challenges or difficulties in algorithm design or theorem proofs. More explanations may help to clarify this paper’s originality.

Author response:
**Response 2**: Thank you for the suggestion. We should certainly add the following paragraph to the camera-ready version of our work (where we will have an extra page for the main paper) highlighting the challenges/difficulties in algorithm design and theorem proofs:

**Challenges or difficulties in algorithm design or theorem proofs**: 
The theoretical results of our work and the proposed algorithms are inspired by Mishchenko et al. (2022), who first proposed a ProxSkip algorithm for solving the minimization problem.  However, our work extends these ideas to the more challenging min-max and VIP settings. The detailed proof procedures of our work differ substantially compared to Mishchenko et al. (2022) as we allow the use of a more general unbiased operator via the Key Assumption 2.2. This key Assumption 2.2 on stochastic estimates allows us to study several variants of the ProxSkip update rule under a single framework, including the stochastic (Corollary 3.5), the deterministic (Corollary 3.6), and a variance-reduced method (Corollary 3.9).

In addition, our convergence analysis requires using the quantity $M\gamma^2 \sigma_t^2$ in the Lyapunov function appearing in our main theorems, which also made the convergence guarantees of our work much different. Finally, for our convergence results, we needed to use recent convergence tricks from the analysis of gradient descent-ascent (GDA) from Beznosikov et al. (2022b) in combination with the ProxSkip ideas, which was also more challenging.

An important point that is also good to have in mind is that the previous works in simple optimization problems (Mishchenko et al. (2022) and Malinovsky et al. (2022)) involve function suboptimality (a concept that cannot be useful in VI problems) in the assumptions and their proof techniques. In our setting and proofs, there are no function values. Thus, the difference between the two pieces of literature does not solely lie in different settings (minimization vs VIPs) but begins at a deeper conceptual level.

Reviewer comment (Evaluation):
Theorem 3.1 shows that ProxSkip-VIP converges to the neighborhood of the solution, while the first experiment shows that the proposed variance-reduced method converges to the exact solution. Please add some words to explain it.

Author response:
**Response 3**: We agree with the reviewer that the general Theorem 3.1 shows that ProxSkip-VIP (with any unbiased estimator) converges to the neighborhood of the solution.  However, we highlight in Lemma 3.8 that for the special case of a variance-reduced variant, the corresponding parameters $D_1=D_2=0$, which in turn drives the result in Theorem 3.1 to converge exactly to the solution. This is why the variance-reduced method converges to the exact solution in the first experiment. The experiment verifies precisely our theoretical findings.

Reviewer comment (Experiments):
Is the choice of the probability ($p=\sqrt{\gamma\mu}$) because of the purpose of analysis? What impact will the change of $p$ have on the performance of the proposed algorithms.

Author response:
**Response 4**: Thank you for the interesting question. The general result of Theorem 3.1 holds under any choice of probability $p$. However, to obtain the accelerated communication complexity in Corollary 3.2, the choice $p=\sqrt{\gamma\mu}$ is needed. As the reviewer pointed out, this is an artifact of the proof techniques related to our theoretical analysis. 
We also agree that different choices of $p$ will lead to different convergence guarantees for the proposed algorithms.
This requires a detailed check on each different choice of $p$. For example, if $p$ is set to be a small enough choice satisfying $\tau=p^2$, via our theory, we can show that the stepsize $\gamma$ is required to satisfy
$
\frac{p^2}{\mu}\leq \gamma\leq \sqrt{\frac{p^2\epsilon}{2(D_1+MD_2)}}.
$
Thus, in some particular cases of parameter settings, there will be no feasible $\gamma$ that leads to convergence. 
We highlight that our proposed parameter choice is universally applicable (there are always choices of all parameters that guarantee convergence).