Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a new federated learning algorithm, named FedMIM, which adopts the multi-step inertial momentum
on the edge devices and enhances the local consistency for free during the training to improve the robustness of the heterogeneity.

In terms of theory the paper, the paper proves that FedMIM achieves the $O(1/\sqrt{SKT})$ convergence rate under the nonconvex setting, where $S$ is the number of selected clients and $K$ is the proper local interval in each communication round. For non-convex function under PL condition, a convergence rate $O(1/T)$ was also proved for FedMIM. 

The paper conduct comprehensive experiments on various real-world datasets and demonstrate the efficacy of the proposed FedMIM against several state-of-the-art baselines.
strength_and_weaknesses: The paper is well written and the main contributions of the work are well presented. I went through the main proofs and the important steps looks also correct.

The paper is easy to follow and the main contributions are clear. 

Suggestion: It would be nice at the point where FedMIMN proposed a pseudocode to explain how one can obtain the classical Local SGD as a special case (if this is possible). 

Missing references: Closely related papers are [1] and [2] below. 

Questions: 
1. Also to the best of my knowledge one of the state-of-the-art methods for solving non-convex problems is the $\tau$-overlap SGP from paper [2]. How your approach is related to this method? 

2. SlowMo from the paper Wang et al. (2019), which is already cited in the paper, is also a fast method. Why is not used in the experiments? 

Missing References:

[1] Koloskova, Anastasia, Nicolas Loizou, Sadra Boreiri, Martin Jaggi, and Sebastian Stich. "A unified theory of decentralized sgd with changing topology and local updates." In International Conference on Machine Learning, pp. 5381-5393. PMLR, 2020.

[2] Assran, Mahmoud, Nicolas Loizou, Nicolas Ballas, and Mike Rabbat. "Stochastic gradient push for distributed deep learning." In International Conference on Machine Learning, pp. 344-353. PMLR, 2019.
clarity,_quality,_novelty_and_reproducibility: Please see the above review for further details.
summary_of_the_review: As I mentioned above, the paper is well written and the main contributions of the work are well presented.

I give a score of "6: marginally above the acceptance threshold" to this submission.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposed a new federated learning algorithm (i.e., FedMIM),  which adopts the multi-step inertial momentum on the edge devices and enhances the local consistency for free during the training  to improve the robustness of the heterogeneity. Theoretically, it proved that 
the FedMIM algorithm achieves the best known convergence rate with a linear speedup property with respect to the number of selected clients and proper local interval in each communication round under the nonconvex setting. Empirically, it conducted comprehensive experiments on various real-world datasets and demonstrate the efficacy of the proposed FedMIM against several state-of-the-art baselines.

strength_and_weaknesses: Strength:

This paper proposed a new federated learning algorithm (i.e., FedMIM),  which adopts the multi-step inertial momentum on the edge devices and  enhances the local consistency for free during the training  to improve the robustness of the heterogeneity. Theoretically, it proved that 
the FedMIM algorithm achieves the best known convergence rate with a linear speedup property with respect to the number of selected clients and proper local interval in each communication round under the nonconvex setting. 

Weakness:

The novelty of this paper is limited. This paper basically follows the existing Nesterov' momentum techniques.
clarity,_quality,_novelty_and_reproducibility: The writing can significantly be improved. 
The novelty of this paper is limited. This paper basically follows the existing Nesterov' momentum techniques.
The motivation of this paper is not clear. 

summary_of_the_review: This paper proposed a new federated learning algorithm (i.e., FedMIM),  which adopts the multi-step inertial momentum on the edge devices and  enhances the local consistency for free during the training  to improve the robustness of the heterogeneity. Theoretically, it proved that  the FedMIM algorithm achieves the best known convergence rate with a linear speedup property with respect to the number of selected clients and proper local interval in each communication round under the nonconvex setting. Empirically, it conducted
comprehensive experiments on various real-world datasets and demonstrate the efficacy of the proposed FedMIM against several state-of-the-art baselines.

Some Comments:

1. The motivation of this paper is not clear. Why could multi-step inertial momentum on the edge devices 
 enhance local consistency ?

2. What is the parameter $I$ in the FedMIM algorithm ？

3. How to choose the tuning parameters $\alpha_j$ and $\beta_j$ in the FedMIM algorithm ?

4. Reproducibility is a crucial aspect of any work based on experiments. 
Your code should be available to the reviewers throughout all stages of the submission and publicly.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This work propose a federated learning algorithm, named FedMIM, which adopts the multi-step inertial momentum on the edge devices and enhances the local consistency for free during the training to improve the robustness of the heterogeneity. Specifically, the authors incorporate the weighted global gradient estimations as the inertial correction terms to guide both the local iterates and stochastic gradient estimation, which can reckon the global objective optimization on the edges’ heterogeneous dataset naturally and maintain the demanding consistent iteration locally.  
strength_and_weaknesses: strength: multi-step inertial momentum on the edge devices to enhances the local consistency

weakness: lack of novelty compared to the server side momentum (Karimireddy et al., 2020b) or client side momentum Xu et al. (2021) 
clarity,_quality,_novelty_and_reproducibility: lack on novelty

lack of code to reproduce experiment results 
summary_of_the_review: federated learning is a great area to study. this work adopts the multi-step inertial momentum on the edge devices to improve the robustness of the heterogeneity.  However, the novelty of the paper is not well-established.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper proposes a new variant of federated learning with momentum, establishes its convergence rate and shows its empirical performance. The main hinderance to a higher score for this paper is the lack of novelty in either the problem setting, the algorithm or its analysis. 

This is reflected both in the comments of the reviewers, and in their scores. There is now a rich enough body of literature with similar algorithms and analyses that with this exact setup it may be hard to stand apart even if it is well written and technically sound.
justification_for_why_not_higher_score: Lack of novelty in setting or algorithm, similarity to many previously published works.
Decision: Reject