Title: FedQS: Optimizing Gradient and Model Aggregation for Semi-Asynchronous Federated Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: Semi-Asynchronous Federated Learning

Aggregation Strategies

Convergence Analysis
Abstract: Federated learning (FL) enables collaborative model training across multiple parties without sharing raw data, with semi-asynchronous FL (SAFL) emerging as a balanced approach between synchronous and asynchronous FL. However, SAFL faces significant challenges in optimizing both gradient-based (e.g., FedSGD) and model-based (e.g., FedAvg) aggregation strategies, which exhibit distinct trade-offs in accuracy, convergence speed, and stability. While gradient aggregation achieves faster convergence and higher accuracy, it suffers from pronounced fluctuations, whereas model aggregation offers greater stability but slower convergence and suboptimal accuracy. This paper presents FedQS, the first framework to theoretically analyze and address these disparities in SAFL. FedQS introduces a *divide-and-conquer strategy* to handle client heterogeneity by classifying clients into four distinct types and adaptively optimizing their local training based on data distribution characteristics and available computational resources. Extensive experiments on computer vision, natural language processing, and real-world tasks demonstrate that FedQS achieves the highest accuracy, attains the lowest loss, and ranks among the fastest in convergence speed, outperforming state-of-the-art baselines. Our work bridges the gap between aggregation strategies in SAFL, offering a unified solution for stable, accurate, and efficient federated learning. The code and datasets are available at https://github.com/bkjod/FedQS_.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a framework of semi-asynchronous Federated Learning, FedQS, which mainly tackles two key issues: aggregation disparity and inefficiency of global information of client-centric model aggregation methods.\n\nBased on some key observations on distinct continuity in the optimization trajectories of model / gradient aggregation strategies, the authors categorize all clients into four distinct groups and apply different aggregation methods to them. Then, the local updates (local models / gradients) are accordingly aggregated in different methods.\n\nThe theoretical and empirical results shown in this paper clearly demonstrate the superior FL performance over existing semi-asynchronous FL methods.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n - The study is well organized and the proposed framework looks well-designed.\n - The key observation is interesting: the gradient aggregation strategy provides more stable global model training.\n - Divide-and-conquer strategy seems to be practical.\n\n**Weaknesses**\n\nWhile I see several strong points, I also see a couple of serious weak points as follows.\n - [System efficiency analysis] FedQS has a few extra steps compared to typical FedAvg, such as calculating pseudo global gradient, calculating similarity scores, and updating aggregation status table at the server side. The clients also have some extra works such as feedback signal calculation and communications. These extra workload increases overall computational and communication costs as well as memory footprint. However, the paper does not discuss them.\n - [Misleading terms] In Mod2, the 'biased' term may mislead readers. Precisely speaking, it is not a matter of biased or unbiased. It is a matter of strongly biased or weakly biased. E.g., *Fast-and-Unbiased* clients still produce more or less biased updates due to the non-IID data characteristics, although the degree of bias is lower than other clients.\n - [Strong assumptions] In Section 4, the gradient is assumed to have a maximum magnitude $G_c$. Although some previous works used this assumption, I still believe it is quite strong, and it makes the results less convincing.\n - The convergence properties are also not thoroughly analyzed. Both theorems show the intermediate results only. The common convention is to show that the expected average gradient norm is bounded by a constant rather than showing the bound of the loss difference. It may show different convergence properties if the left-hand side is the gradient norm, e.g., complexity.\n - In order to accelerate slow clients, the authors employed momentum in Mod2. However, why not applying it to fast clients too? Wouldn't it further improve the training speed? It makes sense to apply it to slow clients only when it hurts the training speed when applied to the fast clients.\n\nQUESTIONS:\nNo specific questions, but I expect healthful discussions regarding my comments shown in the 'weakness' part.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces FedQS, a framework for optimizing both gradient and model aggregation in semi-asynchronous federated learning, addressing challenges like client heterogeneity, staleness, and data distribution disparities. FedQS uses a divide-and-conquer strategy to classify clients into four types and adapt their training dynamically, achieving exponential convergence rates and balancing the trade-offs between accuracy, convergence speed, and stability. Extensive experiments demonstrate the performance of FedQS across tasks in computer vision, natural language processing, and real-world datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- The paper presents a strong experimental design with detailed settings and comprehensive numerical results, enabling reproducibility and validation of claims.\n\n- It provides a clear discussion of limitations and broader impacts, demonstrating transparency and awareness of the practical challenges in implementing FedQS.\n\nWeaknesses:\n\n- My major concern lies in the convergence guarantees in Section 4.2. Theorems 4.2 and 4.3 show that the function value does not converge exactly to the global optimum, leaving a non-vanishing gap $\\mathcal{U} +\\mathcal{V}$. This deviates from standard convergence results commonly found in the literature, where gradient norm convergence is typically used as a metric for nonconvex problems. Furthermore, the claimed *linear* convergence rates are questionable, as these are generally unattainable for *L*-smooth functions without additional regularity conditions (e.g., strong convexity).\n\n- While I understand the rich body of literature in federated learning, the paper still lacks a discussion of or comparison with more recent works in asynchronous federated learning. Below is a non-exhaustive list of closely related studies:\n\n   [1] Yan W. et al. A. Momentum-Driven Adaptivity: Towards Tuning-Free Asynchronous Federated Learning. In Forty-second International Conference on Machine Learning, 2025.\n   \n   [2] Wang, X., et al. Achieving linear speedup in asynchronous federated learning with heterogeneous clients. IEEE Transactions on Mobile Computuing, 2024.\n\n   [3] Yu T. et al. M. Momentum Approximation in Asynchronous Private Federated Learning. arXiv preprint arXiv:2402.09247, 2024.\n\n   [4] Wang, Y. et al. FADAS: Towards Federated Adaptive Asynchronous Optimization. In International Conference on Machine Learning, 2024.\n\n   [5] Wang, Y. et al. Tackling the data heterogeneity in asynchronous federated learning with cached update calibration. In Federated Learning and Analytics in Practice: Algorithms, Systems, Applications, and Opportunities, 2023.\n\nQUESTIONS:\nSee weaknesses.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

comments: SUMMARY:
This paper addresses three key challenges in semi-asynchronous federated learning – lack of theoretical understanding, inherent aggregation disparity and client-centric limitations. The authors propose the first FL framework FedQS with a divide-and-conquer strategy that classifies clients into four types and adapts their training strategies dynamically to optimize both gradient and model aggregation in SAFL.

STRENGTHS AND WEAKNESSES:
Strength

1. FedQS is a novel SAFL framework that simultaneously optimizes gradient and model aggregation, effectively targeting the accuracy–stability trade-off originated from FedSGD and FedAvg.

2. Solid theoretical analysis is provided to guarantee the convergence of the proposed method.

Weakness

1. The divide-and-conquer strategy needs to compute and aggregate some extra variables, i.e., update speeds and similarities. This step could be time-consuming when the number of clients is large.

QUESTIONS:
How much impact does the divide-and-conquer strategy have on the experiment’s runtime? Are there any ways to speed up the averaging operation across all clients?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces FedQS, a novel framework for optimizing both gradient and model aggregation strategies in Semi-Asynchronous Federated Learning (SAFL). FedQS classifies clients into four types based on update speed and gradient similarity and dynamically adapts local training accordingly. The approach addresses limitations of gradient aggregation and model aggregation in SAFL. Theoretical convergence proofs and extensive experiments on vision, NLP, and real-world tasks demonstrate that FedQS outperforms existing baselines in accuracy, convergence speed, and runtime efficiency.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\nNovelty and significance: The paper is the first to propose a unified framework for simultaneously optimizing gradient and model aggregation in SAFL, addressing a clear gap in the literature.\n\nTechnical soundness: Provides rigorous theoretical convergence analysis under standard FL assumptions, with separate proofs for both aggregation strategies.\n\nComprehensive experiments: Evaluations cover diverse tasks, client heterogeneity settings, and ablation studies that convincingly support the claimed benefits.\n\nWeaknesses\n\nHyperparameter sensitivity: Though there is some analysis, FedQS introduces several hyperparameters that may require careful tuning in practice, potentially limiting usability.\n\nCommunication overhead: it may introduce additional communication overhead due to the need to transmit extra data.\n\nDynamic environments: In the experimental evaluation, although the resource heterogeneity is simulated using a predefined static ratio, client resources are fixed throughout training. While FedQS theoretically supports dynamic adaptation, the absence of experiments under dynamic resource changes limits the empirical validation of its robustness and effectiveness in truly dynamic environments.\n\nQUESTIONS:\nThe authors should clarify the expected additional communication overhead from sharing similarity information and feedback signals in large networks.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}