Title: A Robust Training Method for Federated Learning with Partial Participation
Venue: NeurIPS 2025
Accepted: no
Keywords: Partial participation

Stochastic optimization

Convex optimization

Non-convex optimization
Abstract: Client weighting and partial participation are key techniques in federated learning. They reduce communication costs and maintain a balance in the data used for model training. Numerous strategies are well-established within the research community, leading to growing interest in developing a unified theory. In this paper, we explore this issue in detail. We propose a method that accumulates unused gradients from the current iteration locally and, after full aggregation, leverages them for effective training. Our framework supports a wide class of weighting and sampling heuristics. Furthermore, we show the proposed approach to be robust against clients' periodic disconnection. To validate it, we conduct a series of numerical experiments involving the training of convolutional and transformer-based architectures.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces PPBC, a unified framework for federated learning that addresses the challenges of client weighting and partial participation. PPBC accumulates unused gradients locally during each round and incorporates them after global aggregation to improve training effectiveness. The method is compatible with a wide range of client sampling and weighting strategies and is robust to periodic client disconnections. Experimental results on both convolutional and transformer-based models demonstrate the effectiveness of PPBC in improving performance under partial participation scenarios.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n+ Versatile and unifying framework: The proposed PPBC framework is highly versatile, supporting a wide range of client weighting and sampling strategies under a unified mechanism. This flexibility is a significant contribution, as it allows practitioners and researchers to explore and benchmark different federated learning strategies consistently within the same implementation. \n\n \nWeakness\n- Lack of baseline comparisons with original strategies: The paper evaluates PPBC by applying three existing client sampling/weighting strategies but does not compare PPBC's performance directly against the original implementations of those strategies. Since PPBC integrates these strategies, it is important to assess whether and how closely it replicates their expected training behavior and performance. Such comparisons are critical to verify the framework’s correctness and fidelity. \n\n- Insufficient discussion of convergence consistency: One of the surprising results is that PPBC exhibits similar convergence behavior across different strategies. However, the paper does not provide sufficient explanation or analysis of this phenomenon. Understanding why PPBC converges robustly regardless of the underlying strategy would be important for interpreting its stability and generalization potential. A deeper discussion on this point would enhance the paper’s impact and clarity.\n\nQUESTIONS:\n1) Convergence consistency across strategies: The experiments show that PPBC maintains similar convergence trends regardless of the underlying client sampling and weighting strategies applied. \n\n* Could the authors elaborate on why convergence remains consistent across these strategies? \n\n* Does this imply a generalizable property of PPBC that ensures stable convergence for any compatible strategy, including those not tested in the paper? \n\n* A theoretical justification or an empirical investigation would greatly strengthen the contribution and clarify the mechanism behind this observation. \n\n2) Lack of comparison with original strategies used in PPBC: The evaluation applies several existing strategies within the PPBC framework, but surprisingly, there is no direct comparison with the original versions of those strategies (outside PPBC). \n\n* Why are such baseline comparisons omitted? \n\n* Given that PPBC builds on these strategies, it would be valuable to show whether PPBC can match or even improve their performance under the same conditions. \n\nIncluding these comparisons would improve the clarity of PPBC’s contributions and its fidelity in replicating diverse behaviors. \n\n3) Choice of baselines (SCAFFOLD and FedAvg): The paper includes FedAvg and SCAFFOLD as baselines, but these are relatively dated. \n\n* Could the authors justify the exclusion of more recent and competitive baselines, such as FedDyn, FedNova, or MOON, which are known to perform better in heterogeneous data settings? \n\n* Including these comparisons would provide a stronger and more up-to-date empirical validation of PPBC.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 1, "CLARITY": 2, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a method for partial client participation in the federate learning setup. The proposed method, Partial Participation with Bias Correction (PPBC) lets inactive federated-learning clients cache a gradient surrogate and leverages it during training. The authors provide theoretical analysis of PPBC convergence under different assumptions. Experiments on CIFAR-10 (ResNet-18) across three client-selection heuristics show faster loss reduction and higher accuracy than FedAvg and SCAFFOLD.\n\nSTRENGTHS AND WEAKNESSES:\n***Strengths***\n1) The paper is arranged in a very logical manner\n2) The theoretical analysis covers different type of objectives and is quite elaborate\n3) Extension of core algorithm (Algorithm 1) to account for unavailable device (Algorithm 2) is quite practical\n\n***Weaknesses***\n\n1. The introduction section is not easy to read. It talks about client weighting and client sampling, without mentioning clearly what the relevance is to the current research paper. In Line 116, it is mentioned that “there is still no flawless unified theory of partial participation”. However, it is not clear  why a  flawless unified theory is required and what are the impact of using FedAvg, which requires boundedness of gradients or their difference.\n\n2. Using gradient surrogate seems to be a very important part of Algorithm 1, however, it is not explained how gradient surrogate is calculated and the intuition behind why it is helpful and makes the proposed algorithm perform well. In addition, the literature review does not mention much of prior research related to using gradient surrogates.\n\n3. In section3.1 line 152, it’s mentioned that current methods yield poor results. However, it is not clear what are the results and why are they considered poor\n\n4. No ablation study to understand the impact of gradient accumulation and impact of \\theta\n\n5. The idea of client sampling per section 1.2 is to reduce number of communications to the server, however line 20 in algorithm 1 will cause a communication bottleneck at the server.\n\nQUESTIONS:\n***Questions***\n1.  For the client selection schemes reviewed in the literature, do everyone of them assume inactive clients stay inactive through the entire epoch?\n2. In algorithm 1 line 20, why are the surrogate gradients sent to the server at the end of all iterations? Why are the surrogate gradients not sent once between the iterations?\n3. Referring back to line 115, how does algorithm 1 and the corresponding analysis, relaxes the requirement for boundedness of gradients or their difference.\n4. In assumption 4 what does \\eta signify?\n5. In algorithm 1, why is there no step to account for clients receiving the updated model\n6. From the algorithm and theory perspective, what are the core novel contributions\n7. For the experiments presented in the main paper, how many clients were used.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper targets Federated Learning with partial client participation. The authors first proposed a method that allows devices to accumulate gradient surrogates locally, then let the server account for them after the full aggregation round. The proposed approach\nsupports various weighting and sampling client strategies, including biased ones. The authors also comprehensively analyzed method convergence in both strongly convex and non-convex cases.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1) The proposed framework supports a wide range of client weighting and sampling strategies, including biased ones.\n\n2) The paper provides comprehensive convergence proofs for both non-convex and strongly convex settings, while most existing works focus primarily on convex assumptions.\n\n3) It is novel and practical that Algorithm 2 explicitly considers device dropout, which frequently occurs in real-world federated learning scenarios.\n\nWeaknesses and Questions:\n1) Assumption 3 states that each gradient is similar to the full gradient. I have two questions regarding this assumption:\na) Does \"each gradient\" refer to the gradient on each client?\nb) How can we ensure that the gradients remain sufficiently aligned, especially in highly non-iid scenarios? How robust is the method under poor gradient alignment caused by non-iid?\n\n2) The method applies a stochastic operator to define the selection rule (e.g., zeroing out the weights of unavailable nodes). How does this operator handle newly joined or rejoining clients?\n\n3) It is unclear how to determine the value of H^k (number of iterations per epoch) in Algorithm 1 (PPBC). How does this choice affect performance and convergence?\n\n4) In PPBC, all inactive devices are required to accumulate gradient surrogates, which may introduce overhead. If a device has not been selected for an extended period, how is the growing memory or computational overhead handled?\n\n5) Related to 3) and 4), if a device has not participated in the global aggregation for a long time, could its stale surrogate gradient distort or mislead the global update direction?\n\n6) CIFAR-10 is a relatively small and balanced dataset that is not typically used for evaluating non-iid in federated learning. The method should also be tested on some non-iid benchmarks, such as LEAF.\n\nQUESTIONS:\nPlease see the weakness and questions section.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

comments: SUMMARY:
Prior works on partial participation analysis are typically based on the standard FedAvg algorithm, which does not account for inactive clients. This paper introduces the PPBC, a framework designed for scenarios involving inactive clients. The PPBC framework is general enough to encompass various client weighting and sampling strategies. The authors provide a theoretical analysis of PPBC's convergence rate for both convex and non-convex objectives.

STRENGTHS AND WEAKNESSES:
### **Strengths**

S1. The proposed framework is comprehensive, as it covers a wide range of previously studied client weighting and sampling approaches.

S2. The authors provide a rigorous theoretical analysis of the PPBC's convergence.

S3. Experiments on CIFAR-10 and Food-101 with ResNet-18 and FasterViT confirm the method's effectiveness.

### **Weaknesses**

W1. The paper is difficult to follow; its overall structure and writing require refinement.

W2. The authors claim that leveraging information from inactive clients is one of the key contributions. However, the experiments presented in the main paper are exclusively for the PPBC variant. To support the claim, results for PPBC+ must be included and emphasized in the main paper.

W3. The set of baselines appears insufficient. A comparison with other methods designed for settings with inactive clients is necessary.

W4. In Section 3.3, the line numbers in Algorithm 2 do not match those in Appendix A.

QUESTIONS:
Please refer to the weaknesses.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03