Title: Rethinking Fair Federated Learning from Parameter and Client View
Venue: NeurIPS 2025
Accepted: yes
Keywords: Federated learning

Fairness
Abstract: Federated Learning is a promising technique that enables collaborative machine learning while preserving participant privacy. With respect to multi-party collaboration, achieving performance fairness acts as a critical challenge in federated systems. Existing explorations mainly focus on considering all parameter-wise fairness and consistently protecting weak clients to achieve performance fairness in federation. However, these approaches neglect two critical issues. 1) Parameter Redundancy: Redundant parameters that are unnecessary for fairness training may conflict with critical parameters update, thereby leading to performance degradation. 2) Persistent Protection: Current fairness mechanisms persistently enhance weak clients throughout the entire training cycle, hindering global optimization and causing lower performance alongside unfairness. To address these, we propose a strategy with two key components: First, parameter adjustment with mask and rescale which discarding redundant parameter and highlight critical ones, preserving key parameter updates and decrease conflict. Second, we observe that the federated training process exhibits distinct characteristics across different phases. We propose a dynamic aggregation strategy that adaptively weights clients based on local update directions and performance variations. Empirical results on single-domain and cross-domain scenarios demonstrate the effectiveness of the proposed solution and the efficiency of crucial modules. The code is available at https://github.com/guankaiqi/FedPW.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper aims at solving a challenge  in federated learning： existing fair FLs often focus on improving weak clients during the training process. While this enhances weak clients, it can hinder global optimization, leading to lower overall performance and persistent unfairness. \n\nTo tackle this issue, the paper proposes a new fair FL framework, FedPW, with two main parts：(1) it introduces a Parameter Adjustment technique using masks and rescaling to discard redundant parameters, thus preserving important updates and reducing conflicts. (2) the author(s) propose  a dynamic aggregation function which  adaptively weights clients in training process varies across two phases. \n\nThe experiments show that FedPW achieves better global model accuracy and fairness across clients from different domains compared to multiple baselines.\n\nSTRENGTHS AND WEAKNESSES:\n1. This work proposes a Parameter Adjustment + Adaptive Weighting strategy. The former addresses weight conflicts  in federated learning under multi-objective optimization, while the latter provides a solution to ensure balanced performance across clients from different domains.\nThe inclusion of the Early Phase/Later Phase criterion is a valuable addition that provides insights into when fairness should be integrated into the training process. The proposed solution is logically structured, and the technical details are well-explained, making it a useful contribution to the FL community. \n\nHowever, the adaptive  Weighting may have some practical limitations: \n\n(1) whether accuracy and fairness can achieve Pareto stability, and \n\n(2) whether the final level of fairness can be controlled, which can be guaranteed in other multi-objective gradient descent-based fair FLs.\n\n2. Another strength is the well-designed experiment setup, with sufficient baselines and a variety of experimental scenarios. The effectiveness of the experiments may make this paper a good new fair FL baseline for federated learning with cross-domain clients. \n\nThe corresponding weakness is that the experiments is not completely provided in the code link.\n\nQUESTIONS:\n1. How is the redundancy value in the Fig. 2 (right) y-axis evaluated?\n\n2. The standard deviations are significant in Table 2, and it seems not to represent an ideal fairness level, which means that the accuracy performance disparity among clients reaches up to 20%?\n\nLIMITATIONS:\nThe adaptive  Weighting may have some practical limitations: \n\n(1) whether accuracy and fairness can achieve Pareto stability, and \n\n(2) whether the final level of fairness can be controlled, which can be guaranteed in other multi-objective gradient descent-based fair FLs.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on performance fairness in federated learning. It addresses two key challenges: (1) parameter redundancy, which can lead to inefficient model updates, and (2)  the persistent protection of weak clients, which may act as outliers and reduce overall model quality. To tackle these issues, the authors propose FedPW, a dual-pronged approach: the paper introduces a parameter adjustment strategy to reduce redundancy and proposes an adaptive weighting mechanism to dynamically balance client contributions based on their reliability. The authors conducted extensive experiments, achieving higher average accuracy and lower performance disparity compared to baseline FL methods and fairness-oriented approaches in both single-domain and cross-domain scenarios.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The figures are well-designed and enhance comprehension. The writing is clear and logically structured, making the paper easy to follow.\n\n2. The paper addresses a fundamental and meaningful question: Can an FL algorithm achieve fairness without compromising performance? The authors provide two key insights to mitigate performance degradation in fairness-driven methods.\n\n3. The identification of parameter redundancy and persistent protection issues is well-articulated and supported by explicit evidence.\n\n4. The proposed method is novel and elegantly addresses the stated challenges.\n\n5. The extensive experimental results demonstrate consistent improvements in both performance and fairness.\n\nWeaknesses:\n1. Definition 3.1 introduces performance fairness as the variance of client losses. However, the rationale for choosing this metric over established fairness measures (e.g., Gini coefficient, max-min gap, or Jain’s index) remains unclear. Please justify this choice and compare its implications with alternative metrics.\n\n2. The experimental scope omits validation under extreme conditions, such as large-scale FL settings (>100 clients) or more realistic heterogeneity (e.g., Dirichlet allocation with α=0.01). These scenarios are critical for assessing scalability and robustness.\n\n3. The approach requires all clients to share identical model architectures, which may limit generalization.\n\nQUESTIONS:\n1) How does FedPW generate client-specific parameter masks? Could the authors clarify the criteria for selecting which parameters to mask for each client?\n\n2) What defines the transition between training phases? Are these boundaries based on empirical observations or adaptive metrics?\n\n3) Please specify the communication settings between the client and server (e.g., local training epoch), as these details are critical for reproducibility.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-19"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes FedPW, a novel federated learning framework aimed at improving both global performance and fairness across clients. To address parameter redundancy, it selectively discards insignificant local updates and amplifies consensus parameters that exhibit cross-client agreement. Additionally, it introduces an adaptive aggregation strategy that adjusts client weights dynamically based on the training phase: reinforcing update alignment in early training and emphasizing fairness in later stages. Extensive experiments on single- and cross-domain tasks demonstrate the effectiveness and compatibility of the proposed approach.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths：\n1. This paper presents a technically solid approach with a thoughtfully designed method that tackles practical issues in federated learning. Extensive experiments consistently demonstrate the effectiveness of the proposed method.\n\n2. This paper addresses an important and practical problem in federated learning: achieving fairness without compromising global performance.\n3. This paper is well-written and well-organized.\n\nWeaknesses:\n1.  This method introduces several hyperparameters, but the sensitivity analysis is relatively limited.\n\n2. This method relies on a common model architecture among clients, which could restrict its applicability in practical federated settings where model heterogeneity is present.\n3. The overall framework introduces multiple components, making the system relatively complex and potentially difficult to implement or reproduce. It may hinder adoption in practical federated settings, especially where computational or engineering resources are limited.\n\nQUESTIONS:\n1. The mask ratio $c$ and momentum coefficient $\\beta$ appear to be critical hyperparameters, yet their impact is only briefly discussed. A more thorough analysis of how these parameters influence performance would be valuable.\n\n2. Given that masking, normalization, and consensus computation require non-trivial operations, can the authors quantify the additional cost (e.g., time per round, memory usage)?\n3. Due to the trade-off relationship between the number of communication rounds and local epochs in federated learning systems, so how does the number of **local epochs** (or steps) affect FedPW’s performance?\n4. Some minor issues: Some methods in the related work section lack references, such as FedCurve and Ditto.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper aims to overcome the bias that FL based models face by proposing FedPW, a fairness-aware federated learning method that addresses two challenges: (1) parameter redundancy, where unimportant parameter updates interfere with meaningful gradients, and (2) persistent client protection, where prioritizing weak clients across all rounds leads to convergence degradation. FedPW performs parameter masking/resizing based on local update magnitudes and applies a phase-adaptive client weighting scheme based on gradient alignment and local loss. Empirical evaluations demonstrate improved performance-fairness trade-offs.\n\nSTRENGTHS AND WEAKNESSES:\n[S1] The paper relies on a known observation that fairness mechanisms may themselves cause unfairness or performance degradation, particularly due to persistent overweighting of weak clients and noisy parameter updates. This is in line with the theory of importance sampling that while importance weighting impacts machine learning models early in training, its effect diminishes over successive training iterations.\n\n[S2] The proposition of a two-phased training heuristic (early: consensus-driven; late: fairness-driven), instantiated using statistical indicators from client gradients and loss, is an interesting bias-mitigation approach.\n\n[S3] The evaluation covers both single-domain and cross-domain FL with baselines and ablations.\n\n[W1] The central idea of parameter redundancy appears to be both under-theorized and lacks any formalization; The redundancy criterion is based entirely on local update magnitude, assuming small updates are unimportant. Is this assumption valid? This needs clarity because of a few points: Small updates could represent delayed or convergent directions. Update magnitude is highly sensitive to batch stochasticity and learning rate. Parameters interacting in high-dimensional manifolds (e.g., batchnorm scales) often have noisy gradients regardless of their importance. For this, I have a question: How do you distinguish between truly redundant updates and temporally small ones due to gradient noise or slow convergence?\n\n[W2] The masking and rescaling steps (Eqs. 5–7) are the mathematical core of the method, but they are treated entirely as black-box heuristics. There is no attempt to analyze: (1) How masking affects the optimization trajectory (e.g., bias, variance, directionality). For this, I have a question: Can you derive any convergence bound for the masked and rescaled FedAvg variant? Under what assumptions on gradient noise does your procedure remain unbiased?\n\n[W3] The dual-phase aggregation strategy (Eq. 11) is intuitive but lacks robustness or theoretical validation. The scheduler uses the relative standard deviations $\\sigma_p$ and $\\sigma_q$ of gradient consensus and loss weights, respectively. This implicitly assumes: (1) gradient directions align early in training ($\\sigma_p$ high), and (2) loss disparities dominate later ($\\sigma_q$ high). However, in practice, these signals can be noisy, correlated, or reversed depending on batch stochasticity, task diversity, or optimizer state. For this, my question is: How sensitive is the scheduler to the batch size, number of clients, or variance scaling?\n\n[W4] The paper defines performance fairness as minimizing the variance of client losses (Definition 3.1). While this is common in FL literature, it may have limitations: variance is symmetric as it penalizes both over- and underperformance. But fairness typically aims to improve the worst-off groups. For this, my question is: are other fairness metrics applicable in this work such as max-loss, minimum accuracy, or worst-group accuracy across clients?\n\n[W5] Several parts of the paper suffer from imprecise explanations or inconsistencies. For instance, in Figure 2 (parameter pruning analysis) The x-axis is not defined. Does \"mask rate\" refer to percentage of parameters, magnitude threshold, or layer-wise pruning?\n\nQUESTIONS:\n1) How do you distinguish between truly redundant updates and temporally small ones due to gradient noise or slow convergence?\n\n2) Can you derive any convergence bound for the masked and rescaled FedAvg variant? Under what assumptions on gradient noise does your procedure remain unbiased?\n\n3) How sensitive is the scheduler to the batch size, number of clients, or variance scaling?\n\n4) Are other fairness metrics applicable in this work such as max-loss, minimum accuracy, or worst-group accuracy across clients?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-25"}