Rethinking Fair Federated Learning from Parameter and Client View
Authors: Kaiqi Guan
Wenke Huang
Xianda Guo
Yueyang Yuan
Bin Yang
Mang Ye
Venue: NeurIPS 2025 poster ()
Keywords: Federated learning
Fairness
Primary area: social_and_economic_aspects_of_machine_learning

Abstract: Federated Learning is a promising technique that enables collaborative machine learning while preserving participant privacy. With respect to multi-party collaboration, achieving performance fairness acts as a critical challenge in federated systems. Existing explorations mainly focus on considering all parameter-wise fairness and consistently protecting weak clients to achieve performance fairness in federation. However, these approaches neglect two critical issues. 1) Parameter Redundancy: Redundant parameters that are unnecessary for fairness training may conflict with critical parameters update, thereby leading to performance degradation. 2) Persistent Protection: Current fairness mechanisms persistently enhance weak clients throughout the entire training cycle, hindering global optimization and causing lower performance alongside unfairness. To address these, we propose a strategy with two key components: First, parameter adjustment with mask and rescale which discarding redundant parameter and highlight critical ones, preserving key parameter updates and decrease conflict. Second, we observe that the federated training process exhibits distinct characteristics across different phases. We propose a dynamic aggregation strategy that adaptively weights clients based on local update directions and performance variations. Empirical results on single-domain and cross-domain scenarios demonstrate the effectiveness of the proposed solution and the efficiency of crucial modules. The code is available at https://github.com/guankaiqi/FedPW.

Rethinking Fair Federated Learning from Parameter and Client View
Kaiqi Guan',, I, Wenke Huang", Xianda Guo', Yueyang Yuan', Bin Yang', Mang Yel
1 National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence, Hubei Key Laboratory of Multimedia and Network Communication Engineering. School of Computer Science, Wuhan University, Wuhan, China. {guankaiqi, wenkehuang, yemang) Brrau cn
Abstract
Federated Learning a promising technique that enables collaborative machine learning while preserving participant privacy. With respect to multi-party collab- oration, achieving performance fairness acts as a critical challenge federated systems. Existing explorations mainly focus considering all parameter- wise fairness and consistently protecting weak clients to achieve performance fairness in federation. However, these approaches neglect two critical issues. 1) Parameter Redundancy: Redundant parameters that are unnecessary for fairness training may conflict with critical parameters update, thereby leading performance degrada- tion. 2) Persistent Protection: Current fairness mechanisms persistently enhance weak clients throughout the entire training cycle, hindering global optimization and causing lower performance alongside unfairness. To address these, we pro- pose strategy with two key components: First, parameter adjustment with mask and rescale which discarding redundant parameter and highlight critical ones, preserving key parameter updates and decrease conflict. Second, we observe that the federated training process exhibits distinct characteristics across different phases. We propose dynamic aggregation strategy that adaptively weights clients based on local update directions and performance variations. Empirical results on single-domain and cross- cross-domain scenarios demonstrate the effectiveness of the proposed solution and the efficiency of crucial modules. The code is available at httpp://github com/ htmm//uaaaii//ee/P
Introduction
Federated Learning (FI.) is a collaborative machine learning framework that enables multiple clients jointly train global model 38 31 18
26
57
30
27
58
19
without sharing raw data. Clients process data locally and periodically send model updates to the server, which aggregates these updates into a global model. This training paradigm effectively addresses data island and privacy issues. However, due to data heterogeneity [58] 23 intermittent client participation, and system heterogeneity, the model is prone to unfairness, which diminishes FL's generalization capability.
is a central research focus in federated learning. Existing approaches can categorized into three types: client selection [16 42 weight allocation [39 35
Improving performance fairness [
[9]
[8], and personalized local models [33]. For instance, to enhance the performance of underperforming clients, the federated server may assign them larger aggregation weights, amplifying their influence
Equal Contribution. Corresponding Author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Parameter Redundancy
(b) Persistent Protection
Clienti
Conflict D Directions
Subset Parameters
Critical Set
Redundant
① F airness Goal
4
Skewed Model
Aggropale Elkes
Client
Aggregate
Unequal Contributions
Parameter Redundant Parameter
@ Skew towards Weak Clients
Tpac 3 Persistent Protection
Eyeck X
Figure Problem illustration of existing fairness methods: Traditional approaches consider all model parameters for fairness, but not all parameters are equally important. Redundant parameters can conflict with key ones, disrupting their crucial contributions. Additionally, a long-tern weighting strategy that favors weak clients is a classic approach. However, since weak clients often deviate from the global trend, this strategy can lead to skewed initial training direction, resulting in undertrained model and performance degradation.
on model updates. However, these methods typically compromise the performance of the global model. But training an effective global model is the primary goal of FL. This provokes our thinking:
Can we design an algorithu for FL that promotes fairesss while improving the performance of the global model?
We identify two primary causes of global model degradation: Parameter Redundancy: Due to the high redundancy in model parameters, a significant portion of parameters are inherently non- essential, making it unnecessary for all parameters to participate in model aggregation. Owing to the over-parameterized characteristics of deep neural networks [10 [1 [59], not all parameters contribute equally to fitting the domain distribution. Specifically, only a subset of critical core parameters plays decisive role in model training, while other non-core, less important parameters introduce noise that disrupts the parameter space. Moreover, these marginal parameters are prone to conflicting with other critical parameters, leading to performance degradation and unfairness. Persistent Protection: Fairness approaches typically protect weak clients throughout the entire training cycle, leading to suboptimal model optimization. From conventional fairness perspective, the existence of weak clients is paramount to ensuring equitable performance across heterogeneous participants. However, we argue that such clients may inherently act as outliers within the federated ecosystem. If the system rigidly applies a static prioritization strategy favoring underperforming clients across all training phases, the global model may become excessively influenced by their gradient directions during early stages, leading to a deviation in the model's optimization trajectory and resulting in performance degradation and unfairness.
To address the aforementioned problems, we propose FedPW Fair Federated Learning via Parameter Adjustment and Adaptive Weighting). For problem I we leverage Parameter Adjustment address this challenge. Specifically, we observe that discarding small updates to parameters can reduce conflicts with clients from other domains without negatively affecting the model's performance on its own domain. Interestingly, we also find that different domains exhibit varying levels of tolerance for parameter discarding, which is inversely correlated with the domain's 's complexity. Therefore, we apply a domain-specific drop rate to discard the tail parameters of each client, ensuring fairness in the discarding process while mitigating conflicts between domains. However, the effect the discarding operation is limited. To further reduce confusion during aggregation, we identify a set of consensus parameters for amplification, making the global model's 's update direction more stable and consistent, while scaling the discarded parameter updates back to their original magnitude, which is shown to benefit the model's performance 660 15].
For Problem II we propose an adaptive weighting mechanism that dynamically adjusts model aggregation weights based on the evolving training dynamics. We observe that the training process exhibits distinct phases. Early stages may benefit from reinforcing consensus to stabilize joint training. while later stages require emphasis on domain diversity to enhance fairness. To implement this insight, we dynamically allocate aggregation weights using the dot product between client parameter updates and loss variations. Our method achieves transition from reinforcing consensus to emphasizing fairness, resulting in excellent performance and fairness. The details are presented in Sec. 3.3

---

In this paper, FedPW consists of two main components. First, parameter adjustment alleviates parameter update conflicts by discarding certain parameters, and strengthens consensus to emphasize important parameters across multiple parties. Second, through the reweighting strategy, FedPW ensures balanced performance across clients from different domains, guaranteeing fairness the di- verse update directions during multi-party collaboration. The method is straightforwar to implement and focuses on improving the aggregation step, making it easily compatible with other federated learning methods. The main contributions are summarized as follows:
Re-examining Why might Fairness Methods Harm Model Performance. Full parameter participa- tion introduce unnecessary conflicts via redundant parameters, undermining model training. A Novel Partial-pant Partiet-paraneter Dynamic Aggregation Framework. We discard minor updates while amplifying critical ones to adjust the model parameters. Furthermore, we propose an adaptive weighting strategy based on the dynamic characteristics of the training process, which mitigates the negative impact of lagging clients hindering the global model during the early stages of training. Extensive Experimental Validation. We conduct experiments on single-domain and cross-domain scenarios. With ablations, we validate the efficacy of FedPW and the indispensability of modules.
2 Related Work
2.1 Heterogeneous Federated Learning
Statistical heterogeneity across parties, commonly referred to the non-IID problem, poses signifi- cant challenges in Federated I Learning (FL.). The pioneering work FedAvg (38] demonstrated notable
performance degradation under heterogeneous data settings. To address this, many approaches
employ regularization terms to constrain local training. For instance, FedProx (34] introduced a
proximal term to mitigate divergence between local and global models, while FedDyn T. FedCurv,
and PFedMe 49 adopted similar regularization strategies. Methods like MOON [1 FCCL [17].
FedUFO 622. FedProto [50), FPL [8], and FedProc [40] incorporate alignment- based penalty terms
to harmonize feature representations across clients, addressing data heterogeneity. SCAFFOLD [2]
proposed a control variate mechanism correct client drift by reducing gradient divergence. Other approaches tackle heterogeneity via prototype- based communication. FedProto [50) aligns global and local class prototypes to handle label distribution skew, though it primarily targets single-domain label skew. For cross-domain challenges, FPL. [88 leverages clustered prototypes to generate unbi-
ased global representations, while FedGA 63 and FedDG (36) focus on domain generalization for unseen target domains. However, these methods often involve full parameter updates during training, which introduces redundancy. Our approach emphasizes parameter adjustment to prioritize critical parameters, effectively resolving confficts arising from redundant parameters across diverse domains.
2.2 Fair Federated Learning
Fairness has been a key focus in Federated Learning, with various concepts proposed, such as Performance Fairness [3 (22, Collaboration Fairness [37 64 55), and Group Fairness [9] 61 6). Performance Fairness, which aims to ensure similar accuracy across clients, is one of the most widely studied areas [32 39. Some methods address this by modifying client selection strategies. For selection strategy based on multi-armed bandit theory, choosing clients with higher local loss promote faimess and consistency. Other approaches focus on adjusting aggregation weights to unify training outcomes. A notable example is AFL [39]. which minimizes maximum loss to improve the performance of the perft-perf devices. In contrast, q-FFL. 35 introduces exponentially scaled weights to penalize clients with higher loss, leading to a more balanced accuracy distribution. FedHEAL 3 leverages the distance between local models and the global model to constrain unfair disparities. FedFV (53) uses cosine similarity to detect and resolve gradient conflicts iteratively, converging Pareto-stable solution. Ditto proposes a personalized federated learning framework that employs penalty term to control the degree of model personalization, thereby achieving fairness and robustness. FedCE (2 leverages client contribution estimation as global model aggregation weights, demonstrating improved Performance Fairness and Collaboration Fairness. However, these methods often exhibit persistent tendency to protect underperforming clients throughout the entire training cycle, which may hinder model training. Our method leverages the training dynamics, applying different weighting strategies at different stages, thereby achieving dual excellence in both generalization and fairness.
instance, CCB-CSEE 5 uses a communication-effin
3

---

3 Methodology
3.1 Preliminary
Federated Learning and Performance Fairness. In typical federated learning, system consists of K' clients, each with private data D2 [ ) At the start of each communication round I, the server shares the global model w' with all clients. Each client initializes its local model w with w", performs local optimization using its data, and sends the updated parameters back to the server. The server then aggregates these updates using weighted averaging:
w - w v/
Σ
(( (luf,S)), Ei),
w
Σ Auu
VEBr
Here, B& is a mini-batch sampled from the local dataset Dx. E represents query instance, and 17 is the local learning rate. The optimization objective is to minimize global loss:
min F(w) =E Aefa(w),
(2)
where d is the weight of client and fa(w) is the loss of local model with parameters w.
Definition 3.1. (Performance Fairness) Given two trained models, and w. model is considered to provide a fairer solution to the federated learning objective (2) if its performance across the m
devices is more uniform compared to model w, i.e. (F&(W)) var [Fx
(Fa((t) (P((ū) (
3.2 Parameter Adjustment
100
100
Motivation. Previous work has shown that model parameters often contain significant re-
90
USPS
SUTIN Scllari
dundancy [60] 15 7
S 52). with only
small set of key parameters driving perfor- mance, while most parameters redundant and ineffective. To investigate parameter re- dundancy, conducted observational ex- periment on the Digits dataset. We trained on 5 clients across four domains for 20 epochs, gradually increasing the mask rate while monitoring performance. As shown in Fig. [ all domains showed almost no performance change when a small number of parameters were discarded, with per- formance degradation only occurring when a large proportion of parameters was discarded, indicating the redundancy of the parameters. Our findings revealed that each domain exhibited significant pa- rameter redundancy, and domains with lower performance had less redundancy(Fig.. 2 If redundant parameters are included in aggregation, they can overwhelm important updates, leading to confusion in the global model. Therefore, minimizing this redundancy is essential to ensure effective model updates. Our approach focuses on discarding unimportant parameters and enhancing the updates of key ones. The specific process is as follows:
SVN
DSL3
Webcus
SYN
0.00 0.25
0.30
0.75
1.00
38 20
0.75 0:50 ais 090 035 1.00
Figure 2: Study on Parameter Redundancy. Left: Model accuracy declines with increasing mask ratio. Right: Redundancy pertooomae relationship across datasets.
Selection of Unimportant Parameters. As previously established, the parameter updates Awf exhibit significant variation: while most parameters undergo negligible changes ( (Awwf,)) - 0), a critical minority demonstrate substantial updates. We prune insignificant parameters by first representing the G-dimensional update vector:
Awi [Awi......,
Unimportant parameters are defined as those below threshold тą = ( rf (0, 1] is the client-specific mask rate. As demonstrated in Fig, 2 parameter redundancy inversely correlates with client performance (quantified by training loss). In our experiments, various methods to increase the parameter redundancy for clients with lower loss were proven effective, with the inverse loss being the simplest and most effective approach. Thus, we compute rf using the inverse of smoothed loss % from Eq. (9), where is a pyper-paramettr representing the average mask ratio.
(((-))))))))))) )orte(((Au((((LL)))) where
1/4
rk c. ΣK 1/9
(4)

---

Discurding Purameters
Musked Ljveeies
Consensus-rivveffeee Training)
F Training)
Soried Magnitater
Fl +1 Fq.
Main Duncase
Aufi
Ansj
O
O
Dlvcanded Paraaaderr
2
*
3
Resculing Consensus Parameters
Eq
Outlir
Aas'' Blggr
Clene
St amp
Staler of
Eg.
Eig.
pt
Mevcale
Clans Marked Ejstses
Global Ejelste
Resealed Global Ejstnse
Redunddnt Paramettr
ComsCHANS Parsmeterr
Clionr A
(8) Parameter Adjustment
Adaptivve Weighting
Figure 3: Architecture illustration FedPW. FedPW consists of two core components: 1 The left box refers to Parameter Adjustment (PA), which adaptively discards redundant parameters and amplifies consensus ones (Sec. 3.2 reducing interference among parameters. @ The right box represents Adaptive Weighted Aggregation (AWA), where we dynamically assign weights to each client according to the evolving training process (Sec. 3.3 In this way, conflicts among clients can be mitigated while reinforcing consensus.
Discarding Redundant Parameters. We use mask Mis TRG based on magnitudes focus important parameters which is used to discard the smallest subset of updates. The mask is defined as:
1, if Arl 2 Te:
M
(5)
0, otherwise.
Finally, the masked updates are Aw, Auf MR. where @ denotes the Hadamard product. These masked updates are subsequently aggregated through AW = A/Awf, where X denotes the weighting coefficients derived from Eq. (IT)
Consensus Parameter Rescaling. To enhance directional consensus in global updates, amplify parameters exhibiting cross-client agreement through a rescale process. First, we construct the client update matrix AW"
|mrf. ................................ Awk] ERKXG and perform normalization:
Aw
AWES
Aw
AuFii
(6)
Aw
Eg Awiy
This applies row-wise and column-wise normalization AW", enabling cross-client comparability of update directions while preserving their relative importance.
The consensus degree of parameter is quantified by its standard deviation oj = std ( across clients. We then amplify the most consistent parameters (those with the lowest o,) corre- sponding to the bottom p-quantile, where p' K Σ r is the average mask rate. The purpose of using p¹ to determine the amplification set is to rescale the aggregated gradients back to their original magnitude, which shown benefit the model's performance [5).
i Simp
(i of I sorted [[][[ G'}.
mi
otherwise,
where m and m denote the average magnitudes discarded and amplified parameters respectively. This rescaling ensures model outputs remain stable relative to the pre-discarding state. The final aggregated update becomes W W + AW"
3.3 Adaptive Weighted Aggregation
Motivation. Our analysis of federated learning reveals staged characteristics in collaborative training. shown in 4 The two sides of the green line exhibit different behaviors, representing two phases:
Phase I: Early Training. The initial phase exhibits strong client alignment, where test accuracy improves rapidly through collective gradient coherence. However, while most clients converge to beneficial update directions, the clients in the SYN domain show lower cosine similarity with other clients. These divergent directions may skew the dominant direction, compromising the generalization
5

---

1300
111
200
(a) Cosine similarity between clients
(b) Loss and accuracy curve
(c) Variation of op and o
Figure 4: Illustration of Training Dynamics. The federated training has phases: early phase (left of green line) and later phase (right green line). The two phases exhibit distinctly different characteristics. Please refer to Sec. 3.3 for a detailed discussion.
ability of the global model. Our strategy therefore enforces directional consensus during this phase, selectively amplifying clients contributing coherent, generalizable updates.
Phase II: Later Training. As training progresses, client updates evolve toward orthogonality (near- zero cosine similarity), signaling exhausted consensus-driven gains. Here, the very underperformers that posed risks in Phase become valuable sources of exploratory gradient diversity. Their divergent directions now prevent premature convergence and enable fairer parameter distributions across heterogencous clients. We correspondingly shift weighting priorities to elevate these previously marginalized contributors, transforming gradient confficts into shimess-ennancing signals.
This phased paradigm fundamentally motivates our dual-term adaptive weighting framework. By dynamically rebalancing generalization and fairness priorities in response to emergent training signatures, we transcend the limitations of static aggregation schemes.
3.3.1 Consensus-Driven Generalization
The design of our generalization term stems from fundamental intricate and interdependent relation- ship between gradient alignment patterns and collective learning progress dynamics. By analyzing the first-order Taylor expansion of the global loss reduction:
A Ciotal
(C)
(w'
nd') - c (w'))
(g)
7
(8)
1=1
1=1
==1
(gi g/).
establish that clients exhibiting higher gradient coherence (larger E, (g) g/}) contribute more significantly to overall loss minimization. In practical training, g is equivalent to Aw", and we assign more weight those with greater sum of dot products with other clients. This pairwise dot product formulation serves two purposes: it robustly captures directional consensus across skewed distributions through gradient similarity metries, while simultancously suppressing clients with divergent updates that could destabilize the global model. Additionally, we employ momentum updates [25], with the momentum coefficient decaying using parameter 7ps which helps constrain the oscillations that typically occur as the system approaches convergence.
Σ (Aw(, Aw!) Aww Aw?)
Apin. = =(1 11--818 (T7)) 4- + B7v
(9)
(Aw), Aw!)
=h + pl
P
7/p
ΣK
where (Aw(, Aw!) represents the average value the dot products between clients during epoch 3.3.2 Diversity-Enhanced Fairness
A natural approach to achieving fairness, as defined in (1), is to reweight the aggregation process by assigning higher weights to devices with poor performance. We use the loss as a proxy for perfor- mance and assign higher weights to clients with higher loss values, applying the same momentum

---

update to the fairness score described previously.
C
4 C
Agi (1 - (1-87)) Aqh
+ 87
Σ, M
D
(10)
< 1-1 Avi. &i
Σ oj
To reduce the number of syper-parameters, adopt the same 8 as in Eq. (9. and 7i serves the same role as "p. This mechanism aims to shift the weights in favor of disadvantaged clients, thereby achieving more uniform performance while also accounting for domain diversity.
3.3.3 Phase-Adaptive Weight Synthesis
Our phased analysis reveals a critical duality in federated optimization objectives: the alignment of updates and the disparity in loss exhibit opposing dominance patterns across different training phases. To empirically validate this phase-dependent dichotomy, we quantify the client-wise standard deviations of generalization (o/)) and fairness (ot) weights throughout training. As shown in Fig. 4c of dominates in the early rounds, while of surpasses it in later stages.
Early Phase: of o high gradient alignment variability among clientsleads to prioritizing caasensus-drivens updates to maximize collective progress.
Later Phase: of >> of. the later-phase exhibits diminishing gradient coherence but increasing loss disparity, necessitating interventions that emphasize fairness.
This systematic inversion motivates adaptive weighting mechanism, which dynamically rebalances the two objectives based on their relative variability:
A of of
0
Pi of + +z
(11)
This veriance-regulated synthesis automatically shifts emphasis between consensus-seeking and disparity-reductin modes throughout the training dynamics.
3.4 Discussion and Limitation
Comparison with Analogous Methods. AFL 39,, Q-FFL [35 and FedCE 22 prioritize weak clients using single metrics (e.g. loss/accuracy), but simply increasing their weights is not appropriate. In contrast, our method first suppresses then amplifies updates for straggling clients, aligning with natural learning dynamics. This strategy maintains update consistency without sacrificing domain like FedL.F address update conflicts via gradient projection, trivial parameters still disrupt global optimization. Our parameter adjustment selectively discards non-critical updates to preserve essential ones, enhancing both generalization and fairness.
diversity. While prior works 44 4
Discussion on Parameter Adjustment. Our method addresses update conflicts in large-scale FL through selective parameter pruning. This strategy enables performance gains for conflicting clients with minimal self- aff-degradation, achieving collective enhancement. Concurrent gradient rescaling preserves original gradient norm magnitudes while amplifying critical parameters, effectively reducing parameter space conflicts and stabilizing multi-client collaboration.
Limitations. Our method employs parameter adjustment and adaptive weighting adjust model aggregation. However, setting the hyper-parameter e to excessively high values may cause instability, exhibits sensitivity selection. Our approach requires all participating clients to maintain identical network architecture specifications, which may limit the broader applicability of this method.
4
Experiments
We perform experiments on image classification tasks in various single-domain and cross-domain scenarios validate the superiority of our framework FedPW.
4.1 Experiment Setup
Datasets. Following
we evaluate our method on single-domain datasets Fashion-MMn
[54] Ci (28). Cifar 100, and cross-domain datasets Digits [29] and Office-Caltech (Μ].
7

---

Table 1: Comparison of Average Accuracy (and Standard Deviation) with baselines in single-domain scenarios.
FMNIST
CIFAR-11 10
CIFAR-100
Methods
Dir(0.1)
Par-1
Pat-2
Dir(0.1)
Pat-1
Pat-2
Die(O.1)
Pat-1
Pat-2
FedAvg
87.0(11.5)
82.3(14.2)
82,9(11.4)
68.1(15.0)
56.2(20.4)
68.7(19.3)
37.1(7.5)
20.5(15.7)
21.9(15.3)
q-FFL
86.1(9
84.0(13.5)
79.3(9.6)
67.8(14.1)
58.6(17.5)
68.8(18.1)
37.9(6.8)
23.1(13.9) 17.6(15.1)
AFL
87.0(9.5)
82.2(17.2)
83.7(11.8)
64.3(13.8
56.5(14.4)
65.6(14.0)
39.1(7.0)
17.5(16.6)
26.7(14.6)
Ditto
79.9(10.6)
75.5(214)
77.8(10.)
57.9(13.1)
45.3(11.3)
52.6(14.2)
27.9(7.4)
18.1(20.1)
11.2((1.4)
FedProx
81.9(10.0)
84.1(119))
84.348.8)
53.4(13.4)
57.3(11.))
56.8(12.2)
18.7(5.9)
20.3(14.7)
19.1(12.0)
FedFV
82.8(9.4)
88.5(11.7)
81.6(13.4)
68.1(13.3)
55.1(18.3)
69.4(18.9)
37.3(6.9
18.9(14.5)
21.9(13.7)
FedCKA
89.0(10.1)
79.3(188)
87.4(10.9)
67.3(4.3)
56.4(20.4)
69.3(14.4)
37.1(7.0)
19.6(1577
21.3(14.4)
FedSAC:
84.1(13.1)
74.5(23.6)
83.4(18.1)
58.7(14.3)
44.8(15.3)
55.8(15.3)
33.4(6.1)
25.4(14.)
7.5(3.1)
FedGCR
85.7(10.0)
83.2(13.5)
84.0(11.3)
66.6(13.8)
61.5(19.1)
65.5(16.9)
38.6(6.7)
20.6(14.5)
19.9(15.6)
FedHeal
85.7(11.3)
85.8(13.9)
84.0(11.))
71.0(14.4)
58.7(22.4)
66.8(14.1)
38.7(6.8)
19.6(14.)
22.8(14.3)
FedAA 86.8(7.7)
85,7(8.8)
87.9(7.0)
73.8(12.7)
73.2(10.4)
71.4(11.3)
38.1(7.0)
34,2(12.7) 27.4(13.2)
FedPW
88.2(7.1)
89.4(8.0)
90.2(6.6)
75.3(1..))
75.1(10.3)
76.39..9
41.2(66.6)
36.2(14.2)
38.4(12.6)
Table 2: Cross-domain comparison of Average Accuracy (AVG) and Standard Deviation (STD) with baseline.
Digiis
Office-Catech
Methods
MNIST
USPS
SVIN
SYN
AVG1
STD 4
Amazon
DSLR
Caltech
Webcam
AVG 1
STD 1
FedAvg
93.23
91.01
79.13
40.02
75.85
24.67
72.36
56.93
59.19
45.92
58.60
10.85
+APL
93.73
95.42
75.42
44.25
76.71
10.35
23.27
11.40
64.34
65.65
57.21
47.52
58.68 68 10.08
8:31 12.34
1F-FFL
93.89
77.93
76.78
10.35
12.19
64.75
52,60
51.33
$7.02 11.56
6.26 14.29
(FdHEAL
93.12
94.12
79.13
46.36
78.18 12.33
22.29
12:36
67.49
66.81
59,82
54.83
62.24 (3.6)
6.03 p4.22
4FedPW
94.28
93.62
80.76
49.43
79.52
21.00
68.64
65.95
59.76
58.62
63.24
4.83
FedProx
93.64
91.14
80.53
41,93
76.81
23.94
70.31
57.83
59.86
44.79
58.20
10.48
+AFL
93.72
95:23
75.44
43.15
76.89
10.05
24.22 10:25
67.18
63.52
59.65
53.08
60.86
6.03 14.46
FFL
94.05
93.49
75.73
44.36
76.91
10.30
23.31
62.27
73.62
$4.39
$5.46
61.44
13.24
8.84 11.04
:FeHHEAL
92.15
93.58
78.89
77.31
10.50
22.78
11.16
66.17
72.65
58.09
$6.95
15.25
7.37 pa:11
#FedPW
94.33
92.46
80.19
48.62
78.90
19.09
21.14
12:80
68.40
70.67
59,86
$8.94
64,47 19.95
5.94 p4.64
MOON
76.40
60.29
60.13
10.40
+AFL
93.14
95.12
74.68
44.48
76.86
10.46
23.46
66.70
68.20
61.54
54.50
62.74
12.01
6.18
FFL
92.31
94.51
75.98
43.67
76.62
10.22
23.47
(1:71
64,90
65.85
53.88
58.93
60.89
5.59 14.11
+FedHEAL
93.23
94.31
80.81
45.12
78.37
11.07
23:00
12:38
68.16
64.95
59.17
59.51
62.95
y2.02
4.37 16.00
iFedPW
93.71
94.61
81.53
79.53
21.68 (3.50
66.73
65.84
59.47
60.34
yo.rn
3.72 (6.c8
FodDyo
94:15
94.82
80.29
40.48
77:44
25.53
70.11
61.56
59.78
48.15
59.90
9.04
+AFL
96.14
70.95
11775
10.12
50.99
10.10
8.24 10.30
FFL
94.71
94.26
75.33
42.79
76.77
Jo.a7
24.39
11.14
62.99
66.44
55,76
55.36
60.14
10.24
5.47 19.97
+FedHEAL
94.61
95.72
79.94
43.72
78.50
24.27
67.55
60.53
58.86
$3.38
60.08
5.84 13:20
iFedPW
94.61
95.72
80.06
46.84
79.31
11.87
22.79
66.93
63.27
58.83
$4.67
60.93
5.33
1a.71
Independeni Methuds
Dito
93.62
91.58
79.55
40.65
76.42
24.65
56.94
69.
56.73
56.82
61.04
6.42
FedFV
94.70
76.77
25.45
61.83
71.72
58.92
61.96
7.15
FedCKA
91.53
94.72
78.52
46.18
78.25
22.48
67.24
64.03
59.76
49.10
60.03
7.91
FedGCR
93.14
95.12
78.26
44.48
77.75
23.42
66.70
68.20
62.54
$1.50
62.99
6.14
FedSAC
92.16
93.75
78.61
41.67
76.55
24.22
52.39
51.68
55.37
44.16
50.90
4.77
FedAA
92.91
92.28
78.57
47.16
77.73
21.43
68.71
56.83
56.02
61.11
5.92
FedPW
94.28
93.62
80.76
79.52
21.00
68.64
63.95
58.62
63.24
4.83
Data Heterogeneity. To simulate heterogeneous clients in FL, we consider three scenarios: (1) Dir(a): We simulate m clients in Dirichlet heterogeneous partition. The smaller a is, the more imbalanced the local distribution is. (2) Pat-1: It constructs difficult data-island scenario where each client only has data from one class. (3) Pat-2: We follow FedAvg to build pathological non-III data where each client has data from two classes.
Model. For the single-domain scenario, we conduct experiments with CNN(two convolutional layers) [53]. For cross-domain scenarios, we use ResNet- 10 [14] whose feature vector dimension is 512. Note that all methods use the same network architecture for fair comparison across different tasks.
Counterparts. We compare our method with FedAvg 38 and fairness-focuss FL approaches: AFL [39 , F-F [35), and FedHEAL (both integrable). For non-integrable frameworks like Ditto (3). FedFV (53], 4 FedSAC[[36] and FedAA [3). we perform full end-to-end benchmarking.
This ensures comprehensive evaluation across all baseline categories.
3 43 in the single-domain setting, we employ 100 clients for 3,000 communication epochs, where all federated learning methods exhibit minimal or accuracy
Implement Details. Following
8

---

improvement beyond this point. Each epoch involves 10% client participation. We use the SGD optimizer with learning rate of 0.1 and a batch size of 50. For the cross-domain setting, we allocate 20 clients per task and equal clients per domain, with clients randomly assigned to domains. The training runs for 200 communication epochs with T local updates per round. Fach epoch involves all clients. The SGD uses learning rate of 0.001, and momentum is 0.9. The batch sizes are 64 for Digits and 16 for Office-Caltech. We fix the random seed to ensure reproduction and conduct experiments the NVIDIA 3090Ti. The hyperparameter settings are detailed in Sec. 4.3
Evaluation Metric. Following 34 38), Top-1 accuracy is adopted for model performance evaluation. For the single-domain setting, we use the standard deviation of accuracy across clients, while for the cross-domain setting, we use the standard deviation across domains for fair evaluation. We conduct experiments three times and utilize the accuracy of the last five epochs as the final performance.
4.2 Comparison to State-of-the-Arts
We benchmark FedPW against contemporary approaches addressing Performance Fairness in FL, with comprehensive results presented Tab. and Sec. [.1 Our method establishes new state-of-the- art performance, achieving superior mean accuracy while maintaining the lowest standard deviation across several scenarios. The convergence analysis in Fig. Fig./5further demonstrates FedPW's accelerated training dynamics compared to existing methods and several key observations summarized: Cifar10
Cifar100
Digits
Office-Caltech
Fadkig Dimo
Q.FFL
AFL
FedCKA FedSAC
FeaGCR
FddPW
Coemsnniaelenn Epoch
Conmuniaation Epoch
Figure 5: Comparison of convergence of average accuracy with counterparts. Please see details in Sec. 4.2
FedPW achieves an optimal trade-off between overall performance and fairness. Through the synergistic collaboration of the PA and AWA modules, FedPW simultaneously promotes fairness while improving the performance of the global model.
Existing fairness-oriented methods risk degrading the accuracy of global model or advantaged clients. For instance, q-FFL and AFL exhibit performance degradation compared to FedAvg Cifar- 10, and similar phenomena occur with SVHN in the Digits benchmark.
FedPW safeguards performance for disadvantaged parties. Underperforming domains like SYN in Digits and Webcam in Office-Caltech achieve improvements under FedPW's framework.
4.3 Diagnostic Experiments
Compatibility Study. To validate the compatibility of FedPW, we compared the results of several indely-adopted FI. methods, FedAvg The results are shown Sec. 4.
338 FedProx
34.
FedDynn
11, without and with FedPW.
Table 3: Ablation study multiple datasets. Please refer to Sec.
4.3
for detailed discussion.
Setting
FMNIST
CIFAR-10
CIFAR-100
Digits
Office-Caltech
PA
AWA
AVG f
STD 4
AVG
STD 4
AVG
STD
AVG f
STD +
AVG
STD |
87.03
11.21
68.13
15.02
37.12
7.48
75.85
24.67
58,60
10.85
88,05
10.34
72.88
13.94
38.64
7.81
77.85
23.62
60.65
9.92
87.34
8.62
72.12
12.84
40.01
7.27
78.58
22.08
62.39
5.05
88.22
7.13
75.28
11.10
41.22
6.57
79.52
21.00
63.24
4.83
Ablation Study. We conducted an ablation study to analyze the contributions of Parameter Adjustment (PA) and Adaptive Weighted Aggregation (AWA) components, as summarized Tab. 3 Our findings indicate that each module positively contributes to performance, with optimal results achieved through their combination.
Table 4: AVG(STD) under different number of
clients Digits.
Client scales
Methods
20
60
100
FedAvg 75.9(24.7) 86.2(16.7) 87.1(16.0)
FedPW
79.5(21.0) 88.9(9.8) 91.3(7.8)

---

Sepper-parameter Analysis. We systematically investigate the impact of two critical hyper- parameters: the client selection rate (Eq. (4) and the momentum coefficient / (Eq. ( Here, is solely used for momentum updates to mitigate drastic fluctuations during model training. Fig. shows that | has a limited impact on model performance, though the error bars indicate that increasing leads to more stable accuracy. The optimal values are set as defaults for subsequent experiments.
2110 Mase
4TD Bardor
STONJ
16
AMRE
AND Jeckea
14
12.4
000
04
03
855
60.
(a) Cifarl0 for
(b) Digits for 8
(c) Cifar for
(d) Digits for c
Figure 6: Hyper-parameter study with variant (Eq. 99 and variant e (Eq.
See details in Sec. 4.3
Conclusion
In this paper, we explore the fairness challenges arising from domain skew in heterogeneous federated learning. We propose simple yet effective federated learning algorithm, FedPW, to address two critical issues: Parameter Redundancy and Persistent Favoritism. Specifically, we utilize gradient information from model training to selectively discard and reinforce parameters. Furthermore, by leveraging training dynamics across epochs, our method achieves adaptive weighted aggregation. The effectiveness of FedPW has been extensively validated against several popular methods across various classification tasks. We hope that this work will serve foundation for future research.
Acknowledgement
This work is supported by National Natural Science Foundation of China under Grant (62361166629) 623B2080, 62501428), the Major Project of Science and Technology Innovation of Hubei Province (2024BCA003, 2025BEA002), and the Innovative Research Group Project of Hubei Province under Grants 2024AFA017. The supercomputing system at the Supercomputing Center of Wuhan University supported the numerical calculations in this paper.
References
[1] Durmus Alp Emre Acar, Yue Zhao, Ramon Matas, Matthew Mattina, Paul Whatmough, and Venkatesh Saligrama. Federated learning based on dynamic regularization. In ICLR, 2021.
[2] Tian Bowen, Lai Songning, Wu Jiemin, Shuai Zhihao, Ge Shiming, and Yue Yutao. Beyond task vectors: Selective task arithmetic based on importance metrics. arXiv preprint arXiv:2411.1613,, 2024.
[3] Yuhang Chen, Wenke Huang Huang, and Mang Ye. Fair federated learning under domain skew with local consistency and domain diversity. In CVPR, 2024.
[4] Shu-Ling Cheng, Chin- Yuan Yeh, Ting-An Chen, Eliana Pastor, and Ming-Syan Chen. Fedger: Achieving performance and fairness for federated learning with distinct client types via group customization and reweighting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 11 11498-11 1506, 2024.
[5] Yae Jee Cho, Samarth Gupta, Gauri Joshi, and Osman Yağan. Bandit-based communication- efficient client selection strategies for federated learning. In 2020 54th Asilomar Conference on Signals, Systems, and Computers, pages 1066-1069. IEEE, 2020.
[6] Sen Cui, Weishen Pan, Jian Liang, Changshui Zhang, and Fei Wang. Addressing algorithmic disparity and performance inconsistency in federated learning. In NewrIPS, pages 26091-26102, 2021.
[7] Pala Tej Deep, Rishabh Bhardwaj. and Soujanya Poria. Della-merging: Reducing interference in model merging through magnitude-based sampling, arXiv preprint arXiv:2406.11677 2024.
10

---

[8] Guodong Du, Junlin Lee, Jing Li, Runhua Jiang, Yifei Guo, Shuyang Yu, Hanting Liu, Sim Kuan Goh, Ho-Kin Tang, Daojing He, al. Parameter competition balancing for model merging. arXiv preprini arXiv:2410.02336, 2024.
[9] Yahya H Ezzeldin, Shen Yan, Chaoyang He, Emilio Ferrara, and A Salman Avestimehr. Fairfed: Enabling group fairness in federated learning. In AAAI, pages 7494-7502, 2023.
[10] Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. arXiv preprini /rX3vv/803..om 03635, 2018.
[11] Boqing Gong, Yuan Shi, Fei Sha, and Kristen Grauman. Geodesic flow kernel for unsupervised domain adaptation. In CVPR, pages 2066-2073. 2012.
[12] Gregory Griffin, Alex Holub, and Pietro Perona. Caltech-256 object category dataset. 2007.
[13] Jialuo He, Wei Chen, and Xiaojin Zhang. Fedaa: A reinforcement learning perspective on adaptive aggregation for fair and robust federated learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 17085-17093, 2025.
[14] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pages 770-778, 2016.
[15] Chenyu Huang, Peng Ye, Tao Chen, Tong He, Xiangyu Yue, and Wanli Ouyang. Emr- Emr-merging: Tuning-free geghpperformacc model merging. arXiv preprint arXiv:2405.17461 2024.
[16] Tiansheng Huang, Weiwei Lin, Wentai Wu, Ligang He, Keqin Li, and Albert Y Zomaya. bffistnnyy-boosting client selection scheme for federated learning with fairesss guarantee. IEEE Transactions on Parallel and Distributed Systems, 32(7): 2020.
[17] Wenke Huang, Mang Ye, and Bo Du Learn from others and be yourself in heterogeneous federated learning. In CVPR, 2022.
[18] Wenke Huang, Mang Ye, Zekun Shi. He Li, and Bo Du. Rethinking federated learning with domain shift: prototype view. In CVPR, pages 16312-16322, 2023.
[19) Wenke Huang, Mang Ye, Zekun Shi, Guancheng Wan, He Li, Bo Du, and Qiang Yang. A federated learning for generalization, robustness, fairness: A survey and benchmark. arXiv, 2023.
[20] Jonathan J. Hull. A database for handwritten text recognition research. IEEE PAMI, pages 550-554, 1994.
[21] Yongzhe Jia, Xuyun Zhang, Hongsheng Hu, Kim-Kwang Raymond Choo, Lianyong Qi, Xiao- long Xu, Amin Beheshti, and Wanchun Dou. Dapperfl: Domain adaptive federated learning with model fusion pruning for edge devices. arXiv preprint 24X1v:2412.05823,, 2024.
[22] Mcirui Jiang, Holger R Roth, Wenqi Li, Dong Yang. Can Zhao, Vishwesh Nath, Daguang Xu, Qi Dou, and Ziyue Xu. Fair federated medical image segmentation via client contribution estimation. In CVPR, 2023.
[23] Xuefeng Jiang, Sheng Sun, Yuwei Wang, and Min Liu. Towards federated learning against noisy labels via local self-regularization. In CIKM, pages 862-873, 2022.
[24] Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank J Reddi, Sebastian U Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for on-device federated learning. In ICML,, 2020.
[25] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
[26] Jakub Konečný, H Brendan McMahan, Daniel Ramage, and Peter Richtárík, Federated optimiza- tion: Distributed machine learning for on-device intelligence. arXiv preprint arXiv: 1610.0227, 2016.
11

---

[27] Zhiqiang Kou, Jing Wang, Jiawei Tang, Yuheng Jia, Boyu Shi, and Xin Geng, Exploiting multi-label correlation in label distribution learning. In Kate Larson, editor, Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24, pages 4334-4334. International Joint Conferences on Artificial Intelligence Organization, 8 2024. Main Track.
[28] A. Krizhevsky and G. Hinton. Learning multiple layers features from tiny images. Master's thesis, Department of Computer Science, University of Toronto, 2009.
[29] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, pages 2278-2324, 1998.
[30] Qinbin Li, Yiqun Diao, Quan Chen, and Bingsheng He. Federated learning on non-iid data silos: An experimental study. ICDE, pages 965-978, 2022.
[31] Qinbin Li, Bingsheng He, and Dawn Song. Model-contrastive federated learning. In CVPR, pages 10713-10722, 2021.
[32] Tian Li, Shengyuan Hu, Ahmad Beirami, and Virginia Smith. Federated multi-task learning for competing constraints. arXiv preprint arXiv:2012.04221, 2020.
[33] Tian Li, Shengyuan Hu, Ahmad Beirami, and Virginia Smith. Ditto: Fair and robust federated learning through personalization. In ICML, pages 6357-6368, 2021.
[34] Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. In MLSys, 2020.
[35] Tian Li, Maziar Sanjabi, Ahmad Beirami, and Virginia Smith. Fair resource allocation in federated learning, In ICLR, 2020.
[36] Quande Liu, Cheng Chen, Jing Qin, Qi Dou, and Pheng-Amn Heng. Feddg: Federated domain generalization on medical image segmentation via episodic learning in continuous frequency space. In CVPR, pages 1013-1023, 2021.
[37] Lingjuan Lyu, Xinyi Xu, Qian Wang, and Han Yu. Collaborative fairness in federated learning. Federated Learning: Privacy and Incentive, pages 189-204, 2020.
[38] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. wawpiiiccaaooofffffiiiiee learning of deep networks from decentralized data. In AISTATS, pages 1273-1282, 2017.
[39] Mehryar Mohri, Gary Sivek, and Ananda Theertha Suresh. Agnostic federated learning. In ICMI., pages 4615-4625. PMLR, 2019.
[40] Xutong Mu, Yulong Shen, Ke Cheng, Xueli Geng, Jiaxuan Fu, Tao Zhang, and Zhiwei Zhang. Fedproc: Prototypical contrastive federated learning on non-iid data. arXiv preprint arXiv:2109. 12273, 2021.
[41] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. In NeurIPS Workshop. 2011.
[42] Takayuki Nishio and Ryo Yonetani. Client selection for federated learning with heterogeneous resources in mobile edge. In ICC 2019-2019 IEEE international conference on communications (ICC), pages 1-7, JEEE, 2019.
[43] Zibin Pan, Chi Li, Fangchen Yu, Shuyi Wang, Haijin Wang. Xiaoying Tang, and Junhua Zhao. Fedlf: Layer-wise fair federated learning. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pages 14527-14535, 2024.
[44] Zibin Pan, Shuyi Wang, Chi Li, Haijin Wang, Xiaoying Tang, and Junhua Zhao. Fedmdfg: Federated learning with multi-gradient descent and fair guidance. In Proceedings of the AAAI Conference Artificial Intelligence, volume 37, pages 9364-9371, 2023.
12

---

[45] Prasun Roy, Subhankar Ghosh, Saumik Bhattacharya, and Umapada Pal. Effects of degradations on deep neural network architectures. arXiv preprint arXi::1807.10008 2018.
[46] Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. Adapting visual category models to new domains. In ECCV, pages 213-226, 2010.
[47] Yuxin Shi, Han Yu, and Cyril Leung. Towards fairness-aware federated learning. IEEE TNNLS, 2023.
[48] Yaqi Sun, Shijing Si, Jianzong Wang, Yuhan Dong, Zhitao Zhu, and Jing Xiao. fair federated learning framework with reinforcement learning. In 2022 International Joint Conference on Neural Networks (IJCNN), pages 1-8. IEEE, 2022.
[49] Canh T. Dinh, Nguyen Tran, and Josh Nguyen. Personalized federated learning with moreau envelopes. In NeurlPS, pages 21405, 2020.
[50] Yue Tan, Guodong Long, Lu Liu, Tianyi Zhou, Qinghua Lu, Jing Jiang, and Chengqi Zhang. Fedproto: Federated prototype learning across heterogeneous clients. In AAAI, 2022.
[51] Yuandong Tian, Tina Jiang, Qucheng Gong, and Ari Morcos. Luck matters: Understanding training dynamics of deep relu networks. arXiv preprint arXiv: 13405, 2019.
[52] Ke Wang. Nikolaos Dimitriadis, Guillermo Ortiz-Jimenez, François Fleuret, and Pascal Frossard. Localizing task information for improved model merging and compression. arXiv preprint arXiv:2405.07813 2024.
[53] Zheng Wang, Xiaoliang Fan, Jianzhong Qi, Chenglu Wen, Cheng Wang, and Rongshan Yu. Federated learming with fair averaging. In IJCAI, 2021.
[54] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist A novel image dataset for benchmarking machine learning algorithms. arXiv preprint trXiv:1708.0775 2017.
[55] Xinyi Xu, Lingjuan Lyu, Xingjun Ma, Chenglin Miao, Chuan Sheng Foo, and Bryan Kian Hsiang Low. Gradient driven rewards to guarantee fairness in collaborative machine learning. In NeurlPS, volume 34, pages 16104-16117,002
[56] Kunda Yan, Sen Cui, Abudukelimu Wuerkaixi, Jingfeng Zhang, Bo Han, Gang Niu, Masashi Sugiyama, and Changshui Zhang. Balancing similarity and complementarity for federated learning. 2024.
[57] Qiang Yang. Yang Liu, Tianjian Chen, and Yongxin Tong. Federated machine learning: Concept and applications. ACM TIST, pages 1-19, 2019.
[58] Mang Ye, Xiuwen Fang, Bo Du, Pong C Yuen, and Dacheng Tao. Heterogeneous federated learning: State-of-the-arr and research challenges. CSUR, 2023.
[59] Yun Ye, Ganmei You, Jong-Kas Kae Fwu, Xia Zhu, Qing Yang, and Yuan Zhu. Channel pruning via optimal thresholding. In Neural Information Processing: 27th International Conference, ICONIP 2020, Bangkok, Thailand, November 18-22, 2020, Proceedings, Part V 27, pages 508-516. Springer, 2020.
[60] Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. Language models are super mario: Absorbing abilities from homologous models as free lunch. In Forty-first International Conference on Machine Learning, 2024.
[61] Yuchen Zeng, Hongxu Chen, and Kangwook Lee. Improving fairness via federated learning. arXiv preprint arXiv:21 15545, 2021.
[62] Lin Zhang, Yong Luo, Yan Bai, Bo Du, and Ling- Yu Duan. Federated learning for non-iid data via unified feature learning and optimization objective alignment. In ICCV, pages 4420 4428,,42, 2021.
[63] Ruipeng Zhang, Qinwei Xu, Jiangchao Yao, Ya Zhang, Qi Tian, and Yanfeng Wang. Federated domain generalization with generalization adjustment. In CVPR, pages 3954-3963 June 2023.
[64] Zirui Zhou, Lingyang Chu, Changxin Liu, Lanjun Wang, Jian Pei, and Yong Zhang. Towards fair federated learning. In ACM SIGKDD, pages 4100-4101, 2021.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction clearly state the main claims of our work, including overview of FedPW. In addition, the main contributions and existing limitations are logically outlined in Sec.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Justification: In the dedicated section (Sec. 3.4p. analyze the limitations of FedPW, including its sensitivity to hyper-paraman parameter. Moreover, all clients should maintain identical network architecture specifications, which may limit the broader applicability.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
14

---

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: This paper does not include formal theoretical results or theorems that require specific assumptions and complete proofs. The methodology presented in Sec. B empirical and algorithmic, accompanied by the corresponding descriptions and formulas.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
This paper provides sufficient experimental details reproduce the results, including model backbone types, datasets with partitioning methods, the number of tasks, related training parrmpaaameeerss specific evaluation metrics and exact baseline settings.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
15

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
Code is accessible in this paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurIPS code and data submission guidelines (https //nips.ccpput wwtpcccccppccccceee casollcc/giieee wwwedeeee/ooebbbmeeeeeee
ssionoolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: We provide all necessary training and evaluation details, including model, datasets. dataset splits per task, batch size, and methods to control the non-lll IID level in Sec.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Please refer Sec. 2
Guidelines:
16

---

The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report I-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out of range (c.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please refer Sec.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: Our work addresses the issue of catastrophic performance degradation caused by traditional fairness methods and has been validated across diverse data scenarios. The study does not involve human subjects or sensitive personal data, presents no adverse real-workd impacts, and fully complies with the NeurIPS ethical guidelines.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
17

---

Answer:
[NA]
Justification:
There is no societal impact of the work performed.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our work does not release any new pretrained models, image generators, or datasets that pose risk of misuse. We conducted the evaluation using publicly available datasets and backbones. The additional settings required in the experiment are commonly used in current methods, and there was misuse threat.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
All the external assets used this paper, including the ResNet10 backbone and all baselines, as well as the Fasion-MNIST, CIFAR-10, CIFAR-100, Digits and Office- Caltech datasets, have been correctly eited and accompanied by the corresponding references. Their licenses respected and unauthorized or crawled content has been used.
Justification:
18

---

Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
19

---

Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality the research, declaration is not required.
Answer: [NA]
Justification: None of the core methods, including any important, original, non-standard components, rely on LLMs. We also do not use LLMs to generate data, experimental results, similar
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM for what should or should not be described.
20

---

A
Method Algorithm
Algorithm 1: FedPW
Input:
Communication rounds T, local epochs &, number of participants K, k participant private data DR. private model wg.
Output: The final global model w²
Server: initialize the global model w
for ( = 0,
-1 do
Client:
for 1, ... K in parallel do w/ f- W
for c 1, 2, do
w + w n /VC Dx)
Auf & w/. W
Server:
AWA FedPW(Aur)
for 1, ..., G do
L W W + AW
FedPW( (ww') ): for = 1,2..... K do
/+ Adaptive Weighting /
Pr. Api, (p) 1-1 App
in Eq.
Ge: Aqk - (g(-1 Aq
3) Eq. 10
Af < Pi) in Eq. 11
/* Parameter Adjustment */
for 1....... do
Mi,i + Eq. 5
Awk, = AWA. M
af (Awg, :) in Eq.
AW; a ΣK Aw
return AW!
Parameter Adjustment. This module refines client updates through two-stage process. In the first stage, redundant parameters are pruned by computing client-specific mask rates rf. from inverse training losses and applying binary masks M in Eq. 5 Parameters below the adaptive threshold Th are removed, eliminating about (1 rA: x 100% aggregation noise. In the second stage, consensus- based rescaling applied using Eq. ( masked updates are normalized, consistent parameters are identified via cross-client standard deviation, and emphasized by scaling factors af to preserve gradient magnitude while reinforcing aligned updates that support stable convergence.
least significant parameters and reducing
Adaptive Weighting. This module balances generalization and faimess across training. In each round, generalization weights p are computed via gradient alignment (Eq. (9)). while fairness weights of are derived from training losses (Eq. (10)). The two components are then adaptively combined using the variance-based rule in Eq. (TT), shifting from alignment-driven weighting in early
rounds to fairness-oriented weighting later, guided by the relative variability of the two distributions.
B Details of Experiments
43 evaluate the efficacy of our method on single-domain datasets Fashion-Mnist, Cifar 10, Cifar 100, and cross-domain datasets Digits and Office-caltech.
Datasets. Following 18 21
Fasion-MNIST has 60k train and 10k test examples from 10 classes.
Cifar10 28 contains 50k, 104 images for training, validation. Each image 32 32 size from 10 different classes, e.g., airplanes, cars, and birds.
Cifar 28 contains 50% and 10k images with 32 x 32 for 100 classes.
Digits 29 20 45 includes four domains: MNIST(M), USPS (U), SVHN (SV) and SYN
(SY) with 10 cat- 427 egories (digit number from 0 to 9).
21

---

Office-Caltech [ consists four domains: Caltech (C), Amazon (A), Webcam (W) and DSLR (D), which is formed of overlapping classes between Office31 46 and 255 [2].
Hyper-parameter Study Our method involves only two hyperparameters: the mask ratio and the momentum coefficient The mask ratio governs the average masking level in the PA component, where smaller values make FedPW resemble FedAvg and reduce potential gains, while excessively large values may cause instability due to insufficient trainable parameters. The coefficient 3 controls the smoothness of adaptive updates in the AWA component; extremely small values degrade it to non-momentum variant, while overly large values may delay the system's responsiveness training dynamics. The following tables present comprchensive hyperparameter evaluation results.
Table 5: AVG(STD) under varying 3 across different datasets.
3 FMNIST
0.1
0.3
0.5
0.7
0.9
85.1(8.3)
87.7(7.2)
88.2(7.1)
87.2(7.8)
86.7(6.8)
CIFAR-IO
74.5(10.9)
74.6(11.4)
75.3(11.1)
75.0(11.3)
74.9(11.6)
CIFAR-100
39.6(6.8)
41.2(7.0)
41.2(6.6)
39.8(6.3)
39.3(6.6)
Digits
78.8(21.7)
79.1(21.4)
79.5(21.0)
78.5(21.1)
77.9(21.))
Office-Caltech
62.7(6.3)
61.9(4.8)
63.2(4.6)
62.7(5.9)
62.3(5.8)
Table 6: AVG(STD) under varying across different datasets.
0.1
0.2
0.3
0.4
0.5
FMNIST
86.1(7.1)
87.6(7.4)
88.2(7.1)
86.3(6.9)
86.1(6.7)
CIFAR-10
74.6(10.9)
75.3(10.9)
75.3(11.1)
74.5(12.1)
74.1(11.3)
CIFAR-100
40.2(6.8)
41.0(6.6)
41.2(6.6)
39.3(6.9)
38.6(7.4)
Digits
78.8(21.3)
79.4(21.5)
79.5(21.0)
79.1(21.5)
78.4(22.1)
Office-Caltech
62.1(5.1)
62.3(5.1)
63.2(4.6)
60.1(3.8)
58.9(4.2)
22


Reviews:
[{"review_id": "Guft9oIiC8", "reviewer": "Reviewer_NSd9", "summary": "The paper aims to overcome the bias that FL based models face by proposing FedPW, a fairness-aware federated learning method that addresses two challenges: (1) parameter redundancy, where unimportant parameter updates interfere with meaningful gradients, and (2) persistent client protection, where prioritizing weak clients across all rounds leads to convergence degradation. FedPW performs parameter masking/resizing based on local update magnitudes and applies a phase-adaptive client weighting scheme based on gradient alignment and local loss. Empirical evaluations demonstrate improved performance-fairness trade-offs.", "questions": "1) How do you distinguish between truly redundant updates and temporally small ones due to gradient noise or slow convergence?\n\n2) Can you derive any convergence bound for the masked and rescaled FedAvg variant? Under what assumptions on gradient noise does your procedure remain unbiased?\n\n3) How sensitive is the scheduler to the batch size, number of clients, or variance scaling?\n\n4) Are other fairness metrics applicable in this work such as max-loss, minimum accuracy, or worst-group accuracy across clients?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "[S1] The paper relies on a known observation that fairness mechanisms may themselves cause unfairness or performance degradation, particularly due to persistent overweighting of weak clients and noisy parameter updates. This is in line with the theory of importance sampling that while importance weighting impacts machine learning models early in training, its effect diminishes over successive training iterations.\n\n[S2] The proposition of a two-phased training heuristic (early: consensus-driven; late: fairness-driven), instantiated using statistical indicators from client gradients and loss, is an interesting bias-mitigation approach.\n\n[S3] The evaluation covers both single-domain and cross-domain FL with baselines and ablations.\n\n[W1] The central idea of parameter redundancy appears to be both under-theorized and lacks any formalization; The redundancy criterion is based entirely on local update magnitude, assuming small updates are unimportant. Is this assumption valid? This needs clarity because of a few points: Small updates could represent delayed or convergent directions. Update magnitude is highly sensitive to batch stochasticity and learning rate. Parameters interacting in high-dimensional manifolds (e.g., batchnorm scales) often have noisy gradients regardless of their importance. For this, I have a question: How do you distinguish between truly redundant updates and temporally small ones due to gradient noise or slow convergence?\n\n[W2] The masking and rescaling steps (Eqs. 5–7) are the mathematical core of the method, but they are treated entirely as black-box heuristics. There is no attempt to analyze: (1) How masking affects the optimization trajectory (e.g., bias, variance, directionality). For this, I have a question: Can you derive any convergence bound for the masked and rescaled FedAvg variant? Under what assumptions on gradient noise does your procedure remain unbiased?\n\n[W3] The dual-phase aggregation strategy (Eq. 11) is intuitive but lacks robustness or theoretical validation. The scheduler uses the relative standard deviations $\\sigma_p$ and $\\sigma_q$ of gradient consensus and loss weights, respectively. This implicitly assumes: (1) gradient directions align early in training ($\\sigma_p$ high), and (2) loss disparities dominate later ($\\sigma_q$ high). However, in practice, these signals can be noisy, correlated, or reversed depending on batch stochasticity, task diversity, or optimizer state. For this, my question is: How sensitive is the scheduler to the batch size, number of clients, or variance scaling?\n\n[W4] The paper defines performance fairness as minimizing the variance of client losses (Definition 3.1). While this is common in FL literature, it may have limitations: variance is symmetric as it penalizes both over- and underperformance. But fairness typically aims to improve the worst-off groups. For this, my question is: are other fairness metrics applicable in this work such as max-loss, minimum accuracy, or worst-group accuracy across clients?\n\n[W5] Several parts of the paper suffer from imprecise explanations or inconsistencies. For instance, in Figure 2 (parameter pruning analysis) The x-axis is not defined. Does \"mask rate\" refer to percentage of parameters, magnitude threshold, or layer-wise pruning?", "was_revised": true, "final_justification": "Thanks for answering my questions. I have raised my score keeping in mind that theoretical formalizations still need some work.", "rebuttal": "Dear Reviewer NSd9:\n\nWe sincerely appreciate your time and effort in reviewing our paper. We hope that our responses below will address your concerns and further affirm the quality of our research.\n\n### Weaknesses & Questions\n\n**W1 & Q1: How to distinguish truly redundant parameter updates from temporarily small updates caused by noise or slow convergence?**\n\nA1: Thank you for your insights and concerns. Neural networks are known to contain subnetworks that are functionally equivalent to the full model, implying the presence of redundant parameters. In our work, we treat parameters with consistently small updates as redundant, since their contribution to loss reduction is minimal. This perspective has been widely supported in prior studies [1, 2]. To address the concern that some parameters may have small updates due to noise or slow convergence, we clarify the following:\n\n(1) **Temporarily small updates:** Some parameters may have small updates due to gradient noise, stochasticity, or slow convergence. These temporarily small magnitudes do not reflect true redundancy. Such parameters are not persistently suppressed or discarded, so occasional masking of them does not cause significant negative impact.\n\n(2) **Persistent redundancy:** Truly redundant parameters tend to have consistently small updates over the long term. Our masking strategy captures this pattern and consistently discards such parameters.\n\n(3) **Long-term mask:** Our masking strategy operates continuously over the long term. This long-term perspective ensures that only persistently inactive parameters are consistently masked, while temporarily small updates may be masked occasionally but are not systematically suppressed.\n\nThis distinction between temporarily small updates and persistent redundancy allows us to identify and mask unimportant parameters in a robust manner.\n\n[1] Picking winning tickets before training by preserving gradient flow. In International Conference on Learning Representations (ICLR), 2021.\n\n[2] Gradient flow in sparse neural networks and how lottery tickets win. In Proceedings of the AAAI Conference on Artificial Intelligence, 2022.\n\n\n**W2 & Q2: Convergence bound and bias analysis.**\n\nA2: Our current masking strategy is not strictly unbiased, as it favors the preservation of high-magnitude parameters. However, the resulting bias remains limited. We decompose the global update as:\n$$\nw^{t+1} = w^t - \\eta \\cdot \\left( \\alpha^t \\odot \\sum_k \\lambda_k \\mathcal{M}_k g_k \\right),\n$$\n\nand define the total bias as:\n$$\nb := \\mathbb{E}[\\alpha^t \\odot \\sum_k \\lambda_k \\mathcal{M}\\_k g_k] - \\sum_k \\lambda_k g_k = b_{\\text{mask}} + b_{\\text{amp}},\n$$\nwith bounds:\n$$\n\\|b_{\\text{mask}}\\| \\le \\sqrt{c_{\\max}} \\cdot G, \\quad\n\\|b_{\\text{amp}}\\| \\le \\frac{m_d}{m_a} \\cdot G.\n$$\nAssuming $F(w)$ is $L$-smooth and $\\mu$-strongly convex, we obtain the convergence bound:\n$$\n\\mathbb{E}[F(w^T)] - F(w^*) \\le \\mathcal{O}\\left( \\frac{1}{T} + \\eta^2 G^2 \\left( \\sqrt{c_{\\max}} + \\frac{m_d}{m_a} \\right)^2 \\right).\n$$\nIn typical settings, $\\sqrt{c_{\\max}} < 1$, $\\frac{m_d}{m_a} \\ll 1$, and $\\eta G \\ll 1$, implying that the bias term $b$ remains significantly smaller than $1/T$. This guarantees convergence to a bounded neighborhood with controlled directional bias.\n\n\n**W3 & Q3: Dual-phase scheduler robustness and sensitive justification.**\n\nA3: To assess the scheduler’s sensitivity, we focus on **two key aspects**:  \n(1) **Stability**: whether the dual-phase phenomenon that the scheduler relies on is consistently observable and stable across settings, and  \n(2) **Effectiveness**: whether the scheduler can consistently achieve the intended objectives in both phases under different configurations.\n\nFor (1), we confirm the presence of the dual-phase behavior in all tested settings and report the proportion of training epochs in each phase, denoted as $E_{\\text{Phase I}}$ and $E_{\\text{Phase II}}$. In general, $E_{\\text{Phase II}}$ accounts for a larger portion of the training, indicating that the scheduler tends to prioritize fairness for most of the time.  \nFor (2), we quantify how concentrated the scheduler’s selection is in each phase. Specifically, we report $K \\cdot \\lambda_{\\text{top-25\\%-p}}$ and $K \\cdot \\lambda_{\\text{top-25\\%-q}}$, which represent the average weight assigned to the top 25% of clients (in terms of $p_k^t$ and $q_k^t$, respectively), scaled by the total number of clients $K$. Clients with higher $p_k$ (or $q_k$) values are those more beneficial for generalization (or fairness), and they are expected to be favored in the corresponding phase. A stable value of $K \\cdot \\lambda_{\\text{top-25\\%}}$ greater than 1 indicates that the scheduler consistently prioritizes these top clients as intended.\n\nThe following tables present our results. The first two rows present the accuracy and corresponding standard deviation (in parentheses) for our method and FedAvg. These results suggest that the scheduler is not overly sensitive to batch stochasticity or system heterogeneity, and its decision rule remains stable and reliable across different settings.\n\n*Table: Sensitive Analysis on Bacth Size in Digits*\n\n| | 4          | 16         | 64         | 256        |\n|-|-|-|-|-|\n| FedAvg                                | 88.7(11.9) | 86.7(14.9) | 75.9(24.7) | 69.8(25.5) |\n| FedPW                                 | 92.3(9.0)  | 89.4(7.3)  | 79.5(21.0) | 77.3(21.5) |\n| $E_{\\text{Phase I}}$                  | 0.47       | 0.33       | 0.36       | 0.35       |\n| $E_{\\text{Phase II}}$                 | 0.53       | 0.67       | 0.64       | 0.65       |\n| $K \\cdot \\lambda_{\\text{top-25\\%-p}}$ | 1.32       | 1.35       | 1.29       | 1.33       |\n| $K \\cdot \\lambda_{\\text{top-25\\%-q}}$ | 1.45       | 1.31       | 1.38       | 1.41       |\n\n*Table: Sensitive Analysis on Number of Clients in Digits*\n\n|                                       | 4          | 12         | 20         | 60         | 100        | 200        |\n|-| ---------- | ---------- | ---------- | ---------- | ---------- | ---------- |\n| FedAvg                                | 47.6(27.6) | 68.7(28.9) | 75.9(24.7) | 86.2(16.7) | 87.1(16.0) | 88.6(15.5) |\n| FedPW                                 | 56.3(25.4) | 74.9(24.6) | 79.5(21.0) | 88.9(9.8)  | 91.3(7.8)  | 93.4(7.1)  |\n| $E_{\\text{Phase I}}$                  | 0.41       | 0.35       | 0.36       | 0.39       | 0.44       | 0.42       |\n| $E_{\\text{Phase II}}$                 | 0.59       | 0.65       | 0.64       | 0.61       | 0.56       | 0.58       |\n| $K \\cdot \\lambda_{\\text{top-25\\%-p}}$ | 1.35       | 1.32       | 1.29       | 1.26       | 1.23       | 1.25       |\n| $K \\cdot \\lambda_{\\text{top-25\\%-q}}$ | 1.41       | 1.37       | 1.38       | 1.34       | 1.27       | 1.26       |\n\n*Table: Sensitive Analysis on Variance Scaling (Max Value of $\\sigma_p$ and $\\sigma_q$)*\n\n|                                       | FMNIST     | CIFAR-10   | CIFAR-100 | Digits     | Office-Caltech |\n| ------------------------------------- | ---------- | ---------- | --------- | ---------- | -------------- |\n| Max $\\sigma_p$                        | 0.035      | 0.112      | 0.259     | 0.045      | 0.031          |\n| Max $\\sigma_q$                        | 0.038      | 0.058      | 0.183     | 0.032      | 0.025          |\n| FedAvg                                | 87.0(11.5) | 68.1(15.0) | 37.1(7.5) | 75.9(24.7) | 58.6(10.9)     |\n| FedPW                                 | 88.2(7.1)  | 75.3(11.1) | 41.2(6.6) | 79.5(21.0) | 63.2(4.6)      |\n| $E_{\\text{Phase I}}$                  | 0.29       | 0.41       | 0.38      | 0.36       | 0.52           |\n| $E_{\\text{Phase II}}$                 | 0.71       | 0.59       | 0.62      | 0.64       | 0.48           |\n| $K \\cdot \\lambda_{\\text{top-25\\%-p}}$ | 1.25       | 1.35       | 1.53      | 1.29       | 1.41           |\n| $K \\cdot \\lambda_{\\text{top-25\\%-q}}$ | 1.21       | 1.29       | 1.48      | 1.38       | 1.22           |\n\n\n\n**W4 & Q4: Additional details on variance-based fairness metrics.**\n\nA4: As shown in the following table, We additionally evaluate fairness using max-loss, minimum accuracy, the group gap between top-10% and bottom-10% clients, Gini index, and Jain’s fairness index. These results show that our method substantially improves the performance of the most disadvantaged clients across diverse fairness definitions.\n\n*Table: Fairness Scores under Different Fairness Metrics*\n\n| Method  | max-loss $\\downarrow$ | min-acc $\\uparrow$ | group gap $\\downarrow$ | Gini index $\\downarrow$ | Jain's index $\\uparrow$ |\n| ------- | --------------------- | ------------------ | ---------------------- | ----------------------- | ----------------------- |\n| FedAvg  | 0.097                 | 39.39              | 47.23                  | 0.410                   | 0.936                   |\n| q-FFL   | 0.091                 | 42.66              | 46.57                  | 0.257                   | 0.942                   |\n| FedHeal | 0.094                 | 41.95              | 47.46                  | 0.213                   | 0.945                   |\n| FedPW   |**0.080**|**44.59**| **44.13**                  | **0.180**                   | **0.951**                   |\n\n**W5: Further clarification on Figure 2 and the x-axis.**\n\nA5: We apologize for your confusion caused. The x-axis labeled \"mask rate\" indeed refers to the percentage of parameters (ranked by magnitude) that are pruned. This definition aligns with the meaning of $r_k^t$ in our paper, representing the percentage of parameters being discarded. The left plot illustrates how performance varies across different domains as the mask ratio increases, where the inflection point indicates the maximum tolerable pruning ratio. The right plot displays the corresponding mask ratios and accuracy levels at which these inflection points occur across different datasets."}, {"review_id": "5s9JOeMc2N", "reviewer": "Reviewer_Jzhw", "summary": "This paper proposes FedPW, a novel federated learning framework aimed at improving both global performance and fairness across clients. To address parameter redundancy, it selectively discards insignificant local updates and amplifies consensus parameters that exhibit cross-client agreement. Additionally, it introduces an adaptive aggregation strategy that adjusts client weights dynamically based on the training phase: reinforcing update alignment in early training and emphasizing fairness in later stages. Extensive experiments on single- and cross-domain tasks demonstrate the effectiveness and compatibility of the proposed approach.", "questions": "1. The mask ratio $c$ and momentum coefficient $\\beta$ appear to be critical hyperparameters, yet their impact is only briefly discussed. A more thorough analysis of how these parameters influence performance would be valuable.\n\n2. Given that masking, normalization, and consensus computation require non-trivial operations, can the authors quantify the additional cost (e.g., time per round, memory usage)?\n3. Due to the trade-off relationship between the number of communication rounds and local epochs in federated learning systems, so how does the number of **local epochs** (or steps) affect FedPW’s performance?\n4. Some minor issues: Some methods in the related work section lack references, such as FedCurve and Ditto.", "limitations": "yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths：\n1. This paper presents a technically solid approach with a thoughtfully designed method that tackles practical issues in federated learning. Extensive experiments consistently demonstrate the effectiveness of the proposed method.\n\n2. This paper addresses an important and practical problem in federated learning: achieving fairness without compromising global performance.\n3. This paper is well-written and well-organized.\n\nWeaknesses:\n1.  This method introduces several hyperparameters, but the sensitivity analysis is relatively limited.\n\n2. This method relies on a common model architecture among clients, which could restrict its applicability in practical federated settings where model heterogeneity is present.\n3. The overall framework introduces multiple components, making the system relatively complex and potentially difficult to implement or reproduce. It may hinder adoption in practical federated settings, especially where computational or engineering resources are limited.", "was_revised": true, "final_justification": "While I appreciate the clarifications, I remain unconvinced by the claim that the proposed method is compatible with heterogeneous model structures. The current formulation appears to assume homogeneous architectures across clients, and the authors have not provided sufficient technical details or empirical support to justify generalization to heterogeneous settings.\n\nMoreover, I find the overall completeness of the work to be lacking. In particular, more extensive experimental analysis would be necessary to better support the claimed generality and effectiveness of the method.\n\nTherefore, while the paper presents some interesting ideas, I still have reservations about its readiness for publication at this stage.", "rebuttal": "Dear Reviewer Jzhw:\n\nThank you for the insightful comment. We appreciate your recognition of the innovative aspects of our motivation and the clarity of our writing. Thank you for your time and effort in reviewing our paper. We hope that our responses below will address your concerns and further affirm the quality of our research.\n\n### Weakness\n\n**W1 & Q1: Additional hyperparameter sensitivity analysis.**\n\nA1: Our method involves only two hyperparameters: the mask ratio $c$ and the momentum coefficient $\\beta$. The mask ratio $c$ governs the average masking level in the PA component, where smaller values make FedPW resemble FedAvg and reduce potential gains, while excessively large values may cause instability due to insufficient trainable parameters. The coefficient $\\beta$ controls the smoothness of adaptive updates in the AWA component; extremely small values degrade it to a non-momentum variant, while overly large values may delay the system’s responsiveness to training dynamics. To further examine sensitivity, we supplemented results across all datasets. Experiments indicate that the method performs robustly across a range of hyperparameter settings, showing stable behavior without strong sensitivity.\n\n*Table: Average Accuracy(and Standard Deviation) Under Varying $\\beta$*\n\n|                | 0.1        | 0.3        | 0.5        | 0.7        | 0.9        |\n| -------------- | ---------- | ---------- | ---------- | ---------- | ---------- |\n| FMINST         | 85.1(8.3)  | 87.7(7.2)  | 88.2(7.1)  | 87.2(7.8)  | 86.7(6.8)  |\n| CIFAR-10       | 74.5(10.9) | 74.6(11.4) | 75.3(11.1) | 75.0(11.6) | 74.9(11.6) |\n| CIFAR-100      | 39.6(6.8)  | 41.2(7.0)  | 41.2(6.6)  | 39.8(6.3)  | 39.3(6.6)  |\n| Digits         | 78.8(21.7) | 79.1(21.4) | 79.5(21.0) | 78.5(21.1) | 77.9(22.4) |\n| Office-Caltech | 62.7(6.3)  | 61.9(4.8)  | 63.2(4.6)  | 62.7(5.9)  | 62.3(5.8)  |\n\n*Table: Average Accuracy(and Standard Deviation) Under Varying $c$*\n\n|                | 0.1        | 0.2        | 0.3        | 0.4        | 0.5        |\n| -------------- | ---------- | ---------- | ---------- | ---------- | ---------- |\n| FMINST         | 86.1(7.1)  | 87.6(7.4)  | 88.2(7.1)  | 86.3(6.9)  | 86.1(6.7)  |\n| CIFAR-10       | 74.6(10.9) | 75.3(10.9) | 75.3(11.1) | 74.5(12.1) | 74.1(11.3) |\n| CIFAR-100      | 40.2(6.8)  | 41.0(6.6)  | 41.2(6.6)  | 39.3(6.9)  | 38.6(7.4)  |\n| Digits         | 78.8(21.3) | 79.4(21.5) | 79.5(21.0) | 79.1(21.5) | 78.4(22.1) |\n| Office-Caltech | 62.1(5.1)  | 62.3(5.1)  | 63.2(4.6)  | 60.1(3.8)  | 58.9(4.2)  |\n\n**W2: Discussion on the limitations of assuming homogeneous model architecture.**\n\nA2: Using homogeneous models is a common and convenient setting for fairness-oriented methods. Most fairness-driven FL works adopt this setting, and we follow the same convention. Nonetheless, our approach is compatible with extensions to heterogeneous models. Specifically, our aggregation strategies do not require model alignment, and the pruning mechanism can be adapted to partially shared components, such as aligned layers. Therefore, our framework retains flexibility for broader applicability in non-iid or partially heterogeneous environments.\n\n**W3 & Q2: Discussion on implementation difficulty and resource costs.**\n\nA3: Thank you for your insights and concerns. Our method introduces two components, both operating at the aggregation stage, which are not difficult to implement in practice. These steps are lightweight and can be incorporated as plugins without modifying the client-side logic or disrupting existing FL frameworks. We demonstrate its ease of integration via multiple plug-in experiments in Table 2 (Page 8). \n\nFrom a resource perspective, there is no extra client-side cost, either in computation or communication. The server-side overhead is also relatively small and acceptable. We measured the time and memory consumption on an NVIDIA 3090Ti, as shown in the following table. Additionally, we have supplemented a complexity analysis of our method, where the local epoch is denoted as $E$, batch size as $B$, number of parameters as $G$, number of clients as $K$, and dataset size as $D_k$. Although the complexity varies across these methods, their actual overhead in real-world scenarios is negligible and far smaller than the cost during training. \n\n*Table: Time (s) and Memory (GB) Costs on Digits Datasets*\n\n| Method  | Client Time | Server Time | Client Mem | Server Mem |\n| ------- | ----------- | ----------- | ---------- | ---------- |\n| FedAvg  | **3.58**        | **0.16**        | 0.042      | 0.022      |\n| q-FFL   | **3.58**       | **0.16**        | 0.042      | **0.021**      |\n| FedHeal | **3.58**        | 0.19        | 0.045      | 0.022      |\n| FedPW   | **3.58**        | 0.21        | **0.042**      | **0.021**      |\n\n_Table: Complexity Analysis._\n\n| Method  | Client Time Complexity     | Server Time Complexity                                  | Client Mem Complexity | Server Mem Complexity |\n| ------- | -------------------------- | ------------------------------------------------------- | --------------------- | --------------------- |\n| FedAvg  | $O(E \\cdot B \\cdot G)$     | $O(K \\cdot G)$                                          | $O(D_k + G)$          | $O(K \\cdot G)$        |\n| FedPW   | $O(E \\cdot B \\cdot G)$     | $O(K \\cdot G + K \\cdot c \\cdot \\log(G)  + K^2 \\cdot G)$ | $O(D_k + G)$          | $O(K \\cdot G)$        |\n\n### Question\n\n**Q3: Impact of local training epoch.**\n\nA6: To evaluate how FedPW performs under different numbers of local training steps, we conducted experiments on the Digits cross-domain benchmark with local epochs varying from 5 to 20. The results, shown in the appended table, demonstrate that FedPW consistently outperforms baseline methods across all configurations. This suggests that our method maintains strong robustness to local training length.\n\n*Table: Sensitive Analysis on Local Epoch in Digits*\n\n| local epoch | 5          | 10         | 15         | 20         |\n| ----------- | ---------- | ---------- | ---------- | ---------- |\n| FedAvg      | 72.3(25.8) | 75.9(24.7) | 76.6(23.8) | 78.3(24.1) |\n| q-FFL       | 73.5(22.7) | 76.8(22.5) | 77.8(23.0) | 78.9(22.1) |\n| FedHeal     | 74.8(22.7) | 78.2(22.3) | 78.1(22.9) | 79.5(22.6) |\n| FedPW       | **75.6(21.9)** | **79.5(21.0)** | **79.0(21.1)** | **81.1(21.6)** |\n\n\n**Q4: Missing references for some related methods.**\n\nA7: We sincerely apologize for this oversight and thank the reviewer for the helpful reminder. The missing citations for FedCurve, Ditto, and other relevant works will be properly included in the final version of the paper.\n\nThank you for your valuable feedback and hope that our rebuttal adequately addresses your concerns!"}, {"review_id": "X2ucQcBF5a", "reviewer": "Reviewer_ixbu", "summary": "This paper focuses on performance fairness in federated learning. It addresses two key challenges: (1) parameter redundancy, which can lead to inefficient model updates, and (2)  the persistent protection of weak clients, which may act as outliers and reduce overall model quality. To tackle these issues, the authors propose FedPW, a dual-pronged approach: the paper introduces a parameter adjustment strategy to reduce redundancy and proposes an adaptive weighting mechanism to dynamically balance client contributions based on their reliability. The authors conducted extensive experiments, achieving higher average accuracy and lower performance disparity compared to baseline FL methods and fairness-oriented approaches in both single-domain and cross-domain scenarios.", "questions": "1) How does FedPW generate client-specific parameter masks? Could the authors clarify the criteria for selecting which parameters to mask for each client?\n\n2) What defines the transition between training phases? Are these boundaries based on empirical observations or adaptive metrics?\n\n3) Please specify the communication settings between the client and server (e.g., local training epoch), as these details are critical for reproducibility.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n\n1. The figures are well-designed and enhance comprehension. The writing is clear and logically structured, making the paper easy to follow.\n\n2. The paper addresses a fundamental and meaningful question: Can an FL algorithm achieve fairness without compromising performance? The authors provide two key insights to mitigate performance degradation in fairness-driven methods.\n\n3. The identification of parameter redundancy and persistent protection issues is well-articulated and supported by explicit evidence.\n\n4. The proposed method is novel and elegantly addresses the stated challenges.\n\n5. The extensive experimental results demonstrate consistent improvements in both performance and fairness.\n\nWeaknesses:\n1. Definition 3.1 introduces performance fairness as the variance of client losses. However, the rationale for choosing this metric over established fairness measures (e.g., Gini coefficient, max-min gap, or Jain’s index) remains unclear. Please justify this choice and compare its implications with alternative metrics.\n\n2. The experimental scope omits validation under extreme conditions, such as large-scale FL settings (>100 clients) or more realistic heterogeneity (e.g., Dirichlet allocation with α=0.01). These scenarios are critical for assessing scalability and robustness.\n\n3. The approach requires all clients to share identical model architectures, which may limit generalization.", "was_revised": true, "final_justification": "The authors have resolved all the issues I raised,  particularly regarding the ariance-based fairness metrics and the resource costs raised by Reviewer NSd9. I acknowledge their efforts and will upgrade my score for this submission.\n\nHowever, the authors are recommended to incorporate two key revisions in the final version:\n\n-  The hyperparameter sensitivity analysis;\n\n-  The validation under more extreme FL conditions.", "rebuttal": "Dear Reviewer ixbu:\n\nThank you for your thorough review and insightful feedback. We sincerely appreciate your time and effort. We hope that the following responses effectively address your concerns.\n\n### Weaknesses\n\n**W1: Additional details on variance-based fairness metrics.**\n\nA1: As shown in the following table, We additionally evaluate fairness using max-loss, minimum accuracy, the group gap between top-10% and bottom-10% clients, Gini index, and Jain’s fairness index. These results show that our method substantially improves the performance of the most disadvantaged clients across diverse fairness definitions.\n\n*Table: Fairness Scores under Different Fairness Metrics in Digits*\n\n| Method  | max-loss $\\downarrow$ | min-acc $\\uparrow$ | group gap $\\downarrow$ | Gini index $\\downarrow$ | Jain's index $\\uparrow$ |\n| ------- | --------------------- | ------------------ | ---------------------- | ----------------------- | ----------------------- |\n| FedAvg  | 0.097                 | 39.39              | 47.23                  | 0.410                   | 0.936                   |\n| q-FFL   | 0.091                 | 42.66              | 46.57                  | 0.257                   | 0.942                   |\n| FedHeal | 0.094                 | 41.95              | 47.46                  | 0.213                   | 0.945                   |\n| FedPW   | **0.080**                 | **44.59**              | **44.13**                  | **0.180**                   | **0.951**                   |\n\n**W2: Validation under more extreme FL conditions.**\n\nA2: We acknowledge the importance of evaluating performance under more challenging federated settings. Our original submission already includes experiments with 100 clients and Dirichlet allocation at $\\alpha$ = 0.1 (see Tables 3 and 4). To further validate the robustness of FedPW, we have added new experiments covering larger client scales (from 120 to 300 clients) and more severe data heterogeneity with $\\alpha$ ranging from 0.01 to 0.09. The updated results confirm that FedPW consistently outperforms other methods across all these extreme conditions, demonstrating strong scalability and adaptability.\n\n*Table: Sensitive Analysis on Number of Clients in Digits*\n\n| Number of clients | 120        | 150        | 200        | 300        |\n| ----------------- | ---------- | ---------- | ---------- | ---------- |\n| FedAvg            | 88.3(15.3) | 87.4(15.5) | 88.8(15.1) | 88.0(15.8) |\n| FedPW             | **91.9(7.5)**  | **92.3(7.1)**  | **93.2(7.3)**  | **92.8(7.1)**  |\n\n*Table: Sensitive Analysis on Dirichlet Heterogeneity in CIFAR-10* \n\n| $\\alpha$ | 0.01       | 0.03       | 0.05       | 0.07       | 0.09       |\n| -------- | ---------- | ---------- | ---------- | ---------- | ---------- |\n| FedAvg   | 78.2(13.1) | 82.3(10.2) | 82.1(10.7) | 84.7(10.9) | 86.4(11.7) |\n| FedPW    | **81.3(7.5)**  | 81.8(**7.8**)  | **86.0(7.1)**  | **86.7(7.1)**  | **88.23(7.2)** |\n\n**W3: Discussion on the limitations of assuming homogeneous model architecture.**\n\nA3: Using homogeneous models is a common and convenient setting for fairness-oriented methods. Most fairness-driven FL works adopt this setting, and we follow the same convention. Nonetheless, our approach is compatible with extensions to heterogeneous models. Specifically, our aggregation strategies do not require model alignment, and the pruning mechanism can be adapted to partially shared components, such as aligned layers. Therefore, our framework retains flexibility for broader applicability in non-iid or partially heterogeneous environments.\n\n\n### Question\n\n**Q1: What is the criterion for generating parameter masks?**\n\nA4: FedPW constructs client-specific parameter masks based on the magnitude of local updates. For each client $k$, we sort the elements of the update vector $\\Delta w_k^t$ by absolute value and compute a threshold $\\tau_k$ based on the client-specific mask ratio $r_k^t$. All parameters below this threshold are marked as redundant using a binary mask $M_k^t$, as described in Eq. (5). The mask ratio $r_k^t$ is determined by the inverse of the client’s smoothed training loss, meaning that underperforming clients retain more updates to aid their progress.\n\n**Q2: How is the transition between the two training phases defined?**\n\nA5: The transition between the two training phases in FedPW is not determined by a fixed round count or manually defined schedule. Instead, it is guided by an adaptive, data-driven mechanism. The transition is soft and continuous, governed by the dynamics of training signals rather than a clear boundary, allowing the optimization focus to shift gradually in response to the evolving state of training.\n\n**Q3: What communication settings were used in the experiments?**\n\nA6: For single-domain experiments, we used 3,000 communication rounds with local epoch = 1 and batch size = 50. For cross-domain experiments, we used 200 communication rounds with local epoch = 10. The batch size is 64 for the Digits dataset and 16 for Office-Caltech. Detailed communication and training configurations are described in the Implementation Details section of the paper (Page 8)."}, {"review_id": "yV8A8HyndG", "reviewer": "Reviewer_bBPA", "summary": "The paper aims at solving a challenge  in federated learning： existing fair FLs often focus on improving weak clients during the training process. While this enhances weak clients, it can hinder global optimization, leading to lower overall performance and persistent unfairness. \n\nTo tackle this issue, the paper proposes a new fair FL framework, FedPW, with two main parts：(1) it introduces a Parameter Adjustment technique using masks and rescaling to discard redundant parameters, thus preserving important updates and reducing conflicts. (2) the author(s) propose  a dynamic aggregation function which  adaptively weights clients in training process varies across two phases. \n\nThe experiments show that FedPW achieves better global model accuracy and fairness across clients from different domains compared to multiple baselines.", "questions": "1. How is the redundancy value in the Fig. 2 (right) y-axis evaluated?\n\n2. The standard deviations are significant in Table 2, and it seems not to represent an ideal fairness level, which means that the accuracy performance disparity among clients reaches up to 20%?", "limitations": "The adaptive  Weighting may have some practical limitations: \n\n(1) whether accuracy and fairness can achieve Pareto stability, and \n\n(2) whether the final level of fairness can be controlled, which can be guaranteed in other multi-objective gradient descent-based fair FLs.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "1. This work proposes a Parameter Adjustment + Adaptive Weighting strategy. The former addresses weight conflicts  in federated learning under multi-objective optimization, while the latter provides a solution to ensure balanced performance across clients from different domains.\nThe inclusion of the Early Phase/Later Phase criterion is a valuable addition that provides insights into when fairness should be integrated into the training process. The proposed solution is logically structured, and the technical details are well-explained, making it a useful contribution to the FL community. \n\nHowever, the adaptive  Weighting may have some practical limitations: \n\n(1) whether accuracy and fairness can achieve Pareto stability, and \n\n(2) whether the final level of fairness can be controlled, which can be guaranteed in other multi-objective gradient descent-based fair FLs.\n\n2. Another strength is the well-designed experiment setup, with sufficient baselines and a variety of experimental scenarios. The effectiveness of the experiments may make this paper a good new fair FL baseline for federated learning with cross-domain clients. \n\nThe corresponding weakness is that the experiments is not completely provided in the code link.", "was_revised": true, "final_justification": "- The authors provided well-developed code in the discussion phase and clarified how their method establishes the necessary condition for Pareto stability. \n\n- The provided equations and algorithmic flow make sense, and they have committed to adding further improvements in the revision. \n\n- Compared to existing baselines, this paper advances the fairness of model performance distribution among clients, though without significant improvements. However, I like the perspective of this paper, from handling parameter redundancy, which is novel and interesting.", "rebuttal": "Dear Reviewer bBPA:\n\nThank you very much for your thoughtful questions and concerns about our work. We have carefully considered each comment and provided responses.\n\n### Weaknesses\n\n**W1 & L1: Pareto stability guarantee.**\n\nA1: Thank you for your insightful comment. Since our method is not designed from a multi-objective optimization perspective, we did not explicitly provide a Pareto optimality analysis in the main paper. However, our approach satisfies the necessary conditions for achieving Pareto optimality under certain assumptions. Specifically, the global descent direction in our method is a weighted sum of individual client gradients. In the early stages of training, these gradients tend to be aligned (i.e., having positive pairwise inner products), and in later stages, they become nearly orthogonal. As long as the weights remain positive and not excessively large, the resulting global direction maintains a positive inner product with each client’s gradient, thereby qualifying as a Pareto Descent Direction. This property enables FedPW to approximate the Pareto frontier in practice, promoting a balanced trade-off between accuracy and fairness.\n\n**W2: The provided experimental code is incomplete.**\n\nA2: We sincerely apologize for the earlier incomplete release of our experimental code. The repository has now been fully updated to include all components necessary for reproduction, including complete datasets scripts, training and evaluation pipelines, and baseline implementations. We welcome you to re-evaluate the codebase, and we appreciate your understanding and feedback.\n\n### Questions\n\n**Q1: How is the redundancy value in the Fig. 2 (right) y-axis evaluated?**\n\nA3: We apologize for the confusion. The y-axis in Figure 2 (right, Page 4) does **not** represent the redundancy value itself. Each point corresponds to a particular experiment (as in Figure 2 left), where the x-axis denotes the redundancy ratio and the y-axis shows the resulting test accuracy. For instance, in the CIFAR-10 setting, when the redundancy ratio reaches around 0.85, the corresponding accuracy is approximately 70%.\n\n**Q2: Large std in Table 2 suggests a suboptimal fairness level.**\n\nA4: Thank you for your insights and concerns. As shown in Table 2 (Page 8), the performance varies noticeably across different domains. However, in multi-domain federated fairness settings, significant performance variation across domains is common due to extreme non-IID conditions. Our work focuses precisely on evaluating robustness under such challenging scenarios. Even state-of-the-art fairness methods struggle to achieve small standard deviations in this setting. Despite the seemingly large deviation values, our method achieves substantially lower disparity compared to SOTA baselines, highlighting its effectiveness and robustness in enforcing fairness under practical FL conditions.\n\n### Paper Formatting Concerns\n**F1: Incorrect reference of Equation (10) as Equation (9).**\n\nA5: Thank you for pointing this out. We sincerely apologize for the oversight. There was indeed a mix-up between the references to Equations (9) and (10). We will correct this citation in the final version.\n\n**F2 & F3: Explanation of equations and algorithmic flow.**\n\nA6: We appreciate your valuable suggestion. Due to the coupling between the two components of our method, some equations are cross-referenced, which may have disrupted the overall flow of the algorithm. To clarify the core workflow, we provide a simplified pseudocode that highlights the essential steps. We will include the complete pseudocode in the appendix of the final version and revise the corresponding descriptions to improve clarity.\n\n> For each client $k = 1, ..., K$:  \n> //*Adaptive Weighting*:  \n>\t$p_k^t, \\Delta p_k^t \\leftarrow (p_k^{t-1}, \\Delta p_k^{t-1}, \\beta)$ from Eq.(9)  \n>\t$q_k^t, \\Delta q_k^t \\leftarrow (q_k^{t-1}, \\Delta q_k^{t-1}, \\beta)$ from Eq.(10)  \n>\t$\\lambda_k^t \\leftarrow f(p_k^t, p_k^t)$ from Eq.(11)  \n>//*Parameter Adjustment*:  \n> For $i = 1, ..., G$:  \n>\t$\\mathcal{M}\\_{k,i}^t \\leftarrow$ mask computed from Eq.(5)  \n>\t$\\Delta \\mathbf{w}\\_{k,i}^t = \\Delta w_{k,i}^t \\cdot \\mathcal{M}\\_{k,i}^t$  \n>\t$\\alpha_i^t \\leftarrow$ scaling from Eq.(7)  \n>\t$\\Delta \\mathcal{W}\\_i^t = \\alpha_i^t \\sum_{k=1}^K \\lambda_k \\Delta \\mathbf{w}\\_{k,i}^t$  \n\n**F4:  Clear definitions for $\\sigma_p$ and $\\sigma_q$.**\n\nA7: Thank you for the feedback. Since the definitions of $\\sigma_p$ and $\\sigma_q$ are relatively straightforward, we only described them in words in the original manuscript. Specifically, $\\sigma_p$ denotes the client-wise standard deviation of $p_k$ (defined in Eq. (9)), and $\\sigma_q$ denotes the standard deviation of $q_k$ (defined in Eq. (10)). For completeness and clarity, we provide their explicit mathematical definitions below:\n$$\n\\sigma_p = \\sqrt{\\frac{1}{N} \\sum_{k=1}^N \\left( p_k - \\frac{1}{N} \\sum_{i=1}^N p_i \\right)^2}\n$$\n$$\n\\sigma_q = \\sqrt{\\frac{1}{N} \\sum_{k=1}^N \\left( q_k - \\frac{1}{N} \\sum_{i=1}^N q_i \\right)^2}\n$$"}]

Author rebuttal:

Meta-review: This paper proposes FedPW, a federated learning approach to improve average performance and fairness. FedPW is based on two strategies: using dynamic masks to discard redundant parameter updates and upweight critical ones; using dynamic aggregation strategies to weight clients based on update directions and performance variations. After discussion, reviewers are overall satisfied and believe the paper makes a great contribution in federated learning, especially the technical novelty that conforms to the intuition and common beliefs, the extensive experiment evaluations on both single-domain and cross-domain tasks, and high writing quality. Hence, we recommend acceptance. Congratulations! Please make sure to incorporate reviews in the camera-ready version, especially adding discussion to address concerns on hyperparameter sensitivity analysis, applicability/limitations on heterogenous model architectures, and theoretical results such as convergence bounds.
Decision: Accept (poster)