A Fair Federated Learning Method for Handling Client Participation Probability Inconsistencies in Heterogeneous Environments
Authors: Siyuan Wu
Yongzhe Jia
Haolong Xiang
Xiaolong Xu
Xuyun Zhang
Lianyong Qi
Wanchun Dou
Venue: NeurIPS 2025 poster ()
Keywords: Model-heterogeneous federated learning
Fair federated learning
Inconsistent client participation
Primary area: infrastructure
TL;DR: This paper proposes a fair federated learning method for handling client participation probability inconsistencies in heterogeneous environments

Abstract: Federated learning (FL) is a distributed machine learning paradigm that enables multiple clients to collaboratively train a shared model without exposing their raw data. However, existing FL research has primarily focused on optimizing learning performance based on the assumption of uniform client participation, with few studies delving into performance fairness under inconsistent client participation, particularly in model-heterogeneous FL environments. In view of this challenge, we propose PHP-FL, a novel model-heterogeneous FL method that explicitly addresses scenarios with varying client participation probabilities to enhance both model accuracy and performance fairness. Speciﬁcally, we introduce a Dual-End Aligned ensemble Learning (DEAL) module, where small auxiliary models on clients are used for dual-end knowledge alignment and local ensemble learning, effectively tackling model heterogeneity without a public dataset. Furthermore, to mitigate update conﬂicts caused by inconsistent participation probabilities, we propose an Importance-driven Selective Parameter Update (ISPU) module, which accurately updates critical local parameters based on training progress. Finally, we implement PHP-FL on a lightweight FL platform with heterogeneous clients across three different client participation patterns. Extensive experiments under heterogeneous settings and diverse client participation patterns demonstrate that PHP-FL achieves state-of-the-art performance in both accuracy and fairness. Our code is available at: https://github.com/Siyuan01/PHP-FL-main.

A Fair Federated Learning Method for Handling Client Participation Probability Inconsistencies in Heterogeneous Environments
Siyuan Wu', Yongzhe Jia', Haolong Xiang,, Xiaolong Xu²,
Xuyun Zhang", Lianyong Qi', Wanchun Dou
State Key Laboratory for Novel Software Technology, Nanjing University, China School of Computer and Software, Nanjing University of Information Science and Technology, China School of Computing, Macquarie University, Australia *College of Computer Science and Technology, China University of Petroleum (East China), China
Abstract
Federated learning (FL) is a distributed machine learning paradigm that enables multiple clients to collaboratively train shared model without exposing their raw data. However, existing Fl. research has primarily focused on optimizing learning performance based on the assumption of uniform client participation, with few studies delving into performance fairness under inconsistent client participation, particularly in modee-heteroogeneous FL environments. In view of this challenge, we propose PHP-FL, novel model-hhetroogeneuus FL method that explicitly ad- dresses scenarios with varying client participation probabilities to enhance both model accuracy and performance fairness. Specifically, we introduce a Dual-End Aligned ensemble Learning (DEAL) module, where small auxiliary models on clients are used for dual-end knowledge alignment and local ensemble learning, effectively tackling model heterogeneity without public dataset. Furthermore, to mitigate update conflicts caused by inconsistent participation probabilities, we propose Importance-drivee Selective Parameter Update (ISPU) module, which accurately updates critical local parameters based on training progress. Finally, implement PHP-FL a lightweight FL platform with heterogeneous clients across three different client participation patterns. Extensive experiments under heterogeneous settings and diverse client participation patterns demonstrate that PHP-FL achieves state-of-the-art performance in both accuracy and fairness. Our code is available at: bttps://githann http:///gittau...oi//saaa/////// comm//iyuanno//H--PPL-----
Introduction
Federated Learning (FL) has emerged promising paradigm for enabling decentralized model training across multiple clients without directly sharing their private data [II, 21). By collaboratively learning global model while keeping data localized, FL offers strong privacy guarantees and broad applicability across various domains such as mobile devices, healthcare, and finance [G] 1--55]]-
Despite significant progress, traditional FL methods still face two critical challenges: I) Model het- erogeneity. Traditional FI. assumes that all clients share an identical local model architecture, which is often impractical in real-world due to the diversity in client capabilities. To address this, Model- Heterogencous Federated Learning (MH-FL) has emerged as a promising research paradigm [2. B- 9]. which allows each client to maintain personalized model tailored to its own resource constraints
Cooresponning author: Wanchun Dou (douwe noouwc@nju.cdu.cc)
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Partetaalon Pribusllty
100
Moan: 50.09
200
Mean: 47.33
Pattopatinn Prbbullly (80
-20
Meani 50.00
FNL
PHP-L
7D
40
+p
CDent
(a) Uniform pattern
(b) Normal pattern
(c) Linear pattern
(d) Client accuracy
Figure 1: Left three: three client participation probability patterns (average value is approximately
50%). Right: client accuracy under patterns (a)-(c). Refer to Section 5. for experimental details.
or task requirements. However, most existing MH-FL methods [10-13] primarily aim to ensure com- patibility between diverse model architectures, yet overlook the faimess issues posed arising from inconsistent client participation probabilities in practical deployments. This oversight result in biased models that favor frequently participating clients, ultimately compromising the robustness and fairness of the FL system. II) Unfairness caused by inconsistent client participation. Most existing FL. research [IL. 9. I2- implicitly assume a uniform client participation pattern, where all clients equally likely to participate in each training round, as illustrated in Figure Dal In practical deployments, clients often face heterogeneous conditions, such as intermittent connectiv- ity, fluctuating network bandwidth, and network coverage of base stations [TZ-20]. These internal and external factors lead to client unavailability and result in non-uniform participation probabil- ities, as exemplified in Figure II and Idd which may critically degrade both the overall accuracy and fairess of the FL system. Figure In compares the final client accuracy distributions between FML [2T] (a representative MH-FL method) and our method PHP-FL under three participation pat- terns. While FML suffers significant performance degradation in the normal and linear patterns compared to the uniform participation, PHP-FL maintains stable accuracy with only marginal drops. Furthermore, PHP-FL, demonstrates tighter accuracy distributions, indicating better performance fairness. Although some studies [22-25)] improve overall system performance by proactively se- lecting ligh-availability clients and discarding less efficient ones, such strategies often compromise fairness across clients. Furthermore, numerous fair federated learning methods aim to enhance per- formance fairness through personalized model [|E], [G] weight recalibration [27 2N Neverthe- less, they typically operate under the assumption of uniform client participation, failing to address the challenge of inconsistent client availability. Despite its importance, this issue has received lim- ited attention in the literature, particularly in the context of MH-FL, where the interplay between model heterogeneity and participation imbalance exacerbates the learning challenge.
To address these challenges, we propose PHP-FL., fair federated learning method designed to address scenarios with varying client participation probabilities in model-hheteoggenoous environ- ments. Specifically, to effectively tackle model heterogeneity without relying on public datasets, we introduce Dual-End Aligned ensemble Learning (DEAL) module, which leverages lightweight auxiliary models on clients to align heterogeneous local models and enable ensemble learning to improve the performance of local tasks. Furthermore, to mitigate the adverse effects of update con- flicts that caused by inconsistent client participation probabilities, we propose a Importance-drin Selective Parameter Update (ISPU) module. The ISPU module adaptively updates only the most critical task-relevant parameters based on training progress, allowing clients with different participa- tion frequencies to selectively absorb varying ratios of global knowledge. This design helps reduce gradient conflicts and enhancing fairness. Our main contributions are as follows:
To the best of our knowledge, this is the first work to explicitly address performance unfair- ness caused by inconsistent client participation probabilities in practical FL systems with heterogeneous local models.
We propose PHP-FL, a novel model- heeroeehttroossssuus federated learning method designed to address inconsistent client participation probabilities, aiming to jointly improve both overall accuracy and performance fairness across clients.
We evaluate PHP-FL through extensive experiments on a lightweight FL platform simulat- ing multiple realistic participation patterns. Empirical results on the FaSSion-MNIST and CIFAR-10 datasets demonstrate its state-of-the-art performance, while the ablation study further validates the effectiveness of each proposed module.

---

2 Related Works
Model- Heterogeneous Federated Learning. Hocrrgeectoogeeeouuu Federated Learning (MH- FL) has emerged as a promising research direction [ 6 8 B, 22, [3]] Existing work in this area can be broadly categorized into knowledge distillation-ern based (KD) methods, representation alignment methods, and partial model sharing methods. KD is one of the most widely adopted techniques MH-FL. The studies in [29-611) enable clients with different architectures to distill knowledge through shared or public dataset. However, such reliance on public data limits applicability in privacy-sensitive settings. Another popular line of work, like IIS. 12, [6],, focuses on representa- tion alignment, which aligns feature representations or prototypes rather than raw model parameters, allowing clients to maintain model diversity while contributing to a shared learning objective. Fur- thermore, some studies
adopt partial model sharing strategies, where clients share only specific model components (e.g., shared backbone or predictor) while keeping the other parts dis- tinct, thereby enabling partial compatibility across models. However, few MH-FL methods explicitly address fairness for clients under inconsistent participation probabilities, a critical requirement for equitable and robust deployment.
[ t [El]
Fairness in Federated Learning. Existing research on fairness in federated learning has primarily focused on three key dimensions: (1) Contribution Fairness [. BS, [6]] which involves evaluating each client's contribution to the global model to guide equitable benefit distribution, often using
techniques like Shapley value influence functions [37], 38]; (2) Model Fairness [599 [0)) which
addresses inherent biases in model predictions concerning sensitive attributes, thereby promoting fairness at the prediction level; and (3) Performance Fairness [14 20, 27. II 28), which aims to ensure uniform model performance across clients, typically by minimizing the variance or standard deviation of test accuracy among clients. As prior studies [92, 33| demonstrate, these fairness met- rics often conflict, making it infeasible for a model to simultaneously achieve optimal performance across all dimensions. Our work, therefore, specifically targets performance fairness, aiming en- sure uniform performance across clients while concurrently optimizing overall performance under inconsistent participation probabilities in MH-FL. While another line of research [M] 3E SSS ad- dresses client unavailability by primarily focusing on maintaining the average performance across clients, they do not explicitly ensure performance-lever fairness. Furthermore, these methods are typically designed for homogeneous settings and face significant challenges when generalizing to heterogeneous federated learning environments, where client capabilities vary substantially.
3 Preliminaries
The Global Objective of Federated Learning. Following typical federated learning [II. 26] set- tings, we consider a set of K clients (index by i) with local datasets {D1, D2,..., ..., Dx) } where D, {(z,. ((mz.,;)) ) and
Dj)- In heterogeneous FL environment, each client maintains a unique model wi parameterized by 0, E Rdd,, with dimensional heterogeneity (d, * d,)) arising from hardware constraints (e.g., compute/memory limits) or personalized model specialization. This implies dim( * dim(0,). 3i * j E [K]. The global objective function can be expressed as:
K
K ΣA=1
min (w kR
F F(1w = P.F......
where ps the weight of client i, {w;), represents the set of the client's local models, and F,(un)) = N Σn (1,,,00; My) is the local objective for client with loss function L.
Inconsistent Client Participation Probability. To simulate realistic client availability in feder- ated learning, we consider three types of client participation probability patterns. Let Pi, denote the probability that client actively participates communication round l:
Definition 1
(Uniform Patlern) All clients share an identical and fixed probability (0, of participating in each round, i.e., Pi, a, Vi E (1,2....,,,, ,.), Vl.
Definition 2 (Normal Pattern) The participation probabilities are drawn from a truncated normal distribution simulate natural heterogeneity; Pia N(u, σ), and pi, is clipped to (0,1).
3

---

Definition 3 (Linear Pattern) Client participation probabilities are distributed according to an in- creasing arithmetic sequence: Pit = a+ (i 1)d, i= 1.2..... n, Vt, where a is the first term and d is the common difference. In this pattern, the initial sequence satisfies 0 < P1.t < P2.t <.. ..... lim1 is then randomly shuffled prior to use to eliminate any inherent ordering bias among clients. It models systematic heterogeneity such as tine-varying connectivity or device capacity.
PK. S 1 for round t. This sequence { K
Note that Pi,t is independent of the history and other clients. The three distinct patterns considered enable a comprehensive analysis of how heterogeneous client participation affects both fairness and overall performance in federated learning.
Design Goals. In this paper, we aim to design a MH-FL method under inconsistent client par- ticipation probabilities that not only optimizes the average performance across all clients but also enhances performance fairness. Formally, let ai I....,K) represent the test accuracy on the
i-th client's local test dataset. The Accuracy Metric (AM) is defined as: AM = * Ek= ag. The Fairness Metric (FM) is defined as: FM Std(a), ok), where Std(.) denotes the standard deviation. To this end, our approach seeks to maximize the average local accuracy (AM) while minimizing the performance disparity (FM), ensuring both high overall performance and fair perfor- mance distribution across clients.
4 Methodology
4.1 Overview of PHP-FL
Our method consists of two key modules: dual-end aligned ensemble learning (DEAL in Sec- tion 422), and importance-driver selective parameter update (ISPU in Section A.3.) The DEAL module employs small homogeneous auxiliary model to perform bidirectional representation-logit alignment between local and auxiliary models, which resolves model heterogeneity and enhances overall performance via ensemble learning. To handle inconsistent client participation, the ISPU module selectively updates task-relevant critical parameters using an importance-basee masking mechanism. This approach applies larger updates stragglers accelerate overall convergence, while reducing updates for frequent participants to prevent adverse effects from the stragglers. This ensures efficient knowledge fusion and fair parameter evolution across clients.
As shown in Figure 2. the training process in each communication round of PHP-FL can be sum- marized follows:" ① The server first computes the client-specific auxiliary model G O M for the active client E Ar by pruning non-essential parameters using historical binary mask M... @ Then active clients initialize the personalized auxiliary models g both the local model w, and the auxiliary model gi into a backbone and a predictor, which are used for representation extraction and soft prediction, respectively. At the beginning of local training, the DEAL module first optimizes the ensemble weights X on the adaptation set D,, while w and g. - are frozen. Local training then proceeds using the customized loss Cw. which comprises two components: (1) The dual-end alignment loss LDEAL enabling bidirectional knowledge align- ment through data-free distillation and representation matching. (2) The ensemble learning loss LENS, which further enhances overall model performance. O Following the local training, the ISPU module calculates an update ratio af based on the client's 's total training rounds. It then estimates the top-a important parameters of 9 using the -oom and generates a binary mask Mt. Finally, each active client uploads both its updated local auxiliary model g and the binary mask M to the server. G The server updates the historical mask Mt E Miis for each active client by replacing its entry with the newly received M.. Then the received auxiliary models are aggregated via simple averaging technique to obtain the global auxiliary model go for round + 1.
PHP-FL decomposes
4.2 Dual-End Aligned Ensemble Learning
To address system heterogeneity without relying on public datasets, previous works such as [S] decompose the model wP (parameterized by f) into a backbone and a predictor wp and perform
2 "lgoorih M in Appendix A describes the PHI-FI. algorithm.
We omit the client index and the communication round index for notation simplicity.

---

Local auxiliary models Ae
ge-1
M
ge-l
Server
Praning
Model
Logits-wie addition
Latest records mask Mrists
Pensonalized pruning
Proning
Mask
Element-wfse
operation
Mask and local auxiiiary model
Pruned global auxiliary model
Active Client I A,
Sureed
L.
A
LENS
yrod
Backbone
Predictor
Logits
gurred
4 MOTOE
Topey
Calculate update
Logits
Inference
M
Figure 2: The overview of PHP-FL.
aggregation based on intermediate representations (z(0;; xx)) produced by the backbone. How- ever, it's challenging to classify representations generated by heterogeneous backbones for local predictors. Inspired by the spirit of the mutual learning, some works [86, [7]] leverage logits-level knowledge distillation, where each client co-trains heterogeneous local model w and lightweight homogencous global model g (parameterized by Φ) by aligning only the final outputs of the predic- tors. Unfortunately, this limited alignment fails to facilitate meaningful knowledge transfer to the backbone component, resulting in suboptimal representation learning. To address these limitations, propose DEAL module. In DEAL, both the backbone and predictor components between local and auxiliary models are explicitly aligned. This dual-end alignment ensures effective and rapid fusion of local and global knowledge. To this end, design the following loss function:
m DEAL =
LMMD CNMM(uu,(6;:100 go (6(;); (1))) + DKL (w x (0g
(2)
D.
JED.
Backbone Alignment
Predictor Alignment
where DKL is Kullback-Leible (KL) divergence. The Maximum Mean Discrepancy (MMD) loss LMMD between sets of representations 21 TRa and 22 Rmxd using Gaussian radial basis function (RBF) kernel is computed as:
m Σ
LVMD(Z)
k(f(2)) (()
{(h(2!)),
m
+1
2 nm
T
m
h (0((()) k(2))))
where f IRd, + ddd and h 22 + d represent customizable projection functions designed to standardize feature dimensions, enabling cross-architecture feature alignment between the local model w and global model g when their structures are heterogeneous. The Gaussian RBF kernel k(z, z') exp (->liz- z' 2'|²²) measures similarity between representations, with 1/(20²) con- trolling the kernel bandwidth. This non-parametric metric effectively captures the distance between the distributions of and in the Reproducing Kernel Hilbert space (RKHS). MMD is able to effectively align feature distributions by comparing global statisties via kernel-based embeddings. handles non- IID data robustly, and enables stable optimization [[8].

---

Next, to fully leverage the classification capabilities of both local models and global auxiliary mod- els, we adopt model ensembling [ enhance performance on local tasks:
1 Di
LENS
Σ (CCE(w((; U5) + (cw(Xw((; + (1 - JED,
(4)
where Lce denotes the cross-entropy loss between the predicted label and the ground-truth label. Furthermore, to address the challenges posed by the potential heterogeneous system capabilities between each local model w and the global model g. we set À trainable parameter for each client and randomly hold out tiny adaptability set D from the training set D; (e.g., 10%) for its optimization at the beginning of every communication round. The remaining set on each client for training is denoted as the study set D;. This round-wise resampling of the adaptability set guar- antees that the ensemble weight X; continuously optimized on fresh and unbiased data, thereby effectively mitigating overfitting risks. The learning process of X each client is:
X x 7A
Vx- Vx--Beeeww--- Ler (((; (1-xa X (1-X"=")g(0000 (1d) (5) JEDŞ
Σ
D
where 7Ia is the learning rate for A. Finally, the total training objective of the local model combines the dual-end alignment loss and the ensemble learning loss:
Lup - CHRAL. + LENS:
For symmetry, analogous loss Lg is also computed but omitted here for brevity, it follows the same formulation with reversed inputs. The total losses Lg and fw are used to simultaneously update the homogeneous auxiliary model and the heterogeneous client local model, with learning rates flg and Tw. respectively, as follows:
w t w 1- -Tw D
1
Σ jED
V
w-- E (x),) Lw
g¹ t g" flg
Σ JED;
V 9
(en,)) Lg w X mj.Uj).
D
During the inference stage, clients use the weighted model ensemble for prediction:
ipred arg max maw.aaw(xxxaaaa (w;(0; (;) + (1 {)g((;zz))
(8)
This adaptive weighting mechanism automatically balances the contributions of both models based on their current performance. particularly effective under system heterogeneity, where devices may have varying computational capabilities.
4.3 Importance-Diie Driven Selective Parameter Update
To enable straggling clients quickly catch up upon rejoining training while preserving the leam- ing momentum of more active clients, we propose a novel selective parameter update module, ISPU, which selectively updates task-relevant parameters and suppresses noisy redundant updates. In- stead of directly overwriting the local auxiliary model g f-1 with the global model Gt-1, we perform model fusion by identifying the most significant parameters in g The update ratio a is adap- tively determined by sigmoid- based scheduling function according to the training progress:
N:(1) f+1 0.5
exp 8
where N.(1) -((t==' - 1ii A,) denotes the cumulative number of rounds in which client has participated up to round and 1f-} is the indicator function. Here, 7 E (0, 1] represents the prun- ing threshold and 8 is tunable sharpness hyperparameter. We then apply a binary mask on the parameters of - to retain only the critical parameters and replace them with the corresponding parameters from the global model Gr-1, Common pruning metrics include the ε-norm [[5,, BJ,, Fisher Information Matrix (FIM) [57. 52], and sensitivity-based measures [33, 54]]. Specifically, adopt

---

the f1-norm to evaluate the importance of parameters, which has been proven to be an effective technique for assessing parameter significance [S>, Bl Compared to other metrics, this formulation better captures the importance of task-relevant parameters. After local training, the binary mask M, constructed to update only the top-o important parameters of gf in the next participation round:
1. if d-th parameter topp-a largest of g
Miaa
(10)
0, otherwise
This parameter-wise filtering mechanism helps frequently active clients preserve their learned knowledge while allowing infrequent clients to assimilate global updates more effectively, enabling rapid catch-up up and mitigating knowledge drift. Specifically, at the beginning of round t, the local auxiliary model 9; -1 of each active client At is updated as follows:
- g -M - M."
(11)
where M is the mask matrix obtained by client from its most recent training round and -M denotes the bit-wise inverse of the mask M. This approach ensures clients preserve critical knowl- edge via high- ingortapeteance parameters, filters out conflicts from heterogeneous data and inconsistent training progress.
Experiments
5.1 Experiments Details
Datasets. We evaluate our proposed PHP-FL. on two standard image classification benchmarks: Fashion-MNIST [56] and CIFAR-10 [57]] For both datasets, we adopt 4:1 ratio to split samples into training and test sets. Following previous studies [122, 58% 554, we simulate heterogeneous data distributions by allocating class j proportions to each client k according to Dirichlet distribution (pi,k ~ Dir(3)), where smaller 3 implies more extreme data heterogeneity across clients. We adopt 3 = 0.1 for Fashion-MNIST and B 0.5 for CIFAR-10, respectively. Notably, each client's s local training and test sets share the same distribution.
Models. Our experimental setup employs four heterogeneous model architectures: (1) GoogleNet [60], (2) DenseNet-121 [[]]], (3) EfficientNet-B1 [622], and (4) ResNet-18 [63]] Each client is assigned one of these models based its identifier i, following a round-robin strategy where client receives the model corresponding to i mod 4. Comprehensive comparative results of homogeneous model architectures are provided in Appendix O+
Comparison Baselines. the heterogeneous model experiments, we comprehensively evaluate our method against six state-of-the-ar heterogeneous federated learning algorithms without rely- ing on public data, including FML [RII], FedGen [IC]], FedKD [[]]], FedAPEN [H77], FedTGP [I2], FedMRL [[3] In addition, we also compare a standalone baseline where clients train locally without any aggregation or communication.
Implementation Details. Our experimental framework is built on lightweight MH-FL platform HtFLlib [63]] using PyTorch 2.2.2 [65]] with NVIDIA RTX 3090 GPU. We employ SGD as our optimizer with learning rate of 0.001 and a local batch size of 64. The global training process consists of 100 communication rounds, with total of 20 clients. During each federated training round, clients perform 5 local epochs of training. In each round, client participation follows the three patterns introduced in Section & We repeated all experiments three times with different random seeds and present the averaged results. More details are provided in Appendix B.
Evaluation Metrics. As defined in Section R, we evaluate the performance using the average Top- 1 accuracy (AM) across all clients. In evaluating the faimess of the clients, we adopt the standard deviation (FM) of client accuracy when the Top-1 test accuracy achieved. In PHP-FL, the test accuracy and fairness for each client are derived from the ensemble output of its local and auxiliary models, as computed by Eq.
https://www, toronto. edu http:///wwww..toooooooooooe/////
html

---

Table 1:
Comparison with the Mfte-of-thhe-at methods Fashion-MNIST in the heterogeneous setting. Best in bold and second with underline. indicates improved accuracy (%) and indicates improved standard deviation (%) of accuracy compared with the best baseline, respectively.
Uniform [a = 0.5]
Normal (a = 0.5, 0.2]
Linear
0.05, d
K-2 KTK
Methods
(%)
FM
AM (%)
FM (9)
(%)
FM (8)
Standalone
95.88
9.26 1.37
96.89-10.19
9.37 1.22
95.77-1.0005
9.86.10.53
FML [arXiv20]
89.09110.66
23.13 1.18
88.71
23.32:
88.15:10.73
2-.21
FedGen [ICML21]
93.97.11.13
22.78 1.46
93.91
23.25
93.72:10.93
23.80
FedKD [NC22]
95.67=00.11
9.20=
95.65
9.02
0..17
0.30
FedAPEN [KDD23]
96.79:10.06
6.24:1:.26
96.79:10.07
6.50:10.60
9.073:10.06
6.89
FedTGP [AAA124]
94.35
10.94
94.06+
21.3
94.21 2.44
11,50
FedMRL [NIPS24]
96.06:1:0:4
9.07
9..45:1:0.43
9.25 1.39
95.78 0:009
9.864
PHP-FL (Ours)
97.64:10.04
0.15
97.59-10.04
0.24
97.58-10.03
4.29:10.04
0.85
2.59
0.80
2.26
0.95
2.60
Table 2: Comparison with the state-of-the-art methods on CIFAR-I0 in the heterogeneous setting. Best in bold and second with underline. 1 indicates improved accuracy (%) and J indiccates improved standard deviation (%) of accuracy compared with the best baseline, respectively.
Uniform [a 0.5]
Normal 0.5,
0.2]
Linear
0.05,
Methods
K(K-1)
AM (%6
FM (%)
AM (%)
FM (5) 4
AM (%)
FM 4.
Standalone
56.66+
10.28
56.77:
00.1
00.30
10.21
FMI. [arXv22X))
40.61
46.161t 0.34
1.02
05.7
46.04 1.16
05.85
FedGen [[CML21]
54.27
0..2
54.38+00.02
10.33+00.31
54.50
10.45+0.466
FedKD [NC22]
54.86++0.43
0..43
04.8
10.36:00.33
04.55
10.45
FedAPEN [KDD23]
60.30+40.33
60.35:40031 0.30
9.45
60.30
0,30
FedTGP [AAA124]
53.39
9.73 0.61
53,40:10.52
10.62 1.17
$2.85 1.16
10.28 1.078
FedMRL [NIPS24]
52.80=t0.63
12.81 +: 1.17
52.61 0.77
12.85 1.20
53.1 0.66
12.95 1.27
PHP-FL (Ours)
66.85
6..94:10.39 0.07
8.07:10.24
0.338:10.36
8.09
1 6.55
1.38
1 6.59
1.29
1 6.38
1.31
5.2
Comparison to State-of-the-Arts Methods
We evaluate PHP-FL against several state-of-the- methods in Tables and The experiments are conducted under three distinct client participation patterns: uniform, normal, and linear, represent- ing diverse real-world scenarios. Across all settings, PHP-FL consistently demonstrates superior performance. achieves the highest average accuracy (highest AM values) while simultaneously exhibiting the best fairness (lowest FM values). Compared to the strongest baseline FedAPEN, PHP-FL achieves notable improvements in average performance, boosting AM by up to 0.87% on Fashion-MNIST and a substantial 6.51% on CIFAR-10. At the same time, it enhances fairness by reducing FM by up to 2.48% and 1.33% on the respective datasets, demonstrating the robustness and effectiveness of PHP-FL in addressing model heterogeneity and unfairness caused by inconsistent client participation. Appendix CT further shows its faster convergence and superior performance through accuracy and standard deviation curves.
5.3 Ablation Study
In Table E. we present an ablation study to evaluate the contribution of the DEAL and ISPU mod- ules in PHP-FL under the normal participation pattern. When both modules are disabled, the per- formance significantly degrades, especially CIFAR-10, where the accuracy drops to 59.88%

---

Standalone
FedGen FedKD
FedAPEN
FedMRL PHP-FL (Ours)
FML
FeoTGP
90
50
10.0
85
$4.09
84 06
I1 37
40
20
20
7.5
5.29
73.05
4.00
30
78.20
30
15
14.77
75
2.5
20
(%)
(b) AM (%) @ B
FM
B
0.1
(d) FM (%) 5
Figure 3: Comparison results CIFAR-10 under varying degrees of data distribution heterogeneity across clients. All other settings follow their default configurations.
6.09
0.90
0.12
0.7
CIAR-10
Standard Deviatinn
0.57
0.1
bevizion
0.6
0.90
0.00
0.000
0.97
0.4
0.07
0.03
0.85
Communtration Rounds
(a) Fashion-MNIST
(b) CIFAR-10
(c) Average weight À
Figure Left two: effect of r on performance. Right: average weight À of clients in each round.
Introducing the ISPU module alone brings a modest improvement, highlighting its effectiveness in mitigating update conflicts from inconsistent client participation.
Besides, enabling only the DEAL module yields a more pronounced performance gains. This is achieved by effectively addressing sys- tem heterogeneity through data-free knowledge aligmment and ensemble learning. Notably, enabling both DEAL and ISPU achieves the best performance on both datasets, with 97.59% accuracy on Fashion-MNIST and 66.94% CIFAR-10, 10, demonstrating their complementar- ity and the necessity of their joint design.
Table 3: Ablation study of key modules of PHP-FL under the normal pattern.
Fashion MNIST
CIFAR 10
DEAL
ISPU
AM (%)
(%)
AM (%)
(%
92.86
9.10
59.88
11.24
/
96.72
5.07
62.05
9.47
X
96.98
5.73
63.08
8.16
v
97.59
4.24
66.94
8.07
5.4 Case Studies
To evaluate PHP-FL's robustness under varying data heterogeneity. we conducted additional experiments on CIFAR-10 using the Dirichlet distribution with 0.1 (high heterogeneity) and = 5 (low heterogeneity). As shown in Figure EL PHP-FL consistently outperforms all baselines across both settings. Under high heterogeneity, PHP-FL surpasses the best- performing baseline by 2.09% in accuracy (AM) and reduces the faireess metric (FM) by 1.84%. This advantage becomes even more pronounced under low heterogeneity, where PHP-FL achieves remarkable 8.83% accuracy gain over the next best method while maintaining the best fairness performance. These results clearly show that PHP-FL is not only robust to different levels of data heterogeneity but consistently achieves state-of-the- art performance in both accuracy and faimess.
Robustness to
Effect of the Pruning Threshold 7 on Performance. To investigate the effect of the hyperparam- eter т, we conduct experiments on Fashion-MNIST and CIFAR-I0 under the normal pattern. As shown in Figure -V and HR, on the CIFAR-10 dataset, the accuracy first increases and then decreases, while the standard deviation initially decreases and then increases, with both metrics achieving their best values when is set to 0.2. For the Fashion-MNN MNIST dataset, the performance remains relatively

---

KDE FaDAPEN
6DE FooMRL
KDE PeP
Numbar of Glant
Numbor of Clants
5.50 0.60
0.50 0.60 0.70
0.50
0M0
5.50 0.40
0.50
0.50
6.70
0.00
0.00
0.30 0.60
0.50 0.60 0.70 0.00 0.40
ACOUDCy
ACDUBIC
AcOuscy
Figure 5: The client accuracy distribution at the best achieved AM (best mean accuracy) on CIFAR- 10 dataset under the uniform participation pattern for PHP-FL and two other baseline methods.
stable across different т, and similarly, the best results are also observed at 7 = 0.2. Therefore, we choose r 0.2 the default configuration for all experiments.
Effeet of Adaptive Ensemble Weights. We analyze the behavior of the adaptive ensemble weight by tracking its average value across clients throughout training under the uniform pattern. As depicted in Figure _d. the dynamics of À differ significantly between datasets. For CIFAR-10, 10, the average À is initialized as 0.5 but quickly decreases and stabilizes around 0.43, indicating consis- tent preference for the global model within the ensemble on this more complex dataset. In contrast, on Fashion-MNIST, the average À steadily increases from 0.5 to approximately 0.58 by the end of training, signifying a growing reliance on the specialized local models w. This demonstrates that the adaptive mechanism effectively captures dataset-specific characteristics, dynamically adjusting the ensemble balance between local and global models to leverage their respective strengths during the learning process.
Visualization of Client Accuracy Distribution. To visualize the client accuracy distribution un- der the normal participation pattern, we plot histograms and Kernel Density Estimation (KDE) [66]] curves for different methods CIFAR- 10 dataset. As shown in Figure EL PHP-FL achieves more concentrated accuracy distribution compared to FedAPEN and FedMRL, with clients generally at- taining higher accuracy. Moreover, the differences in client performance significantly reduced under PHP-FL,, highlighting its superiority in both enhancing average performance and promoting fairness across clients.
6 Conclusion
In this paper, we propose PHP-FL, a novel model-heterogeeneuus federated learning method address- ing the critical challenge of enhancing both accuracy and fairness under inconsistent client partic- ipation probabilities. PHP-FL achieves this through two integrated modules: (1) the DEAL mod- ule, which harmonizes heterogeneous models via fata-free knowledge alignment; and (2) the ISPU module, which selectively updates task-relevant parameters to mitigate update conflicts. Evaluated across diverse participation patterns, PHP-FL demonstrates state-of-the-an performance for both ac- curacy and fairness. Ablation study further validates the effectiveness of each module. Our research narrows the divide between idealized uniform participation scenarios and practical heterogeneous FL systems, providing lightweight yet robust solution suitable for real-world implementation.
Limitations. Despite the promising results, PHP-FL has two main limitations:
First, compared to approaches that exchange only lightweight information (e.g., logits, proto- types [24, 122, partial model parameters [II0, [II)), method introduces non-negligible compu- tation and communication overheads." B Although employing a smaller auxiliary model can alleviate this burden, the additional costs from ensemble training and selective parameter updates still persist.
Second, our evaluation is conducted on constrained set of model heterogeneity types, datasets, and client participation patterns. Although PHP-FI. demonstrates robust performance within these scenarios, its generalizability should be further verified on more diverse and large-scale benchmarks.
Forr analysis of the communication and computation costs, please refer Appendix
10

---

Acknowledgments and Disclosure of Funding
This work was supported by the National Key Research and Development Program of China under Grant No. N0.2024YFE02045000 and in part by the National Natural Science Foundation of China under Grant No.92267104.
References
[1] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera Ar- cas. Communication-effitii efficieen learning deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273-1282. PMLR, 2017.
[2] Mang Ye, Xiuwen Fang, Bo Du, Pong C Yuen, and Dacheng Tao. Hetcrogeneous federated learning: State-oftth-aar and research challenges. ACM Computing Surveys, 56(3):1-44, 2023.
[3] Yongzhe Jia, Xuyun Zhang, Hongsheng Hu, Kim-Kwang Raymond Choo, Lianyong Qi, Xiao- long Xu, Amin Beheshti, and Wanchun Dou. Dapperft: Domain adaptive federated learning with model fusion pruning for edge devices. Advances in Neural Information Processing Sys- tems, 37:1 17:13099-1311 13123,22 2024.
[4] Jingwei Liu, Yating Li, Mengjiao Zhao, Lei Liu, and Neeraj Kumar. Epffl: enhancing privacy and fairness in federated learning for distributed e-healthcare data sharing services. IEEE Trunsuctions on Dependable and Secure Computing, 2024.
[5] Pushpita Chatterjee, Debashis Das, and Danda B Rawat. Federated learning empowered ommendation model for financial consumer services. IEEE Transactions on Consumer Elec- tronics, 70( (111)2200-2511 2023.
[6] Shuai Wang, Yexuan Fu, Xiang Li, Yunshi Lan, Ming Gao, et al. Dfrd: Data-free robustness distillation for heterogeneous federated learning. Advances in Neural Information Processing 3ystems,,6:118854 36:17854-17866 2023.
[7] Liping Yi, Gang Wang, Xiaoguang Liu, Zhuan Shi, and Han Yu. Fedgh: Heterogeneous fed- erated learning with generalized global header. Proceedings of the 31st ACM International Conference on Multimedia, pages 8686-8696, 2023.
[8] Xiuwen Fang and Mang Ye. Noise-robust federated learning with model heterogeneous clients. IEEE Transactions on Mobile Computing, 2024.
[9] Siyuan Wu, Hao Tian, Weiran Zhang, Tingtong Zhu Fuwen Tian, Zhehong Wang, and Wanchun Dou. heterogeneous federated learning method based on dual teachers knowl- edge distillation. In International Conference on Advanced Data Mining and Applications, pages 192-207. Springer, 2024.
[10] Zhuangdi Zhu, Junyuan Hong, and Jiayu Zhou. Data-free knowledge distillation for hetero- geneous federated learning. International conference on machine learning, pages 12878- 12889. PMLR, 2021.
[11) Chuhan Wu, Fangzhao Wu, Lingjuan Lyu, Yongfeng Huang, and Xing Xie. Communication- efficient federated learning via knowledge distillation. Nature communications, 13(1):2032, 2022.
[12] Jianqing Zhang, Yang Liu, Yang Hua, and Jian Cao. Fedtgp: Trainable global prototypes with adaptive-margit -er-aaa-n--n-nnneddd contrastive learning for data and model heterogeneity in federated learning. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 16768-16776. 2024.
[13] Liping Yi, Han Yu, Chao Ren, Gang Wang, Xiaoxiao Li, et al. Federated model heteroge- neous matryoshka representation learning. Advances Neural Information Processing Sys- tems, 37:666331-6666555 2024.

---

[14] Tian Li, Shengyuan Hu, Ahmad Beirami, and Virginia Smith. Ditto: Fair and robust feder- ated learning through personalization. In International conference on machine learning, pages 6357-6368. PMLR, 2021.
[15] Yue Tan, Guodong Long, Lu Liu, Tianyi Zhou, Qinghua Lu, Jing Jiang, and Chengqi Zhang. Fedproto: Federated prototype learning across heterogeneous clients. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 8432-8440, 2022.
[16] Jianqiao Zhang, Caifeng Shan, and Jungong Han. Fedgmkd: An efficient prototype feder- ated learning framework through knowledge distillation and discrepancy- aggregation. Advances in Neural Information Processing Systems, 37:118326-118355 2024.
[17] Michal Yemini, Rajarshi Saha, Emre Ozfatura, Deniz Gündüz, and Andrea Goldsmith. Ro- bust federated learning with connectivity failures: A cemindecentaalized framework with col- laborative relaying. arXiv preprint arXiv:2202.11880, 2022.
[18] Hao Ye, Le Liang. and Geoffrey Ye Li. Decentralized federated learning with unreliable com- munications. IEEE journal of selected topics in signal processing, 16(3):487-500, 2022.
[19] Ming Wen, Chengchang Liu, and Yuedong Xu. Communication efficient distributed newton method over unreliable networks. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 38, pages 15832-15840, 2024.
[20] Ming Xiang, Stratis loannidis, Edmund Yeh, Carlee Joe-Wong and Lili Su. Efficient federated learning against heterogeneous and non-stationary client unavailability. Advances in Neural Information Processing Systems, 37:104281-104328, 2024.
[21] Tao Shen, Jie Zhang, Xinkang Jia, Fengda Zhang, Gang Huang. Pan Zhou, Kun Kuang, Fei Wu, and Chao Wu. Federated mutual learning. arXiv preprint arXiv:2006.1677 2020.
[22] Yae Jee Cho, Jianyu Wang, and Gauri Joshi. Towards understanding biased client selection in federated learning. In International Conference on Artificial Intelligence and Statistics, pages 10351- PMLR, 2022.
[23] Tiansheng Huang, Weiwei Lin, Wentai Wu, Ligang He, Keqin Li, and Albert Y Zomaya. An -boocinn client selection scheme for federated learning with fairness guarantee. IEEE Transactions on Parallel and Distributed Systems, 32(7):1552-1564 2020.
[24] Rituparna Saha, Sudip Misra, Aishwariya Chakraborty, Chandranath Chatterjee, and Pallav Ku- mar Deb. Data-centric client selection for federated learning over distributed edge networks.
IEEE Transactions on Parallel and Distributed Systems, (4(2))675-688 2023.
[25] Zhiyuan Ning, Chunlin Tian, Meng Xiao, Wei Fan, Pengyang Wang, Li, Pengfei Wang, and Yuanchun Zhou. Fedges: A generative framework for efficient client selection in federated learning via gradient-based optimization. arXiv preprint arXiv:2405.063/ 2024.
[26] Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. Proceedings of Machine learning and systems, 2:429-450, 2020.
[27] Tian Li, Maziar Sanjabi, Ahmad Beirami, and Virginia Smith. Fair resource allocation in federated learning. arXiv preprint 1905, 10497, 2019.
[28] Yuhang Chen, Wenke Huang, and Mang Ye. Fair federated learning under domain skew with local consistency and domain diversity. Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 12077-12086, 2024.
[29] Daliang Li and Junpu Wang. Fedmd: Heterogenous federated learning via model distillation. arXiv preprint 190.0338. 2019.
[30] Tao Lin, Lingjing Kong, Sebastian U Stich, and Martin Jaggi. Ensemble distillation for ro- bust model fusion federated learning. Advances Neural Information Processing Systems, 33:2351-2363. 2020.
12

---

[31] Wenke Huang, Mang Ye, and Bo Du Learn from others and be yourself in heterogeneous federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 10143-10153, 2022.
[32] Jachee Jang, Heoneok Ha, Dahuin Jung, and Sungroh Yoon. Fedelassavg: Local representation learning for personalized federated learning on heterogeneous neural networks. In Proceedings of the 51 International Conference Parallel Processing, pages 1-10, 2022.
[33] Guangyu Sun, Matias Mendieta, Jun Luo, Shandong Wu, and Chen Chen. Fedperfix: Towards partial model personalization of vision transformers in federated learning. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4988 098884998.00 2023.
[34] Feijie Wu, Xingchen Wang, Yaqing Wang, Tianci Liu, Lu Su, and Jing Gao. FIARSE: Model- heterogeneous federated learning via importance-aware submodel extraction. In The Thirty- eighth Annual Conference on Neural Information Processing Systems, 2024.
[35] Jingwen Zhang, Yuezhou Wu, and Rong Pan. Incentive mechanism for horizontal federated learning based on reputation and reverse auction. In Proceedings of the Web Conference 2021, pages 947-956, 2021.
[36] Meirui Jiang, Holger R Roth, Wenqi Li, Dong Yang, Can Zhao, Vishwesh Nath, Daguang Xu, Qi Dou, and Ziyue Xu. Fair federated medical image segmentation via client contribution estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16302-16311, 2023.
[37] Ruoxi Jia, David Dao, Boxin Wang, Frances Ann Hubis, Nick Hynes, Nezihe Merve Gürel, Bo Li, Ce Zhang, Dawn Song, and Costas Spanos. Towards efficient data valuation based on the shapley value. In The 22nd International Conference Artificial Intelligence and Statistics, pages 1167-1176. PMLR, 2019.
[38] Nurbek Tastan, Samar Fares, Toluwani Aremu, Samuel Horvath, and Karthik Nandakumar. Redefining contributions: stapley-driver federated learning. In International Joint Conference on Artificial Intelligence (IICAI), 2024.
[39] Yahya H Ezzeldin, Shen Yan, Chaoyang He, Emilio Ferrara, and A Salman Avestimehr. Fairfed: Enabling group fairesss in federated learning. In Proceedings of the AAAI confer- ence artificial intelligence, volume 37, pages 7494-7502, 2023.
[40] Zeqing He, Zhibo Wang, Xiaowei Dong, Peng Sun, Ju Ren, and Kui Ren. Towards fair feder- ated learning via unbiased feature aggregation. IEEE Transactions on Dependable and Secure Computing, 2025.
[41] Zheng Wang, Xiaoliang Fan, Jianzhong Qi, Chenglu Wen, Cheng Wang, and Rongshan Yu. Federated learning with fair averaging. arXiv preprint arXiv:2104.149933 2021.
[42] Mingyang Wan, Daochen Zha, Ninghao Liu, and Na Zou. In-processing modeling techniques for machine learning fairness: A survey. ACM Transactions on Knowledge Discovery from Data, 17(3):1-25 2023.
[43] Reuben Binns. On the apparent conflict between individual and group fairness. In Proceedings of the 2020 conference on fairness, accountabiliry, and transparency, pages 514 524, 2020.
[44] Xinran Gu, Kaixuan Huang, Jingzhao Zhang, and Longbo Huang. Fast federated learning in the presence of arbitrary device unavailability. Advances in Neural Information Processing Systems, 34:1 100:1110222000 2021.
[45] Shiqiang
Wang and Mingyue Ji. A lightweight method for tackling unknown participation statisties in federated averaging. In The Twelfth International Conference on Learning Repre- sentations, 2024.
[46] Eunjeong Jeong, Seungeun Oh, Hyesung Kim, Jihong Park, Mehdi Bennis, and Seong-Lyun Kim. Communication-effiiinn on-device machine learning: Federated distillation and aug- mentation under non-iid private data. arXiv preprint 11X1v:1811.11479, 2018.
13

---

[47] Zhen Qin, Shuiguang Deng, Mingyu Zhao, and Xueqiang Yan. Fedapen: Personalized cross- silo federated learning with udaptability statistical heterogeneity. In Proceedings of the 29th ACM SIGKDD conference on knowledge discovery and data mining, pages 1954-1964, 2023.
[48] Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. The Journal of Machine Learning Reseurch, 13(1):723-773, 2012.
[49] Devansh Arpit, Huan Wang, Yingbo Zhou, and Caiming Xiong. Ensemble of averages: Im- proving model selection and boosting performance in domain generalization. Advances in Neural Information Processing Systems, 35:8265-8277 2022.
[50] Xiaopeng Jiang and Cristian Borcea. Complement sparsification: Low-overhead model prun- ing for federated learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 8087-8095, 2023.
[51] Liyang Liu, Shilong Zhang, Zhanghui Kuang, Aojun Zhou, Jing-Hao Xue, Xinjiang Wang, Yimin Chen, Wenming Yang. Qingmin Liao, and Wayne Zhang. Group fisher pruning for prac- tical network compression. In International Conference on Machine Learning, pages 7021- 7032. PMLR, 2021.
[52] Xiyuan Yang, Wenke Huang, and Mang Ye, Fedas: Bridging inconsistency in personalized federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1 11986-119999 11995, 2024.
[53] Michael Santacroce, Zixin Wen, Yelong Shen, and Yuanzhi Li. What matters in the structured pruning of generative language models? arXiv preprint arXiv:2302.03773 2023.
[54] Xinghao Wu, Xuefeng Liu, Jianwei Niu, Guogang Zhu, and Shaojie Tang. Bold but cautious: Unlocking the potential of personalized federated learning through cautiously aggressive col- laboration. In Proceedings of the IEEE/CVF international conference on computer vision, pages 19375-19384, 2023.
[55] Zhuang Liu, Jianguo Li, Zhiqiang Shen, Gao Huang, Shoumeng Yan, and Changshui Zhang. Learning efficient convolutional networks through network slimming. In 2017 IEEE Interna- tional Conference on Computer Vision (ICCV), pages 2755-2763, 2017.
[56] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for bench- marking machine learning algorithms. arXiv preprint 1XXv:/708.00774/ 2017.
[57] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[58] Qinbin Li, Yiqun Diao, Quan Chen, and Bingsheng He. Federated learning on non-iid data si- los: An experimental study. In 2022 IEEE 38th International Conference on Data Engineering (ICDE), pages 965-978. IEEF, 2022.
[59] Jianqing Zhang, Yang Hua, Hao Wang, Tao Song. Zhengui Xue, Rubui Ma. Jian Cao, and Haibing Guan. Gpft: Simultaneously learning global and personalized feature information for personalized federated learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5041-5051, 2023.
(60] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1-9, 2015.
[61] Gao Huang. Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely con- nected convolutional networks. In Proceedings of the IEFE conference on computer vision and pattern recognition, pages 4700-4708, 2017.
[62] Mingxing Tan and Quoe Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference machine learning, pages 6105-6114, PMLR, 2019.
14

---

[63] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recogni- lion, pages 770-778, 2016.
[64] Jianqing Zhang, Yang Liu, Yang Hua, Hao Wang, Tao Song, Zhengui Xue, Ruhui Ma, and Jian Cao. Pfllib: A geginner-friendll and comprehensive personalized federated learning library and benchmark. Journal of Machine Learning Research, 26(50):1-10, 2025.
[65] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
[66] Emanuel Parzen. On estimation of probability density function and mode. The annals of mathematical statistics, 33(3):1065-1076, 1962.
[67] Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. Practical secure aggregation for privacy-preserving machine learning. proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pages 1175-1191,
[68] Galen Andrew, Om Thakkar, Brendan McMahan, and Swaroop Ramaswamy. Differentially private learning with adaptive clipping. Advances in Neural Information Processing Systems, 34:17455-17460 117666 2021.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The Abstract and Introduction Sections accurately reflect the paper's contri- butions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The discussion about the limitations of this work is provided in the Conclu-
sion.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The au- thors should reflect on how these assumptions might be violated practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the ap- proach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-tex system might not be used reliably provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach ad- dress problems of privacy and faimeess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
16

---

Answer: [NA]
Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theo- rems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be comple- mented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: This paper presents comprehensive details of the experimental setup, and further provides the hyperparameters of baseline methods. This paper also provides an URL link our released code.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, if the contribution is specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate the research performed.
While NeurIPS does not require releasing code, the conference does require all sub- missions to provide some reasonable avenue for reproducibility, which may depend on the nature the contribution. For example
(a) the contribution primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results way to re- produce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case au- thors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
17

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: This paper provides URL link our released code. The public datasets used in this paper are properly referenced.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cc.
htaitt//muddes/Coeeuubmiiii wtt.SSubbimiiioooooooiooo for more details.
While we encourage the release of code and data, we understand that this might not be possible, so No an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:)
7 //appe.cc/pblli//uuudda Saape//oooSSbummmieeee ottp://deweodeeaaaaaaaaaaaoooooo sww.niiissiioooooocc.coooooo for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specifyy all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: This paper presents comprehensive details of the experimental setup in the Experimental Section and further provides the hyperparameters of baseline methods in Ap- pendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropri- ate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The paper reports experimental results averaged over 3 runs with correspond- ing standard deviations in the main results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
18

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should prefer- ably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
This paper describes the experimental enviromment in the Experiments Sec- tion and our code.
Justification:
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't i make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
This paper fully complies with the NeurIPS Code of Ethics all respects.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: There is societal impact of the work performed.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
19

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact spe- cific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitiga- tion strategies (e.g.. gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [No]
Justification:
This paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by re- quiring that users adhere usage guidelines restrictions access the model or implementing safery filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
The assets used in this paper properly credited. The license and terms of use are explicitly mentioned and properly respected.
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
20

---

If assets are released, the license, copyright information, and terms of use in the pack- age should be provided. For popular datasets, paperswithcode com/datasets, has curated licenses for some datasets. Their licensing guide can help determine the li- cense of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documenta- tion provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/codeemoodel as part of their submissions structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the pa- per include the full text of instructions given to participants and screenshots, if applicable, well as details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but the main contri- bution of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection. cura- tion, or other labor should be paid at least the minimum wage the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper deseribe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: This paper does involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equiva- lent) may be required for any human subjects research. you obtained IRB approval, you should clearly state this in the paper.
21

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy Ottees//Doerriooccc///ofeerrrerp for what should or should not be described.
22

---

APPENDIX
A
Pseudo codes of PHP-FL
The algorithm outlined in Algorithm Please refer to Section for more details.
Algorithm PHP-FL
Input: Auxiliary small models {g. [g Local models [w]] Total number of rounds T, Dataset partitions (D,)
li=1
Output: Optimized local models [ Ji=1 and auxiliary small models {gf Jim
Initialize historical mask {M, J t 1 and global model go;
for round = 1 to T do
Server Side:
Collect the IDs of active clients A S (1,...,ky) K): Broadcast the pruned auxiliary small model G o M to client E As
[g]: Mf] JiEA, + Client Update:
Update the historical mask M E Mhist for clients in A; M + M₁ Aggregate auxiliary small models: G A.T EiEA, g
Client Update:
for each client E A, in parallel do
Download Gt C M. from server;
9; -;-1 initialize local auxiliary model by Eq. [I]
Randomly divides the training set D, into D. and D;;
for batch (1r, y) D do
A + update ensemble weight by Eq. S
end
for batch (x. v) E D do
L. + Calculate local training loss by Eq. &
Lg +- Calculate the symmetrical loss similar to Eq. B
w. g + update the local model and local auxiliary model by Eq. I
end
a + calculate update ratio by Eq. B:
M + obtain binary mask by Eq. I0:
Upload of and the binary mask M to server;
end end
B Additional Experimental Details
B.1 Hyperparameter Settings
We provide a detailed summary of the hyperparameter configurations used in our experiments in Table H. These settings are carefully selected ensure fair comparison across different baselines. For the proposed PHP-FL, the standardized feature dimension d is set to 512. The adaptability set DΩ consists of 10% randomly sampled data from the training set D,- The ensemble weight X is trained for 10 epochs in each round. Additionally, following the hyperparameter search detailed in Section b_4 and C the pruning threshold and the sharpness factor 8 are set to 0.2 and respectively. Unless otherwise specified, all experiments follow the same training setting.
"Note that these parameter names in different methods are consistent with the original references and are independent of the notation used in our work.
23

---

Table 4: Hyperparameter settings used in our experiments.
Type
Hyperparameters
Value
Local learning rate 7
0.001
Batch size
64
FL training settings
Local epochs per round Total rounds T
100
Number of clients K
a in FML
/ in FML
0.5
Server learning epochs FedGen
100
Server learning rate in FedGen
0.1
d, in FedGen
32
d, in FedGen
512
Tstart in FedKD
0.95
T Tend FedKD
0.98
in FedKD
0.001
Adaptation set ratio in FedAPEN
10%
Framework specific
Server learning epochs in FedTGP
100
7 in FedTGP
100
À in FedTGP
0.1
in FedMRL
128
À in Ditto
0.1
a for CIFAR-10 in FedFV
0.1
for Fashion-MNIST in FedFV
T for CIFAR- in FedFV
10
7 for Fashion-MNIST in FedFV
] in FedHEAL
0.4
in FedHEAL
0.4
in FedAU
B.2 Visualization of Data Distributions
To intuitively illustrate the data heterogeneity across clients our federated learning setting, we plot scatter diagrams based the CIFAR-10 and Fashion-MNIST datasets in Figure K Specifically, we simulate heterogeneous data distributions by allocating the proportion of class j to each client according to Dirichlet distribution (P3,k Dir(B)), where smaller 8 indicates more extreme data heterogeneity across clients. In our experiments, we set 8 = 0.1 for Fashion-MNIST and 8 0.5 for CIFAR-10, respectively.
B.3 Model Architectures Used in Experiments
we utilize four widely recognized Convolutional Neural Network (CNN) architectures with varying designs and complexities. We report the corresponding parameter counts of each model in Table B. These serve as the local models for clients in our heterogeneous federated learning setup:
Introduced the inception module, which performs convolutions with multiple filter sizes in parallel within the same block. It was designed for computational efficiency and the ILSVRC 2014 challenge.
GoogLeNet [a]]:
DenseNet-121 [61]]: Characterized by its dense connectivity pattern, where each layer re- ceives feature maps from all preceding layers within a dense block. This encourages feature reuse, strengthens gradient flow, and improves parameter efficiency. The '121" denotes the number of layers with weights.
EfficientNet-BI [62]: Developed using neural architecture search and compound scaling method that uniformly scales network width, depth, and resolution. It aims to balance model accuracy with computational efficiency (FL.OP and parameters). B1 is a specific scaled version providing good trade-off.

---

(a) Fashion-MNIST
CIFAR-10
Figure 6: The data distribution of 20 clients in our experiments.
ResNet-18 [63]: Employs residual connections (skip connections) that allow gradients to bypass layers, enabling the training of much deeper networks by mitigating the vanish- ing gradient problem. ResNet- 18 is a relatively shallow variant with 18 layers containing weights.
For fair comparison, adopt GoogLeNet [60]] the auxiliary model architecture for all methods that utilize an auxiliary model, including our proposed method PHP-FL, since has the smallest number of parameters among the four candidate models.
Table Parameter counts of the evaluated models. "M" short for million.
Model
Parameter counts
GoogleNet [60]
5.61M
DenseNet- 121 61]
6.96M
EfficientNNet-B 62
6.52M
ResNet- 18 663
11.18M
Additional Experimental Results
C.1 Comparison of Training Curves for Accuracy and Standard Deviation
To evaluate the convergence behavior and training stability of PHP-FL, we compare the training curves in terms of average accuracy and standard deviation across training rounds on Fashion- MNIST and CIFAR-10 under the uniform pattern. As shown in Figure D and & the results on both datasets demonstrate that the proposed method PHP-FL significantly accelerates convergence compared to baselines, while maintaining low and stable standard deviation throughout training, which effectively enhances performance and fairness across clients.
C.2 Comparison to More State-of-the-Arts Methods in the Homogeneous Setting
To further validate the generality and effectiveness of PHP-FL,, we conduct additional experiments in the homogeneous setting and report the results in Table B. Specifically, all clients adopt the iden-
tical GoogLeNet architectures [60]] Additional compared methods include FedFV [АT]] and Fed-
HEAL [28]. which are designed to improve client fairness (Fair-FL), as well as FedAWE [20] and
FedAU [S which address client unavailability (CU-FL). As shown in Table PHP-FL achieves the highest average accuracy (AM) across all patterns, while also maintaining highly competitive performance faimess (FM). Although FedHEAL exhibits slightly better fairness, average accu- racy is significantly inferior compared to PHP-FL. This highlights that PHP-FL not only delivers highest average performance but also achieves fairness that highly competitive with the best fair-FL methods, striking an exceptional balance that surpasses existing methods in overall effective- ness even in the homogeneous setting. These results underscore the robustness and practicality of PHP-FL even in homogeneous FI. scenarios, reaffirming its overall superiority.
25

---

1.00
0.5
Ours
FedGen
0.95
FadMRL
FML
0.4
Local
Fea
0.90
Standard Deviation
Accurey
0.85
0.3
0.80
FedMAL
0.75
FedTGP
Local
0.2
0.70
FML
0.1
0.65
FedKD
FedAPEN
0.60
0.0
80
100
Step
(a) Average accuracy
Standard deviation
Figure 7: Comparison of training curves on Pashion-MNIST.
0.7
0.30
Ours
Local
0.6
FeaTGP
FML
0.25
FeoGen Fea
0.5
Standard Deviation
FeaMRL
FeoAPEN
Accuracy
0.4
Ours
FedTGP
0.3
FedGen
FedMAL
Local
0.10
Fed
FedAPEN
0.05
20
Stop
Stop
Average accuracy
(b) Standard deviation
Figure 8: Comparison of training curves on CIFAR-10.
Table 6: Comparison with the oftte-of-theaar methods on CIFAR-I0 in the homogeneous setting. Best in bold and second with underline.
Uniform 0.5]
Normal [4 0.5,
Linear a 0.05, K(K-1)
Type
Methods
AM (50)
(%)
AM (5)
FM (%)
AM (%) f
FM (%) J
FML [arXiv20]
01.41
13.28 1..31
41.00-10.48
13.46::0.35
39.21
13.37-1-030
FedGen [[CML21]
61.7910.13
8.80=0.46
61.73:0.115
8.95++0.25
61.60
8.99-1.20
FodKD [NC22]
61.26:0008
10.03:0.100
60.96-00.88
9.93
60.73
10.01:10.09
MH-FL
FedAPEN (KDD23)
66.49:10.30
8.14=0.16
66.54:10.31
8.26:1..18
66.60
0.25
FedTGP (AAA124)
60.12-10.48
9.97=0.46
60.15:10.46
10.20+0.31
59.88
10.14:1:0.1
FedMRL [NIPS24]
62.66:10.22
8.60=0.16
62.86:10.22
8.95++0.58
62.58
8.80-0.38
FedFV [UCAI21]
59.75.10.87
11.03 1.63
60.05 1.31
10,54:12.02
1.432-11.45
11.08 11.08-11.60
Fair-FL
FedHEAL [CVPR24]
59.43:10.83
7.03-000 0.76
59.36:1
7.13
59.15:10.57
7.94 1.15
FedAWE [NIPS24]
62.75:10.09
9.39=
62.68:+0..3
9.31
62.54:10.31
9.36-10.28
CU-FL
FodAU [[CL224]
0.22
9.17
0.27
9.21
62.59-10.47
9.35-+0.15
All
PHP-FL (Ours)
67.99 0.16
0.05=
67.88-1
8.08-10.16
67.84-1-0.13
0.12:20.12
26

---

C.3 Performance with Varying Numbers of Clients
In this Section, we compare the performance of different methods with varying numbers of clients on CIFAR- 10. Specifically, the uniform pattern involves 10 clients with full participation in every round: the normal pattern consists of 50 clients whose participation probabilities are sampled from normal distribution (μ 0.2, the linear pattern also includes 50 clients, with participation probabilities increasing linearly from 0.02 by step of 0.36 K-1' yielding the same average of 10 clients per round. As shown in Table M PHP-FL consistently achieves the best accuracy (AM) and performance faimess (FM) across all three patterns. Notably, compared to the strongest baseline FedAPEN, our method improves AM by 6.62%, 6.47%, and 6.1 under the uniform, normal, and linear settings, respectively, while also reducing FM, demonstrating superior robustness and fairness with varying numbers of clients.
0.2), resulting in an average of 10 clients per round; and
Table 7: Comparison with the state-of- of-the-art art methods with varying numbers of clients on CIFAR- 10 in the heterogeneous setting. Best in bold and second with underline.
Uniform (a 1.0]
Normal [1- 0.2, a 0.2]
Linear a
0.02,
0.36
Methods
(%) f FM (%)
AM
FM (%) 4
(%)
FM (%6 1
Standalone
58.89
10.53-+0024
50.73+:1229
14.04 1.40
51.81+00.30
12.81 +00.47
FML [arXiv20]
46.61
36.59:1.117 18.1
17.35 1.22
35.26:10.22
17.10:10.16
FedGen [ICML21]
57.82:10.06
9.45
43.70:1:.03
15.10 1.26
43.35:1
14.92:10.53
FedKD [NC22]
57.46:10.45
11.26:10.33
49.27-10.93
13.98:10063
49.07-10062
13.99
FedAPEN [KDD23]
03.9
9.05
5..99:
1.77
53.76:10.39
12.74
FedTGP [AAA124]
58.45:0.669
0.83
00.15:t0.32
14.74
38.18 1.04
16.08
FedMRL [NIPS24]
58.33
0.62
49.32:
13.64
49.26:10.74
13.26::0227
PHP-FI (Ours)
7..52:10.03
7.97:10.4
59.46:10.83
12.69 1.41
0..88-10.26
11.22:10.37
Table 8: Comparison results on CIFAR-10 dataset under the Markovian participation pattern. All other settings follow their default configurations.
Method
Standalone
FMI
FedGie
FedKi)
FedAPEN
FedTGP
FedMRI.
PHP-FL (Ours)
AM (%)
52.77=1444
0.96
0.89
5..81
57.84+0.553
50.55 1.79
1.70
6..04:10.73
FM (%)
16.10=2.50
2902 14.91
16.824 1.45 13,99
12,14
15.86
11.69 1.35
C.4
Performance under the Markovian Participation Pattern
To better reflect dynamic client availability in real-word scenarios, we have conducted additional experiments under the Markovian participation pattern following FedAU [[5]]. In this pattern, each client follows a two-state Markov chain to determine its participation status in each training round, where the two states correspond "participating" and "not participating" in the current training round. This modeling introduces temporal correlation in client participation behavior while main- taining sufficient randomness, offering a more realistie simulation compared to independently sam- pled participation patterns.
For the parameter settings of the Markovian pattern experiment, we constrain the maximum transi- tion probability from the non- non-pariicipatin state to the participating state to 0.05, thereby avoiding excessively frequent state changes and ensuring realistic participation dynamics. The initial state of each client's 's Markov chain is randomly sampled according to the stationary probability, which is set to 0.5 ensure that approximately half of the clients participate in each round on average. The transition probabilities are carefully calibrated to maintain this stationary distribution throughout the training process, ensuring system stability while introducing participation heterogeneity across clients. As shown in Table HL the experimental results under the Markovian participation pattern further validate PHP-FL.s robustness, consistently achieving superior accuracy and fairness despite the increased dynamic and unpredictable client availability.
27

---

C.5 Effectiveness of Adaptive Selective Updates in the ISPU Module
To further validate the design rationality of the adaptive selective parameter update mechanism in our ISPU module, we conduct an ablation study comparing different update strategies for the local auxiliary model g. focusing on the update ratio a. Specifically, we evaluate the following variants:
Fixed update. Updates fixed ratio of the most important parameters, with set to constant and we adopt 0.5.
Full Update. Updates all parameters without selection, which is equivalent to
Random Update. All parameters are stochastically updated with probability a, where is dynamically determined by our proposed method using Eq. g
Adaptive update (Ours). Dynamically adjusts using Eq. g based client participation history and parameter importance.
As shown in Table E adaptive update mechanism consistently achieves the best accuracy and performance fairness. Full update shows the poorest performance, and while fixed Update offers some stability, surpassed by the adaptive methods. Random update achieves competitive ac- curacy but inferior fairness compared to our method. The findings highlight the superiority of our adaptive selective update mechanism within the ISPU module.
Table 9: Effectiveness of Adaptive Selective Updates in the ISPU module. Results are evaluated under the linear participation pattern on Fashion-MNIS" and CIFAR-10. 10. Best in bold.
FaSSi
CIFAR-10
Different variants
AM (%) f FM (%)
AM (%)
FM (%)
Fixed Update 0.5)
97.43
4.41
64.11
9.14
Full Update (a = 1.0)
96.82
5.73
62.81
8.65
Random Update (dynamic
97.28
6.23
65.97
8.97
Adaptive Update (Ours with dynamic o:)
97.58
4.29
66.78
8.09
C.6 Effect of the Sharpness Factor on Performance
In our proposed PHP-FL, the hyperparameter 5 in Eq. E controls the sharpness of the mapping from local participation frequency to the update ratio in the ISPU module. A larger d causes to approach 1 for frequently participating clients and 0 for infrequent ones, whereas a smaller ó smooths the adjustment, pushing a toward 0.5. As shown in Table II performance relatively stable across a range of 5 values, but we observe that 5 = 5 consistently achieves near-optimal results both accuracy and fairness across Fashion-MNIST and CIFAR-10. Therefore, we set in our experimental configuration.
Table 10: Effect of the hyperparameter 8 on performance. Best in bold and second with underline.
Dataset
Metric
= 0.1
0.5
8 =
=
5
=
10
5 = 50
= 100
AM (%) f
97.54
97.62
97.59
97.61
97.59
97.52
97.51
Fashion-MNIST
FM (%)
4.25
4.13
4.39
4.12
4.60
4.35
4.43
AM (%) f
67.24
67.28
67.06
67.27
66,36
67.17
66.82
CIFAR-10
FM (%)
8.44
8.10
8.12
8.01
8.51
8.49
8.55
C.7 Cost and Efficiency Analysis
Communication Cost. We compare per-round communication cost with baselines terms of the number of parameters transmitted between 20 clients and the server on CIFAR-10. As shown Table II, among the evaluated baselines, methods such as FedGen and FedTGP demonstrate significantly lower communication costs due to their use of partial model sharing and lightweight
28

---

prototypes. Unfortunately, methods relying auxiliary model transmission (e.g., FML, FedKD, FedAPEN, and FedMRL) exhibit communication cost exceeding 200M parameters per round. In contrast, PHP-FI. requires only 1 112.52M parameters per round, which is nearly half the cost of other auxiliary model-based approaches such as FedAPEN and FedMRL.. This reduction is primarily due to clients downloading only the pruned global auxiliary model parameters instead of the full model.
Computation Cost. We also report the total computation cost per round across all clients in terms of FLOPs (floating-pon) point operations). 8 as summarized in Table M Following [[2)] other operations such as data preprocessing are not included in the FLOPs calculation. To ensure a fair comparison, this experiment involves 20 clients with full participation in cach round. All other configurations remain aligned with the main experiments.
According to Table MI PHP-FL incurs the per-round computation cost at 813.7IGFLOPs. This increase is marginal when compared with other auxiliary model-based methods such as FML (753.51G), FedKD (753.51G), FedAPEN (771.01G), and FedMRL (757.34G) Compared to Fed- Gen (391.38G) and FedTGP (387.99G), the increased computation cost primarily arises from the training of additional auxiliary models. Besides, the slightly higher cost of the proposed PHP-FL mainly attributed to the extra training required for ensemble weights. While FedTGP achieves the lowest computation cost, its accuracy significantly lags. Despite this slight increase in cost pared with other auxiliary model-based methods, PHP-FL achieves significantly the best results in both accuracy and performance fairness, shown in the main experiments. The results underscore a compelling trade-off, with our method delivering notable performance gains at the cost of only a slight increase in computation.
Table 11:
Comparison of communication and computation costs per round on CIFAR-10 10, where communication cost is measured by the number of parameters transmitted between 20 clients and the server, and computation cost is evaluated as the total number of FLOPs executed across all 20 clients. "M' and "G" denote million and giga, respectively. For FedKD, the SVD computation cost excluded from this analysis.
Method
Communication Cost
Computation Cost
FML [arXiv20]
224.20M
753.51G
FedGen [ICML21]
5.92M
391.38G
FedKD [NC22]
200.26M
753,51G
FedAPEN [KDD23]
224.20M
771.01G
FedTGP [AAA124]
0.192M
387.99G
FedMRL [NeurPP24]]
224.05M
757.34G
PHP-FL (Ours)
112.52M
813.71G
Efficiency Analysis. In addition, we conducted further experiments to measure the total costs (communication rounds, computation cost, and communication cost) required to reach 50% accuracy compared to baselines. As shown in Figure 9. while PHP-FL's per-round communication cost is higher than non auxiliary model-based methods (e.g., FedGen and FedTGP) due the transmission of auxiliary model, it reaches target accuracy in just 2 rounds, whereas all baselines require at least 5 rounds. Consequently, the total communication and computation costs substantially lower. Furthermore, the per-round costs can be readily optimized in practice by employing smaller auxiliary models or mask quantization (e.g.. bitwidth reduction). Thus, PHP-FL offers a far more efficient path to convergence in practical deployments.
D Discussion
Communication Cost. In the auxiliary model-based methods, active clients often need to upload I. [g] parameters of the global auxiliary model in each communication round, which are typi- cally represented in full-precision floating-point format. Our method similarly requires uploading
'e calculate FLOPs using the thop library: httt:///tttuuoooo///aaa////tttte
29

---

FedGen
FedKD
FedAPEN
FedTGP
FedMRL
S
PHP-FL (Ours)
2400
45
18
16.68
2003
1600
30
TFLORS
12
1.344
MB
1121
120
19
7.44
7.54
800
15
10
3.86
4.54
225
8.26
0
0
Communication Rounds
Computation Cost
Communication Cost
Figure 9: From left to right: Communication rounds, total number of transmitted parameters, and computation FLOPs required to achieve the 50% accuracy on CIFAR-IO under the uniform pattern.
the global auxiliary model but with an additional binary mask matrix M of the same dimension L. Fortunately, since each element rameter, rendering the added communication overhead negligible compared to the transmission of full-precision parameters. Moreover, when downloading the global auxiliary model, clients only need receive L (a (0, 1)) parameters, where the pruning threshold t in Eq E can be adjusted as needed. This flexibility allows us to further reduce communication costs dynamically.
binary value (0 or 1), it only incues 1 bit per pa-
Participation Patterns. We clearly state that make assumptions about the distribution of client participation patterns and allow them to be arbitrary throughout the training process. More- over, similar to [55]. we do not require any prior knowledge of the client sampling process for the proposed method PHP-FL.
Privacy. Similar FedAvg [II]. PHP-FL does not require sharing data client-specific het- erogeneous model parameters. Instead, only the lightweight, homogeneous auxiliary models and the mask matrix related to model parameters uploaded to the server. This design ensures that sensitive local model structures and data remain on the client side, making PHP-FL suitable for privacy-critical applications. Therefore, PHP-FL compatible with standard privacy-preserving mechanisms such as secure aggregation [67]]. Differentially private variants of FedAvg [68] can be seamlessly integrated similarly.
30


Reviews:
[{"review_id": "CGwGqOwqzP", "reviewer": "Reviewer_zpY2", "summary": "This paper addresses the underexplored challenge of performance fairness in model-heterogeneous federated learning (MH-FL) under non-uniform client participation probabilities. The authors propose PHP-FL, which includes two key modules: (1) the Dual-End Aligned ensemble Learning (DEAL) module for bidirectional alignment between local and auxiliary models using MMD and KL loss, and (2) the Importance-driven Selective Parameter Update (ISPU) module, which adaptively updates task-relevant parameters based on participation frequency. Extensive experiments on Fashion-MNIST and CIFAR-10 under three participation patterns demonstrate that PHP-FL achieves strong performance in both accuracy and fairness, outperforming prior MH-FL baselines.", "questions": "See weakness", "limitations": "Yes (clearly acknowledged in Section 6 and the experiments)", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**:\n\nThe paper tackles a timely and practical problem — performance unfairness under inconsistent participation in MH-FL — which has been insufficiently explored in prior work. This brings new perspectives to the FL fairness literature.\n\nThe DEAL module incorporates both representation-level (via MMD loss) and prediction-level (via KL divergence) alignment, and the ISPU module introduces a sigmoid-based update ratio to balance frequent and infrequent clients, both of which are thoughtfully designed and motivated.\n\nPHP-FL consistently outperforms state-of-the-art MH-FL baselines (FedAPEN, FedMRL, etc.) across multiple participation patterns and datasets, with improvements in both average accuracy and fairness. The ablation study validates the complementary effects of DEAL and ISPU.\n\nThe methodology is clearly presented with good visual illustrations (e.g., Fig. 2, Fig. 4), and code is released anonymously, enabling reproducibility.\n\n**Weaknesses**:\n\n- The experiments are only conducted on two relatively simple vision datasets (Fashion-MNIST and CIFAR-10). These may not sufficiently reflect the complexity and scale of real-world MH-FL applications. Adding results on at least one larger-scale or cross-domain dataset (e.g., CelebA, Reddit, or MIMIC-III) would significantly strengthen the empirical evaluation.\n\n\n- The proposed method introduces multiple additional components (auxiliary model training, MMD computation, adaptive masking), which may increase training cost and memory usage. However, there is no analysis of overhead (e.g., training time, FLOPs, memory footprint), nor comparison with lightweight alternatives. This may hinder practical deployment.\n\n- The paper designs three participation patterns (uniform, normal, linear), but these are synthetic and static. In real-world scenarios, client availability is often temporally correlated or bursty. The current evaluation may not reflect PHP-FL’s robustness under such dynamic conditions.", "was_revised": true, "final_justification": "Thank for authors' rebuttal, which addressed my concerns. I will maintain my positive attitude and score.", "rebuttal": "We sincerely thank you for your constructive and helpful comments. Below we address your concerns in order.\n\n---\n\n## Response to Weakness 1:\n\nThanks for the valuable comments. To address this concern, we have conducted additional experiments on CelebA dataset to further evaluate the generalizability of PHP-FL.\n\nFrom Table R1, we still see that the results on CelebA further validate the effectiveness and robustness of PHP-FL, demonstrating consistently superior performance in both accuracy and fairness, even with increased data complexity. \n\n[**Table R1**: Comparison results on CelebA dataset. The target attribute is *Smiling* and all other settings follow their default configurations.]\n\n|     Methods     |     Uniform     |     Uniform     |     Normal      |     Normal      |     Linear      |     Linear      |\n| :-------------: | :-------------: | :-------------: | :-------------: | :-------------: | :-------------: | :-------------: |\n|                 |    AM (%) ↑     |    FM (%) ↓     |    AM (%) ↑     |    FM (%) ↓     |    AM (%) ↑     |    FM (%) ↓     |\n| FedAPEN [KDD23] | $92.20\\\\pm0.14$ | $12.30\\\\pm0.76$ | $92.16\\\\pm0.10$ | $12.04\\\\pm0.43$ | $92.14\\\\pm0.08$ | $12.60\\\\pm1.17$ |\n| FedMRL [NIPS24] | $87.10\\\\pm0.67$ | $15.38\\\\pm0.72$ | $87.13\\\\pm0.70$ | $15.58\\\\pm0.50$ | $87.34\\\\pm0.99$ | $15.86\\\\pm0.37$ |\n|  PHP-FL (Ours)  | $93.43\\\\pm0.50$ | $11.68\\\\pm0.59$ | $93.34\\\\pm0.41$ | $11.89\\\\pm0.50$ | $93.41\\\\pm0.48$ | $12.19\\\\pm0.65$ |\n\n\n---\n\n## Response to Weakness 2:\n\nWe fully agree that a thorough analysis of training cost and memory usage is essential for evaluating practical deployability. We would like to emphasize that a comprehensive analysis addressing these concerns has already been included in the Appendix. Specifically:\n\n1. **Memory Footprint:** Table 5 in Appendix B.3 provides the parameter counts of all evaluated models, providing a clear comparison of their memory requirements.\n2. **Communication and Computation Costs:** Appendix C.6 provides an in-depth analysis of the communication and computation costs, comparing PHP-FL with two state-of-the-art auxiliary model-based baselines. Our experimental results, presented in Figure 8, demonstrate that while PHP-FL incurs slightly higher per-round communication and computation costs, its significantly faster convergence rate ultimately results in much lower overall resource consumption.\n\nIn addition, we conducted further experiments to measure the total costs (communication rounds, computation cost, and communication cost) required to reach 50% accuracy compared to non auxiliary model-based methods. As shown in Table R2, while PHP-FL's per-round communication cost is slightly higher than non auxiliary model-based methods (e.g., FedGen and FedTGP) due to the transmission of auxiliary model, it reaches target accuracy in just 2 rounds, whereas all baselines require at least 5 rounds . Consequently, the total communication and computation costs are substantially lower. Furthermore, the per-round costs can be readily optimized in practice by employing smaller auxiliary models or mask quantization (e.g., bitwidth reduction). Thus, PHP-FL offers a far more efficient path to convergence in practical deployments.\n\n[**Table R2**: Comparison of total costs (communication rounds, computation cost, and communication cost) required to achieve 50% accuracy on CIFAR-10 under the *uniform* pattern in the heterogeneous setting. '/' indicates that the method failed to reach 50% accuracy.]\n\n|          Methods          | FML  | FedGen |  FedKD  | FedAPEN | FedTGP | FedMRL  | PHP-FL (Ours) |\n| :-----------------------: | :--: | :----: | :-----: | :-----: | :----: | :-----: | :-----------: |\n|   Communication Rounds    |  /   |   19   |   10    |    5    |   43   |    6    |       2       |\n| Computation Cost (TFLOPs) |  /   |  7.44  |  7.54   |  3.86   | 16.68  |  4.54   |     1.63      |\n|  Communication Cost (MB)  |  /   | 112.48 | 2002.60 | 1121.00 |  8.26  | 1344.30 |     230.8     |\n\n---\n\n## Response to Weakness 3:\n\nWe appreciate your constructive suggestion. To address this valuable feedback, we have conducted additional experiments under the *Markovian* participation pattern following FedAU [1], which better reflect dynamic client availability in real-word scenarios. In this pattern, each client follows a two-state Markov chain to determine its participation status in each training round, where the two states correspond to \"participating\" and \"not participating\" in the current training round. This modeling introduces temporal correlation in client participation behavior while maintaining sufficient randomness, offering a more realistic simulation compared to independently sampled participation patterns.\n\nFor the parameter settings of the *Markovian* pattern experiment, we constrain the maximum transition probability from the non-participating state to the participating state to 0.05, thereby avoiding excessively frequent state changes and ensuring realistic participation dynamics. The initial state of each client's Markov chain is randomly sampled according to the stationary probability, which is set to 0.5 to ensure that approximately half of the clients participate in each round on average. The transition probabilities are carefully calibrated to maintain this stationary distribution throughout the training process, ensuring system stability while introducing participation heterogeneity across clients.\n\nAs shown in Table R3, the experimental results under the *Markovian* participation pattern further validate PHP-FL’s robustness, consistently achieving superior accuracy and fairness despite the increased dynamic and unpredictable client availability. \n\n[**Table R3:** Comparison results on CIFAR-10 dataset under the *Markovian* participation pattern.  All other settings follow their default configurations.]\n\n| Methods  |   Standalone    |       FML       |     FedGen      |      FedKD      |     FedAPEN     |     FedTGP      |     FedMRL      |  PHP-FL (Ours)  |\n| :------: | :-------------: | :-------------: | :-------------: | :-------------: | :-------------: | :-------------: | :-------------: | :-------------: |\n| AM (%) ↑ | $52.77\\\\pm1.44$ | $42.73\\\\pm0.96$ | $51.93\\\\pm0.84$ | $51.31\\\\pm0.80$ | $57.84\\\\pm0.53$ | $50.55\\\\pm1.79$ | $49.02\\\\pm1.74$ | $64.04\\\\pm0.73$ |\n| FM (%) ↓ | $16.10\\\\pm2.50$ | $19.29\\\\pm2.04$ | $14.91\\\\pm0.38$ | $16.82\\\\pm1.45$ | $13.99\\\\pm3.34$ | $12.14\\\\pm1.66$ | $15.86\\\\pm0.08$ | $11.69\\\\pm1.35$ |\n\n\n> [1] S. Wang and M. Ji. A Lightweight Method for Tackling Unknown Participation Statistics in Federated Averaging. ICLR 2024.\n\n---\n\nOverall, we hope that our responses can fully address your comments and will be grateful for any feedback."}, {"review_id": "cDZIsmGm1p", "reviewer": "Reviewer_i57X", "summary": "This work addresses performance degradation due to nonuniform participation of clients in FL and proposes a new FL framework\ncalled PHP-FL in model heterogeneous FL environment that can improve model accuracy and performance fairness among clients.", "questions": "1. One main concern is about the communication and computation cost to get and train auxiliary models \nfor clients. The cost also depends on the size of auxiliary models.\nIn the appendix the PHP-FL is compared with other auxiliary model-based models from the\nperspective of communication cost, but a comparison with non auxiliary model-based \nmodels is necessary from the perspective of both costs.\n\n2. The impact of masking needs to be investigated in detail. When the local dataset does not\ncontain full information on key features, masking based on the important parameters of\neach local model, may miss some key features. One important reason for the use of a global\nmodel is to utilize some missing key features, if any, for each local client. It seems that masking\nmay reduce the impact of the use of the global model.\n\n3. In eq. 2, backbone alignment and predictor alignment are considered through L_{MMD} and\nD_{KL}, respectively, but the scales of both terms may be different, so does it make sense to consider\nthe simple sum of both terms with different scales? \nIn eq. 3 how do we select the best for the customizable projection functions f and h?\n\n4. In Fig. 2, $M^h_\\*$ appears in the server, but no explanation is provided on it.\nIt seems to be a typo and hence is replaced by $M^h_i$ according to the expanation in section 4.1.\nIn step 6 of section 4.1, no explanation on how to update $M^h_i$ using $M^t_i$. Are they the same?\n\n5. In $L^w_{ENS}$ (eq. 4) the local model w is used in the first term and the second term,\nwhich seems to be overlapping. What happens if only the second term is considered?\n\n6. What model architecture is used in the experiments for the global model? Is it large enough to have\nfull trained information in it?\n\n7. In Fig. 3, it looks like that $\\lambda$ does not converge until communication round 100.\nThere is a possibility that more communication rounds would improve the performance by choosing a better $\\lambda$.\nA discussion on the convergence of $\\lambda$ would help understand the performance behavior.\n\n8. Improvement on unfairness due to frequently and infrequent active participation:\nThe PHP-FL can improve the fairness in accuracy as shown in the experiment. \nSince other auxiliary heterogeneous models also consider auxiliary models in training, what is the main\nreason why the PHP-FL outperforms other models?", "limitations": "The proposed approach has relatively high communication and computational costs by considering auxiliary models \nfor clients.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "The proposed model can achieve good accuracy and performance fairness among clients.\nEach local client has an auxiliary model as well as its local model and trains both models,\nwhich obviously increases computational cost and communication cost. \nMasking information for each client is also uploaded, which also increases communication cost.", "was_revised": true, "final_justification": "I have carefully reviewed the responses. Since my concerns have been addressed, I keep my score.", "rebuttal": "We sincerely thank you for your constructive and helpful comments. Below we address your concerns in order.\n\n---\n\n## Response to Question 1:\n\nWe sincerely appreciate the your valuable suggestion to compare our method with non auxiliary model-based approaches in terms of communication and computation costs. To address this, we have conducted additional experiments measuring the total costs (communication rounds, computation cost, and communication cost) required to achieve 50% accuracy. As shown in Table R1, while non auxiliary model-based baselines such as FedGen and FedTGP exhibit lower per-round communication costs (e.g., 112.48M and 8.26M respectively), our method PHP-FL, though incurring a higher communication cost per round due to auxiliary model transmission (230.8M), achieves significantly faster convergence (2 rounds versus 19 and 43 rounds, respectively). This rapid convergence significantly reduces the overall computational costs (1.63T for PHP-FL versus 7.44T and 16.68T respectively), offering a considerable advantage in overall compute efficiency. Moreover, in practical deployments, the communication cost of PHP-FL can be further optimized by adopting smaller auxiliary models and applying bitwidth reduction techniques to of the mask $M^t_i$.\n\n[**Table R1**: Comparison of total costs (communication rounds, computation cost, and communication cost) required to achieve 50% accuracy on CIFAR-10 under the *uniform* pattern in the heterogeneous setting. '/' indicates that the method failed to reach 50% accuracy.]\n\n|          Methods          | FML  | FedGen |  FedKD  | FedAPEN | FedTGP | FedMRL  | PHP-FL (Ours) |\n| :-----------------------: | :--: | :----: | :-----: | :-----: | :----: | :-----: | :-----------: |\n|   Communication Rounds    |  /   |   19   |   10    |    5    |   43   |    6    |       2       |\n| Computation Cost (TFLOPs) |  /   |  7.44  |  7.54   |  3.86   | 16.68  |  4.54   |     1.63      |\n|  Communication Cost (MB)  |  /   | 112.48 | 2002.60 | 1121.00 |  8.26  | 1344.30 |     230.8     |\n\n\n---\n\n## Response to Question 2:\n\nOur selective parameter update mechanism within the ISPU module is specifically designed to facilitate targeted selective fusion, rather than to block beneficial global knowledge. As defined in Eq.11, the update rule is given by: $\\hat{\\boldsymbol{g}}_i^{t-1} = \\boldsymbol{g}_i^{t-1} \\odot \\neg M_i^h + \\\\mathcal{G}^{t-1} \\odot M_i^h$, where the mask $M_i^h$ identifies the most signiﬁcant parameters of the local auxiliary model. Importantly, these significant parameters are directly updated by replacing their values with those from the global model $\\\\mathcal{G}^{t-1}$. This selective update mechanism ensures that clients can effectively acquire \"missing key features\" or knowledge embedded in the aggregated global model, which is particularly beneficial for straggling clients whose local parameters might be under-trained. Therefore, the masking mechanism does not diminish the global model's influence; rather, it adaptively transfers global knowledge where it is most needed via $M_i^h$, thereby achieving an effective fusion of global and local knowledge.\n\n---\n\n## Response to Question 3:\n\nRegarding the scales of $\\mathcal{L}\\_{MMD}$ and $\\mathcal{D}\\_{KL}$ in Eq. 2, both are fundamental divergence measures, and their composite sum is implicitly normalized by the mini-batch size. In practice, we observe that the joint optimization process naturally balances their contributions towards overall model alignment, as both terms implicitly guide gradient descent to minimize feature dissimilarities without requiring explicit weighting. Moreover, our MMD implementation further standardizes feature scales via internal normalization of representations (as seen in our anonymous code), which inherently aids their joint optimization. \n\nAs for the projection functions $f$ and $h$ in Eq. 3, these are not externally defined hyperparameters. Instead, they represent learnable feature projection layers within the local model's backbone $\\boldsymbol{w}_b$ and the auxiliary model's backbone $\\boldsymbol{g}_b$, respectively. Their parameters are jointly learned during training, enabling adaptive projection of heterogeneous features into a shared latent space for effective alignment.\n\n---\n\n## Response to Question 4:\n\nWe sincerely appreciate your meticulous observation. As described in step 6 of Section 4.1, $M_i^h$ represents the latest historical mask record of client $i$, which is maintained on the server. The update procedure is straightforward: for each active client $i$, the corresponding $M_i^h$ is directly replaced by the newly uploaded mask $M_i^t$ from the current communication round. Additionally, the notation $M^h_{* }$ in Figure 2 is not a typo. It denotes the entire collection of all clients' historical mask records maintained on the server, i.e., $\\\\{M\\_i^h\\\\}\\_{i=1}^K$ , distinguishing it from an individual client's mask $M_i^h$. To prevent further misunderstandings, we have provided a detailed clarification regarding both $M^h_*$ and the update procedure of $M_i^h$ in the revised manuscript.\n\n---\n\n## Response to Question 5:\n\nWe thank the reviewer for this insightful question. This $\\mathcal{L}_{ENS}^{\\boldsymbol{w}}$ loss in Eq.4 comprises two cross-entropy terms, each serving a distinct purpose: the first term is designed to optimize the performance of local model $\\boldsymbol{w}$ on its local data, while the second term focuses on improving the inference performance of the ensemble output (the inference process is shown in Eq.8). Maintaining the local model's performance via the first term is crucial, as a well-trained local model is essential for the subsequent knowledge alignment within our DEAL module. Removing this term would degrade the local model, thereby harming the alignment process and overall performance. We empirically validate this with an ablation study presented below. The results in Table R2 clearly show that omitting the first term leads to a significant performance drop across both datasets, e.g., a 2.73% decrease in accuracy on CIFAR-10 and a 0.47% decrease on Fashion-MNIST, with a corresponding decline in fairness. This confirms that both components are integral to our method's effectiveness.\n\n[**Table R2**: Ablation study on the first term of the $\\mathcal{L}_{ENS}^{\\boldsymbol{w}}$ loss function in Eq.4.]\n\n|   Datasets    | Metrics  | PHP-FL *w/o* first term of $ \\mathcal{L}_{ENS}^{\\boldsymbol{w}}$ |  PHP-FL (Ours)  |\n| :-----------: | :------: | :----------------------------------------------------------: | :-------------: |\n| Fashion-MNIST | AM (%) ↑ |                       $97.11\\\\pm0.12$                        | $97.58\\\\pm0.03$ |\n|               | FM (%) ↓ |                        $5.02\\\\pm0.37$                        | $4.29\\\\pm0.04$  |\n|   CIFAR-10    | AM (%) ↑ |                       $64.05\\\\pm0.20$                        | $66.78\\\\pm0.36$ |\n|               | FM (%) ↓ |                        $8.89\\\\pm0.18$                        | $8.09\\\\pm0.27$  |\n\n\n\n---\n\n## Response to Question 6:\n\nFor a fair comparison, we consistently adopt GoogLeNet as the auxiliary model architecture for all methods, including our PHP-FL. GoogLeNet is selected because it has the smallest parameter count among the heterogeneous model group, while remaining sufficiently large to save comprehensive trained information, ensuring a robust and equitable evaluation across all auxiliary model-based methods.\n\n---\n\n## Response to Question 7:\n\nWe appreciate the your insightful observation regarding the convergence of the average ensemble weight $\\lambda$ in Figure 3. The appearance of slight fluctuations is primarily due to the fine granularity of the y-axis scale, which exaggerates minor variations. In reality, the values of $\\lambda$ exhibit substantial stability towards approximately100 rounds. To validate this, we extended the experiments under the same setup as Figure 3c to 300 communication rounds. The results showed that $\\lambda$ values at round 300 differed by less than 0.01 compared to those at round 100 for both datasets. This stability aligns with the overall model convergence, as evident from the accuracy curves in Figure 6 and Figure 7. Due to the restrictions on image display here, we will adjust Figure 3 in the revised manuscript to more accurately illustrate the convergence behavior of $\\lambda$.\n\n---\n\n## Response to Question 8:\n\nWhile other auxiliary model-based heterogeneous FL methods also leverage auxiliary models, the distinct advantage of PHP-FL lies in its dual-module design, which is specifically tailored to address inconsistent client participation and model heterogeneity simultaneously.\n\nThe DEAL module extends beyond conventional auxiliary model usage by integrating data-free knowledge distillation and representation alignment for robust dual-end knowledge transfer, enabling more effective alignment among diverse local models. Furthermore, the ISPU module directly tackles the fairness challenge arising from inconsistent participation. Unlike existing methods, it adaptively adjusts the update proportion $\\alpha$ and selectively updates parameters based on each client’s participation history and parameter importance. This mechanism ensures that straggling clients receive prioritized global updates for faster catch-up, while frequently active clients retain their unique knowledge without being negatively influenced by lagging participants. This adaptive and importance-driven selection strategy, which is absent in other auxiliary model-based methods, empowers PHP-FL to achieve superior and more consistent fairness, alongside maintaining high accuracy.\n\n---\n\nOverall, we hope that our responses can fully address your comments and will be grateful for any feedback."}, {"review_id": "XAXEWxJwZb", "reviewer": "Reviewer_KfpB", "summary": "This paper proposes an approach for model-heterogeneous federated learning, with an aim to improve the fairness among clients considering the probabilities of their participations. The main contributions consists of two parts, one is the auxiliary model to transfer the knowledge among clients and the second is a selective parameter update mechanism.", "questions": "1）Could using a fixed adaptability set to train the ensemble learning coefficients lead to overfitting, especially when these coefficients are retrained every round?\n2）How does the model perform under varying degrees of data distribution heterogeneity across clients? While the datasets used in the study already exhibit some heterogeneity, it would be valuable to clarify how the model performs under different levels of heterogeneity.\n3）In the experiments, are the performance metrics reported based on the local models of each client or the ensemble-averaged results after aggregation? This was not explicitly clarified in the text.\n4）K is used to represent the total number of clients and a tunable sharpness hyperparameter. I suggest to use two symbols.", "limitations": "I think the limitations addressed by the author are appropriate.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1) The paper is well-organized and the presentation is easy to follow.\n 2) The designs of the two critical components have well-defined motivations. For the Dual-End Aligned Ensemble learning module, the integration of representation alignment and ensemble learning objectives facilitates both knowledge transfer and localized learning. For the Importance-Driven Selective Parameter Update module, the selection of important parameters for update aligns with the needs of adapting to varying participation frequencies among clients.\n3）The experiments are comprehensive. The method was compared with multiple state-of-the-art heterogeneous federated learning approaches. Specifically, three participation probability models were simulated to validate the effectiveness of the proposed method under diverse scenarios. Additional experiments, including results under model-homogeneous settings, are provided in the appendix.\n\nWeaknesses:\n1）As shown in Section 5.2, while there are some improvements, the gains over existing methods are not particularly significant.\n2）The method lacks theoretical analysis. For instance, there is no theoretical justification for why the selective parameter update mechanism adapts to different participation probability models. Intuitively, defining Equation (9) specifically for different participation probability models might yield better performance. Additionally, no theoretical analysis of convergence is provided.\n3） The proposed approach incurs notable additional computational overhead.", "was_revised": true, "final_justification": "Considering the contributions and the rebuttal, I keep my score.", "rebuttal": "We sincerely thank you for your constructive and helpful comments. Below we address your concerns in order.\n\n---\n\n## Response to Weakness 1:\n\nWe appreciate your observation regarding the performance gains. While some gains may appear modest, PHP-FL consistently achieves superior accuracy and fairness, especially under challenging conditions. For instance, on the Fashion-MNIST dataset, compared to the best baseline, PHP-FL achieves a 0.87% improvement in accuracy (AM) and a significant 2.48% reduction in the fairness metric (FM), indicating enhanced fairness. On the more complex CIFAR-10 dataset, the gains are even more substantial, with a 6.51% increase in AM and a 1.33% improvement in FM. \n\nMore importantly, PHP-FL consistently achieves state-of-the-art performance for both accuracy and fairness across all evaluated scenarios. This dual enhancement, particularly in balancing these often conflicting objectives, marks a substantial advancement in heterogeneous federated learning, where enhancing one metric often compromises the other.\n\n---\n\n## Response to Weakness 2:\n\nThank you for your insightful comments. Our selective parameter update mechanism (ISPU) is specially designed to be adaptive, focusing on task-relevant parameters while suppressing noisy or redundant updates. This design serves multiple purposes: it enables straggling clients to rapidly catch up upon rejoining training, maintains the learning momentum of active clients, mitigates conflicts arising from heterogeneous data, and ensures the retention of critical knowledge via high-importance parameters. Crucially, this mechanism operates without requiring prior knowledge of client participation patterns, which are inherently unpredictable in real-world scenarios. While some existing FL pruning methods pre-define pruning ratios [1] or adjust pruning ratios dynamically based on metrics like accuracy or training rounds [2, 3], our approach takes a pragmatic heuristic strategy. By estimating client lag from participation history to determine the update ratio $\\alpha$, we address the uncertainty of client availability in a practical and effective manner. PHP-FL's superior performance, further validated under a realistic *Markovian* participation pattern ( please see **Response to Weakness 3** from **Reviewer zpY2**), confirms the robustness of this design.\n\nRegarding theoretical convergence, prior studies have already proven the convergence of FL under unknown participation rates [4] and with model pruning mechanisms [5]. Our PHP-FL does not violate the convergence guarantees of FL. Specifically, the convergence of each local heterogeneous model $\\boldsymbol{w}_i$ can be directly inferred from existing FL convergence analyses, as these models are trained locally with standard optimization procedures and the bias resulting from aggregation has been eliminated. For the homogeneous auxiliary model $\\boldsymbol{g}_i$, its training with frozen local models is also guaranteed to converge under challenges such as submodel heterogeneity, non-uniform training, and data heterogeneity, as demonstrated in works like RAM-Fed [6], which establishes a convergence rate of $O(1/\\sqrt{T})$ in a comparable setting. The convergence of both the local and auxiliary models ensures that their ensemble outputs will reach a stable state. This conclusion is further supported by the empirical convergence behaviors observed in experimental results.\n\n> [1] Li, A, et al. Hermes: an efficient federated learning framework for heterogeneous mobile clients. MobiCom 2021.\n>\n> [2] L. Yi , et al. FedPE: Adaptive Model Pruning-Expanding for Federated Learning on Mobile Devices. TMC 2024.\n>\n> [3] Y. Jia, et al. DapperFL: domain adaptive federated learning with model fusion pruning for edge devices. NeurIPS 2024.\n>\n> [4] S. Wang, et al. A Lightweight Method for Tackling Unknown Participation Statistics in Federated Averaging. ICLR24.\n>\n> [5] H, Zhou, et al. On the convergence of heterogeneous federated learning with arbitrary adaptive online model pruning. arXiv 2022.\n>\n> [6] Y. Wang, et al. Theoretical Convergence Guaranteed Resource-Adaptive Federated Learning with Mixed Heterogeneity. KDD 2023.\n\n---\n\n## Response to Weakness 3:\n\nIn PHP-FL, the main source of additional computation can be derived from training the homogeneous auxiliary model $\\boldsymbol{g}_i^t$. However, compared to other auxiliary model-based methods like FedAPEN and FedMRL, our analysis in Table 10 of Appendix C.6 shows that PHP-FL incurs only a modest increase in FLOPs per communication round (5.54% and 7.40% respectively). Importantly, despite this slightly higher per-round computational cost, PHP-FL's faster convergence rate (as shown in Figure 6 and Figure 8 in Appendix) results in significantly lower overall computational requirements to reach the target performance. This highlights our method's efficiency in practical scenarios.\n\nWe further evaluated the total costs for achieving 50% accuracy compared to non auxiliary model-based methods. As shown in Table R1, while non auxiliary baselines like FedGen and FedTGP exhibit lower per-round communication costs, our method PHP-FL, though incurring a higher communication cost per round due to auxiliary model transmission, achieves much faster convergence. This rapid convergence significantly reduces the overall computational costs, offering a considerable advantage in overall compute efficiency. Additionally, in practical deployments, the communication cost of PHP-FL can be further reduced by adopting smaller auxiliary models and applying bitwidth reduction techniques to the mask $M^t_i$.\n\n[**Table R1**: Comparison of total costs (communication rounds, computation cost, and communication cost) required to achieve 50% accuracy on CIFAR-10 under the *uniform* pattern. '/' indicates that the method failed to reach 50% accuracy.]\n\n|          Methods          | FML  | FedGen |  FedKD  | FedAPEN | FedTGP | FedMRL  | PHP-FL (Ours) |\n| :-----------------------: | :--: | :----: | :-----: | :-----: | :----: | :-----: | :-----------: |\n|   Communication Rounds    |  /   |   19   |   10    |    5    |   43   |    6    |       2       |\n| Computation Cost (TFLOPs) |  /   |  7.44  |  7.54   |  3.86   | 16.68  |  4.54   |     1.63      |\n|  Communication Cost (MB)  |  /   | 112.48 | 2002.60 | 1121.00 |  8.26  | 1344.30 |     230.8     |\n\n---\n\n## Response to Question 1:\n\nTo specifically prevent such overfitting, we randomly hold out a tiny adaptability set $D_i^a$ from each client's local training set $D_i$ at the beginning of every communication round. This round-wise resampling of the adaptability set guarantees that the ensemble coefficient $\\lambda_i$ is continuously optimized on fresh and unbiased data, thereby effectively mitigating overfitting risks. We have added a detailed explanation of this mechanism in the revised manuscript.\n\n---\n\n## Response to Question 2:\n\nWe appreciate your constructive suggestion. To evaluate PHP-FL's robustness under varying data heterogeneity, we conducted additional experiments on CIFAR-10 using the Dirichlet distribution with $\\beta=0.1$ (high heterogeneity) and $\\beta=5$ (low heterogeneity). As shown in Table R2, PHP-FL consistently outperforms all baselines across both settings. Under high heterogeneity, PHP-FL surpasses the best-performing baseline by 2.09% in accuracy (AM) and reduces the fairness metric (FM) by 1.84%. This advantage becomes even more pronounced under low-heterogeneity, where PHP-FL achieves a remarkable 8.83% accuracy gain over the next best method while maintaining the best fairness performance. These results clearly show that PHP-FL is not only robust to different levels of data heterogeneity but consistently achieves state-of-the-art performance in both accuracy and fairness.\n\n[**Table R2**: Comparison results on CIFAR-10 under varying degrees of data distribution heterogeneity across clients. All other settings follow their default configurations.]\n\n| $\\beta$ | Metrics  |   Standalone    |       FML       |     FedGen      |      FedKD      |     FedAPEN     |     FedTGP      |     FedMRL      |  PHP-FL (Ours)  |\n| :-----: | :------: | :-------------: | :-------------: | :-------------: | :-------------: | :-------------: | :-------------: | :-------------: | :-------------: |\n|  $0.1$  | AM (%) ↑ | $84.89\\\\pm0.07$ | $80.69\\\\pm0.16$ | $78.34\\\\pm0.37$ | $84.08\\\\pm0.04$ | $85.97\\\\pm0.12$ | $78.20\\\\pm0.49$ | $84.58\\\\pm0.14$ | $88.06\\\\pm0.04$ |\n|         | FM (%) ↓ | $14.57\\\\pm0.20$ | $18.01\\\\pm0.78$ | $20.93\\\\pm0.19$ | $14.94\\\\pm0.26$ | $13.58\\\\pm0.36$ | $22.93\\\\pm1.37$ | $15.66\\\\pm0.81$ | $11.74\\\\pm0.25$ |\n|   $5$   | AM (%) ↑ | $34.81\\\\pm0.10$ | $23.69\\\\pm0.25$ | $34.15\\\\pm0.36$ | $32.48\\\\pm0.31$ | $38.67\\\\pm0.31$ | $33.09\\\\pm0.30$ | $31.48\\\\pm0.24$ | $47.50\\\\pm0.43$ |\n|         | FM (%) ↓ | $4.89\\\\pm0.21$  | $6.19\\\\pm0.15$  | $4.67\\\\pm0.17$  | $4.54\\\\pm0.41$  | $5.43\\\\pm0.33$  | $4.37\\\\pm0.44$  | $8.29\\\\pm0.27$  | $4.21\\\\pm0.47$  |\n\n---\n\n## Response to Question 3:\n\nThank you for the valuable question. As detailed in our methodology, each client $i$'s prediction is generated by the weighted model ensemble of its local model $\\boldsymbol{w}_i$ and the auxiliary model $\\boldsymbol{g}_i$ (Eq.8). Consequently, and in line with our Design Goals (Section 3), each client's test accuracy and fairness are computed from this ensemble output. The overall Accuracy Metric (AM) and Fairness Metric (FM) are reported as the average across all clients, ensuring the evaluation directly reflects the practical performance experienced by clients.\n\n---\n\n## Response to Question 4:\n\nWe appreciate the your careful observation regarding the use of the symbol $K$. To resolve this ambiguity, we have revised the manuscript by using distinct symbols to separately denote the total number of clients and the sharpness hyperparameter.\n\n---\n\nOverall, we hope that our responses can fully address your comments and will be grateful for any feedback."}, {"review_id": "uEWl6L42hd", "reviewer": "Reviewer_VyUg", "summary": "This paper addresses the challenge of inconsistent client participation in Federated Learning. The proposed approach consists of two key modules: the DEAL module, which employs a homogeneous auxiliary model to align heterogeneous local models and facilitate ensemble learning for improved local accuracy, and the ISPU module, which adaptively updates task-relevant parameters using an importance-based masking mechanism. The authors support their method with experimental results.", "questions": "1: In the introduction (line 27), the authors mention that it is impractical to assume all clients share an identical global model. However, the DEAL module still relies on a small homogeneous auxiliary model. Can this assumption be relaxed? If not, what are the key challenges in doing so?\n\n2: In Figure 3c, the values of $\\lambda$ for both datasets appear not to converge. Could the authors clarify whether convergence is expected?\n\n3: The paper lacks theoretical analysis. \n\nMinor:\n\n1: According to Definition 3, should the linear pattern plot in Figure 1c be sorted in ascending order to better reflect the defined metric?\n\n2: In line 127, the sentence is imprecise. The variable $p_i$ implicitly depends on other clients since it is defined as a function of $i$ (the client index), which may introduce cross-client dependencies.", "limitations": "NA", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1: The paper addresses the issue of performance unfairness resulting from inconsistent client participation—a practical and important challenge in real-world federated learning.\n\n2: The proposed method leverages the DEAL and ISPU modules to guide learning in a way that improves both local accuracy and performance fairness across clients.\n\n3: The paper is well-written and easy to follow.\n\nWeakness:\n\nSee the Question section.", "was_revised": true, "final_justification": "The authors have thoroughly addressed my concerns in the rebuttal. Therefore, I am raising my score to 4.", "rebuttal": "We sincerely thank you for your constructive and helpful comments. Below we address your concerns in order.\n\n---\n\n## Response to Question 1:\n\nThank you for your meticulous observation. The statement in Line 27 refers specifically to the client local models $\\boldsymbol{w}_i$, which are indeed heterogeneous and not assumed to be identical. In contrast, the small auxiliary model $\\boldsymbol{g}_i$ in DEAL serves a shared, compact knowledge representation space, designed to facilitate efficient data-free distillation and representation alignment across diverse client models. Its homogeneity simplifies the distillation process by providing a stable common reference for knowledge transfer, without imposing any identical model constraints on the clients' local models.\n\nRelaxing the homogeneity assumption for the auxiliary model would introduce significant challenges:\n\n1. **Architectural Mismatch:** In each round, the server aggregate the auxiliary models from all clients to synthesize a global model $\\\\mathcal{G}$. If the auxiliary models were architecturally heterogeneous, this aggregation would become infeasible, breaking the mechanism for integrating public information and reverting the problem to standard model-heterogeneous federated learning.\n2. **Semantic Alignment:** Without a shared architectural basis, ensuring that feature representations from different auxiliary models are semantically aligned is extremely difficult. Any solution would likely require complex and costly techniques, potentially negating the efficiency of our method.\n\nTherefore, employing a homogeneous auxiliary model represents a practical and scalable design choice, effectively balancing efficient knowledge transfer with the inherent heterogeneity of client models.\n\n---\n\n## Response to Question 2:\n\nThank you for your insightful comments. The appearance of slight fluctuations is primarily due to the fine granularity of the y-axis scale, which exaggerates minor variations. In reality, the values of $\\lambda$ demonstrate substantial stability towards approximately 100 rounds. To validate this, we extended our experiments under the same setup as Figure 3c to 300 communication rounds. The results showed that the $\\lambda$ values at round 300 differed by less than 0.01 compared to those at round 100 for both datasets. This stability aligns with the overall model convergence trends, as evident from the accuracy curves shown in Figure 6 and Figure 7. Due to the restrictions on image display here, we will adjust Figure 3 in the revised manuscript to more accurately illustrate the convergence behavior of $\\lambda$.\n\n---\n\n## Response to Question 3:\n\nRegarding theoretical convergence, our framework builds upon established principles: prior studies have already proven the convergence of FL under unknown participation rates [1] and with model pruning mechanisms [2,3]. Our PHP-FL does not violate the convergence guarantees of FL. Specifically, the convergence of each local heterogeneous model $\\boldsymbol{w}_i$ can be directly inferred from existing FL convergence analyses [4,5], as these models are trained locally with standard optimization procedures and the bias resulting from aggregation has been eliminated. For the homogeneous auxiliary model $\\boldsymbol{g}_i$, its training with frozen local models is guaranteed to converge under challenges like submodel heterogeneity, non-uniform training, and data heterogeneity, as demonstrated in works like RAM-Fed [6], establishes a convergence rate of $O(1/\\sqrt{T})$ in a comparable setting. The convergence of both the local and auxiliary models ensures that their ensemble output will reach a stable and consistent state. This conclusion is further supported by the empirical convergence behaviors demonstrated in experimental results.\n\n> [1] S, Wang, et al. A Lightweight Method for Tackling Unknown Participation Statistics in Federated Averagig. ICLR24.\n>\n> [2] H, Zhou et al. On the convergence of heterogeneous federated learning with arbitrary adaptive online model pruning. arXiv 2022.\n>\n> [3] J, Yuang, et al. Model pruning enables efficient federated learning on edge devices. TNNLS 2022.\n>\n> [4] L, Xiang, et al. On the convergence of fedavg on non-iid data. arXiv 2019.\n>\n> [5] L, Yi, et al. Federated model heterogeneous matryoshka representation learning. Neurips 2024.\n>\n> [6] Y, Wang, et al. Theoretical Convergence Guaranteed Resource-Adaptive Federated Learning with Mixed Heterogeneity. KDD 2023.\n\n---\n\n## Response to Minor Question 1:\n\nWe appreciate the reviewer's insightful observation regarding Figure 1c and its connection to Definition 3. While Definition 3 defines client participation probabilities as linearly ordered, our experimental implementation (please refer to our anonymous code), introduces a permutation operation to $\\\\{p_i^t\\\\}_{i=1}^K$ to ensure randomness and eliminate any implicit ordering bias among clients. Consequently, Figure 1c visualizes the permuted order of these linearly distributed participation probabilities, which aligns with our experimental setup. To avoid potential misunderstanding, we have added a detailed explanation in the revised manuscript. \n\n---\n\n## Response to Minor Question 2:\n\nAs explained in **Response to Minor Question M1**, although $p_i^t$ is initially defined as a function of client index $ i $ to establish the distribution of participation probabilities (e.g., *linear* pattern), our experimental implementation introduces a permutation operation on the set $\\\\{p_i^t\\\\}_{i=1}^K$. This permutation randomly reassigns the participation probabilities among clients, ensuring that the probability assigned to client $i$ is independent of the client index and other clients’ probabilities. This design eliminates cross-client dependencies, preserving fairness and randomness in practice. We have included a detailed explanation of this permutation mechanism in the revised manuscript to prevent potential misunderstandings.\n\n---\n\nOverall, we hope that our responses can fully address your comments and will be grateful for any feedback."}]

Author rebuttal:

Meta-review: This paper studies fairness in model-heterogeneous federated learning under non-uniform client participation. The proposed framework, PHP-FL combines DEAL for dual-end alignment and ISPU for adaptive parameter updates, aiming to improve both accuracy and fairness across clients. The method is well-motivated and shows consistent improvements over baselines on Fashion-MNIST and CIFAR-10. The main concerns are that experiments are limited to relatively simple datasets and the additional overhead introduced by auxiliary models and masking is not analyzed. Nevertheless, the contribution is clear, the design is sound, and the results are convincing. All reviewers give weak accept. I recommend acceptance as a poster. I also expect the authors can address all concerns of reviewers according to rebuttal in the camera ready version.
Decision: Accept (poster)