Confusion-Resistant Federated Learning via Diffusion-Based Data Harmonization on Non-IID Data
Authors: xiaohong chen
Canran Xiao
Yongmei liu
Venue: NeurIPS 2024 poster (2024)
Keywords: Federated Learning
Non-IID Data
Importance Sampling
Diffusion model
Primary area: diffusion_based_models

Abstract: Federated learning has become a pivotal distributed learning paradigm, involving collaborative model updates across multiple nodes with private data. However, handling non-i.i.d. (not identically and independently distributed) data and ensuring model consistency across heterogeneous environments present significant challenges. These challenges often lead to model performance degradation and increased difficulty in achieving effective communication among participant models. In this work, we propose Confusion-Resistant Federated Learning via Consistent Diffusion (CRFed), a novel framework designed to address these issues. Our approach introduces a new diffusion-based data harmonization mechanism that includes data augmentation, noise injection, and iterative denoising to ensure consistent model updates across non-i.i.d. data distributions. This mechanism aims to reduce data distribution disparities among participating nodes, enhancing the coordination and consistency of model updates. Moreover, we design a confusion-resistant strategy leveraging an indicator function and adaptive learning rate adjustment to mitigate the adverse effects of data heterogeneity and model inconsistency. Specifically, we calculate importance sampling weights based on the optimal sampling probability, which guides the selection of clients and the sampling of their data, ensuring that model updates are robust and aligned across different nodes. Extensive experiments on benchmark datasets, including MNIST, FashionMNIST, CIFAR-10, CIFAR-100, and NIPD, demonstrate the effectiveness of CRFed in improving accuracy, convergence speed, and overall robustness in federated learning scenarios with severe data heterogeneity.

Confusion-Resistari Federated Learning via ciffussio-BBase Data Harmonization on Non-IID Data
Xiaohong 1.2. Canran Xiao Yongmei Liu
School Business, Central South University, Changsha, Hunan 410083, China
Xiangjiang Laboratory, Changsha, Hunan 410205, China
3 School of Advanced Interdisciplinary Studies, School of Management Science and Engineering,
Hunan University of Technology and Business, Changsha, Hunan 410205, China
4 Urban Smart Governance Laboratory, Changsha, Hunan 410083, China c888778038163 com, xiaocanran@csu, edu. cn, Liuyongxeicn01663.com
Abstract
Federated learning has become a pivotal distributed learning paradigm, involving collaborative model updates across multiple nodes with private data. However, handling non-i.i.d. (not identically and independently distributed) data and en- suring model consistency across heterogeneous environments present significant challenges. These challenges often lead to model performance degradation and increased difficulty in achieving effective communication among participant mod- els. In this work, we propose Confusion-Resistant Federated Learning via Consis- tent Diffusion (CRFed), novel framework designed address these issues. Our approach introduces a new diffusion- data harmonization mechanism that includes data augmentation, noise injection, and iterative denoising sistent model updates across non-i.i.d. data distributions. This mechanism aims reduce data distribution disparities among participating nodes, enhancing the coordination and consistency of model updates. Moreover, we design a confusion- resistant strategy leveraging an indicator function and adaptive learning rate adjust- ment to mitigate the adverse effects of data heterogeneity and model inconsistency. Specifically, we calculate importance sampling weights based the optimal sam- pling probability, which guides the selection of clients and the sampling of their data, ensuring that model updates are robust and aligned across different nodes. Extensive experiments on benchmark datasets, including MNIST, FashionMNIST, CIFAR-I0, CIFAR- -I0, and NIPD, demonstrate the effectiveness of CRFed in im- proving accuracy, convergence speed, and overall robustness in federated learning scenarios with severe data heterogeneity.
Introduction
20176] has emerged as a powerful paradigm for dis- tributed machine lcarning, enabling multiple clients to collaboratively train a shared model without exchanging raw data. This approach addresses critical concerns around data privacy and security, which are increasingly significant in various sectors such as healthcare Antunes et all 2022 fi- nance [Chatterjee et all, 2023),, and IoT [[l 2020a. Pan all 2023, Yao et all 2024]. How- ever, one of the fundamental challenges in FL is dealing with non-independent and identically dis- tributed (non-IID) data, which can significantly impair the performance and convergence of global model [Zhu all 2021].
Federated Learning (FL) [MMMaban et all
"Corresponding author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Model assregation
Local training client Local training of dientj
Global model
Centralized training
Model parameter space
Server
KA
client
client if!
Federated learning Non-i.i.d ... data
(a) FedAvg
(b) CRfed
Figure 1: Problem illustration of federated learning on Non-ii.d data.
As illustrated in Figure M. FL on non-lID data often suffers from issues like divergent model up- dates and inconsistent global models. Client models trained on heterogeneous data distributions tend to diverge ct (023), making difficult for the server to aggregate them into coherent global model. This divergence is due to inconsistencies in data sources [[xao and [022 and distributions across clients [Duan al, 202T]. This problem leads to reduced accuracy and slower convergence rates, highlighting the need for effective solutions to handle data heterogeneity. Exist- ing research has made significant strides in improving the robustness and efficiency of FL in non-IlI settings. Notable methods include FedProx [[ all 20206), which adds a proximal term to handle 1 and FedGen [[guyen et all, 2021 in- troduce sophisticated strategies like contrastive learning and data generation to mitigate the effects of data heterogeneity. Despite these advancements, issues related to data distribution disparities and model inconsistency persist, limiting the scalability and effectiveness of FL. real-world scenarios.
heterogeneity. Techniques such MOON [ et
Given the critical gaps in existing FL approaches, particularly their limited robustness to severe non-IID data distributions, there a pressing need for more resilient and adaptive solutions. This research is motivated by the necessity enhance FL's capability handle heterogeneous data effi- ciently. The primary objective of this study is to develop a novel FL framework, Confusion-Resse -esistant Federated Learning via Consistent Diffusion (CRFed), which integrates advanced mechanisms to ad- dress data distribution disparities and enhance model consistency across clients.
Our contributions can be summarized as follows:
1. We propose a novel indicator function that dynamically adjusts sample weighting based loss values and uncertainties, facilitating self-paced learning approach that prioritizes more difficult samples over time.
2. Our framework employs diffusion-basee mechanism harmonize data distributions, in- volving iterative noise injection and denoising processes that align local data with the de- sired distribution.
3. We implement a strategic client selection method based on the indicator function, ensuring the inclusion of the most reliable clients, which enhances the robustness and consistency of model updates.
4. Our extensive experimental evaluations demonstrate that CRFed achieves state-of-the-arr the-art performance on benchmark datasets. CRFed outperforms existing methods significantly in terms of accuracy and convergence speed under various non-IlD settings.
2 Related Work
2.1
NID Challenge in Federated Learning
The issue non-IID data was initially highlighted by FedAVG [MeMahan 2017a), and has since been demonstrated that this challenge can significantly hinder the convergence and overall performance of the global model [Zhao et all, 2018. LLetall 2019). Numerous studies, categorized client-centric methods, have been proposed to tackle this problem by adjusting the local training

---

objectives using insights from the global model and the local models of other clients [Wang et all 202T]. For instance, FedProx [Llttal 20205] introduced a proximal term constrain local updates by leveraging the global model. SCAFFOLD Karimireddy et all, 2020 utilized control variates to correct for local training drift, while FedDyn [Acar et all, 2021] introduced a dynamic regularizer
for parallelizing gradients among clients. MOON [L anl 202T applied contrastive learning to minimize the discrepancy between model representations, thereby correcting local training.
Despite their contributions, these methods fall short of fully resolving the core of the non-IlD is- sue and may experience performance limitations in scenarios with highly skewed data distributions [L 20221]. Beyond client-side udjustments, the server also plays role in mitigating the ad- verse effects of non-III data by calibrating the biased global model post-aggregation. For example, CCVR [Luo all 2021] uses virtual representations from an approximated Gaussian mixture model to correct the classifier. FedFTG [Zhang et all 2022] employs data-free knowledge distillation to refine the global model with the knowledge derived from local models. Additionally, strategies such as client clustering [Ghossh all, 2020, [Long et all, 2023] and client selection [Zhaang et all 2021. Wang et al, 2020) can be implemented by the server to alleviate the non-IlD problem. IFCA
Ghoohh et all 2020 iteratively estimates client cluster identities based on local empirical loss and updates model parameters for each cluster via gradient descent.
2.2 Importance Sampling in Federated Learning
In federated learning (FL.,, data sampling strategies are vital for enhancing distributed training effi- ciency. [Muor et atl 20201] proposed selecting local training data based on user-end data correlation analysis. This led to dynamic sampling strategies like [Lac M121aa] where training sample importance is determined by model gradient magnitudes. Similarly, [Rizk el all, 20222] used gradi- ent norms derive sampling weights, minimizing theoretical convergence bounds. However, these methods require immediate gradient computations, increasing local overhead, and assume convex loss functions, which may not apply deep learning models [Rizk all 2021T]. Therefore, develop- ing importance sampling methods suitable for deep learning-basse based FL remains an open challenge.
FL convergence be theoretically analyzed due the model aggregation mechanism [98, 101- 102, 150], with experimental validation for deep learning tasks [Wan et all, 2021J]. Most studies rely on theoretical derivations, limiting practical application. This study aims use diffusion model to automate the modeling of optimal sampling strategies in FL.
Method
3.1
Overview
The CRFeed framework, shown in Figure M addresses challenges posed by non-i.i.d. data in FL.. Our approach integrates a diffusion mechanism and a confusion-resistant strategy to ensure consistent and robust model updates across heterogeneous data distributions. The core idea of CRFed that the performance of client i's data on the global model reflects its contribution to the training process. By using an optimal indicator function, we determine the optimal data sampling probability for each client, enhancing training efficiency and model performance.
The framework comprises several key components: the current global model downloaded by clients at time t; the Model Encoder and Meta-model, which process the global model and client-specific data for the diffusion process; indicator function, computed using client i's data the global model; the Diffusion-based Data Harmonization Mechanism, which uses data augmentation, noise injection, and probabilistic modeling to mitigate data distribution disparities; and the Distribution Decoder, which aligns the denoised data distribution with the desired distribution.
3.2 Indicator Function and Meta-model
The Indicator Function [x44, .,) is designed to measure the reliability of the i-th sample's 's loss value 1, and its associated uncertainty d.- The design is inspired by self eeff-pace learning [lEan et all 20m Castells et all 20200) which adjusts the weights of samples based on their loss values, allowing for a gradual learning process from easy to difficult samples. means that each client should adopt a self-paced learning paradigm, sampling its data in a way that allows the global model to learn from simple to complex tasks. The Indicator Function captures this performance and is defined as follows:
the context of federated learning, this
3

---

0
4
5 6 6
7 7 8 9
1
Client i
2
Model eucoder
Mela-model
5 6
0
1
4
8
0
/ 5 9
07 9
Current global model
P
Po
Project lion
Low flow
Diffusion
Concal
Dendiing
Sampling
Process
N(O,
Iteratively Denolse
Distribution decoder
Desired distribution
Figure 2: CRFed Framework. The process begins with the current global model, which is down- loaded clients. The model encoder processes the global model, and the meta-model is obtained. This meta-model is then projected into gigeer-dimensional space and concatenated with the in- dicator function, forming the combined representation 25. The diffusion-based data harmonization mechanism adds noise to this representation and iteratively denoises to achieve the desired distri- bution. The distribution decoder then aligns the denoised data distribution. Client i's data is sampled based on importance sampling weights calculated as the ratio of the optimal sampling probabil- ity P. the original data distribution Po. This ensures that the sampled data aligns with the desired distribution, following curriculum learning approach that progresses from easy to difficult samples, thus enhancing overall model performance.
1x 0.) = (4 - TTo, + A(logo,)²
(I)
where À is pre-set regularization coefficient. T is a confidence threshold that determines the diffi- culty of the sample based on its loss value. It can either be fixed constant or dynamically adjusted weighted average during the training process.
The Indicator Function can be further explained using the following steps: for each client i, the loss value 4 of each sample calculated on the current global model, and the uncertainty a, of each sample is estimated based its difficulty. The Indicator Function Ix (4,, or,) is then used to assign weights to the samples, with easier samples having lower weights and more difficult samples having higher weights. This adaptive weighting mechanism ensures that the model focuses samples over time, leading to improved learning efficiency and robustness.
on difficult
Theorem 3.1. In the CRFed framework, using the indicator function (x(4, ensures stable and convergent updates for heterogeneous federated learning. For an appropriately chosen learning rate 7, the model update rule for client i iteration t.
(1-nn(o+++ +7 7)K 2X logo; Vol..
(2)
of
guarantees a decreasing step size, promoting convergence. Moreover, CRFed achieves a tighter bound on update steps than FedAvg. indicating faster convergence under the same conditions.
The proof of Theorem 3 can be found in Appendix A
Given the global model f, the optimal uncertainty of can be derived through the following theorem:
Theorem 3.2. The optimal uncertainty of for given loss value is obtained by minimizing the Indicator Function Ix(4,, o.). The solution is given by:
oi =exp (-W
21 max (-2 --- (-2)4-7))
(3)
where W.) is the Lambert W function.

---

We rewrite the indicator function by setting o; = e": reduce the derivative condition to a Lambert W form under domain constraints, and thus obtain closed-form solution for oj: the complete derivation is presented in Appendix A
For each sample indexed by i, we determine the optimal uncertainty of function of its loss 4- Specifically, let /x 04, be the indicator function defined before. Then, selecting of that minimizes [x(1i σ.) induces an adaptive weighting scheme:
(144 o7(4))
(4)
ensuring that when is relatively large. the corresponding optimal of decreases, thus increasing w1(44) and emphasizing more difficult samples. Conversely, smaller 4 values lead to larger of and lower wx(4), indicating that easier samples receive diminished focus. As result, the distribution defined by of is optimally aligned with the current global model 0r, conforming to the self-paced learning principle.
Since (4,4, σι) can regarded as quantifying each client's contribution relative to Oz, we embed O, within the diffusion model as follows: using an autoregressive encoder E, we compress Og to a meta-model 6e = E(O.). Subsequently, oe is projected into a high-dimensional representation P(oi), and concatenated with Ix (4,, o,) to form
2j concat(P(oi). o)).
We then feed 2; into the diffusion model, denoted by DiffusionModel( 22). to refine the data distri- bution with respect the global context. Further implementation details about the encoder E are provided in Appendix A.4
3.3 Diffusion-based Data Harmonization
The diffusion-based data harmonization mechanism is a critical component of the CRFed frame- work, responsible for mitigating data distribution disparities and ensuring consistent model updates across heterogeneous environments. The harmonization process shown Figure which in- volves adding noise to the data distribution and then iteratively denoising it to achieve the desired distribution. The workflow of this mechanism can be divided into two main processes: the forward diffusion process and the reverse denoising process.
Forward Diffusion Process Suppose that a training sample x is of a certain distribution, denoted (xx)). In the forward diffusion process, Gaussian noise with variance ;3 E (0. 1) is added gradually to the sample Xy for T steps, resulting in a latent sample Xy ~ NN The process is defined as follows:
((z1r|(z) = II q(z z1-1),
(6)
q(z, 21-1) Nizz v1 8,
(7)
Using notations a; = 1 - Bg and ā; = III a's the sample can be defined directly as:
2, = Vānzi +
e~~
(8)
Reverse Denoising Process The reverse denoising process aims to sample reversely from žr through transition probabilities g(z(-1lz;) for timesteps T 1 through 1, yiclding sample drawn from ((z)). The transition q (z)) is a Gaussian distribution, tractable when conditioned on 2;:
qiz--iZe 2,) = N(z 2)) B.I,
(9)
where the mean ü. and variance 3 are calculated as:
5

---

1 - Ot
(4 2i)
(10)
V
a22
- at-1 Br.
B.
(11)
Okt.
The reverse transition probability po po(zi-1 (2)) relies on the entire data distribution and is approxi- mated through neural network:
Pe (z-- -i t), Eg(ze, t)),
(12)
The detailed derivations and computations for these processes can refer to A. In the context of federated learning, the reverse denoising process starts from the optimal indicator function / ob- tained in the forward diffusion process. By progressively denoising, we obtain the optimal sampling probability for client i, ensuring that the final data distribution aligns with the desired distribution. This minimizes the impact of data heterogeneity and ensures robust model updates across all clients.
3.4 Confussinn-eeistarn Strategy
The confusion-resistant strategy is designed address the challenges posed by data heterogeneity and model inconsistency in federated learning. It consists of three key components: client selection based on the indicator function, data sampling using the diffusion- harmonization mechanism, and adaptive learning rate adjustment.
Client Selection Strategy To mitigate the adverse effects of data heterogeneity, we select clients based on their indicator function (4, o²), which quantifies the reliability of their data. Clients with the lowest indicator values, reflecting higher data reliability, are chosen for training. This approach follows the curriculum learning paradigm, where lower values indicate better data:
Selected Clients (1,, o,) ST},
(13)
where y is a dynamically adjusted threshold ensuring the selection of the most suitable clients.
Data Sampling Strategy For each selected client, the optimal sampling probability P" is deter- mined through the reverse denoising process, starting from the optimal indicator function /*. This ensures that the sampled data aligns with the desired distribution, enhancing the robustness of model updates. The importance sampling weight w; is calculated as follows:
P Po
(14)
where Po is the original data distribution. Using Wi, sample the local training data (D) (D sampled Sample(D,, w.)). This sampling ensures the effective sampling probability aligns with P..
The distribution decoder, which is implemented noised data distribution. The autoencoder trained to map the denoised samples back the desired distribution, further ensuring that the data used for training is aligned with the ideal distribution. Refer to the 4 for more details of distribution decoder.
autoencoder, is then used to decode the de-
Adaptive Learning Rate Adjustment The learning rate тι for each client is adjusted based on the indicator function value, enhancing the influence of more reliable data:
Ixll, max, Ix (1),)
Th
7/0
(15)
where is the base learning rate.
The complete computational process(pseudocode) of CRFed is provided in the A.S

---

Experiments
4.1
Experiment Setup
Datasets Our experiments are conducted on four widely used benchmark
datasets:
MNIST
LeCun et all [1998].
Fashion-
MNIST Xiao et all
2017].
CIFAR-
Initial distrihution: Ph.
Meta Model Loss
10
Krizhevsky all 2009). and CIFAR-
100
Krzhevsky et all 2009]. To simulate
Non-lID data scenarios, utilize the Dirichlet
distribution [Yurochkin et all, 2019 to generate non-IlI partitions with varied concentration parameters, 3. Smaller values / lead to more imbalanced data distributions among clients, thereby increasing levels of data het- erogeneity. In experiments, we set / to 0.5 to reflect this imbalance. In all experiments, we simulate federated learning environment with 10 edge nodes, i.c., K 10. For the MNIST and FasHionMNIST datasets, each node has 600 data samples. For the CIFAR-10 10 dataset, each node has 500 data samples. For CIFAR-100, the partitioning strategy remains the same, ensuring that each client's local data distribution varies significantly, simulat- ing real-world federated learning scenarios. MNIST and FasHionMNIST datasets consist of grayscale images of size x 28 pixels, with 10 classes. CIFAR-10 and CIFAR-100 contain color images of size 32 x 32 pixels.
Adl Noise
Training
m2
Denobling Process
Nelse Predicter
Nelo Fredktee
Figure 3: The diffusion-bar data harmonization mechanism in CRFed framework. The process involves a forward diffusion process where Gaus- sian noise is added to the initial data distribution, transforming it into a latent representation. This followed by reverse denoising process that it- eratively removes the noise, aligning the data dis- tribution with the desired target distribution.
Additionally, we use the NIPD dataset [Ym atl 2023]] a benchmark specifically designed for federated learning in person detection tasks with Non-III data. This dataset provides real- world non-lll) scenario to test the generaliza- tion of CRFed.
Competing Methods Apart from FedAvg |McMahan et all 2010 we compare the proposed algorithm with several benchmarking FL algorithms specialized for solving the non-IID problem.
including FedProx all 20206), MOON aLietall 20216), and FedGen [Nguyen et al, 2021] We also compare our method against HFMDS-FL. [Li all 2022!), FRAug [Chen et all 2023), G- FML [Yang et all 2023], FedCD [Long et all 2023), FedNP [Wi etal (023), and FedDPMS [Chen and Vikaln. 2023), which are recent state-of-the-ee approaches addressing non-IID data issues in federated learning.
Hyperparameter For local training, the settings are as follows: MNIST with = 5, B 10, = 5 x 10-3, FashionMNIST with E 5, B 100, n 2 x 10-4, CIFAR-10 and CIFAR- 100 with E = 5, applied. In the CRFed framework, key hyperparameters include maximum global rounds (Tc) set to 100. local training cycles (E₁) per global round set to 1, regularization coefficient (A) set to 0.1, dynamically adjusted confidence threshold (7), and client selection threshold (γ) initially set to 0.5. These parameters are fine-tuned based on preliminary experiments to ensure training efficiency and model performance. The experiments were conducted using NVIDIA GeForce RTX 4060 GPU, which has 8GB of VRAM. Detailed configurations of model structure are provided in A.G.
100, 7 = x 10-4 Momentum optimization with a coefficient of 0.5
Evaluation metrics The primary evaluation metrics for our experiments focus accuracy and the number of training rounds needed to reach convergence, addressing the challenges posed by non-IID data in federated learning- Accuracy is measured at the same training round across different models to ensure fair comparison. Convergence is assessed by the number of rounds required to achieve a target accuracy, which reflects the model's stability and efficiency. For the NIPD dataset,
7

---

we use mean Average Precision (mAP) as the evaluation metric. Following [Wang et all 2020]. all reported results are averaged over five runs with different random seeds account for variability.
4.2
Performance Comparison
Table I: Test accuracy of CRFed and the competing methods on five datasets. We run five trials with different random seeds and report the mean accuracy.
Scheme
MNIST
FashionMNNIST
CIFAR-10
CIFAR-100
NIPD (mAP)
FedAvg
[McMahu cl
0.976
0.847
0,650
0.821
FedProx [Lelal 202205
0.978
0.844
0,655
0.826
MOON Lict clall 202
0.980
0.846
0.674
0,372
0.836
FedGen
Nguyen
20211
0,982
0.862
0,672
0,369
0.841
HFMDS-FL
[Lieaal 40/22)
0.982
0.868
0.678
0.846
FRAug
[hen
0.981
0.865
0.675
0,374
0.851
G-FML
Yang all 21/2
0.983
0.870
0.681
0,378
0.854
FedCD
Long cI all, [1/K
0.982
0.867
0.677
0,376
0.861
FedNP
[Wi [1726)
0.982
0.869
0.680
0.377
0.863
FedDPMS
[KChen and Vikal A026)
0.983
0.876
0.680
0.386
0.871
CRFed
0.985
0.878
0.683
0.389
0.882
Accuracy comparison Table M presents the test accuracy of CRFed compared to several feder- ated learning algorithms under a highly heterogeneous setting (3 0.5). Our proposed method shows notable improvements over FedAvg [McMahan et al, | with relative gains of 0.9% on MNIST, 3.7% on FashionMNIST, 5.1% on CIFAR-10, 7.5% on CIFAR-100, and a significant 7.4% improvement in mAP the NIPD dataset. This highlights CRFed's robustness in handling non-lID data distributions. CRFed consistently outperforms other methods across the datasets, underscoring its effectiveness in federated learning scenarios with severe data heterogeneity.
Effeet of Data Heterogeneity We analyze the impact of data hetero- geneity on the performance of the top 5 models by varying the Dirichlet concentration parameter 3. Table E shows the performance these mod- els CIFAR- 100 and NIPD datasets 3 values ranging from 0.1 to 0.5. As expected, the performance gener- ally decreases with smaller 3 values due to increased data heterogeneity. Table 2 indicates that as 3 decreases, representing higher data heterogeneity, the performance of all models declines. CRFed consistently outperforms other methods across different settings, demonstrating its robustness in handling data heterogeneity. Notably, the relative performance gap between CRFed and other methods widens as decreases, highlighting its efficacy in more challenging federated learning scenarios.
Table 2: Performance of top 5 models CIFAR-100 and NIPD datasets under different & values.
CIFAR-100
NIPD (mAP)
Scheme
0.1
0.3
0.5
0.1
0.3
0.5
FedDPMS
0.270
0.330
0.386
0.751
0.810
0.871
FRAug
0.268
0.328
0.374
0.746
0.800
0.851
G-FML
0.265
0.329
0.378
0.748
0.802
0.854
FedCD
0.275
0.333
0.376
0.750
0.808
0.861
CRFed
0.280
0.345
0.389
0.760
0,820
0.882
Effect of Increasing Edge Nodes Figure presents the performance of the top 5 models on CIFAR-100 and NIPD datasets the num- ber of edge nodes K increases from 10 to 100. Across all models, performance generally im- proves with higher K values, reflecting better data utilization. Notably, CRFed shows the most significant gains, with accuracy increasing from 0.389 0.425 on CIFAR-101 100 and mAP from 0.882 to 0.920 on NIPD. This demon- strates CRFed's superior scalability and effec- tiveness in handling more edge nodes, mak- ing it robust in federated learning environments with increasing data sources.
Figure 4: Effect of Increasing Edge Nodes

---

Figure 5: Test accuracy across federated training rounds for top 5 models FMNIST, CIFAR-10. and CIFAR-106 datasets.
Convergence Rate The convergence perfor- mance of the top five models FMNIST, CIFAR-10, and CIFAR-100 datasets depicted in Figure B As observed, CRFed demon- strates significantly faster and more stable con- vergence compared to the competing methods across all three datasets. This superior perfor- mance is attributed the diffusion- harmonization mechanism, which effectively aligns data distributions, and the confusion- resistant strategy that selects reliable clients and adaptively adjusts learning rates, ensuring effi- cient and robust training even in highly hetero- geneous environments.
100
data
Figure 6: Ablation study results CIFAR- 10, CIFAR-100, and NIPD datasets. The har charts show the accuracy CIFAR-10 and CIFAR-100 datasets, while the line plot represents the mAP on the NIPD dataset.
4.3 Ablation Study
To evaluate the contribution of each component in the CRFed framework, we conduct an ablation study by removing altering specific components and observing the impact model performance. Removing the Indicator Function and using uniform sampling led to sigmificant performance drops, with CIFAR-10 10 accuracy falling from 0.683 to 0.661, CIFAR- 100 from 0.389 to 0.365, and NIPD mAP from 0.882 to 0.861. Excluding the Diffusion-based Data Harmonization (DDH) mechanism resulted in reduced accuracy CIFAR-10 (0.670), CIFAR- 100 (0.373), and NIPD mAP (0.870), highlighting its role in aligning data distributions. Replacing strategic client selection with random selection markedly decreased performance, emphasizing the importance of reliable client selection. Fixing the learning rate instead of adapting it slowed convergence and destabilized training. These findings validate the theoretical and practical significance of our proposed components in improving federated learning performance.
4.4 Comparison with Importance Sampling Methods
Previous importance sampling methods typically require prior analysis of the data relevance each client-side [[Hsu et all 2020, [aa et all, 2022) or necessitate deriving optimal sampling weights based on assumptions such as the convexity of the loss function er 2022, atll 2024]]. While these methods offer strong thcoretical guarantees, they are somewhat limited in their adapt- ability to real-world federated learning (FL) scenarios. For instance, both FedIR [[ss 2020 and Harmony [[lanaea 2022] assume that the server has knowledge the local distributions of all clients. Although this assumption does not violate the be challenging to obtain weal-world applications.
principles of FL, it can
In contrast, our CRFed does not depend on these assumptions. Instead, it iteratively adjusts the data distributions during the FL process itself, enabling the model to dynamically harmonize the diverse, non-IID data across clients without requiring explicit distributional assumptions centralized ac- cess to all client data distributions. Guided by the indicator function, our CRFed can derive the optimal sampling strategy for each local node.

---

Moreover, as shown in Ta empirical experiments demonstrate that the diffusion model achieves superior performance, outperforming other benchmark methods.
is worth noting that this comparison is not en- tirely fair, as each importance sampling method operates under different assumptions. For ex- ample, ISFL requires validation set up- date the empirical gradient Lipschitz constants for each local model, while FedIR requires all clients to upload the conditional distribution of images given class labels to match the target distribution. Nevertheless, our CRFed outper- forms the others even under less restrictive con- ditionsunlike ISFedAvg and ISFI,, it does not require assumptions about the loss function or gradient variance, and unlike FedIR and Har- mony, does not require centralized access to all client data distributions before calculating the importance sampling weights.
Table 3: The performance of different importance sampling methods on CIFAR- 100 under various values.
CIFAR-100
Method
3 = 0.1
B = 0.3
B = 0.5
ISFedAvg
0.232
0.285
0.305
ISFL
0.237
0.296
0.314
FedIR
0.258
0.311
0.352
Harmony CRFed
0.246
0.313
0.354
0.280
0.345
0.389
5 Conclusion
In conclusion, this study tackles the pressing challenge of handling non-i.i.d. data in federated learn- ing environments. We propose the CanfusionnResistaarn Federated Learning via Consistent Diffusion (CRFed) framework. This framework introduces a novel Indicator Function that dynamically adjusts sample weighting, facilitating self-paced learning paradigm that prioritizes more difficult samples over time. Additionally, our diffusion- aligned data distributions through iterative noise injection and denoising processes, mitigating the adverse effects of data heterogeneity. Our strategic client selection method, guided by the Indicator Function, ensures that the most reliable clients are chosen for training, thus improving the robustness and consistency of global model updates.
data harmonization mechanism ensures consistent and
Despite the promising results, our approach has certain limitations. The reliance on complex diffu- sion mechanisms and adaptive strategies may introduce computational overhead, which could a concern for resource-constrainct environments. Future work should focus optimizing the com- putational efficiency of the CRFed framework and exploring its applicability to a broader range of real-world federated learning scenarios [Zhang et all 20233].
6
Acknowledgement
This research was funded by the Basic Science Center Project for National Natural Science Foundation of China (Grant No: 720881011 the Xiangjiang Laboratory Major Project(Grant No: 23XJ01007), and the Fundamental Research Funds for the Central Universities of Central South University(Grant No: 1053320214050).
References
D. A. E. Acar, Y. Zhao, R. M. Navarro, M. Mattina, P. N. Whatmough, and V. Saligrama. Federated learning based on dynamic regularization. arXiv preprint arXiv:211 1.04263, 2021.
R. S. Antunes, C. André da Costa, A. Küderle, I. A. Yari, and B. Eskofier. Federated learning for healthcare: Systematic review and architecture proposal. ACM Transactions on Intelligent Systems and Technology (TIST), 13(4):1-23, 2022.
T. Castells, P. Weinzaepfel, and J. Revaud. Superloss: A generic loss for robust curriculum learning. Advances in Neural Information Processing Systems, 15:4308-4319 2020.
P. Chatterjee, D. Das, and D. B. Rawat. Federated learning empowered recommendation model for financial consumer services. IEEE Transactions on Consumer Electronics, 2023.
H. Chen and H. Vikalo. Federated learning in non-iid settings aided by differentially private syn- thetic data. In Proceedings of the IEEF/CVE Conference on Computer Vision and Pattern Recog- nition, pages 5026-5035, 2023.
10

---

H. Chen, A. Frikha, D. Krompass, J. Gu, and V. Tresp. Fraug: Tackling federated learning with non-iid features via representation augmentation. Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4859-4859, 2023.
M. Duan, D. Liu, X. Ji, Y. Wu, L. Liang, X. Chen, Y. Tan, and A. Ren. Flexible clustered federated learning for client-level data distribution shift. IEEE Transactions on Parallel and Distributed Systems,
2021.
Y. Fan, R. He, J. Liang, and B. Hu. Self-paced learning: An implicit regularization perspective. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31, 2017.
A. Ghosh, Chung, Yin, and Ramchandran. An efficient framework for clustered federated learning. Advances in Neural Information Processing Systems, 33: 13:19986-19559 2020.
T.M. H. Hsu, H. Qi, and M. Brown. Federated visual classification with real-world data distribution. In Computer Vision-ECCC 2020: 16th European Conference, Glasgow; UK, August 23-28, 2020, Proceediings, Part X 16, pages 76-92. Springer, 2020.
S. P. Karimireddy, S. Kale, M. Mohri, S. Reddi, S. Stich, and A. T. Suresh. Scaffold: Stochastic controlled averaging for federated learning. In International conference on machine learning, pages 5132-5143. PMLR, 2020.
A. Krizhevsky, G. Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEFE, 86(1 (1((1))227822))) 23224, 1998.
A. Li, L.. Zhang, J. Tan, Qin, Wang, and X.-Y. Li. Sample-level data selection for federated learning. In IEEE INFOCOM 2021 Conference on Computer Communications, pages 1-10. IEEE, 2021a.
L. Li. Y. Fan, M. Tse, and K.-Y. Y. Lin. review of applications in federated learning. Computers Industrial Engineering. 146:106854, 2020a.
Q Li, B. He, and D. Song. Model-contrastive federated learning. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 10713-10722, 2021b.
Q. Li, Y. Diao, Q. Chen, and B. He. Federated learning on non-iid data silos: An experimental study. In 2022 IEEE 38th international conference on data engineering (ICDE), pages 965-978. IEEE, 2022.
Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith. Federated optimization in heterogeneous networks. Proceedings of Machine learning and systems, 2:429-450, 2020b.
X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang. On the convergence of fedavg on non-iid data. arXiv preprint arXiv: 1907.02/8,, 2019.
2. Li, Y. Sun, J. Shao, Y. Mao, H. Wang, and Zhang. Feature matching data synthesis for non-iid federated learning. JEEE Transactions on Mobile Computing, 2024.
Y. Long, Z. Xue, L. Chu, T. Zhang, J. Wu, Y. Zang, and J. Du. Feded: A classifier debiased federated learning framework for non-iid data. In Proceedings of the 31st ACM International Conference on Mullimedia, pages 8994-9002, 2023.
M. Luo, F. Chen, D. Hu, Y. Zhang, J. Liang, and J. Feng. No fear of heterogeneity: Classifier calibration for federated learning with non-iid data. Advances in Neural Information Processing Systems, 34:5972-5984, 2021.
B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas. Communication-effiiiennn learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273-1282. PMLR, 2017a.
B. McMahan, F. Moore, D. Ramage, S. Hampson, and B. A. y Arcas. Communication-effcienn learning deep networks from decentralized data. In Artificial intelligence and statistics, pages 1282. PMLR, 2017b.
11

---

D. C. Nguyen, M. Ding. P. N. Pathirana. A. Seneviratne, and A. Y. Zomaya. Federated learning for covid-19 detection with generative adversarial networks in edge cloud computing. IEEE Internet of Things Journal, 9(12):10257-10277, 2021.
X. Pan, Yao, H. Kou, Wu, and Xiao. Harmonicnerf: Geometry-informed synthetic view augmentation for 3d scene reconstruction in driving scenarios. In ACM Multimedia 2024, 2023.
Redmon and A. Farhadi. Yolov3: An incremental improvement. arXiv preprint arXiv: 1804.800442220 02767, 2018.
E. Rizk, S. Vlaski, and A. H. Sayed. Optimal importance sampling for federated learning. In ICASSP 2021-2021 IEEE International Conference on Acousties, Speech and Signal Processing (ICASSP), pages 10995-3099 IEEE, 2021.
E. Rizk, S. Vlaski, and A. H. Sayed. Federated learning under importance sampling. IEEE Transac- fions on Signal Processing, 70:5381-5334 2022.
C. Tian, I.. L. Shi, Wang, and C. Xu. Harmony: Heterogeneity-awanl hierarchical management for federated learning system. In 2022 55th IEÉE/ACM International Symposium on Micrnarchi- tecture (MICRO), pages 631-645 IEEE, 2022.
T. Tuor, S. Wang, B. J. Ko, C. Liu, and K. K. Leung. Data selection for federated learning with relevant and irrelevant data clients, arXiv preprint arXiv:2001.08300, page 64, 2020.
S. Wan, J. Lu, P. Fan, Y. Shao, C. Peng, and K. B. Letaief. Convergence analysis and system design for federated learning over wireless networks. IEEE Journal on Selected Areas in Communica- tions, 39(12):3622-3633 2021.
H. Wang, Z. Kaplan, Niu, and B. Li. Optimizing federated learning on non-iid data with rein- forcement learning. In IEEE INFOCOM 2020-IEEE conference on computer communications, pages (698-17071 IEEE, 2020.
J. Wang, Q. Liu, H. Liang, G. Joshi, and H. V. Poor. A novel framework for the analysis and design of heterogeneous federated learning. IEEE Transactions on Signal Processing, 69:5234-5249, 2021.
X. Wu, H. Huang, Y. Ding, H. Wang, Y. Wang, and Q. Xu. Fednp: Towards non-iid federated learning via federated neural propagation. In Proceedings of the AAAI Conference Artificial Intelligence, volume 37, pages 10499, 2023.
C. Xiao and Y. Liu. A multifrequency data fusion deep learning model for carbon price prediction. Journal of Forecasting, 2024.
H. Xiao, K. Rasul, and R. Vollgraf. Fashion-mnis novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747 2017.
L. Yang, Huang, W. Lin, and J. Cao. Personalized federated learning on non-iid data via group- based meta-learning ACM Transactions on Knowledge Discovery from Data, 17(4):1-20, 2023.
Yao, Y. Lai, H. Kou, T. Wu, and R. Liu. Qe-bev: Query evolution for bird's eye view object detection in varied contexts. In ACM Multimedia 2024, 2024.
M. Ye, Fang. Du, C. Yuen, and D. Tao. Heterogeneous federated learning: Bate-of-the-aa and research challenges. ACM Computing Surveys, 56(3):1-44, 2023.
K. Yin, Z. Ding, Z. Dong, D. Chen, Fu, X. Ji, G. Yin, and Z. Wang. Nipd: A federated learning person detection benchmark based on real-world non-iid data. arXiv preprint MXXi:2336611993 2023.
M. Yurochkin, M. Agarwal, S. Ghosh, K. Greenewald, N. Hoang, and Y. Khazaeni. Bayesian nonparametrie federated learning of neural networks. International conference on machine learning, pages 7252-7261, PMLR, 2019,
12

---

L. Zhang, L. Shen, L. Ding, D. Tao, and L.-Y. Duan. Fine-tuning global model via data-free knowl- edge distillation for non-iid federated learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 00174-0113, 2022.
W. Zhang, X. Wang, P. Zhou, W. Wu, and X. Zhang. Client selection for federated learning with non-iid data in mobile edge computing. IEEE Access, 9:24462-24474. 2021.
Z. Zhang, X. Hu, J. Zhang, Y. Zhang. H. Wang, L. Qu, and Z. Xu. Fedlegal: The first real-world federated learning benchmark for legal nlp. In Proceedings of the 61st Annual Meeting of the Association for Compulational Linguistics (Volume 1: Long Papers), pages 3492-3507, 2023.
Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, and V. Chandra. Federated learning with non-iid data. arXiv preprint arXiv: 1056::88000005500 00582, 2018.
H. Zhu, Xu, Liu, and Y. Jin. Federated learning on non-iid data: survey. Neurocomputing, 465:371-390, 2021.
Z. Zhu, Y. Shi, P. Fan, C. Peng, and K. B. Letaief. Isfl: Federated learning for non-iid data with local importance sampling. IEEE Internet of Things Journal, 2024.
13

---

A Appendix
A.1 Supplementary Explanation of the Diffusion-based Data Harmonization Mechanism
Forward Diffusion Process In the forward diffusion process, Gaussian noise with variance 33 E (0, 1) is added gradually to the sample Xy for T steps. The process is defined as:
T ((11T1Z) = II 1=1
(2) 21-1),
(16)
q(z zt-1) NZz v1 1
(17)
Using notations az, 1 Bg and ae = =II ass, the sample 2, can be defined directly
2, v von4 vI ā,e, e~~(0,11)
(18)
Reverse Denoising Process The reverse denoising process aims to sample reversely from 2t through transition probabilities g(Ze-i/zd for timesteps T 1 through 1 to obtain sample drawn from (z;). The transition (Zz (z) a Gaussian distribution, tractable when conditioned on 2;:
q 1 Ze. 2,) N( 24). Bc))
(19)
where the mean i and variance 3, calculated as:
at
i. 2.)
23 v1 Cs. 61
(20)
Vaz
B. 1 ---11 Be. āi
(21)
The reverse transition probability pe (p: (z)) relies on the entire data distribution and is approxi- mated through neural network:
Po (z 21) = M(zz-1;H4(zz t),
(22)
where Eo(,, () = 3.I and the mean M4g (ze, depends on a noise sample eg (zz. () learned by
neural network. The learning process is guided by the objective function:
L Er, [[|e Eo(2,, t)
(23)
while the output sample is obtained as:
Uut co(2),)) + 0,2,
24
2t
(24)
VI
where z ((0 and otherwise.
A.2 Proof of Theorem 3.1
Proof. For convenience, define w; w.(6.) = of (4- log k, ,whcch can be regarded of the effective weight induced by the indicator function Ix(l,, a,). It appears in the local update of client i. Hence, the update rule can be written as:
Oc+1 0, mw Valy.
(25)
Mcanwhile, assume the local loss function 4(0) satisfies two common assumptions:
14

---

1. L-smoothness (Lipschitz continuity of gradients): The gradient of (0) is L-Lipschitz i.e.
V1:(0)) (1,()))) S L 1 02ll
for all 0,, 02.
(26)
2. Bounded below: There exists constant such that
(()) 2 17
(27)
for all A.
From L-smoothness, we have for the update from iteration to + 1:
1(((+1) & (((0)) + (V)(()), 8.11 - 01) + マ 0c+1 - 0ell2
(28)
Using (25), we get
0.+1 - 0. - (o.)
(29)
Substituting back yields
L
4(0++1) S 4(00) nww [V((0)))1 + 2 1 wf [VV1(0,))
L ((0)) (nws 2 1² w/ w?? (1))1)
(30)
we ensure 7)4 S 7. or more conservatively no 5 21 then
1
1) wi
11 w/
2
(31)
2
Hence, (50) implies
1
4(0+41) 1 4(6,) -
2 n wi [4,()))
(32)
Therefore, if 17 is chosen such that 17 < 21 then (00) is guaranteed to decrease every step. forming a monotonically decreasing sequence.
By the bounded-below assumption (,(0) 2 17, the loss cannot decrease indefinitely. Consequently, (1, (0.)) converges. From standard telescoping sum argument, at iteration T, we estimate the sum of squared gradient norms:
1(00))4((0m) 4(0+11) 4(0.) 2
2 [4(00) 12
<
(33)
Wi
7 min; (wi (0.)
t=
1=0
minn((w((.)) and is properly chosen, then the average gradient norm goes to as -> oc, ensuring global convergence.
In FedAvg, the single-step update typically has the form 0c+1 = 0, FedAAv Vol, (0.). CRFed, we have additional factor (..). Thus the effective learning rate becomes 7er (0,). If (.)).) < (which can be ensured by proper design of the indicator function many scenarios), then Mat 17, i.e., CRFed is more conservative (or adaptive) in its step size. This helps avoid large updates caused by data heterogeneity and promotes more stable convergence. Consequently. CRFed obtains tighter convergence bound because it mitigates the gradient-directional deviation brought on by heterogeneous data.
15

---

A.3 Proof of Theorem 3
Proof. Recall that the Indicator Function is defined I14,, 0.) (4- - A (log (logg;)). o,)², we wish to find the optimal of that minimizes this function for a given 4,
To simplify the differentiation, introduce two transformations:
4 and I₃ = log 0s.
(34)
Hence, we have
(35)
Under these new variables, the Indicator Function becomes
I44, oz) = (4 - 7) e + X (xx)²² = 2 + xf).
(36)
Since is just a constant multiplier, minimizing 1, is equivalent to minimizing (xx)
+ 27.
(37)
We now compute the derivative
(x;) with respect to v, and set it to zero to find the critical points:
d
G th + xf]
Gi e th
2x 0.
(38)
dz.
Rearrange this to isolate exponential terms:
c -22
(39)
Note that this step implicitly assumes X; < 0 if the right-hand side is negative, depending on the sign of c. We proceed manipulate this into a standard Lambert form. Multiply both sides by 2 e
2 G GG22. 20.
(40)
Let Then,
y = 2; :
(41)
y z/2 c 20.
(42)
Meanwhile, by definition of y, Hence, we arrive at
y = 24 =t
(43)
32
24 =
4
2x
(44)
where W(-) is the Lambert W function, the inverse function of + 2
In practice, the argument of the Lambert W function must lie a domain where the function real-valued. By restricting
4-72 -2 e
(45)
we ensure that
2 1 E'
(46)
2X 2
Therefore, to handle the case when 4 T
< 6.
we take
max
4 4-).
(47)
which ensures the Lambert W functions argument stays within the valid real domain. Consequently, the solution for x; becomes
24 = - w( Fx max( -2, 4 43.4-)))
(48)
Recalling a; e we conclude:
(1, exp( W( max(-2, 4, -7))
(49)
16

---

A.4 Design and Training Details of the Model Encoder and Distribution Decoder
In the CRFed framework, both the model encoder and distribution decoder play crucial roles in ensur- ing effective data harmonization and robust model updates. These components are implemented us- ing autoencoder architectures, designed compress and reconstruct data representations efficiently.
A.4.1 Model Encoder
The model encoder E' is responsible for compressing the global model parameters 6, into a lower- dimensional meta-model representation Φt. The encoder architecture comprises several fully con- nected layers activated by ReI.U functions, followed by linear transformation layer to produce the final compressed representation.
Mathematically, given the input global model parameters 0, E L2, the encoder outputs a compressed representation Ø: E(O,) Idd where d' < d. The transformation is defined as follows:
h1 ReLU + h22 ReLu + by) = ReLU(W +
Wouthe + bout
(50)
The training of the model encoder involves minimizing the mean squared error (MSE) between the original model parameters and their reconstructions. The loss function is given by:
LE
Σ
02 0,,1
(51)
where 6, E- and N the number of samples. The optimization is performed using the Adam optimizer with learning rate n,. The detailed architecture includes an input layer of size d, hidden layers of sizes [128, 64, 32), and an output layer of size d' 16.
A.4.2 Distribution Decoder
The distribution decoder D aims to transform the denoised latent representations z, back into the desired data distribution. Like the encoder, the decoder uses a series of fully connected layers with ReLU activations, culminating in linear layer to reconstruct the data.
Given the input latent representation 2; E Rd", the decoder outputs the reconstructed data & D(z)) Rd The transformations are defined
h1 = ReLU(W[(22 + b2) = ReLU + bb)i = ReLU(W +1)8 = W te + blout
(52)
The training process for the distribution decoder also minimizes the MSE, defined as:
CD
X1.
x1,
(53)
where R2, = (zz,) and is the number of samples. The optimization employs the Adam optimizer with learning rate 17. The architecture details include input layer of size d' 16. hidden layers of sizes [32, 64, 128), and an output layer size d.
A.4.3 Architectural Details
The model encoder and distribution decoder share similar architectural approach, emphasizing efficient compression and reconstruction through deep learning techniques. The key parameters for both autoencoders are summarized follows:
Model Encoder:
Input layer size: d
Hidden layer sizes: [128, 64, 32]
Output layer size: d' = 16
17

---

Activation function: ReLU
Learning rate: 0.001
Batch size: 32
Distribution Decoder:
Input layer size: d' 16
Hidden layer sizes: [32, 64, 128]
Output layer size:
Activation function: ReLU
Learning rate: 0.001
Batch size: 32
These architectural and training details ensure that the CRFed framework effectively handle non- i..d. data distributions, facilitating robust and consistent model updates across federated learning environments.
A.5 Pseudocode for CRFed
The complete computational process of CRFed is illustrated in Algorithm M
Algorithm 1 ConffusionRReistaan Federated Learning via Consistent Diffusion (CRFed)
Require: Maximum global rounds Ta, local training cycles E;, client weights {mx}. local datasets {D learning rate Tlu indicator function threshold y
1: Initialize global model parameters Og
2: Initialize local model parameters {6)) and importance sampling weights {wf 1}
3: Set <
4: while To Es do
5: for each client i do
6: Sample local data D.ampled based on importance weights w
7: Train local model 0 on D. campled
8:
end for
9:
if
mod E == 0 then
10:
Each client uploads local model {0F] } to the server
Server aggregates the global model: 0. +- Σk= T
12:
for each client & do
Compute optimal indicator function /"
14:
Calculate optimal sampling probability P. = ReverseDennise((F)
15:
Calculate importance sampling weights w; 7h
Pr
Sample new local data Daampled based on updated importance weights w Train local model 0 on D sampled
17:
18:
Update local model 8% +- 0g
19:
Adjust learning rate: 17, 170 maxy (y
end for end if
21:
t+1
23:
end while
Output: Global model 0,, local models {0/}
A.6 Detailed Model Structure
The detailed configuration of the models used in experiments is provided below. Each table outlines the layers and parameters for the respective datasets.
All weights are initialized with normal distribution (mean 0, standard deviation 0.1) and biases with a constant value of O.I. These settings ensure that the models are well-prepared for training and capable of achieving high performance the respective datasets.
18

---

Table 4: Model structure for MNIST and FashionMNIST datasets
Layer
Type
Output Channels/Units
Additional Information
Input
28x28 grayscale images
Convolutional
16
5x5 filters, stride I, padding 'SAME'
Activation
ReLU
Max Pooling
2x2 window, stride 2
Convolutional
32
5x5 filters, stride I, padding 'SAME'
Activation
ReLU
Max Pooling
2x2 window, stride 2
Fully Connected
512
Activation
ReLU
Fully Connected
10
Softmax
Table 5: Model structure for CIFAR-IO and CIFAR-100 datasets
Layer
Type
Output Channels/Uniss
Additional Information
Input
32x32 RGB images
Convolutional
64
5x5 filters, stride padding 'SAME'
Activation
ReLU
Max Pooling
2x2 window, stride 2
Convolutional
64
5x5 filters, stride padding 'SAME'
Activation
ReLU
Max Pooling
2x2 window, stride 2
Fully Connected
1600
Activation
ReLU
Fully Connected
512
Activation
ReLU
5
Fully Connected
10 (CIFAR- 10) 100 (CIFAR- 100)
Softmax
In the NIPD dataset, adopted the classic YOLOv3 [Redmon and Parhadi MUK model.
19

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
The main claims in the abstract and introduction are supported by detailed descriptions, empirical evaluations, and theoretical analysis provided in the body of the paper (Sections 1, 3, and 4).
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
fine to include aspirational goals motivation as long it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The paper discusses the limitation in Section 5, addressing potential biases
from large language models.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The au- thors should reflect on how these assumptions might be violated practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the ap- proach. For example, a facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to ad- dress problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't it acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
20

---

Answer: [Yes]
Justification: The paper includes a detailed proof for the convergence theorem in Section 3, providing all necessary assumptions and complete proof.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theo- rems.
The proofs can either appear the main paper or supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should comple-
mented by formal proof' provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
The paper specifies the datasets used, the training and testing details, the hy hyperparameters, and the evaluation metrics Section 4, ensuring that the experiments can be reproduced.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all sub- missions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results way to re- produce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case au- thors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
21

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
The data and code is provided.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
mublic/guides/Cie www.oooSaaaiaioooooooooooo
for more details.
While we encourage the release of code and data, understand that this might not be possible, so No is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (ittps:
//nips.com
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specifyy all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The paper provides comprehensive details the training and testing setups, including data splits, hyperparameters, and optimizer settings in Section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropri- ate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Since some baselines involve randomness using k-means, the paper did 95% significance test using 10 repeated results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoo bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
22

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK report I-sigma error bars, but one should state it. The authors should prefer- ably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: The paper provides details the computational resources used, including the
type of GPUs and the total amount of compute required for the experiments, as mentioned in Section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics httpps//eeerrppo.ccpoooo/////sts http///pwbbbbieiiiiicciieeeeeeer
Answer: [Yes]
Justification: The research adheres to the NeurIPS Code of Ethics, ensuring transparency, reproducibility, and consideration of ethical implications throughout the study.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper discusses the potential positive impacts of improving clustering techniques for various applications and mentions possible negative impacts such as biases introduced by large language models and the risk of misuse in surveillance, along with mitigation strategies in Scction
23

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact spe- cific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitiga- tion strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper does not involve the release of data models that have high risk for misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by re- quiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The paper properly credits the creators of the datasets and models used, and mentions the licenses and terms of use, as detailed in Section
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
24

---

The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license. copyright information, and terms of use in the pack- age should be provided. For popular datasets, paperswi thcode cow /ataattsstt has curated licenses for some datascts. Their licensing guide can help determine the li- cense of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors are encouraged reach out to the asset's ss creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documenta- tion provided alongside the assets?
Answer: [Yes]
Justification:
The new datasets and code introduced the paper are well documented.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the pa- per include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contri- bution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, tion, or other labor should be paid at least the minimum wage the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
25

---

Depending on the country in which research is conducted, IRB approval (or equiva- lent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do include any information that would break anonymity (if applicable), such as the institution conducting the review.
26


Reviews:
[{"review_id": "FMjpRIlus7", "reviewer": "Reviewer_nZWV", "summary": "The work proposes a new FL approach, called CRFed, for addressing data heterogeneity in FL settings. CRFed relies on a diffusion based approach for harmonizing clients data heterogeneity  by performing data noise injection and iterative denoising, followed by a curriculum learning approach, which employs an indicator function to assign weights to training samples and indicate samples selection. The authors conduct of number of experiments across various domains and FL environments to showcase the promise of their approach.", "questions": "Comments, textual corrections and typos:\n- Is the definition of the indicator function and the use of Lambert function based on previous work? Please cite accordingly and explain why these formulas were used. Moreover, why there is a substraction of confidence from the loss value?, also variables' reported value range in the indicator function is not clear (i.e., range of $\\sigma_i, \\tau$).\n- Why the adaptive learning rate is adjusted based on the clients' indicator function? this is not clear at all.\n- How often is the noise/denoise operation performed? At the beginning of training, at every federation round or after r-rounds? If it is done post-initialization then have you measured the effect of the approach in terms of wall-clock time? How expensive is the proposed approach in terms of time convergence (not round)? It would be great if you could include these results as well.\n- Figure 4's is not readable. What is the $\\beta$ value used for the two domains? Moreover, have you created 100 partitions and sub-sampling clients at each round or consider all available clients at every round?\n- Please fix your citations style and add missing space between the text and the reference throughout the paper.", "limitations": "No.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- A self-paced (curriculum) learning approach for each client based on samples' loss and uncentrainty.\n- Thorough empirical evaluation against many existing methods in multiple federated environments.\n- Ablation results show the importance of each proposed component (indicator function, diffusion mechanism, client selection)", "weaknesses": "- Some of the applied methods seem very ad-hoc and further elaboration is needed on their selection.\n- More information is needed on the update frequency of the harmonization approach and respective curriculum sequence of the clients' training samples.\n- Lack of theoretical framework limits the contribution of this work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# W1: Further elaboration on the selection of the methods\n\n**Response:** Thanks for pointing this out. Below, we offer a detailed explanation and theoretical basis for the key methods employed in CRFed framework.\n\n1. The Indicator Function $I_{\\lambda}(l_i, \\sigma_i)$ is designed to dynamically adjust sample weighting based on loss values and uncertainties, **inspired by self-paced learning** [1]. **The construction of the indicator function is grounded in the principles of curriculum learning**, which prioritizes simpler tasks before progressively addressing more complex ones. The design of the Indicator Function $I_{\\lambda}(l_i, \\sigma_i)$ is intended to improve training convergence efficiency. The proof of this assertion can be found in global rebuttal.\n\n2. The diffusion-based data harmonization mechanism is the core to our approach, aiming to mitigate data distribution disparities. This method is based on the principles of denoising diffusion probabilistic models (DDPMs), which have demonstrated success in various generative tasks. The iterative process of noise addition and removal aligns data distributions effectively, reducing heterogeneity across clients. **Through the diffusion-based data harmonization mechanism, we can ensure that the data distributions of all clients gradually converge during the noise addition and removal processes**, thereby reducing the impact of data heterogeneity on model updates.\n\n# W2: More information on the update frequency of the harmonization approach and respective curriculum sequence of the clients' training samples.\n\n**Response:** Thanks for the suggestion. In the CRFed framework, the diffusion-based data harmonization process is integrated into each global aggregation round. This means the harmonization update occurs once per global round, ensuring timely adjustments to data distributions across clients.\n\n- **Maximum global rounds ($T_G$):** 100\n- **Local training cycles per global round ($E_l$):** 1\n\nOur curriculum learning approach involves dynamically adjusting the sampling weights of clients' training samples based on their difficulty, as measured by the Indicator Function $I_{\\lambda}(l_i, \\sigma_i)$. This function is recalculated at each local training cycle to reflect the latest state of the global model. The curriculum sequence progresses from easier to more difficult samples, facilitating a self-paced learning paradigm.\n\n# W3: Theoretical framework.\n\n**Response:** Thanks for the suggestion. In the global rebuttal, we have added theoretical analysis.\n\n# Q1：More discussion on the Indicator Function.\n\n**Response:** We appreciate the reviewer's detailed comments. Our indicator function, inspired by confidence-aware cross-entropy [2], includes a loss-amplifying term and a regularization term to amplify high-loss samples' contributions while regularizing uncertainty estimation. The Lambert W function helps find the optimal uncertainty $\\sigma_i^*$, ensuring difficult samples are appropriately weighted during training. Subtracting the confidence threshold $\\tau$ from the loss value $l_i$ centers the loss values, making it easier to prioritize difficult samples (i.e., those with $l_i > \\tau$). The uncertainty $\\sigma_i$ ranges from $10^{-3}$ to $10$ and $\\tau$ is dynamically adjusted based on the weighted average loss, initially set to 1.0 and updated every 10 rounds to reflect the dataset's evolving difficulty. We will include these details in future versions of the manuscript. Thank you for your valuable feedback.\n\n# Q2: Discussion on the adaptive learning rate.\n\n**Response:** Thank you for your comment. Adjusting the learning rate $\\eta_i$ based on $I_{\\lambda}(l_i, \\sigma_i)$ ensures that clients with more challenging data receive higher learning rates, allowing significant updates. This balances the learning pace among clients, preventing dominance by any single client and ensuring equitable contributions. Higher indicator function values typically correspond to more difficult or uncertain data, leading to slower convergence with a uniform learning rate. Increasing the learning rate for these clients accelerates their learning, speeding up overall convergence.\n\nAdditional experiments (see **Table 1 on the one-page pdf**) show that models with adaptive learning rates based on the indicator function demonstrate faster convergence and higher final accuracy compared to fixed learning rates.\n\n# Q3. The principle and details of noise/noise operation.\n\n**Response:** Thank you for your comment. \n\nThe noise/denoise operation in the CRFed framework is performed at every global federation round. This ensures consistent alignment of data distributions across clients, maintaining effective model updates. After each global model aggregation, noise is added and the denoising process adjusts the data distributions before the next round of local training begins.\n\nWe have measured the wall-clock time to evaluate the computational overhead introduced by the proposed approach, results are shown in **Table 2 on the one-page pdf**.  Although our system involves additional computations per round,  he total convergence time is comparable. \n\n# Q4. Discussion on the $\\beta$ and experimental details.\n\n**Response:** Thank you for your comment. In our experiments, we used the Dirichlet distribution to create non-IID data partitions. The $\\beta$ value, which controls the degree of data heterogeneity, was set to 0.5 for both domains in the experiments presented in Figure 4. We have considered all available clients at every round of federated learning in our experiments. We hope these clarifications address your concerns.\n\n# Q5. Formatting issue.\n\n**Response:**  Thank you for pointing out this. We will fix the citations style in future version.\n\n# Reference\n[1]  Self-paced learning: An implicit regularization perspective. AAAI, 2017.\n\n[2] Data parameters: A new family of parameters for learning a differentiable curriculum. Neurips, 2019.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sJP1nVemW7", "reviewer": "Reviewer_cbew", "summary": "This paper introduces CRFed, a framework designed to handle the challenges of non-i.i.d. data in federated learning. By using a diffusion-based data harmonization mechanism and a confusion-resistant strategy, CRFed aims to reduce data distribution differences among participating nodes and improve model consistency. Extensive experiments show that CRFed significantly enhances accuracy and convergence speed compared to existing methods.\nIn my opinion, the diffusion-based data harmonization mechanism is an innovative approach to dealing with data distribution differences. This paper not only introduces new theoretical concepts but also validates them through comprehensive experiments, making it a significant contribution to the field.", "questions": "Please refer to the weaknesses.", "limitations": "The limitations are discussed in the paper by the authors. There is no potential negative societal impact.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The paper exhibits several strengths that highlight its contributions and impact on the field of federated learning:\n1. The introduction of a diffusion-based data harmonization mechanism is a fresh approach to tackling data distribution disparities. I think this idea has great potential to improve the stability of learning despite client heterogeneity. There's a mapping relationship between local and global data distributions, and using a diffusion model to capture this relationship is really innovative.\n2.The paper conducts extensive experiments on various non-i.i.d. datasets, including MNIST, FashionMNIST, CIFAR-10, CIFAR-100, and NIPD. Especially with NIPD, which is a very challenging dataset, I congratulate the authors for achieving impressive performance on it.\n3.The framework's ability to handle an increasing number of edge nodes and its adaptive learning rate adjustment contribute to improved scalability and training efficiency. This makes CRFed a practical solution for real-world federated learning scenarios.", "weaknesses": "There are some areas where the paper could be improved to enhance its clarity, robustness, and applicability:\n1.The method involves several hyperparameters, like the variance of Gaussian noise, the regularization coefficient, and the dynamically adjusted confidence threshold. A more thorough sensitivity analysis of these parameters would make the paper more complete.\n2.In cases where data distributions are extremely diverse or have intrinsic properties that are difficult to capture through these processes, the effectiveness of the harmonization might be limited. \n3.While the method aims to enhance model consistency and reduce data disparities, the additional communication overhead introduced by the diffusion mechanism and importance sampling is not discussed.\n4.Theorem 3.1 is valuable. However, during the derivation process, the purpose of equation (3) needs to be clearly explained.\n5.\"I suggest highlighting the data showing performance advantages in Table 2 in bold.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# W1. Sensitivity analysis.\n\n**Response:** We appreciate the reviewer's insightful comment regarding the sensitivity analysis of our hyperparameters. To address this, we conducted additional experiments to analyze the sensitivity of the key hyperparameters in our CRFed framework, namely the variance of Gaussian noise ($\\beta_t$), the regularization coefficient ($\\lambda$), and the dynamically adjusted confidence threshold ($\\tau$).\n\nWe performed the sensitivity analysis on the CIFAR-10 and CIFAR-100 datasets, which were also used in the original experiments. The default values for the hyperparameters in our original experiments were:\n\n- Variance of Gaussian noise, $\\beta_t$: 0.1\n- Regularization coefficient, $\\lambda$: 0.1\n- Dynamically adjusted confidence threshold, $\\tau$: dynamically adjusted starting from 0.5\n\nWe varied each hyperparameter while keeping the others fixed to their default values and measured the test accuracy. The results are summarized in  **Table 3,4,5 on the on-page pdf**. The results indicate that our CRFed framework is relatively robust to variations in these hyperparameters.\n\n# W2. Additional experiments addressing scenarios where data distributions are extremely diverse.\n\n**Response:** We appreciate the reviewer's concern regarding the effectiveness of our harmonization mechanism in handling extremely diverse data distributions. To address this, we have conducted additional experiment to evaluate and demonstrate the robustness and adaptability of our CRFed framework under such challenging conditions.\n\nTo empirically validate the effectiveness of our approach under extreme data heterogeneity, we conducted additional experiments using the CIFAR-100 dataset with even smaller Dirichlet concentration parameters ($\\beta$) to simulate highly imbalanced and diverse data distributions. Specifically, we set $\\beta$ to 0.01 and 0.05 to create scenarios with extreme non-IID characteristics. The results are shown in **Table 6 on the one-page pdf** and demonstrate that CRFed framework significantly outperforms the baseline FedAvg method even under extreme data heterogeneity. \n\n# W3. Discussion on additional overhead.\n\n**Response:** Thank you for your comment.  We have measured the wall-clock time to evaluate the computational overhead introduced by the proposed approach.   Results are shown in **Table 2 on the one-page pdf**. We observe that the wall-clock time per round for CRFed is slightly higher compared to other methods due to the additional noise/denoise operations. However, the total convergence time is comparable to other methods. Although CRFed involves additional computations per round, the convergence in terms of accuracy and model robustness is achieved efficiently. The slight increase in computational cost is balanced by the improved model performance, making it a worthwhile trade-off.\n\n# W4. Discussion on equation (3).\n\n**Response:** We thank the reviewer for recognizing the value of Theorem 3.1. We acknowledge that the purpose of equation (3) in the derivation process may not have been sufficiently clear. The purpose of this equation is to define the indicator function $I_{\\lambda}(l_i, \\sigma_i)$, which measures the reliability of each sample based on its loss value $l_i$ and associated uncertainty $\\sigma_i$. This function is critical for the self-paced learning mechanism in our framework, where samples are prioritized based on their difficulty and uncertainty.\n\n# W5. Formatting issue.\n\n**Response:** Thanks for the suggestion. In future versions, we will make the corresponding annotations.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BZ1P5S7DGj", "reviewer": "Reviewer_FLG7", "summary": "This paper presents a framework called CRFed to address the significant challenges posed by non-i.i.d. data in federated learning environments. This work introduces a diffusion-based data harmonization mechanism that effectively reduces disparities in data distributions across different nodes. Additionally, the paper proposes a confusion-resistant strategy that leverages an adaptive indicator function based on importance sampling.\nOverall, this paper's writing is clear and easy to follow. The figures are well-drawn, allowing for a quick understanding of the research motivation and methodological design. The core contribution of this paper is the introduction of a diffusion-based data harmonization method to obtain valuable data distribution for global model aggregation, which is a very brave and innovative idea. Additionally, how to use synthetic data to enhance training effectiveness has long been an open question in the field of FL, and this work clearly provides a very promising approach. Therefore, I recommend that this paper be accepted.", "questions": "See above.", "limitations": "The authors have thoroughly outlined the limitations of their work as well as the potential negative societal impacts.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The method described in the paper is presented very clearly, the formulas are well-expressed, and the charts are clear.\n2. The diffusion-based data harmonization mechanism is especially creative. This method uses Gaussian noise injection and iterative denoising to gradually align local data distributions with a desired global distribution. This process is clearly explained in Equations (7) to (12) and greatly reduces the impact of data differences. The detailed explanation of the forward and reverse processes, as shown in Figure 2, highlights a smart and practical way to handle non-i.i.d. data.\n3.  The results in Tables 1 and 2 consistently show that CRFed outperforms other methods in both accuracy and convergence speed. The detailed comparison with various state-of-the-art methods under different $\\beta$ values and edge node configurations highlights the practical applicability and scalability of the proposed framework.", "weaknesses": "1. Some notations could be more clearly defined. For instance, in the definition of the Indicator Function $ I_\\lambda (l_i, \\sigma_i) $, it's not immediately clear how $\\tau$ (the confidence threshold) is dynamically adjusted or chosen. A more detailed explanation of how $\\tau$ impacts the learning process and its optimal selection criteria would be beneficial.\n2. The iterative denoising process might face scalability issues with very large datasets or a high number of iterations. The paper should evaluate the performance and efficiency of the denoising steps in such scenarios to understand the method’s scalability better.\n3. It would be beneficial to include a brief discussion on future work in the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# W1. Some notations could be more clearly defined. \n\n**Response:** Thanks for pointing this out. In our framework, $\\tau$ represents a confidence threshold that determines the difficulty level of samples based on their loss values. The term $(l_i - \\tau) \\sigma_i$ in the Indicator Function adjusts the weight of each sample based on its loss value $l_i$ relative to the threshold $\\tau$. This adjustment mechanism prioritizes samples with higher loss values (more difficult samples) when $l_i > \\tau$, giving them higher weights, while samples with lower loss values (easier samples) are given lower weights. The impact of $\\tau$ on the learning process is twofold:\n\n- By setting an appropriate $\\tau$, the model can focus on learning from more difficult samples first, thereby implementing a form of curriculum learning. This helps the model to progressively handle more complex patterns in the data, leading to improved generalization and robustness.\n- $\\tau$ can be dynamically adjusted during the training process to reflect the model's evolving understanding of the data. Initially, $\\tau$ can be set to a lower value to focus on easier samples, and as training progresses, $\\tau$ can be increased to prioritize more difficult samples. This dynamic adjustment ensures that the model continually challenges itself, preventing stagnation and promoting continuous improvement.\n\n# W2. Evaluation the performance and efficiency of the denoising steps.\n\n**Response:** We appreciate the reviewer's concern regarding the scalability of the iterative denoising process, especially when dealing with very large datasets or a high number of iterations. To address this, we have conducted additional experiments to evaluate the performance and efficiency of the denoising steps under such scenarios.\n\nWe conducted experiments on the CIFAR-100 and NIPD datasets, varying the dataset size and the number of iterations in the denoising process. The key parameters evaluated include:\n\n- Dataset sizes: Full CIFAR-100 (50,000 samples) and NIPD (80,000 samples)\n- Number of denoising iterations: 10, 50, 100, 200\n\nThe results  are shown in **Table 7 on the one-page pdf**. We can find that the performance improvements (in terms of accuracy for CIFAR-100 and mAP for NIPD) saturate as the number of iterations increases. These results demonstrate that while the iterative denoising process is effective, the benefits in performance diminish beyond a certain number of iterations. Therefore, for practical applications, we recommend limiting the number of denoising iterations to around 100, where a good balance between performance and efficiency is achieved. This approach ensures that the method remains scalable even for large datasets.\n\n# W3. A brief discussion on future work.\n\n**Response:**  Thanks for the suggestion. Future work could explore more sophisticated noise injection techniques in the diffusion-based data harmonization process. For instance, adaptive noise schemes that consider the specific characteristics of local data distributions could potentially improve the alignment of data across clients. We will include relevant discussions in future version to inspire further work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4iRolCkLA6", "reviewer": "Reviewer_CpsC", "summary": "This paper proposes an importance sampling method with a diffusion model to achieve data harmonization in federated learning with non-i.i.d. data. The proposed method utilizes the indicator function from self-paced learning to measure the reliability of loss on each client and calculates the optimal data distribution according to the indicator function. The proposed method is evaluated in five different datasets empirically.", "questions": "1.\tWhy is the optimal uncertainty obtained by minimizing the indicator function? \n\n2.\tHow is the distribution decoder trained? Since the decoder should output an unknown distribution from the denoised latent representation, there should not be a ground truth $x_{t_i}$ when training the decoder.\n\n3.\tAre the diffusion model, the model encoder, and the distribution decoder trained locally or globally? What is the training overhead of these components?", "limitations": "See the weaknesses and questions above.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "This paper proposes a novel method to measure the importance and difficulty of each data sample, and samples the local data with importance sampling to achieve lower model update divergence. Experiments are conducted on five different datasets, with a comparison to ten other baselines, sufficiently showing the superior performance of the proposed method.", "weaknesses": "Despite the sufficient experimental results, the poor clarity of this paper weakens its soundness. There are several confusing points in the context of this paper:\n\n1.\tIt is unclear what the indicator function means and how it is derived. As the core of this paper, the indicator function is only provided with a formulation and never explained how it is derived. Additionally, it is not straightforward to understand the relationship between the indicator function and the optimal data distribution.\n\n2.\tHow the diffusion model is trained and used is not discussed in the paper. What we know from the paper is that the diffusion model takes the model embedding and the indicator function together to generate the optimal data distribution. It is weird why the diffusion model is necessary here. It seems that what we need is only a method that can estimate the optimal data distribution according to the indicator function.\n\n3.\tThe definitions of the indicator function in Eq. (1) and Eq. (16) seem to be inconsistent. The indicator function is calculated for each sample in Eq. (1), while it is calculated for each client in Eq. (16). \n\nAfter all, the core idea of this paper is simple: re-assigning sampling probability to achieve overall balanced training data distribution such that the model update divergence can be mitigated. It is doubtful whether such a complicated framework is necessary to achieve this goal, while the improvement seems to be limited in the final converged accuracy.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# W1. Discussion on Indicator Function.\n\n **Response:** We appreciate the reviewer's insightful comments. Below is a comprehensive explanation:\n\n(1) **Motivation of the Indicator Function**.   The Indicator Function $I_{\\lambda}(l_i, \\sigma_i)$ is designed to dynamically adjust sample weighting based on loss values and uncertainties, **inspired by self-paced learning** [1]. It follows curriculum learning principles, prioritizing simpler tasks before more complex ones, to improve training convergence efficiency. **The proof of this assertion can be found in global rebuttal**.\n\n(2) **Derivation of the Indicator Function**. Our design is heuristic, similar to confidence-aware cross-entropy [2]. It consists of a loss-amplifying term and a regularization term, enhancing high-loss sample contributions while regularizing uncertainty estimation. To further discussion, see the convergence speed discussion of CRFed with FedAvg in the global rebuttal.\n\n(3)  **The Relationship Between the Indicator Function and the Optimal Data Distribution.** Please refer to the **response to Q1** for detailed explanation.\n\n# W2. Necessity of diffusion model\n\n **Response:** Thanks for your comment. In CRFed, the diffusion-based data harmonization process is integrated into each global aggregation round. This means the harmonization update occurs every time the global model is aggregated and broadcasted to the clients, specifically set to once per global round. The model is trained online during the federated learning process.\n\nTo address the reviewer's concern, we conducted comparative experiments with alternative methods to estimate the optimal data distribution based on the indicator function:\n\n- **Simple Weighted Sampling (SWS)**: Directly adjusts sampling probabilities based on indicator function values without additional transformation.\n- **Kernel Density Estimation (KDE)**: Uses KDE to smooth and adjust data distribution based on the indicator function.\n\nThe results, shown in **Table 8 on the one-page pdf**, indicate the diffusion model achieves the best performance. **The diffusion model's iterative denoising process effectively harmonizes data distributions**, ensuring robust and consistent alignment, crucial for improving convergence and performance.\n\nFrom a practical perspective, CRFed harmonizes these data distributions, ensuring consistent updates and improved convergence. CRFed outperforms state-of-the-art methods and remains effective, as its training can be performed during the local phase(although running on server), ensuring real-time operation of the FL system.\n\n# W3. Issue about Eq. (1) and Eq. (16)\n\n **Response:** : We appreciate the reviewer's observation. The indicator function is defined at the client level but computed from the sample level. Specifically, Eq. (1) calculates the indicator function for each sample, while Eq. (16) aggregates it at the client level. There is no inconsistency between Eq. (1) and Eq. (16). We will improve clarity in future version.\n\n# Q1. Why is the optimal uncertainty obtained by minimizing the indicator function?\n\n**Response:** Thank you for your comment. Minimizing the indicator function is crucial for:\n\n- **Balancing Loss and Uncertainty:** Minimizing $I_{\\lambda}(l_i, \\sigma_i)$ ensures higher importance for samples with higher loss values while controlling uncertainty.\n- **Preventing Overfitting:** The regularization term $\\lambda (\\log \\sigma_i)^2$ discourages high uncertainties, preventing overfitting to unreliable samples.\n\nAdditionally, minimizing the indicator function helps control the parameter update magnitude, as shown by:\n\n$$\n\\left| \\theta_{t+1} - \\theta_t \\right| = \\eta \\left| \\left( \\sigma_i^* + (l_i - \\tau) k + 2\\lambda \\frac{\\log \\sigma_i^*}{\\sigma_i^*} k \\right) \\nabla_{\\theta} l_i \\right| \\leq \\eta C \\left| \\nabla_{\\theta} l_i \\right|,\n$$\n\nwhere $\\eta$ is the learning rate, and $C$ is a constant that bounds the term $\\left( \\sigma_i^* + (l_i - \\tau) k + 2\\lambda \\frac{\\log \\sigma_i^*}{\\sigma_i^*} k \\right)$. This inequality indicates that by choosing $\\sigma_i^*$ to minimize the indicator function, we can control the magnitude of the parameter updates.\n\n# Q2. Discussion on the distribution decoder\n\n**Response:** Thank you for your comment. The role of the Distribution Decoder is essentially to map the latent representation $z_t$ to the distribution space, so the training is conducted offline on a known ground truth distribution. For a specific task, using the task's test set (or validation set) is a good choice. Other details can be found in section A.2.2 of the original text. We hope these explanations can address your concern.\n\n#  Q3. Details of model training\n\n**Response:** Thank you for your comment. The diffusion model, the model encoder, and the distribution decoder are all trained globally. Specifically, the model encoder and the distribution decoder are trained offline. \n\nTo quantify the training overhead introduced by our approach, we have measured the wall-clock time per round and the total convergence time. The results are shown in **Table 2 on the one-page pdf**. We observe that the wall-clock time per round for CRFed is slightly higher compared to other methods due to the additional noise/denoise operations, but the total convergence time is comparable to other methods. The slight increase in computational cost is balanced by the improved model performance, making it a worthwhile trade-off.\n\n## Reference\n\n[1]  Self-paced learning: An implicit regularization perspective. AAAI, 2017.\n\n[2] Data parameters: A new family of parameters for learning a differentiable curriculum. Neurips, 2019.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
# Conclusion

We sincerely thank all the reviewers for their insightful and valuable comments! Overall, we are encouraged that they find the contributions of our work noteworthy and valuable. Here is a summary of the key points acknowledged by the reviewers:

- The proposed CRFed framework, including the diffusion-based data harmonization mechanism and confusion-resistant strategy, is well-received.  (all reviewers)
- The theoretical foundations and detailed explanations provided for key methods are appreciated(Reviewers cbew and FLG7), although some areas required further clarification (Reviewers nZWV and CpsC).
- The empirical validation on multiple datasets and the comprehensive set of experiments, including sensitivity analysis and ablation studies, were considered robust. CRFed achieved SOTA results. (all reviewers)
- The scalability and efficiency of the proposed methods, as well as their potential impact on federated learning applications, were highlighted positively (Reviewers nZWV, cbew, and FLG7).

We have addressed the issues according to the reviews, which can be summarized as follows:

- We have provided a more thorough explanation and theoretical basis for the Indicator Function $I_{\lambda}(l_i, \sigma_i)$, including detailed derivations and convergence analysis. (**We include this in the global rebuttal**)
- Additional sensitivity analysis of key hyperparameters has been included, with results presented for the CIFAR-10 and CIFAR-100 datasets.( **Table 3,4,5 on the one-page pdf**)
- We conducted further experiments to evaluate the computational efficiency, robustness and effectiveness of CRFed.( **Table 1,2, 6,7,8 on the one-page pdf**)
- Minor corrections and improvements, such as fixing citation styles。

# Theoretical analysis 
We have added relevant theories on the design of Indicator Function \(I_{\lambda}(l_i, \sigma_i)\)in CRFed, including convergence analysis and convergence speed analysis.
## Convergence

Consider a simplified federated learning framework where the global model parameters $\theta$ are updated at iteration $t$ as follows:

$$
\theta_{t+1} = \theta_t - \eta \sum_{i=1}^n \nabla_{\theta} I_{\lambda}(l_i, \sigma_i),
$$

where $\eta$ is the learning rate and $n$ is the number of clients. For simplicity, we consider a single client and expand $\nabla_{\theta} I_{\lambda}(l_i, \sigma_i)$:

$$
\nabla_{\theta} I_{\lambda}(l_i, \sigma_i) = \nabla_{\theta} \left( (l_i - \tau) \sigma_i + \lambda (\log \sigma_i)^2 \right).
$$

By the chain rule, we have:

$$
\nabla_{\theta} I_{\lambda}(l_i, \sigma_i) = \sigma_i \nabla_{\theta} l_i + (l_i - \tau) \nabla_{\theta} \sigma_i + 2\lambda \frac{\log \sigma_i}{\sigma_i} \nabla_{\theta} \sigma_i.
$$

Based on the definition of the optimal $\sigma_i^*$, $\nabla_{\theta} \sigma_i^*$ can be approximated as proportional to $\nabla_{\theta} l_i$, i.e.,

$$
\nabla_{\theta} \sigma_i^* \approx k \nabla_{\theta} l_i,
$$

where $k$ is a constant. Thus, $\nabla_{\theta} I_{\lambda}(l_i, \sigma_i^*)$ simplifies to:

$$
\nabla_{\theta} I_{\lambda}(l_i, \sigma_i^*) = \left( \sigma_i^* + (l_i - \tau) k + 2\lambda \frac{\log \sigma_i^*}{\sigma_i^*} k \right) \nabla_{\theta} l_i.
$$

Since $\sigma_i^*$ is obtained by minimizing the indicator function, we have:

$$
\sigma_i^* \approx \exp \left( W \left( \frac{-(l_i - \tau)}{2\lambda} \right) \right).
$$

Finally, the model update rule can be expressed as:

$$
\theta_{t+1} = \theta_t - \eta \left( \sigma_i^* + (l_i - \tau) k + 2\lambda \frac{\log \sigma_i^*}{\sigma_i^*} k \right) \nabla_{\theta} l_i.
$$

To prove convergence, we note that the step size of parameter updates is finite:

$$
\left| \theta_{t+1} - \theta_t \right| = \eta \left| \left( \sigma_i^* + (l_i - \tau) k + 2\lambda \frac{\log \sigma_i^*}{\sigma_i^*} k \right) \nabla_{\theta} l_i \right| \leq \eta C \left| \nabla_{\theta} l_i \right|,
$$

where $C$ is a constant. Thus, as long as the learning rate $\eta$ is appropriately chosen, the update step size will gradually decrease, ensuring the convergence of the model.

## Convergence speed

Next, we compare the convergence speed of our method with the FedAvg.  

For FedAvg, the update rule is:

$$
\theta_{t+1} = \theta_t - \eta \frac{1}{n} \sum_{i=1}^n \nabla_{\theta} l_i,
$$

where the update step size is (**I'm not sure why the following formulas are not displaying correctly. I will correct this as soon as possible**):

$$
\left| \theta_{t+1} - \theta_t \right|_{\text{FedAvg}} = \eta \left| \frac{1}{n} \sum_{i=1}^n \nabla_{\theta} l_i \right|
$$

For our method with the indicator function, the update step size is:

$$
\left| \theta_{t+1} - \theta_t \right|_{I_{\lambda}} = \eta \left| \left( \sigma_i^* + (l_i - \tau) k + 2\lambda \frac{\log \sigma_i^*}{\sigma_i^*} k \right) \nabla_{\theta} l_i \right|.
$$

To demonstrate that our method converges faster or has a tighter bound, we analyze the total update step sizes over all clients.

For FedAvg:

$$
\sum_{i=1}^n \left| \theta_{t+1} - \theta_t \right|_{\text{FedAvg}} = \eta \sum_{i=1}^n \left| \nabla_{\theta} l_i \right|.
$$

For our method:

$$
\sum_{i=1}^n \left| \theta_{t+1} - \theta_t \right|_{I_{\lambda}} = \eta \sum_{i=1}^n \left| \left( \sigma_i^* + (l_i - \tau) k + 2\lambda \frac{\log \sigma_i^*}{\sigma_i^*} k \right) \nabla_{\theta} l_i \right|.
$$

When $\sigma_i^* = \exp \left( W \left( \frac{-(l_i - \tau)}{2\lambda} \right) \right)$, $\lambda \geq \frac{-(l_i - \tau)}{2e}$, and $\tau = l_{\max} - 2 \lambda \ln \left( \frac{1}{k} \right)$, we can ensure $\left| C_i \right| \leq 1$, thus:

$$
\left| C_i \nabla_{\theta} l_i \right| \leq \left| \nabla_{\theta} l_i \right|.
$$

Thus,

$$
\sum_{i=1}^n \left| C_i \nabla_{\theta} l_i \right| \leq \sum_{i=1}^n \left| \nabla_{\theta} l_i \right|.
$$

Therefore, the update step size for CRFed is less than that of FedAvg. We will provide the complete theoretical proof in final version.

---

Next, we address each reviewer's detailed concerns point by point. Thanks!

Meta-review: The paper proposes a method to address data heterogeneity in FL by using a diffusion to harmonize clients data heterogeneity, with data noise injection and iterative denoising. The reviewers are generally positive, and all improved their ratings after engaging in the rebuttal process (including the lowest score).
Decision: Accept (poster)