FedPD: Defying data heterogeneity through privacy distillation
Authors: Zhiqin Brian Yang
Yonggang Zhang
Yu Zheng
Zhenheng TANG
Xiaowen Chu
Hao Peng
Bo Han
Venue: Submitted to ICLR 2023 (2023)

Abstract: Model performance of federated learning (FL) typically suffers from data heterogeneity, i.e., data distribution varies with clients. Advanced works have already shown great potential for sharing client information to mitigate data heterogeneity. Yet, some literature shows a dilemma in preserving strong privacy and promoting model performance simultaneously. Revisiting the purpose of sharing information motivates us to raise the fundamental questions: Which part of the data is more critical for model generalization? Which part of the data is more privacy-sensitive? Can we solve this dilemma by sharing useful (for generalization) features and maintaining more sensitive data locally? Our work sheds light on data-dominated sharing and training, in a way that we decouple original training data into sensitive features and generalizable features. To be specific, we propose a \textbf{Fed}erated \textbf{P}rivacy \textbf{D}istillation framework named FedPD to alleviate the privacy-performance dilemma. Namely, FedPD keeps the distilled sensitive features locally and constructs a global dataset using shared generalizable features in a differentially private manner. Accordingly, clients can perform local training on both the local and securely shared data for acquiring high model performance and avoiding the leakage of not distilled privacy. Theoretically, we demonstrate the superiority of the sharing-only useful feature strategy over sharing raw data. Empirically, we show the efficacy of FedPD in promoting performance with comprehensive experiments.

Under review as a conference paper at ICLR 2023
FEDPD: DEFYING DATA HETEROGENEITY THROUGH PRIVACY DISTILLATION
Anonymous authors Paper under double-blind review
ABSTRACT
Model performance of federated learning (FL) typically suffers from data hetero- geneity, i.e., data distribution varies with clients. Advanced works have already shown great potential for sharing client information to mitigate data heterogeneity. Yet, some literature shows dilemma in preserving strong privacy and promoting model performance simultaneously. Revisiting the purpose of sharing information motivates us to raise the fundamental questions: Which part the data is more critical for model generalization? Which part of the data is more privacy-sensitee Can we solve this dilemma by sharing useful (for generalization) features and maintaining more sensitive data locally? Our work sheds light data-dominated sharing and training, in a way that we decouple original training data into sensitive features and generalizable features. To be specific, Distillation framework named FedPD to alleviate the privacy-performant dilemma. Namely, FedPD keeps the distilled sensitive features locally and constructs global dataset using shared generalizable features in a differentially private manner, Ac- cordingly, clients can perform local training on both the local and securely shared data for acquiring high model performance and avoiding the leakage of not distilled privacy. Theoretically, we demonstrate the superiority of the sharing-only useful feature strategy over sharing raw data. Empirically, we show the efficacy of FedPD in promoting performance with comprehensive experiments.
propose Federated Privacy
INTRODUCTION
Federated learning (FL.), as an emerging protection paradigm, receives increasing attention re- cently (Kairouz et al.] 2021 Li et al. 202 Yang et al. 2019). which preserves data privacy without transmitting pure data. In general. distributed clients collaboratively train global model by aggregating gradients (or model parameters). However, distributed data can cause heterogeneity
issues (McMahan et al. 2017 ei al. 2022 2020 Zhao et al. 2018), due to diverse computing capability and MoD data distribution across federated clients. It results in unstable convergence
and degraded performance.
To address the challenge of heterogeneity, the seminal work, federated averaging (FedAvg) (McMahan eet al., (017). proposes weighted averaging to overcome Non-lII data distribution when sharing selected local parameters in each communication round. Despite addressing the diversity of computing and communication, FedAvg still struggles with the client drift issue (Karimireddy et al., 2020). Therefore, recent works try to resolve this issue by devising new learning objectives (Li al.
2020), designing new aggregation strategies (Yurochkin et al. 2019) and constructing information for sharing (Zhao et al. 20188 Yoon et al. 2021). Among these explorations, sharing relevant information across clients provides a straightforward and promising approach to mitigate data heterogeneity.
However, recent works point out dilemma in preserving strong privacy and promoting model performance. Specifically, (Zhao et al. 2018) show that a limited amount of sharing data could significantly improve training performance. Unfortunately, sharing raw data, synthesized data, logits and statistical information (Luo et al. 202 Goetz& & Tewari 2020 Hao et al. 2021 Karimireddy et al. 2020) can incur high privacy risks. To protect clients' privacy, differential privacy (DP) provides de facto standard way for provable security quantitatively. The primary concern in applying DP
is about performance degradation (Tramer & Bonch 2020). Thus, solving the above dilemma can contribute promoting model performance while preserving strong privacy.

---

Under review as a conference paper at ICLR 2023
1.1 SYSTEMATIC OVERVIEW OF FEDPD
To solve the dilemma, we revisit the purpose of sharing information: sharing raw data benefits model generalization while violating privacy leakage. This motivates us to raise the fundamental questions:
(I) Is it necessary to share complete raw data features to mitigate data heterogeneity?
We find that some data features are more important than others to train global model. Therefore, an intuitive approach is to divide the data features into two parts: one part for model generalization, named generalizable features, and the other part with clients' privacy, named sensitive features. Then, the dilemma can be solved by sharing generalizable features and keeping sensitive features locally throughout the training procedure. The insight is that the sensitive features in the data are kept locally, and the generalizable features intrinsically related to generalization are shared across clients. Accordingly, numerous decentralized clients can share generalizable features without privacy concerns and construct global dataset to perform local training.
(2) How to divide data features into generalizable features and sensitive features?
It is challenging identify which part of the data more important for model generalization and which part is more privacy-sensitive. To resolve this challenge, we propose a novel framework named Federated Privacy Distillation (FedPD). FedPD introduces a competitive mechanism by decomposing IRd with dimension d into generalizable features 2i E Ted and sensitive features æ, IR" i.e., = B + x.. In FedPD. sensitive features D, aim to cover almost all information in the data æ, while the generalizable features Fy compete with v, for extracting sufficient information to train models such that models trained on &g can generalize well. Consequently, the sensitive features are almost the same as the data while models trained on generalizable features generalize well.
(3) What is the difference between sharing raw data features and partial features?
To ensure that sharing the generalizable features Fg cannot expose FL the danger of privacy leakage, we follow the conventional style in applying differential privacy to protect generalizable features By shared across clients. Our trick is that most information in data has been distilled as sensitive features a. which is very secure and kept locally. In other words, we only need a relatively small noise to protect Eg without the need to fully protect the raw data æ, yet achieving a much stronger privacy than the straightforward protection (i.e., directly sharing with differential privacy). Intuitively, sharing partial information in the data is more accessible to preserve privacy than sharing complete information, which is fortunately consistent with our theoretical analysis.
1.2
OUR RESULTS AND CONTRIBUTION
To tackle data heterogeneity, we propose a novel framework with privacy, which constructs a global dataset using securely shared data and performs local training both the local and shared data, shedding new light on data-dominated sharing schemes. To show the efficacy, we deploy FedDP on four popular FL algorithms, including FedAvg, FedProx, SCAFFOLD, and FedNova, and conduct experiments on various scenarios with respect to different amounts of devices and varying degrees of heterogeneity. Our extensive results show that FedPD achieves considerable performance gains on different FL algorithms. Our solution not only improves model performance in FL but also provides strong security, which is theoretically guaranteed from the lens of differential privacy.
Our contributions are summarized as follows:
We raise a foundation question: whether is necessary to share complete raw data features when sharing privacy data for mitigating data heterogeneity in FL.
We answer the question by proposing a plug-and-playy framework named FedPD, where data features are divided into generalizable features and sensitive features. In FedPD, the sensitive features are distilled in a competitive manner and kept locally, while the generalizable features are shared in a differentially private manner construct global dataset.
We give a new perspective employing differential privacy that adds noise to partial data features instead of the complete raw data features, which is theoretically superior to the raw data sharing strategy.
Extensive experiments demonstrate that FedPD can considerably improve the performance of FL models.
2

---

Under review as a conference paper at ICLR 2023
Server
Local classifier
Auxiliary
Client Local training
Client
Client
Local training
Local training
Globally shared
Privacy distillation
Privacy distillation
Privacy distillation
Protected generalizable features
Privacy distillation
Local training
Protected generalizable features (xp:))
Classifier
Classiffer
6(-:0) #))
X,
Globally shared data
(p)
Raw data
Protection(x,)
(3)
Ly
Figure 1: FL Framework with the plug-in in FedPD. Clients generate generalizable features and add noise protection get protected generalizable features zp during privacy distillation process. The protected generalizable features F are collected from numerous distributed clients to construct globally shared dataset while sensitive features F, are kept locally. During local training procedure, local raw data and a subset of globally shared data jointly train the local model for global aggregation. CA denotes the Eq. [ and CF is the Eq. 3 in our paper.
RELATED WORK
Federated Learning with heterogeneous data. The classic FL algorithm FedAvg (McMahan et 2017) suffers from serious performance degradation when meeting severe Non-IID data. To address the data heterogeneity problem, series of works propose new learning objective calibrate the updated direction of local training from being too far away from the global model, including
FedProx et al. 2020). FedIR (Hsu al., 2020), SCAFFOLD (Karimireddy
al., (0200) and
MOON (Li et al. 2021a And some works propose designing new model aggregation schemes like
FedAvgM (Hsu et al. 20192. FedNova (Wang et al. 2020b),FddMA FedMA (Wang et
FedBN (Li
al. 2021c).
Another promising direction is sharing some data, which mainly focuses on synthesizing and sharing data of different clients mitigate client drift (Zhao et al. 2018 Jeong al. 2018 Long al. 2021). To avoid privacy leakage caused by sharing data, some methods share the statistics of data Yoon et al. [2021] Shin al. 2022).. which still contains
data content. Some methods distribute intermediate features (Hao et al. 2021), logits (Chang et al. 2 Luo et al. [202T), or the learned new embedding (Tan al. 2022). Although these tacties enhance privacy at some degree, advanced
attacks can still successfully reconstruct raw data given shared data (Zhao et al. 2020). Unlike prior research, we exploit DP to ensure privacy of shared data and then analyze grivac--pefforman trade-off.
Differential privacy in federated learning. Recent works on model memorization and gradient leakage confirm that model parameters are seemingly secure (Carlini et al. (019). Training with differential privacy (Zhu et al. 2019 Nasr al., 2019) feasible solution avoid some attacks, albeit at some loss in utility. Differential privacy quantifies what extent individual privacy in statistical dataset is preserved while releasing the established model over specific datasets.
In FL, training with differential privacy, i.e., adding noise to the model/data, originally aims to protect local information of each client (Yuan et al. 2019; Thakkar et al., 2019). Some works analyze the relation between convergence and utility in FL (Huang et al. 2020 Wei et al. 2020). A series of works in DP add noise to gradients or model parameters in FL to protect model privacy (Kim et al. 2021: van der Hoeven 2019 Triasteyn & Faltings 2019: Sun et al.) 2021). Unlike model-based protection, our work aims to protect data privacy and mitigate the client drift issue. We provide a detailed discussion of exciting works in Appendix A.5

---

Under review as a conference paper at ICLR 2023
3 METHODOLOGY AND DETAILED CONSTRUCTION
This section elaborates Federated Privacy Distillation (FedPD), which is illustrated in Figure | Our insight is to keep sensitive features on the client's s side locally and share generalizable features globally in differentially private manner. FedPD endows each client its local raw data features and generalizable features from others during local training, thus defying data heterogeneity.
3.1 DIVISION OF TWO TYPES OF FEATURES FOR PRIVATE DATA
Differential privacy (DP) is promising in FL protections, but sharing all raw data in a DP manner typically causes performance degradation. Recall that the goal of sharing information is to benefit the model generalization rather than to collect private information. Therefore, we suggest to share useful features (generalizable features) in data while keeping most features locally (sensitive features), such that shared features benefits global generalization and locally kept features avoids privacy leakage.
Ideally, if we could identify the sensitive features H, and the generalizable features zg. could be able to solve the privacy-peffrrmant dilemma. Intuitively, sensitive features Ts contain most information of data, while generalizable features Zy contains the nonsensitive part that help global generalization in FL. To resolve the dilemma in protecting privacy and promoting performance, we can keep the sensitive features locally while sharing generalizable feature protected under differentially private guarantee. The major challenge here is that the intersection of two types of features as aforementioned may not be the empty set, making it challenging distill privacy.
3.2 PRIVACY DISTILLATION
To address this issuc, propose competitive mechanism to perform privacy distillation. Therein, the generalizable features aim to train models for generalizing well on the raw data, while the sensitive features compete with the generalizable features to construct the raw data. Consequently, the sensitive features is almost the same as the data while models trained on generalizable features generalizing well on the raw data. We propose two approaches instantiate the competitive mechanism for privacy distillation, i.e., making generalizable features useful for model generalization while keeping sensitive features almost the same as the raw data, i.e., covering almost all information of raw data.
3.2. OPTIMIZATION VIEW
A straightforward approach is distill private information in a meta manner (Finn et al., 2017).
Specifically, we employ a generative model, e.g., a variational auto-encoder (VAE), G(:; 0) parame-
terized with 0 to achieve the goal of covering all information of raw data, i.e., 2;, G(: () aims to
reconstruct æ. Mcanwhile, to ensure the generalizable features, Tg = e- D. - G-: = xg (0).
useful for model generalization, we train an auxiliary classifier A(: w) parameterized with using
such that A(: w) trained on Fy performs well on the raw data æ. Then, we can formalize the task oy privacy distillation into the following optimization problem as:
min E (x, v) C(A(#; (i)). H(x,(()), 8.1. (b() arg min E (0, v) C(A((=,, w). (0) = x G( 0).
Here, is the label of the sample and the generalizable features (0), (0) is function of e denoting the parameters of classifier A(x; H()x(()) is the information entropy of (0) and .(.,)) represents the cross-entropy loss. We can see that every possible parameter 0 is paired with model trained on the corresponding generated data a.,(0). (0). Thus, solving the optimization problem equivalent to searching for parameters 0 to generate the generalizable features x, (0) with minimum can perform
information entropy. Moreover, the model A((x, (0); w), y) trained using well on the raw data.
However, the proposed non-convex optimization problem is non arivall We employ a simple yet effective trick widely used in reinforcement learning (Mnih et al. 2015). Specifically, alternatively update G(æ;; 8) over æ via stochastic gradient descent and update (0); w) over (0). Moreover, minimize upper bound of H H(v, (0) with the variance of m. (0) following (Ahuja et al. 2021).
4

---

Under review as a conference paper at ICLR 2023
3.2.2 GENERALIZATION VIEW
Besides the optimization approach, we also provide a generalization view distill privacy. In high level, we aim to train model A(: w) using Ty such that A(::w) can generalize well on æ, i.e. samples drawn from different distribution. Therefore, we should model how the performance on the generated data transfers to the raw data. To derive a detailed connect between these two distributions, the metric to measure the generalization performance should be defined clearly. According to the margin theory (Koltchinskii & Panchenko, 2002) that maximizing the margin between data points and the decision boundary achieves strong generalization performance, we relate such a margin the generalization performance:
Definition 3.1 (Margin). We define the margin for a classifier A(; w) on a distribution P with distance metric d: Mm, =
(z. =
inf A(x') + y d(<x,_,) .x)..
Built upon the defined margin that quantifies the degree of generalization performance, we can quantify the generalization performance of A(; a given distribution. To be specific, large margin means strong generalization performance.
A recent work (Tang et al., 2022 shows that the margin is intrinsically related to the distribution discrepancy in the representation space, i.c., the distance between distributions sampling æ and that sampling yg. Thus, we propose minimizing the distribution discrepancy of the generated distribution and the raw distribution in the representation space:
min E (xxy) ((A(x; w), y) + (()(z, w) y) + ((z,())) + (0)), r(x)).
(2)
where d is the distance metric used in the definition of margin and (0)) stands for the representa- tion of (0) generated by the classifier A.
3.3 DIFFERENTIALLY PRIVATE GENERALIZABLA FEATURES
The proposed privacy distillation methods make it possible to keep most (private) information locally while sending the generalizable features to the server. However, for case of calculation of information entropy, we employ the variance of generalizable features surrogate, which may cause privacy leakage. This breaks the original intention of federated learning in protecting privacy. Thus, the shared generalizable features should be protected. Accordingly, the server can construct global dataset using these generalizable features and send the dataset back to clients for local training.
To avoid privacy leakage, additional noise (e.g., Gaussian or Laplacian) is added to generalizable features Zgg i.e., Tp - 2g (N, Then, clients send xp to the server to construct a globally shared dataset. Using the global dataset, clients can train classifier F(:; () parameterized by á with the local and shared data, :
min Cr(0) = (x. y) ((F(2; Φ). v) + E (2p, y) ((F(ppp v).
Algorithm D Summarizess the training procedure of FedAvg with FedPD, To make sure the framework can be used without privacy concern, we further provide the corresponding analysis. Before that, we introduce the definition of differential privacy, which we used for adding i.i.d noise to generalizable features.
Definition 3.2. (Differential Privacy). A randomized mechanism M provides (c, diifferrrltt privacy (DP) if for any two neighboring datasets D and D' that differ in a single entry, VS S (Mnge(M),
Pr(M(D) E S) S e (e(M(DD)) E ES +..
where is the privacy budget and 8 is the failure probability.
Our added noise to By is proportional to the sensitivity, as defined in Definition
3.3
The concept of
sensitivity is originally used for sharing dataset for achieving (c, 8) ddffferrent privacy. Later, follow Theorem 3.4 analyze the privacy on globally shared data.
Definition 3.3. (Sensitivity). The sensitivity of a query function F D -> TR. for any two neighboring
datasets D, D' is,
A max D.D
[(D) - F(D')ll.
where || denotes In or Toz norm.
5

---

Under review as a conference paper at ICLR 2023
Algorithm 1 FedAvg with FedPD
server input: initial 6", communication round R
client k's input: local epochs E, local datasets DK learning rate T/e Initialization: server distributes the initial model Φ all clients.
Globally shared dataset DD' generating. + Detail Algorithm
Server Executes:
for each round 2,
R do server samples subset of clients S, S (), Eies, [Di|
................................ .......
client & samples subet of globally shared dataset D.: D ((D ID
server communicates ø" to selected clients & E S, and sampled sharing data D
for each client k E S, in parallel do g++1 E- t Clien(Training(k; 0², D
end for Φ++1 Σ
ID' KES, a -E
end for
ClientTraining(t; Φb D. ): for each local epoch with j = 0,... E- do
Ōk.++1 + OR.j ? V CF (0), i... Eq. end for Return Ø server
Theorem 3.4. For any e 0,8 1], and 8 E [0, 1}, the class of (e, -idfffereenl private mech- privacy under k-fold adaptive composition for (c (e]kk///(( 2llgg//18))
anisms satisfies (Ez, 1 (111-(1-8)11 (1-))))
i min (ke, (e) 1)) 2 log(e +
Since x, is kept by the corresponding client, an adversary views nothing, which can be regarded as adding a sufficiently large noise to make it random enough. Considering all clients' data as a whole, we use a relatively small a (i.e., de < Od + a.) for achieving much smaller privacy loss, summarized in Theorem 3.5
Theorem 3.5. Given identical privacy requirement, de of FedDP is much less than that is suppos- edly added to raw data in conventional FL.
Given (e, d each client side, we utilize composition theorem to analyze overall privacy in FedPD. In summary, FedPD protects two types of data features using different protective manners, ie., small noise for generalizable features and extremely large noise for sensitive features, and thus attains higher model performance and stronger security in the same time.
EXPERIMENTS AND EVALUATION
4.1 EXPERIMENT SETUP
Federated Non-III Datasets. We conduct experiments over various popular image classification datasets, including CIFAR-10, CIFARI00 (Krizhessky et al.) 2009). FaahhionMNNIST(MMNISS (Xiao et al.) 2017). ). and SVHN (Netzer et al. (201)). We use latent dirichlet sampling (LDA) (Hsu et al. 2019) to simulate Non-III distribution with TO and 100 clients. The primary thought is to draw a q ~ (ap( from Dirichlet distribution, where a controls the heterogeneity degree. Here, the less is, the more severe Non-IID IID distribution generate. In our experiments, with two different degrees by LDA including & 0.1 and a = 0.05. Besides, in order to prove that framework works well under with Non- IID partitions. We also test other two kinds of partition strategy: (1) #C = (McMahan et al. 2017: Li et al., 2002);: each client only has different labels from dataset, and A controls the unbal anced degree. (2) Subset method (Zhao et al. 2018): each client
partition
datasets

---

Under review as a conference paper at ICLR 2023
Table
Results with/without FedPD on CIFAR-10
centralized training ACC 95.48%
w/(w)) FedPD
ACC1
Gaint
Round
Speedupt
ACC1
Gaint
Round
Speedup?
0.1, B
1. K 10 (Target ACC =79%)
0.05, E
1, K
10 (Tareet ACC =69%)
FedAvg
92.34(79.35)
12.991
39(284)
x7.3(x 1.0)
90.02(69.36)
20.661
44(405)
x9.2(x110)
FedProx
92.12(83.06)
9,061
62(192)
x.56(x1.5)
90.73(78.98)
11.75t
48/203)
x8.4(x2.0)
SCAFFOLD
89.66(83.67)
5.991
34(288)
x..4(x100
81.04(37.87)
45.171
37(None)
x 10.9(None)
FedNova
92.23(80.95)
11.281
33(34))
x8.6(x0.8)
(6521(65.08)
26.131
32(None)
x 12.7(Nome)
a 0.1,
5,
10 (Target ACC =85%)
a 0.1,
1, K
100 (Target ACC =49%)
FodAvg
93.24(83.79)
9.45
17(261)
x15.4(x
84.06(49.72)
34.341
163(967)
x5.9(x 1.0)
FedProx
91.39(82.32)
8.97 f
76(None)
x3.4(Nonc)
87.01(50.01)
37.001
127(831)
x..6(x1.2)
SCAFFOLD
92.34(85.31)
7.03
15(66)
x17.0(x4..0
79.60(52.76)
26.841
171(627)
x5.7(x 1.5)
FedNova
92.85(86.21)
6.64 f
31(120)
x8.4(x2.2)
86.64(45.97)
40.671
199(None)
x4.9(None)
"Round" means the communication rounds that arrive the target accuracy. 4 and f indicates smaller
(larger) values are better. "None" implies not attaining the target accuracy during the entire training process, All the "Speedup" calculated by comparing with vanilla FedAvg "Round" in different Non-IID partition scenarios.
has all classes from the data, but one dominant class far away outnumhers other classes. These three partition methods mainly include label skew and quantity skew. The visualization of data distribution is shown in Figure in Appendix A.I
Models, Metrics and Baselines. We use ResNet- 18 (He et al. 2016) both in content extractor during sharing data generation process and classifier in FL. And we exploit 3-VAE (Higgins et al. 2016) for privacy distiller for privacy-preserving. We evaluate the model performance on two popular metrics in FL, i.e. the communication rounds to reach the target accuracy and the best accuracy in whole training process. Note that the target accuracy different scenarios. We conduct FedAvg (McMahan et al., (017) and some other popular methods including FedProx (Li et al., 2020). SCAFFOLD (Karimireddy et al., (020). FedNova (Wang et al.) 2020b), with without FedPD, to explore the potency of our method. We conduct all algorithms with local epochs E = and E The detailed hyper-parameters of each FL algorithm and privacy distillation in different datasets are listed in A.3.1
as the best accuracy of vanilla FedAvg in
4.2 EXPERIMENTAL. RESULTT
Main Results. The results on CIFAR-10, CIFAR-100, FMNIST, and SVHN are shown respectively in Tables D 5 and 6 which demonstrates that FedPD has significant performance gain. We also show the convergence speed of different algorithms on CIFAR-10 10 with 0.1, E = 1, M = 10 in Figure 2a which shows that FedPD can also greatly improve the convergence rate.
Table Results with/without FedPD on CIFAR-100
centralized training ACC 75.56%
w((wo) FedPD
ACC1
Gaint
Round J
Speedupt
ACC1
Gaint
Round J
Speedupt
0.1, E
1, K 10 (Target ACC =67%)
0.05, E 1, K 10 (Tageel ACC =61%)
FedAvg
69.64(67. 84)
1.8t
283(495)
x1.7 (x1.0)
68.49(62.01)
6.481
137(503)
x3.7(x1.0)
FedProx
70.02(65.34)
4.68 t
233(Nonc)
x2.1(None)
69.03(61.29)
7.741
141(485)
x3.6(1.0)
SCAFFOLD
70.14(67.23)
2.91
198(769)
x2.5(x 0.6)
69.32(58.78)
10.54
81(Nonc)
x6.2(None)
FedNova
70.48(67.990 988)
2.5t
147(432)
x3.4(x1.1)
68.92(60.53)
8.391
87(None)
5.8(None)
0.1, E
K.
10 (Target ACC =69%)
a 0.1,
1, J.K
100 (Targe ACC =48%)
FodAvg
70.96(69.34)
1.62
79(276)
60.58(48.21) x.)
12.37t
448(967)
x2.2(x
FedProx
69.66(62.32)
7.34
285(None)
x1.0(None)
(4867))
18.91f
200(932)
x..8(x
SCAFFOLD
70.76(70.23)
0.531
108(174)
x2.6( 1.6)
66.67(51.03)
15.64
(83()
x..3(x112)
FedNova
69.98(69.78)
0.21
89(290)
x3.1(x100)
67.62(48.03
19.59h
198(976)
x4.9(x
More figures of convergence speed of other experiments are shown in appendix A.4
7

---

Under review as a conference paper at ICLR 2023
Table Experiment results of different Non-IID partition methods CIFAR-10 10 with 10 clients.
Table 4: Experiment results with different noise adding CIFAR-10 10.
Purtition
Text Accuracy wi(wlo) FedPD
Test Accuracy on Ditlerent Noise
FedAvg
FedProx
SCAFFOLD
Noise
FedAvg
FedProx
SCAFFOLDD
#C =
0.1
92 22.34 79.35)
89.23:42.54
92.12183.06)
88.17/58.45
84.4396.82
39.66183.77)
92.23(80.955 95)
89.54/15.44
Gaussian Noise
92.34
92.12 91.36
89.66 91.24
92.23
89.11/32.87
Laplacian Noise
92.30
91.73
Test Accuracy [%)
Yos Accoumen [N]
90
60-
Piina E
60
X
93
Fecterssneet 1o
Fectes
250
500
750
1000
Round
Noiec
Training
(a) Test Accuracy CIFAR-10 with 0.1, K
(b) Test Accuracy FMNIST with different noise level o² ,ob-
(c) Two clasifiers trained dif- ferent data form and test accuracy on æ, Ta, and Hy respectively.
Gaussian Filter for better sualization.
taining various privacy clower
is preferred).
Figure 2: Experiments of the relationship between privacy and performance.
(a) Globally Shared Data
(b) Model Inversion Attack x;
(c) Model Inversion Attack xp
Figure 3: Model Inversion Attack Results. White-Box attack globally shared data æ, and generaliz- able features Zy. respectively. The result of being attacked is in (b) and (c) to compare with shared data xp in (a)
Privacy and performance. To explore the relationship between privacy level and performance, we conduct experiments with different o². As shown in Figure 2b the performance decreases with the increasing protection strength. Another Laplacian noise report comparable results with Gaussian noise listed in Table In conelusion, limited communication resources. Another question is, can the globally shared data be inferred by some attack methods? To answer this question, widely used in the literature to reconstruction our shared data. The results on Figure 3blindicates that only privacy distillation still have risk of privacy leakage. Figure also be a strong testimony for the differential privacy of noise adding on generalizable features. Furthermore, FedPD can give a strong private information protection. The original image can be found in Appendix A..
suggest sacrificing part of the privacy when encountering
model inversion attack (He al. 2019).
Different number of clients. Table Table 2 Table and Table show that FedPD strengthen the performance and speed up the convergence both TO and 100 chents. Especially 100 clients in CIFAR-10 and CIFAR 100 have noteworthy enhancement. The reason may be that FL on CIFAR-10 and CIFARIO0 with 100-clients has more diverge data distribution than FMNIST. With FedPD, the missed data knowledge can be well replenished.
Different data heterogeneity. Table Table 2 Table 5 and 6 that high Non-IID degree (a=0.05) achieve a better improvement than lower unbalanced degree (a=0.1), which also indicates that FedPD can well defend against data heterogeneity. Moreover, Table 3 shows that other two kinds of heterogeneity partition more performance decline compared with LDA (a 0.1). and FedPD attains comparable improvement with LDA a = 0.1, indicating FedPD insensitive to other Non-IIID data distribution.
8

---

Under review as a conference paper at ICLR 2023
Table 5:
Results with/without FedPD on FMNIST
centralized training ACC 95.64% w((w/o) FedPD
ACC1
Gaint
Round
Speedupt
ACC1
Gaint
Round 4
Specdupt
0.1,
1, K
10 (Target ACC= =86%)
0.05, E
1, K
10 (Target 77C))
FedAvg
92.34(86.73)
5.611
14(121)
x8.6(x100)
90.69(78.34)
12.351
16(420)
x26.3(x 1.0)
FedProx
92.09(87.73)
4.36
32(129)
x2.1(x0.9)
89.68(82.03)
7.651
16(44)
x26.3(9.5)
SCAFFOLD
91.62(86.31)
3.891
29(147)
x4.2(x 0.8)
80.48(76.63)
3,851
139(Nonc)
(6.2(None)
FeuNova
92.39(87.03)
5.361
18(88)
x..7(x1.4)
89.72(79.98)
9,741
16(531)
x *26,3(x 0.8)
a 0.1,
5, K
10 (Target ACC =87%)
a 0.1,
1,
100 (Target ACC =90%)
FodAvg
92.26(88.43)
4.831
19(276)
x14.5(x 1.0)
92.71(90.21)
2.5t
243(687)
x2.8(x 1.00)
FedProx
91.79(86.63)
5.161
34(None)
92.82(90.17) (8.1(Nonc)
2.651
284(501)
x2.4(x1.44)
SCAFFOLD
92.92(87.21)
5.71f
8(112)
(32.5(x2.5)
90.28(84.87)
5,411
952(None)
x0.7 (None)
FedNova
92.30(87.67)
4.63f
8(187)
x34.5(x1.5)
91.04(85.32)
5.721
(89(None)
x1.2(None)
Table 6: Results with/without FedPD on SVHN
centralized training ACC 96.56% w/(w/o) FedPD
ACC1
Gain
Round
Speedupt
ACC
Gaint
Round
Speedupt
a 0.1, E
1, K
10 (Target ACC =88%)
0.05, E = 1, K 10 (Target ACC =82%)
FedAvg
93.21 34)
4.871
105(264)
x2.3(x1.00)
93.49(82.76)
10.731
194(365)
x1.9(x 1.0)
FedProx
91.80(86.23)
5.574t
233(None)
x x1.1(Nome)
93.21(79.43)
13.787
37(None)
x9.9(Nune)
SCAFFOLD
88.41(80.12)
8.291
357(None)
x0.(None)
90.27(75.87)
14.41
64(None)
xS.7(None)
FedNova
92.98(89.23)
3.751
113(276)
x2.3(x100)
93.05(82.32)
10.731
37(731)
x9.9(x0.5)
0.1,
5,
(Target ACC =87%)
0.1,
1,
100 (Targel ACC =89%)
FedAvg 93.77(87.24)
6,531
105(128)
x1.2(x1.0)
91.04(89.32)
1.721
763(623)
x0.8( 1.0)
FedProx
91.15(77.21)
13.94f
142(Nonc)
x0.9(None)
91.41(88.76)
2.65f
733(645)
x0.8(x
SCAFFOLD
93.78(80.98)
12.81
20(None)
x6.4(Nonc)
92.73(88.322 32)
4.411
507(687)
x1..((0.9)
TedNova
93.66(89.03)
4.631
52(177)
x2.5(x0.7)
84.05(81.87)
2.181
None(None)
None(None)
Different local epochs. To test the effect of local epoch E, we choose E = 1 and E = 5 with the same Non-IID degree (a = 0.1) and client number (K = 10). We run 1000 rounds with I epoch local training and 400 rounds for 5 epochs local update. The results show that FedPD is robust to the
local epochs.
Other Facts of FedPD. For intuitive understanding of why we utilize V, as a substitute of raw data x without drastic performance degradation. We train two different networks separately on æ and T, on CIFAR-10 and test them on æ, x.. and Eg. respectively. The results presented in Figure 2c As we can see, the most useful features for downstream tasks are contained in Eg. More experimental details are presented in Appendix A.4
5 CONCLUDING REMARKS
In this paper, we observe that model gains a substantial performance assisted by generalizable features. Later we conduct DP to protect generalizable features and contruct globally shared dataset for defying heterogeneity in FL. Our contribution lies in not only improving model performance in Non- IID scenarios, but also inspiring a new viewpoint data-dominated secure sharing, e.g., distillation data before knowledge learning. We expect that our work could simulated further data-dominated sharing in FL or other popular learning algorithms.
Our framework shows suprior results against model inversion attack, yet we have not finished exploring data poisoning attack, given the shared data. We conduct preliminary experiments on data poisoning attacks, in which some clients send Gaussian noise to the server, causing performance degradation and slow convergence. Limited storage communication resources may limited the power of FedPD, since FedPD introduces extra storage overhead. We leave as our future work to explore the friendly FedPI).

---

Under review as a conference paper at ICLR 2023
ETHIC STATEMENT
This paper does not raise any ethical concerns. This study does not involve any human subjects, practices to data set releases, potentially harmful insights, methodologies and applications, potential conflicts of interest and sponsorship, ieteriiinatioo/bis/ffainnn concerns, privacy and security issues, legal compliance, and research integrity issues.
REPRODUCIBILIIY STATEMENT
To make all experiments reproducible, we have listed all detailed hyper-parameters of each FL algorithm and privacy distillation on different datasets in A.3.1 Due to the privacy concerns, we will upload the anonymous link of source codes and instructions during the discussion phase make only visible to reviewers. All definitions can be found in Section And the complete proof can be found in Appendix AA.
REFERENCES
Kartik Ahuja, Ethan Caballero, Dinghuai Zhang, -ha-CChisttophe Gagnon-Audet, Yoshua Bengio, Ioannis Mitliagkas, and Irina Rish. Invariance principle meets information bottleneck for out-of- distribution generalization. Advances in Neural Information Processing Systems, 34:3438-4550,
2021.
Kuntai Cai, Xiaoyu Lei, Jianxin Wei, and Xiaokui Xiao. Data synthesis via differentially private markov random fields. Proceedings of the VLDB Endowment,
2021.
Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernjj Kos, and Dawn Song. The secret sharer: Evaluating and testing unintended memorization in neural networks. In USENIX Security, 2019.
Hongyan Chang, Virat Shejwalkar, Reza Shokri, and Amir Houmansadr. Cronus: Robust and heteroge- neous collaborative learning with black-box box knowledge transfer. arXiv preprint 1722 2019.
Antoine Chatalic, Vincent Schellekens, Florimond Houssiau, Yves-Alexandre De Montjoye, Laurent Jacques, and Rémi Gribonval. Compressive learning with privacy guarantees. Information and Inference: A Journal of the IMA, 11(1):251-305 2022.
Yae Jee Cho, Jianyu Wang, and Gauri Joshi. Client selection in federated learning: Convergence analysis and power-of-choice selection strategies. arXiv preprint arXiv:2010.01243, 2020.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. International conference on machine learning. pp. 1126-1135. PMLR, 2017.
Jack Goetz and Ambuj Tewari. Federated learning via synthetic data. arXiv:2008. 04489, 2020.
arXiv preprint
Weituo Hao, Mostafa El-Khamy, Jungwon Lee, Jianyi Zhang, Kevin J Liang, Changyou Chen, and Lawrence Carin Duke. Towards fair federated learning with zero-shot data augmentation. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pp-
3310-3319, 2021.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770-778, 2016.
Zecheng He, Tianwei Zhang, and Ruby B Lee. Model inversion attacks against collaborative inference. In Proceedings of the 35th Annual Computer Security Applications Conference, pp. 148-162, 2019.
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with a constrained variational framework. 2016.
10

---

Under review as a conference paper at ICLR 2023
Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. Measuring the effects of non- con-identical data distribution for federated visual classification. arXiv preprint arXiv:1909.063300 2019.
Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. Federated visual classification with real- world data distribution. In European Conference on Computer Vision, pp. 76-92. Springer, 2020.
Zonghao Huang, Rui Hu, Yuanxiong Guo, Eric Chan-Tin, and Yanmin Gong. DP-ADMM: admm- based distributed learning with differential privacy. In IEEE TIFS, 2020.
Eunjeong Jeong. Seungeun Oh, Hyesung Kim, Jihong Park, Mehdi Bennis, and Seong-Lyur Kim. Communicationn-effciieee on-device machine learning: Federated distillation and augmentation under non-iid private data. arXiv preprint arXiv: 181.1.149914 2018.
Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. Ad- vances and open problems in federated learning. Foundations and Trends@ in Machine Learning, 14(1-2):1 2021.
Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for federated learning. In International Conference Machine Learning, pp. 5132-5143, PMLR, 2020.
Muah Kim, Onur Günlü, and Rafael F, Schaefer. Federated learning with local differential privacy: Trade-offs between privacy, utility, and communication. In ICASSP. IEEE, 2021.
Vladimir Koltchinskii and Dmitry Panchenko. Empirical margin distributions and bounding the generalization error of combined classifiers. The Annals of Statistics, 30(1):1-50, 2002.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Qinbin Li, Bingsheng He, and Dawn Song. Model-contrastive federated learning. In Proceedings of the IEE//CV Conference on Computer Vision and Pattern Recognition, pp. 10713-10722, a.
Qinbin Li, Zeyi Wen, Zhaomin Wu, Sixu Hu, Naibo Wang, Yuan Li, Xu Liu, and Bingsheng He. A survey on federated learning systems: vision, hype and reality for data privacy and protection. IEEE Transactions Knowledge and Data Engineering, 2021b.
Qinbin Li, Yiqun Diao, Quan Chen, and Bingsheng He. Federated learning on non-iid data silos: An experimental study. In 2022 IEFE 38th International Conference on Data Engineering (ICDE), pp- 965-978. IEFE, 2022.
Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. Proceedings of Machine Learning and Systems, 2:429-450, 2020.
Xiaoxiao Li, Meirui Jiang, Xiaofei Zhang, Michael Kamp. and Qi Dou. Fedbn: Federated learning on non-iid features via local batch normalization. arXiv preprint 1rXiv:2102..0662, 20
Tao Lin, Lingjing Kong, Sebastian U Stich, and Martin Jaggi. Ensemble distillation for robust model fusion federated learning. Advances in Neural Information Processing Systems, 33:2351-2363, 2020.
Yunhui Long, Boxin Wang. Zhuolin Yang, Bhavya Kailkhura, Aston Zhang, Carl Gunter, and Bo Li. G-pate: Scalable differentially private data generator via private aggregation of teacher discriminators. Advances Neural Information Processing Systems, 34:2965-299 -9777, 2021.
Mi Luo, Fei Chen, Dapeng Hu, Yifan Zhang, Jian Liang, and Jiashi Feng. No fear of heterogeneity: Classifier calibration for federated learning with non-iid data. Advances in Neural Information Processing Systems, 34:5972-59844 2021.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera Arcas. Communication -iffiiattion-efiiennttt learning of deep networks from decentralized data. In Artificial intelli- gence and statistics, pp. 1273-1288 1282. PMLR, 2017.

---

Under review as a conference paper at ICLR 2023
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Mare G Bellemare, Alex Graves, Martin Riedmiller, Andreas Fidjeland. Georg Ostrovski, et al. Human-level control through deep reinforcement learning, nature, 518(7540):52-5333 2015.
Milad Nasr, Reza Shokri, and Amir Houmansadr. Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning. In SP, 2019.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
JaungJae Shin, Chihoon Hwang, Joongheon Kim, Jihong Park, Mehdi Bennis, and Seong-Lyun Kim. Xor mixup: Privacy-preserving data augmentation for one-shot federated learning. arXiv preprint arXiv:2006.05 2020.
Lichao Sun, Jianwei Qian, and Xun Chen. LDP-FL: practical private aggregation in federated learning with local differential privacy. In IJCAI, 2021.
Yue Tan, Guodong Long, Lu Liu, Tianyi Zhou, Qinghua Lu, Jing Jiang, and Chengqi Zhang. Fedproto: Federated prototype learning across heterogeneous clients. In AAAI Conference on Artificial Intelligence, volume 1, pp. 3, 2022.
Zhenheng Tang, Yonggang Zhang, Shaohuai Shi, Xin He, Bo Han, and Xiaowen Chu. Virtual homogeneity learning: Defending against data heterogeneity in federated learning, arXiv preprint arXiv:2206.022655, 2022.
Om Thakkar, Galen Andrew, and Brendan McMahan. Differentially private learning with adaptive clipping. CoRR, tts//9055038877 2019. http:://aaxxii.... www./o/gg///////eeeeeeeeeeeeeeee 03871
Florian Tramer and Dan Boneh. Differentially private learning needs better features (or much more data). arXiv preprint arXiv:201 1,6660 2020.
Aleksei Triastcyn and Boi Faltings. Federated learning with bayesian differential privacy. In Big Data. IEEE, 2019.
Dirk van der Hoeven. User-specified local differential privacy in unconstrained adaptive online learning. In NeurlPS, 2019.
Hongyi Wang, Mikhail Yurochkin, Yuekai Sun, Dimitris Papailiopoulos, and Yasaman Khazaeni. Federated learning with matched averaging. arXiv preprint arXiv:2002.064440 2020a.
Jianyu Wang, Qinghua Liu, Hao Liang, Gauri Joshi, and H Vincent Poor. Tackling the objective inconsistency problem in heterogeneous federated optimization. Advances in neural information processing systems, 33:7611-7623, 2020b.
Kang Wei, Jun Li, Ming Ding, Chuan Ma, Howard H. Yang, Farhad Farokhi, Shi Jin, Tony Q S. Quek, and H. Vincent Poor. Federated learning with differential privacy: Algorithms and performance analysis. In IEEE TIFS, 2020.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747 2017.
Qiang Yang. Yang Liu, Tianjian Chen, and Yongxin Tong. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology (TIST), 10(2):1-19, 2019.
Tehrim Yoon, Sumin Shin, Sung Ju Hwang, and Eunho Yang. Fedmix: Approximation of mixup under mean augmented federated learning. arXiv preprint arXiv:2107.00255 2021.
Danni Yuan, Xiaoyan Zhu, Mingkui Wei, and Jianfeng Ma. Collaborative deep learning for medical image analysis with differential privacy. In GLOBECOM, 2019.
Mikhail Yurochkin, Mayank Agarwal, Soumya Ghosh, Kristjan Greenewald, Nghia Hoang, and Yasaman Khazaeni. Bayesian nonparametric federated learning of neural networks. In International Conference on Machine Learning, pp. 7252-7261. PMLR, 2019.
12

---

Under review as a conference paper at ICLR 2023
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz Paz. mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412, 2017.
Lin Zhang, Li Shen, Liang Ding, Dacheng Tao, and Ling-Yo Yu Duan. Fine-tuning global model via data-free knowledge distillation for non-iid federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10174-10183, 2022.
Nanxuan Zhao, Zhirong Wu, Rynson WH Lau, and Stephen Lin. What makes instance discrimination good for transfer learning? arXiv preprint arXiv:2006. 2020.
Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. Federated learning with non-iid data. arXiv preprint 1rX.v:/806.00082.. 2018.
Ligeng Zhu, Zhijian Liu, and Song Han. Deep leakage from gradients. In NeurlPS, 2019.
13

---

Under review as a conference paper at ICLR 2023
A
APPENDIX
A.I
VISUAL
We show the visualization of data distribution in Figure 4 The LDA partition and the #C partition have the label skew and the quantity skew simultancously. And the Subset partition only has the label skew.
client
(a) LDA (a 0.1)
(b) #C
(c) Subset
Figure 4: Data distribution in various FL heterogeneity scenario. Different colors denote different labels and the length of each line denote data number.
A.2 GLOBALLY SHARED DATA
We display the globally shared data zp from four different datasets and the raw data compare our privacy protection. Firstly, the raw data in Figure shown in Figure 5
Figure Raw Data Model Inversion Attack.
Figure 6: Globally Shared Data up CIFAR-10.
A.3 MORE DETAILS OF FEDPD
Algorithm give us an intuitive explanation of how we deploy FedPD on FL algorithm c.g., FedAvg and Algorithm illustrates the procedure generate globally shared data.
14

---

Under review as a conference paper at ICLR 2023
Algorithm 2 Globally Shared Data Generation
Server input: generation process communication round T, noise mean jr, noise level 0² Client k's imput: local epochs Q, local datasets D
Initialization: server distributes the initial model w. 6 all clients,
Server Executes:
for each round t = 1,2,.... . T do
server samples subset of clients S (L,
Eies, (Di)
server communicates w², 0f to all clients
for each client & E Sg in parallel do "k.q-1 0 t CeeenGGenerationTTaiinggn end for w(+1, 00+1 1S DD'l wh, 11.Q-11 L.kES, end for for all clients with A: = 0,... do
uu'," ,0, M, 02)
Dí SharedDataGenerationn
0x. μl o²)
send D to server to construct globally shared dataset D end for
CiientGeneration Trainingg w.
6, (,
for each local epoch with O.... .Q- do
Wk.g++1 0k.g+1 +- PrivacyDistillationn (wk, Osr uh σ²) using Eq end for
Return Wk.Q-1, 1h.0-110x.Q-11 to server
A.3.1 HYPER-PARAN PARAMETERS
We fine-tuned learning rates in 0.0001, 0.001, 0.01,0.1 and report the best results and corresponding learning rate. In most case, use 0.01 the learning rate except SCAFFOLD and FedNova in SVHN under the a 0.1, E = 1, K = 100 setting, the learning rate 0.0001 and 0.001, respectively. Batch size is set as 64 in when K 10 and 32 for K 100. The number of clients selected for
aggregation on server side is per round for K 10, and 10 for = 100. The noise level in our experients is (0,00.15))
A.3.2 TRICK FOR FEDPD
sb van :1:
Tex Acctescy
50
=dy A2440
60
40
100
200
300
400
Round
Figure 7: The pink line indicates 6.25% dataset to train VAE and Auxiliary 200 rounds jointly and the whole dataset for only VAE G(::0) training 200 rounds in the following. Furthermore, the yellow uses the same data in the former 200 rounds the pink line but the complete dataset to train Auxiliary Classifier A(::w).
In addition, provide insight experiment the need for mixupdata (Zhang al. 2017) augmentation in our approach shown in Figure | As we can see, the absence of data leads to poor generalization of the auxiliary classifier on and adequate data for VAE G still has bad effects.
15

---

Under review as a conference paper at ICLR 2023
95.
Acc
Tea Accuray
Text Accuracy
Proowage
Privecy
-0-
92
65
75
90
0.2
03
0.1
Noise Level
Notse Level of
Noke Level of
(a) Test Accuracy on CIFAR-10 with different noise level o²
(b) Test Accuracy on SVHN with different noise level o²
(c) Test Accuracy on CIFAR-100
with different noise level o²
Figure 8: Prvaay-Performance results on different datasets
Tort Accuracy [Nd
80
Test Accuracy [&d
Accuracy [%]
80
60
60-
60
40
40
40
Packig Fecto
Tading
Tucleg Feaff
Pespyp
20
Teckfoa =fieto
Fediova
Tastis
Teslon
0
250
500
750
1000
100
200
300
400
250
500
750
1000
Round
Round
Round
(a) a 0.1, E 1, 10
(b) a 0.1, F = ==
(c) 001, F= 100
Figure 9: Convergence comparison on CIFAR-10.
Test Accuracy (16)
60
Test Accuracy [86]
60
Test Accuracy [56]
60
40
20
Teclp-Ficlt
Tasigg
20-
fasleg-facts Fedrro-fad00
Tackog Teciic
20
Recho-
0
F- mm
0
Prelou Prom
0
250
500 Round
750
1000
100
200 Round
300
400
250
500
750
1000
Round
(a) 0.1, E 1, 10
(b) 0.1, E =
=
0.1, E
100
Figure 10: Convergence comparison on CIFAR-100.
Test Accurary [%6
80
Test Accuracy aaryy [%]]
800
80
60
60
40
Pedleg Pecto
Fadleo
Tooley Foo
Pea0 Recheeifeat5
20
Peshaa 17PPPo 1ea
20
Telfrrm Teiff
20
fed507a
250
500
750
1000
100
200 Round
300
400
250
500
750
1000
Round
Round
(a) =-1, 10
(b) 0.1, E 5, = 10
(c) 0.05, F K 10
Figure 11: Convergence comparison on SVHN.
A.4 MORE EXPERIMENT RESULTS
A.5 MORE RELATED WORK
Federated Learning with heterogeneous data. In FL,, all distributed clients jointly train model across various distributed datasets for user privacy protection, while local data is not accessible to other clients. FedAvg (McMahan et al. 2017) is the first work proposed to reduce communication overhead and preserve privacy by more local training epochs and fewer communication rounds. However, some studies (Zhao al. 2018 Li al.) 2022) have pointed out that the divergence between FedAvg and centralized training is slight in the IID case. But, in heterogeneous distribution, there is considerable divergence between the different clients and centralized training, and the gap
16

---

Under review as a conference paper at ICLR 2023
Text Accuracy [16]
Test Accuracy [%)
80
Accuracy
60
60-
60
40
40
SCAROLD
20
20-
Reffrrm hefff
20
250
500
750
1000
100
200
300
400
250
500
750
1000
(a) a 0. 1, 100
(b) a = 0.1, F
== 10
a 0.05, F F, 10
Figure 12: Convergence comparison FMNIST.
accumulates during the FedAvg weighted aggregation, leading to the performance degradation of FL models.
Recently, a series work propose new learning objective to calibrate the update direction of local training from being too far away from the global model. FedProx (Li et al. as the regularization term in the objective function and provides a theoretical guarantee of convergence.
2020) adds a Le distance
Similarly, novel objective function also introduced in FedIR (Hsu et al. 2020) over a mini-batch by self-normalized weights to address the non-identical class distribution. SCAFFOLD (Karimireddy jet al., 2020) restricts the model using previous information. Besides, MOON (Li et al. [202 a) introduces constrastive learning at the model level to correct the divergence between clients and server.
Mcanwhile, recent works propose designing model aggregation schemes. FedAvgM (Hsu et al. 2019) performs momentum on the server side. FedNova (Wang et al. 2020b) adopts normalized averaging method to eliminate objective inconsistency. A study (Cho al. 2020) also indicates that biasing client selection with higher local loss can speed up the convergence rate. The coordinate-wise averaging of weights also induce noxious performance. FedMA (Wang et al., 2020a) conducts Bayesian non-parametric strategy for heterogeneous data. FedBN (Li et al. (202cc) focus on feature shift Non-IID and perform local batch normalization before averaging models.
Another existing direction for tackling data heterogeneity is sharing data. This line of works mainly to assemble the data of different clients to construct a global IID dataset, mitigating client drift by replenishing the lack of information of clients (Zhao et al. 2018). Existing methods include synthesizing data based on the raw data by GAN (Jeong et al., 2018 Long et al. 2021]. However, the
synthetic data is generally relatively similar to the raw data, leading to privacy leakage at some degree.
Adding noise to the shared data is another promising strategy (Chatalic al., [022 Cai et al. 202T). Some methods employ the statistics of data (Yoon et al. | et al.) 2020) to synthesize for sharing, which still contains some raw data content. Other methods distribute intermediate features (Hao et al.) 2021). logits (Chang et al. 2019 Luo et al. ct al. (002)). These tactics will increase the difficulty of privacy protection because some existing methods reconstruct images based on feature inversion methods (Zhao et al. 2020). Most of the above methods share information without privacy guarantee or with strong prevecciie but poor performance, posing the sowacy-perffrmance dilemma.
2021), or learn the new embedding (Tan)
Concretely, in FD (Jeong et al. 2018) all clients leverages a generative model collaboratively for data generation in homogeneous distribution. For better privacy protection, G-PATE (Long et al. 202 1) performs discriminators with local aggregation in GAN. Fed-ZDAA(Fedd-ZDAA. depending on which side to play augmentation, introduce zero-shot data augmentation by gathering intermediate activations and batch normalization(BB (BNN) statistics to generate fake data. Inspired by mixup data, MAFL (Yoon et al. 202T) propose FedMIx to share information by averaging local data which also brings about the privacy problem. Cronus (Chang et al., (019) transmit the logits information while CCVR (Luo et al. 2021) collect statistical inforamtion of logits to sample fake data. FedFTG (Zhang et al. 2022) use a generator to explore input space of local model and transfer local knowledge to global model. FedDF (Lin et al. (020)) utilizes knowledge distillation based on unlabeled data generator and then conduct AVGLOGITS. The main difference between FedDF and FedPD is that our method distill the privacy kept locally rather than distilling knowledge. We provide multi steps to protect privacy with drastic performance gain.
(Hao et al. 2021).
17

---

Under review as a conference paper at ICLR 2023
Differential privacy with federated learning. Recent works model memorization and gradient leakage confirms that model parameters are seemingly secure. Carlini et.al (Carliii et al. 2019) found that seaennddddadd-e-ssaae
memorization of sensitive data occurs early during training with no relation to data rarity and model size. Training with differential privacy (Zhu et al. 2019) Nasr et al. 2019) is feasible solution to avoid serious consequences, albeit at some loss in utility.
Differential privacy is a framework to quantify to what extent individual privacy in statistical dataset preserved while releasing the established model over specific datasets. It has spawned a large set of research topics in data-releasing mechanism and noise-adding mechanism. Particularly, noise-adding mechanism has been widely utilized in various differentially private learning algorithms for protecting whether an individual is in the dataset or not.
In federated settings, training with differential privacy, i.c., adding noise to the model/data, originally aims to protect local information of each clients. Say, an adversary should not be able to discern whether a client's s data was used for early training. Here, we summarize some works with high citation or from top venue. Yuan et al (Yuan et al. (019) apply differential privacy to protect medical images by adopting famous AlexNet and Gaussian mechanism. Huang et al (Huang et al.) 2020) integrate an approximate augmented L Lagrangian function and Gaussian noise mechanism for balancing utility and privacy in FL. Wei etal (Wei et al., 2020 perturb carly-trained parameters locally by adding noises before uploading them to server for aggregation. Both Huang al and Wei et al are first (to their knowledge) to analyze the relation between convergence and utility in FL. Andrew et al (Thakkar al.] 2019) explore
in federated setting rather than using fixed one. They show that adaptive clipping to gradients can perform as well as any fixed clip chosen by hand.
adaptive clipping
Kim et al (Kim et al.] 2021) provide noise variance bound that guarantees local DP after multiple rounds of parameter aggregations. They introduce a trilemma in privacy, utility, and transmission rate of federated stochastic gradient decent. Hoeven al (van der Hoeven 2019) introduce data- dependent bounds and apply symmetric noise in online learning, which allows data provider to pick noise distribution. Triastcyn al (Triasteyn Faltings) 2019) adapt the notion of Bayesian differential privacy to federated learning and make necessary analyses on privacy guarantee. Sun et al (Sun et al., (02T) explicitly vary ranges of weights at different layers in a DNN, and shuffle high-dimensional parameters at an aggregation for easing explodes of privacy budgets. All works above start to apply DP and its variants to federated setting for different goals/scenarios, which thus provide underlying security as DP guarantees.
A.6 DIFFERENTIAL PRIVACY
Proof of Theorem 3.4 is here.
Proof.
Definition A.1. (Privacy Loss). Let M: RR be a randomized mechanism with input domain D and range R. Let D, D' be pair of adjacent dataset and aux be an auxiliary input. For an outcome E P, the privacy loss at o is defined by:
(1)
Pr[M(aux, D) o] log Pr[M(aux, D*)
We need to compute the privacy loss on an outcome o as a random variable when the random mechanism operates on two adjacent database D and D'. Privacy loss is random variable that accumulates the random noise added to the algorithm/model.
We aim at an exact analysis on privacy via compositing multiple random mechanisms. For simplifica- tion, we start with particular random mechanism M and then generalize it. The mechanism M does not depend on database or the query but relies on hypothesis hp. For hp = O. the outcome O of M independent and identically distributed from a discrete random distribution Ohp=o ~ pt.o. (pt)(()) is defined to be: 8 for o = 0; (1 - ô)e² /(1 + e²) for 1; 6)/(1 + e²) for = 2: 0 for o = 3. For hp = 1, the outcome O of M. is Ohp== ~ pi.l. pr is defined to be: 0 for = 0: (1 - /) + e²) for
1; (1 8)e² /1 + et) for 2; for = 3.
Let R(c, 5) be privacy region of a single access to M.. Privacy region consists of two rejection regions with errors, i.e., rejecting true null-hypothesis (type-I error) and retaining false null-hypothesis
18

---

Under review as a conference paper at ICLR 2023
(type-Il error). Let c 6 be M parameters for defining privacy. R(M, D, D') of any mechanism M can be regarded as an intersection of {( ()) privacy regions. For an arbitrary mechanism M, we need to compute its privacy region using the (c), 8%) pairs. Let D. D' be neighboring databases and O be the outputting domain. Define (symmetric) P. P' be probability density function of the outputs MDD), M(D'), respectively. Assume a permutation и over O such that ('()) = (()))
Let denote the complement of rejection region. Since R(M, D') is convex, we have
P(S)) 2 -EAP'(S) + 5% = P(S) e P'(()))))))))))))))))) 8
Define Dt.r (P, (P() c P'S)). Thus, M's privacy region is the {(e) o & [0, co)] s.t. P(o) = e P')), = Dt, P')}. Next, we consider composition on random
mechanisms : M, By accessing M!. P(O o......, O =;;) - (o,))
By algebra on two discrete distributions,
(P (P')) = 1 (1 8) +
-e /11 + tcg
1=0
Hence, privacy region is an interaction of i regions, parameterized by 1 - (1 -1-60 (1)-))1,(11 - (6,).
19


Reviews:
[{"review_id": "r3Yi0isT9qV", "reviewer": "Reviewer_Yumu", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes an algorithm that distill and privatize the generalizable features to improve FL training.", "summary_of_the_review": "The idea seems interesting, but I'm mostly concerned about the privacy analysis.", "strength_and_weaknesses": "Strength:\nThe paper proposes an interesting idea to distill the generalizable features from samples to improve training.\n\nWeakness:\n1. The DP analysis is a bit unclear.\n- The paper says that we'll add noise to x_g to form x_p and all the x_p would be sent to the server. That looks like local DP which could usually cause problem to utility. But then the paper says \"considering all clients' data as a whole\", which seems to suggest something very different from local DP. The authors did not provide a rigorous of sensitivity and privacy analysis.\n- It is also unclear whether we're having a per-example privacy or per-user privacy.\n- I didn't see the privacy values in the experiments either.\n2. The paper claims to separate the generalizable and sensitive features, but it looks more like generalizable vs \"non-generalizable\" features / the reminder features to me. I don't think the paper has explained much about why the reminder features contain anything sensitive.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "GxzeBpdeJg", "reviewer": "Reviewer_52C4", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies privacy in a federated learning setting. It aims to improve upon methods that directly privatize data points or gradient updates using differential privacy by instead separating data points into \"sensitive\" and \"generalizable\" features and then only requiring the client to publish a noisy version of their generalizable features. Each client then constructs their own model based on all shared noisy data and their own raw data. Intuitively, the scale of noise required to privatize the generalizable features may be substantially smaller than that needed to privatize the raw data (as required for local DP), and this enables more accurate final models. The paper concludes with experimental results .", "summary_of_the_review": "The paper's explanation of its algorithm is unclear, no proofs or code are offered for verifying its reasoning, the privacy guarantee of the algorithm it sketches is suspect even apart from these technical details, and one of the results appears to be plagiarized, with an incorrect proof, from previous work. ", "strength_and_weaknesses": "Strengths: The idea of separating useful and sensitive features is conceptually interesting.\n\nWeaknesses: I think the paper has three serious problems.\n\n1. The description of the algorithm is unclear. After reading Section 3.2 several times, I still don't understand how the data distillation process works. Is it performed entirely locally? If so, what is the meaning of the E_{(x,y)} term? As far as I can tell, the paper never makes the actions taken by each client locally explicit. Algorithm 1 abstracts it away as a subroutine in Algorithm 2 in the appendix, and Algorithm 2 abstracts it away as calling an undefined PrivacyDistillation algorithm using Equation 1, though I don't see where the user's actual data point enters into the algorithm. If indeed the distillation process is entirely local, I'm confused about how generalizability is even measured -- how can somebody with one data point reason about which parts of it generalize? Do users have multiple data points? Turning to the privatization step, I do not see which part of the paper actually derives the additive noise scale required to privatize the generalizable features x_g when constructing the globally shared database. The relevant result appears to be Theorem 3.5, which only offers a vague qualitative statement that FedPD's noise standard deviation is \"much less than\" that of conventional federated learning, without a proof. Overall, the paper's clarity is low.\n\n2. The privacy argument seems wrong. As far as I understand it, the paper does not attempt to ensure that the distillation process itself is private. This makes me question the value of adding noise to the distilled generalizable features. The paper is not precise enough for me to make this argument exact, but it seems plausible that the scale of the generalizable feature is itself private information, i.e., certain data points will produce generalizable features with noticeably larger norms than other data points. If indeed noise is scaled to this norm (as I assume it must be -- if not, what is the advantage over privatizing the entire data point?) then the scale of the noise added alone could leak information. As mentioned above, the paper doesn't seem to actually describe how the noise is scaled, so I can't verify this concern, but I think it's a red flag that the paper does not attempt to reason about this possibility.\n\n3. The paper's Theorem 3.4 comes out of nowhere, its significance is not explained (are users applying it to privatize k of their generalizable points?) and it seems to have been plagiarized from previous work (see \"Details of Ethics Concerns\" below).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "CTVPryekDM", "reviewer": "Reviewer_Lg1w", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a novel method to tackle data heterogeneity in federated learning. Specifically, the authors proposed viewing the data as an addition of public features and private features and generate a set of public features to construct a globally public dataset to aid training. The empirical results demonstrate significant improvement over prior efforts.", "summary_of_the_review": "Although the experimental results seem good, critical privacy arguments are missing in the current paper (1. bounding sensitivity, 2. communicating label info). Based on current content, justifications and improvements are needed for this work. \n\n**UPDATES**: After reading the responses and other reviewers' comments, I still do not find the privacy argument convincing. The main concerns still lies in unbounded sensitivity and unmasked label info. Therefore, I decide to maintain my initial evaluation.", "strength_and_weaknesses": "Strengths:\n- The method is novel and the empirical results look promising.\n\nWeaknesses:\n- I'm concerned with the privacy guarantee imposed in the work. It doesn't seem that the authors state the clipping bound for Gaussian/Laplacian Mechanism nor does Theorem 3.4 include any information about how the privacy parameter $\\varepsilon$ relates to the noise scale $\\sigma$ and the clipping bound. That makes me concerned about whether the experimental results comes from unbounded $x_g$? Could the authors explain how sensitivity is bounded in the experiments and theory?\n- I'm confused with the separation of $x_s$ and $x_g$ here. Seems that $x_s$ has never been used alone, it seems that the solution is equivalent to generating synthetic data $x_g$ from true data $x$ using a generative model and send the synthetic data to the server. Could the authors explain the motivation of having $x_s+x_g=x$?\n- When communicating the synthetic data to the server, how are the labels communicated? It seems that $p(y|x)$ and $p(y|x_g)$ are the same? If that is the case, how to communicate the label information privately?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "7lkcMjxf3q_", "reviewer": "Reviewer_Kf6q", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper tries to tackle the problem of data heterogeneity in federated learning with differential privacy by attempting to divide the features into private and generalizable features. They ask the question of what is necessary to share to learn global models and can the remaining data stay on the client as local models. They provide empirical evaluation of their claims.", "summary_of_the_review": "In light of the strengths and weaknesses mentioned above, the weaknesses outweigh the strengths and I recommend the paper for rejection.", "strength_and_weaknesses": "Strengths:\n1. The paper proposes an interesting idea to try and improve federated learning training with differential privacy.\nWeaknesses:\n1. Theorem 3.4 is the standard adaptive composition result of differential privacy and it is not clearly stated that this is not one of the papers contributions.\n2. Theorem 3.5 is an informal statement at best, and in no way shows why the necessary noise to be added is lesser to just the generalizable features. Is the L2 sensitivity lowered in some way?\n3. In the experimental evaluations, the high near central training accuracies make it seem like there is some personalization done using the private data on each client $x_s$. Is this the case? If it is the case, the baseline of no local personalization is quite weak. Unfortunately, I couldn't verify this since the code wasn't attached.  \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject