One-shot Federated Learning via Synthetic Distiller-Distillate Communication
Authors: Junyuan Zhang
Songhua Liu
Xinchao Wang
Venue: NeurIPS 2024 poster (2024)
Keywords: Federated Learning
Dataset Distillation
Primary area: other

Abstract: One-shot Federated learning (FL) is a powerful technology facilitating collaborative training of machine learning models in a single round of communication. While its superiority lies in communication efficiency and privacy preservation compared to iterative FL, one-shot FL often compromises model performance. Prior research has primarily focused on employing data-free knowledge distillation to optimize data generators and ensemble models for better aggregating local knowledge into the server model. However, these methods typically struggle with data heterogeneity, where inconsistent local data distributions can cause teachers to provide misleading knowledge. Additionally, they may encounter scalability issues with complex datasets due to inherent two-step information loss: first, during local training (from data to model), and second, when transferring knowledge to the server model (from model to inversed data). In this paper, we propose FedSD2C, a novel and practical one-shot FL framework designed to address these challenges. FedSD2C introduces a distiller to synthesize informative distillates directly from local data to reduce information loss and proposes sharing synthetic distillates instead of inconsistent local models to tackle data heterogeneity. Our empirical results demonstrate that FedSD2C consistently outperforms other one-shot FL methods with more complex and real datasets, achieving up to 2.6 $\times$ the performance of the best baseline. Code: https://github.com/Carkham/FedSD2C

One-shot Federated Learning via Synthetic Distiller-Distillate Communication
Junyuan Zhang 1.2
Songhua Liu' Xinehao Wang's
National University of Singapore' junyuanpk@gmail. com, songhua. liu uud.....dduu xinchao@nus.edmig nus.edu.sg
Beihang University
Abstract
One-shot Federated learning (FL) is powerful technology facilitating collaborative training of machine learning models in single round of communication. While its superiority lies in communication efficiency and privacy preservation compared iterative FL., one-shot FL often compromises model performance. Prior research has primarily focused on employing data-free knowledge distillation to optimize data generators and ensemble models for better aggregating local knowledge into the server model. Prior research has primarily focused on employing data- free knowledge distillation to optimize data generators and ensemble models for better aggregating local knowledge into the server model. However, these methods typically struggle with data heterogeneity, where inconsistent local data distributions can cause teachers to provide misleading knowledge. Additionally, they may encounter scalability issues with complex datasets due to inherent two- step information loss: first, during local training (from data to model), and second, when transferring knowledge the server model (from model to inversed data). In this paper, we propose FedSD2C, a novel and practical one-shot FI. framework designed address these challenges. FedSD2C introduces distiller synthesize informative distillates directly from local data to reduce information loss and proposes sharing synthetic distillates instead of inconsistent local models tackle data heterogeneity. Our empirical results demonstrate that FedSD2C consistently outperforms other one-shot FL methods with more complex and real datasets, achieving to 2.6 x the performance of the best baseline. Code: https:// github. com/ Carrkaam/FodSD2C
1 Introduction
Federated learning (FL) has emerged as a cutting-edge technology that enables training global model across multiple clients without sharing their raw data [1. Original FL requires multiple
communication rounds for exchanging information between clients and servers. While this paradigm
yields better global model by frequent communication, such high communication costs along with the risk of connection drop errors make impractical and intolerable in real-world FL applications [2] 3. Moreover, frequent communication poses security risks such as man-in-the-minn theddiddee attacks 5
and privacy concerns 6
To address these issues, one-shot FL 7 has been proposed, requiring only single communication round, significantly reducing communication costs and concurrently diminishing vulnerability to malicious interception. Due to one communication round property, One-shot FL can also be easily scaled up to large-scale client scenarios, especially for cross-device settings [ Despite sufficient benefits, the limitation of a single communication round makes one-shot FL. fall short in accuracy compared to conventional multiple-round FL.
"Corresponding Author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

(1) 1 2
Two-tier information loss
Server
W
Wj
Ws
D
Client
S
Server model
Ensemble model
(2) Low quality of synthetic data
DENSE
Co-Boosting
*
Our FedSD2C
Real Data
DENSE
Co-Boosting
Our FedSD2C
Bad samples
Figure 1: Illustration of issues in one-shot FL based on DFKD: (1) Information loss occurs during the transfer from local data to the model and from the model back to the inversed data. (2) t-SNE plots of feature distributions of data generated by DENSF(left A), Co-Boosting(middl (edgh *). We randomly select five different classes (indicated by different colors) of real and synthetic data from TmneelmageNet. Bad samples are data generated by the DFKD-based method that deviates from the distribution of local real data.
), and our
To facilitate effective knowledge transfer from client models to the server model within single round, most previous one-shot FL. methods 7 g 9.
[0] have focused on knowledge distillation. Early approaches [1] 7 knowledge distillation to transfer knowledge from ensemble of client models 12 to the server model. While effective, these methods often require additional public datasets, which can be cumbersome unfeasible in real-world scenarios [3], Alternatively, data-free knowledge distillation (DFKD) is introduced to avoid the need for public datasets. For example, DENSE employs Generative Adversarial Networks (GANs) [4] data generators and an ensemble of client models as a discriminator to synthesize diverse data for knowledge transfer to server model in data-free manner. Co-Boosting [T0] extends DENSE by proposing mutually reinforcing approach to enhance synthetic data and the ensemble model for server training.
Nevertheless, such method of generating data implies two-tier information loss. First, due model capacity limitations, client models may struggle to encapsulate all information about local data, affecting the quality of the generated data. Second, the generated data do not fully represent the information within the model, as they are produced from random noise without explicit guidance. Therefore, some classes of synthetic data cannot have similar feature distributions to the original real data, as depicted in Figure I Moreover, data heterogeneity (15) in FL can result in inconsistent and misleading predictions from local models [16]. which has been shown to hinder knowledge distilla- tion [8]. Consequently, the server model trained on such noisy and information-loss generated data typically suffers significant performance degradation, particularly on complex datasets [19
In this paper, we propose FedSD2C (One-shot Federated Learning via Synthetic Distiller-Distillate Communication), novel and practical one-shot FL framework that introduces a pre-defined distiller for informative, privacy-enhance
and conmunication-effccienn distillate communication. In specific, FedSD2C first adopts V-information (00 based Core-Set selection method to distill the local dataset into an informative Core-Set. By capturing the diversity and realism through V-information, the distilled Core-Set fully encapsulates the information of the local data domain for training robust server model. However, directly transmitting the Core-Set, which may include the original samples, poses potential privacy risks and incurs significant communication costs, especially for high- resolution images. In this regard, FedSD2C employs two techniques to further distill the Core-Set into distillates, thereby enhancing privacy and reducing communication costs: Utilizing Fourier transform perturbation to alter the amplitude components of the Core-Set samples for distillate initialization, enhancing privacy while retaining semantic content; 2) Employing a pre-trained Autoencoder 21 provided by the server as distiller to distill the perturbed Core-Set into distillates and optimizing its V-information to be as close as possible to original Core-Set, thus minimizing information loss. Finally, clients transmit synthetic distillates to the server instead of inconsistent

---

models for knowledge transfer. Compared to generating noisy knowledge from inconsistent client models with two-tier information loss, end-to-end distillate synthesis minimizes information loss and their aggregation mitigates the impact of data heterogeneity. Through extensive experiments over various real-world datasets we show that our proposed method significantly surpasses the generated-based one-shot FL. methods. The contributions of this paper are:
We propose a new one-shot FL framework named FedSD2C which proposes to share synthetic distillates instead of generating noisy data from inconsistent models for server-side training.
To mitigate the potential of privacy leakage and reduce communication costs, propose two techniques: distillate initialization with Fourier transform perturbation and distillate synthesis with pre-trained Autoencoder.
We conduct extensive experiments over various datasets and settings. The results demon- strate the effectiveness of the proposed method which achieves up 2.7 the performance of the best baseline.
2 Related Work
2.1 One-shot Federated Learning
One-shot federated learning was first proposed by (22). which introduces a method to aggregate server model by distilling knowledge from ensemble of client models using public datasets. FedKT [I]] propose hierarchical knowledge transfer framework, enabling various types of classi- fication models. While their approaches demonstrate promising results, the requirement of public datasets which is inaccessible for privacy or transmission reasons limits their practical applications. To address the limitations associated with public datasets, DENSE [] introduces DFKD process utilizing an additional data generator trained on the ensemble model. Considering the challenge of high statistical heterogeneity, FedCVAE 9 proposes replacing the local training task with training conditional Variation Autoencoders. Furthermore, Co-Boosting [10] aims to enhance the perfor- mance of both the data generator and the ensemble model through a two-tier process. Despite these advancements, generating data through DFKD involves a two-tier information loss. Simultaneously. the inconsistency among client models due to data heterogeneity [T5] 23 24 25), further degrades the quality of generated data, introducing label noise and thus limiting the performance of the server model. In this work, we tackle these problems from the perspective of sharing synthetic distillates. By utilizing Core-Set selection and pre-trained Autoencoders as distillers, our proposed methods distill diverse and informative data for server training.
2.2 Dataset Distillation in Federated learning
Dataset Distillation (DD) was first introduced by (26). aiming to distill the knowledge of datasets into synthetic data while preserving the performance of the model trained on it. Early dataset distillation methods are formulated as a bi-level optimization problems (27]. where the outer loop optimize the
synthetic data via gradient matching 28 29 30,, distribution matching [1 32
and performance
matching (26], while the inner loop progressively trains model on the synthetic data. Considering
34] based on kernel ridge regression are proposed to decouple the bi-level optimization, thereby reducing training cost. These methods demonstrate comparable performance in non-complex datasets like CIFARIO 10 and are
computational resources constraints, single-level optimization methods 333
implemented in FL to tackle communication bottlenecks 36), data heterogeneity [37 38 39 and
one-shot FL [40] 36). However, these methods require significant computational resources, making them impractical for edge devices with limited capability in FL. Additionally, they may struggle to effectively distill high-resolution datasets.
Methodology
3.1 Overview
We proposed FedSD2C to alleviate the two-tier information loss inherent in one-shot FL methods based on DFKD and account for data heterogeneity by synthetic distiller-distillate communication.
3

---

Core-Set selection
Distillate synthesis
Distillate synthesis
Fourier
transform
syn
X
Client
X
perturbttion
z
upload
Server
Core-Set selection
Ws
Distillate synthesis
Z
S
Server model
X
Client.
X
Encoder and Decoder of the pre- pre-trined Auteencoder
Figure 2: Framework of proposed FedSD2C.
The details FedSD2C are described in Figure 2 and Algorithm In the preparation phase of one-shot FL, the server distributed pre-trained Autoencoder [21] as the distiller to each client. Sub- sequently, clients synthesize informative, privacy-enhanced, and communication-effiiiee distillates for server-side training. Specifically, ensure the distillates fully encompass local information, clients first utilize a V-information-baat mative Core-Set from their local data domains. Aiming further reduce the communication costs and enhance the privacy of distillates, clients then perturb the Core-Set with Fourier transform for distillate initialization and employ the received pre-trained Autoencoder to optimize distillates in compact latent space via V-information alignment with the Core-Set. Finally, clients transmit the distillates, and the server decodes them using the pre-trained Autoencoder for training. We will now delve into the details of each component.
Core-Set selection method to extract diverse and infor-
3.2 V-information based Core-Set Selection
V-information 20 was first proposed to measure the mutual information between X and Y con- strained on predictive family which denoted as:
Iv(X + Y) Hy()) (Y
(I)
where ()) and (X) denote the predictive V-entropy conditioned on or X.
Core-Set selection is type of dataset distillation method that focuses on preserving subset of the original training dataset containing only valuable or representative samples. The objective is to enable models trained on this subset achieve performance similar to those trained on the entire dataset. Typically, this is achieved by minimizing certain criteria, such data distribution [41]. In our case, the emphasis lies on maximizing the diversity and information content the subset. Therefore, in the context of V-information, Core-Set selection can be reformulated as:
(X Y,) arg max (x, + Y;) X,Y
where X, and Y, denotes the images and labels in the original datasets and (X,, Y,) denotes the selected Core-Set. The intuition behind this equation is that Core-Set should include sufficient information and provide concise representation corresponding to original datasets, constrained by observers V.
Inspired by 42 maximize the V-information of the Core-Set from two levels. First, we identify the most informative image segments within each image by evaluating patches extracted at various scales from each image. Second, we select the top-ipe with the highest V--information for each class to construct the final Core-Set. The algorithm description can be found at Appendix A] Since model pre-trained V-information 42), we proposed using local pre-trained models as the observer models (predictive family V) conduct D-information-base Core-Set selection on local datasets. Given that pre-trained local models are commonly present in one-shot FL 8 77 their utilization does not violate the practicality of our approach.
original datasets
optimal observer for approximating the
3.3 Distilling Core-Set into Distillates with Pre-trained Autoencoders
In this section, we will describe how to synthesize informative, privacy-enhanced, and communication- efficient distillates using pre-trained Autoencoders, Although Core-Set significantly reduces dataset

---

size, communication costs remain a challenge for high- nigh-resolution data in real-world applications. Moreover, while sharing Core-Set provides consistent resistance to membership inference attacks [4) transmitring patches may still risk exposing sensitive information of original images. Therefore, further enhancing the privacy of shared data is essential. To alleviate these concerns, we propose two novel techniques: 1) distillate initialization with Fourier transform perturbation, which alters the amplitude components of the Core-Set samples to enhance privacy while retaining semantic content; and 2) distillate synthesis with pre-trained Autoencoders which act the distillers. The pre-trained Autoencoder converts the perturbed samples into distillates by optimizing their V-information to be as close as possible to the original Core-Set samples, minimizing information loss. We will now discuss each component in detail below. The process is described in Algorithm I
Distillate initialization with Fourier transform perturbation. A typical privacy-enhancee tech- nique for sharing synthetic data adding noise [44]. Although this approach can blur the visual information of the synthetic data, it can also destroy important semantic information and seriously degrade model performance. Our approach leverages a well- kell-known property of the Fourier transform: the phase component of the Fourier spectrum encodes high-level semantic information, whereas the amplitude component captures low-level details [45] 46). Inspired by this, we propose a novel privacy-enhance method that perturbs the amplitude components in Core-Set samples through the Fourier transform to reduce visual information while preserving semantic information. Given an image z, its Fourier transform can be formulated as:
F(x) A(x) xe -ixP((()
(3)
where A(x), P(x) depict the amplitude and phase components respectively. We then perturb the amplitude information via linearly interpolating:
A(z) (1 A)A(T)) + XA(x*)
(4)
where the À is a scaling coefficient and x² can be other images or random noise. Then, we combine the perturbed amplitude spectrums with the original phase component to generate the perturbed Core-Set sample:
== x (x((()))
(5)
where (z) defines the inverse Fourier transform which can be calculated with the FFT algo- rithm 47 effectively.
Distillate synthesis with pre-trained Autoencoders. The Fourier transform perturbation serves to protect visual privacy but also compromises the realism of images, resulting in inconsistent V- information between the perturbed Core-Set and the original Core-Set. To address this issue, we propose optimizing the alignment of the Y-information between the perturbed Core-Set and the original Core-Set to reconstruct key information. One straightforward and efficient approach is to optimize the perturbed Core-Set in pixel space [48] 49]. However, this method can be prone to overfitting into nigh-frequency patterns that only match the observer model [50]. Such overfitting is detrimental to training global model due inconsistent local models caused by data heterogeneity. Leveraging the powerful priors obtained from large-scale datasets, a pre-trained Autoencoder can decode latent representations into generalizable images. As result, optimizing in the latent space acts as a regularization method that encourages synthetic data to be more generalizable, thereby making pre-trained Autoencoder an ideal distiller for local Core-Sets distillation. Furthermore, compact latent representations can reduce communication costs and mitigate privacy leakage if the latent intercepted by attackers. Consequently, we employ a pre-trained Autoencoder on the client to distill the Core-Set into informative, privacy-enhanced, and communication-efiienn transmit them the server for model training. On the server side, they are decoded by the decoder and are expected to maintain similar Y-information to the original Core-Set from the perspective of the observer model while remaining visually unidentifiable.
distillates, and
In specific, in the preparation phase of one-shot FL, the server first distributes a pre-trained Au- toencoder to n clients, denoted as & and D for encoder and decoder, respectively. Each client i then conducts y-information-bart based selection to construct a Core-Set (X;, 8, Y i = 1, 1,2.... n with diverse information regarding the original local datasets. Subsequently, client i learns a latent set
Z' |21 initialized by {E ())))))) xrj X which is perturbed with Fourier transform, such that the {3 as close as possible to the corresponding data in the Core-Set:
arg min [xx + Yy) Tv. ((D((( r3cl
2'
5

---

Algorithm One-shot Federated Learning via Synthetic Distiler-Dissillate Communication
Require:
Client local model (+))),)), pre-trained VAE (D,E), Server model parameter number of clients n, training iterations of local synthesis Tsyn, training iterations of server model Terns learning rate Nayn and Vty
1: Server distributes pre-trained VAE (D, E) to clients.
2: for each client i = 1, , do
3: (X3, Y2) &- CoresetSelection(()
D See Algorithm 2 in Appendix
4: for each z) X; do
5: Perturb z) via Equations (3)
and (5)
6: end for
7: Initialize latent set z {8(x)) xx
using pre-trained VAE decoder &
8: for / 1, Syn do
9:
for mini-batch (=' x²)) E (Z) do
10:
Compute synthetic loss Ssyn based on Equation (7)
11:
+- Tyyn V2thuyn
12:
end for end for
13:
14: Generate soft label for each synthetic latents Y = (F(()
15:
Transmit S (2', Y to the server
end for
17: Combine client synthetic data into S =
2", U.....
18: for = 1,... Tern do
19:
for mini-batch (z, y) E (z.Y.) do
20: Compute Livn based on Equation (8)
21:
+ 0 Mer Vihn
22:
end for
23:
end for
As Core-Set selection employs the local pre-trained models as the observer models Vi, the reformu- lated Eqquttion (6) and objective function can be formulated as:
arg min (Ax(D(=))) - h=(
(D(e))) 1 N
Sagn
Σ
he hx(3t)
where h.()) denotes the feature extractor of pre-trained local model of client i, and w/ and 2f
paired. By minimizing Cayns we synthesize a set of latent variables Z { |22 that contains diverse information of local data domains.
Finally, clients transmit the latent set Z' along with the corresponding soft label Y predicted by local models to the server. The server combines the synthetic local distillates from each client S = (Z. Y.). then uses decoder D reconstruct the images from data (z. y) Y.) and distills the knowledge by minimizing the following objective function:
Lirn
Σ (L(f(b(D()))))))
(8)
(2.Y)
where fh(()) denotes the server model. By minimizing the KL loss, we can transfer the local knowledge in the distillate to the server model.
Discussion on privacy. We first consider whether an attacker can train performant model with the intercepted distilled data and labels during transmission 40 99. Because the attacker cannot know that the distilled data encoded by VAE, nor can the attacker access the pre-trained VAE encoder, which can be easily achieved by being predefined offline or via encryption, it is hard for the attacker to reproduce an effective model. For model inversion and membership inference attacks, according to [51]. there has been prior research has performed these attacks solely using distilled data and

---

Table Accuracy of different one-shot FL methods over three datasets with ConvNet and ResNet- 18. We vary the {0.1, 0.3, 0.5) to simulate different levels of data heterogeneity for Tiny-ImageNe and Imagenette and use pre-defined splits for OpenImage. "Central" means that clients send all their local data to the server for centralized training, representing the upper bound of model performance.
ImageNettt
Tiny-ImageNet
Openlmage
Model
Methods
0.1
0.3
Central
89.60
49.73
33.61
FedAVG
10.68+0.23
10.04=0.10
9.83+-0.77
0.17
F-DAFL
44.95:1072
52.23=0.233
58.34:0.55
5.25#0.41
8.89=0.61
10.28=0.1 10
3.36=0.56
Conv
DENSE
42.09:40.68
48.64=1191
54.74=0.75
11.45:0.08
14.69+0.48
15.15=0.22
7.00=0.84
Boosting
39.36:10.70
16.1
58.60:1.02
6.66
0.81=00.26
10.75=0.11 11
13.59:10.99
FedSD2C
10.68:0.20
57.89 0.96
58.17:0.51
20.73:10.12
0.15
24.10:0.30
23.00:10.24
Central
90.00
61.98
34.17
FedAVG
9.86=0.13
10.06=00.0
10.76=0.35
1,68=0.16
F-DAFL
37.86210.8
39.52=00.46
46,06=0.16
7.91
12.30:10.36
13.31=0.56
12.75:10.14
ResNet-18
DENSE
38.37:40.66
47.85=2.17
49.78+2.11
8.88
13.05=4..36
17.24=0.43
14.85+00.2
Boosting
27.06:10.61
28.53=00886
30.53::111
10.29:10.43
14.35:10.93
16.39=0.59
9.52:11552
FedSD2C
47.52:10.51
53.69 0.17
55.90=0.53
26.83:10.10
29.92:10.37
31.66
22.69:00.14
have also revealed the advantage dataset distillation in this regard. Therefore, we employ the synthetic data the reconstructed samples for the evaluation of model inversion attacks. Furthermore, we compare our proposed Fourier transform perturbation with other privacy-enhanced techniques, including adding random noise synthetic samples and data augment 53 Experimental results can be found in Section 4.3.
labels and previous works [52 36 35 43
4 Experiments
4.1
Experimental Setup
Datasets and partitions. We conduct experiments on three real-world image datasets with different ranges of resolution including Tiny-ImageNet [54], ImageNette (55]. and OpenImage [56]. Tiny- ImageNet contains 10000 images of 64 x64 resolution across 200 classes. ImageNette is a widely used subset of 10 classes from ImageNet-1 K [57] with 9469 color images, resized to 128 x 128. Openlmage is large-scale real-world vision dataset with over 9 million images of 256 resolution. To simulate data heterogeneity in real-worke applications of one-shot FL, we use Dirichlet distribution to generate non-lID data to generate non-IID local data, as in (58) for Specifically, for client i, we sample pi. Dir(a) to allocate a P proportion of class k client i. The parameter controls the degree of data heterogeneity, with smaller indicating severe data heterogeneity. The set to 0.1 by default unless otherwise stated. For OpenImage, we randomly choose n real-world clients from FedScale [59] and use their corresponding test sets to form global sets. We set the default number of clients n to 10, unless otherwise specified.
and ImageNette.
Baseline methods and Configurations. We compare our proposed FedSD2C with existing methods: FedAVG D. DENSE (8) and Co-Boosting [10). Following [8] TO], we also introduce DAFL. (60) with one-shot FL settings, denoted as F-DAFL. We use two different model architectures: ConvNet 28 and R [6] for all methods. In FedSD2C, the image per class ipc is set to 50 for Tiny- ImageNet and ImageNette, and 10 for OpenImage. We set the Fourier transform coefficient À = 0.8 and use a public pre- traitrdined Autoencoder from Stable Diffussion (62) by default for all tasks. For distillate synthesis, we set T 50, Msyn 0.1 by default. Mem is set to 0.2 for Tiny- ImageNet and 0.02 for ImageNette and OpenImage. More experimental details can be found in the Appendix.
4.2 Evaluation Results
To evaluate the effectiveness of our method, we conduct experiments under various non-IID set- tings with a 10.1, 0.3, 0.5} for Tiny- ImageNet and Imagenette and pre-defined splits [59] for OpenImage. As illustrated in Table 1, our proposed FedSD2C surpasses all other methods in most settings. In particular, under extreme data heterogeneity(a = 0.1), FedSD2C achieves up to 1.3x, 2.6x, and 1,8x the accuracy of the best baseline on ImageNette, Tiny-ImageNer and OpenImage, respectively. This superior performance is attributed to FedSD2C's approach of sharing synthetic
7

---

Table Accuracy, PSNR and SSIM of FedSD2C combining different Laplace and Gaussian indicate adding corresponding noise into synthetic distillates without Fourier transform initialization. FedMix denotes averaging two real samples from Core-Set to synthesize data. "-" indicates no privacy-enhanced technique is combined.
techniques.
Privacy-rrserving techniques
ImageNette
Tiny-ImageNet
ConvNerf
ResNet-18f
PSNR1
SSIML
ConvNert
ResNet-18 18
PSNRI
SSIM,
51.87
51.82
22.62
28.29
Ours(A 0.1)
51.26
50.55
23.48
73.20
22.03
28. 28.22
20.54
54.89
Ours(A 0.5)
51.36
48,97
19.97
64.23
21.77
28.09
18.06
44.18
Ours(A 0.8)
50.68
47.52
16.42
50.80
20.85
26.83
16.95
35.89
Laplace(s = 0.2.p = 0.1)
48.61
45.25
24.02
81.66
21.30
27.48
22.25
73.09
Gaussiam(s== = 0.2,, 0.1)
48.31
46,70
24.82
85.89
21.48
27.51
23.38
78,90
Laplace(s 0.2.p 0.2)
45.61
38,01
20.05
73.13
19.32
23.66
19.99
64.51
Gaussian(s 0.2,p 0.2)
45.81
38.09
20.30
76.11
19.32
23.52
20.35
68.56
FedMix
41.86
37.76
16.88
58.93
13.86
16.26
16.43
56.91
distillates rather than inconsistent local models, thereby mitigating the impact of data heterogeneity. Moreover, FedSD2C demonstrates the independence from model structures. In contrast, other meth- ods struggle to adapt to different model structures and complex datasets. For instance, at a = 0.5, Co-Boosting with ResNet- 18 achieves only half the accuracy of ConvNet ImageNette, whereas FedSD22C maintains consistent performance. This discrepancy arises because differences in model capacity affect their ability to condense local knowledge and the two-tier information loss during data generation increases the difficulty of transferring local knowledge to the server model, resulting in poor robustness to complex datasets and varied networks. In contrast, the shared distillates in FedSD2C are synthesized through end-to-end local distillation, mitigating information loss during knowledge transfer.
4.3 Analysis of Our Method
4.3.1 Privacy Evaluation
For privacy ovaluation, we consider an connestbuutcurronn server attempting to reconstruct client data from distillates. We compare our proposed FedSD2C with other privacy-enhanced techniques and FedMix [33. In the random
for sharing synthetic data, including adding random noise 44 63
noise approach, we incorporate it into FedSD2C by removing the Fourier transform perturbation and instead directly using Core-Set samples for initialization. We then add random noise to the synthetic distillates before transmitting them to the server, following the methods in 44 633. Specifically, Given latent z, a perturbation coefficient p, randomly generated noise e and its scale parameter 8, the data to be shared is formulated as 2 (1 2) + e x s. FedMix (53) proposes using linear interpolation of real samples to preserve privacy. In this approach, we synthesize data by averaging each two real samples from the Core-Set. For our proposed Fourier transform perturbation, we vary the x = {0.1, 0.5, 0.8} and observe the variations in performance and privacy protection. To quantitatively evaluate the privacy protection of the synthetic data, we employ the Peak Signal-to- Noise Ratio (PSNR) and Structure Similarity Index Measure (SSIM). A higher PSNR or SSIM value indicates greater similarity between the synthetic samples and the original samples, which implies more severe privacy leakage. We calculate the average PSNR and SSIM values of all the synthetic samples.
As depicted in Table 2 PSNR and SSIM values, it comes at the expense of significant performance degradation. The application of random noise requires a delicate balance between performance and privacy protection. For example, with perturbation coefficient p 0.2, it offers similar privacy protection to that of FedMix, but the performance drops approximately 10% compared to p 0.1. However, the p 0.1 setting increases the risk of privacy leakage. In comparison, the synthetic distillates generated by our proposed FedSD2C achieve comparable PSNR values with them. This suggests that our proposed Fourier transform perturbation offers effective privacy protection for the real data sample. Furthermore, FedSD2C consistently outperforms other methods in terms of accuracy while maintaining a minimal performance degradation compared no privacy protection techniques. This indicates that FedSD2C strikes balance between privacy preservation and performance. We also
although FedMix provides better privacy protection, as evidenced by lower
8

---

Table 3: Comparison of communication costs and accuracy 0.1 with ResNet-18. Results highlighted in bold represent outcomes with default ipc settings. Acc. and Comm. denote accuracy and communication costs, respectively.
ImageNette
Tiny-Iman Liny-ImaeeNet
OpenImage
Method
apc
ipc
Acc.
Comm.
Acc.
Comm.
Acc.
Comm.
DENSE
38.37
44MB
8.88
44MB
14.85
44MB
Co-Boosting
27.06
44MB
10.29
44MB
7.00
44MB
FedSD2C w/o AE
19.90
0.48MB
3.60
2.2MB
12.89
2.6MB
20
43.10
0.23MB
23.23
L.MB
5
20.73
1.6MB
FedSD2C
50
50.68
0.5MB
26.83
2.
10
22.69
2.0MB
80
56.13
0.73MB
27.71
2.9MB
15
23.49
2.7MB
perform membership inference attacks FedSD2C and other methods, please refer to Appendix C.2 for more details.
4.3.2 Scalability of Communication Efficiency
By employing Core-Set selection and communication-effieee distillate communication, our FedSD2C condenses local data mere MBs, while the model trained these condensed data exhibits comparable performance, as shown in Table 3 Specifically, the communication costs of FedSD2C for sharing synthetic distillate is most 4% of that of sharing model. Notably, we exclude the communication costs of sending and receiving pre-trained Autoencoder, as this can be pre-defined offline, allowing for the reuse of multiple one-shot FL tasks. Given the considerable capacity for communication costs, we further investigate the scalability of communication efficiency and perfor- mance. Our experimental results demonstrate that increased data transmission enhances the diversity of compressed data, leading to further improvements in performance. By increasing ipe from 20 80, the accuracy boosts by 13.03% and 4.48% on ImageNette and Tiny-ImageNet respectively. Additionally, we compare FedSD2C without Autoencoder at equivalent communication costs, where the absence of synthesized images limits performance to best half of the default setting. The communication--fficiiee of FedSD2C highlights its practicality in real-world applications.
4.3.3 Impact of Pre-trained Autoencoder
Pre-trained Autoencoders are typically trained on natural data domains [64], while practical ap- plications of federated learning often involve a broader range of domains, such as medical im- ages. This raises the question of whether pre-trained Autoen- coders remain effective when ap- plied different domain and whether our proposed FedSD2C can adapt to these differences. investigate this, we evaluate per- formance using a medical dataset FO [65].
52-
Accurney
50-
FedSD2C Co- Beosting DENSE
Accumcy
52-
48
Random
Pre mained
44
50
300
500
800
1000
50
300
500
sóo 1000
Iteration of synthesis T,m
Iteration of synthesis T,,.
(a) COVID-FI.
(h) ImageNette
Figure 3: (a) Experiments on the medical image data domain. Adopting pre- pre-trained Autoencoders on other data domains can reduce performance. However, this can be mitigated by increasing Tayn- (b) Experiments of FedSD2C with randomly initialized downsampling and upsampling modules (blue line) compared to pre-trained Autoencoders (orange line) on ImageNette. Without pre-trained knowledge, FedSD2C requires a higher T Tayn for distil- late synthesis but can still achieve comparable results. ResNet-18 is used for both experiments.
As shown in Figure 3al using the default synthesis iteration of Taym 50 yields suboptimal results. By increasing Tegm to 1000, the performance improves and then stabilizes. This observation suggests that the pre-trained knowledge of Autoencoders may influence the speed of distillate synthesis convergence. To fur- ther validate this, we replace the encoder and decoder of pre-trained Autoencoders with randomly initialized downsampling and upsampling modules. We vary Tayn from 50 to 1000 and compare

---

Table 4: Accuracy on Imay-ImageNer with different client amounts.
Number of clients
F-DAFL
DENSE
Co-Boosting
FedSD2C
0.1
4.97
11.36
7.37
21.92
n=20
a 0.3
7.66
13.69
9.95
22.00
& 0.5
10.01
14.86
10.45
22.87
& = 0.1
3.99
8.32
7.05
21.48
n=50
0.3
5.92
12.25
8.80
22.38
a 0.5
6.94
13.06
8.90
22.58
a 0.1
3.12
7.87
4.27
20.34
n=100
& = 0.3
5.26
10.22
7.15
21.86
Q 0.5
6.30
11.49
8.32
21.76
its performance with employing pre-trained Autoencoders ImageNette, setting ipc 80 for better illustration. FFiuure 3b demonstrates that as Tsyn increases, the performance of FedSD2C with randomly initialized modules improves progressively, eventually matching the performance of FedSD2C with pre-trained Autoencoders. In summary, while pre-trained knowledge can enhance convergence rate, FedSD2C achieve comparable performance by adjusting T Tayns demonstrating its adaptability across domains.
4.3.4 Impact of Client Scales
As practical FL deployments often involve participating clients [59], evaluate FedSD2C with various numbers clients n = {20, 50, 100] and maintain consistent communication budget by setting êpc (40, 20, 10), respectively. We compare these methods under Image-ma with data heterogeneity a = 10.1, 0.3, 0.5} for partitions and employ ConyNet. As depicted in Table 4 FedSD22 consistently achieves the highest accuracy the number of clients increases. Moreover, FedSD2C demonstrates greater robustness to the number of participants. Specifically, as the number of participants changes, the accuracy of FedSD2C fluctuates within only 1%. In contrast, the accuracy of F-DAFL, DENSE, and Co-Boosting dropped by up to 4.71 %, 3.49%, and 3.10%, respectively under different settings. This further validates the utility of sharing synthesized distillates real-world one-shot FL applications.
Limitations
The local distillation process introduces additional computational overhead. While Core-Set selection requires no training and the distillate synthesis process only requires 50 iterations with a speed of 0.4s/per image RTX3090, still imposes higher resource requirements compared to the method of sharing model. One direction worth exploring is to integrate with the model market [66] enable clients synthesize distillates once for permanent use.
the local device
6 Conclusion
In this paper, we propose a new one-shot FL framework driven by distiller-distllltte communica- tion, denoted FedSD2C, alleviate the information loss of knowledge transfer and impacts of data heterogeneity. FedSD2C compresses the local data into Core-Set with V-information and employs pre-trained Autoencoder as the distiller distill informative, communication--ffiiieent and privacy-enhanced distillates from Core-Set. Moreover, We discuss FedSD2C's resistance to attackers intercepting distillate communications and attacks from honest-but-curious servers and introduce Fourier transform perturbation to further minimize the risk of privacy leakage. Empirical results validate the effectiveness of FedSD2C in transferring local knowledge to the server in one-shot FL while balancing communication efficiency and privacy protection.
Acknowledgment
This project is supported by the National Research Foundation, Singapore, under its AI Singapore Programme (AISG Award No: AISG2-RP-2021-023))
10

---

References
[1] Brendan Memahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera Y Arcas. Communication-effiee learning of deep networks from decentralized data. In Artificial Intelligence and Statistics, pages 1273-1282. PMLR, 2017.
[2] Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine, 37(3):50 60, 2020.
[3] Peter Kairouz, H Brendan Mcmahan, Brendan Avent, AurELien Bellet, Mchdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, and Others. Advances and open problems in federated learning. Foundations and Trends@ in Machine Learning, 14(1-2)11210, 2021.
[4] Junyuan Zhang, Shuang Zeng, Miao Zhang. Runxi Wang. Feifei Wang, Yuyin Zhou, Paul Liang, and Liangqiong Qu. Flhetbench: Benchmarking device and state heterogeneity in federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages June 2024.
[5] Derui Wang, Chaoran Li, Sheng Wen, Surya Nepal, and Yang Xiang. Man-in-the-middh attacks against machine learning classifiers via malicious generative models. IEEE Transactions on Dependable and Secure Computing, (85))2007420877, 2020.
[6] Hongxu Yin, Arun Mallya, Arash Vahdat, Jose Alvarez, Jan Kautz, and Pavlo Molchanov. See through gradients: Image batch recovery via gradinversion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 16337-16346, 2021.
[7] Neel Guha, Ameet Talwalkar, and Virginia Smith. arxiv:1902.11177 2019.
One-shot federated learning. arxiv preprint
[8] Jie Zhang, Chen Chen, Bo Li, Lingjuan Lyu, Shuang Wu, Shouhong Ding, Chunhua Shen, and Chao Wu. Dense: Data-free one-shot federated learning. Advances in Neural Information Processing Systems, 35:21414-21422 2022.
[9] Clare Elizabeth Heinbaugh, Emilio Luz-Ricea, and Huajie Shao. Data-free one-shot federated learning under very high statistical heterogeneity. In ICLR, 2022.
[10] Rong Dai, Yonggang Zhang. Ang Li, Tongliang Liu, Xun Yang, and Bo Han. Enhancing one-shot federated learning through data and ensemble co-boosting, arxiv preprint arxiv:240.110770, 2024.
[11] Qinbin Li, Bingsheng He, and Dawn Song. Practical one-shot federated learning for cross-silo setting. arxiv preprint arxiv:2010.000 01011, 2020.
[12] Xingyi Yang, Daquan Zhou, Songhua Liu, Jingwen Ye, and Xinchao Wang. Deep model reassembly. Advances in neural information processing systems, 35:25739-25777 2,753,, 2022.
[13] Zhuangdi Zhu, Junyuan Hong, and Jiayu Zhou. Data-free knowledge distillation for heterogeneous federated learning. International Conference on Machine Learning, pages 12878-12889. PMLR, 2021.
[14] Ian Goodfellow. Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in Neural Information Processing Systems, 27, 2014.
[15] Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. Federated learning with non-iid data. arXiv preprint 1806.008000 2018.
[16] Jiawei Shao, Fangzhao Wu, and Jun Zhang. Seleetive knowledge sharing for privacy- federated distillation without good teacher. Nature Communications, 15(1):349, 2024.
[17] Sidak Pal Singh and Martin Jaggi. Model fusion via optimal transport. Advances in Neural Information Processing Systems, 33:22045-22055, 2020.
[18] Xingyi Yang, Jingwen Ye, Xinchao Wang. Factorizing knowledge neural networks. In European Conference on Computer Vision, pages 73-91. Springer, 2022.
[19] Shikang Yu, Jiachen Chen, Hu Han, and Shuqiang Jiang. Data-free knowledge distillation via feature exchange and activation region constraint. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 24266 24275, 2023.

---

[20] Yilun Xu, Shengjia Zhao, Jiaming Song, Russell Stewart, and Stefano Ermon. A theory of usable information under computational constraints. arxiv preprini arxiv:2002. 2020.
[21] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arxiv preprint arxiv:1312.6114. 2013.
[22] Neel Guha, Ameet Talwalkar, and Virginia Smith. One-shot federated lcarning. arxiv preprint arxiv: 11175, 2019.
[23] Liangqiong Qu. Yuyin Zhou, Paul Liang. Yingda Xia, Feifei Wang, Ehsan Adeli, Fei-Fei, and Daniel Rubin. Rethinking architecture design for tackling data heterogeneity in federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10061-10071, 2022.
[24] Hangyu Zhu, Jinjin Xu, Shiging Liu, and Yaochu Jin. Federated learning non-iid data: A survey. Neurocomputing, 465:371-390, 2021.
[25] Shuang Zeng. Pengxin Guo, Shuai Wang, Jianbo Wang. Yuyin Zhou, and Liangqiong Qu Tackling data heterogeneity in federated learning via loss decomposition. International Conference on Medical Image Computing and Computer-Assiste Intervention, pages 707-717. Springer, 2024.
[26] Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba, and Alexei A Efros. Dataset distillation. arxiv preprint arxiv: 181 1.10599, 2018.
[27] Ruonan Yu, Songhua Liu, and Xinchao Wang. Dataset distillation: A comprehensive review. PAMI, 2023.
[28] Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. Dataset condensation with gradient matching. ICLR, 2021.
[29] George Cazenavette, Tongzhou Wang, Antonio Torralba, Alexci A Efros, and Jun- Zhu. Dataset distillation by matching training trajectories. In Proceedings of the IEEE/CVF Conference Computer Vision and Patiern Recognition (CVPR), pages 4750-4759, 2022.
[30] Ruonan Yo, Songhua Liu, Jingwen Ye, and Xinchao Wang. Teddy: Efticient large-scale dataset distillation apyrooaapero matching. European Conference on Computer Vision, pages 1-17. Springer, 2025.
[31] Bo Zhao and Hakan Bilen. Datasct condensation with distribution matching, In WACV, pages 6514-6523, 2023.
[32] Kai Wang, Bo Zhao, Xiangyu Peng. Zheng Zhu, Shuo Yang. Shuo Wang. Guan Huang. Hakan Bilen, Xinchao Wang, and Yang You. Cafe: Learning to condense dataset by aligning features. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 2022.
[33] Timothy Nguyen, Zhourong Chen, and Jachoon Lee, Datasct meta-lcarning from kemee ridge-regression. ICLR, 2021.
[34] Timothy Nguyen, Roman Novak, Lechao Xiao, and Jaehoon Lee. Dataset distillation with infinitely wide convolutional networks. Advances in Neural Information Processing Systems, 34:5186-51999 2021.
[35] Yuanhao Xiong, Ruochen Wang, Minhao Cheng, Felix Yu, and Cho-Jui Hsich. Feddm: Iterative distribution matching for coamuuncaaiion-fffiiie federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 16323-16332, 2023.
[36] Rui Song, Dai Liu, Dave Zhenyu Chen, Andreas Festag, Carsten Trinitis, Martin Schulz, and Alois Knoll. Federated learning via decentralized dataset distillation in resource-constrained edge environments. In IJCNN, pages 1-10. IEEE, 2023.
[37] Renjie Pi, Weizhong Zhang, Yueqi Xie, Jiahui Gao, Xiaoyu Wang, Sunghun Kim, Qifeng Chen. Dynafed: Tackling client data heterogeneity with global dynamics. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12177-12186. 2023.
[38] Ping Liu, Xin Yu, and Joey Tianyi Zhou. Meta knowledge condensation for federated learning. ICLR, 2023.
[39] Jie Zhang. Xiaohua Qi, and Bo Zhao. Federated generative learning with foundation models, arXiv preprint arXiv:2306. 100064 2023,
12

---

[40] Yanlin Zhou, George Pu. Xiyao Ma, Xiaolin Li, and Dapeng Wu. Distilled one-shot federated learning. arxiv preprint arxiv:2009.0799) 079999 2020.
[41] Max Welling. Herding dynamical weights to learn. In International Conference on Machine Learning. pages 1121-1128, 2009.
[42] Peng Sun, Bei Shi, Daiwei Yu, and Tao Lin. on the diversity and realism of distilled dataset: An efficient dataset distillation paradigm. arxiv preprint arxiv:2312.0353 2023.
[43] Hanlin Lu, Changchang Liu,, Ting He, Shiqiang Wang, and Kevin S Chan. Sharing models or coresets: A study based on membership inference attack. arxiv preprint arxiv:2007 02977, 2020.
[44] Yue Tan, Guodong Long, Jie Ma, Lu Liu, Tianyi Zhou, and Jing Jiang. Federated learning from pre- trained models: A contrastive learning approach. Advances in Neural Information Proessing Systems, 15:19332-19388 2022.
[45] Qinwei Xu, Ruipeng Zhang. Ya Zhang, Yanfeng Wang. and Qi Tian. A fourier- framework for domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages -43922, 2021.
[46] Leon N Piotrowski and Fergus W Campbell. A demonstration of the visual importance and flexibility of spatial-frequency amplitude and phase. Perception, 11(3):337-346, 1982.
[47] Henri Nussbaumer and Henri J Nussbaumer. the Faxt Fourier Transform. Springer, 1982.
[48] Hongxu Yin, Pavlo Molchanov, Jose M Alvarez, Zhizhong Li, Arun Mallya, Derek Hoiem, Niraj Jha, and Jan Kautz. Dreaming to distill: Data-free knowledge transfer via deepinversion. In Proceedings of the IEFF/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 87 871558724, 2020.
[49] Zeyuan Yin, Eric Xing, and Zhiqiang Shen. Squeeze, recover and relabel: Dataset condensation at imagenet scale from a new perspective. Advances in Neural Information Processing Systems, 36, 2024.
[50] Jonas Geiping, Hartmut Bauermeister, Hannah DrOGe, and Michael Moeller. Inverting gradients-how casy is to break privacy in federated lcarning? Advances in Neural Information Processing Systems, 33: 33:16937-1694) 2020.
[51] Jiawei Shao, Zijian Li, Wenqiang Sun, Tailin Zhou, Yuchang Sun, Lumin Liu, Zehong Lin, and Jun Zhang. A survey of what to share in federated learning: Perspectives on model utility, privacy leakage, and communication efficiency. arxiv preprint arxiv:2307.1063: 2023.
[52] Tian Dong, Bo Zhao, and Lingjuan Lyu. Privacy for free: How does dataset condensation help privacy? In International Conference on Machine Learning, pages 5378-5396. PMLR, 2022.
[53] Tehrim Yoon, Sumin Shin, Sung Ju Hwang, and Eunho Yang. Fedmix: Approximation mixup under mean augmented federated learning. arxiv preprint arxiv:2107.00233 2021.
[54] Le and Xuan Yang. Tiny imagenet visual recognition challenge. Cs 231n, 7(7):3. 2015.
[55] Imagenette.
[56] Google Open Images Dataset.
html
[57] Deng. Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei. Imagenet: large-scale hierarchical image database. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 248 255. IEEE, 2009.
[58] Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. Measuring the effects of non-identical data distribu- tion for federated visual classification. arxiv preprint
2019.
[59] Fan Lai, Yinwei Dai. Sanjay Singapuram, Jiachen Liu, Xiangfeng Zhu, Harsha Madhyastha, and Mosharaf Chowdhury. Fedscale: Benchmarking model and system performance of federated learning at scale. In International Conference on Machine Learning, pages 11144 11827, PMLR, 2022.
[60] Hanting Chen, Yunhe Wang, Chang Xu, Zhaohui Yang, Chuanjian I Liu, Boxin Shi, Chunjing Xu, Chao Xu, and Qi Tian. Data-free learning of student networks. In Proceedings of the IEEF/CVF International Conference on Computer Vision (ICCV), pages 3514-3522, 2019.
[61] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 770-778, 2016.
13

---

[62] Robin Rombach, Andreas Blattmann. Dominik Lorenz. Patrick Esser, and BjORn Ommer. High-resolution image synthesis with latent difffusion models. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition (CVPR), pages 10684-10695, 2022.
[63] Jianging Zhang. Yang Liu, Yang Hua, and Jian Cao. An upload-efficient scheme for transferring knowledge from a server-side pre-trained generator to clients in heterogeneous federated learning. arxiv preprint httv:240031576666 2024.
[64] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, and Others. Laion-5b: An open large- scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 35:25278 25294, 2022.
[65] Rui Yan, Liangqiong Qu, Qingyue Wei, Shih- Shih-Cheng Huang, Liyue Shen, Daniel Rubin, Lei Xing, and Yuyin Zhou. Label-effiiient superviee federated learning for tackling data heterogencity in medical imaging. IEEE Transactions on Medical Imaging, 2023.
[66] Manasi Vartak, Harihar Subramanyam, Wei-En Lee, Srinidhi Viswanathan, Saadiyah Husnoo, Samuel Madden, and Matei Zaharia. Modeldb: A system for machine learning model management. Proceedings of the Workshop on teetan-ii-thh-aaee Data Analytics, pages 1-3, 2016.
[67] Alex Krizhevsky, Geoffrey Hinton, Others. Learning multiple layers of features from tiny images. 2009.
[68] Michacl Acrni, Jie Zhang, and Florian TramER. Evaluations of machine learning privacy defenses are mislcading. arxiv preprint arxiv:2404. 17399, 2024.
[69] Martin Abadi, Andy Chu, lan Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pages 308-318, 2016.
14

---

A V-information based Core-Set Selection Description
Algorithm S1 y-information-basee based Core-Set Selection
Require: Client local dataset (X,, Yt) Client local feature extractor h, image per class ipc
X4- t- 0
2: Level I: identifying the most informative image segments
for each 2 (Xe, Yi) do
4: {x²)
K [ k=1 f extract K patches with multiple scales from r.
K do 6: Calculate V-information score se <- -L(h(x*),
for
end for
8: select x' with highest score s'
X U((2l'''.. s'}
10: end for
Level 2: select the top-èpe patches with highest V-information
12: X,, Y, +
for each class c E Yú do
14:
if size of {<zy, Vy s3)}. y ipe then
select top-ipc [(r5, My ) via 8/
16:
X. Y. U..n)) U(un
end end for return Y.)
18:
B More Experimental Details
We use the SGD optimizer with momentum=0.9, learning rate=0.01 and weight decay=0.0001 for clients' local training. The batch size is set to 128 and local epoch is 200. For all generation based methods, we set the resolution of the generated images to 64 x 64, 128 x 128 and 256 x 256 for Tiny-ImageNer ImageNette and Openlmage, the number of generated images in each batch 128, and the learning rate of the generator is 0.001, the latent dimension 256, iteration for training generator is 30, using Adam for optimization. The server model is optimized with SGD with momentum 0.9, the learning rate is 0.01, and the training epochs are 200. The synthesized batch size and server model training batch size is both 128. In DENSE, set X 1 for BN loss and A2 = 0.5 for diversity loss. In Co- Boosting, the perturbation strength is set to 8/255 and the step size μ 0.1/n. In Core-Set selection stage of FedSD2C, for each image T;, employ the torchvision. transfora. htw.amaannnmmeaiaeeeeeeeeamm oeero K times to generate a collection of patches. For patch size, set the scale=(0.08, 1.0), which is collect diverse image patches. Following [42), employ ConvNet-A for Tiny-ImageNet, ConvNet-5 for ImageNette and ConvNet-6 for OpenImage. All methods are implemented with Pytorch and conducted GeForce RTX 3090.
Details of t-SNE plots in Figure 1 We randomly select a client and five classes from its local dataset (Tiny- ImageNet) and employs its local model (ResNet- 18) to extract features. The feature is extracted from the final layer (before the classifier). We then use t-SNE plots to illustrate the feature distribution.
C Additional Experiments
C.1 Additional datasets
CIFAR-10. We set the resolution of generated images to 32x32 for CIFAR-10 67 and keep all other settings the same. As shown in Table ST there an initial performance discrepancy at the standard setting of ipc 50. This occurs because our method prioritizes efficiency with large datasets rather than low-resolution ones. However, upon increasing the amount of synthetic data (ipc 500), method can still achieve comparable results.
15

---

Table S1: Performance on CIFAR-10 with ResNet-18.
Method
0.1
0.3
0.5
Comm.
DENSE
47.75
54.53
66.05
44MB
Co-Boosting
53.33
61.75
68.99
44MB
FedSD2C (ipc 50)
44.72
48.23
51.14
0.03MB
FedSD2C (ipc 500)
56.95
62.37
65.06
0.24MB
COVID-FL. We crop the images of COVID-FL [65]] into 128x 128, set the T = 1000 as in Sec tion 4.3.3 and keep all the other settings the same. The results in Table Siidiiaate that our FedSD2C still acheive better results compared to DENSE and Co-Boosting.
Table S2: Performance COVID-FL datasets with ResNet-18.
Method
a 0.1
= 0.3
a 0.5
DENSE
46.38
57.55
62.83
Co-Boosting
45.07
60.27
65.81
FedSD2C
52.65
62.50
66.68
C.2 Membership Inference Attack
To further validate the effectiveness of our method, we employ an improved version of LiRA 68 to conduct Membership Inference Attacks we use the distillates uploaded by the client to train a new model and conduct membership inference attacks on that model. For the sharing model-based methods, we perform membership inference attacks on the models uploaded by the clients. The client model is ResNet-18 with a = 0.1, ipc = 50. We set the images of Core-Set as the canary (target data 2),, this is the most serious case of our methods. The results confirm that our approach does not introduce more privacy risk than the sharing model-based approach, even for the most vulnerable targets.
methods. When attacking each client, for FedSD2C,
Table S3: Membership Inference Attack.
Method
TPR @ FPR=0.1%
Sharing model-based methods
22.81
(DENSE, Co-Boosting)
FedSD2C
20.13
C.3 Wavelet transform perturbation
We explore the use of wavelet transforms to replace the Fourier transforms during Fourier transform perturbations. We use ResNet- 18 Tiny-ImageNet with 0.1, ipc 50. The results indicate that Wavelet Transform offers greater scalability in privacy protection. By increasing X, the PSNR/SSIM be reduced to as low as 12.90/15.30 5..0. When the accuracy is comparable to that of Fourier transform (Wavelet = 0.5 vs. Fourier 0.8), the PSNR and SSIM of Wavelet transform is lower.
Table S4: Comparison between Wavelet transform and Fourier transform.
Acc.
PSNR
SSIM
Wavelet(A 0.1)
28.05
18.86
44.76
Fourier(A = 0.1)
28.22
20.54
51.50
Wavelet(A 0.5)
26.91
15.22
27.34
Fourier(A -0.5)
28.09
18.06
43.26
Wavelet( À -0.8)
26.06
12.90
15.30
Fouricr(A 0.8)
26.83
16.95
35.89
C.4 Ablation study on Core-Set selection
In this section, we perform ablation experiments to explore the significance of Y-information Core-Set
selection. We use ResNet-18 18 with 0.1, ipc 50. Compared with V-information Core-Set
16

---

selection, the accuracy of random selection decreased by 3.5 and 5,46 on Finy-mmaeNNet and ImageNette, respectively. We also report the performance of uploading Core-Set directly, which achieve the best performance. However, without distillate synthesis, it will increase the cost of communication and the risk of privacy leakage.
Table S5: Performance of different selection strategy. Core-Set denotes that clients directly upload their local Core-Set, which leads to privacy issue. FedSD2C w/ random selection denotes replacing D-information-base Core-Set selection with random selection.
Tiny-ImageNet
ImageNette
Core-Set FedSD2C w/ random selection
31.01
60.54
23.32
42.06
FedSD2C
26.83
47.52
C.5 Integrating with Differential Privacy
According 33]. introducing DP-SGD [69] during the distillate synthesis stage can provide the- oretical privacy guarantees for our method. We perform experiments of integrating DP-SGD in our method on ImageNt with ResNet-18 (a 0.1, ipc 50) to provide clear view of the trade-offs involved.
Table S6: Performance of integrating DP-SGD
8
00
FedSD2C 22.92
25.13
26.01
26.83
D
Visualization
Figure S1: Visualization of synthetic distillate reconstructed by the pre-trained Autoencoder compared to the original sample on Tiny-ImageNee. The image style similar, but with enhanced privacy protection.
17

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The contributions are empirically validated in the main body.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations in Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
18

---

Justification:
the paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibiliyy
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide a detailed description of FedSD2C and the parameters for imple- menting the baseline in Section 4 and the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
19

---

Answer: [Yes]
Justification: Our code is attached to the supplementary material and is released at: https //github. com/ Carkham/raaaaaae FedSD2C
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurlPS code and data submission guidelines (https
wtt.ggiddde/eeSSuuunnrnn ssicnpolicy) for more details. httpolliicggiddeeCoCoooo
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide a detailed description of FedSD2C and the parameters for imple- menting the baseline in Section 4 and the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification:
We report the standard deviation in Table
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
20

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We describe the computational resources used in the Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
We do not violate Code of Ethics
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss the potential impacts about privacy issues.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
21

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We cite all sources used and comply with their licenses.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
22

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not release assets.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
We do not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
We do not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
23


Reviews:
[{"review_id": "enztZz8DrS", "reviewer": "Reviewer_nVDn", "summary": "This paper proposes a one-shot federated learning approach designed to enhance privacy protection, communication efficiency, and model performance. Firstly, the authors introduce a Core-Set selection method based on V-information to extract the most informative data from the original dataset. The amplitude spectrum of the images in the Core-Set is then perturbed using a Fourier transform, and these perturbed images are input into an Autoencoder to obtain their representations. Finally, these representations are transmitted to the server, which decodes the image information from the representations for training.", "questions": "1. The paper utilizes several large datasets, but is the method effective for low-resolution datasets such as CIFAR-10?\n\n2. The authors should provide the model performance using only the Core-Set, where clients send all their original Core-Set data to the server for centralized training.\n\n3. Please provide additional visual metrics beyond PSNR to make the results more convincing, such as SSIM.\n\n4. The authors should conduct ablation experiments to demonstrate the performance of each part of the proposed method. For example, evaluating the model performance without the Core-Set data selection step.\n\n5. Since only representations are ultimately transmitted, and attackers cannot recover images from representations, I wonder if the image perturbation stage is still necessary?", "limitations": "Yes, have discussed the limitations", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The one-shot learning method proposed in this paper enhances privacy protection, transmission efficiency, and model performance compared to previous methods. The approach appears innovative, employing theoretically grounded techniques such as the Core-Set selection method, and using an Autoencoder and decoder for data transmission, which improves both transmission efficiency and model performance.", "weaknesses": "1. In the Core-Set Selection stage, the authors do not explicitly define the patches used in Level 1: identifying the most informative image segments. I have the following questions: Are the patches the same size across different datasets? How many patches are extracted from each image? Does the size or number of patches affect the results? If so, how can one determine the optimal patch size?\n\n2. In the perturbation stage, the authors do not provide sufficient reasoning for using the Fourier transform. Why must the Fourier transform be used instead of other transformations? For example, does using a wavelet transform and merging its low-frequency components achieve similar effects? Additionally, the authors mention that the merged image can be random noise. Is this too idealistic? Does using particularly weak random noise also achieve similar effects?\n\n3. During the transmission phase using an Autoencoder, does the performance of the model depend on the pre-training of the Autoencoder? If a pre-trained model is not used, does this method still work effectively?\n\n4. It would be beneficial for the authors to create a framework diagram of the entire method to enhance readability.\n\n\n5. The paper contains many capitalization and punctuation errors, such as in line 50 and line 229. Additionally, please carefully review the citations in the article as they are quite disorganized, including issues with formatting, capitalization, and more. Please ensure consistency throughout.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "***Q1 Patch size, number of patch and influence of number of patch***\n\n**A1** We apologize for the unclear statement. For each image $x_i$, we employ the `torchvision.transform.RandomResizeCrop` $K$ times to generate a collection of patches. For patch size, we set the `scale=(0.08, 1.0)`, which is to collect diverse image patches, so we don't set a fixed size. For the number of patches, we perform experiments to determine the best $K$ empirically. The table below indicates that performance improves with an increased $K$, stabilizing when $K$ reaches 10. Consequently, we have empirically set $K=10$ based on these observations.\n\n| $K$          | 1     | 3    | 5     | 10    | 20    | 30    |\n| ------------ | ----- | ---- | ----- | ----- | ----- | ----- |\n| TinyImagenet | 16.58 | 19.8 | 21.66 | **23.34** | 23.38 | 23.29 |\n| ImageNette   | 52.89 | 54.8 | 55.06 | **55.13** | 54.06 | 53.45 |\n\n***Q2 Motivation for Fourier transfomation. Merging random noise***\n\nWe would like to thank Reviewer nVDn for the suggestion. The employment of the Fourier transform is for its ability to balance privacy preservation with information retention. Indead, wavelet transform is also suitable for our proposed methods. But our decision was driven by its simplicity. We appreciate the reviewer's insight and will consider exploring the wavelet transform and its potential benefits in future research. \n\nFor image perturbations, we focus on preserving the frequency components of the Core-Set sample. The information in the amplitude component can be reconstructed in the latent space of the Autoencoder based on the image priors during synthesis. Since the noise does not follow the image a priori, merging with it will have a performance impact, but it can also lead to greater privacy protection. In response, we conducted an experiment to replace the merged images with Gaussian noise (ResNet18, $\\alpha=0.1,ipc=50$, Tiny-ImageNet). The experimental results show that noise merging can be used as a supplement to the need for stronger privacy protection.\n\n| Method | Acc.  | PSNR  |\n| ---------------------- | ----- | ----- |\n| FedSD2C (Merging with Gaussian noise) | 22.21 | 12.91 |\n| FedSD2C                | 26.83 | 16.95 |\n\n**Q3 Effectiveness of Autoencoder, only Core-Set and performance of Core-Set selection step**\n\n1) The introduction of pre-trained Autoencoders is crucial as it provides generalized image priors. This image prior helps synthetic distillate to prevent overfitting to localized data patterns, thus reducing the negative effects of data heterogeneity. The lower resolution of the latent space representations can significantly boost communication efficiency. Our experimental results presented in Table 3 of our paper, confirm the benefits of employing a pre-trained Autoencoder. Moreover, we conduct experiments **w/o Autoencoder** under the same setting (ResNet18, $\\alpha=0.1,ipc=50$). The table clearly illustrates the effectiveness of our method even when not using a pre-trained model. However, the performance gain is substantial, highlighting the advantage of pre-training in our method.\n2) Thanks for your feedback. We now add the results of only using Core-Set (ResNet18, $\\alpha=0.1,ipc=50$) and perform an ablation study by replacing Core-Set selection with random selection. As depicted in the table, better performance can be achieved by simply transferring Core-Set, but this comes at the cost of privacy compromise. Random selection struggles to capture the necessary data diversity and representativeness, resulting in the poorest performance.\n\n| Method | Tiny-ImageNet | ImageNette |\n| --------------------------------------------- | --------- | ---------- |\n| Core-Set                                      | 31.01     | 60.54      |\n| FedSD2C w/o AE                                | 24.35     | 46.43      |\n| FedSD2C w/o Core-Set (Random selection) | 23.32     | 42.06      |\n| FedSD2C                                       | 26.83     | 47.52      |\n\n***Q4 Experiments on CIFAR-10***\n\nThanks for the pertinent comments. Our approach focuses on efficiency on large data rather than low-resolution datasets. Core-Set selection stage only requires inference, and distillate synthesis effectively reduces the number of parameters by optimizing in the latent space. This efficiency-tailored approach results in the synthesis of a more compact data compared to the DFKD-based methods. We have conducted a thorough evaluation of our approach on the CIFAR-10 dataset. As depicted in Table 2 (PDF), there is an initial performance discrepancy at the standard setting of $ipc=50$. However, upon increasing the amount of synthetic data ($ipc=500$), our method achieves results comparable results.\n\n***Q5  Lack of SSIM.***\n\nAs suggestions, we include the SSIM in our privacy evaluation. The results indicate that our method achieves the best tradeoff between performance and privacy protection.\n\n| Privacy-preserving | Acc.  | PSNR  | SSIM  |\n| ------------------------- | ----- | ----- | ----- |\n| ours ($\\lambda=0.8$)      | 20.85 | 16.95 | 35.89 |\n| $Gaussian (s=0.2, p=0.2)$ | 19.32 | 23.52 | 68.56 |\n| $Gaussian (s=0.2, p=0.1)$ | 21.48 | 27.51 | 78.90 |\n| FedMix                    | 13.86 | 16.26 | 56.91 |\n\n***Q6 The necessity of image perturbations***\n\nAlthough it is not possible for attackers to directly reconstruct images from their representations, there remains a risk of a semi-honest server attempting to infer private information from the data. By introducing image perturbation, we make it significantly more challenging for any adversary to deduce private information from synthetic distillate. Empirical results on the Model Inversion attack and Membership Inference attack (Table 2 of PDF file) demonstrate that our method achieves a superior balance between privacy protection and utility.\n\n***Q7 Punctuation errors and framework diagram***\n\nWe would like thank the Reviewer nVDn for suggestions. We will revise them in final version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mj8CnzWUID", "reviewer": "Reviewer_EquD", "summary": "The paper presents FedSD2C, a novel one-shot federated learning (FL) framework that aims to improve communication efficiency, privacy preservation, and model performance. The approach addresses issues with data heterogeneity and information loss by synthesizing informative distillates from local data and sharing these instead of local models. Empirical results show that FedSD2C significantly outperforms existing one-shot FL methods.", "questions": "1. What are the key hyper-parameters used in the core-set selection algorithm?\n2. According to Algorithm 2, the core-set includes patches with different scales. How does the encoder handle patches with different scales?\n3. There is still a huge performance gap between FedSD2C and Central in Table 1. What are the main factors that result in the performance drop?", "limitations": "The authors have pointed out the computational overhead on local devices as the limitation.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. One-shot FL is a potential direction that can significantly minimize communication costs in FL.\n2. This approach does not rely on sharing private data, which protects the data privacy.\n3. Using V-information and fourier transform perturbation is interesting.\n4. Experimental results show significant improvements.", "weaknesses": "1. A critical question on the optimal observer of approximating the V-information is that, the local model is trained locally, which may not indicate good V-information of the global datasets, i.e., all local dataets.\n2. The privacy is not strictly guaranteed. The perturbation with 3, 4, 5 cannot ensure data privacy.\n3. Figure 2 shows the reconstruction is very similar with the original images.\n4. There is no theoretical analysis on privacy and generalization of the proposed method.\n5. The pre-trained autoencoder plays a key role in the framework. But the experimental study doesn't investigate the impact of the pre-trained autoencoder on the performance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "***Q1 A critical question on the optimal observer of approximating the V-information is that, the local model is trained locally, which may not indicate good V-information of the global datasets, i.e., all local datasets.***\n\nThanks for the detailed comments. In the context of one-shot federated learning, where global datasets are inaccessible, our method aims to distill optimal local datasets. It's essential to recognize that while local models are indeed tailored to their respective local data domains, this specialization is not a limitation but rather an advantage. This localized understanding positions the local model as the most effective observer for the Core-Set selection.\n\n***Q2 The privacy is not strictly guaranteed. The perturbation with 3, 4, 5 cannot ensure data privacy. Figure 2 shows the reconstruction is very similar with the original images. There is no theoretical analysis on privacy and generalization of the proposed method.***\n\nThanks for the detailed comments. Our paper emphasizes the empirical contributions of using the Fourier transform to enhance the privacy of synthetic data. In this regard, our paper performed Model Inversion Attacks to validate that our approach provides the best trade-off between privacy and performance.\nThe image perturbations applied are designed to protect individual private information while still allowing for the synthesis of key image patterns. Therefore, similar image styles do not mean that private information has been compromised. \nTo further validate the effectiveness of our method, we employ an improved version of LiRA[1] to conduct Membership Inference Attacks on our methods. We set the raw images of Core-Set as the canary (target data $x$), as this is the most serious case of our methods. The results confirm that our approach does not introduce more privacy risk than the sharing model-based approach, even for the most vulnerable targets. Furthermore, according to Theorem 3.2 of [2], introducing DP-SGD during the distillate synthesis stage can provide theoretical privacy guarantees for our method.\n\n| Method | TPR@FPR=0.1 |\n| ---------------------------------- | ----------- |\n| Sharing Model (DENSE, Co-Boosting) | 22.81       |\n| FedSD2C                            | **20.13**      |\n\n[1] Aerni, M, et al. Evaluations of Machine Learning Privacy Defenses are Misleading. 2024.\n\n[2] Xiong Y, et al. FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learning. 2023.\n\n***Q3 The pre-trained autoencoder plays a key role in the framework. But the experimental study doesn't investigate the impact of the pre-trained autoencoder on the performance.***\n\nThanks for the detailed comments. In fact, we have explored the impact of pre-trained Autoencoders on communication efficiency in Table 3 of our paper. The results show that the introduction of pre-trained Autoencoders reduces the communication cost and achieves better results. To further validate its effectiveness, we perform experiments w/ and w/o Autoencoder during distillate synthesis ($ipc=50,\\alpha=0.1$). The table clearly illustrates the performance gain, highlighting the advantage of pre-trained Autoencoders in our method.\n\n| Method | TinyImage | ImageNette |\n| -------------- | --------- | ---------- |\n| FedSD2C w/o AE | 24.35     | 46.43      |\n| FedSD2C        | 26.83     | 47.52      |\n\nIn addition, we conducted experiments on the medical dataset COVID-FL [1] to verify that the employment of Autoencoder can be extended to more different data domains. The results are as follows:\n\n| Method | 0.1   | 0.3   | 0.5   |\n| ------- | ----- | ----- | ----- |\n| DENSE   | 46.15 | 57.55 | 62.83 |\n| CoBoost | 45.07 | 60.27 | 65.61 |\n| FedSD2C | **52.65** | **62.50** | **66.68** |\n\n[1] Yan R, et al. Label-Efficient Self-Supervised Federated Learning for Tackling Data Heterogeneity in Medical Imaging. 2023.\n\n***Q4 What are the key hyper-parameters used in the core-set selection algorithm? According to Algorithm 2, the core-set includes patches with different scales. How does the encoder handle patches with different scales?***\n\nWe apologize for the unclear statement. For each image $x_i$, we employ the `torchvision.transform.RandomResizeCrop` $K$ times to generate a collection of patches. Each patch with different scales will be resized to the resolution of its original image. We will revise this in the final version.\n\n***Q5 There is still a huge performance gap between FedSD2C and Central in Table 1. What are the main factors that result in the performance drop?***\n\nThanks for the question. This gap can be attributed to the inherent challenges of one-shot federated learning. For example, the data heterogeneity across different clients can lead to the generation of noisy soft labels, which can impede the server model from extracting accurate knowledge from the synthetic data. \nThere is a natural trade-off between communication efficiency, which is crucial in federated learning to minimize communication overhead, and the performance of the server model, which requires sufficient information to learn effectively. Despite these challenges, it is important to recognize our method's strength. Our approach demonstrates the lowest performance gap and best communication efficiency among comparable methods, thereby offering optimal practical utility for one-shot federated learning environments. We believe that the trade-offs our approach entails are justified and favorable, given the current state of other methods and the inherent constraints of one-shot federated learning.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rJApbgItSb", "reviewer": "Reviewer_3gL8", "summary": "This paper proposes a one-shot FL method (FedSD2C), utilizing V-information to select local core set data and server-pretrained autoencoder and Fourier-domain perturbation to ensure privacy preservation for local \"distillate\" sharing. In comparison to existing works such as DENSE and Co-Boosting, FedSD2C can reduce information loss in one-shot FL and improve up to 2.7x global model performance.", "questions": "- Can you more rigorously demonstrate the privacy-preservation statement? For example, differential privacy is a widely accepted concept. Could you please show the provable privacy-preservation guarantee like DP?\n\n- It is unclear how Figure 1 is generated. The authors should detail the datasets, model, and necessary information. Otherwise, comparing the figures is meaningless.\n\n- What is the computational cost for the core-set selection step? \n\n- It is unclear which parts of the algorithm contribute to handling the heterogeneous data. It seems that coreset selection could contribute well if intentionally sampling (almost) balanced data.\n\n- What is the number of clients for Table 1?", "limitations": "Limitation was discussed", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper tackles an important setting in FL and the one-shot performance outperforms listed related works.", "weaknesses": "- The assumption that the server holds an autoencoder is pretty strong, as the autoencoder must be trained in the clients' data domain to ensure it works. \n\n- The paper only showed that Fourier-based perturbation can 'visually' protect privacy by using PSNR as a metric. Although the paper mentioned MIA, it did not evaluate existing privacy attacks. \n\n- The clarity of paper writing can be improved. Essential details are missing to understand the contributions. See more in the Questions section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "***Q1 The assumption that the server holds an autoencoder is pretty strong, as the autoencoder must be trained in the clients' data domain to ensure it works.***\n\nThanks for the detailed comments. The large-scale datasets used to pre-train the autoencoder contain diverse data, which is enough to cover the data domains of most clients. For untrained data domains, such as medical dataset COVID-FL [1], we conducted experiments to verify that Autoencoder can be extended to more different data domains. Moreover, Thanks to the growing open source community, autoencoders pre-trained for different data domains are also readily available. All of these illustrate the practices of employing a pre-trained autoencoder. The experimental results on medical datasets are as follows:\n\n| Method | 0.1   | 0.3   | 0.5   |\n| ------- | ----- | ----- | ----- |\n| DENSE   | 46.15 | 57.55 | 62.83 |\n| CoBoost | 45.07 | 60.27 | 65.61 |\n| FedSD2C | **52.65** | **62.50** | **66.68** |\n\n[1] Yan R, et al. Label-Efficient Self-Supervised Federated Learning for Tackling Data Heterogeneity in Medical Imaging. 2023.\n\n***Q2 The paper only showed that Fourier-based perturbation can 'visually' protect privacy by using PSNR as a metric. Although the paper mentioned MIA, it did not evaluate existing privacy attacks. Can you more rigorously demonstrate the privacy-preservation statement? For example, differential privacy is a widely accepted concept. Could you please show the provable privacy-preservation guarantee like DP?***\n\nThanks for the comments. Our paper emphasizes the empirical contributions of using the Fourier transform to enhance the privacy of synthetic data. In this regard, our paper performed Model Inversion Attacks to validate that our approach provides the best trade-off between privacy and performance.\n\nTo further validate the effectiveness of our method, we employ an improved version of LiRA[1] to conduct Membership Inference Attacks on our methods. We set the raw images of Core-Set as the canary (target data $x$), as this is the most serious case of our methods. The results confirm that our approach does not introduce more privacy risk than the sharing model-based approach, even for the most vulnerable targets. Furthermore, according to Theorem 3.2 of [2], introducing DP-SGD during the distillate synthesis stage can provide theoretical privacy guarantees for our method.\n\n| Method | TPR@FPR=0.1 |\n| ---------------------------------- | ----------- |\n| Sharing Model (DENSE, Co-Boosting) | 22.81       |\n| FedSD2C                            | **20.13**      |\n\n[1] Aerni, M, et al. Evaluations of Machine Learning Privacy Defenses are Misleading. 2024.\n\n[2] Xiong Y, et al. FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learning. 2023.\n\n\n***Q3 Unclear statement of Figure 1 and What is the number of clients for Table 1?***\n\nWe apologize for the unclear statement. We employ one of the local models (ResNet18) to extract features from its corresponding local dataset, and the synthetic data of DENSE, Co-Boosting and our proposed method. The feature is extracted from the final layer (before the classifier), and we use t-SNE plots to illustrate the feature distribution. This visual comparison aims to demonstrate the effectiveness of our method in capturing the data distribution. For all experiments presented in our paper, the default number of clients involved is 10, unless otherwise specified. We will include these details in our final version.\n\n***Q4 What is the computational cost for the core-set selection step?***\n\nThanks for the valuable question. In the Core-Set selection stage, the client runs $K$ (number of patch) inferences on each image $x_i\\in X_t$ ($X_t$ indicates local dataset), i.e. the computational cost = $\\mathrm{FLOPs}\\times \\|X_t\\| \\times K$\n\n***Q5 It is unclear which parts of the algorithm contribute to handling the heterogeneous data. It seems that coreset selection could contribute well if intentionally sampling (almost) balanced data.***\n\nThanks for the detailed comments. Data heterogeneity leads to inconsistencies in local models (known as client drift), which is a significant challenge for sharing model-based methods. Our method addresses this issue by transmitting synthetic distillate, which avoids the necessity for ensembling inconsistent local models and, consequently, mitigates the effect of client drift. Moreover, a pre-trained autoencoder is introduced to optimize distillate in its latent space. This manner prevents overfitting to patterns that are only recognized by the local model, which further mitigates the impact of data heterogeneity.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aKCf1iJEF9", "reviewer": "Reviewer_Bcom", "summary": "The paper introduces FedSD2C, a new one-shot Federated Learning (FL) framework designed to address challenges in existing methods. Previous approaches have used data-free knowledge distillation to improve one-shot FL, but these methods struggle with data heterogeneity and scalability issues.  FedSD2C aims to solve these issues by:\n1.Introducing a distiller to synthesize informative distillates directly from local data, reducing information loss.\n2.Sharing synthetic distillates instead of inconsistent local models to address data heterogeneity.\nThe authors claim that empirical results show FedSD2C outperforms other one-shot FL methods, especially with more complex and real datasets.", "questions": "see above", "limitations": "see above", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "This paper focuses on one-shot Federated Learning (FL), which is an intriguing topic. \nThe paper presents a comprehensive set of experiments, both on model performance and privacy.", "weaknesses": "Thanks for the authors' efforts in presenting this paper. After carefully reading it, I have the following questions and comments:\n\n1. On line 52, the values 4.21 and 2.06 are not clearly explained. I suggest adding a figure or table to provide more specific details about these numbers.\n2. While I can imagine that transmitting synthetic data could improve the global model's performance more easily than transmitting the model itself, this inevitably leads to privacy concerns. Although the authors provide some data reconstruction experiments, the results are not entirely convincing:\n    a) To my knowledge, dataset distillation/coreset selection does not inherently protect privacy [1]. I suggest the authors include additional experiments on Membership Inference Attacks (MIA) (based on results from data-free KD [2], I suspect coresets provide even less privacy protection, since data-free KD doesn't use any 'real' training data ).\n    b) According to the privacy onion concept [3], memorization is relative, then in your method the selected images may be at higher risk of privacy leakage compared to those not selected.\n\n3. Compared to other methods, the proposed method communication overhead seems larger. It requires to transmit both the auto-encoder and a large amount of synthetic data.\n\n4. The method relies heavily on Stable Diffusion's auto-encoder, which seems to utilize very strong prior information. How well would your method perform on datasets where diffusion models are not as proficient, such as medical datasets?\n\n5. To my knowledge, another one-shot FL work[4] that relies on Stable Diffusion achieved results of about 75.0 on ImageNette, which appears significantly better than your 55.90. \n\n[1] No Free Lunch in \"Privacy for Free: How does Dataset Condensation Help Privacy\"\n\n[2] Evaluations of Machine Learning Privacy Defenses are Misleading\n\n[3] The Privacy Onion Effect: Memorization is Relative\n\n[4] Federated Generative Learning with Foundation Models", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "***Q1 On line 52, the values 4.21 and 2.06 are not clearly explained. I suggest adding a figure or table to provide more specific details about these numbers.***\n\nWe would like to thank the Reviewer Bcom for suggestions. We will include a detailed figure to better illustrate this in the final version.\n\n***Q2 While I can imagine that transmitting synthetic data could improve the global model's performance more easily than transmitting the model itself, this inevitably leads to privacy concerns. Although the authors provide some data reconstruction experiments, the results are not entirely convincing: a) To my knowledge, dataset distillation/coreset selection does not inherently protect privacy [1]. I suggest the authors include additional experiments on Membership Inference Attacks (MIA) (based on results from data-free KD [2], I suspect coresets provide even less privacy protection, since data-free KD doesn't use any 'real' training data ). b) According to the privacy onion concept [3], memorization is relative, then in your method the selected images may be at higher risk of privacy leakage compared to those not selected.***\n\nThanks for the insightful comments. As suggestion, we perform Membership Inference Attacks on our methods. We employ an improved version of LiRA[1]and set the raw images of Core-Set as the canary (target data $x$), as this is the most serious cases of our methods. Please note that we consider a semi-honest server, so the victim model for us is a model trained on synthetic data, and for the sharing model method is the client-uploaded local model. The results confirm that our approach does not introduce more privacy risk than the sharing model-based approach, even for the most vulnerable targets. Furthermore, according to Theorem 3.2 of [2], introducing DP-SGD during the distillate synthesis stage can provide theoretical privacy guarantees for our method.\n\n| Method | TPR@FPR=0.1 |\n| ---------------------------------- | ----------- |\n| Sharing Model (DENSE, Co-Boosting) | 22.81       |\n| FedSD2C                            | **20.13**      |\n\n[1] Aerni, M, et al. Evaluations of Machine Learning Privacy Defenses are Misleading. 2024.\n\n[2] Xiong Y, et al. FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learning. 2023.\n\n***Q3 Compared to other methods, the proposed method communication overhead seems larger. It requires to transmit both the auto-encoder and a large amount of synthetic data.***\n\nThanks for the comments. In fact, our method **does not** require server-to-client transmission. The pre-trained Autoencoder is available from public repositories. Instead, in our framework, only the synthetic distillate is transmitted from the client to the server, with no other data or model being transferred. As illustrated in Table 3 of our paper, our method significantly reduces the communication cost to a mere 0.5 MB, in contrast to the 44 MB required by sharing model-based methods. This comparison underscores the superiority of our approach in communication efficiency.\n\n***Q4 The method relies heavily on Stable Diffusion's auto-encoder, which seems to utilize very strong prior information. How well would your method perform on datasets where diffusion models are not as proficient, such as medical datasets?***\n\nThanks for the detailed comments. The large-scale datasets used to pre-train the Autoencoder contain diverse data, which is enough to cover the data domains of most clients. For untrained data domains, such as medical dataset COVID-FL [1], we conducted experiments to verify that Autoencoder can be extended to more different data domains. All of these illustrate the practices of employing a pre-trained autoencoder. The experimental results on medical datasets are as follows:\n\n| Method | 0.1   | 0.3   | 0.5   |\n| ------- | ----- | ----- | ----- |\n| DENSE   | 46.15 | 57.55 | 62.83 |\n| CoBoost | 45.07 | 60.27 | 65.61 |\n| FedSD2C | **52.65** | **62.50** | **66.68** |\n\n[1] Yan R, et al. Label-Efficient Self-Supervised Federated Learning for Tackling Data Heterogeneity in Medical Imaging. 2023.\n\n***Q5 To my knowledge, another one-shot FL work[4] that relies on Stable Diffusion achieved results of about 75.0 on ImageNette, which appears significantly better than your 55.90.***\n\nThanks for bringing a related work to our attention. In fact, it is not absolutely fair to compare our approach to theirs. Firstly, the approach in work [1] demands significantly more computational resources and data. Their method requires 400 times the data volume compared to ours ($ipc=20,000$ vs. $ipc=50$). Our method is more cost-effective in terms of data synthesis. It relies solely on an autoencoder, eliminating the need for Stable Diffusion, which is a computationally intensive process. Secondly, the method from work [1] necessitates prior knowledge of the category name and assumes that the images adhere to the prior distribution of Stable Diffusion. This assumption can be restrictive, as it may not be applicable to diverse datasets or specialized domains such as medical imaging, where Stable Diffusion may not be able to synthesize X-rays or similar images based solely on labels.\n\n[1] Zhang J, et al. Federated Generative Learning with Foundation Models. 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear Reviewers and ACs,

We would like to thank the reviewers' insightful reviews and constructive comments on our manuscript. We have carefully considered all the suggestions and made the following changes:

1. We have included an additional datasets. By doing so, we aim to demonstrate our methods can adapt to low-resolution datasets (CIFAR-10) and vairous data domains (medical dataset COVID-FL[1])
2. To further substantiate the privacy-preserving capabilities of our proposed method, we have performed Membership Inference Attack and provided comprehensive explanations to mitigate privacy concerns.

Thank you once again for your valuable feedback.

[1] Yan R, et al. Label-Efficient Self-Supervised Federated Learning for Tackling Data Heterogeneity in Medical Imaging. 2023.

Meta-review: This paper proposes a one-shot FL method (FedSD2C), utilizing V-information to select local core set data and server-pretrained autoencoder and Fourier-domain perturbation to ensure privacy preservation for local "distillate" sharing. In comparison to existing works such as DENSE and Co-Boosting, FedSD2C can reduce information loss in one-shot FL and improve up to 2.7x global model performance.

Weakness:

The biggest concern is the privacy protection. Although authors conduct experiments for MIA, but this is not sufficient to show the privacy protection.

•	The assumption that the server holds an autoencoder is pretty strong, as the autoencoder must be trained in the clients' data domain to ensure it works. 

•	The paper only showed that Fourier-based perturbation can 'visually' protect privacy by using PSNR as a metric. Although the paper mentioned MIA, it did not evaluate existing privacy attacks. Given that communication efficiency and privacy are highlighted as key contributions (contribution 2), revising the privacy-related statements could necessitate substantial changes to the original submission and might reduce overall contributions. 

•	As for the experiment on privacy attacks, since the model's performance is too low (far below 90%), it will also lead to biased MIA results. Therefore, it is not recommended to claim in the paper that your method can protect privacy (without any differential privacy guarantee).
Decision: Accept (poster)