One-shot Federated Learning via Synthetic Distiller-Distillate Communication

Authors: junyuan  zhang
songhua  liu
xinchao  wang
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-10 15:17:59

Abstract: One-shot Federated learning (FL) is a powerful technology facilitating collaborative training of machine learning models in a single round of communication. While its superiority lies in communication efficiency and privacy preservation compared to iterative FL, one-shot FL often compromises model performance. Prior research has primarily focused on employing data-free knowledge distillation to optimize data generators and ensemble models for better aggregating local knowledge into the server model. However, these methods typically struggle with data heterogeneity, where inconsistent local data distributions can cause teachers to provide misleading knowledge. Additionally, they may encounter scalability issues with complex datasets due to inherent two-step information loss: first, during local training (from data to model), and second, when transferring knowledge to the server model (from model to inversed data). In this paper, we propose FedSD2C, a novel and practical one-shot FL framework designed to address these challenges. FedSD2C introduces a distiller to synthesize informative distillates directly from local data to reduce information loss and proposes sharing synthetic distillates instead of inconsistent local models to tackle data heterogeneity. Our empirical results demonstrate that FedSD2C consistently outperforms other one-shot FL methods with more complex and real datasets, achieving up to 2.6 $\times$ the performance of the best baseline. Code: https://github.com/Carkham/FedSD2C

Main paper: 
 introduction: 
 Federated learning (FL) has emerged as a cutting-edge technology that enables training a global model across multiple clients without sharing their raw data [1] . Original FL requires multiple communication rounds for exchanging information between clients and servers. While this paradigm yields a better global model by frequent communication, such high communication costs along with the risk of connection drop errors make it impractical and intolerable in real-world FL applications [2, 3, 4] . Moreover, frequent communication poses security risks such as man-in-the-middle attacks [5] and privacy concerns [6] .To address these issues, one-shot FL [7] has been proposed, requiring only a single communication round, significantly reducing communication costs and concurrently diminishing vulnerability to malicious interception. Due to one communication round property, One-shot FL can also be easily scaled up to large-scale client scenarios, especially for cross-device settings [3] . Despite its sufficient benefits, the limitation of a single communication round makes one-shot FL fall short in accuracy compared to conventional multiple-round FL. (2) t-SNE plots of feature distributions of data generated by DENSE(left ▲), Co-Boosting(middle ■), and our FedSD2C(right ⋆). We randomly select five different classes (indicated by different colors) of real and synthetic data from Tiny-ImageNet. Bad samples are data generated by the DFKD-based method that deviates from the distribution of local real data.To facilitate effective knowledge transfer from client models to the server model within a single round, most previous one-shot FL methods [7, 8, 9, 10] have focused on knowledge distillation.Early approaches [11, 7] use knowledge distillation to transfer knowledge from an ensemble of client models [12] to the server model. While effective, these methods often require additional public datasets, which can be cumbersome or unfeasible in real-world scenarios [13] . Alternatively, data-free knowledge distillation (DFKD) is introduced to avoid the need for public datasets. For example, DENSE [8] employs Generative Adversarial Networks (GANs) [14] as data generators and an ensemble of client models as a discriminator to synthesize diverse data for knowledge transfer to server model in a data-free manner. Co-Boosting [10] extends DENSE by proposing a mutually reinforcing approach to enhance synthetic data and the ensemble model for server training.Nevertheless, such a method of generating data implies two-tier information loss. First, due to model capacity limitations, client models may struggle to encapsulate all information about local data, affecting the quality of the generated data. Second, the generated data do not fully represent the information within the model, as they are produced from random noise without explicit guidance. Therefore, some classes of synthetic data cannot have similar feature distributions to the original real data, as depicted in Figure 1 . Moreover, data heterogeneity [15] in FL can result in inconsistent and misleading predictions from local models [16] , which has been shown to hinder knowledge distillation [17, 18] . Consequently, the server model trained on such noisy and information-lossy generated data typically suffers significant performance degradation, particularly on complex datasets [19] .In this paper, we propose FedSD2C (One-shot Federated Learning via Synthetic Distiller-Distillate Communication), a novel and practical one-shot FL framework that introduces a pre-defined distiller for informative, privacy-enhanced, and communication-efficient distillate communication. In specific, FedSD2C first adopts a V-information [20] based Core-Set selection method to distill the local dataset into an informative Core-Set. By capturing the diversity and realism through V-information, the distilled Core-Set fully encapsulates the information of the local data domain for training a robust server model. However, directly transmitting the Core-Set, which may include the original samples, poses potential privacy risks and incurs significant communication costs, especially for highresolution images. In this regard, FedSD2C employs two techniques to further distill the Core-Set into distillates, thereby enhancing privacy and reducing communication costs: 1) Utilizing Fourier transform perturbation to alter the amplitude components of the Core-Set samples for distillate initialization, enhancing privacy while retaining semantic content; 2) Employing a pre-trained Autoencoder [21] provided by the server as a distiller to distill the perturbed Core-Set into distillates and optimizing its V-information to be as close as possible to original Core-Set, thus minimizing information loss. Finally, clients transmit synthetic distillates to the server instead of inconsistent models for knowledge transfer. Compared to generating noisy knowledge from inconsistent client models with two-tier information loss, end-to-end distillate synthesis minimizes information loss and their aggregation mitigates the impact of data heterogeneity. Through extensive experiments over various real-world datasets , we show that our proposed method significantly surpasses the generated-based one-shot FL methods. The contributions of this paper are: 2 Related Work 
one-shot federated learning: 
 One-shot federated learning was first proposed by [22] , which introduces a method to aggregate a server model by distilling knowledge from an ensemble of client models using public datasets.FedKT [11] propose a hierarchical knowledge transfer framework, enabling various types of classification models. While their approaches demonstrate promising results, the requirement of public datasets which is inaccessible for privacy or transmission reasons limits their practical applications.To address the limitations associated with public datasets, DENSE [8] introduces a DFKD process utilizing an additional data generator trained on the ensemble model. Considering the challenge of high statistical heterogeneity, FedCVAE [9] proposes replacing the local training task with training conditional Variation Autoencoders. Furthermore, Co-Boosting [10] aims to enhance the performance of both the data generator and the ensemble model through a two-tier process. Despite these advancements, generating data through DFKD involves a two-tier information loss. Simultaneously, the inconsistency among client models due to data heterogeneity [15, 23, 24, 25] , further degrades the quality of generated data, introducing label noise and thus limiting the performance of the server model. In this work, we tackle these problems from the perspective of sharing synthetic distillates. By utilizing Core-Set selection and pre-trained Autoencoders as distillers, our proposed methods distill diverse and informative data for server training.Dataset Distillation (DD) was first introduced by [26] , aiming to distill the knowledge of datasets into synthetic data while preserving the performance of the model trained on it. Early dataset distillation methods are formulated as a bi-level optimization problems [27] , where the outer loop optimize the synthetic data via gradient matching [28, 29, 30] , distribution matching [31, 32] and performance matching [26] , while the inner loop progressively trains a model on the synthetic data. Considering computational resources constraints, single-level optimization methods [33, 34] based on kernel ridge regression are proposed to decouple the bi-level optimization, thereby reducing training cost. These methods demonstrate comparable performance in non-complex datasets like CIFAR10 and are implemented in FL to tackle communication bottlenecks [35, 36] , data heterogeneity [37, 38, 39] and one-shot FL [40, 36] . However, these methods require significant computational resources, making them impractical for edge devices with limited capability in FL. Additionally, they may struggle to effectively distill high-resolution datasets.3 MethodologyQuestion: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: We discuss the limitations in Section 5Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
overview: 
 We proposed FedSD2C to alleviate the two-tier information loss inherent in one-shot FL methods based on DFKD and account for data heterogeneity by synthetic distiller-distillate communication. The details of FedSD2C are described in Figure 2 and Algorithm 1. In the preparation phase of one-shot FL, the server distributed a pre-trained Autoencoder [21] as the distiller to each client. Subsequently, clients synthesize informative, privacy-enhanced, and communication-efficient distillates for server-side training. Specifically, to ensure the distillates fully encompass local information, clients first utilize a V-information-based Core-Set selection method to extract diverse and informative Core-Set from their local data domains. Aiming to further reduce the communication costs and enhance the privacy of distillates, clients then perturb the Core-Set with Fourier transform for distillate initialization and employ the received pre-trained Autoencoder to optimize distillates in a compact latent space via V-information alignment with the Core-Set. Finally, clients transmit the distillates, and the server decodes them using the pre-trained Autoencoder for training. We will now delve into the details of each component.V-information [20] was first proposed to measure the mutual information between X and Y constrained on predictive family V which is denoted as:EQUATION) where H V (Y |∅) and H V (Y |X) denote the predictive V-entropy conditioned on ∅ or X.Core-Set selection is a type of dataset distillation method that focuses on preserving a subset of the original training dataset containing only valuable or representative samples. The objective is to enable models trained on this subset to achieve performance similar to those trained on the entire dataset. Typically, this is achieved by minimizing certain criteria, such as data distribution [41] . In our case, the emphasis lies on maximizing the diversity and information content of the subset. Therefore, in the context of V-information, Core-Set selection can be reformulated as:EQUATIONwhere X t and Y t denotes the images and labels in the original datasets and (X s , Y s ) denotes the selected Core-Set. The intuition behind this equation is that Core-Set should include sufficient information and provide a concise representation corresponding to original datasets, constrained by observers V.Inspired by [42] , we maximize the V-information of the Core-Set from two levels. First, we identify the most informative image segments within each image by evaluating patches extracted at various scales from each image. Second, we select the top-ipc with the highest V-information for each class to construct the final Core-Set. The algorithm description can be found at Appendix A. Since a model pre-trained on original datasets can serve as an optimal observer for approximating the V-information [42] , we proposed using local pre-trained models as the observer models (predictive family V) to conduct V-information-based Core-Set selection on local datasets. Given that pre-trained local models are commonly present in one-shot FL [8, 7] , their utilization does not violate the practicality of our approach.In this section, we will describe how to synthesize informative, privacy-enhanced, and communicationefficient distillates using pre-trained Autoencoders. Although Core-Set significantly reduces dataset size, communication costs remain a challenge for high-resolution data in real-world applications. Moreover, while sharing Core-Set provides consistent resistance to membership inference attacks [43] , transmitting patches may still risk exposing sensitive information of original images. Therefore, further enhancing the privacy of shared data is essential. To alleviate these concerns, we propose two novel techniques: 1) distillate initialization with Fourier transform perturbation, which alters the amplitude components of the Core-Set samples to enhance privacy while retaining semantic content; and 2) distillate synthesis with pre-trained Autoencoders which act as the distillers. The pre-trained Autoencoder converts the perturbed samples into distillates by optimizing their V-information to be as close as possible to the original Core-Set samples, minimizing information loss. We will now discuss each component in detail below. The process is described in Algorithm 1.Distillate initialization with Fourier transform perturbation. A typical privacy-enhanced technique for sharing synthetic data is adding noise [44] . Although this approach can blur the visual information of the synthetic data, it can also destroy important semantic information and seriously degrade model performance. Our approach leverages a well-known property of the Fourier transform: the phase component of the Fourier spectrum encodes high-level semantic information, whereas the amplitude component captures low-level details [45, 46] . Inspired by this, we propose a novel privacy-enhanced method that perturbs the amplitude components in Core-Set samples through the Fourier transform to reduce visual information while preserving semantic information. Given an image x, its Fourier transform can be formulated as:F(x) = A(x) × e -j×P(x)(3) where A(x), P(x) depict the amplitude and phase components respectively. We then perturb the amplitude information via linearly interpolating:EQUATIONwhere the λ is a scaling coefficient and x * can be other images or random noise. Then, we combine the perturbed amplitude spectrums with the original phase component to generate the perturbed Core-Set sample:EQUATIONwhere F -1 (x) defines the inverse Fourier transform which can be calculated with the FFT algorithm [47] effectively.Distillate synthesis with pre-trained Autoencoders. The Fourier transform perturbation serves to protect visual privacy but also compromises the realism of images, resulting in inconsistent Vinformation between the perturbed Core-Set and the original Core-Set. To address this issue, we propose optimizing the alignment of the V-information between the perturbed Core-Set and the original Core-Set to reconstruct key information. One straightforward and efficient approach is to optimize the perturbed Core-Set in pixel space [48, 49] . However, this method can be prone to overfitting into high-frequency patterns that only match the observer model [50] . Such overfitting is detrimental to training a global model due to inconsistent local models caused by data heterogeneity. Leveraging the powerful priors obtained from large-scale datasets, a pre-trained Autoencoder can decode latent representations into generalizable images. As a result, optimizing in the latent space acts as a regularization method that encourages synthetic data to be more generalizable, thereby making pre-trained Autoencoder an ideal distiller for local Core-Sets distillation. Furthermore, compact latent representations can reduce communication costs and mitigate privacy leakage if the latent is intercepted by attackers. Consequently, we employ a pre-trained Autoencoder on the client to distill the Core-Set into informative, privacy-enhanced, and communication-efficient distillates, and transmit them to the server for model training. On the server side, they are decoded by the decoder and are expected to maintain similar V-information to the original Core-Set from the perspective of the observer model while remaining visually unidentifiable.In specific, in the preparation phase of one-shot FL, the server first distributes a pre-trained Autoencoder to n clients, denoted as E and D for encoder and decoder, respectively. Each client i then conducts V-information-based selection to construct a Core-Set (X i s , Y i s ), i = 1, 2 • • • , n with diverse information regarding the original local datasets. Subsequently, client i learns a latent setZ i = {z j } |Z i | j=1 initialized by {E(x i j )} |X i s | j=1 , x i j ∈ X i s which is perturbed with Fourier transform, such that the {D(z i j )} |Z i |j=1 is as close as possible to the corresponding data in the Core-Set:arg min EQUATION(X i s , Y i s ) ← CoresetSelection(i) ▷ See Algorithm 2 in Appendix 4:for each x i j ∈ X i s do 5:Perturb x i j via Equations ( 3), ( 4) and ( 5)6:end for 7:Initialize latent setZ i = {E(x i j )} |X i s |j=1 using pre-trained VAE decoder E 8:for t = 1, • • • T syn do 9:for mini-batch(z i , x i s ) ∈ (Z i , X i s ) do 10:Compute synthetic loss L syn based on Equation (7) 11:z i ← z i -η syn ∇ z i L syn 12:end for 13:end for 14:Generate soft label for each synthetic latentsY i s = {f i (h i (D(z i j )))} |Z i | j=115:Transmit S i = (Z i , Y i s ) to the server 16: end for17: Combine client synthetic data into S = (Z, Y s ) = (Z 1 ∪ • • • ∪ Z n , Y 1 s ∪ • • • ∪ Y n s ) 18: for t = 1, • • • , T trn do 19: for mini-batch (z, y) ∈ (Z, Y s ) do 20:Compute L trn based on Equation (8) 21:θ ← θ -η trn ∇ θ L trn 22:end for 23: end for As Core-Set selection employs the local pre-trained models as the observer models V i , the reformulated Equation ( 6) and objective function can be formulated as:EQUATIONwhere h i (•) denotes the feature extractor of pre-trained local model of client i, and x i j and z i j are paired. By minimizing L syn , we synthesize a set of latent variablesZ i = {z j } |Z i | j=1 that contains diverse information of local data domains.Finally, clients transmit the latent set Z i along with the corresponding soft label Y i s predicted by local models to the server. The server combines the synthetic local distillates from each client S = (Z, Y s ). It then uses decoder D to reconstruct the images from data (z, y) ∈ (Z, Y s ) and distills the knowledge by minimizing the following objective function:EQUATIONwhere f (h(•)) denotes the server model. By minimizing the KL loss, we can transfer the local knowledge in the distillate to the server model.Discussion on privacy. We first consider whether an attacker can train a performant model with the intercepted distilled data and labels during transmission [40, 9] . Because the attacker cannot know that the distilled data is encoded by VAE, nor can the attacker access the pre-trained VAE encoder, which can be easily achieved by being predefined offline or via encryption, it is hard for the attacker to reproduce an effective model. For model inversion and membership inference attacks, according to [51] , there has been no prior research has performed these attacks solely using distilled data and labels and previous works [52, 36, 35, 43] have also revealed the advantage of dataset distillation in this regard. Therefore, we employ the synthetic data as the reconstructed samples for the evaluation of model inversion attacks. Furthermore, we compare our proposed Fourier transform perturbation with other privacy-enhanced techniques, including adding random noise to synthetic samples and data augment [53] . Experimental results can be found in Section 4.3.1.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
experimental setup: 
 Datasets and partitions. We conduct experiments on three real-world image datasets with different ranges of resolution including Tiny-ImageNet [54] , ImageNette [55] , and OpenImage [56] . Tiny-ImageNet contains 10000 images of 64×64 resolution across 200 classes. ImageNette is a widely used subset of 10 classes from ImageNet-1K [57] with 9469 color images, resized to 128×128. OpenImage is a large-scale real-world vision dataset with over 9 million images of 256×256 resolution. To simulate data heterogeneity in real-world applications of one-shot FL, we use Dirichlet distribution to generate non-IID data to generate non-IID local data, as in [58] for Tiny-ImageNet and ImageNette. Specifically, for client i, we sample p i k Dir(α) to allocate a p i k proportion of class k to client i. The parameter α controls the degree of data heterogeneity, with smaller α indicating severe data heterogeneity. The α is set to 0.1 by default unless otherwise stated. For OpenImage, we randomly choose n real-world clients from FedScale [59] and use their corresponding test sets to form global sets. We set the default number of clients n to 10, unless otherwise specified.Baseline methods and Configurations. We compare our proposed FedSD2C with existing methods: FedAVG [1] , DENSE [8] and Co-Boosting [10] . Following [8, 10] , we also introduce DAFL [60] with one-shot FL settings, denoted as F-DAFL. We use two different model architectures: ConvNet [28] and ResNet-18 [61] for all methods. In FedSD2C, the image per class ipc is set to 50 for Tiny-ImageNet and ImageNette, and 10 for OpenImage. We set the Fourier transform coefficient λ = 0.8 and use a public pre-trained Autoencoder from Stable Diffussion [62] by default for all tasks. For distillate synthesis, we set T syn = 50, η syn = 0.1 by default. η trn is set to 0.2 for Tiny-ImageNet and 0.02 for ImageNette and OpenImage. More experimental details can be found in the Appendix.To evaluate the effectiveness of our method, we conduct experiments under various non-IID settings with α = {0.1, 0.3, 0.5} for Tiny-ImageNet and Imagenette and pre-defined splits [59] for OpenImage. As illustrated in Table 1 , our proposed FedSD2C surpasses all other methods in most settings. In particular, under extreme data heterogeneity(α = 0.1), FedSD2C achieves up to 1.3×, 2.6×, and 1.8× the accuracy of the best baseline on ImageNette, Tiny-ImageNet and OpenImage, respectively. This superior performance is attributed to FedSD2C's approach of sharing synthetic distillates rather than inconsistent local models, thereby mitigating the impact of data heterogeneity. Moreover, FedSD2C demonstrates the independence from model structures. In contrast, other methods struggle to adapt to different model structures and complex datasets. For instance, at α = 0.5, Co-Boosting with ResNet-18 achieves only half the accuracy of ConvNet on ImageNette, whereas FedSD2C maintains consistent performance. This discrepancy arises because differences in model capacity affect their ability to condense local knowledge and the two-tier information loss during data generation increases the difficulty of transferring local knowledge to the server model, resulting in poor robustness to complex datasets and varied networks. In contrast, the shared distillates in FedSD2C are synthesized through end-to-end local distillation, mitigating information loss during knowledge transfer.For privacy evaluation, we consider an honest-but-curious server attempting to reconstruct client data from distillates. We compare our proposed FedSD2C with other privacy-enhanced techniques for sharing synthetic data, including adding random noise [44, 63] and FedMix [53] . In the random noise approach, we incorporate it into FedSD2C by removing the Fourier transform perturbation and instead directly using Core-Set samples for initialization. We then add random noise to the synthetic distillates before transmitting them to the server, following the methods in [44, 63] . Specifically, Given latent z, a perturbation coefficient p, randomly generated noise e and its scale parameter s, the data to be shared is formulated as z = (1 -p)z + e × s. FedMix [53] proposes using linear interpolation of real samples to preserve privacy. In this approach, we synthesize data by averaging each two real samples from the Core-Set. For our proposed Fourier transform perturbation, we vary the λ = {0.1, 0.5, 0.8} and observe the variations in performance and privacy protection. To quantitatively evaluate the privacy protection of the synthetic data, we employ the Peak Signal-to-Noise Ratio (PSNR) and Structure Similarity Index Measure (SSIM). A higher PSNR or SSIM value indicates greater similarity between the synthetic samples and the original samples, which implies more severe privacy leakage. We calculate the average PSNR and SSIM values of all the synthetic samples.As depicted in Table 2 , although FedMix provides better privacy protection, as evidenced by lower PSNR and SSIM values, it comes at the expense of significant performance degradation. The application of random noise requires a delicate balance between performance and privacy protection. For example, with a perturbation coefficient p = 0.2, it offers similar privacy protection to that of FedMix, but the performance drops approximately 10% compared to p = 0.1. However, the p = 0.1 setting increases the risk of privacy leakage. In comparison, the synthetic distillates generated by our proposed FedSD2C achieve comparable PSNR values with them. This suggests that our proposed Fourier transform perturbation offers effective privacy protection for the real data sample. Furthermore, FedSD2C consistently outperforms other methods in terms of accuracy while maintaining a minimal performance degradation compared to no privacy protection techniques. This indicates that FedSD2C strikes a balance between privacy preservation and performance. We also By employing Core-Set selection and communication-efficient distillate communication, our FedSD2C condenses local data to mere MBs, while the model trained on these condensed data exhibits comparable performance, as shown in Table 3 . Specifically, the communication costs of FedSD2C for sharing synthetic distillate is at most 4% of that of sharing model. Notably, we exclude the communication costs of sending and receiving pre-trained Autoencoder, as this can be pre-defined offline, allowing for the reuse of multiple one-shot FL tasks. Given the considerable capacity for communication costs, we further investigate the scalability of communication efficiency and performance. Our experimental results demonstrate that increased data transmission enhances the diversity of compressed data, leading to further improvements in performance. By increasing ipc from 20 to 80, the accuracy boosts by 13.03% and 4.48% on ImageNette and Tiny-ImageNet respectively. Additionally, we compare FedSD2C without Autoencoder at equivalent communication costs, where the absence of synthesized images limits performance to at best half of the default setting. The communication-efficiency of FedSD2C highlights its practicality in real-world applications. Adopting pre-trained Autoencoders on other data domains can reduce performance. However, this can be mitigated by increasing T syn . (b) Experiments of FedSD2C with randomly initialized downsampling and upsampling modules (blue line) compared to pre-trained Autoencoders (orange line) on ImageNette. Without pre-trained knowledge, FedSD2C requires a higher T syn for distillate synthesis but can still achieve comparable results. ResNet-18 is used for both experiments.Pre-trained Autoencoders are typically trained on natural data domains [64] , while practical applications of federated learning often involve a broader range of domains, such as medical images. This raises the question of whether pre-trained Autoencoders remain effective when applied to a different domain and whether our proposed FedSD2C can adapt to these differences. To investigate this, we evaluate performance using a medical dataset COVID-FL [65] .As shown in Figure 3a , using the default synthesis iteration of T syn = 50 yields suboptimal results. By increasing T syn to 1000, the performance improves and then stabilizes. This observation suggests that the pre-trained knowledge of Autoencoders may influence the speed of distillate synthesis convergence. To further validate this, we replace the encoder and decoder of pre-trained Autoencoders with randomly initialized downsampling and upsampling modules. We vary T syn from 50 to 1000 and compare its performance with employing pre-trained Autoencoders on ImageNette, setting ipc = 80 for better illustration. Figure 3b demonstrates that as T syn increases, the performance of FedSD2C with randomly initialized modules improves progressively, eventually matching the performance of FedSD2C with pre-trained Autoencoders. In summary, while pre-trained knowledge can enhance convergence rate, FedSD2C can achieve comparable performance by adjusting T syn , demonstrating its adaptability across domains.As practical FL deployments often involve participating clients [59] , we evaluate our FedSD2C with various numbers of clients n = {20, 50, 100} and maintain consistent communication budget by setting ipc = {40, 20, 10}, respectively. We compare these methods under on Tiny-ImageNet with data heterogeneity α = {0.1, 0.3, 0.5} for partitions and employ ConvNet. As depicted in Table 4 , FedSD2C consistently achieves the highest accuracy as the number of clients increases. Moreover, FedSD2C demonstrates greater robustness to the number of participants. Specifically, as the number of participants changes, the accuracy of FedSD2C fluctuates within only 1%. In contrast, the accuracy of F-DAFL, DENSE, and Co-Boosting dropped by up to 4.71%, 3.49%, and 3.10%, respectively under different settings. This further validates the utility of sharing synthesized distillates in real-world one-shot FL applications. 
limitations: 
 The local distillation process introduces additional computational overhead. While Core-Set selection requires no training and the distillate synthesis process only requires 50 iterations with a speed of 0.4s/per image on RTX3090, it still imposes higher resource requirements on the local device compared to the method of sharing model. One direction worth exploring is to integrate with the model market [66] to enable clients to synthesize distillates once for permanent use. 
conclusion: 
 In this paper, we propose a new one-shot FL framework driven by distiller-distillate communication, denoted as FedSD2C, to alleviate the information loss of knowledge transfer and impacts of data heterogeneity. FedSD2C compresses the local data into Core-Set with V-information and employs a pre-trained Autoencoder as the distiller to distill informative, communication-efficient, and privacy-enhanced distillates from Core-Set. Moreover, We discuss FedSD2C's resistance to attackers intercepting distillate communications and attacks from honest-but-curious servers and introduce Fourier transform perturbation to further minimize the risk of privacy leakage. Empirical results validate the effectiveness of FedSD2C in transferring local knowledge to the server in one-shot FL while balancing communication efficiency and privacy protection.A V-information based Core-Set Selection Description {x k } K k=1 ← extract K patches with multiple scales from x. for k = 1 to K do 6: Calculate V-information score s k ← -L(h(x k ), y) end for 8: select x ′ with highest score s ′ X ′ T {(x ′ , y, s ′ )} 10: end for Level 2: select the top-ipc patches with highest V-information 12: X s , Y s ← ∅ for each class c ∈ Y t do 14: if size of {(x ′ j , y j , s ′ j )}, y j = c ≥ ipc then select top-ipc {(x ′ j , y j )} ipc j=1 via s ′ j 16: X s {x ′ j } ipc j=1 ,Y s {y j } ipc j=1 end if 18: end for return (X s , Y s ) 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: We discuss the potential impacts about privacy issues.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA]Justification:Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We cite all sources used and comply with their licenses.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. 
 
 Appendix: We use the SGD optimizer with momentum=0.9, learning rate=0.01 and weight decay=0.0001 for clients' local training. The batch size is set to 128 and local epoch is 200. For all generation based methods, we set the resolution of the generated images to 64 × 64, 128 × 128 and 256 × 256 for Tiny-ImageNet, ImageNette and OpenImage, the number of generated images in each batch is 128, and the learning rate of the generator is 0.001, the latent dimension is 256, iteration for training generator is 30, using Adam for optimization. The server model is optimized with SGD with momentum 0.9, the learning rate is 0.01, and the training epochs are 200. The synthesized batch size and server model training batch size is both 128. In DENSE, we set λ 1 = 1 for BN loss and λ 2 = 0.5 for diversity loss. In Co-Boosting, the perturbation strength is set to ϵ = 8/255 and the step size µ = 0.1/n. In Core-Set selection stage of FedSD2C, for each image x i , we employ the torchvision.transform.RandomResizeCrop K times to generate a collection of patches. For patch size, we set the scale=(0.08, 1.0), which is to collect diverse image patches. Following [42] , we employ ConvNet-4 for Tiny-ImageNet, ConvNet-5 for ImageNette and ConvNet-6 for OpenImage. All methods are implemented with Pytorch and conducted on GeForce RTX 3090.Details of t-SNE plots in Figure 1 We randomly select a client and five classes from its local dataset (Tiny-ImageNet) and employs its local model (ResNet-18) to extract features. The feature is extracted from the final layer (before the classifier). We then use t-SNE plots to illustrate the feature distribution.C.1 Additional datasets CIFAR-10. We set the resolution of generated images to 32×32 for CIFAR-10 [67] and keep all the other settings the same. As shown in Table S1 , there is an initial performance discrepancy at the standard setting of ipc = 50. This occurs because our method prioritizes efficiency with large datasets rather than low-resolution ones. However, upon increasing the amount of synthetic data (ipc = 500), our method can still achieve comparable results.selection, the accuracy of random selection decreased by 3.5 and 5.46 on Tiny-ImageNet and ImageNette, respectively. We also report the performance of uploading Core-Set directly, which achieve the best performance. However, without distillate synthesis, it will increase the cost of communication and the risk of privacy leakage. According to [35] , introducing DP-SGD [69] during the distillate synthesis stage can provide theoretical privacy guarantees for our method. We perform experiments of integrating DP-SGD in our method on Tiny-ImageNet with ResNet-18 (α = 0.1, ipc = 50) to provide a clear view of the trade-offs involved. Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 


Decision: This paper proposes a one-shot FL method (FedSD2C), utilizing V-information to select local core set data and server-pretrained autoencoder and Fourier-domain perturbation to ensure privacy preservation for local "distillate" sharing. In comparison to existing works such as DENSE and Co-Boosting, FedSD2C can reduce information loss in one-shot FL and improve up to 2.7x global model performance.

Weakness:

The biggest concern is the privacy protection. Although authors conduct experiments for MIA, but this is not sufficient to show the privacy protection.

•	The assumption that the server holds an autoencoder is pretty strong, as the autoencoder must be trained in the clients' data domain to ensure it works. 

•	The paper only showed that Fourier-based perturbation can 'visually' protect privacy by using PSNR as a metric. Although the paper mentioned MIA, it did not evaluate existing privacy attacks. Given that communication efficiency and privacy are highlighted as key contributions (contribution 2), revising the privacy-related statements could necessitate substantial changes to the original submission and might reduce overall contributions. 

•	As for the experiment on privacy attacks, since the model's performance is too low (far below 90%), it will also lead to biased MIA results. Therefore, it is not recommended to claim in the paper that your method can protect privacy (without any differential privacy guarantee).

Reviews: review: paper_summary: paper_summary: This paper proposes a one-shot federated learning approach designed to enhance privacy protection, communication efficiency, and model performance. Firstly, the authors introduce a Core-Set selection method based on V-information to extract the most informative data from the original dataset. The amplitude spectrum of the images in the Core-Set is then perturbed using a Fourier transform, and these perturbed images are input into an Autoencoder to obtain their representations. Finally, these representations are transmitted to the server, which decodes the image information from the representations for training.
strength_weakness: strength_weakness: 1. In the Core-Set Selection stage, the authors do not explicitly define the patches used in Level 1: identifying the most informative image segments. I have the following questions: Are the patches the same size across different datasets? How many patches are extracted from each image? Does the size or number of patches affect the results? If so, how can one determine the optimal patch size?

2. In the perturbation stage, the authors do not provide sufficient reasoning for using the Fourier transform. Why must the Fourier transform be used instead of other transformations? For example, does using a wavelet transform and merging its low-frequency components achieve similar effects? Additionally, the authors mention that the merged image can be random noise. Is this too idealistic? Does using particularly weak random noise also achieve similar effects?

3. During the transmission phase using an Autoencoder, does the performance of the model depend on the pre-training of the Autoencoder? If a pre-trained model is not used, does this method still work effectively?

4. It would be beneficial for the authors to create a framework diagram of the entire method to enhance readability.


5. The paper contains many capitalization and punctuation errors, such as in line 50 and line 229. Additionally, please carefully review the citations in the article as they are quite disorganized, including issues with formatting, capitalization, and more. Please ensure consistency throughout.
questions: questions: 1. The paper utilizes several large datasets, but is the method effective for low-resolution datasets such as CIFAR-10?

2. The authors should provide the model performance using only the Core-Set, where clients send all their original Core-Set data to the server for centralized training.

3. Please provide additional visual metrics beyond PSNR to make the results more convincing, such as SSIM.

4. The authors should conduct ablation experiments to demonstrate the performance of each part of the proposed method. For example, evaluating the model performance without the Core-Set data selection step.

5. Since only representations are ultimately transmitted, and attackers cannot recover images from representations, I wonder if the image perturbation stage is still necessary?
limitations: limitations: Yes, have discussed the limitations
paper_summary: paper_summary: The paper presents FedSD2C, a novel one-shot federated learning (FL) framework that aims to improve communication efficiency, privacy preservation, and model performance. The approach addresses issues with data heterogeneity and information loss by synthesizing informative distillates from local data and sharing these instead of local models. Empirical results show that FedSD2C significantly outperforms existing one-shot FL methods.
strength_weakness: strength_weakness: 1. A critical question on the optimal observer of approximating the V-information is that, the local model is trained locally, which may not indicate good V-information of the global datasets, i.e., all local dataets.
2. The privacy is not strictly guaranteed. The perturbation with 3, 4, 5 cannot ensure data privacy.
3. Figure 2 shows the reconstruction is very similar with the original images.
4. There is no theoretical analysis on privacy and generalization of the proposed method.
5. The pre-trained autoencoder plays a key role in the framework. But the experimental study doesn't investigate the impact of the pre-trained autoencoder on the performance.
questions: questions: 1. What are the key hyper-parameters used in the core-set selection algorithm?
2. According to Algorithm 2, the core-set includes patches with different scales. How does the encoder handle patches with different scales?
3. There is still a huge performance gap between FedSD2C and Central in Table 1. What are the main factors that result in the performance drop?
limitations: limitations: The authors have pointed out the computational overhead on local devices as the limitation.
paper_summary: paper_summary: This paper proposes a one-shot FL method (FedSD2C), utilizing V-information to select local core set data and server-pretrained autoencoder and Fourier-domain perturbation to ensure privacy preservation for local "distillate" sharing. In comparison to existing works such as DENSE and Co-Boosting, FedSD2C can reduce information loss in one-shot FL and improve up to 2.7x global model performance.
strength_weakness: strength_weakness: - The assumption that the server holds an autoencoder is pretty strong, as the autoencoder must be trained in the clients' data domain to ensure it works. 

- The paper only showed that Fourier-based perturbation can 'visually' protect privacy by using PSNR as a metric. Although the paper mentioned MIA, it did not evaluate existing privacy attacks. 

- The clarity of paper writing can be improved. Essential details are missing to understand the contributions. See more in the Questions section.
questions: questions: - Can you more rigorously demonstrate the privacy-preservation statement? For example, differential privacy is a widely accepted concept. Could you please show the provable privacy-preservation guarantee like DP?

- It is unclear how Figure 1 is generated. The authors should detail the datasets, model, and necessary information. Otherwise, comparing the figures is meaningless.

- What is the computational cost for the core-set selection step? 

- It is unclear which parts of the algorithm contribute to handling the heterogeneous data. It seems that coreset selection could contribute well if intentionally sampling (almost) balanced data.

- What is the number of clients for Table 1?
limitations: limitations: Limitation was discussed
paper_summary: paper_summary: The paper introduces FedSD2C, a new one-shot Federated Learning (FL) framework designed to address challenges in existing methods. Previous approaches have used data-free knowledge distillation to improve one-shot FL, but these methods struggle with data heterogeneity and scalability issues.  FedSD2C aims to solve these issues by:
1.Introducing a distiller to synthesize informative distillates directly from local data, reducing information loss.
2.Sharing synthetic distillates instead of inconsistent local models to address data heterogeneity.
The authors claim that empirical results show FedSD2C outperforms other one-shot FL methods, especially with more complex and real datasets.
strength_weakness: strength_weakness: Thanks for the authors' efforts in presenting this paper. After carefully reading it, I have the following questions and comments:

1. On line 52, the values 4.21 and 2.06 are not clearly explained. I suggest adding a figure or table to provide more specific details about these numbers.
2. While I can imagine that transmitting synthetic data could improve the global model's performance more easily than transmitting the model itself, this inevitably leads to privacy concerns. Although the authors provide some data reconstruction experiments, the results are not entirely convincing:
    a) To my knowledge, dataset distillation/coreset selection does not inherently protect privacy [1]. I suggest the authors include additional experiments on Membership Inference Attacks (MIA) (based on results from data-free KD [2], I suspect coresets provide even less privacy protection, since data-free KD doesn't use any 'real' training data ).
    b) According to the privacy onion concept [3], memorization is relative, then in your method the selected images may be at higher risk of privacy leakage compared to those not selected.

3. Compared to other methods, the proposed method communication overhead seems larger. It requires to transmit both the auto-encoder and a large amount of synthetic data.

4. The method relies heavily on Stable Diffusion's auto-encoder, which seems to utilize very strong prior information. How well would your method perform on datasets where diffusion models are not as proficient, such as medical datasets?

5. To my knowledge, another one-shot FL work[4] that relies on Stable Diffusion achieved results of about 75.0 on ImageNette, which appears significantly better than your 55.90. 

[1] No Free Lunch in "Privacy for Free: How does Dataset Condensation Help Privacy"

[2] Evaluations of Machine Learning Privacy Defenses are Misleading

[3] The Privacy Onion Effect: Memorization is Relative

[4] Federated Generative Learning with Foundation Models
questions: questions: see above
limitations: limitations: see above

Comments: title: Thank you for raising the score!
A friendly reminder
Response
Response
Thank you for the rebuttal
Response to rebuttal
Thanks for the response
Thank you for raising the score!
response
Paper discussion
Looking Forward to Further Discussions
Looking Forward to Further Discussions
Looking Forward to Further Discussions
Looking Forward to Further Discussions
comment: Thank you very much for your acknowledgment of our rebuttal. We will include the results into our revision.
Thanks for the feedback of Reviewer Bcom and we are encouraged that most of the concerns and questions are addressed. As mentioned by the reviewer, we will definitely include the results in the rebuttal into our revision.

We have noted that the current rating still tends toward the negative. We kindly request clarification on any unresolved issues that might be affecting the reviewer's rating. Please feel free to share any remaining concerns. We are fully committed to addressing these issues during the remainder of the review discussion period. We greatly appreciate your efforts and look forward to your additional feedback.
Dear Reviewer 3gL8,

Thank you for your constructive feedback. 

1. **Pre-trained Autoencoder**. We acknowledge that the effectiveness of a pre-trained autoencoder can be significantly reduced when applied to different data domains. When validating on the medical dataset COVID-FL, we set the number of iterations for distillate synthesis to $T_{syn}=1000$, which is much more than the $T_{syn}=50$ used for natural dataset, and observed a severe performance degradation when set to $T_{syn}=50$ for COVID-FL. We believe the primary impact may be on convergence speed. To illustrate, we conducted an experiment where we replaced the encoder and decoder of the pre-trained autoencoder with randomly initialized downsample and upsample layers, respectively, during distillate synthesis. The experiment is conducted on ImageNette with ResNet18 ($\alpha=0.1,ipc=80$). As illustrated in the table below, random initialization achieved comparable performance to the pre-trained Autoencoder after twenty times more iterations. The results illustrate two conclusion: 1) our method is not sensitive to the data domain and can achieve good results with more optimization iterations, and 2) the image prior in the pre-trained Autoencoder can speed up the optimization of distillate synthesis, and a slow convergence speed may be a potential indicator of the failure of a pre-trained autoencoder.

| Method and Dataset                  | Accuracy | Iteration |
| ----------------------------------- | -------- | --------- |
| Pre-trained AE, COVID-FL          | 46.45    | 50        |
| Pre-trained AE, COVID-FL          | 52.65    | 1000      |
| Pre-trained AE, ImageNette        | 56.13    | 50        |
| Random Initialization, ImageNette | 54.54    | 1000      |

2. **Privacy**. We acknowledge that our method does not provide provable privacy protection. Our primary emphasis is on empirical contributions, demonstrating that our method is superior in empirical evaluation compared to other baselines. As suggested by Reviewer nVDn, we have included the SSIM metric to further validate our method. Additionally, we perform experiments of integrating DP-SGD in our method on Tiny-ImageNet with ResNet18 ($\alpha=0.1,ipc=50$) to provide a clear view of the trade-offs involved. The results are shown below:

|         | $\epsilon=1$ | $\epsilon=4$ | $\epsilon=8$ | $\epsilon=\infty$ |
| ------- | ------------ | ------------ | ------------ | ----------------- |
| FedSD2C | 22.92        | 25.13        | 26.01        | 26.83             |

Thank you once again for your valuable comments. Your insights have been invaluable in refining our approach and understanding the practical implications of our work. We will include all these discussions in our final version.
Dear Reviewer nVDn,

We would like to thank you for your constructive feedback.

As suggestion, we perform experiments using the Wavelet Transform on Tiny-ImageNet with ResNet18($\alpha=0.1,ipc=50$). The results indicate that Wavelet Transform offers greater scalability in privacy protection. By increasing $\lambda$, the PSNR/SSIM can be reduced to as low as 12.90/15.30. When the accuracy is comparable to that of Fourier transform (Wavelet $\lambda=0.5$ vs. Fourier $\lambda=0.8$), the PSNR/SSIM of Wavelet transform is lower. We sincerely appreciate your innovative comments and will include this discussion in our final version.

|                        | Acc.  | PSNR  | SSIM  |
| ---------------------- | ----- | ----- | ----- |
| Wavelet($\lambda=0.1$) | 28.05 | 18.86 | 44.76 |
| Fourier($\lambda=0.1$) | 28.22 | 20.54 | 51.50 |
| Wavelet($\lambda=0.5$) | 26.91 | 15.22 | 27.34 |
| Fourier($\lambda=0.5$) | 28.09 | 18.06 | 43.26 |
| Wavelet($\lambda=0.8$) | 26.06 | 12.90 | 15.30 |
| Fourier($\lambda=0.8$) | 26.83 | 16.95 | 35.89 |
Thank you for the rebuttal, which partially addresses my original questions. 

It would be highly beneficial to delve deeper into the required capacity of an autoencoder, especially when it is trained on a domain different from the one being applied. The exploration of circumstances under which adaptation might fail, along with potential techniques to mitigate such failures or indicators that could signal these failures, is crucial. A deeper understanding of these underlying assumptions would significantly enhance the practical value of the proposed method.

Additionally, I share the same concerns as Reviewer Bcom regarding the original submission's emphasis on privacy. Given that communication efficiency and privacy are highlighted as key contributions (contribution 2), revising the privacy-related statements could necessitate substantial changes to the original submission and might reduce overall contributions. I appreciate the inclusion of the new experiment with LiRA in the rebuttal. To further substantiate the privacy claims, it would be advisable for the authors to incorporate a more comprehensive empirical evaluation. Additionally, the rebuttal mentions that integrating differential DP into data generation could bolster privacy. However, it would be valuable to understand how this integration might impact model performance in practical scenarios.

To clarify, my follow-up questions are not intended to request additional experiments but rather to better understand the merits and limitations of the work and to have a clearer picture of the revision plan.
Thanks for the clarification and responses. I will rasie my rating to 5.
Dear authors,

Thanks for your response. You have addressed most of my concerns, so I will raise the score to 5. However, one remaining concern is that you need to compare the Fourier transform and wavelet for different scenarios. For certain scenarios, wavelet may outperform the Fourier transform.
We sincerely appreciate the time and effort you have invested in reviewing our paper, and your insightful feedback has been a critical factor in enhancing the overall quality of our work. 

Following your comments, we will revise the privacy statement in the abstract and introduction to accurately reflect its limitations and ensure there is no overstatement. Additionally, we will include a discussion on work[1], as well as all other comments.

Once more, we appreciate the time and effort you've dedicated to our paper.

[1] Zhang J, et al. Federated Generative Learning with Foundation Models. 2023.
Thanks for authors' response. 
As for the experiment on privacy attacks, since your model's performance is too low (far below 90%), it will also lead to biased MIA results. Therefore, I don't recommend claiming in the paper that your method can protect privacy (without any differential privacy guarantee).
For another one-shot FL work[1], I think they also tested the method on medical datasets. 

After reading the rebuttal, I am happy with increasing the score to a 5 or 6 (if there is no overstatement in the final version).

[1] Zhang J, et al. Federated Generative Learning with Foundation Models. 2023.
Hi all,

Thanks again for your reviews!
If you haven't done so already, please respond to the rebuttals before the end of the author-reviewer discussion period. If you don't have any further questions for the authors then a simple "thank you for the rebuttal" would suffice.

All the best,
Area Chair
Dear Reviewer Bcom,

Thank you once again for your constructive comments and the effort you put into reviewing our submission. Please let us know if our response has addressed your concerns. We are more than happy to address any further comments you may have.

Thanks!
Dear Reviewer 3gL8,

We would like to thank you once again for the insightful feedback and great efforts in reviewing our paper. Please let us know if you have follow-up concerns, and we are eager to engage in any further comments.

Thanks!
Dear Reviwer EquD,

We would like to thank you again for your constructive comments and kind effort in reviewing our submission. We kindly ask you to inform us if our replies have successfully resolved your concerns, and we are more than happy to address any further comments.

Thanks!
Dear Reviewer nVDn,

We sincerely appreicate your great efforts in reviewing our submission. Your constructive comments really help improve our paper. Please do let us know if our response has addressed your concerns, and we are more than happy to address any further comments.

Thanks!