Dual Defense: Enhancing Privacy and Mitigating Poisoning Attacks in Federated Learning

Authors: runhua  xu
shiqi  gao
chao  li
james  joshi
jianxin  li
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 16:51:05

Abstract: Federated learning (FL) is inherently susceptible to privacy breaches and poisoning attacks. To tackle these challenges, researchers have separately devised secure aggregation mechanisms to protect data privacy and robust aggregation methods that withstand poisoning attacks. However, simultaneously addressing both concerns is challenging; secure aggregation facilitates poisoning attacks as most anomaly detection techniques require access to unencrypted local model updates, which are obscured by secure aggregation. Few recent efforts to simultaneously tackle both challenges offen depend on impractical assumption of non-colluding two-server setups that disrupt FL's topology, or three-party computation which introduces scalability issues, complicating deployment and application. To overcome this dilemma, this paper introduce a Dual Defense Federated learning (DDFed) framework. DDFed simultaneously boosts privacy protection and mitigates poisoning attacks, without introducing new participant roles or disrupting the existing FL topology. DDFed initially leverages cutting-edge fully homomorphic encryption (FHE) to securely aggregate model updates, without the impractical requirement for non-colluding two-server setups and ensures strong privacy protection. Additionally, we proposes a unique two-phase anomaly detection mechanism for encrypted model updates, featuring secure similarity computation and feedback-driven collaborative selection, with additional measures to prevent potential privacy breaches from Byzantine clients incorporated into the detection process. We conducted extensive experiments on various model poisoning attacks and FL scenarios, including both cross-device and cross-silo FL. Experiments on publicly available datasets demonstrate that DDFed successfully protects model privacy and effectively defends against model poisoning threats.

Main paper: 
 introduction: 
 Federated learning (FL) [18] is gaining popularity as a collaborative model training paradigm that provides primary privacy protection by eliminating the need of sharing private training data. Based on the participants' scale, FL is typically divided into two categories: cross-silo FL and cross-device FL [17] . Cross-device FL typically involves numerous similar devices, while cross-silo FL usually includes fewer participants like organizations. Recent studies show that FL mainly confronts two types of threats: privacy risks from curious adversaries attempting to compromise data privacy through methods like membership inference and model inversion attacks, and security risks from Byzantine adversaries looking to damage the final model's integrity with backdoors or by lowering its accuracy [2, 24, 14, 11, 3, 1, 34] .To mitigate privacy risks in FL, researchers have developed a range of techniques to bolster privacy. These encompass differential privacy-based aggregation [32] , as well as secure aggregation approaches using homomorphic encryption [41] , functional encryption [35] , and secure multi-party computation [6, 43] . Aside from privacy concerns, many studies have proposed strategies to identify and mitigate potentially harmful updates during the model aggregation phase, thereby safeguarding the global model against adversarial attacks. Notable Byzantine-resistant aggregation mechanisms encompass the Krum fusion method [5] , cosine defense aggregation mechanism [29, 38] , and median/mean-based strategies like clipping median and trimmed mean strategies [40] . Research in these two areas has been conducted separately, and addressing both issues at once continues to be challenging. This difficulty arises because secure aggregation makes it easier for adversarial attacks to occur, as most anomaly detection methods need access to "unencrypted" local model updates that secure aggregation protects.Few recent efforts [39, 13, 16, 43, 15, 23, 9, 20] to tackle both challenges simultaneously often depend on differential privacy techniques [39, 13, 16, 22, 12] , which can degrade model performance due to added noise, or rely on impractical non-colluded two-server assumption that disrupts FL's topology [43, 15, 23, 9, 20] , complicating its deployment and application. In light of these limitations, a critical yet overlooked question is how to create a straightforward dual defense strategy that simultaneously strengthens privacy protection and mitigates poisoning attacks without introducing new participant roles or altering the single-server multiple-clients structure?To address this dilemma, this paper proposes a Dual Defense approach that simultaneously enhances privacy protection and combats poisoning attacks in Federated learning (DDFed), without changing the structure of current FL frameworks. DDFed initially leverages cutting-edge cryptographic technology, specifically fully homomorphic encryption (FHE), to securely aggregate model updates without the impractical assumption of non-colluding two-server setups and ensures strong privacy protection by permitting only the aggregation server to perform secure aggregation in the dark. To tackle the challenge of detecting malicious models within encrypted model updates, DDFed introduces a novel two-phase anomaly detection mechanism. This approach enables cosine similarity computation over encrypted models and incorporates a feedback-driven collaborative selection process, with additional measures to prevent potential privacy breaches from Byzantine clients incorporated into the detection mechanism. Our main contributions are summarized as follows:• We introduce a dual defense strategy that simultaneously boosts privacy and combats poisoning attacks in federated learning. This is achieved by integrating FHE-based secure aggregation with a mechanism for detecting malicious encrypted models based on similarity.• To effectively detect malicious models in encrypted updates, we propose a novel two-phase anomaly detection mechanism with extra safeguards against potential privacy breaches by Byzantine clients during the detection process. Additionally, we introduce a clipping technique to bolster defenses against diverse poisoning attacks.• We carried out comprehensive experiments on multiple model poisoning attacks and federated learning scenarios, covering both cross-device FL and cross-silo FL. Our experiments with publicly accessible datasets demonstrate DDFed's effectiveness in safeguarding model privacy and robustly defending against model poisoning threats. 
related works: 
 Privacy Risks and Countermeasures in FL The fundamental design of FL ensures that all training data stays with its owner, offering basic privacy. However, it still exposes vulnerabilities to inference attacks, which allow adversaries to extract information about the training data used by each party [24, 27, 2, 24, 14, 11] . In some cases, the risk of private information leakage may be unacceptable. Therefore, several defenses have been suggested to mitigate these risks, including differential privacy (DP) and secure aggregation (SA), based on various cryptographic primitives such as (partial) homomorphic encryption [21, 41] , threshold Paillier [30] , functional encryption [36] , and pairwise masking protocols [6] .Poisoning Risks and Countermeasures in FL. Besides privacy inference attacks, FL is also susceptible to poisoning attacks, where adversaries can compromise certain clients and manipulate their data or models to intentionally worsen the global model's performance by introducing corrupted updates during training. This paper focuses on untargeted model attacks, whose goal is to significantly diminish the effectiveness of the global model through methods such as Inner Product Manipulation (IPM) attack [34] , scaling attack [1] , and "a little is enough" (ALIE) attack [3] . Several strategies have been developed to counteract the impact of attacks, ensuring they don't compromise model performance. These strategies fall into two categories: client-side and server-side defenses. Client-side defenses adjust the local training algorithm with a focus on secure client updates [28] , whereas server-side defenses [5, 29, 38, 40] either reduce the influence of updates from malicious clients through adjusted aggregation weights or use clustering techniques to aggregate updates from trustworthy clients only. However, these defense strategies typically operate under the assumption that model updates are not encrypted, which contradicts the objectives of privacy-focused secure aggregation defense strategies.Private and Robust Federated Learning. In privacy-preserving FL, identifying poisoning attacks is harder because of the need to balance local model privacy with the detection of harmful models. Only a few existing studies like those mentioned in [39, 13, 16, 15] employ Byzantine-resilient aggregation through differential-privacy techniques. This approach necessitates a compromise between privacy and model accuracy. Additionally, recent initiatives have been launched to address this problem through diverse methods by using various secure computation technologies. These include 3PC [9] , which faces scalability limitations; an oblivious random grouping method constrained by its design for partial parameter disclosure [43] ; and both additive secret sharing [20] and twotrapdoor homomorphic encryption [23] , which depend on the impractical assumption of non-colluding dual servers.3 Dual Defense Federated Learning Framework EQUATIONHere, L i represents the local loss function for each client's data, and W W W i are the local model parameters specific to client C i . The term D i refers to the private dataset of client i, with |D i | indicating its size in terms of sample count. In short, the goal of general FL is to learn an optimal global model W W W G across m clients. This is achieved by periodically synchronizing the model parameters from all clients using specified fusion algorithms like FedAvg and its variants, with the aggregation server A over several training rounds.Due to various malicious activities, including inference attacks that aim to steal private information from legitimate clients and poisoning attacks designed to undermine model integrity by degrading its performance, existing privacy-preserving FL often relies on a secure aggregation mechanism [21, 41, 30, 36, 6] . Typically, without loss of generality, during the t-th federated learning training round, each client C i secures its local model update W W W i -referred to as [[W W W i ]] throughout this paperbefore transmitting it to the aggregation server. This is achieved by using various privacy-enhancing technologies such as homomorphic encryption and secure multi-party computation.Threat Assumption. DDFed tolerates an adversary, capable of corrupting any subset of local clients at a specified ratio r ATTACK , s.t., r ATTACK < 0.5, to carry out model poisoning attacks that degrade the global model's performance. Additionally, we assume the aggregation server A is semihonest (honest-but-curious), meaning it adheres to the protocol but seeks to glean as much private information as possible. Similarly, the compromised clients C ADV i can conduct privacy inference attacks like those performed by A. In summary, regarding privacy preservation, both the inquisitive A and the corrupted client subset aim to extract private information from benign clients; however, only the corrupted client subset will also initiate model poisoning attacks to undermine the global model. 
framework details: 
 Objective of DDFed. DDFed is designed to bolster privacy protection and mitigate model poisoning attacks seamlessly within the existing FL framework. Unlike existing private and robust approaches [39, 13, 16, 43, 15, 23, 9, 20] that add new participant roles or depend on differential privacy, which may compromise model performance, DDFed maintains effectiveness efficiently. DDFed introduces a dual defense strategy that combines fully homomorphic encryption (FHE) for secure data aggregation with an optimized similarity-based mechanism to detect malicious models, ensuring unparalleled privacy protection and security against model poisoning attacks.Similarity-based methods are commonly used in existing studies for anomaly detection models [29, 38] . Specifically, it computes the cosine similarity between each local model update of training round t and the global model from the previous round t -1:EQUATIONwhere α i denotes the angle between global model weights W W W i . Subsequently, each client submits their list of similarity scores ( 5 ⃝). It's important to note that at this stage, malicious clients may tamper with their similarity scores in an attempt to prevent detection of their compromised models. Since a benign client will honestly and accurately decrypt and select trustworthy clients group via threshold-based filter, and hence their results should be consistent. Therefore, A uses a majority voting strategy to acquire the final client score list, i.e., the voted s s s (t) ( 6 ⃝). Next, A normalizes s s s (t) and generates the fusion weight ( 7 ⃝). Here, DDFed employs FedAvg's approach by weighting the aggregation according to dataset size proportions in current training round ( 8 ⃝) . Finally, each client C i receives the aggregated global model[[W W W (t)G ]], decrypts it, and initiates the (t + 1)-th round of DDFed training ( 9⃝).Private and Robust Malicious Model Detection. As observed in [38] , the distribution of local data labels can be more effectively represented in the weights of the last layer than in other layers. Consequently, DDFed employs a similar approach to enhance the efficiency of detecting anomalies, as it requires performing similarity computation on encrypted model updates. Given that FHE supports only basic mathematical operations, and the similarity-based anomaly model detection mechanism needs complex operations like division (as shown in equation 2), comparison and sorting operations, DDFed breaks it down into two stages: secure similarity computation and feedback-driven collaborative selection. In the rest of the paper and during our experimental evaluation, we adhere to the layer section settings described in [38] . However, DDFed can be easily extended to support strategies for detecting malicious models using full layers. Additional experiments are detailed in Appendix A.2.4 to demonstrate the impact of layer sections on the DDFed framework.Secure Similarity Computation. To circumvent division operations, DDFed necessitates that all clients pre-process their inputs for normalization and shifts the task of comparing similarity scores to the client side. This is because clients possess the FHE private key, allowing them to obtain the similarity score in plaintext. Formally, we have the following:EQUATIONwhere each client C i prepares theW W W (t) i ∥W W W (t) i ∥2andW W W (t-1) G ∥W W W (t-1) G ∥2in advance, and then encrypts them using FHE encryption algorithm. Next, the aggregation server S verifies received [[W W W (t-1) G ∥W W W (t-1) G ∥2]] and perturbs local inputs and conducts secure inner-product computation as follows:EQUATIONMotivation of Similarity Score Perturbation. DDFed aims to simultaneously address privacy and poisoning risks. This means it not only considers model poisoning attacks but also prevents adversarial clients from inferring private information from other benign clients by exploiting decrypted similarity scores and previous global models. To mitigate this privacy risk, DDFed improves secure innerproduct computation by introducing perturbations into each normalized and encrypted model update. Specifically, DDFed uses (ε, δ)-differential privacy with a Gaussian mechanism as its method of perturbation,∆ (t) = N (0, σ 2 ), σ = ∆f √ 2 ln(1.25/δ) ε, where (ε, δ) represents the parameters of the DP mechanism and ∆f denotes sensitivity.It's important to note that our perturbation affects only the anomaly detection phase and does not change the encrypted model updates that are to be aggregated. Consequently, the final aggregated model retains its accuracy, just as it would with a standard aggregation mechanism. Furthermore, our experiments indicate that the perturbation noise does not affect the effectiveness of anomaly detection.Even at ε = 0.01, which offers strong privacy protection, DDFed still performs well and delivers good model performance.Feedback-driven Collaborative Selection. As shown in the threat model, DDFed tolerates less than 50% malicious clients, indicating that over half of the clients are benign and will execute the steps honestly and correctly as designed. DDFed employs a feedback-driven collaborative selection approach to filter out potentially malicious models. Specifically, upon receiving the encrypted [[s s s′ (t) ]],each client C i first decrypts to acquire s s s′ (t) iusing the FHE private key SK. Next, each client C i independently decrypts the similarity scores, sorts them, and selects trustworthy clients s s s f f f W W W (t)s s s (t) in the aggregation group, where f(t) j = |Dj | j∈s s s (t) |Dj | .FHE-based Secure Aggregation with Clipping. DDFed's secure aggregation leverages the FHE cryptosystem, specifically the CKKS instance [8] , which excels in arithmetic operations on encrypted real or complex numbers and stands as one of the most efficient methods for computing with encrypted data. Formally, the aggregation server performs secure aggregation as[[W W W (t) G ]] = ⟨[[W W W (t) ]], f f f W W W (t) s s s (t) ⟩. Once receiving the aggregated global model [[W W W (t) G ]], each client C i uses their private key to decrypt it, obtaining the final global model W W W (t)G in plaintext via the FHE decryption algorithm. In contrast to current approaches in private and robust FL, DDFed uniquely enables each benign client to execute a clipping operation before the next training round. This enhancement is designed to counteract more sophisticated model poisoning attacks that conventional similarity-based methods [29, 38] fail to address, as will be shown in the experiments section.Based on the threat model discussed earlier, DDFed prevents an honest-but-curious aggregation server from potentially inferring private information from accessible model updates. Additionally, it also withstands a subset of local clients, compromised by an adversary, to launch model poisoning attacks and attempt to infer private information from other benign clients during the anomaly model detection phase.In terms of privacy risks, DDFed utilizes FHE primitives to ensure cryptographic-level privacy protection. This means the aggregation server processes each operation without any insight into the model update (in the dark), eliminating any chance of inferring private information from local model updates. Furthermore, to counter potential inferences by corrupted clients exploiting decrypted similarity scores, DDFed incorporates a perturbation method where DP noise is added during the secure similarity computation phase. Due to space limitations, the formal DP-enhanced perturbation analysis is provided solely in Appendix A.3.Regarding the risk of poisoning attacks, DDFed adopts similarity-based anomaly detection technologies with additional optimizations such as perturbation-based similarity computation and postaggregation clipping. These enhancements bolster the robustness of its aggregation mechanism. Our experiments demonstrate that DDFed effectively resists a range of continuous poisoning attacks, including IPM, SCALING, and ALIE attacks, which will be elaborated in Section 4. 
experimental setup: 
 Datasets and Implementation. We assessed our proposed DDFed framework using publicly available benchmark datasets: MNIST [19] , a collection of handwritten digits, and Fashion-MNIST (FMNIST) [33] , which includes images of various clothing items, offering a more challenging and diverse dataset for federated learning tasks. We create non-iid partitions for all datasets based on previous research [38, 43] , using a default q value of 0.5, where a higher q reflects greater degrees of non-iid. We assess the framework's performance using a nine-layer CNN model with 225k parameters, secured by the FHE cryptosystem in each training round. This secure aggregation is implemented through TenSEAL library [4] . The experimental DDFed is available on the GitHub repository.Baselines and Default Setting. We compare our proposed method DDFed with well-known FL fusion algorithms and robust aggregation methods, including Krum [5] , Cos Defense [38] , and median/mean-based approaches like Median, Clipping Median, and Trimmed Mean strategies [40] .We exclude baselines such as FLTrust [7] or RFFL [37] because they require server-side validation data or are incompatible with client sampling, making them impractical for real-world applications. Additionally, we omit secure robust approaches [9, 20, 23, 43] that depend on complex secure aggregation techniques due to their requirement for additional non-colluding participants, which alters the original structure of the federated learning framework. Note that the core contribution of this paper is not to propose new model poisoning defense approaches, but to enhance existing popular defenses with privacy features-specifically, server-side similarity-based defenses. Therefore, the experiments aim to evaluate how these privacy-preserving features affect the original defense methods, rather than defending against recent attack techniques and strategies as shown in works like [26, 31, 42, 10, 25] .To assess defense performance, we evaluated the proposed work against popular model poisoning attacks: Inner Product Manipulation (IPM) attack [34] , scaling attack [1] , and the "a little is enough" (ALIE) attack [3] . Unless otherwise mentioned, we assume a default attacker ratio of 0.3 among all participants as malicious clients. The attacks commence at the 50th round and persist until training ends. The default FL training involves 10 clients randomly chosen from 100 for each communication round. Furthermore, we employ a batch size of 64 with each client conducting local training over three epochs per round using an SGD optimizer with a momentum of 0.9 and a learning rate of 0.01. Our DDFed implementation's default epsilon (ε) value is set to 0.01 unless specified differently.Performance of Defense Effectiveness under Various Attacks. Figure 2 demonstrates the effectiveness of our DDFed method compared to baseline methods in countering three prevalent model poisoning attacks, with an attacker ratio set at 0. performance. Note that DDFed remains robust even when attackers target the system from the start of training. Due to space constraints, we present the defense effectiveness against various cold-start attacks in Appendix A.2.3. In summary, our DDFed method achieves the best comprehensive defense performance.Impact of Attacker Ratio. To further investigate the impact of attacker ratio in the DDFed framework, we conducted experiments with various attacker ratio settings. It's important to note that DDFed operates under the security assumption that at least half of the participants must be benign (i.e., r attacker < 0.5), therefore, in our experiments, the attacker ratio setting is ranged from 0.1 to 0.4. As shown in Figure 3 , the proportion of attackers among all clients does not significantly affect our proposed DDFed method. This suggests that it can effectively counter three types of model poisoning attacks. Additionally, we observed that under an ALIE attack scenario, our method may require approximately 10-20 training rounds to recover from the continuous attack, depending on the dataset evaluated.Compatibility with Cross-device and Cross-silo FL Scenarios. To explore how the number of clients affects our DDFed framework and to confirm its compatibility with two common federated learning scenarios, i.e., cross-device and cross-silo, we conducted multiple experiments. These experiments had an attacker ratio fixed at 0.3, with client counts varying from 10 to 100. In cross-silo FL, client numbers are typically small, often ranging from a few to several dozen; however, for simulating the cross-device FL scenario in our study, we used 100 clients due to their generally larger population. As illustrated in Figure 4 , our DDFed framework effectively defends against all three attacks across various client number settings. This suggests that the performance of DDFed is not significantly affected by the number of clients, indicating its suitability for both cross-silo and cross-device FL scenarios. Furthermore, a higher number of client settings may result in relatively large fluctuations during training rounds immediately following the attack; however, the model training ultimately converges steadily, unaffected by the continuous attack. To the best of our knowledge, DDFed offers a dual defense strategy that simultaneously boosts privacy protection and fights against poisoning attacks in FL, without altering the existing FL framework's architecture. DDFed utilizes FHE for top-notch privacy, enabling the aggregation server to perform similarity calculations and aggregation without directly accessing model updates. Additionally, DDFed introduces perturbation techniques to block attempts by malicious clients to infer information from similarity scores. It further employs similarity-based anomaly detection, enhanced with strategies like perturbation and post-aggregation clipping, to protect against various types of poisoning attacks. However, DDFed has not fully explored two related questions: how can we relax the attacker ratio restriction (i.e., r ATTACK < 0.5) while still ensuring effective dual defense? And how can we adapt DDFed to more complex FL scenarios, such as dropout and dynamic participant groups? We leave these questions open for future research. Currently, DDFed only enhances existing popular defenses, such as similarity-based strategies with privacy features. Extending DDFed to support other or more recent defense strategies remains an open question. 
conclusion: 
 To tackle the dual challenges of privacy risks and model poisoning in federated learning, we introduce DDFed, a comprehensive approach that strengthens privacy protections and counters model poisoning attacks. DDFed enhances privacy by using an FHE-based secure aggregation mechanism and addresses encrypted poisoned model detection through an innovative secure similarity-based anomaly filtering method. This method includes secure similarity computation with perturbation and feedbackdriven selection process to distinguish safe model updates from potentially harmful ones. Our approach has been rigorously tested against well-known attacks on diverse datasets, demonstrating its effectiveness. We believe our work sets a solid foundation for future advancements in secure and robust federated learning.  
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: The paper poses no such risks. Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: This paper have state which version of the asset is used. Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA] Justification: Paper does not release new assets. Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: (t) i ← TRAIN(W W W (t-1) G ); 11 Ci encrypts local model [[W W W (t) i , W W W L,(t) i ]] ← FHE.ENCPK([[W W W (t-1) G ]]); 12 Ci sends out [[W W W (t) i , W W W L,(t) i ]] to A; 13 A waits and collects {[[W W W (t) i , W W W L,(t) i ]]} i∈[1,...,m] ; 14 A retrieves [[W W W L,(t-1) G]] from previous round and prepares perturbation ∆ (t) ; A generates fusion weights f f f w s s s (t) using s s s (t) ;15 A performs [[s s s ′ (t) ]] ← {⟨[[W W W L,(t) i ]] + ∆ (t) , [[W W W L,(t-1) G ]]⟩} i∈[1,...,20 A conducts secure aggregation [[W W W (t) G ]] ← ⟨[[W W W (t) ]], f f f W W W (t) s s s (t) ⟩ and sends [[W W W (t) G ]] to {C1, ..., Cm};The DDFed algorithm is outlined in Algorithm Once all encrypted model updates are collected from the clients, the aggregation server begins secure similarity computations using the abstracted last layer of these updates. It introduces differential privacy by adding perturbation noise and sends them back to each client for collaborative decryption and selection of benign clients.Following this, based on majority votes, the aggregation server determines final aggregation groups and calculates fusion weights. Finally, it securely aggregates these with the fusion weights to produce an encrypted global model and concludes that round of federated learning (FL) training.Figure 6 presents the experimental findings on how different ε values affect perturbations during the secure similarity computation phase, with experiments focusing on an IPM attack scenario and involving 100 clients. These tests were carried out using the MNIST and Fashion-MNIST (FMNIST) datasets. For both datasets, we explored a range of epsilon values from 0.01 to 0.1, noting that lower epsilon values indicate enhanced privacy through increased noise addition. Initially, all configurations demonstrated high accuracy levels; however, performance fluctuations became evident following the attack. Specifically, the MNIST dataset exhibited a notable decrease in accuracy at certain epsilon settings, while the FMNIST dataset showed more moderate variations in performance. Ultimately, both datasets achieved relatively stable model accuracy. Determining the optimal ε setting is task-dependent and remains an area for future investigation. The primary purpose that we initiated the attack at round 50 is to demonstrate the effectiveness of defense mechanisms and clearly show the comparative effects of different defense methods before and after an attack. This setup can also illustrate how various defensive measures impact training convergence and model quality, even without attacks.DDFed is resilient to poisoning attacks from the beginning of training. Our design is not constrained by the attack's initiation round. Supplementary experimental results as reported in Table 4 on the FMNIST dataset with 100 clients in a non-iid setting support this claim.In the main body of the paper, we use only the last layer for similarity computation because our primary goal is to integrate privacy-preserving functionality into existing poisoning defense strategies rather than optimizing these mechanisms. Our exploration shows that similarity-based methods and their variants provide comprehensive defense effectiveness, robust against various threat scenarios such as server reliance on validation data, types of model poisoning attacks, and the number of compromised clients. Therefore, we selected a typical similarity-based defense strategy (Cosine Defense) as a starting point to enhance privacy-preserving features. Our approach can easily extend to other similarity-based detection variants using full layers for secure similarity computation. As shown in Table 5 , we conducted additional experiments with full-layer secure similarity computation on a larger dataset (CIFAR10) under various attacks. Differential privacy is a mathematical framework designed to provide privacy guarantees for individuals in a dataset. The standard definition of differential privacy is as follows:A randomized algorithm M is said to be (ε, δ)-differentially private if, for any two adjacent datasets D and D ′ (i.e., datasets differing by only one element), and for any subset of outputs S ⊆ Range(M), the following inequality holds:EQUATION)where ε is the privacy budget parameter, which controls the trade-off between privacy and utility. A smaller ε indicates stronger privacy. δ (delta) is a small probability that accounts for the possibility of the privacy guarantee being violated.The Gaussian mechanism is a specific method to achieve differential privacy by adding Gaussian noise to the output of a function. The definition of the Gaussian mechanism is as follows:Given a function f and any two adjacent datasets D and D ′ , the sensitivity of f is defined as:EQUATIONThe Gaussian mechanism adds noise drawn from a Gaussian distribution with mean 0 and standard deviation σ, where σ is determined by:EQUATIONThus, the Gaussian mechanism is defined as:EQUATIONwhere N (0, σ 2 ) denotes a Gaussian distribution with mean 0 and variance σ 2 . By adding Gaussian noise in this manner, the Gaussian mechanism ensures that the output satisfies (ε, δ)-differential privacy.The DDFed framework aims to enhance privacy protection and mitigate poisoning attacks within federated learning systems by integrating FHE and a similarity-based anomaly detection system. To further bolster privacy, DDFed incorporates DP during the similarity score computation process. This section provides a theoretical analysis of the differential privacy levels maintained by each participant in the DDFed framework, specifically focusing on clients during similarity score computation and feedback stages, as well as the aggregation server during model aggregation and similarity score processing.In the similarity score computation phase, each client normalizes its local model updates before submitting them. To ensure DP, Gaussian noise is added to these normalized updates. By adding Gaussian noise, each client's similarity score computation adheres to (ε, δ)-differential privacy, ensuring that the privacy of the client's data is preserved even in the presence of adversaries.During the feedback phase, clients decrypt the similarity scores and submit their results. Since these scores have already been DP due to the added Gaussian noise, the privacy level remains at (ε, δ)-differential privacy. This ensures that even when clients provide feedback, their privacy is not compromised.In the model aggregation phase, the aggregation server receives encrypted model updates from clients.While FHE inherently provides a high level of security for these crucial parameters, the aggregation server further ensures privacy by applying DP during the similarity score calculation. The server aggregates the encrypted updates without accessing the plaintext data, thereby maintaining the privacy of the individual model updates.For the similarity score processing phase, the aggregation server handles the scores submitted by clients, which have already been protected using differential privacy. Consequently, the server does not need to apply additional privacy mechanisms during this phase. The DP guarantees provided during the similarity score computation phase by clients are sufficient to protect the overall process.Based on the analysis, the privacy levels for each client in DDFed framework can be summarized as follows. During the similarity score computation phase, clients achieve (ε, δ)-differential privacy by adding Gaussian noise to their normalized model updates. During the feedback phase, clients maintain (ε, δ)-differential privacy as the similarity scores they submit have already been differential private.By thoughtfully designing and selecting parameters, the DDFed framework can provide robust privacy protection and maintain high model performance. The use of FHE for critical parameters and differential privacy for similarity scores ensures a balanced and comprehensive approach to privacy protection, addressing both security and utility needs effectively.Generally, the reader may concern about whether [[x]] + ∆ equals x + ∆, where x is under FHE protection. However, this depends on the precision of the employed FHE schemes. Proving such a statement theoretically may require delving into the specific construction algorithm of the FHE scheme, which is beyond the scope of machine learning-oriented venues. Justification: There is no societal impact of the work performed. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 


Decision: Thank you for your submission. The reviewers agree that the paper provides an interesting and novel idea. The problem definition is important, and the combination of FHE with differential privacy seems to be moderately practical, at least for smaller models (this can perhaps be justified because small models are the only kinds of models that can be trained in client-level federated learning deployments on mobile devices). At the same time, the reviewers pointed out some concerns. I believe that if the authors can address the first one in particular, the paper would be significantly strengthened. 

(1) The evaluation seems to be limited in terms of the attacks and defenses evaluated. In the cat-and-mouse game of backdoor defenses, it is important (and common) to compare against a wide swath of both attacks and defenses, particularly in the absence of theoretical robustness guarantees. The authors have added an adaptive attack during the rebuttal, which is a good start, but there is a wide literature of attacks and I would like to see a more thorough review, and evaluation against relevant attacks. Additionally, they have not addressed the comment of reviewer zJWB to add additional defenses. In particular, I am concerned that this defense algorithm may not compare favorably to more recent defenses than the baselines evaluated in this work. For example, in addition to the methods listed by reviewer zJWB (which are indeed important reference points), how does this defense compare to methods such as [1-3]? (This is just a short list, please conduct a more thorough review). What do you lose in terms of backdoor robustness by adding your secure aggregation technique? This field is moving very fast, and a comprehensive evaluation requires a more thorough comparison to related work. 
[1] Pillutla et al, Robust aggregation for federated learning
[2] Wang  et al, Towards a defense against federated backdoor attacks under continuous training
[3] Zhang et al, Flip: A provable defense framework for backdoor mitigation in federated learning

Note: even if DDFed performs worse than these other defenses in terms of backdoor robustness, it's not a deal-breaker in my view, because the scheme is providing other benefits, namely secure aggregation. However, I do think it's important to include this empirical evaluation for completeness. 

(2) There were some concerns about the threat model, and the assumption that clients do not collude with the server. While I agree that this is difficult to ensure in practice, it is also true that many papers in the literature make this assumption. Hence, this concern did not factor into my evaluation.

Reviews: review: paper_summary: paper_summary: This paper introduces Dual Defense Federated Learning (DDFed), a framework designed to tackle two major challenges in federated learning: privacy breaches and poisoning attacks. By integrating fully homomorphic encryption, DDFed securely aggregates model updates, thereby enhancing privacy protection without the need for impractical non-colluding two-server setups. Furthermore, it incorporates a two-phase anomaly detection mechanism for encrypted model updates, which includes secure similarity computation and feedback-driven collaborative selection.
strength_weakness: strength_weakness: The novelty of the work primarily lies in the combination of techniques rather than theoretic novelty in individual components. The multiple steps and interactions between clients and the server could introduce significant computational and communication overhead. This raises questions about the framework’s scalability in large-scale FL settings with thousands of clients. Furthermore, the paper does not provide a detailed comparison of the computational costs and latency introduced by FHE and the anomaly detection mechanism compared to baseline methods.
questions: questions: 1. What is the time complexity on server-side and client-side separately?
2. How does the proposed method scale with the number of clients?
3. What is the threshold for the proportion of compromised clients that DDFed can effectively handle?
limitations: limitations: Yes
paper_summary: paper_summary: This paper introduces a Dual Defense Federated learning (DDFed) framework. DDFed simultaneously boosts privacy protection and mitigates poisoning attacks leveraging fully homomorphic encryption (FHE). The experiments with publicly accessible datasets demonstrate DDFed’s effectiveness in safeguarding model privacy and robustly defending against model poisoning threats.
strength_weakness: strength_weakness: Major concerns:

- The methods fails to address the non-colluding problem. As the clients hold the same private key, the server would easily obtain the private gradient if it collaborate with one of the client.

- The robust aggregation protocol relies on the cosine similarity with the global model from previous round, raising two problem: 1) why do the author select this method as the aggregation rule? Could the framework extend to other aggregation protocols? Please include discussion for the choice and comparison of defense approaches.  2) The protocol relies on the assumption that the inital round consists of all benign clients. How to ensure the condition?

Minor concerns:

- The experiments uses two simple dataset, MNIST and Fashion-MNIST. Please consider more complex dataset and models.

- Table 1 shows that the training time takes only 2 more seconds than FedAvg. The result is quite conter-intuitive as the FHE takes significant time for aggregation and there's no improvement on the FHE algorithm itself. How many clients are involved in the evaluation? Please consider more clients (>100) and larger model size.
questions: questions: Refer to weakness.
limitations: limitations: The authors discuss the limitation.
paper_summary: paper_summary: This paper proposes a novel Byzantine-robust and differentially private federated learning (FL) framework, named as Dual Defense (DDFed). To guarantee the privacy, DDFed utilizes a secure similarity computation based on fully homomorphic encryption, without leaking client’s privacy to either the server or any potential compromised clients. To address the Byzantine attacks, DDFed uses a novel feedback-driven collaborative selection method to filter out malicious clients by majority of votes.
strength_weakness: strength_weakness: 1. DDFed only considers the last layer in the detection process. Although this approach makes DDFed more efficient compared to other methods, it is vulnerable to attackers who can bypass DDFed by injecting poisoned weights into other hidden layers.
2. The authors add differential privacy (DP) noise to the encrypted parameters in a manner commonly used for non-encrypted parameters. They should provide either theoretical proof or experimental evidence to verify that the DP noise remains effective when the parameters are decrypted.
3. The authors should extend their experiments. For instance, they should use more complex datasets (e.g., CIFAR-10) and larger models (e.g., ResNet). Additionally, they should compare DDFed with more advanced attacks [1] and defenses (e.g., Multi-Krum and Flame [2]). Furthermore, the authors should demonstrate the robustness of DDFed even when the attacker targets the model from the beginning of the FL training.
4. The authors should better discuss DDFed with previous methods that do not use fully homomorphic encryption (FHE) and sophisticated differential privacy (DP) noise utilization [3-5], further illustrating the strengths of DDFed.

[1] Fang, Minghong, et al. "Local model poisoning attacks to {Byzantine-Robust} federated learning." 29th USENIX security symposium (USENIX Security 20). 2020.

[2] Nguyen, Thien Duc, et al. "{FLAME}: Taming backdoors in federated learning." 31st USENIX Security Symposium (USENIX Security 22). 2022.

[3] Miao, Yinbin, et al. "Privacy-preserving Byzantine-robust federated learning via blockchain systems." IEEE Transactions on Information Forensics and Security 17 (2022): 2848-2861.

[4] Guo, Hanxi, et al. "SIREN+: Robust Federated Learning with Proactive Alarming and Differential Privacy." IEEE Transactions on Dependable and Secure Computing (2024).

[5] Liu, Xiaoyuan, et al. "Privacy-enhanced federated learning against poisoning adversaries." IEEE Transactions on Information Forensics and Security 16 (2021): 4574-4588.
questions: questions: 1. Why the authors choose to start the attack after round 50? Will DDFed still be robust when the attackers attack the global model from the beginning of the training?
2. Why is only the last layer considered in the secure similarity computation? Attackers could easily bypass such detection by injecting poisoned weights into other hidden layers.
3. Will FHE influences the privacy protection of DP noise? Please also refer to the second Cons in the Weakness section.
4. Is the cosine similarity between the weights a good indicator? The attacker could add restrictions to the poisoned parameters to influence the global model without flip the cosine similarity.
limitations: limitations: The authors should further clarify the limitations of the proposed method. Specifically, they should address the fact that it only considers the last layer in the detection process and adds DP noise to encrypted data in the same manner as plain data. If the author could well address my concerns from theses two perspectives, I will give a higher score.
paper_summary: paper_summary: This paper introduces DDFed, which combines FHE (F8lly Homomorphic Encryption) and similarity computation/collaborative selection, to achieve privacy protection and attack mitigation, respectively.  The idea of this paper is to leverage FHE within Federated Learning (FL) as a strategy for protecting privacy while addressing the problems it introduces in mitigating attacks.
strength_weakness: strength_weakness: I am somewhat concerned about the practicality of the method described in the paper, as well as the practicality of some settings, See Questions.
questions: questions: I appreciate the author's dual focus and have also reviewed the code provided.
However, I require further clarification before considering the paper for acceptance.
First, in line 186, to circumvent division operations, DDFed moves the comparing tasks to the client side, but why should this task be calculated on the client side if some of the clients are malicious?

Second, the paper demonstrates that DDFed incurs higher training costs compared to other methods. It reports an average training round duration exceeding 12 seconds on small datasets like MNIST and FMNIST, which are only 28x28 in size. Moreover, while FHE typically involves significant communication overhead, the paper lacks detailed experimental results or a discussion on this aspect. Given that communication costs can rise exponentially with increasing model parameters, the practicality of this method is questionable.
limitations: limitations: See questions.

Comments: title: Thanks for the clarification. I've raised my score to 6.
Thanks for the reply.
Thanks for the clarification
comment: Thank you for your clarification. Please ensure that you include the experimental results and references discussed during the rebuttal in the next version of the main text, as they are crucial to the soundness of your paper. My concerns have been largely addressed, so I further raise my score to 6. Good luck!
Thank you for your additional comments.

We acknowledge that SecAgg (Google's original work), which uses pairing masking with secret sharing technologies, can address client-server collusion. However, this is just one research direction in the field of secure aggregation within FL. Other approaches such as HE, functional encryption, additive secret sharing, and conventional MPC are also significant and valuable contributions to this field. As we have illustrated above, these methods remain practical. We want to emphasize that not all non-SecAgg works are impractical or unacceptable. We kindly ask the reviewer to consider this point.

The cosine scores do not necessarily improve with higher values. In our released open-source code, we have selected the threshold using an adaptive approach.

Regarding convergence proof, we agree with the reviewer that our similarity-based approach differs slightly from ShildFL. Due to limited response time, we may not provide a full theoretical proof at this moment; however, our experiments on three datasets demonstrate experimental convergence guarantees. A detailed proof will be included in the final version.

Concerning DP concerns raised by another reviewer: Our experimental results show it does not affect benign client selection.

Finally, thank you again for your further comments—they have indeed helped us improve the quality of our paper. We hope our responses address these concerns and positively influence the final decision in this round of discussions.
Thank your for the further comments.

PSNR is calculated using the Mean Squared Error (MSE) between the original and reconstructed images. We just realized that we made a mistakes in the above reported table due to limited response time and our negligent. DP-based version actually has a lower PSNR than the non-DP version.

Regarding the mentioned three comparison solution, 
- Scheme 1 is actually the Cosine Defense solution. Without HE, it does not provide privacy-preserving functionality, resulting in any inversion attack works well as they attack on native FL. It is reported above, where PSNR is 28.5.
- Scheme 2 is reported above, where the PSNR is actually 18.2.
- Scheme 3 is our work. As we give a short theoretical analysis above, according to most HE related work, it actually does not impact the computing precision in the context of FL. Due to limited response time, we cannot provide detailed experiments right now. We will include those experimental results in the final version. 

Finally, we appreciate the reviewer's further comments again. We hope our response has addressed these issues and will continue to positively influence the final decision in this round of discussions.
Thanks for your reply. I still have the following comments:

> To our knowledge, most existing papers—may be approximately 99%—do not consider client-server collusion. 

Actually, SecAgg[1][2] algorithm has rigorous security/threshold analysis for the client-server collusion. Since one of the paper's major contribution is to solve the non-colluding problem, it might be insufficient to consider merely the two-server collusion (the more reliable one) while ignoring the client-server collusion (the more vulnerable one). Enhancing your framework to include the collusion threshold between clients and servers would strengthen its impact.

> Second, our work does not actively ensure that clients' local weights are normalized. In fact, non-normalized local weights will deviate more from the global model, making them easier to detect and filter using similarity-based approaches.

This argument sounds inaccurate. If $\|w_i\|$ is greater than 1, then un-normalizing in fact makes $cos(\alpha_i)$ greater. That it, skipping the normalization would put the adversaries at an advantage.

> The convergence guarantee has already been proven in previous similarity-based solutions.

The similarity-based methods with convergence guarantee differs from your methods. For example, in ShieldFL, they use the similarity between current gradient and previous gradients to filter outliers (instead of the cosine simlarity between local and global weights). Your aggregation method follows that proposed by [3], but they didn't provide convergence guarantee.

> Differential Privacy (DP) perturbation is only applied to secure similarity computation and does not affect the secure aggregation stage, hence it does not impact convergence....We will include an analysis of the formal convergence guarantee in the appendix of the final version.

I understand that DP perturbation provides strong privacy guarantee, and that the DP noises are not added to the aggregation results. However, it would impact the selection of benign clients. I look forward the analysis of the formal convergence guarantee.

Anyway, I appreciate your feedback and will consider that in the next discussion stage.

[1] Bonawitz, Keith, et al. "Practical secure aggregation for privacy-preserving machine learning." proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 2017.

[2] Bell, James Henry, et al. "Secure single-server aggregation with (poly) logarithmic overhead." Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security. 2020.

[3] Yaldiz, Duygu Nur, Tuo Zhang, and Salman Avestimehr. "Secure Federated Learning against Model Poisoning Attacks via Client Filtering." ICLR 2023 Workshop on Backdoor Attacks and Defenses in Machine Learning.
Thank you for the response, but there's still a point of confusion in your results: How did you compute the PSNR? Typically, a higher PSNR indicates a more successful inversion, so it's surprising that the DP-based version has a much higher PSNR than the non-DP version.

The reason I requested the inversion attack results is that it could serve as complementary evidence to support your DP+HE scheme, alongside the theoretical proof. Specifically, I expected a comparison of three schemes:
1. No DP, No HE (baseline)
2. With DP, No HE (previous DP-based method)
3. With DP, With HE, and DP noise on encrypted data (your method)

If your method's results show comparable defense effectiveness to the traditional DP-based method, it would strongly support the validity of adding DP noise to encrypted data.
**Resp to collusion concern**:
First, we argue that the assumption of client-server collusion is uncommon and does not affect practicality in this field for the following reasons:
1. While assuming no client-server collusion might introduce vulnerabilities, it is an extreme case in real-world applications. As noted by the reviewer, if a server can actively attack the FL system (e.g., by introducing fake clients), it could infer more private information, but easily bypass server-side defenses against model poisoning attacks due to its ability to break aggregation rules. Typically, the threat model assumption involves a trade-off and does not aim to cover all extreme cases.
2. Assuming no client-server collusion remains practical for real-world applications. FL is typically classified into cross-silo FL and cross-device FL from an application perspective. In cross-silo scenarios, servers are usually authoritative organizations within or between industries. In cross-device scenarios, cloud service providers often act as servers. In these cases, servers are generally honest-but-curious rather than active attackers. We acknowledge that the no-client-server-collusion assumption may not cover all scenarios but believe it is still reasonable and practical for most applications since there is no one-size-fits-all solution.
3. To our knowledge, most existing papers—may be approximately 99%—do not consider client-server collusion. For instance, one of the referenced papers [1,2] does not assume such collusion either; paper [2], which shares similar assumptions with ours, explicitly states that it does not address client-server collusion.
In summary, our work aligns with the majority of existing solutions and we believe these solutions have their respective application value in specific scenarios.

Second, our work does not actively ensure that clients' local weights are normalized. In fact, non-normalized local weights will deviate more from the global model, making them easier to detect and filter using similarity-based approaches. 
In DDFed, a benign client will correctly follow the protocol and generate normalized local weights accurately. If the client is adversarial, it may or may not perform normalization; however, this does not provide an advantage in circumventing the defense method.

**Resp to concerns of choosing similarity-based detection method**:
We did not include the formal convergence guarantee in the current version for two reasons:
1. The convergence guarantee has already been proven in previous similarity-based solutions. Our work follows this method, enhancing it with privacy-preserving functionality using our proposed solution, without changing the aggregation rule. Therefore, covering this is not a contribution of our paper.
2. Differential Privacy (DP) perturbation is only applied to secure similarity computation and does not affect the secure aggregation stage, hence it does not impact convergence.
Additionally, as noted by the reviewer, our work relates to ShieldFL. We use a similar similarity-based detection approach but employ different privacy-enhancing technologies. Thus, the formal convergence is akin to that in ShieldFL. We will include an analysis of the formal convergence guarantee in the appendix of the final version.


Regarding the concern about DP-based perturbation, we may not have correctly implemented the reader access control policy in our previous response. Actually, we initially addressed reviewer zJWB's concerns and adjusted the reader policy so you can track all other responses. For your convenience, here is our response to this concern raised by reviewer zJWB.

In Appendix Section 3, we provided an overview of privacy analysis for differential privacy-enhanced FHE-based secure similarity computation but did not include a formal proof. Generally, we believe the reviewer's concern is whether $[[x]] + \Delta$ equals $x + \Delta$, where $x$ is under FHE protection. This depends on the precision of the employed FHE schemes. Proving this theoretically would require delving into the specific construction algorithm of the FHE scheme, which is beyond our scope.
This paper uses CKKS FHE constructions that natively support high-precision secure computation on floating-point numbers. Therefore, adding DP noise to encrypted similarity results does not degrade performance.
As suggested by reviewer zJWB, we conducted supplementary experiments on CIFAR10 using a simulated DDFed setup where DP noise was added to non-encrypted parameters. The reported results support this claim.

|Approaches|Attacks Type|Acc|
|--|--|--|
|DDFed (Simulated)|IPM|70.21|
||ALIE|64.3|
||SCALINE|69.82|
|DDFed(Our work)|IPM|70.31|
||ALIE|64.62|
||SCALINE|69.6|

We invite the reviewer to continue following our response thread with reviewer zJWB for further updates.

Again, we appreciate the reviewer's suggestions. We hope our response addresses these concerns and influences the final decision positively.
**Resp to advance attacks**: 
We thoroughly examined the advanced attack suggested by the reviewer, as referenced in [1] above. That paper only discusses attacks on Krum and its variant, Trimmed Mean, and Median—all of which are not conventional similarity-based approaches. Therefore, we adopted a similar attack strategy and implemented a dynamic attack: conducting n attack iterations after n non-attack iterations to collect "before-attack" global models. We then averaged the crafted models (using the most effective attack) with the "before-attack" global models for subsequent periods of attack iterations. The results on CIFAR10 show that DDFed remains effective under such dynamic attacks.

| Approaches       | Attacks Type | Acc   |
| ---------------- | ------------ | ----- |
| FedAvg           | NO ATTACK    | 70.16 |
| DDFed (Our work) | IPM          | 70.31 |
|                  | ALIE         | 64.62 |
|                  | SCALINE      | 69.6  |
|                  | Dynamic ALIE | 64.52 |

**Resp to DP protection and evaluation**:
We would like to emphasize that the DP-based perturbation is used to prevent potential privacy leakage from similarity scores. Specifically, it prevents adversaries from inferring private information about benign clients by exploiting decrypted similarity scores and previous global models, rather than directly targeting a specific client's model update as in the previous DGL attack.

As shown in formula 4 of the paper, suppose a client can infer private information; they must (i) solve a multivariate linear equation problem with only one equation successfully—given $C$, $B$, and $C=<A,B>$, find all parameters of $A$—and (ii) break the DP-based perturbation to find $W^{t-1}_i$. To our knowledge, we do not believe an adversary can accomplish step (i) because each round's $A$ is not fixed. If the similarity computation does not use full layers, this challenge becomes even greater.

However, let's consider a harsh condition: during convergence phases when model parameters tend to stabilize and A is approximately fixed while using full-layer models for similarity computing. We then conducted a simple inversion attack (e.g., DLG attack) as suggested by the reviewer. We assessed the PSNR results of both DP-based perturbation and non-DP perturbation on the MNIST dataset. The results below demonstrate the effectiveness of the DP-based similarity score perturbation.

| Approaches         | Attacks    | PSNR (avg of last 10 round) |
| ------------------ | ---------- | --------------------------- |
| DP Perturbation    | DLG Attack | 28.5                        |
| No DP Perturbation | DLG Attack | 18.2                        |

**By the way**, we didn't set up special access control for readers; we just used the default settings when inputting the last round's response. We also haven't provided a general response yet for that round. As suggested, we've revised the scope of previous responses and added a general response to address the current status of the rebuttal.

Finally, we appreciate the reviewer's concerns and suggestions again. We hope our response has addressed these issues and will continue to positively influence the final decision in this round of discussions.
Still, thank you for the valuable comments given earlier.
Thanks for the response and additional experiments. However, some of my concerns remains.

**Collusion between clients and server**: Firstly, assuming no collusion between clients and servers poses great vulnerabilities in real-life, particularly with expanding client pools. Servers can easily introduce fake clients to undermine this assumption. Recent studies [1][2] have successfully addressed privacy concerns and ensured robust aggregation, eliminating the need of this assumption. Secondly, the paper mentioned by the authors [3] relies on two servers to validate the gradient normalization. The authors remove the two-server setting at the cost of simply skipping the normalization step. How to ensure that the clients' local weights are normalized?

**Choise of similarity-based detection methods**: I understand that the solution does not rely on the assumption that the initial round of training must include only benign clients. However, the aggregation protocol the authors choose does not have formal convergence guarantee. As is mentioned by another reviewer, I expect the theoretical proof under the case with and without DP noise.

[1] Lycklama, H., Burkhalter, L., Viand, A., Küchler, N., & Hithnawi, A. (2023, May). Rofl: Robustness of secure federated learning. In 2023 IEEE Symposium on Security and Privacy (SP) (pp. 453-476). IEEE.

[2] So, J., Güler, B., & Avestimehr, A. S. (2020). Byzantine-resilient secure federated learning. IEEE Journal on Selected Areas in Communications, 39(7), 2168-2181.

[3] Ma, Z., Ma, J., Miao, Y., Li, Y., & Deng, R. H. (2022). ShieldFL: Mitigating model poisoning attacks in privacy-preserving federated learning. IEEE Transactions on Information Forensics and Security, 17, 1639-1654.
Thanks for the clarification. Although my questions have been answered, my opinion regarding the contribution has not changed. Therefore, my decision remains unchanged.
Your rebuttal has addressed my first concern regarding the last layer setting and has partially resolved my concern about the DP noise addition, so I will raise my score to 5.

However, as mentioned in my review, I still recommend that the authors evaluate the robustness of DDFed against more advanced attacks (e.g., adaptive attacks) and assess the DP performance using a simple inversion attack (if the theoretical proof is time-consuming). If the authors can further address these concerns, I will consider raising my score again.

P.S. It seems that the authors did not correctly set the readers of the rebuttal, preventing me from accessing the general response as well as the rebuttal to other reviewers.
We appreciate the reviewer's comments and the raised concerns.

**Resp to Q1:**
DDFed moves the comparison task to the client side because no FHE scheme efficiently supports both comparison operations and floating-point numerical computation over encrypted model updates. FHE schemes are generally classified into three categories: bit-wise approaches like FHEW and TFHE, word-wise approaches like BGV and BFV, and CKKS for efficient floating-point computations. There is currently no perfect FHE solution that efficiently handles all types of computational operations. Each category is more efficient for specific applications, leading to trade-offs when employing a particular FHE scheme.

In this paper, we use FHE to protect FL local models with millions of floating-point parameters, prioritizing computational efficiency. We adopt the CKKS scheme as our underlying FHE solution and delegate comparison tasks to the client side since CKKS does not efficiently support comparison operations.

We encountered issues with potential malicious clients in our initial DDFed design, as noted by the reviewer. This prompted us to propose a feedback-driven collaborative selection mechanism. Similar to blockchain consensus technology, this mechanism can tolerate up to 50% of clients being malicious. This aligns with our threat model assumption that DDFed can handle adversaries with the ratio lower than 0.5.  


**Resp to Q2:**
The introduction of FHE for privacy-preserving federated learning often raises computational concerns due to secure computation over encrypted models and communication overhead from the larger size of encrypted model updates, as noted by the reviewer.

From a computational perspective, the cost of FHE for privacy enhancement in FL depends on the size of the trained model, not the dataset. In our experiment, evaluating models on the MNIST and FMNIST datasets involved about 0.23 million parameters. Additionally, as shown in our released prototype implementation, we run each training round sequentially with each client rather than in parallel. We believe that if DDFed is applied to a real FL scenario where all local training and operations are conducted in parallel, the total training time will be lower than reported in Table 1.

The primary goal of this paper is to validate the effectiveness of the proposed dual defense approaches. As indicated in the released open-source code, we implemented a simulated FL framework rather than an actual FL system at this prototype stage. Building an FL system in a distributed real-world network environment slightly deviates from the core focus of this paper. Therefore, we did not evaluate communication overhead by measuring network latency caused by transferring encrypted model updates instead of plaintext model updates.

Formally, depending on the security parameter settings, the communication overhead is linearly related to the model size, rather than an exponential relationship. In our CKKS setting with a security parameter of 128, we use the above-mentioned model with 0.23 million parameters as an example. Our manual measurement shows that the original model size is approximately 0.9MB, while the encrypted model size is about 20.4MB. This indicates that the communication payload overhead is roughly 20 times greater compared to a non-FHE protected solution.

In summary, our work may introduce additional computational and communication costs similar to most existing FHE-based privacy-preserving federated learning solutions. However, this design provides a strong guarantee of privacy preservation. While applying DDFed in a cross-device FL scenario involving thousands of devices might raise practical concerns, we believe it remains feasible for cross-silo FL scenarios where organizations or companies with higher network bandwidth and servers conduct FL training. It is worth noting that the cross-silo FL scenario is an important type of application according to recent surveys on federated learning.

Again, we appreciate the reviewer's comments. We hope our response addresses these concerns and positively influences the final decision with higher rating scores.
We appreciate the reviewer's concerns and suggestions. 

**Resp to major concern 1:** 
Collusion between the server and clients is beyond the scope of our threat model assumptions in Section 3.1. In PPFL, each solution includes a threat model that defines the adversary's capabilities and behavior, following the security research tradition of defining threat boundaries before proposing solutions since no solution can claim absolute security.
Our threat model assumes an honest-but-curious server that does not collude with any clients, consistent with most related work's assumptions. Most current PPFL approaches cannot handle such collusion issues, which is a strong security assumption.
We believe our work represents substantial progress in this field. Our solution eliminates the need for a non-colluding two-server assumption found in most recent related works, making private and robust FL solutions more practical.

**Resp to major concern 2:**
Based on our exploration in model poisoning attacks and defenses, our findings indicate that similarity-based detection methods and their variants provide excellent and comprehensive defense outcomes regarding the server's reliance on validation data, types of model poisoning attacks, number of compromised clients, and attacking round. This is why we have chosen the (cosine) similarity-based poisoning model detection methodology.
Additionally, our approach can easily extend to similarity-based poisoning model detection variants. However, it does not provide a unified support to all other defense methods such as Krum or those relying on the assumption that the server has partial validation datasets. The reviewer's question represents a promising future direction. Regarding comparisons with other defense approaches, we have demonstrated their effectiveness in existing experimental evaluations, such as in Figure 2. We will include discussions of these choices in the final version.

Besides, our solution does not rely on the assumption that the initial round of training must include only benign clients. In fact, our work supports starting attacks from the first training round, meaning compromised clients can be present initially. To substantiate this claim, we conducted additional experiment, comparing our approach with baselines by initiating attacks in the first training round on the FMNIST with 100 clients in a non-iid setting (see reported results below).
The reviewer's misunderstanding likely arises from our experimental setup and results. We began the attack at round 50 to demonstrate the effectiveness of defense mechanisms and to clearly show the comparative effects of different defense methods before and after an attack. This helps illustrate how various defensive measures impact training convergence and model quality, even in the absence of attacks.

|Approaches|Attacks Type|Acc|
|--|--|--|
|FedAvg|IPM|0|
||ALIE|10.1|
||SCALINE|0|
|Krum|IPM|69.05|
||ALIE|73.69|
||SCALINE|69.95|
|Median|IPM| 67.57|
||ALIE| 76.57|
||SCALINE|74.03|
|Clip Median|IPM|61.1|
||ALIE|73.8|
||SCALINE|75.49|
|Trimmed Mean|IPM|0|
||ALIE|43.29|
||SCALINE|0|
|Cosine Defense|IPM|81.87|
|| ALIE| 82.97|
|| SCALINE| 81.11|
|DDFed (Our work)|IPM|83.32|
||ALIE|80.97|
||SCALINE|83.05|

**Resp to minor concern 1:**
Given the limited response time, we conducted an additional experiment to evaluate our approach against related baselines on the CIFAR10 using a ResNet model with 100 clients in a non-iid setting. As shown in the tables below, our proposed method remains effective under various attacks.

|Approaches|Attacks Type|Acc|
|--|--|--|
|FedAvg|NO ATTACK|70.16|
||IPM|0|
||ALIE|10|
|| SCALINE|0|
|Cosine Defense|IPM|68.04|
||ALIE|68.27|
|| SCALINE|69.05|
|DDFed (Our work)|IPM|70.31|
||ALIE|64.62|
||SCALINE|69.6|

**Resp to minor concern 2:**
Using HE for privacy enhancement does not add significant overhead in FL recently, contrary to what most non-cryptography researchers might expect. Despite the inherent challenges of developing HE technology, the minor computational overhead reported in Table 1 is due to the following factors:
1) The scale of HE used is relatively small, with total parameters of models being about 0.23 million.
2) Our design mechanism involves only simple computations over the encrypted model, with just one layer of multiplication depth. It is well-known that multiplication depth significantly impacts efficiency of HE.
As suggested, we conducted additional experiments on the CIFAR10 dataset using a ResNet model (approximately 11.2 million parameters) and 100 clients. The results showed that it takes an extra 974 seconds (16 minutes) compared to the non-protected FedAvg baseline. This increase is reasonable and acceptable given that our proposed dual defense approach offers strong privacy and security guarantees.

We appreciate the reviewer's concerns and suggestions. We hope our response addresses these concerns and influences the final decision positively.
We appreciate the reviewer's concerns and suggestions.

**Resp to Q1:**
The primary purpose that we initiated the attack at round 50 is to demonstrate the effectiveness of defense mechanisms and clearly show the comparative effects of different defense methods before and after an attack. This setup can also illustrate how various defensive measures impact training convergence and model quality, even without attacks.

DDFed is resilient to poisoning attacks from the beginning of training. Our design is not constrained by the attack's initiation round. Supplementary experiments on the FMNIST dataset with 100 clients in a non-iid setting support this claim.

|Approaches|Attacks Type|Acc|
|--|--|--|
|FedAvg|IPM|0|
||ALIE|10.1|
||SCALINE|0|
|Krum|IPM|69.05|
||ALIE|73.69|
||SCALINE|69.95|
|Median|IPM| 67.57|
||ALIE| 76.57|
||SCALINE|74.03|
|Clip Median|IPM|61.1|
||ALIE|73.8|
||SCALINE|75.49|
|Trimmed Mean|IPM|0|
||ALIE|43.29|
||SCALINE|0|
|Cosine Defense|IPM|81.87|
|| ALIE| 82.97|
|| SCALINE| 81.11|
|DDFed (Our work)|IPM|83.32|
||ALIE|80.97|
||SCALINE|83.05|

**Resp to Q2:**
We use only the last layer for similarity computation because our main goal is to add privacy-preserving functionality to existing poisoning defense strategies, more than on optimizing existing defense mechanisms. 
Based on our exploration, similarity-based methods and their variants offer a comprehensive defense effectiveness that is robust to various threat assumption setting such as server reliance on validation data, types of model poisoning attacks, number of compromised clients. Therefore, we chose one typical similarity-based defense strategy (Cosine Defense) as a starting point to enhance privacy-preserving features. Our approach can easily extend to other similarity-based detection variants that use full layers for secure similarity computation. We conducted additional experiments considering full-layer secure similarity computation on a larger dataset (CIFAR10) under various attacks.

|Approaches|Attacks Type|Acc|Time Cost (min) (60 rounds)|
|--|--|--|--|
|FedAvg|NO ATTACK|70.16|46.23|
||IPM|0|46.62|
||ALIE|10|46.89|
||SCALINE|0|46.48|
|DDFed (Last Layer)|IPM|70.3|50.66|
||ALIE|64.62|51.3|
||SCALINE|69.61|51.63|
|DDFed (Full Layers)|IPM|69.84|58.95|
||ALIE|69.73|58.78|
||SCALINE|68.89|59.01|

**Resp to Q3:** 
In the appendix 3, we provided an overview privacy analysis of differential privacy-enhanced FHE-based secure similarity computation but did not include a formal proof. Generally, we believe the reviewer is concerned about whether $[[x]]+ \Delta$ equals $x + \Delta$, where $x$ is under FHE protection. However, this depends on the precision of the employed FHE schemes. Proving such a statement theoretically may require delving into the specific construction algorithm of the FHE scheme, which is beyond the scope of machine learning-oriented venues.

This paper utilizes CKKS constructions, which natively support high-precision secure computation on floating-point numbers. As a result, adding DP noise to encrypted similarity results does not degrade performance. To validate this, we conducted supplementary experiments on CIFAR10 using a simulated DDFed setup where DP noise was added to non-encrypted parameters. The reported results support this claim.

|Approaches|Attacks Type|Acc|
|--|--|--|
|DDFed (Simulated)|IPM|70.21|
||ALIE|64.3|
||SCALINE|69.82|
|DDFed(Our work)|IPM|70.31|
||ALIE|64.62|
||SCALINE|69.6|

**Resp to Q4:** 
As stated in response 2, the primary goal and contribution of this paper are to enhance existing model poisoning defense strategies with privacy preservation functionality, addressing both privacy and model poisoning issues in FL simultaneously. Therefore, we did not focus heavily on indicator selection. 

Our exploration in this field shows that similarity-based model poisoning defense methods and their variants provide comprehensive defense effectiveness when considering threat model assumptions such as server reliance on validation data, types of model poisoning attacks, number of compromised clients, and attacking rounds. While it may not be optimal for a specific threat model assumption, it offers relatively good overall defense quality as demonstrated by related papers. 

This paper focuses more on privacy-preserving functionality than on model poisoning defense strategies. Therefore, we did not conduct the latest attacks [1-2] for comparison, as suggested by the reviewer, because it is outside the focus of this study.

**Supplementary response:**
We also conducted additional experiments to evaluate our work on the CIFAR10 using a ResNet model as shown in ***Resp to Q2*** section. Regarding related work, we have discussed the referenced study [5], but omitted studies [3-4]. These will be included in the final version.

Finally, we hope our response, particularly regarding the last layer setting and DP noise addition, addresses these concerns and positively influences the final decision.
We appreciate the reviewer's positive decision and those raised concerns. The challenge of the paper lies in resolving the dilemma where detecting model poisoning requires plaintext model updates from each client, while privacy protection demands safeguarding these updates.

This is our first attempt to address this dilemma using a homomorphic encryption scheme as the privacy-enhancing technology. Unlike other methods such as secret sharing, pairwise masking, and functional encryption, the native design of the homomorphic encryption-based secure aggregation approach prevents the aggregation server from obtaining both anomaly detection results and aggregated models. Thus, such a design results in introducing an additional interaction step compared to the original federated learning paradigm and incurs extra time costs due to privacy protection. However, this is inevitable. Existing privacy-preserving federated learning solutions typically sacrifice either computational efficiency or communication efficiency—or both—to provide strong privacy and security guarantees.

In our paper, we note that feedback-driven collaborative selection requires an additional round of interaction due to the native security model of homomorphic encryption. In our next research plan, we aim to use advanced cryptographic computation technologies like functional encryption, which have different security models and can eliminate the need for extra interaction rounds.

From a computational cost perspective, the use of homomorphic encryption for privacy enhancement does not add significant overhead actually. This is contrary to what most non-cryptography researchers might expect, as shown by the initial time cost results in Table 1.
Apart from the inherent challenges of developing homomorphic encryption technology, we believe the minor computational overhead reported in Table 1 is due to the following factors:
1) The scale of homomorphic encryption used is relatively small. The total parameters for evaluating models on the MNIST and FMNIST datasets are about 0.23 million. As requested by other reviewers, we will also include evaluations on larger datasets and models such as CIFAR10 in the final version.
2) Our design mechanism does not involves complex computations over the encrypted model, with only one layer of multiplication depth. It is well-known that multiplication depth significantly impacts efficiency in homomorphic encryption.
Besides, although we provided a computational cost comparison with other approaches, we did not measure latency. As shown in the released open-source code, we implemented a simulated FL framework rather than an actual FL system at this prototype stage. Our goal is to validate the effectiveness of the proposed dual defense approaches. Building an FL system in a distributed real-world network environment deviates from the core focus of this paper and will be addressed in future research stages.

Here are specific responses to the raised questions:
1) Compared to native FL paradigms or existing non-private solutions, our work introduces homomorphic encryption to enhance privacy preservation and support poisoning model detection. Due to the introduction of homomorphic encryption, providing a precise formal time complexity analysis is challenging. We believe the reviewer would like a breakdown of the time costs involved. In addition to the original federated learning training time, extra time is required for homomorphic encryption processes such as encryption, decryption, and secure computation (i.e., inner-product and addition among encrypted models and fusion weights or perturbation noise). Each client is responsible only for encryption and decryption, while all secure computations are performed by the server.
2) Due to limited experimental conditions, we only report results for client sizes ranging from 10 to 100. Theoretically, our proposed framework supports a larger scale of clients. Since the input sizes for encryption, decryption, and secure computation are linear relative to the number of clients, the time cost is also linear based on current reported results. For example, in evaluating the MNIST dataset as shown in Table 1, compared to a non-private solution, our approach requires approximately an additional 2 seconds per training round with 100 clients participating in FL training. Therefore, scaling up to thousands of clients may take about 20 seconds per round without considering network latency.
3) As stated in Section 3.1 regarding threat assumptions (i.e., the threat model in security and privacy papers), our work can handle scenarios where less than half of the clients are compromised. We will further clarify these statements and analyses in the final version.

We hope we have addressed the reviewer's concerns. We will improve the final version of this paper and thank the reviewers for their valuable comments.