SafeNet: The Unreasonable Effectiveness of Ensembles in Private Collaborative Learning
Abstract —Secure multiparty computation (MPC) has been proposed to allow multiple mutually distrustful data owners to jointly train machine learning (ML) models on their combined data.
However, by design, MPC protocols faithfully compute the training functionality, which is susceptible to poisoning and privacy attacks, as shown by the adversarial ML community.
In this work, we argue that model ensembles, implemented in our framework called SafeNet, are a highly MPC-amenable way to avoid many adversarial ML attacks.
The natural partitioning of data amongst owners in MPC training allows this approach to be highly scalable at training time, provide provable protection from poisoning attacks, and provable defense against a number of privacy attacks.
We demonstrate SafeNet’s efficiency, accuracy, and resilience to poisoning on several machine learning datasets and models trained in end-to-end and transfer learning scenarios.
For instance, SafeNet reduces backdoor attack success significantly, while achieving 39 × faster training and 36 × less communication than the four-party MPC framework of Dalskov et al.
Our experiments show that ensemble learning retains these benefits even in many non-iid settings.
The simplicity, cheap setup, and robustness properties of ensemble learning make it a strong first choice for training ML models privately in MPC.
I. I NTRODUCTION
Machine learning (ML) has been successful in a broad range of application areas such as medicine, finance, and recommendation systems.
Consequently, technology companies such as Amazon, Google, Microsoft and IBM provide machine learning as a service (MLaaS) for ML training and prediction.
In these services, data owners outsource their
ML computations to a set of more computationally powerful servers.
However, in many instances, the client data used for
ML training or classification is sensitive and may be subject to privacy requirements.
Regulations such as GDPR, HIPAA and PCR, data sovereignty issues, and user privacy concern are common reasons preventing organizations from collecting user data and training more accurate ML models.
These privacy requirements have led to the design of privacy-preserving
ML training methods, including the use of secure multiparty computation (MPC).
Recent literature in the area of MPC for ML proposes privacy-preserving machine learning (PPML) frameworks [1], [31], [32], [71], [73], [75], [91], [92], [95] for training and inference of various machine learning models such as logistic regression, neural networks, and random forests.
In these models, data owners outsource shares of their data to a set of servers and the servers run MPC protocols for ML training and prediction.
An implicit assumption for security made in this setup is that the underlying datasets provided by the data owners during training is not influenced by an adversary.
However, research in adversarial machine learning has shown that data poisoning attacks pose a high risk to the integrity of trained ML models [10], [43], [47], [52].
Data poisoning becomes a particularly relevant threat in PPML systems, as multiple data owners contribute secret shares of their datasets for jointly training a ML model inside the MPC, and poisoned samples cannot be easily detected.
Furthermore, the guarantees of MPC provide privacy against an adversary observing the messages in the protocol, but does not protect against any sensitive information leaked by the model about its training set.
Many privacy attacks are known to allow inference on machine learning models’ training sets, and protecting against these attacks is an active area of research.
In this paper, we study the impact of these adversarial machine learning threats on standard MPC frameworks for private ML training.
Our first observation is that the security definition of MPC for private ML training does not account for data owners with poisoned data.
Therefore, we extend the security definition by considering an adversary who can poison the datasets of a subset of owners, while at the same time controlling a subset of the servers in the MPC protocol.
Under our threat model, we empirically demonstrate that poisoning attacks are a significant threat to the setting of private ML training.
We show the impact of backdoor [25], [47] and targeted [43], [58] poisoning attacks on four MPC frameworks and five datasets, using logistic regression and neural networks models.
We show that with control of just a single owner and its dataset (out of a set of 20 owners contributing data for training), the adversary achieves 100% success rate for a backdoor attack, and higher than 83% success rate for a targeted attack.
These attacks are stealthy and cannot be detected by simply monitoring standard ML accuracy metrics.
To mitigate these attacks, we apply ensembling technique from ML, implemented in our framework called SafeNet, which, in the collaborative learning setting we consider, is an effective defense against poisoning attacks, while also simultaneously preventing various types of privacy attacks.
Rather than attempting to implement an existing poisoning defense in MPC, we observe that the structure of the MPC threat model permits a more general and efficient solution.
Our main insight is to require individual data owners to train
ML models locally, based on their own datasets, and secret share the resulting ensemble of models in the MPC.
We filter out local models with low accuracy on a validation dataset, and use the remaining models to make predictions using a majority voting protocol performed inside the MPC.
While this permits stronger model poisoning attacks, the natural partitioning of the MPC setting prevents an adversary from poisoning more than a fixed subset of the models, resulting in a limited number of poisoned models in the ensemble.
We perform a detailed analysis of the robustness properties of SafeNet, and provide lower bounds on the ensemble’s accuracy based on the error rate on the local models in the ensemble and the number of poisoned models, as well as a prediction certification procedure for arbitrary inputs.
The bounded contribution of each local model also gives a provable privacy guarantee for
SafeNet.
Furthermore, we show empirically that SafeNet successfully mitigates backdoor and targeted poisoning attacks, while retaining high accuracy on the ML prediction tasks.
In addition, our approach is efficient, as ML model training is performed locally by each data owner, and only the ensemble filtering and prediction protocols are performed in the MPC.
This provides large performance improvements in ML training compared to existing PPML frameworks, while simultaneously mitigating poisoning attacks.
For instance, for one neural network model, SafeNet performs training 39 × faster than the [31] PPML protocol and requires 36 × less communication.
Finally, we investigate settings with diverse data distributions among owners, and evaluate the accuracy and robustness of
SafeNet under multiple data imbalance conditions.
To summarize, our contributions are as follows:
Adversarial ML-aware Threat Model for Private Machine
Learning.
We extend the MPC security definition for private machine learning to encompass the threat of data poisoning attacks and privacy attacks.
In our threat model, the adversary can poisoned a subset t out of m data owners, and control
T out of N servers participating in the MPC.
The attacker may also seek to learn sensitive information about the local datasets through the trained model.
SafeNet Ensemble Design.
We propose SafeNet, which adapts ensembling technique from ML to the collaborative
MPC setting by having data owners train models locally and aggregation of predictions is performed securely inside the
MPC.
We show that this procedure gives provable privacy and security guarantees, which improves as models become more accurate.
We also propose various novel extensions to this ensembling strategy which make SafeNet applicable to a wider range of training settings (including transfer learning and accommodating computationally restricted owners).
SafeNet’s design is agnostic to the underlying MPC framework and we show it can be instantiated over four different MPC frameworks, supporting two, three and four servers.
Comprehensive Evaluation.
We show the impact of existing backdoor and targeted poisoning attacks on several existing
PPML systems [4], [31], [35] and five datasets, using logistic regression and neural network models.
We also empirically demonstrate the resilience of SafeNet against these attacks, for an adversary compromising up to 9 out of 20 data owners.
We report the gains in training time and communication cost for SafeNet compared to existing PPML frameworks.
Finally, we compare SafeNet with state-of-the-art defenses against poisoning in federated learning [17] and show its enhanced certified robustness even under non-iid data distributions.
II. B ACKGROUND AND R ELATED W ORK
We provide background on secure multi-party computation and poisoning attacks in ML, and discuss related work in the area of adversarial ML and MPC.
A. Secure Multi-Party Computation
Secure Multi-Party Computation (MPC) [7], [34], [44], [50], [98] allows a set of n mutually distrusting parties to compute a joint function f , so that collusion of any t parties cannot modify the output of computation ( correctness ) or learn any information beyond what is revealed by the output ( privacy ).
The area of MPC can be categorized into honest majority [4], [7], [14], [22], [74] and dishonest majority [33], [34], [44], [72], [98].
The settings of two-party computation (2PC) [64], [65], [78], [98], three parties (3PC) [3], [4], [74], and four parties (4PC) [23], [31], [46], [51] have been widely studied as they provide efficient protocols.
Additionally, recent works in the area of privacy preserving ML propose training and prediction frameworks [1], [61], [73], [75], [81], [82], [91], [92] built on top of the above MPC settings.
Particularly, most of the frameworks are deployed in the outsourced computation setting where the data is secret-shared to a set of servers which perform training and prediction using MPC.
B. Data Poisoning Attacks
In a data poisoning attack, an adversary controls a subset of the training dataset, and uses this to influence the model trained on that training set.
In a backdoor attack [25], [47], [77], an adversary seeks to add a “trigger” or backdoor pattern into the model.
The trigger is a perturbation in feature space, which is applied to poisoned samples in training to induce misclassification on backdoored samples at testing.
In a targeted attack [57], [58], [86], the adversary’s goal is to change the classifier prediction for a small number of specific test samples.
Backdoor and targeted attacks can be difficult to detect, due to the subtle impact they have on the ML model.
C. Related Work
While both MPC and adversarial machine learning have been the topic of fervent research, work connecting them is still nascent.
We are only aware of several recent research papers that attempt to bridge these areas.
Recent works [20], [62] show that MPC algorithms applied at test time can be compromised by malicious users, allowing for efficient model extraction attacks.
Second, Escudero et al.
[39] show that running a semi-honest MPC protocol with malicious parties can result in backdoor attacks in the resulting SVM model.
Both these works, as well as our own, demonstrate the difficulty of aligning the guarantees of MPC with the additional desiderata of trustworthy machine learning.
We demonstrate the effectiveness of data poisoning attacks in MPC for neural networks and logistic regression models, and
C t
C t +
C m
S
S
S T S T +
S N
S N −
SOC/MPC Paradigm
Poisoned
Corrupted
Honest ... propose a novel ensemble training algorithm in SafeNet to defend against poisoning attacks in MPC.
Model ensembles have been proposed as a defense for ML poisoning and privacy attacks in prior work in both the centralized training setting [9], [53] and the collaborative learning setting [17], [27].
Compared to centralized approaches, which process a single dataset, we are able to leverage the trust model of MPC, which limits the number of poisoned models in the ensemble and can provide stronger robustness and privacy guarantees.
Ensembles have also been proposed in
MPC to protect data privacy [27] and in federated learning to provide poisoning robustness [17].
Our work provides a stronger privacy analysis, protecting from a broader range of threats than [27], and additionally offers robustness guarantees.
We provide a more detailed comparison with these approaches in Section III-F.
III. S AFE N ET : U SING E NSEMBLES IN MPC
We describe here our threat model and show how to implement ensembles in MPC.
We then show that ensembling gives us provable robustness to poisoning and privacy adversaries.
A. Threat Model
Setup.
We consider a set of m data owners C
= ∪ mk =1 C k who wish to train a joint machine learning model M on their combined dataset D = ∪ m k =1 D k .
We adopt the Secure
Outsourced Computation (SOC) paradigm [1], [14], [31], [32], [73], [75], [82], [91], [92] for training model M privately, where the owners secret-share their respective datasets to a set of outsourced servers, who execute the MPC protocols to train
M .
The final output is a trained model in secret-shared format among the servers.
A single training/testing sample is expressed as ( x i , y i ) , where x i is the input feature vector and y i is its corresponding true label or class.
We use D k =
( X k , y k ) to denote dataset of data owner C k participating in the training process.
Matrix X k denotes a feature matrix where the number of rows represent the total training samples possessed by C k and y k denotes the corresponding vector of true labels.
Adversary in the SOC.
Given a set S = {S 1 , . . .
, S N } of servers, we define an adversary A soc , similar to prior work [1], [31], [73], [75], [82], [92].
A soc can statically corrupt a subset
S T ⊂ S of servers of size at most T < N .
The exact values of N and T are dependent on the MPC protocols used for training the ML model privately.
We experiment with twoparty, three-party, and four-party protocols with one corrupt server.
MPC defines two main adversaries: i) Semi-honest : Adversary follows a given protocol, but tries to derive additional information from the messages received from other parties during the protocol; ii) Malicious : Adversary has the ability to arbitrarily deviate during the execution of the protocol.
Security Definition.
MPC security is defined using the real world - ideal world paradigm [15].
In the real world, parties participating in the MPC interact during the execution of a protocol π in presence of an adversary A .
Let REAL [ Z , A , π, λ ] denote the output of the environment Z when interacting with
A and the honest parties, who execute π on security parameter λ .
Effectively, REAL is a function of the inputs/outputs and messages sent/received during the protocol.
In the ideal world, the parties simply forward their inputs to a trusted functionality
F and forward the functionality’s response to the environment.
Let IDEAL [ Z , S , F , λ ] denote the output of the environment
Z when interacting with adversary S and honest parties who run the protocol in presence of F with security parameter λ .
The security definition states that the views of the adversary in the real and ideal world are indistinguishable:
Definition 1.
A protocol π securely realizes functionality F if for all environments Z and any adversary of type A soc , which corrupts a subset S T of servers of size at most T < N in the real world, then there exists a simulator S attacking the ideal world, such that IDEAL [ Z , S , F , λ ] ≈ REAL
[ Z , A soc , π, λ ] .
Poisoning Adversary.
Existing threat models for training ML models privately assume that the local datasets contributed towards training are not under the control of the adversary.
However, data poisoning attacks have been shown to be a real threat when ML models are trained on crowd-sourced data or data coming from untrusted sources [10], [52], [76].
Data poisoning becomes a particularly relevant risk in PPML systems, in which data owners contribute their own datasets for training a joint ML model.
Additionally, the datasets are secret shared among the servers participating in the MPC, and potential poisoned samples (such as backdoored data) cannot be easily detected by the servers running the MPC protocol.
To account for such attacks, we define a poisoning adversary
A p that can poison a subset of local datasets of size at most t < m .
Data owners with poisoned data are called poisoned owners , and we assume that the adversary can coordinate with the poisoned owners to achieve a certain adversarial goal.
For example, the adversary can mount a backdoor attack, by selecting a backdoor pattern and poison the datasets under its control with the particular backdoor pattern.
Poisoning Robustness: We consider an ML model to be robust against a poisoning adversary
A p , who poisons the datasets of t out of m owners, if it generates correct class predictions on new samples with high probability.
We provide bounds on the level of poisoning tolerated by our designed framework to ensure robustness.
Our Adversary.
We now define a new adversary
A psoc for our threat model (Figure 1) that corrupts servers in the MPC and poisons the owners’ datasets:
– A psoc plays the role of A p and poisons t out of m data owners that secret share their training data to the servers.
– A psoc plays the role of A soc and corrupts T out N servers taking part in the MPC computation
Note that the poisoned owners that A psoc controls do not interfere in the execution of the MPC protocols after secretsharing their data and also do not influence the honest owners.
Functionality F pTrain .
Based on our newly introduced threat model, we construct a new functionality F pTrain in Figure 2 to accommodate poisoned data. Input computes O ( D D m a m f training
[Figure]
Security against A psoc .
A training protocol Π train is secure against adversary
A psoc if: (1) Π train securely realizes functionality F pTrain based on Definition 1; and (2) the model trained inside the MPC provides poisoning robustness against data poisoning attacks.
Intuitively, the security definition ensures that A psoc learns no information about the honest owners’ inputs when T out of N servers are controlled by the adversary, while the trained model provides poisoning robustness against a subset of t out of m poisoned owners.
Extension to Privacy Adversary.
While MPC guarantees no privacy leakage during the execution of the protocol, it makes no promises about privacy leakage that arises by observing the output of the protocol.
This has motivated a combination of differential privacy guarantees with MPC algorithms to protect against privacy leakage for both the intermediate execution as well as the output of the protocol.
For this reason, we also consider adversaries seeking to learn information about data owners’ local datasets by observing the output of the model, as done in membership inference [19], [85], [99] and property inference attacks [42], [87], [102].
Recent works have used data poisoning as a tool to further increase privacy leakage [21], [69], [89] of the trained models.
Consequently, we can extend our threat model to accommodate a stronger version of A psoc that is also capable of performing privacy attacks by observing the output of the trained model.
B
Given our threat model in Figure 1, existing PPML frameworks provide security against an A soc adversary, but they are not designed to handle an A psoc adversary.
We show experimentally in Section IV that PPML frameworks for private training are susceptible to data poisoning attacks.
While it would be possible to remedy this by implementing specific poisoning defenses (see Section V-C for a discussion of these approaches), we instead show that we can take advantage of the bounded poisoning capability of A psoc to design a more general and efficient defense.
Intuitively, existing approaches train a single model on all local datasets combined, causing the model’s training set to have a large fraction of poisoned data ( t/m ), which is difficult to defend against.
Instead, we design SafeNet, a new protocol which uses ensemble models to realize our threat model and provide security against A psoc .
In addition to successfully mitigating data poisoning attacks,
SafeNet provides more efficient training than existing PPML and comparable prediction accuracy.
Figure 3 provides an overview of the training and inference phases of SafeNet.
SafeNet trains an ensemble E of multiple models in protocol Π train , where each model M k ∈ E is trained locally by the data owner C k on their dataset.
This partitioning prevents poisoned data from contributing to more than t local models.
Each data owner samples a local validation dataset and trains the local model M k on the remaining data.
The local models and validation datasets are secret shared to the outsourced servers.
We note that this permits arbitrarily corrupted models, and poisoned validation datasets, but SafeNet’s structure still allows it to tolerate these corruptions.
In the protocol running inside the MPC, the servers jointly implement a filtering stage for identifying models with low accuracy on the combined validation data (below a threshold ϕ ) and excluding them from the ensemble.
The output of training is a secret share of each model in the trained ensemble E .
In the inference phase, SafeNet implements protocol Π pred , to compute the prediction y k of each shared model M k on test input x inside the MPC.
The servers jointly perform majority voting to determine the most common predicted class y on input x , using only the models which pass the filtering stage.
An optional feature of SafeNet is to add noise to the majority vote to enable user-level differential privacy protection, in addition to poisoning robustness.
Our SafeNet protocol leverages our threat model, which assumes that only a set of at most t out of m data owners are poisoned.
This ensures that an adversary only influences a limited set of models in the ensemble, while existing training protocols would train a single poisoned global model.
We provide bounds for the exact number of poisoned owners t supported by our ensemble in Theorem 6.
Interestingly, the bound depends on the number of data owners m , and the maximum error made by a clean model in the ensemble.
The same theorem also lower bounds the probability that the ensemble predicts correctly under data poisoning performed by the t
Local Data 𝐷 !
Client Query 𝑥 s
SafeNet
Local Training 𝐷 !"
← SelectRandom(𝐷 ! )
𝑀 !
← LocalTrain(𝐷 !
∖𝐷 !" )
Local Validation Dataset
Secure Outsourced Computation 𝐷 "#$ ←∪ !
𝐷 !"
AccVal !
← Accuracy(𝑀 ! ,𝐷 "#$ )
Secret shared validation datasets 𝐷 !"
Global Validation Dataset
Filter model 𝑀 !
if AccVal !
<
𝜙
Model validation accuracy
Filtering stage
Ensemble of Models 𝐸 ← (𝑀 % ,…𝑀 & )
Secret shared models 𝑀 !
Π !"
#$%
Ensemble of Models 𝑦 !
← 𝑀 !
(𝑥)
Model predictions 𝑦 ← MajorityVote(𝑦 % ,…,𝑦 & )
Ensemble final prediction
Secure Outsourced Computation Π &"’(
[Figure]
poisoned owners, and we validate experimentally that, indeed, SafeNet provides resilience to stealthy data poisoning attacks, such as backdoor and targeted attacks. Another advantage of SafeNet is that the training time to execute the MPC protocols in the SOC setting is drastically reduced as each M k ∈ E can be trained locally by the respective owner.
We detail below the algorithms for training and inference in SafeNet.
C. SafeNet Training and Inference
To train the ensemble in SafeNet, we present our proposed ensemble method in Algorithm 1.
We discuss the realization in MPC later in Appendix B. Each owner C k separates out a subset of its training dataset D v k ∈ D k and then trains its model M k on the remaining dataset D k \ D v k .
The trained model M k and validation dataset D v k is then secret-shared to the servers.
The combined validation dataset is denoted as
D val = m (cid:83) i =1 D v i .
We assume that all users contribute equal-size validation sets to D val .
During the filtering stage inside the
MPC, the validation accuracy AccVal of each model is jointly computed on D val .
If the resulting accuracy for a model is below threshold ϕ , the model is excluded from the ensemble.
The filtering step is used to separate the models with low accuracy, either contributed by a poisoned owner, or by an owner holding non-representative data for the prediction task.
Under the assumption that the majority of owners are honest, it follows that the majority of validation samples are correct.
If
C k is honest, then the corresponding M k should have a high validation accuracy on D val , as the corresponding predicted outputs would most likely agree with the samples in D val .
In contrast, the predictions by a poisoned model M k will likely not match the samples in D val .
In Appendix A, we compute a lower bound on the size of the validation dataset as a function of the number of poisoned owners t and filtering threshold ϕ , such that all clean models pass the filtering stage with high probability even when a subset of the cross-validation dataset
D val is poisoned.
Given protocol Π train that securely realizes Algorithminside the MPC (described in Appendix B), we argue security as follows:
Theorem 2.
Given a privacy-preserving machine learning training framework that tolerates T out of N server corruptions, and an error bound of at most p on each clean model in the ensemble, protocol Π train is secure against adversary
A psoc who poisons t < m 2 1 − 2 p 1 − p out of m data owners and corrupts
T out of N servers.
The proof of the theorem is given in Appendix C, after we introduce the details of MPC instantiation and how Π train securely realizes F pTrain in Appendix B-3.
During inference, the prediction of each model M k is generated and the servers aggregate the results to perform majority voting.
Optionally, differentially private noise is added to the sum to offer userlevel privacy guarantees.
The secure inference protocol Π pred and its security proof are given in Appendix B and C, respectively.
Here, we demonstrate the accuracy, poisoning robustness and privacy guarantees that SafeNet provides.
We first show how to lower bound SafeNet’s test accuracy given that each clean model in the ensemble reaches a certain accuracy level.
We also give certified robustness and user-level privacy guarantees.
All of our guarantees improve as the individual models become more accurate, making the ensemble agree on correct predictions more frequently.
Robust Accuracy Analysis.
We provide lower bounds on
SafeNet accuracy, assuming that at most t out m models in the SafeNet ensemble E are poisoned, and the clean models have independent errors, with maximum error rate p < 1 − ϕ , where ϕ is the filtering threshold.
Theorem.
(Informal) Let A psoc be an adversary who poisons at most t out of m data owners and corrupts T out of N servers.
Assume that the filtered ensemble E has at least m − t clean models, each with a maximum error rate of p < 1 − ϕ .
If the number of poisoned owners is at most m (1 − 2 p ) 2(1 − p ) , ensemble E correctly classifies new samples with high probability, which is a function of m , ϕ , t and p .
The formal theorem and the corresponding proof can be found in Appendix A.
Poisoning Robustness Analysis.
Our previous theorem demonstrated that SafeNet’s accuracy on in-distribution data is not compromised by poisoning.
Now, we show that we can also certify robustness to poisoning on a per-sample basis for arbitrary points, inspired by certified robustness techniques for adversarial example robustness [29].
In particular, Algorithmdescribes a method for certified prediction against poisoning, returning the most common class y predicted by the ensemble on a test point x , as well as a bound on the number of poisoning owners t which would be required to modify the predicted class.
Sometimes, the algorithm can return t = 0 , in which case SafeNet provides no robustness for the test point.
Input: m data owners; Ensemble of models E =
{M i } mi =1 ;
Testing point x ; Differential Privacy parameters ε, δ .
C OUNTS = (cid:80) mi =1 M i ( x ) + DPN OISE ( ε, δ ) y,
c y =
M OST C OMMON
( C OUNTS ) //
most common predicted class with noisy count y ′ , c y ′ =
S ECOND M OST C OMMON ( C OUNTS ) //
second most common predicted class with count t = ⌈ ( c y − c y ′ ) / 2 ⌉ −return y, t
We first analyze the poisoning robustness when privacy of aggregation is not enabled in the following theorem.
Theorem 3.
Let E be an ensemble of models constructed in the presence of adversary A psoc .
Assume that given input x , the ensemble generates prediction y
= E ( x ) without DPN OISE and Algorithm 2 outputs ( y, t ) .
Then, if A psoc poisons at most t data owners, it could not have changed E ’s prediction to y ′ ̸ = y .
Proof.
If an adversary’s goal were to cause y ′ to be predicted on input x , their most efficient strategy is to flip y predictions to y ′ .
If y were the ensemble prediction, it must have at least ⌊ c y + c y
[Equation]
⌋ model predictions, and the second most common prediction
y
′ would have at most ⌊ c y + c y ′⌋ model predictions.
Corrupting these predictions then requires flipping at least ( c y − c y ′ )
/ 2 predictions from y to y ′ .
Overall, this requires at least ⌈ ( c y − c y ′ ) /
2 ⌉ poisoned data owners.
Thus, an adversary poisoning at most t = ⌈ ( c y − c y ′ ) /
2 ⌉ − 1 data owners still generates the same prediction y on x .
Privacy Analysis.
Recent work by McMahan et al.
[70] introduced the notion of user-level differential privacy where the presence of a user in the protocol should have imperceptible impact on the final trained model.
We show that, given our threat model, SafeNet provides the strong privacy guarantee of user-level differential privacy, which also implies examplelevel differential privacy.
This privacy guarantee can protect against model extraction and property inference attacks, in addition to membership inference attacks.
Theorem 4.
When DPN OISE function samples from a Laplace random variable Lap (2 /ε ) , Algorithm 2 satisfies user-level εdifferential privacy.
Proof.
Observe that replacing a local model obtained from a data owner in our framework only changes C OUNTS for two classes by 1 on any given query, so it has an ℓ 1 sensitivity of 2.
As a result, Lap (2 /ε ) suffices to ensure that user-level ε -differential privacy holds.
The main crux of Theorem 4 is that no model can influence C OUNTS too much, an observation also made by
PATE [79] and the CaPC [27] framework, but they only considered example-level differential privacy, which protects against membership inference attacks, but not the stronger attacks that user-level differential privacy prevents.
This limitation is inherent in PATE, as the central training set is split to train multiple models.
However, our stronger analysis holds for SafeNet in the private collaborative learning setting, as we start with pre-existing partitions of benign and poisoned datasets.
We prove Theorem 4 by considering Laplace noise, but various improvements to PATE using different mechanisms such as
Gaussian noise and other data-dependent approaches [79], [80], can also be included in our framework.
Combining Robustness and Privacy.
Adding differentially private noise prevents Algorithm 2 from returning the exact difference between the top two class-label counts, making it only possible to offer probabilistic robustness guarantees.
That is, the returned t is actually a noisy version of the “true” t ∗ , where t ∗ is used to certify correctness.
However, for several choices of the DPNoise function, the exact distribution of the noise is known, making it easy to provide precise probabilistic guarantees similar to those provided by Theorem 3.
For example, if Gaussian noise with scale parameter σ is used to guarantee DP, and PredGap returns t , then we know that the true t ∗ is larger than t − k with probability Φ( k/σ ) , where Φ denotes the Gaussian CDF.
Also note that prior work has shown, for local differentially private protocols, the assumption of users correctly randomizing their messages leads to increased vulnerability [18], [26].
Our protocol avoids this vulnerability by generating noise in the MPC, not trusting any individual user to generate noise.
E. Extensions
In addition to providing various guarantees, we offer a number of extensions to our original SafeNet design.
Transfer Learning.
A major disadvantage of SafeNet is its slower inference time compared to a traditional PPML framework, requiring to perform a forward pass on all local models in the ensemble.
However, for transfer learning scenario, we propose a way where SafeNet’s inference runs almost as fast as the traditional framework.
In transfer learning [37], [59], a pretrained model M B , which is typically trained on a large public dataset, is used as a “feature extractor” to improve training on a given target dataset.
In our setting, all data owners start with a common pre-trained model, and construct their local models by fine tuning M B ’s last l layers using their local data.
We can then modify the prediction phase of SafeNet to reduce its inference time and cost considerably.
The crucial observation is that all local models differ only in the weights associated to the last l layers.
Consequently, given a prediction query, we run M B upto its last l layers and use its output to compute the l layers of all the local models to obtain predictions for majority voting.
The detailed description of the modified SafeNet algorithm is given in Appendix D-A. Note that this approach achieves the same robustness and privacy guarantees as described in Section III-D, as long as M B was originally not tampered with.
If M B is untrustworthy, it may already be backdoored [67], or may be easier to run other types of attacks [94].
Integration Testing.
While SafeNet can handle settings with non-iid data distributions among data owners, the local models accuracies might be impacted by extreme non-iid settings (we analyze the sensitivity of SafeNet to data imbalance in Section
IV-H).
In such cases, SafeNet fails fast , allowing the owners to determine whether or not using SafeNet is the right approach for their setting.
This is possible because SafeNet’s training phase is very cheap, making it possible to quickly evaluate the ensemble’s accuracy on the global validation set.
If the accuracy is not good enough, the owners can use a different approach, such as a standard MPC training.
SafeNet’s strong robustness guarantees and an efficient training phase makes it an appealing first choice for private collaborative learning.
It is possible for this filtering to be circumvented if the adversary can participate multiple times [12].
However, this is straightforward to mitigate by bounding the number of submissions.
Furthermore, should an adversarial model slip into the ensemble, the robustness guarantees of Theoremwill still hold, but with one extra poisoned model.
Low Resource Owners.
If a data owner does not have sufficient resources to train a model on their data, they cannot participate in the standard SafeNet protocol.
In such situations, computationally restricted owners can defer their training to
SafeNet, that can use standard MPC training approaches to train their models.
Training these models in MPC increases the computational overhead of our approach, but facilitates broader participation.
We provide the details of this modification in Appendix D-B and also run an experiment in Appendix
E-A to verify that SafeNet remains efficient, while retaining the same robustness and privacy properties.
F. Comparison to Existing Ensemble Strategies
Model ensembles have been considered to address adversarial machine learning vulnerabilities in several prior works.
Here, we discuss the differences between our analysis and previous ensembling approaches.
a) Ensembles on a Centralized Training Set: Several ensemble strategies seek to train a model on a single, centralized training set.
This includes using ensembles to prevent poisoning attacks [54], [63], as well as to provide differential privacy guarantees [79] or robustness to privacy attacks [88].
Due to centralization, none of these techniques can take advantage of the partitioning of datasets.
As a result,
protection from poisoning is only capable of handling a small number of poisoning examples, whereas our partitioning allows large fractions of the entire dataset to be corrupted.
PATE, due to data centralization, can only guarantee privacy for individual samples, whereas in our analysis, the entire dataset of a given owner can be changed, providing us with user-level privacy.
b) CaPC [27]: Chouquette-Choo et al.
[27] propose
CaPC, which extends PATE to the MPC collaborative learning setting.
Their analysis gives differential privacy guarantees for individual examples.
Our approach extends their analysis to a differential privacy guarantee for the entire local training set and model, to provide protection against attacks such as property inference and model extraction.
In addition, our approach also provides poisoning robustness guarantees which they cannot, as they allow information to be shared between local training sets.
c) Cao et al.
[17]: Recent work by Cao et al.
[17] gave provable poisoning robustness guarantees for federated learning aggregation.
They proposed an ensembling strategy, where, given m data owners, t of which are malicious, they construct an ensemble of (cid:0) mk (cid:1) global models, where each model is trained on a dataset collected from a set of k clients.
Our poisoning robustness argument in Theorem 3 coincides with theirs at k = 1 , a setting they do not consider as their approach relies on combining client datasets for federated learning.
Additionally, k = 1 makes their approach vulnerable to data reconstruction attacks [13], an issue SafeNet does not face as the attack directly violates the underlying security guarantee of the MPC.
We experimentally compare both approaches on a federated learning dataset in Section V-D and show that our approach outperforms [17].
A. Experimental Setup
We build a functional code on top of the MP-SPDZ library [56] 1 to assess the impact of data poisoning attacks on the training phase of PPML frameworks.
We consider four different MPC settings, all available in the MP-SPDZ library: i) two-party with one semi-honest corruption (2PC) based on [30], [35]; ii) three-party with one semi-honest corruption (3PC) based on Araki et al.
[4] with optimizations by [32], [73]; iii) three-party with one malicious corruption based on
Dalskov et al.
[31]; and iv) four-party with one malicious corruption (4PC), also based on [31].
Note, that both semihonest and malicious adversaries possess poisoning capability; their roles change only inside the SOC paradigm.
In all the PPML frameworks, the data owners secret-share their training datasets to the servers and a single ML model is trained on the combined dataset.
Typically, real number arithmetic is emulated by using 32 -bit fixed-point representation of fractional numbers.
Each fractional number x ∈ Z 2 ℓ is represented as ⌊ x · 2 f ⌉ , where ℓ and f denote the ring size and precision, respectively.
We set ℓ = 64 and f = 16 .
Probabilistic truncation proposed by Dalskov et al.
[31], [32] is applied after every multiplication.
In the MPC library implementation, the sigmoid function for computing the output probabilities is replaced with a three-part approximation [22], [31], [75].
In SafeNet, models are trained locally using the original sigmoid function.
We implement softmax function using the method of Aly et al.
We perform our experiments over a LAN network on a 32 -core server with 192 GB of memory allowing up to 20 threads to be run in parallel.
B. Metrics
We use the following metrics to compare SafeNet with existing PPML framework:
Training Time is the time taken to privately train a model inside the MPC (protocol Π train ).
As is standard practice [14], [22], [23], [31], [73], [75], this excludes the time taken by the data owners to secret-share their datasets and models to the servers as it is a one-time setup phase.
Communication Complexity is the amount of data exchanged between the servers during the privacy-preserving execution of the training phase.
Test Accuracy is the percentage of test samples that the ML model correctly predicts.
Attack Success Rate is the percentage of target samples that were misclassified as the label of attacker’s choice.
Robustness against worst-case adversary We measure the resilience of SafeNet at a certain corruption level c against a powerful, worst-case adversary.
For each test sample, this adversary can adaptively select any subset of c owners, arbitrarily modifying the model to change the test sample’s classification.
This is the same adversary considered in Algorithm 2 and by
Theorem 3, any any model which is robust against this attack has a provably certified prediction.
We measure the error rate on testing samples for this worst-case adversarial model.
C. Datasets and Models
We give a descriptions of the datasets and models used for our experiments below.
MNIST.
The MNIST dataset [38] is a 10 class classification problem which is used to predict digits between 0 and 9 .
We train a logistic regression model for MNIST.
Adult.
The Adult dataset [38] is for a binary classification problem to predict if a person’s annual income is above $50K.
We train a neural network with one hidden layer of sizenodes using ReLU activations.
Fashion.
We train several neural networks on the Fashion dataset [96] with one to three hidden layers.
The Fashion dataset is a 10-class classification problem with 784 features representing various garments.
All hidden layers havenodes and ReLU activations; the output layer uses softmax.
CIFAR-10.
The CIFAR-10 dataset [60] is a 10 class image dataset.
CIFAR-10 is harder than other datasets we consider, so we perform transfer learning from a ResNet-50 model [48] pretrained on the ImageNet dataset [36].
We fine tune only the last layer, freezing all convolutional layers.
EMNIST.
The EMNIST dataset [28] is a benchmark federated learning image dataset, split in a non-iid fashion by the person who drew a given image, where we select 100 EMNIST clients in our experiments.
D. Dataset Partitioning and Model Accuracy
We conduct our experiments by varying the number of data owners.
We split MNIST and Adult datasets acrossparticipating data owners, while we use 10 owners for Fashion and CIFAR-10 datsets.
For the EMNIST dataset used for comparison with prior work on federated learning we useowners.
Each owner selects at random 10% of its local training data as the validation dataset D v j .
All models are trained using mini-batch stochastic gradient descent.
To introduce non-iid behavior in our datasets (except for
EMNIST, which is naturally non-iid), we sample class labels from a Dirichlet distribution [49].
That is, to generate a population of non-identical owners, we sample q ∼ Dir ( αp ) from a Dirichlet distribution, where p characterizes a prior class distribution over all distinct classes, and α > 0 is a concentration parameter which controls the degree of similarity between owners.
As α → ∞ , all owners have identical distributions, whereas as α → 0 , each owner holds samples of only one randomly chosen class.
In practice, we observe α = 1000 leads to almost iid behavior, while α = 0 .
1 results in an extreme imbalanced distribution.
The default choice for all our experiments is α = 10 , which provides a realistic noniid distribution.
We will vary parameter α in Appendix E-A.
We measure the accuracy of a local model trained by individual data owners and our SafeNet ensemble.
Table I provides the detailed comparison of the accuracy of the local [Table] and ensemble models across all four datasets.
We observe that
SafeNet consistently outperforms local models, with improvements ranging from 4.09% to 25.14%.
The lowest performance is on CIFAR-10, but
in this case SafeNet’s accuracy is very close to fine-tuning the network using the combined dataset, which reaches 65% accuracy.
E. Implementation of Poisoning Attacks
Backdoor Attacks.
We use the BadNets attack by Gu et al.
[47], in which the poisoned owners inject a backdoor into the model to change the model’s prediction from source label y s to target label y t .
For instance, in an image dataset, a backdoor might set a few pixels in the corner of the image to white.
The BadNets attack strategy simply identifies a set of k target samples { x ti } ki =1 with true label y s , and creates backdoored samples with target label y t .
We use k = 100 samples, which is sufficient to poison all models.
To run backdoor attacks on models trained with standard
PPML frameworks, the poisoned owners create the poisoned dataset D ∗ j by adding k poisoned samples and secret-sharing them as part of the training dataset to the MPC.
The framework then trains the ML model on the combined dataset submitted by both the honest and poisoned owners.
In SafeNet, backdoor attacks are implemented at the poisoned owners, which add k backdoored samples to their dataset D j and train their local models M ∗ j on the combined clean and poisoned data.
A model trained only on poisoned data will be easy to filter due to low accuracy, making training on clean samples necessary.
The corrupt owners then secretshare both the model M ∗ j and validation set D v j selected at random from D j to the MPC.
Targeted Attacks.
We select k targeted samples, and change their labels in training to a target label y t different from the original label.
The models are trained to simultaneously minimize both the training and the adversarial loss.
This strategy has also been used to construct poisoned models by prior work [57], and can be seen as the output of the unconstrained version of state-of-the-art Witches’ Brew targeted attack [43], which proposes a clean-label poisoning attack.
The next question to address is which samples to target as part of the attack.
We use two strategies to generate k = 100 target samples, based on an ML model trained by the adversary over the test data.
In the first strategy, called
TGT-Top, the adversary chooses examples classified correctly with high confidence by a different model.
Because these examples are easy to classify, poisoning them should be hard.
We also consider an attack called TGT-Foot, which chooses low confidence examples, which are easier to poison.
For both strategies, the adversary replaces its label with the second highest predicted label.
We compare these two strategies for target selection.
The difference between targeted and backdoor attacks is that targeted attacks do not require the addition of a backdoor trigger to training or testing samples, as needed in a backdoor attack.
However, the impact of the backdoor attack is larger.
Targeted attacks change the prediction on a small set of testing samples (which are selected in advance before training the model), while the backdoor attack generalizes to any testing samples including the backdoor pattern.
F. Evaluation on Logistic Regression
We start with DIGIT 1/7 dataset, a subset of MNIST data using only digits 1 and 7, for which we evaluate the computational costs and the poisoning attack success, for both traditional PPML and our newly proposed SafeNet framework.
We perform our experiments over four underlying MPC frameworks, with both semi-honest and malicious adversaries.
Table II provides a detailed analysis of the training time and communication complexity for both existing PPML and
SafeNet frameworks.
Note that the training time and communication cost for the PPML frameworks is reported per epoch times the number of epochs in training.
The number of epochs is a configurable hyper-parameter, but usually at least 10 epochs are required.
On the other hand, the training time and communication reported for SafeNet is for the endto-end execution inside the MPC, independent of the number of epochs.
We observe large improvements of SafeNet over the existing PPML frameworks.
For instance, in the semihonest two-party setting, SafeNet achieves 30 × and 17 × improvement in running time and communication complexity, respectively, for n = 10 epochs.
This is expected because
SafeNet performs local model training, which is an expensive phase in the MPC.
[Table]
To mount the backdoor attack, the backdoor pattern sets the top left pixel value to white (a value of 1).
We set the original class as y s = 1 and target class as y t = 7 .
Figure 4 (a) shows the success rate for the 3PC PPML and SafeNet frameworks by varying the number of poisoned owners between 0 and 10.
We tested with all four PPML settings and the results are similar.
We observe that by poisoning data of a single owner, the adversary is successfully able to introduce a backdoor in the PPML framework.
The model in the PPML framework predicts all k = 100 target samples as y t , achieving 100% adversarial success rate.
In contrast, SafeNet is successfully able to defend against the backdoor attack, and provides 0% attack success rate up to 9 owners with poisoned data.
The test accuracy on clean data for both frameworks is high at around 98 .
98% even after increasing the poisoned owners to 10 .
[Figure]
We observe in Figure 4 (b) that for the TGT-Top targeted attack, a single owner poisoning is able to successfully misclassify 98% of the target samples in the PPML framework.
As a consequence, the test accuracy of the model drops by ≈ 10% .
In contrast, SafeNet works as intended even at high levels of poisoning.
For the TGT-Foot attack in Figure 4 (c), the test accuracy of the 3PC PPML framework drops by ≈ 5% .
The attack success rate is 94% for the 3PC PPML, which is decreased to 21% by SafeNet, in presence of a single poisoned owner.
The accuracy drop and success rate vary across the two strategies because of the choice of the target samples.
In TGTFoot, the models have low confidence on the target samples, which introduces errors even without poisoning, making the attack succeed with slightly higher rate in SafeNet.
Still,
SafeNet provides resilience against both TGT-Top and TGTFoot for up to 9 out of 20 poisoned owners.
Worst-case Robustness.
Figure 4 (d) shows the worst-case attack success in SafeNet, by varying the number of poisoned owners c ∈ [1 , 10] and allowing the attacker to poison a different set of c owners for each testing sample (i.e., the adversarial model considered in Algorithm 2 for which we can certify predictions).
Interestingly, SafeNet’s accuracy is similar to that achieved under our backdoor and targeted attacks, even for this worst-case adversarial scenario.
Based on these results we conclude that: (1) the backdoor and targeted attacks we choose to implement are as strong as the worst-case adversarial attack, in which the set of poisoned owners is selected per sample; (2) SafeNet provides certified robustness up to 9 out of 20 poisoned owners even under this powerful threat scenario.
Multiclass Classification.
We also test both frameworks in the multiclass classification setting for both Backdoor and
Targeted attacks on MNIST dataset and observe similar large improvements.
For instance, in the semi-honest 3PC setting, we get 240 × and 268 × improvement, respectively, in training running time and communication complexity for n =epochs, while the success rate in the worst-case adversarial scenario does not exceed 50% with 9 out of 20 owners being poisoned.
This experiment shows that the robust accuracy property of our framework translates seamlessly even for the case of a multi-class classification problem.
The details of the experiment are deferred to Appendix E.
G. Evaluation on Deep Learning Models
We evaluate neural network training for PPML and SafeNet frameworks on the Adult and Fashion datasets.
We provide experiments on a three hidden layer neural network on Fashion in this section and include additional experiments in Appendix E.
Table III provides a detailed analysis of the training time, communication, test accuracy and success rate for the 4PC
PPML framework and SafeNet using one poisoned owner.
We observe that SafeNet has 39 × and 36 × improvement in training time and communication complexity over the PPML framework, for n = 10 epochs.
The SafeNet prediction time is on average 26 milliseconds to perform a single secure prediction, while the existing PPML framework takes on average 3 .milliseconds for the same task.
We believe this is a reasonable cost for many applications, as SafeNet has significant training time improvements and robustness guarantees.
For the BadNets backdoor attack we set the true label y s as a ‘T-Shirt’ and target label y t as ‘Trouser’.
We test the effect of both TGT-Top and TGT-Foot attacks under multiple poisoned owners, and also evaluate another variant of targeted attack called TGT-Random, where we randomly sample k
=target samples from the test data.
Figure 5 provides the worstcase adversarial success of SafeNet against these attacks.
We observe that SafeNet provides certified robustness for TGTRandom and TGT-Top up to 4 out of 10 poisoned onwers, while the adversary is able to misclassify more target samples in the TGT-Foot attack.
The reason is that the k selected target samples have lowest confidence and models in the ensemble are likely to be in disagreement on their prediction.
MPC
Setting
Framework
Training (s
GB
Backdoor Attack
[Table] [Table]
Malicious
[Table]
[Figure]
Here, we evaluate our SafeNet extensions introduced in Section III-E. First, we show that on applying our transfer learning extension to SafeNet, its inference overhead falls dramatically on Fashion and CIFAR-10 datasets. For the Fashion dataset, we use the same setup as earlier with m = 10 data owners, and a three-layered neural network, where each data owner finetunes only the last layer ( l = 1 ) of the pre-trained model.
We observe that for each secure inference, SafeNet is now only 1 . 62 × slower and communicates 1 .
26 × more on average than the PPML framework, while the standard SafeNet approach is about 8 × slower due to the evaluation of multiple ML models.
We observe even better improvements for CIFAR10 dataset.
Here, we use a state-of-the-art 3PC inference protocol from [61], built specially for ResNet models.
In our setting, each owner fine-tunes the last layer of a ResNet50 model, which was pre-trained on ImageNet data.
SafeNet reaches 62.8% accuracy, decaying smoothly in the presence of poisoning: 51.9% accuracy tolerating a single poisoned owner, and 39.8% while tolerating two poisoned owners.
The cost of inference for a single model is an average of 59.9s, and
SafeNet’s overhead is negligible (experimental noise has a larger impact than SafeNet); SafeNet increases communication by only 0.1%, increasing around 7MB over the 6.5GB required for standard inference.
Next, we analyze the behavior of SafeNet under different non-iid settings by varying the concentration parameter α .
We use the same Fashion dataset setup from Section IV-G.
We observe that as α decreases, i.e., the underlying data distribution of the owners become more non-iid, SafeNet’s accuracy decreases, as expected, but SafeNet still achieves reasonable robustness even under high data imbalance (e.g., α = 1 ).
In extremely imbalanced settings, such as α = 0 . 1 ,
SafeNet can identify low accuracy during training and data owners can take actions accordingly.
We defer the details for this extension to Appendix E-A, which also includes analyzing attack success rates under extreme non-iid conditions.
V. D ISCUSSION AND C OMPARISON A. SafeNet’s Scalability and Modularity Scalability
The
training and prediction times of SafeNet inside the MPC depend on the number of models in the ensemble and the size of the validation dataset.
The training time increases linearly with the fraction of training data used for validation and the number of models in the ensemble.
Similarly, the prediction phase of SafeNet has both runtime and communication scaling linearly with the number of models in the ensemble.
However, we discussed how transfer learning can reduce the inference time of SafeNet.
Modularity.
Another key advantage of SafeNet is that it can use any MPC protocol as a backend, as long as it implements standard ML operations.
We demonstrated this by performing experiments with both malicious and semi-honest security for four different MPC settings.
As a consequence, advances in ML inference with MPC will improve SafeNet’s runtime.
SafeNet can also use any model type implementable in MPC; if more accurate models are designed, this will lead to improved robustness and accuracy.
B. Instantiating SafeNet in Practice
There are two aspects the data owners need to agree upon before instantiating SafeNet: i)
The MPC framework used for secure training and prediction phase and ii) the parameters in Theorem 6 to achieve poisoning robustness.
The owners agree upon the number of outsourced servers N participating in the MPC, the number of corrupted servers T along with the role of the adversary (semi-honest or malicious) in the MPC and consequently choose an appropriate training framework that satisfies this criteria.
The owners then agree upon a filtering threshold ϕ and the number of poisoned owners t that can be tolerated.
Once these parameters are chosen the maximum allowed error probability of the local models trained by the honest owners based on Lemma 5 and Theorem 6, can be computed as p < min( m (1 − ϕ )
− t m − t , m − 2 t 2( m − t ) ) , where m denotes the total number of data owners.
Given the upper bound on the error probability p , each honest owner trains its local model while satisfying the above constraint.
We provide a concrete example on parameter selection as follows:
We instantiate our Fashion dataset setup, with m = 10 data owners participating in SafeNet.
For the MPC, we set N = 3 servers, tolerating
T = 1 corruption in the presence of a malicious adversary and choose [31] as our privacy-preserving training framework.
For poisoning robustness, we set ϕ = 0 .
3 and the number of poisoned owners to t
= 2 .
This gives us the upper bound on max error probability as p < 0 .
Also the size of the global validation dataset is | D val | > 92 samples, i.e., each data owner contributes 10 cross-validation samples each such that the constrained is satisfied.
We observe that none of the clean models are filtered during training and the attack success rate of the adversary for backdoor attacks remains the same even after poisoning 3 owners, while our analysis holds for t =poisoned owners.
Thus, in practice SafeNet is able tolerate more poisoning than our suggested analysis.
C. Comparing to poisoning defenses
Defending against poisoning attacks is an active area of research, but defenses tend to be heuristic and specific to attacks or domains.
Many defenses for backdoor poisoning attacks exist [24], [66], [90], [93], but these strategies work only for Convolutional Neural Networks trained on image datasets; Severi et al.
[84] showed that these approaches fail when tested on other data modalities and models.
Furthermore, recent work by Goldwasser et.al [45] formulated a way to plant backdoors that are undetectable by any defense.
In contrast,
SafeNet is model agnostic and works for a variety of data modalities.
Even if an attack is undetectable, the adversary can poison only a subset of models, making the ensemble robust against poisoning.
In certain instances SafeNet can tolerate around 30% of the training data being poisoned, while being attack agnostic.
SafeNet is also robust to stronger model poisoning attacks [5], [8], [40], which are possible when data owners train their models locally.
SafeNet tolerates model poisoning because each model only contributes to a single vote towards the final ensemble prediction.
D. Comparison with Federated Learning
Federated Learning (FL) is a distributed machine learning framework that allows clients to train a global model without sharing their local training datasets to the central server.
However, it differs from the PPML setting we consider in the following ways: (1) Clients do not share their local data to the server in FL, whereas PPML allows sharing of datasets; (2) Clients participate in multiple rounds of training in FL, whereas they communicate only once with the servers in
PPML; (3) Clients receive the global model at each round in
FL, while in SafeNet they secret-share their models once at the start of the protocol; and, finally, (4) PPML provides stronger confidentiality guarantees such as privacy of the global model.
It is possible to combine FL and MPC to guarantee both client and global model privacy [41], [55], [103], but this involves large communication overhead and is susceptible to poisoning [68].
For example, recent work [6], [8], [97] showed that malicious data owners can significantly reduce the learned global model’s accuracy.
Existing defenses against such owners use Byzantine-robust aggregation rules such as trimmed mean [101], coordinate-wise mean [100] and Krum [11], which have been show to be susceptible to backdoor and model poisoning attacks [40].
Recent work in FL such as
FLTrust [16] and DeepSight [83] provide mitigation against backdoor attacks.
Both strategies are inherently heuristic, while SafeNet offers provable robustness guarantees.
FLTrust also requires access to a clean dataset, which is not required in our framework, and DeepSight inspects each model update before aggregation, which is both difficult in MPC and leads to privacy leakage from the updates, a drawback not found in SafeNet.
An important privacy challenge is that federated learning approaches permit data reconstruction attacks when the central server is malicious [13].
SafeNet prevents such an attack, as it directly violates the security guarantee of the
MPC, when instantiated for the malicious setting.
We compare experimentally SafeNet with the Cao et al.
[17] FL approach in Appendix E-D and show that our approach consistently achieves higher certified accuracy over prior work.
VI. C ONCLUSION
In this paper, we extend the security definitions of MPC to account for data poisoning attacks when training machine learning models privately.
We consider a novel adversarial model who can manipulate the training data of a subset of owners and control a subset of servers in the MPC.
We then propose SafeNet, which performs ensembling in MPC, and show that our design has provable robustness and privacy guarantees, beyond those offered by existing approaches.
We evaluate SafeNet using logistic regression and neural networks models trained on five datasets by varying the distribution similarity across data owners.
We consider both end-to-end and transfer learning scenarios.
We demonstrate experimentally that SafeNet achieves even higher robustness than its theoretical analysis against backdoor and targeted poisoning attacks, at a significant performance improvement in the training time and communication compared to prior MPC frameworks. VII. A CKNOWLEDGMENTS
We thank Nicolas Papernot and Peter Rindal for helpful discussions and feedback.
This research was sponsored by the U.S. Army Combat Capabilities Development Command Army Research Laboratory under Cooperative Agreement Number W911NF-13-2-0045 (ARL Cyber Security CRA) and by the Department of Defense Multidisciplinary Research Program of the University Research Initiative (MURI) under contract W911NF-21-1-0322.
A PPENDIX A S AFE N ET A NALYSIS
In this section we first provide a detailed proof on the size of the validation dataset D val such that all clean models clear the filtering stage of the training phase of our framework.
We then provide a proof on achieving lower bounds on the test accuracy of our framework given all clean models are a part of the ensemble.
The main idea of deriving the minimum size of D val uses the point that the errors made by a clean model on a clean subset of samples in D val can be viewed as a Binomial distribution in ( m − t ) n and p , where n denotes the size of the validation dataset D v k contributed by an owner C k .
We can then upper bound the total errors made by a clean model by applying
Chernoff bound and consequently compute the size of D val .
Lemma 5.
Let A p soc be an adversary who poisons t out of m data owners and corrupts T out of N servers, and thus contributes t poisoned models to ensemble E , given as output by Algorithm 1.
Assume that Π train securely realizes functionality F pTrain and every clean model in E makes an at most where ϕ is the filtering threshold.
If the validation dataset has at least (2+ δ ) m log1 /ϵ
δ 2 ( m − t ) p samples and 0 ≤ t < m (1 − ϕ − p ) (1 − p ) , then all clean models pass the filtering stage of the training phase with probability at least 1 − ϵ , where δ = (1 − ϕ ) m − t ( m − t ) p − 1 and ϵ denotes the failure probability.
Proof.
Assume that each owner contributes equal size validation dataset D v k of n samples, then the combined validation set
D val collected from m data owners is comprised of mn i.i.d.
samples.
However, given an adversary A psoc from our threat model, there can be at most t poisoned owners contributing tn poisoned samples to D val .
We define a Bernoulli random variable as follows:
[Equation]
where X i denotes if a clean model makes an error on the i th clean sample in the validation dataset.
Then there are
Bin (( m − t ) n, p ) errors made by the clean model on the clean subset of samples in D val .
Note that, a model passes the filtering stage only when it makes ≥ ϕmn correct predictions.
We assume that the worst case where the clean model makes incorrect predictions on all the tn poisoned samples present in D val .
As a result, the clean model must make at most (1 − ϕ )
mn − tn errors on the clean subset of D val with probability 1 − ϵ .
We can upper bound the probability the model makes at least (1 − ϕ )
mn + 1 − tn errors with a multiplicative Chernoff bound with δ > 0 :
[Equation]
where µ = ( m − t ) np (the mean of Bin ( mn − tn, p ) ) and δ = (1 − ϕ ) m − t ( m − t ) p .
The chernoff bound gives that the probability the clean model makes too many errors is at most e [Equation]
= ϵ .
Then it suffices to have this many samples:
[Equation]
where ϵ denotes the failure probability and t < m (1 − ϕ − p ) (1 − p ) .
The inequality on t comes from requiring δ > 0 .
As a visual interpretation of Lemma 5, Figure 6 shows the minimum number of samples required in the global validation dataset for varying number of poisoned owners t and error probability p .
We set the total models m = 20 , the failure probability ϵ = 0 . 01 and the filtering threshold ϕ
= 0 .
The higher the values of t and p , the more samples are required in the validation set.
For instance, for p = 0 . 20 and number of poisoned owners t = 8 , all clean models pass the filtering stage with probability at least 0 .
99 when the validation set size has at least 60 samples.
We use a similar strategy as above to compute the lower bound on the test accuracy.
On a high level, the proof follows [Figure] by viewing the combined errors made by the clean models as a Binomial distribution Bin ( m − t, p ) .
We can then upper bound the total errors made by all the models in the ensemble by applying Chernoff bounds and consequentially lower bound the ensemble accuracy.
Theorem 6.
Assume that the conditions in Lemma 5 hold against adversary
A psoc poisoning at most t < m [Equation] p owners and that the errors made by the clean models are independent.
Then E correctly classifies new samples with probability at least p c
= (1 − ϵ )
[Equation]
, where µ ′ = ( m − t ) p and δ ′ = m −
[Equation]
2 µ ′ − 1 .
Proof.
Lemma 5 shows that, with probability > 1 − ϵ , no clean models will be filtered during ensemble filtering.
Given all clean models pass the filtering stage, we consider the worst case where even the t poisoned models bypass filtering.
Now, given a new test sample, m − t clean models have uncorrelated errors each with probability at most p , the error made by each clean model can be viewed as a Bernoulli random variable with probability p
and so the total errors made by clean models follow a binomial X ∼ Bin ( m − t, p ) .
We assume that a new sample will be misclassified by all t of the poisoned models.
Then the ensemble as a whole makes an error if t + Bin ( m − t, p ) > m/ 2 .
We can then bound the probability this occurs by applying Chernoff bound as follows:
[Equation]
where µ ′ =
( m − t ) p is the mean of X and δ ′ =
m − 2 t 2 µ ′ − 1 > 0 .
Then the probability of making a correct prediction can be lower bounded by:
[Equation]
given the number of poisoned models
[Equation]
The inequality on t comes from the constraint δ ′ > 0 for the Chernoff bound to hold.
Note that, the above bound holds only when all the clean models pass the filtering stage, which occurs with probability at least 1 − ϵ by Lemma 5.
Then the bound on the probability of making a correct prediction by the ensemble can be written as:
[Equation]
A PPENDIX B R EALIZATION IN MPC
To instantiate SafeNet in MPC, we first describe the required
MPC building blocks, and then provide the SafeNet training and secure prediction protocols.
1) MPC Building Blocks:
The notation (cid:74) x (cid:75) denotes a given value x secret-shared among the servers.
The exact structure of secret sharing is dependent on the particular instantiation of the underlying MPC framework [4], [14], [22], [23], [35], [46].
We assume each value and its respective secret shares to be elements over an arithmetic ring Z 2 ℓ .
All multiplication and addition operations are carried out over Z 2 ℓ .
We express each of our building blocks in the form of an ideal functionality and its corresponding protocol.
An ideal functionality can be viewed as an oracle, which takes input from the parties, applies a predefined function f on the inputs and returns the output back to the parties.
The inputs and outputs can be in clear or in (cid:74) ·
(cid:75) -shared format depending on the definition of the functionality.
These ideal functionalities are realized using secure protocols depending on the specific instantiation of the MPC framework agreed upon by the parties.
Below are the required building blocks:
Secure Input Sharing.
Ideal Functionality F shr takes as input a value x from a party who wants to generate a (cid:74) · (cid:75) -sharing of x, while other parties input ⊥ to the functionality.
F shr generates a (cid:74) · (cid:75) -sharing of x and sends the appropriate shares to the parties.
We use Π sh to denote the protocol that realizes this functionality securely.
Secure Addition.
Given (cid:74) · (cid:75) -shares of x and y , secure addition is realized by parties locally adding their shares (cid:74) z (cid:75) =
(cid:74) x
(cid:75) + (cid:74)
y (cid:75) , where z = x + y .
Secure Multiplication:.
Functionality F mult takes as input (cid:74) · (cid:75) - shares of values x and y , creates (cid:74) · (cid:75)
-shares of z = xy and sends the shares of z to the parties.
Π mult denotes the protocol to securely realize F mult .
Secure Output Reconstruction.
F op functionality takes as input (cid:74) · (cid:75) -shares of a value x and a commonly agreed upon party id pid in clear.
On receiving the shares and pid , F op reconstructs x and sends it to the party associated to pid .
Secure Comparison.
F comp functionality takes as input a value a in (cid:74) · (cid:75) -shared format.
F comp initializes a bit b = 0 , sets b = 1 if a > 0 and outputs it in (cid:74) · (cid:75) -shared format.
Protocol Π comp is used to securely realize F comp .
Secure Zero-Vector.
F zvec functionality takes as input a value
L in clear from the parties.
F zvec constructs a vector z of all zeros of size L and outputs (cid:74) · (cid:75) -shares of z .
Π zvec denotes the protocol that securely realizes F zvec .
Secure Argmax.
F amax functionality takes as input a vector x in (cid:74) · (cid:75) -shared format and outputs (cid:74) · (cid:75) -shares of a value OP , where OP denotes the index of the max element in vector x .
Π amx denotes the protocol that securely realizes F amax .
2) ML Building Blocks: We introduce several building blocks required for private ML training, implemented by existing MPC frameworks [14], [73], [75], [92]:
Secure Model Prediction.
F M pred functionality takes as input a trained model M and a feature vector x in (cid:74) · (cid:75)
-shared format.
F M pred then computes prediction Preds =
M ( x ) in onehot vector format and outputs (cid:74) · (cid:75) -shares of the same.
Π M pred denotes the protocol which securely realizes functionality
F M pred .
Secure Accuracy.
F acc functionality takes as input two equal length vectors y pred and y in (cid:74) · (cid:75) -shared format.
F acc then computes the total number matches (element-wise) between the two vectors and outputs # matches | y | in (cid:74) · (cid:75) -shared format.
Π acc denotes the protocol which securely realizes this functionality.
3) Protocols:
We propose two protocols to realize our
SafeNet framework in the SOC setting.
The first protocol Π train describes the SafeNet training phase where given (cid:74) · (cid:75) - shares of dataset D v k and model M k , with respect to each owner C k , Π train outputs (cid:74) · (cid:75) -shares of an ensemble E of m models and vector b val .
The second protocol Π pred describes the prediction phase of SafeNet, which given (cid:74) · (cid:75) -shares of a client’s query predicts its output label.
The detailed description for each protocol is as follows:
SafeNet Training.
We follow the notation from Algorithm 1.
Our goal is for training protocol Π train given in Figureto securely realize functionality F pTrain (Figure 2), where the inputs to F pTrain are (cid:74) · (cid:75) -shares of D k
= D v k and a k = M k , and the corresponding outputs are (cid:74) · (cid:75) -shares of O = E and b val .
Given the inputs to Π train , the servers first construct a common validation dataset (cid:74) D val (cid:75) = ∪ mk =1 (cid:74) D v k (cid:75) and an ensemble of models (cid:74) E (cid:75) = { (cid:74) M k (cid:75) } mk =1 .
Then for each model M k ∈ E , the servers compute the validation accuracy (cid:74)
AccVal k (cid:75) .
The output (cid:74) AccVal k (cid:75) is compared with a pre-agreed threshold ϕ to obtain a (cid:74) · (cid:75) -sharing of b val k , where b val k
=
if AccVal k > ϕ .
After execution of Π train protocol, servers obtain (cid:74) · (cid:75) -shares of ensemble E and vector b val .
The security proof of Π train protocol as stated in Theoremin Section III-C is given in Appendix C.
SafeNet Prediction.
Functionality F pred takes as input party id cid , (cid:74) · (cid:75) -shares of client query x , vector b val and ensemble
E = { (cid:74) M k
(cid:75) }
mk =1 and outputs a value OP , the predicted class label by ensemble E on query x .
Protocol Π pred realizes F pred as follows:
Given (cid:74) · (cid:75) -shares of x , b val and ensemble E , the servers initialize a vector z of all zeros of size L .
For each model M k in the ensemble
E , the servers compute (cid:74) · (cid:75) -shares of the prediction Preds =
M k ( x ) in one-hot format.
The element b val k in vector b val is multiplied to each element in vector Preds .
The (cid:74) Preds (cid:75) vector is added to (cid:74) z (cid:75) to update the model’s vote towards the final prediction.
If b val k = 0 , then after multiplication vector
Protocol Steps: The perform the following b
Protocol Steps: The the following:
– Execute Π zvec protocol with L as the input, where L denotes the number of distinct class labels and
[Table]
Preds is a vector of zeros and does not contribute in the voting process towards the final prediction.
The servers then compute the argmax of vector (cid:74) z (cid:75) and receive output (cid:74) OP (cid:75) from Π amx , where OP denotes the predicted class label by the ensemble.
The appropriate (cid:74) · (cid:75) -shares of OP is forwarded to the client for reconstruction.
Theorem 7.
Protocol Π pred is secure against adversary
A psoc who poisons t out of m data owners and corrupts T out of
N servers.
Proof.
The proof is given below in Appendix C.
For concise security proofs, we assume the adversary
A psoc performs a semi-honest corruption in the SOC paradigm, but our proofs can also be extended to malicious adversaries in the MPC.
We prove that protocol Π train is secure against an adversary of type A psoc .
Towards this, we first argue that protocol Π train securely realizes the standard ideal-world functionality F pTrain .
We use simulation based security to prove our claim.
Next, we argue that the ensemble E trained using Π train protocol provides poisoning robustness against A psoc .
Theorem 2.
Given a privacy-preserving machine learning training framework that tolerates T out of N server corruptions, and an error bound of at most p on each clean model in the ensemble, protocol Π train is secure against adversary
A psoc who poisons t < m 2 1 − 2 p 1 − p out of m data owners and corrupts
T out of N servers.
Proof.
Given the training framework securely realizes each of the building block used in protocol Π train , we now argue Π train securely realizes functionality F pTrain .
Let A psoc be a real-world adversary that semi-honestly corrupts T out of N servers at the beginning of the protocol Π train .
We now present the steps of the ideal-world adversary (simulator) S f for A psoc .
Note that, in the semi-honest setting S f already posses the input of A psoc and the final output shares of b val .
S f acts on behalf of N − T honest servers, sets their shares as random values in Z 2 ℓ and simulates each step of Π train to the corrupt servers as follows:
– No simulation is required to construct (cid:74) · (cid:75) -shares of ensemble E and validation dataset D val as it happens locally.
– S f simulates messages on behalf of honest servers as a part of the protocol steps of Π zvec with public value m as the input and eventually sends and receives appropriate (cid:74) · (cid:75) -shares of b val to and from A psoc .
– For k ∈ [1 , m ] : – S f simulates messages on behalf of honest servers, as a part of the protocol steps of Π M pred , with inputs to the protocol as (cid:74) · (cid:75) -shares of M k and D val and eventually sends and receives appropriate (cid:74) · (cid:75) -shares of PREDS k to and from A psoc .
– S f simulates messages on behalf of honest servers, as a part of the protocol steps of Π acc , with inputs to the protocol as (cid:74) · (cid:75) -shares of PREDS k and y D val and eventually sends and receives appropriate (cid:74) · (cid:75) -shares of AccVal k to and from A psoc .
– No simulation is required for subtraction with threshold ϕ as it happens locally.
– S f simulates messages on behalf of honest servers, as a part of the protocol steps of Π comp , with inputs to the protocols as (cid:74) · (cid:75) -shares of AccVal − ϕ and at the end S f instead sends the original shares of b val k as shares of b ′ associated to A psoc .
– No simulation is required to assign (cid:74)
b val k (cid:75) =
(cid:74) b ′ (cid:75) .
The proof now follows from the fact that simulated view and real-world view of the adversary are computationally indistinguishable and Π train securely realizes functionality F pTrain .
Now given the output of Π train protocol is an ensemble
E , we showed in the proof of Theorem 6 that E correctly classifies a sample with probability at least p c .
As a result the underlying trained model also provides poisoning robustness against A psoc .
We use a similar argument to show protocol Π pred is secure against adversary A psoc .
Theorem 7.
Protocol Π pred is secure against adversary
A psoc who poisons t out of m data owners and corrupts T out of
N servers.
Proof.
Let A psoc be a real-world adversary that poisons t out of m owners and semi honestly corrupts T out of N servers at the beginning of Π pred protocol.
We present steps of the ideal-world adversary (simulator) S f for A psoc .
S f
on behalf of the honest servers, sets their shares as random values in Z 2 ℓ and simulates each step of Π pred protocol to the corrupt servers as follows:
– S f simulates messages on behalf of honest servers as a part of the protocol steps of Π zvec with public value L as the input and eventually sends and receives appropriate (cid:74) · (cid:75) -shares of z to and from A psoc .
– For k ∈ [1 , m ′ ] : – S f simulates messages on behalf of honest servers, as a part of the protocol steps of Π M pred , which takes input as (cid:74) · (cid:75) -shares of M k and x .
S f eventually sends and receives appropriate (cid:74) · (cid:75) -shares of Preds to and from A psoc .
– For every multiplication of (cid:74) b val k (cid:75) with respect to each element in Preds , S f simulates messages on behalf of honest servers, as a part of the protocol steps of Π mult , which takes input as (cid:74) · (cid:75) -shares of Preds j and b val k .
S f eventually sends and receives appropriate (cid:74) · (cid:75) -shares of b val k
× Preds j to and from A psoc .
– No simulation is required to update (cid:74) z (cid:75) as addition happens locally.
– S f simulates messages on behalf of honest servers, as a part of the protocol steps of Π amx , which takes input as (cid:74) · (cid:75) -shares of z .
At the end S f instead forwards the original (cid:74) · (cid:75) -shares of OP associated to A psoc .
The proof now simply follows from the fact that simulated view and real-world view of the adversary are computationally indistinguishable.
Poisoning robustness argument follows from the fact that the ensemble E used for prediction was trained using protocol Π train which was shown to be secure against
A psoc in Theorem 2.
This concludes the security proofs of our training and prediction protocols.
A. Inference phase in Transfer Learning Setting
We provide a modified version of SafeNet’s Inference algorithm in the transfer learning setting, to improve the running time and communication complexity of SafeNet.
Algorithmprovides the details of SafeNet’s prediction phase below.
Input: Secret-shares of backbone model M B , ensemble of m fine-tuned models E = {M 1 , . . .
, M m } , vector b val and client query x .
// MPC computation in secret-shared format
– Construct vector z of all zeros of size L , where L denotes the number of distinct class labels.
– Run forward pass on M B with input x upto its last l layers, where p denotes the output vector from that layer.
– For k ∈ [1 , m ] : - Run forward pass on the last l layers of M k with input as p .
Let the output of the computation be Preds , which is one-hot encoding of the predicted label.
- Multiply b val k to each element of Preds . -
Add z = z + Preds .
– Run argmax with input as z and obtain OP as the output.
return
B. Training with Computationally Restricted Owners In
this section we provide a modified version of SafeNet’s
Training Algorithm, to accommodate when a subset of data owners are computationally restricted, i.e., they can not train their models locally.
Algorithm 4 provides the details of
SafeNet’s training steps below.
A PPENDIX E A DDITIONAL E XPERIMENTS A. Evaluation of SafeNet Extensions
a
) Integration Testing:
Here, we evaluate the performance of SafeNet by varying the concentration parameter α to manipulate the degree of data similarity among the owners
The experiments are performed with the same neural network architecture from Section IV-G on the Fashion dataset.
Figure 9 gives a comprehensive view of the variation in test accuracy and attack success rate for backdoor and targeted attacks over several values of α .
We observe that as α decreases, i.e., the underlying data distribution of the owners becomes more non-iid, the test accuracy of SafeNet starts to drop.
This is expected as there will be less agreement between the different models, and the majority vote will have a larger chance of errors.
In such cases it is easier for the adversary to launch an attack as there is rarely any agreement among the models in the ensemble, and the final output is swayed towards the target label of attackers’ choice.
Figure 9 shows that for both targeted and backdoor attacks, SafeNet holds up well until α reaches [Figure] extremely small values ( α = 0 . 1 ), at which point we observe the robustness break down.
However, the design of SafeNet allows us to detect difference in owners’ distributions at early stages of our framework.
For instance, we experiment for α = 0 .
1 and observe that the average AccVal accuracy of the models is 17% .
Such low accuracies for most of the models in the ensemble indicate non-identical distributions and we recommend not to use SafeNet in such cases.
b) Low Resource Users:
We instantiate our Fashion dataset setup in the 3PC setting and assume 2 out of 10 data owners are computationally restricted.
We observe SafeNet still runs 1 . 82 × faster and requires 3 .
53 × less communication compared to the existing PPML framework, while retaining its robustness against poisoning and privacy attacks.
B. Logistic Regresssion, Multiclass Classification
We use the same strategies for the Backdoor and Targeted attacks on the MNIST dataset.
For BadNets, we select the initial class y s = 4 and the target label y t = 9 , and use the same y t = 9 for the targeted attack.
Table IV provides a detailed analysis of the training time, communication, test accuracy, and success rate for both frameworks, in presence of a single poisoned owner.
The worst-case adversarial success for SafeNet is in Figure 10.
The slow rise in the success rate of the adversary across multiple attacks shows the robust accuracy property of our framework translates smoothly for the case of a multi-class classification problem.
[Figure]
u cce
[Figure]
a t e ( i n
[Figure]
[Figure]
C. Evaluation on Deep Learning Models
Experiments on Fashion Dataset.
We present results on one and two layer deep neural networks trained on the Fashion dataset.
We perform the same set of backdoor and targeted attacks as described in Section IV.
Tables V and VI provide detailed analysis of the training time, communication, test accuracy, and success rate for traditional PPML and SafeNet frameworks.
We observe similar improvements, where for instance in the 4PC setting, SafeNet has 42 × and 43 × improvement in training time and communication complexity over the PPML framework, for n = 10 epochs for a two hidden layer neural network.
Figure 11 shows the worst-case attack success in SafeNet (attacker can choose any subset of corrupted owners per sample).
Results are similar to Figure 5. s
Backdoor Attack
Targeted Attack
[Table] [Figure] [Figure] [Table]
[Figure]
[Table]
[Figure]
[Table] [Figure] [Figure] [Table]
time and communication reported for is
[Table] [Table]
[Figure]
Experiments on Adult Dataset.
We use a similar attack [Figure] strategy as used for logistic regression model in Section IV-E.
We observe that no instance is present with true label y
=for feature capital-loss = 1 .
Consequently, we choose a set of k = 100 target samples { x ti } ki =1 with true label y s = 0 , and create backdoored samples { Pert ( x ti ) , y t = 1 } ki =1 , where
Pert ( · ) function sets the capital-loss feature in x t to 1 .
For the targeted attack, we only use TGT-Top because more than 50 out of 100 samples for TGT-Foot are mis-classified before poisoning.
Table VII provides the training time and communication complexity of both PPML and SafeNet frameworks.
Figure 12 (a) and (b) provide the success rates in both frameworks and show the resilience of SafeNet against backdoor and targeted attacks.
D. Comparison with Federated Learning
We experimentally compare SafeNet to the federated learning-based approach of Cao et al.
[17], who also gave provable robustness guarantees in the federated averaging scenario.
We instantiate their strategy for EMNIST dataset and compare their Certified Accuracy metric to SafeNet’s, with m = 100 data owners, k
= { 2 , 4 } and FedAvg as the base algorithm.
To ensure both approaches have similar inference
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
times, we fix the ensemble size to 100 models, each trained using federated learning with 50 global and local iterations.
Figure 13 shows that SafeNet consistently outperforms [17], in terms of maintaining a high certified accuracy in the presence of large poisoning rates.
Moreover, their strategy is also particularly expensive at training time when instantiated in MPC.
During training, their approach requires data owners to interact inside MPC to train models over multiple rounds.
By contrast, SafeNet only requires interaction in MPC at the beginning of the training phase, making it significantly faster.