Model Extraction Attacks on Split Federated Learning
Anonymous Author(s)
AffiliationAddress email
Abstract
Federated learning (FL) is a popular collaborative learning scheme involvingmultiple clients and a server.
FL focuses on client’s data privacy but exposesinterfaces for Model Extraction (ME) attacks.
As FL periodically collects andshares model parameters, a malicious client can download the latest model and thussteal model Intellectual Property (IP).
Split Federated Learning (SFL), a recentvariant of FL, splits the model into two, giving one part of the model (clientside model) to clients, and the remaining part (server-side model) to the server.
While SFL was primarily designed to facilitate training on resource-constraineddevices, it prevents some ME attacks by blocking prediction queries.
In this work,we expose the vulnerability of SFL and show how ME attacks can be launchedby malicious clients querying the gradient information from server-side.
Wepropose five ME attacks that differ in the gradient usage in data crafting, generating,gradient matching and soft-label crafting as well as in the attacker data availabilityassumptions.
We show that the proposed ME attacks work exceptionally well for
SFL.
For instance, when the server-side model has five layers, our proposed MEattack can achieve over 90% accuracy with less than 2% accuracy degradation with
VGG-11 on CIFAR-10.
Introduction
Federated Learning (FL) [McMahan et al., 2017] has become increasingly popular thanks to its abilityto protect users’ data privacy and comply with General Data Protection Regulation (GDPR) policy.
In
FL, clients locally update their model copies and the FL server collects them by averaging the modelparameters, then distributing the averaged model again to its clients.
Such a setting only allows modelparameters to be shared with the server and direct data sharing is avoided.
One drawback of FL isits clients need to train the entire model locally, which is usually challenging for resource-limitededge devices.
As the countermeasure, Split Federated Learning (SFL) scheme [Thapa et al., 2020] isproposed as a variant of FL.
In SFL, a neural network is split into a client-side model and a server-sidemodel, where the client-side model is shared among multiple clients and processed locally on theirdevices.
During training, clients offload the intermediate activations to server, where the heavy-dutycomputation is performed at the heavy-duty server and the computed gradients are sent back to clients.
SFL follows the same model averaging routine as FL to synchronize the model.
SFL avoids collectingclients’ raw data and also reduces the computational overhead at the client-end.
In addition to the computation advantage introduced by SFL, it can also provide model IP protectionwhich is absent in FL.
The high training cost of high-performance NN makes the NN model a valuable
Intellectual Property (IP).
Unlike FL which handles the entire NN over to the clients, SFL preservesthe server-side model which prevents potential IP theft (Fig. 1 (a)).
Moreover, according to ourinvestigation, SFL shows resistance to Model Extraction (ME) attack [Tramèr et al., 2016, Jagielskiet al., 2020].
In an ME attack, the model IP can be acquired by querying a publicly accessible
[Figure]
Split Federated Learning (b)
[Figure]
Split Federated Learning (b) ℒ
[Figure]
prediction API of the model.
Prior ME attacks cannot be applied to SFL as the protocol does notallow prediction query access, as illustrated in Fig.
1 (b).
As prior ME attacks fail in SFL, we propose SFL-specific ME attacks in this work.
Assumingthe client-side model is a white box, the attacker posing as a participant client can get gradientinformation of inputs sent to the server.
We propose five variants of ME attacks on SFL enumerated as
Craft-ME, GAN-ME, GM-ME, Train-ME, and SoftTrain-ME.
These ME attack variants extensivelycover different gradient usage, including data crafting, data generating, gradient matching andsoft label crafting, as well as assumptions on data such as no data (including randomly generatednoise data)
[Truong et al., 2021], only auxiliary data (out-of-distribution data) and training data (indistribution data).
We consider both train-from-scratch and fine-tuning SFL applications as gradientconsistency is different.
We benchmark the performance of five ME attacks on SFL for these twocases and show that ME attacks can succeed without any data in fine-tuning applications.
We alsoshow that ME performance strongly correlates with the #layers in the server-side model and fail whenthe #layers in the server-side model increase.
However, increasing layers in server-side model reduceslayers in client-side model, which compromises clients’ privacy.
Finally, we conclude that using L1regularization during training can improve SFL schemes’ resistance to ME attacks.
In summary, wemake the following contributions:
• We define the unique threat model in SFL and propose five ME attacks.
These attacks
differ in the usage of gradients (data crafting, data generating, gradient matching and soft label crafting) and data assumptions (no data, only auxiliary data and training data).
To our knowledge, this is the first work that studies ME attacks for SFL.
• We study the performance of the proposed attacks and find that even when the attacker has
no data, the model can be extracted with high accuracy.
With auxiliary data or with training
data, ME attack performance can be further improved.
For a 5-layer-in-server SFL, the
strongest ME attack can derive a surrogate model with over 90% accuracy, and less than 2%
accuracy degradation compared to the original VGG-11 on CIFAR-10 model.
• We find the ME attack performance decreases when more layers are present in server-side
model.
However, such a split model configuration compromises clients’ privacy.
We alsoshow resistance to ME attack can be improved by regularizing the client-side model.
Related Work
Model-split learning schemes.
The key idea for model-split learning schemes is to split the modelso that part of it is processed in the client and the rest is offloaded to the server.
This idea was firstproposed in Kang et al. [2017], Teerapittayanon et al.
[2017], Liu et al. [2018] for inference tasks.
Gupta and Raskar [2018] then extended this idea for split learning, a collaborative multi-client neuralnetwork training.
However, the round-robin design needed clients to learn sequentially and thus has ahuge disadvantage in terms of training time.
Split federated learning (SFL).
In this paper, we consider the SFL scheme where the clients processtheir local models in parallel followed by periodic synchronization as in FedAvg [McMahan et al.
This is the SFL-V2 scheme that is introduced in Thapa et al.
[2020] for its better accuracyperformance.
The detailed process is shown in Algorithm 1.
At the beginning of each epoch, serverperforms the synchronization of client-side model and sends the updated version to all clients.
Then,
Algorithm
Require:
For M clients, instantiate private training data ( X i , Y i ) for 1 , 2 , ..., M .
Server-side model S has N layers and client-side model C i has L − N layers.
1: initialize C i , S 2: for epoch t ← 1 to num_epochs do 3:
C ∗ = 1 M (cid:80) Mi
=1 C i {Model Synchronization} 4:
C i ← C ∗ for all
i 5: for step s ← 1 to num_batches do 6: for client i ← 1 to M in Parallel do 7: data batch ( x i , y i )
← ( X i , Y i ) 8:
A i =
C i ( W C i ; x i ) {Client forward; send A i to Server} 9: end for 10: for client i ← 1 to M in Sequential do 11:
L = L CE ( S ( W S ; A i ) , y i ) {Server forward} 12: ∇ A i L ← back-propagation {Server backward, send ∇ A i L to Client} 13:
Update W S ; 14: end for 15: for client i ← 1 to M in Parallel do 16: ∇ x i L ← back-propagation {Client backward} 17:
Update W C i ; 18: end for 19: end for 20: end for clients perform forward propagation locally till layer L − N (the last layer of client-side model),sending the intermediate activation A i to the server (line 8).
Server accepts the activation and labely i sent from clients, and uses them to calculate the loss and initiates the backward process (line 9).
The backward process (line 10) consists of several steps: server performs backward propagation onthe loss, updates server-side model and sends back gradient ∇ A i L to clients.
Clients then continuethe backward propagation on their client-side model copies and perform model updates accordingly.
While FedGKT [He et al., 2020] leaks prediction logits to clients, SFL does not, making it a promisingcandidate against ME attacks.
Model extraction (ME) attack.
ME attack targets model prediction service APIs and retrievesconfidence score or prediction label for given inputs.
The vulnerability of a DNN model to ME is firstshown in Tramèr et al.
Jagielski et al.
[2020] shows that high fidelity model extraction canbe achieved with fewer queries and the surrogate model can be used for launching more successfuladversarial attacks.
A successful ME attack not only breaches the model IP, but also makes the modelmore vulnerable to attacks.
ME attack also supports transferable adversarial attacks [Goodfellowet al., 2014], mainly targeted ones [Madry et al., 2017] against the victim model.
It can also be usedto perform bit-flip attacks [Rakin et al., 2019]; with hardware expertise, a few bit flips on modelparameters can degrade ResNet-18 model accuracy to below 1%.
Data privacy in SFL.
Similar to FL, SFL scheme also has data privacy concerns.
The most seriousone is its vulnerability to MI attacks.
In model-based MI attack [Fredrikson et al., 2015], the attackertrains an inverted version of client-side model and can directly reconstruct raw inputs from theintermediate activation.
Recent works [Vepakomma et al., 2020, Li et al., 2022] point out thisvulnerability and provide practical ways to mitigate MI.
However, defenses only work well if theclient-side model has enough number of layers.
Threat Model
3.1 Attacker assumptions.
Objectives.
According to Jagielski et al.
[2020], there are three model extraction (ME) attackobjectives: i) functional equivalence, ii) high accuracy, and iii) high fidelity.
However, achieving functional equivalence is difficult in practical applications.
Hence, most of the existing practical ME attacks on NN models focus on achieving high accuracy and fidelity.
To achieve the accuracygoal, the attacker wants to obtain a model that maximizes the prediction correctness.
To achieve thefidelity goal, the attacker wants to derive a model with a similar decision boundary as the victimmodel before launching adversarial attacks [Biggio et al., 2013].
Capabilities.
We assume the attacker acts as a client in a multi-client SFL scheme against theserver (model owner).
Consider the SFL scheme outlined in Fig.
We assume the entire model has atotal of L layers (or layer-like blocks, i.e. BasicBlock in ResNet) out of which the server processes Nlayers.
The attacker holds white-box assumption on the client-side model (consists of L − N layers),that is, it knows the exact model architecture and parameters for those layers.
The attacker holdsa grey-box assumption on the N -layer server-side model, that is, it knows its architecture and lossfunction while the model parameters are unknown.
Also, we assume server blocks the predictionqueries thus neither logits nor prediction labels are accessible by clients during training, but serverallows gradient queries to let client-side models be updated.
Based on a client’s activation A = C ( x )and its label y , gradient information ∇ A L is computed and sent back to clients.
In addition, weassume the attacker can perform gradient queries on any input x , including malicious ones.
3.2 Analysis
Partial model extraction problem.
Since the attacker already has a white-box assumption of theclient-side model, the attacker only needs to extract the server-side model to reveal the entire model.
This results in an easier problem setting.
We observe client-side model heavily regularizes the featurespace of its output (input of server-side model), making ME attacks easier to succeed, especiallywhen N is small.
As shown in Fig.
2, on a VGG-11 [Simonyan and Zisserman, 2014] model on
CIFAR-10 dataset, as N becomes smaller, tSNE embeddings of intermediate features with differentlabels are easier to distinguish.
For N = 2 , ME attack is as simple as separating different clusterswith a linear layer.
To show that the extracting part of the model with ME is easier, we study existing ME attacks onthe server-side model, by assuming that prediction access is allowed .
Specifically, we investigate
CopyCat CNN [Correia-Silva et al., 2018], Knockoff-random
[Orekondy et al., 2019] and data-free
ME [Truong et al., 2021].
As shown in Fig.
3 (a), with auxiliary data (CIFAR-100) and enough querybudget, both attacks derive a surrogate model with very high accuracy even for a large N setting.
Moreover, attacker with no data can also succeed with data-free ME as shown in Fig.
3 (a).
When thequery budget is equal to 2 million, the data-free ME can extract the model with high accuracy evenwhen N is equal to 5.
Consistency of gradient query.
For fine-tuning applications [Park et al., 2021], attackers getconsistent gradient information from gradient query, as server-side model parameters are frozenor updated with a very small learning rate.
Gradient consistency is very beneficial for ME attacks.
However, for a training-from-scratch usage, queries to SFL model obtain inconsistent gradientinformation as the server-side model drastically changes during training.
As shown in Fig.
3 (c), forthe same query input, the gradient is drastically different in different epochs.
Proposed Model Extraction Attack
Previously, Milli et al.
[2019] demonstrated that using gradients to reveal one-layer linear transformation is trivial.
Given f ( x ) =
W T x , one can directly infer W from a single gradient query given that
[Table]
[Table]
Correct Label (1K) Confidence Score (1K) Correct Label (10K) Confidence Score (10K)
[Table]
Avg.
Change (x
[Table]
Inconsistent
Gradient
[Table]
x 0 .
Epoch
[Table]
N
[Table]
Correct Label (1K) Confidence Score (1K) Correct Label (10K) Confidence Score (10K)
[Table]
[Table]
Knockoff
[Table] [Table]
N =
[Table]
N
[Table]
N [Table] EntireModel
H u n d r e d s
CopyCat
[Table] [Table]
[Table] [Table]
[Table]
Inconsistent
[Table] [Table]
[Table] [Table]
Correct Label (1K
[Table]
Confidence Score
[Table]
1K) Correct Label (10K) Confidence Score (10K)
[Table]
CopyCat
[Table] [Table]
[Table] [Table]
[Table] [Table]
N
[Table]
N
[Table]
N
[Table]
EntireModel
H u n d r e d s
CopyCat
[Table]
Avg.
Change (x10-6)
Inconsistent Gradient
x 0 .
Epoch    124477093116 139x 0 .
Avg.
Absolute Change (x10-4)
Epoch (c)
W T = ∇ x f ( x ) .
However, using gradient only can go no further than one layer.
Milli et al.
[2019]shows that to recover a two-layer ReLU network of the form f ( x ) = (cid:80) h n =1 g
x ) i W i
A T i x , whereg ( x )
= 1 { Ax > 0 } , A is of R
h × d and W is of R h , using input gradient can recover the absolutevalue of normal vectors | W i
A i | for i ∈ [ h ] .
In order to get the sign information of W i A i , predictionquery is required which is not supported by SFL’s threat model.
So in this paper, we investigate approximate ME attacks that differ in the data assumptions andgradient usage.
We propose five ME attacks as shown in Table 1.
We also include a naive baselinescheme that train the surrogate model from scratch without access to either the client-side model orgradients.
Despite the differences, the five proposed attack methods follow the same strategy, that is,they all train a randomly initialized surrogate server-side model from scratch. We provide a detailedillustration of proposed five attacks in Appendix A.2.
[Table]
4.1 ME attacks without training data
We first consider two cases where the attacker does not have training data: one is that attacker doesnot use any data or use randomly generated noise data. This case is motivated by Truong et al.
[2021]which shows that when an attacker does not have a similar enough dataset, it is better not to use it atall.
The other case is when the attacker has an auxiliary dataset with different labels from the victim’straining data (for example, CIFAR-10 and CIFAR-100).
Crafting model extraction (Craft-ME).
Inspired by Han et al.
[2018], where data-label pairs(referred to as instances) with small-loss are shown to present useful guidance for knowledgedistillation, we propose a simple method to craft small-loss instances using gradient queries and usethem to train the surrogate model.
We initialize random input x r for every class label c , and use thegradient ∇ x r L to update x r .
For each input, updating is repeated for a number of steps.
By varyinglabel c , a collection of small-loss instances is derived during SFL training.
Then, a surrogate model istrained from scratch on these small-loss instances.
GAN-based model extraction (GAN-ME).
Recent work [Truong et al., 2021] proposes a GANbased approach for data-free ME.
The key idea is to use a generator G to continually feed fake inputsto the victim model V and surrogate model S , and use confidence score matching to let S approach
V .
However, the confidence score matching needs prediction query which is not allowed in our case.
Thus, we adapt the GAN-based method to gradient-query-only case and propose a two-step method:
First, a conditional-GAN (c-GAN) model G ( z | c ) is initialized.
The attacker trains the generatorduring victim model’s training by generating fake data x f and label c and performing gradient queriesto update G .
After the training is done, generator G is used to supply small-loss instances ( x f , c ) totrain the surrogate model (the unknown part).
We observe a serious mode collapse problem duringthe GAN training.
So we utilize the distance-aware training introduced in Yang et al.
[2019], toencourage the c-GAN to generate more diverse small-loss instances.
In the new method, the trainingof the generator is not based on a min-max game, or on traditional GAN training.
Instead, it simplytrains the generator toward minimizing cross-entropy loss.
While the generator G fails to generatenatural-looking inputs even upon convergence, it generates abundant small-loss instances for everylabel, and divergence loss helps it generate a good variety of outputs.
During SFL training, thegenerator can adjust itself to the changing server-side model.
Gradient matching model extraction (GM-ME).
Gradient matching (GM) in ME attack has beeninvestigated in Jagielski et al.
[2020], Milli et al.
[2019] and is used in combination with predictionquery to improve the extraction performance.
Since, in SFL, prediction query is not allowed, wenavigate this strict threat model’s restriction by adopting gradient matching (GM) loss.
For a givenlabel y i , GM loss has the following form:
[Equation]
(1) where, x i denote inputs, C denotes client-side model, S and V denotes the surrogate model andvictim model, respectively.
For each input, attacker would query gradients with different label y
i toget as much information as possible.
This attack performs extremely well for small N but degradessignificantly for a larger N .
Its performance also depends on the domain similarity between theauxiliary dataset and the victim dataset.
4.2 ME attacks with training data.
Next we discuss the case when the attacker has a subset of training data, corresponding to the strongestdata assumption .
Training-based model extraction (Train-ME).
For attackers with a subset of the training data,derivation of an accurate surrogate model can be done using supervised learning (through minimizingthe cross entropy loss on the available data).
We call this Train-ME, similar idea is also adopted in Fuet al.
[2022] to extract the entire model of the other party.
Train-ME only relies on the white-boxassumption of the client-side model, using it to initialize the surrogate model and does not need touse the gradient query at all.
Surprisingly, it is one of the most effective ME attacks.
Gradient-based soft label training model extraction (Soft-train-ME).
If gradient query is allowedand a subset of training data is available, the attacker can achieve better ME attack performancecompared to Train-ME.
To utilize gradients, a naive idea is to combine the GM loss with cross-entropyloss in Train-ME.
However, our initial investigation shows they are not compatible; the cross-entropyloss term usually dominates and the GM loss even hurts the performance.
An alternative approach isto use soft label.
We build upon the method in Gu et al.
[2020] which shows that gradient informationof incorrect labels is beneficial in knowledge distillation, and use it for surrogate model training.
Specifically, for each input x i , gradients of the ground truth label as well as incorrect labels arecollected (a total N C , where N C is the number of classes).
For an input x i with true label c , its softlabel q ki of k -th ( k ̸ = c ) label is computed as follows:
[Equation]
where, e k denotes flattened gradients of label k , q ki denotes soft label for the k-th label k and α is aconstant ( α > 0 . 5 ).
The derived ( x i , q i ) pair is then used in the surrogate model training in additionto the true label c (which is the only difference from the Train-ME).
Model Extraction Performance
In this section, we demonstrate the performance of the proposed ME attacks and the baseline attackon SFL schemes.
All experiments are conducted on a single RTX-3090 GPU.
For the SFL modeltraining, we set the total number of epochs to 200, and use SGD optimizer with a learning rate of0.05 and learning rate decay (multiply by factor of 0.2 at epochs 60, 120 and 160).
We assume allclients participate in every epoch with an equal number of training steps.
To perform ME attacks,the attacker uses an SGD optimizer with a learning rate of 0.02 to train the surrogate model and wereport the best accuracy and fidelity.
We evaluate accuracy of the surrogate model on the validationdataset.
We use the label agreement as fidelity, defined as the percentage of samples that the surrogateand victim models agree with over the entire validation dataset, as in Jagielski et al.
Detailedsettings for each ME attack are described in Appendix A.1.
5.1 ME attack on SFL with fine-tuning based training
We first perform the proposed ME attacks on fine-tuning SFL version with consistent gradient query.
Here we use a pre-trained model and set the number of gradient queries to 100K. On a victim VGG-11model on CIFAR-10 dataset, whose original accuracy is 91.89%, performance of all five ME attacksare shown in Table 2.
For each of the ME attacks, we vary hyper-parameters and report the onethat achieves the best attack performance; details on different hyper-parameters are included in Appendix A.3.
When N = 2 , all five ME attacks can achieve near-optimal accuracy and fidelityperformance.
For Craft, GAN and GM ME, the accuracy drops to around 80% when N is 5, it sharplydrops to below 40% when N is 8.
For Train and SoftTrain ME, accuracy slightly degrades when N is5, and reduces to around 70% when N is 8.
These results show that ME attack performance stronglycorrelates with the #layers in server-side model.
With an increasing #layers in server-side model, MEattack performance reduces as the extraction problem becomes harder with more unknown parametersand more complicated input feature space.
Observation.
Different attacks present different and interesting characteristics.
Craft-ME has asteady attack performance and can succeed even with a tight query budget.
GAN-ME needs a largequery budget to train the c-GAN generator towards convergence but can achieve better accuracyand fidelity than Craft-ME for N ≤ 5 .
GM-ME requires an auxiliary dataset that is similar to thetraining data.
If CIFAR-100 is used to attack CIFAR-10 model, GM-ME achieves almost perfectextraction for small N .
However, its performance degrades if MNIST or SVHN are used as auxiliarydatasets.
For a high N , the surrogate model fails to converge on the GM loss, and its extractionperformance suffers from a sharp drop.
For attacks with training data such as Train and SoftTrain
MEs, both accuracy and fidelity are much higher than attacks without training data.
When N ≥ 6 , SoftTrain-ME can achieve slightly better accuracy and fidelity than Train-ME.
[Table]
5.2 ME attack on training-from-scratch SFL
Next, we investigate the proposed ME attack performance in training-from-scratch SFL case.
A goodattack-time-window for gradient-based ME attacks is at the end of training when gradients do not varyas much and the model converges.
So for Craft, GAN and GM-ME, we launch the attack at epoch160 to get more consistent gradients.
As the model is updated by multiple clients, the percentage of malicious clients also affects the ME attack performance.
We found that with more benign clients(or fewer malicious inputs), the server-side model returns more consistent gradients to the attacker.
Attack performance for three attacks are shown in Table 2 for 10-client SFL training-from-scratchcase.
Results with other hyperparameter settings are included in Appendix A.4.
Observation.
Because of the poisoning effect of malicious inputs sent by the attacker in gradientbased attacks, in all the methods except Train-ME, the victim model’s accuracy suffers from a 2-3%degradation, resulting in a less accurate surrogate model.
For gradient-based attacks without trainingdata (Craft, GAN and GM MEs), we notice a sharp ( > 3% ) performance drop in both accuracyand fidelity compared to the consistent query case.
The sharp drop in ME attack performance iscaused by inconsistent gradients .
Take Craft-ME as an example.
Crafted inputs that have a smallloss at an earlier epoch of the training can have a large loss in the final model.
Training surrogatemodel with large amount of inconsistent information results in poor training accuracy.
Comparedto Craft-ME, GAN-ME is more robust to inconsistent gradients as the generator can adjust itselfto the change of server-side model, resulting in a better extraction performance.
However, when
N is larger, the generator does not converge well and its performance drops drastically.
Last butnot the least, GM-ME completely fails with inconsistent gradients, even for small N .
This impliesthat the GM loss is super sensitive to inconsistent gradients and is only effective in consistent querycases.
A comparison of SoftTrain-ME and Train-ME shows that SoftTrain’s advantage diminishesbecause of the poisoning effect.
Thus, Train-ME is more effective for an attacker with training datain training-from-scratch SFL.
5.3 ME attack performance for other architectures and datasets
We perform extensive analysis on the attack performance of the proposed ME schemes on otherarchitectures and datasets.
We use Train-ME attack, the best performer for both fine-tuning andtraining-from-scratch cases and use the accuracy as the performance measure.
To test ME attack ondifferent datasets, we perform Train-ME attack with 1K training data on VGG-11 model with Nset to 5, on MNIST [LeCun, 1998], FMNSIT [Xiao et al., 2017], SVHN [Netzer et al., 2011] and
CIFAR-100 datasets [Krizhevsky et al., 2009] (in addition to CIFAR-10).
As shown in Fig.
4 (a), forall datasets except CIFAR-100, ME attack achieves accuracy very close to the original.
For CIFAR100, the extracted accuracy is over 20% below the original because of its task difficulty.
Additionally,we also test Train-ME performance with 2% and 20% ImageNet training data of Mobilenet-V2 on
ImageNet dataset.
As shown in Fig.
4 (b), ME attack fails badly due to the complexity of ImageNetdataset [Deng et al., 2009], resulting in a high accuracy gap of 10% when N is set to 2.
For differentarchitectures, we choose Resnet-20, Resnet-32 [He et al., 2016] and Mobilenet-V2 [Sandler et al.
,2018] on CIFAR-10 dataset (with necessary adaptations).
For Resnet and Mobilenet family, weassign last 4 layer-blocks and 1 FC layer to server-side model.
As shown in Fig.
4 (c), with thesame proportion of layers (5 out of 11) being assigned to server-side model, ME attack is muchless effective on Resnet-20 than on VGG-11.
A comparison of the performance of Resnet-32 and
Mobilenetv2 with similar proportion of layers being assigned to server-side (5 out of 17 and 20,respectively), ME on Resnet-32 is also much worse than on MobilenetV2.
This indicates Resnetarchitecture is more resistant to ME attack.
Discussion
Our evaluation showed that ME attack performance drops with increasing #layers in server-sidemodel.
Thus, a simple idea to improve resistance to ME attack is to use a larger N .
However thisimplies that the #layers in client-side model would be smaller, thereby undermining clients’ dataprivacy.
Data privacy in SFL can be represented by the Mean Square Error (MSE) performance ofan MI attack as outlined in Li et al.
The implementation detail is included in Appendix B.1.
As shown in Fig.
4 (d), extracted accuracy decreases with a larger N , and MSE decreases too.
Thismeans MI attack can provide more precise reconstruction which compromises data privacy.
We willextensively consider the tradeoff between ME resistance and data privacy in future.
Defense Method.
We also investigate defensive methods against ME attacks on SFL.
According tothe analysis in Fig. 2, extraction is easier because client-side model is leaked to the attacker.
Thus, tomake ME attacks more difficult, we can restrict the client-side model’s feature extraction capabilities. Towards this goal, we apply L1 regularization with three different strength ( λ = 5e-5, 1e-4 and 2e-4)
(b) 1.
N=4 N=5 N=
N=
N o r m
E rr o r
MNIST
FMNIST SVHN
CIFAR0 0.
M S E
Fidelity
Area: Computational Overhead 76M 57M 19M 10M 0.
M S E
Fidelity
L1Norm
InvDistCorr Original
N=
[Figure]
Original Accuracy
[Figure]
[Figure]
Extracted Fidelity
[Figure]
Original
[Figure]
resistance
[Figure]
(b) 1.
[Figure]
N
[Figure]
N
[Figure]
N
[Figure]
N
[Figure]
N r m
E rr o r
MNIST
FMNIST SVHN
CIFAR0 0.
M S E
Fidelity
Area: Computational Overhead 76M 57M 19M 10M 0.
M S E
Fidelity
L1Norm
InvDistCorr Original
N=
[Figure]
N
[Figure]
N=
[Figure]
[Table]
% data 20% data
[Figure]
[Figure]
Extracted Accuracy
Original
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Figure]
M
[Table]
Improve
[Table]
resistance (b) (b) 1.
N=4 N=5 N=
N=
N o r m E rr
o r
MNIST
FMNIST SVHN
CIFAR0 0.
[Table]
M S E
Fidelity
Area: Computational Overhead 76M
10M
[Table]
Fidelity
L1Norm
InvDistCorr Original
[Table]
[Figure] [Figure]
[Table] [Table]
[Figure]
[Table]
(c) (b) 1.
N=4 N=5 N=
N=
N o r m E rr
o r
MNIST
FMNIST SVHN
CIFAR-
[Table]
[Figure]
[Table]
M S E
Fidelity
Area: Computational Overhead 76M 57M 19M 10M 0.
M S E
Fidelity
L1Norm
InvDistCorr Original
N=
[Figure]
r
[Figure]
d s
[Figure]
Extracted
[Figure]
L1Norm
[Figure]
Improve ME resistance
[Figure]
(b) 1.
N=4 N=5 N=
N=
N o r m E rr
o r
MNIST
FMNIST SVHN
CIFAR0 0.
[Figure]
M
[Figure]
E
Fidelity
Area: Computational Overhead 76M 57M
[Figure] [Figure]
M S E
Fidelity
L1Norm
InvDistCorr Original
N
[Figure]
on the client- model to penalize its weight magnitude.
As shown in Fig.
4 (e), this simple defense effectively improves the resistance to ME attack (specifically, Train-ME with 1K data) without hurting data privacy.
While there is some accuracy degradation, as shown in Appendix A.5, this demonstratesthe potential of using regularization to defend ME attacks.
Ablation Study
ME attack with non-IID data.
We consider the non-IID (independent and identically distributed)case where the attacker only has data from C classes of CIFAR-10.
Results presented in Appendix A.6show that ME attack performance is still good for C = 5 but degrades sharply when C = 2 .
Adversarial attack based on successful ME attack.
As mentioned before, the goal of ME attackis to launch more successful adversarial attacks.
We perform transfer adversarial attacks using asurrogate model extracted by the strongest Train-ME attack.
As shown in Appendix A.7, SFL withproper N achieves better resistance to adversarial attacks.
ME attack without architecture information.
In Appendix A.8, we investigate simple variants(longer, shorter, wider, and thinner) of the original architecture as the surrogate model architecture.
We find that the performance of ME attacks is similar for the different architectures – the exception is
GM-ME which fails for different surrogate architectures.
Conclusion
In this work, we study the model IP protection capability of SFL and its resistance to model extraction(ME) attack.
We propose five viable ME attack methods for the threat model where gradient queryis allowed but prediction query is not allowed.
For the case when there are enough number oflayers in server-side model, model IP can be protected well and transfer adversarial attack is notsuccessful.
However, data privacy could be compromised and so this factor needs to be consideredin the development of such schemes.
We also point out a possible way of defending such attacksthrough regularization and plan to expand on it in the near future.
Broader Impact.
This work points out the vulnerability of Split Federated Learning (SFL), to modelextraction attacks, and should prevent a naive adoption of SFL as a model IP protection method.
Webelieve that the attacks presented here would initiate research in the development of defense schemesto mitigate such attacks, help design a more robust SFL and possibly help in the design of neuralnetwork models that are inherently resilient to such attacks.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] or[N/A] .
You are strongly encouraged to include a justification to your answer either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section x.•
Did you include the license to the code and datasets?
[No] The code and the data areproprietary.•
Did you include the license to the code
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] We provide the limitations of the
proposed attacking methods by experiments described in Section 5.2 and Section 5.3.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] As
discussed in Section 1, this paper provides attacking methods to gain proprietary model
information which breaches Intellectual Property.
However, we intend to point out
the vulnerability of SFL scheme, which makes entities aware of the danger and hence
prevents them from being attacked.
We also provide potential defensive methods to
mitigate our proposed attacks.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] We will include
them in the supplemental material.
(b) Did you specify all the training details (e
g., data splits, hyperparameters, how theywere chosen)?
[Yes] We will open-source our code which includes all the required
information.
(c) Did you report error bars (e g., with respect to the random seed after running experiments multiple times)?
[No]
(d) Did you include the total amount of compute and the type of resources used (e.g., type
of GPUs, internal cluster, or cloud provider)?
[Yes]
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[No]
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[No]
e) Did you discuss whether the data you are using curating contains personally identifiableinformation or offensive content
[No
If you used crowdsourcing or conducted research with human subjects...
=(a)
Did you include the full text of instructions given to participants and screenshots, ifapplicable?
[N/A](
b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A](
c)
Did
you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?
N/ A]