Model Extraction Attacks on Split Federated Learning
Authors: Jingtao Li
Adnan Siraj Rakin
Xing Chen
Li Yang
Zhezhi He
Deliang Fan
Chaitali Chakrabarti
Venue: NeurIPS 2022 Submitted (2022)
Keywords: Split Federated Learning
Model Extraction Attack

Abstract: Federated learning (FL) is a popular collaborative learning scheme involving multiple clients and a server. FL focuses on client's data privacy but exposes interfaces for Model Extraction (ME) attacks. As FL periodically collects and shares model parameters, a malicious client can download the latest model and thus steal model Intellectual Property (IP). Split Federated Learning (SFL), a recent variant of FL, splits the model into two, giving one part of the model (client-side model) to clients, and the remaining part (server-side model) to the server. While SFL was primarily designed to facilitate training on resource-constrained devices, it prevents some ME attacks by blocking prediction queries. In this work, we expose the vulnerability of SFL and show how ME attacks can be launched by malicious clients querying the gradient information from server-side. We propose five ME attacks that differ in the gradient usage in data crafting, generating, gradient matching and soft-label crafting as well as in the attacker data availability assumptions. We show that the proposed ME attacks work exceptionally well for SFL.  For instance, when the server-side model has five layers, our proposed ME attack can achieve over 90% accuracy with less than 2% accuracy degradation with VGG-11 on CIFAR-10.

Model Extraction Attacks on Split Federated Learning
Anonymous Author(s) Affiliation Address email
Abstract
Federated learning (FL) is popular collaborative learning scheme involving multiple clients and server. FL focuses on client's data privacy but exposes interfaces for Model Extraction (ME) attacks. As FL periodically collects and shares model parameters, a malicious client can download the latest model and thus steal model Intellectual Property (IP). Split Federated Learning (SFL), a recent variant of FL, splits the model into two, giving one part of the model (client- side model) clients, and the remaining part (server-side model) the server. While SFL was primarily designed to facilitate training on resource-constrained devices, it prevents some ME attacks by blocking prediction queries. In this work, we expose the vulnerability of SFL and show how ME attacks can be launched by malicious clients querying the gradient information from server-side. We propose five ME attacks that differ in the gradient usage in data crafting, generating, gradient matching and soft-label crafting as well as in the attacker data availability assumptions. We show that the proposed ME attacks work exceptionally well for SFL. For instance, when the server-side model has five layers, our proposed ME attack can achieve over 90% accuracy with less than 2% accuracy degradation with VGG-11 on CIFAR-10.
2
10
11
13
14
15
17
Introduction
Federated Learning (FL.) [MMMhhan et al. 2017] has become increasingly popular thanks to its ability 20 to protect users' data privacy and comply with General Data Protection Regulation (GDPR) policy. In FL., clients locally update their model copies, and the FL server collects them by averaging the model parameters, then distributing the averaged model again to its clients. Such setting only allows model 23 parameters be shared with the server, and direct data sharing avoided. One drawback of FL is
21
22
24 its clients need to train the entire model locally, which is usually challenging for resource- limited edge devices. As the countermeasure, Split Federated Learning (SFL) scheme (Thapa al. 2020 is proposed as a variant of FL. In SFL, a neural network is split into client-side model and server-side model, where the client-side model is shared among multiple clients and processed locally on their devices. During training, clients offload the intermediate activations to server, where the heavy-duty computation is performed at the heavy-duty server and the computed gradients are sent back to clients.
25
26
27
28
29
30
SFL follows the same model averaging routine as FL. to synchronize the model. SFI. avoids collecting clients' raw data and also reduces the computational overhead at the client-end.
31
32 In addition to the computation advantage introduced by SFL, it can also provide model IP protection which is absent FL. The high training cost of high-performance NN makes the NN model valuable Intellecttal Property (IP). Unlike FL which handles the entire NN over to the clients, SFL preserves the sideer-side model which prevents potential IP theft (Fig. [ (a)). Moreover, according to our investigation, SFL shows resistance to Model Extraction (ME) attack (Tramer et al. [016]] Jagielski
33
34
35
36
al. 2020 In an ME attack, the model IP can be acquired by querying publicly accessible 37
Submitted to 36th Conference on Neural Information Processing Systems (NeurIPS 2022). Do not distribute.

---

White- Box
Parameter Unknown
Parameter Unknown
Activation A
Forward L
Gradient v,L
L
Backward
Prediction S(A)
Client-side model
Server-side model
Server-side model
Figure 1: Model Extraction (ME) attack in SFL. (a) The attacker knows the client-side model parameters but does not know the server-side model parameters. (b) Server-side model allows gradient query access but does not allow prediction queries.
38
prediction API of the model. Prior ME attacks cannot be applied to SFL as the protocol does not allow prediction query access, illustrated in Fig. (b).
40
As prior ME attacks fail SFL,, we propose SFL.-specific ME attacks in this work. Assuming the client-side model is white box, the attacker posing participant client can get gradient information inputs sent to the server. We propose five variants ME attacks on SFL enumerated as Craft-ME, GAN-ME, GM-ME, Train-ME, and SoffTrain-ME. These ME attack variants extensively cover different gradient usage, including data crafting, data generating, gradient matching and soft label crafting, as well as assumptions on data such as no data (including randomly generated noise data) [Trung et al., 2021], only auxiliary data (out-of-ding -ifooff--dtrbuutiin data) and training data (in- distribution data). We consider both Train-from-scratee and fine-tuning SFL applications gradient consistency different. We benchmark the performance of five ME attacks on SFL for these two cases and show that ME attacks can succeed without any data in fine-tuning applications. We also show that ME performance strongly correlates with the #layers in the server-side model and fail when the #layers in the server-side model increase. However, increasing layers in server-side model reduces layers in client-side model, which compromises clients' privacy. Finally, we conclude that using L1 regularization during training can improve SFL schemes' resistance ME attacks. In summary, we make the following contributions:
41
42
44
46
48
50
52
53
54
55
We define the unique threat model SFL and propose five ME attacks. These attacks differ in the usage of gradients (data crafting, data generating, gradient matching and soft label crafting) and data assumptions (no data, only auxiliary data and training data). To our knowledge, this is the first work that studies ME attacks for SFL.
56
57
58
59
We study the performance of the proposed attacks and find that even when the attacker has data, the model be extracted with high accuracy. With auxiliary data or with training data, ME attack performance can be further improved. For a 5-layer-in-server SFL, the strongest ME attack derive surrogate model with 90% accuracy, and less than 2% accuracy degradation compared to the original VGG-11 on CIFAR- 10 model.
60
51
83
64
We find the ME attack performance decreases when more layers are present in server-side model. However, such split model configuration compromises clients' privacy. We also show resistance to ME attack can be improved by regularizing the client-side model.
86
Related Work
67
Model-split learning schemes. The key idea for model-split learning schemes is to split the model 89 so that part of it is processed in the client and the rest is offloaded to the server. This idea was first proposed in Kang et al. (2017), Teerapittayanon et al. [2017). Liu et al. (2018) for inference tasks. Gupta and Raskar 2018 then exlended this idea for split learning. a collaborative multi-client neural network training. However, the round-robin design needed clients to learn sequentially and thus has huge disadvantage in terms of training time.
68
71
73
74 Split federated learning (SFL). In this paper, we consider the SFL scheme where the clients process their local models in parallel followed by periodic synchronization as in FedAvg [McMahan et al.. 2017]. This is the SFL-V2 scheme that is introduced in Thapa et al. (2020) for its better accuracy performance. The detailed process is shown in Algorithm|] [ the beginning of each epoch, server performs the synchronization of client-side model and sends the updated version to all clients. Then,
75
77

---

Algorithm Split Federated Learning
Require: For M clients, instantiate private training data (X,, Y;) for 1,,,... M. Server-side model S has N layers and client-side model C₁ has L N layers.
1: initialize C, S
2: for epoch f- to num _em__s do
3: C =
-i=1 C;
(Model Synchronization)
4: C for all
5:
for step s t to to num batches do for client i <- 1 to M in Parallel do
6:
7:
data batch (z,, y.) +- (X,, Y,)
8:
A, (,
CClient forward; send A, to Server)
9:
end for
10:
for client t 1 to M in Sequential do
C Lor(S((ss:: A.), y:) A.),
(Server forward)
12:
VA,C + back-propagation
(Server backward, send V W. C to Client)
13:
Update Ws; end for
14:
15:
for client i +- 1 to M in Parallel do
V Va,L C +- back propagatto
(Client backward
17:
Update Wc,;
end for
19: end for
20: end for
79 clients perform forward propagation locally till layer L N (the last layer of client-side model),
80 sending the intermediate activation A, the server (line 8). Server accepts the activation and label
81 y: sent from clients, and uses them to calculate the loss and initiates the backward process (line 9).
82 The backward process (line 10) consists of several steps: server performs backward propagation on
83 the loss, updates server-side model and sends back gradient V A, C to clients. Clients then continue
84 the backward propagation on their client-side model copies and perform model updates accordingly. While FedGKT [He et al. (2020) leaks prediction logits to clients, SFL. does not, making it promising candidate against ME attacks.
86
87
Model extraction (ME) attack. ME attack targets model prediction service APIs and retrieves confidence score prediction label for given inputs. The vulnerability
DNN model to ME first shown in Tramer et al. (016). Jagielski et al. (0200 shows that high fidelity model extraction can be achieved with fewer queries and the surrogate model can be used for launching more successful adversarial attacks. A successful ME attack not only breaches the model IP, but also makes the model more vulnerable to attacks. ME attack also supports transferable adversarial attacks [Goodfellow 93
88
89
90
91
92
et al. 2014), mainly targeted ones Madry et al. 2017 against the victim model. It can also be used with hardware expertise, few bit flips on model parameters can degrade ResNet- 88 model accuracy to below 1%.
94
to perform bit-flip attacks [Rakin al.] 2019
95
96
Data privacy in SFL. Similar FL, SFL scheme also has data privacy concerns. The most serious one is its vulnerability to MI attacks. In model-based MI attack [Fredrikson et al.) [2015]. the attacker trains inverted version of client-side model and directly reconstruct raw inputs from the
87
98
99
intermediate activation. Recent works [Vepakomma et al., 2020 Li et al. [022] point out this vulnerability and provide practical ways mitigate MI. However, defenses only work well the client-side model has enough number of layers.
100
102
3
Threat Model
103
3.1
Attacker assumptions.
104
Objectives. According to Jagielski al. (2020), there are three model extraction (ME) attack objectives: i) functional equivalence, ii) high accuracy, and iii) high fidelity. However, achieving
3

---

tSNE on Internediate activation (N
1-SNE on Intermediate activation
on intermediate activation IN
LSNG
t-SNE
t-SNE 1
t-SNE
t-SNE
Figure 2: tSNE analysis of intermediate activation with different N (8, 5, from left to right) on a VGG11 model on CIFAR-10 10 dataset. N denotes number of layers in server-side model in SFL.
106
functional equivalence is difficult in practical applications. Hence, most of the existing practical ME attacks on NN models focus on achieving high accuracy and fidelity. To achieve the accuracy goal, the attacker wants to obtaín a model that maximizes the prediction correctness. achieve the fidelity goal, the attacker wants to derive model with similar decision boundary as the victim model before launching adversarial attacks [Biggio et al. 2013].
107
108
110
Capabilities. We assume the attacker acts as client in a multi-client SFL scheme against the server (model owner). Consider the SFL scheme outlined in Fig. [ We assume the entire model has total of L layers (or layer-like blocks, i.c. BasicBlock in ResNet) out of which the server processes N layers. The attacker holds white-box assumption on the client-side model (consists of L N layers), that is, it knows the exact model architecture and parameters for those layers. The attacker holds a grey-box assumption on the N-layer server-side model, that is, knows its architecture and loss function while the model parameters are unknown. Also, we assume server blocks the prediction queries thus neither logits nor prediction labels are accessible by clients during training, but server allows gradient queries to let client-side models be updated. Based on a client's activation A C(xr) and its label y. gradient information A is computed and sent back to clients. In addition, we assume the attacker can perform gradient queries on any input æ, including malicious ones.
112
113
114
115
116
118
120
122
3.2
Analysis
123
Partial model extraction problem. Since the attacker already has white-box assumption of the dliett-siee model, the attacker only needs to extract the server-side model to reveal the entire model. This results in an easier problem setting. We observe client-side model heavily regularizes the feature space of its output (input of server-side model), making ME attacks easier to succeed, especially when N is small. As shown in Fig- 2 on a VGG-11 [Simonyan and Zisserman, 2014) model on
124
125
126
127
128
CIFAR-10 10 dataset, as N becomes smaller, tSNE embeddings of intermediate features with different
129
labels are easier to distinguish. For N = 2, ME attack is as simple as separating different clusters with linear layer.
131 To show that the extracting part of the model with ME is easier, we study existing ME attacks on the server-side model, by assuming that prediction access is allowed. Specifically, we investigate CopyCat CNN Morreia-Silva
132
133
al., 20188) Knockoff-random [Orekondy al. 2019] and data-free (Truong et al. 2021 As shown in Fig. (a), with auxiliary data (CIFAR- 100) and enough query budget, both attacks derive surrogate model with very high accuracy even for large N setting.
134
ME
135
136
Moreover, attacker with no data can also succeed with data-free ME as shown in Fig. 3 (a). When the
137
query budget equal to 2 million, the data-free ME can extract the model with high accuracy even when N is equal to 5.
139
Consistency of gradient query. For fine-tuning applications PPar et al. 2021 attackers get consistent gradient information from gradient query, as server-side model parameters are frozen or updated with very small learning rate. Gradient consistency very beneficial for ME attacks. However, for a training- teining-frommseee cerm-scratch usage, queries to SFL model obtain inconsistent gradient information as the server-side model drastically changes during training. As shown in Fig. (c), for the same query input, the gradient is drastically different in different epochs.
140
141
142
143
144
4 Proposed Model Extraction Attack
Previously, Milli al. (209 demonstrated that using gradients to reveal one-layer linear transforma- 146
tion is trivial. Given f(z) TTz, x,, can directly infer W from single gradient query given that

---

CopyCat (1K)
Knackoff-random (1K)
Data free (1M) Data free (2M)
Data free (500K)
Avg. Absolute Change (x10-4)
CopyCat (10K)
Knockoff-randon (10K)
90%
90%
4
75%
75%
60%
60%
2
45%
45%
30%
30%
47
70
93
116
139
162
185
N 2
N 5
=
Entire Model
N = 2 N 5 N
Entire
Model
Epoch
(a)
(b)
(c)
Figure 3: Analyses of SFL threat model. Existing ME attack performance on VGG-11 CIFAR-
10 model with different N, assuming prediction query access is allowed: (a) ME attacks using
CIFAR-100 100 as auxiliary dataset; (b) Data-free ME attack that demonstrates ME attack on part of the model is much easier than ME attack on the entire model; (c) Inconsistent gradient problem in training- itom-scratch SFL. The y-axis denotes the change in gradient (lower means more consistent) for the same inputs in different epochs.
W = V=./ (a)). However, using gradient only can go further than one layer. Milli et al. 2019
148
Σx=1 Il (x), W; A æ, where g(x) 1 (Ar > 0}. A is of Raxd and W is of R", using input gradient can recover the absolute
shows that recover two-layer ReLU network of the form [(x) Σ
149
150
151
value of normal vectors W, for i E [h]. In order to get the sign information of W.A,, prediction
152
query is required which is not supported by SFL's threat model.
159
So in this paper, we investigate approximate ME attacks that differ in the data assumptions and gradient usage. We propose five ME attacks as shown in Table We also include naive baseline scheme that train the surrogate model from scratch without access to either the client-side model or gradients. Despite the differences, the five proposed attack methods follow the same strategy, that is, they all train a randomly initialized surrogate server-side model from scratch. We provide a detailed illustration of proposed five attacks in Appendix A.2
155
156
157
158
Table 1: Model Extraction Attack Methods in SFL
Method
Data Assumption
Prediction Query
Gradient Usage
Client-side model Usage
Craft-ME
None
No
Data Crafting
Initialization
GAN-ME
None
No
Data Generator
Initialization
GM-ME
Natural Auxiliary
No
Gradient Matching
Initialization
Train-ME
Limited Training
No
None Soft Label Crafting
Initialization
SoftTrain-ME
Limited Training
No
Initialization
Naive Baseline
Limited Training
No
None
None
4.1
ME attacks without training data
160
We first consider two cases where the attacker does not have training data: one is that attacker does any data or use randomly generated noise data. This case is motivated by Truong et al.) [202]] which shows that when an attacker does not have a similar enough dataset, it is better not to use at all. The other case is when the attacker has auxiliary dataset with different labels from the victim's 's training data (for example, CIFAR-10 and CIF)
162
184
165
Crafting model extraction (Craft-ME).. Inspired by Han et al. 2018 where data-label pairs (referred to as instances) with small-loss are shown to present useful guidance for knowledge distillation,
186
propose a simple method to craft small-loss instances using gradient queries and use them to train the surrogate model. We initialize random input x, for every class label c, and use the gradient V_. L to update E,,. For each input, updating is repeated for number of steps. By varying label c, a collection of soall-loss instances is derived during SFL training. Then, a surrogate model is trained from scratch on these small-loss instances.
168
170
172
GAN-based model extraction (GAN-ME). Recent work [Truong et al., 2021 proposes a GAN- based approach for data-free ME. The key idea is to use generator G to continually feed fake inputs 174 to the victim model V and surrogate model S, and use confidence score matching to let S approach
V. However, the confidence score matching needs prediction query which is not allowed in our case.
5

---

178
Thus, we adapt the GAN-based method to gradient- Sundiyet-query--ee query- case and propose a two-step method: First, a conditional-GAN (c-GAN) model G(z|c) is initialized. The attacker trains the generator during victim model's training by generating fake data f and label and performing gradient queries to update G. After the training done, generator is used supply small-loss instances f to train the surrogate model (the unknown part). We observe a serious mode collapse problem during the GAN training. So we utilize the distance-aware training introduced in Yang et al. (2019). to encourage the c-GAN to generate more diverse small-loss instances. In the new method, the training of the generator is not based on a min-max game, or on traditional GAN training. Instead, it simply trains the generator toward minimizing cross-entropy loss. While the generator G fails to generate natural- -loukigg imputs even upon convergence, it generates abundant small-loss instances for every label, and divergence loss helps it generate good variety of outputs. During SFL training. the generator can adjust itself to the changing server-side model.
177
179
180
181
185
185
187
188
Gradient matching model extraction (GM-ME), Gradient matching (GM) in ME attack has been investigated in Dagielski et al. [2020), Milli et al. [019) and is used in combination with prediction query to improve the extraction performance. Since, in SFL, prediction query is not allowed, we navigate this strict threat model's restriction by adopting gradient matching (GM) loss. For a given
189
191
label ye, GM loss has the following form:
LGM V ((S(C(=)) Va, Va, ((V()())))
(I)
193 where, I; denote inputs, C' denotes client-side model, S and V denotes the surrogate model and victim model, respectively. For each input, attacker would query gradients with different label y: to get much information as possible. This attack performs extremely well for small N but degrades significantly for a larger N. Its performance also depends on the domain similarity between the auxiliary dataset and the victim dataset.
194
195
196
198
4.2
ME attacks with training data.
199
Next we discuss the case when the attacker has subset of training data, corresponding to the strongest data assumption.
200
201
Training-based model extraction (Train-ME). For attackers with subset of the training data, derivation of an accurate surrogate model can be done using supervised learning (through minimizing the cross entropy loss on the available data). We call this Train-ME, similar idea is also adopted in Fu al. 2022 extract the entire model of the other party. Train-ME only relies on the white-box assumption of the client-side model, using it to initialize the surrogate model and does not need to use the gradient query at all. Surprisingly, it is one of the most effective ME attacks.
202
203
204
205
208
207
Gradient-basse based soft label training model extraction (Soft-train-ME), If gradient query is allowed achieve better ME attack performance
and a subset of training data is available, the attacker compared to Train-ME. To utilize gradients, a naive idea is to combine the GM loss with cross-entropy loss Train-ME. However, our initial investigation shows they are not compatible; the cross-entropy loss term usually dominates and the GM loss even hurts the performance. An alternative approach is use soft label. We build upon the method in Gu et al. (2020) which shows that gradient information of incorrect labels is beneficial in knowledge distillation, and use it for surrogate model training.
209
210
211
212
213
214
Specifically, for each input 2;, gradients of the ground truth label as well as incorrect labels are collected (a total Ne. where No is the number of classes). For an input æ, with true label c, its soft
215
label of of k-th (k o e) label is computed as follows:
216
cos(et,et e
di = a1 NN (c + 11
where, et denotes flattened gradients of label k; of denotes soft label for the k-th label k and a is 218 constant (a > 0.5). The derived (æ;; q:) pair then used in the surrogate model training in addition to the true label (which is the only difference from the Train-ME).

---

220 5 Model Extraction Performance
221
In this section, we demonstrate the performance of the proposed ME attacks and the baseline attack on SFL schemes. All experiments are conducted on a single RTX-3090 GPU. For the SFL model training. we set the total number of epochs to 200, and use SGD optimizer with a learning rate of 0.05 and learning rate decay (multiply by factor of 0.2 epochs 60, 120 and 160). We assume all clients participate in every epoch with an equal number of training steps. To perform ME attacks,
223
225
226
the attacker uses an SGD optimizer with learning rate of 0.02 to train the surrogate model and we report the best accuracy and fidelity. We evaluate accuracy of the surrogate model on the validation dataset. We use the label agreement fidelity, defined as the percentage of samples that the surrogate and victim models agree with over the entire validation dataset, as in Jagielski et al. 22200. Detailed settings for each ME attack are described in Appendix A.
227
228
229
230
231
ME attack on SFL with fine-tuning based training 5.1
We first perform the proposed ME attacks on fine-tuning SFL version with consistent gradient query. Here we use a per-trrriintn preitrrined model and set the number of gradient queries to 100K. On a victim VGG-11 model on CIFAR- 10 dataset, whose original accuracy is 91.89%, performance of all five ME attacks are shown in Table 2 For each of the ME attacks, we vary hyper- papper-prrmeters and report the one that achieves the best attack performance; details different prper-pprameters
233
235
included in Appendix [A.3] When N = 2, all five ME attacks can achieve near-optimal accuracy and fidelity performance. For Craft, GAN and GM ME, the accuracy drops around 80% when N is 5, it sharply drops to below 40% when N is 8. For Train and SoftTrain ME, accuracy slightly degrades when N is and reduces to around 70% when N 8. These results show that ME attack performance strongly correlate with the #layers in server-side model. With an increasing #layers in server-side model, ME attack performance reduces as the extraction problem becomes harder with more unknown parameters and more complicated input feature space.
237
238
239
240
241
242
243
O Observation. Different attacks present different and interesting characteristics. Craft-ME has steady attack performance and can succeed even with a tight query budget. GAN-ME needs a large query budget to train the c-GAN generator towards convergence but can achieve better accuracy and fidelity than Craft-ME for N S 5. GM-ME requires an auxiliary dataset that is similar to the training data. If CIFAR- 100 is used to attack CIFAR-10 10 model, GM-ME achieves almost perfect extraction for small N. However, its performance degrades if MNIST or SVHN are used as auxiliary datasets. For high N. the surrogate model fails to converge on the GM loss, and its extraction performance suffers from sharp drop. For attacks with training data such as Train and SoftTrain
245
246
247
248
249
250
251
252
MEs, both accuracy and fidelity are much higher than attacks without training data, When N 2 6, Trainrain-Mi can achieve slightly better accuracy and fidelity than Train-ME.
253
Table 2: ME attack performance SFL on fine-tuning and training-from- scorscct applications. The victim is a VGG-11 model on CIFAR-10 with 91.89% validation accuracy. For Train, Srft'raii and Naive baseline, for the fine-tuning setting, data assumption is 1K training data (randomly sampled), and for the vain-froomssrateee setting, the number of clients is 10 and each client has 5K training data.
Fine-tuning
Training-frum from-scratch
Metric
N
Craft
GAN
GM
Train
SofTTrain
Naive
Craft
GAN
GM
Train
SoftTrain
Naive
2
91.64
91.86
92.02
92.05
91.99
85,99
85,99
53.06
90.58
90.31
Accuracy (%)
5
83.46
84,93
80.28
90.82
90.48
49.64
35.58
40.03
12.13
89.86
87.02
72.63
35.48
12.45
70.28
71.32
17.49
10.88
56.78
2
98.23
98.42
99.87
99.29
99.10
92.37
89.59
54.63
99.34
98.87
Fidelity (%)
5
86.32
87.49
84.33
94.84
94.67
50.62
41.32
38.72
11.87
95.40
89.83
72.62
8
36.11
18.62
12.63
71.79
72.45
15.63
17.44
10.67
80.01
57.78
254
5.2
ME attack on training-from-scratte SFL
255
SFL case. A good for bradient-based ME attacks is at the end of training when gradients do not vary 257 as much and the model converges. So for Craft, GAN and GM-ME, we launch the attack at epoch
Next, we investigate the proposed ME attack performance in
attack-time-
160 to get more consistent gradients. As the model is updated by multiple clients, the percentage of
7

---

259
malicious clients also affects the ME attack performance. We found that with more benign clients (or fewer malicious inputs), the server-side model returns more consistent gradients to the attacker. Attack performance for three attacks are shown in Table 2 for 10-client SFL.
260
262
case. Results with other hyperparameter settings are included Appendix A.44
263
Observation. Because of the poisoning effect of malicious inputs sent by the attacker in gradient- based attacks, in all the methods except Train-ME, the victim model's accuracy suffers from a 2-3% degradation, resulting in a less accurate surrogate model. For gradient-based attacks without training data (Craft, GAN and GM MEs), we notice a sharp (> 3%) performance drop in both accuracy and fidelity compared to the consistent query case. The sharp drop in ME attack performance is caused by inconsistent gradients. Take Craft-ME as an example. Crafted imputs that have a small loss at an earlier epoch of the training can have a large loss in the final model. Training surrogate model with large amount of inconsistent information results in poor training accuracy. Compared to Craft-ME, GAN-ME is more robust to inconsistent gradients as the generator can adjust itself to the change of server-side model, resulting in a better extraction performance. However, when N is larger, the generator does not converge well and its performance drops drastically. Last but
264
285
266
267
268
289
270
272
274 not the least, GM-ME completely fails with inconsistent gradients, even for small N. This implies that the GM loss is super sensitive to inconsistent gradients and is only effective in consistent query and Train-ME shows that SoftTrain's 's advantage diminishes because of the poisoning effect. Thus, Train-ME is more effective for attacker with training data in training-froom from-scratch SFL.
276 cases. A comparison of SoftTrain
278
279
5.3
ME attack performance for other architectures and datasets
We perform extensive analysis the attack performance of the proposed ME schemes on other architectures and datasets. We use Train-ME attack, the best performer for both fine-tuning and
281
282
cases and use the accuracy as the performance measure. To test ME attack on different datasets, we perform Train-ME attack with 1K training data on VGG-11 model with N
maiiing-froomsseeee
283
284
set to 5, on MNIST [LeCun] 1998]. FMNSIT [Xiao al., 2017]. SVHN (Netzer et al. 201 and
285
CIFAR- 100 datasets KKrzhevsky et al. 2009] (in addition to CIFAR-10). As shown in Fig. 4 (a), for
286
all datasets except CIFAR- ME attack achieves accuracy very close to the original. For CIFAR-
287
100, the extracted accuracy is over 20% below the original because of its task difficulty. Additionally,
286
we also test Train-ME performance with 2% and 20% ImageNet training data of Mobilenet-V V2 on ImageNet dataset. As shown in Fig. 4 (b). ME attack fails badly due to the complexity of ImageNet dataset (Deng et al. 2009). resulting in high accuracy gap of 10% when N set to 2. For different architectures, we choose Resnet-20, Resnet-32 (He et al. 2016 and Mobilenet- V2 [Sandler et al.
289
290
291
292
2018] on CIFAR-10 dataset (with necessary adaptations). For Resnet and Mobilenet family, we assign last 4 foyer-block: and FC layer to server-side side model. As shown in Fig. (c), with the same proportion of layers (5 out of 11) being assigned to server-side model, ME attack is much less effective on Resnet-20 than on VGG-11. A comparison of the performance of Resnet-32 and Mobilenet22 with similar proportion of layers being assigned to server-side (5 out of 17 and 20, respectively), ME on Resnet-32 is also much worse than on MobilenetV2. This indicates Resnet architecture is more resistant to ME attack.
293
294
296
298
Discussion
299
300
Our evaluation showed that ME attack performance drops with increasing #layers in server-side model. Thus, simple idea to improve resistance to ME attack is to use a larger N. However this implics that the #layers in client-side model would be smaller, thereby undermining clients' data privacy. Data privacy in SFL can be represented by the Mean Square Error (MSE) performance of an MI attack as outlined Li ct al. [022]. The implementation detail is included in Appendix [. As shown in Fig. 4 (d), extracted accuracy decreases with a larger N, and MSE decreases too. This means MI attack can provide more precise reconstruction which compromises data privacy. We will extensively consider the tradeoff between ME resistance and data privacy in future.
302
304
306
307
308
Defense Method. We also investigate defensive methods against ME attacks on SFL. According to the analysis in Fig. 2 extraction is easier because client-side model is leaked to the attacker. Thus, to make ME attacks more difficult, we can restrict the client-side model's feature extraction capabilities.
310
Towards this goal, we apply LI regularization with three different strength (A = 5e-5, le-4 and 2c-4)
8

---

Original Accuracy Extracted Accuracy
Original Accuracy Extracted Accuracy
24 data
20% data
100%
100%
75%
Original Accuracy: 72.19%
70%
70%
60%
40%
40
MNIST
FMNIST
SVHN
CIFAR1O
GIFAR100
Vogli
Resnet20
Resnet32
Mobile22
45%
N=2
N
N=4
N=5
(b)
Increasing N
N=4
L1 Regularization
Original (N=5)
0.04
0.03
0.03
MSE
0.01
0.02
N=8
MSE
0.02
0.01
Increasing X
Improved ME resistance
0
70
75
85 Extracted Accuracy
80
95
80
85
90
95
100
Extracted Fidelity
(d)
(c)
Figure 4: Top row: Extensive results on model extraction (ME) attacks. (a) ME attack performance of VGG-11 on other datasets. (b) ME attack performance of MobilenetV2 on ImageNet. (c) ME attack performance of other architectures on CIFAR-10 dataset. Bottom row: (d) Tradeoff between ME resistance and data privacy (MSE). (c) LI regularization effective defense for ME attacks.
312
on the client-side model to penalize weight magnitude. As shown in Fig. (e), this simple defense effectively improves the resistance to ME attack (specifically, Train-ME with IK data) without hurting data privacy. While there is some accuracy degradation, as shown in Appendix A.3 this demonstrates the potential of using regularization to defend ME attacks.
314
7
Ablation Study
316
317
ME attack with non-IID data. We consider the non-lID (independent and identically distributed)
318
case where the attacker only has data from C classes of CIFAR-IO. 10. Results presented in Appendix A.6 show that ME attack performance is still good for C = 5 but degrades sharply when C' = 2.
319
Ad Adversarial attack based successful ME attack. As mentioned before, the goal of ME attack is to launch more successful adversarial attacks. We perform transfer adversarial attacks using a surrogate model extracted by the strongest Train-ME attack. As shown in Appendix A.7 SFL with proper N achieves better resistance to adversarial attacks.
321
323
ME attack without architecture information. In Appendix A.8 we investigate simple variants (longer, shorter, wider, and thinner) of the original architecture as the surrogate model architecture. We find that the performance of ME attacks is similar for the different architectures the exception is GM-ME which fails for different surrogate architectures.
325
327
Conclusion
328
In this work, we study the model IP protection capability of SFL and its resistance model extraction (ME) attack. We propose five viable ME attack methods for the threat model where gradient query allowed but prediction query not allowed. For the case when there are enough number of layers in server-side model, model IP can be protected well and transfer adversarial attack is not successful. However, data privacy could be compromised and so this factor needs be considered in the development of such schemes. We also point out a possible way of defending such attacks through regularization and plan to expand on it in the near future.
330
332
333
334
335
336
Broader Impact. This work points out the vulnerability of Split Federated Learning (SFL), to model extraction attacks, and should prevent naive adoption of SFL
model IP protection method. We helieve that the attacks presented here would initiate research in the development of defense schemes 338
to mitigate such attacks, help design a more robust SFL and possibly help in the design of neural network models that are inherently resilient to such attacks.
340

---

References
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Sındić, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. Joint European conference on machine learning and knowledge discovery in databases, pages 387-402. Springer, 2013.
343
344
Sebastian Caldas, Sai Meher Karthik Duddu, Peter Wu, Tian Li, Jakub Konečný, H Brendan McMahan, Virginia Smith, and Ameet Talwalkar. Leaf: A benchmark for federated settings. arXiv preprint urXiv:1812.01097.
348
347
2018.
348
Jacson Rodrigues Correia-Silva, Rodrigo F Berriel, Claudine Badue, Alberto F de Souza, and Thiago Oliveira- Santos. Copycat cnn: Stealing knowledge by persuading confession with random non-labeled data, In 2018 International Joint Conference on Neural Networks (IJCNN, pages 1-8. IEFE, 2018.
349
350
351
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hicrarchical image databasc. In 2009 JEEE conference on computer vision and pattern recognition, pages 248 2255, Icce, 2009.
352
353
354
Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confidence informa- tion and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pages 1322-13333 2015.
355
356
357
Chong Fu, Xuhong Zhang, Shouling Ji, Jinyin Chen, Jingzheng Wu, Shanqing Guo, Jun Zhou, Alex X Liu, and Ting Wang. Lahel inference attacks against vertical federated learning. In 31st USENIX Security Symposiun (USENIX Security 22), Boston, MA, 2022.
358
360
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.600 2014.
381
362
Jindong Gu, Zhiliang Wu, and Volker Tresp. Introspective learning by distilling knowledge from online self-explanation In Proceedings of the Asian Conference on Computer Vision, 2020.
383
384
Otkrist Gupta and Ramesh Raskar. Distributed learning of deep neural network over multiple agents. Journal of Network and Computer Applications, 116:1-8 2018.
385
386
Bo Han, Quanming Yao, Xingrui Yu. Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. Advances in neural information processing systems, 31, 2018.
367
368
369
Chaoyang He, Murali Annavaram, and Salman Avestimehr. Group knowledge transfer: Federated learning of large cnns at the edge. Advances in Neural Information Processing Systems, 33: 14068-140000, 2020.
370
371
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770-778, 2016.
372
373
Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex Kurakin, and Nicolas Papernot. High accuracy and high fidelity extraction of neural networks. 29th USENIX Security Symposium (USENIX Security 20). pages 1345-1362, 2020.
374
375
378
Yiping Kang, Johann Hauswald, Cao Gao, Austin Rovinski, Trevor Mudge, Jason Mars, and Lingjia Tang. Neurosurgeon: Collaborative intelligence hetween the cloud and mobile edge. ACM SIGARCH Computer Architecture News, 45(1) (665))66,,,,,, 2017.
377
378
379
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Yann LeCun. The MNIST database of handwritten digits. htp:://ann. lecun. com/exdb/mmistf/ 1998.
381
Jingtao Li, Adnan Siraj Rakin, Xing Chen, Zhezhi He, Deliang Fan, and Chaitali Chakrabarti. Ressfl: A
382
resistance transfer framework for defending model inversion attack in split federated learning, 2022.
383
Peng Liu, Bozhao Qi. and Suman Banerjee. Edgeeye: An edge service framework for real-time intelligent video analytics. In Proceedings of the 1st international workshop on edge systems, analytics and networking, pages
384
1 6, 2018.
386
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706..6000 2017.
387
388
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera Arcas, Communication- efficient learning of deep networks from decentralized data. In Artificial Intelligence and Statisties, pages PMLR, 2017.
389
10

---

391
Smitha Milli, Ludwig Schmidt, Anca D Dragan, and Moritz Hardt. Model reconstruction from model ex- planations. In Proceedings of the Conference Fairness, Accountability, and Transparency, pages 1-9, 2019.
393
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
385
396
Tribhuvanesh Orekondy, Bernt Schiele, and Mario Fritz. Knockoff nets: Stealing functionality of black-box models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 4954 4963, 2019.
398
Sangjoon Park, Gwanghyun Kim, Jeongsol Kim, Boah Kim, and Jong Chul Ye. Federated split task-agnostic vision transformer for covid-19 19 exr diagnosis. Advances in Neural Information Processing Systems, 34, 2021.
400
401
Adnan Siraj Rakin. Zhezhi He, and Deliang Fan. Bit-flip attack: Crushing neural network with progressive bit search. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 1211-1220.
403
2019.
404
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chich Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEFE conference on computer vision and pattern recognition, pages 4510-4520, 2018.
406
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409. 1556, 2014.
408
Surat Tecrapittayanon, Bradley MeDanel, and Hsiang-Tsung Kung. Distributed deep neural networks over the cloud, the edge and end devices. In 2017 IEEE 37th international conferrce on distributed computing systems (ICDCS), pages 328-339. IEEE, 2017.
410
411
Chandra Thapa, Mahawaga Arachchige Pathum Chamikara, Seyit Camtepe, and Lichao Sun. Splitfed: When federated learning meets split learning. arXiv preprint arXiv:2004. 12088, 2020.
413
Florian Tramèr, Fan Zhang. Ari Juels, Michael K Reiter, and Thomas Ristenpart, Stealing machine learning models via prediction apis. In 25th {USENIX} Security Symposium ((USENIX) Security 16), pages 601-618, 2016.
416
Jean-Baptiste Truong, Pratyush Maini, Robert J Walls, and Nicolas Papernot. Data-free model extraction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17700,
418
419
2021.
420
Pranecth Vepakomma, Abhishck Singh, Otkrist Gupta, and Ramesh Raskar. Nopeck: Information Icakage reduction to share activations in distributed deep learning. In 2020 International Conference on Data Mining Workshops (ICDMW), pages 933-942. IEEE, 2020.
421
422
423
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv: 07747, 2017.
424
Dingdong Yang, Seunghoon Hong, Yunseok Jang, Tianchen Zhao, and Honglak Lee. Diversity-sensitive conditional generative adversarial networks. arXiv preprint 1911v:190,,09024 2019.
426
Checklist
428
The checklist follows the references. Please read the checklist guidelines carefully for information on how to answer these questions. For each question, change the default [TODO] to [Yes], [No], or [N/A]. . You are strongly encouraged to include a justification your answer, either by referencing the appropriate section of your paper or providing brief inline description. For example:
429
430
432
Did you include the license to the code and datasets? [Yes] See Section x.
Did you include the license the code and datasets? [No] The code and the data proprietary.
434
435
Did you include the license to the code and datasets? [N/A]
Please do not modify the questions and only use the provided macros for your answers. Note that the Checklist section does not count towards the page limit. In your paper, please delete this instructions
437
block and only keep the Checklist section heading above along with the questions/answers below.
11

---

439
1. For all authors...
440
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
442
(b) Did you describe the limitations of your work? [Yes] We provide the limitations of the proposed attacking methods by experiments described in Section 5.2 Section 5.3
443
(c) Did you discuss any potential negative societal impacts of your work? [Yes] As discussed in Section [ this paper provides attacking methods to gain proprictary model information which breaches Intellectual Property. However, we intend to point out the vulnerability of SFL scheme, which makes entities aware of the danger and hence prevents them from being attacked. We also provide potential defensive methods to mitigate our proposed attacks.
445
447
449
450
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
451
452
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
455
3. If you ran experiments..
458
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (cither in the supplemental material or as a URL)? [Yes] We will include them in the supplemental material.
457
458
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] We will open-source our code which includes all the required information.
480
462
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [No]
463
464
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes]
465
466
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]]
468
(b) Did you mention the license of the assets? [No]
469
(c) Did you include any new assets either the supplemental material or as URL? [No]
470
(d) Did you discuss whether and how consent was obtained from people whose data you're 're
using/curating? [No]
472
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [No]
473
474
5. If you used crowdsourcing or conducted research with human subjects...=
475
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
476
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
478
479
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
480
12


Reviews:
[{"review_id": "euJ40q_hPyf", "reviewer": "Reviewer_HaF2", "summary": "The authors expose the vulnerability of split Federated Learning and show how model extraction attacks can be launched by malicious clients querying the gradient information from server-side.\n\nThey proposed five different variants of model extraction attacks (ME) on split federated learning. The attacks use different gradient schemes, including data crafting, data generating, gradient matching and soft label crafting. They also made different assumptions for the data such as no data, only auxiliary data (out-of-distribution data) and training data (in-distribution data). They interestingly find that in a 5-layer-in-server SFL, an ME attack can derive a surrogate model with over 90% accuracy, and less than 2% accuracy degradation.\n\nFor the experiment they tried both fine-tuning the SFL models and training from scratch for different gradient consistency.\nThey showed that the ME attacks can succeed even without any data when fine-tuning. They also concluded that the ME attacks would succeed better with the increase of layers on the server side. ", "questions": "What do authors think about their possible way of defending such attacks through regularization? How good is that? \n\nHave the authors also looked the LEAF datasets for the experiments? LEAF seems to be the standard dataset for FL papers, with natural client partitions and other FL constraints.", "limitations": "The defense mechanism is quite basic and it would be nice to see some strong defense mechanisms with more analyses and numerical experiments", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- It’s an original work and perhaps the first one on the possible attacks on SFL systems.\n- Quality of the work is good, and the writing it’s quite clear. \n\nWeaknesses:\n- Some related work is missing. The authors mention that Split Learning is firstly introduced in reference \"[Gupta and Raskar, 2018]\" while the idea goes further back and there are already several papers on the field. For example (non exhaustive list):\n* Kang, Yiping, et al. \"Neurosurgeon: Collaborative intelligence between the cloud and mobile edge.\" ACM SIGARCH Computer Architecture News 45.1 (2017): 615-629.\n* Yousefpour, Ashkan, et al. \"Guardians of the deep fog: Failure-resilient DNN inference from edge to cloud.\" Proceedings of the First International Workshop on Challenges in Artificial Intelligence and Machine Learning for Internet of Things. 2019.\n* Liu, Peng, Bozhao Qi, and Suman Banerjee. \"Edgeeye: An edge service framework for real-time intelligent video analytics.\" Proceedings of the 1st international workshop on edge systems, analytics and networking. 2018.\n* Surat Teerapittayanon, Bradley McDanel, and HT Kung. 2017. Distributed deep neural networks over the cloud, the edge and end devices. In Distributed Comput- ing Systems (ICDCS), 2017 IEEE 37th International Conference on. IEEE, 328–339.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "XmMKYe6dfr", "reviewer": "Reviewer_sNJB", "summary": "This paper investigates model extraction attacks on split federated learning. The paper assumes that the attacker cannot be able to access the model predictions from the server. Hence, the paper leverages the gradients to extract the server-side model. Five model extraction attack methods are proposed for different data assumptions.", "questions": "Please clarify the no prediction assumption (weakness 1) and the no data assumption (Weakness 3).", "limitations": "The authors addressed the limitations and potential negative societal impact.", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The paper investigates five model extraction attacks against SFL.\n\n2. The evaluation considers several interesting scenarios, such as non-iid data distributions, adversarial attacks, and attacks without knowing model architectures.\n\nWeaknesses:\n1. The paper claims that the client-side attacker cannot use the existing model extraction attacks in the split federated learning because the client cannot get the predictions from the server. However, in the inference phase, the clients can get the predictions. The proposed attacks can be further compared with the existing model extraction attacks through predictions in the inference phase.\n\n2. The novelty of this paper is limited. Most proposed attacks leverage the existing model extraction attacks. Train-ME is the most effective attack. However, Train-ME is a very straightforward attack, which does not even need the gradient query. This may indicate that the model extraction attacks in SFL is not a trivial problem.\n\n3. The no data assumption is confusing. If the attacker is a client, the attacker should at least have access to the client’s data. \n\n4. It would great if the proposed attacks could be compared with some baselines. For example, since the attacker is one of the clients, what is the attack performance when the attacker uses the client’s data to train a model?\n\n5. The experiment only considers 10 clients. However, in the real-world settings, the number of clients should be much large. What’s the performance of the proposed attacks when a large number of clients participate in the SFL?\n\n6. The proposed attacks do not perform well on complex datasets (e.g., ImageNet, CIFAR100) and complex models (e.g., ResNet). \n\n7. The scope of the paper is pretty narrow. The proposed attacks can be only applied to split federated learning.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Qg-gVEdwo0", "reviewer": "Reviewer_WXHo", "summary": "The authors perform five different model extraction attacks in settings of split federated learning (SFL). Such attacks exploit gradient information from the server-side to conduct model extraction attacks. Besides, this paper proposed an approach to obtain the necessary shadow data for constructing attackers.", "questions": "  1. Craft-ME method, both the surrogate model and input x_r are randomly initialized, and the surrogate model generates x_r since the server-side model won't provide loss respect with x_r. Even though we can generate plausibly (x_r, c) labels via such a random surrogate model, how can these collected (x_r, c) instances be applied to train the surrogate model well?\n  2. Gradient matching model extraction (GM-ME): since the prediction query is disallowed, V is unknown in this setting. how to compute \\nabla_{x_i} L(V(C(x_i)), y_i) (see line 196, eq. 1)? Normally, Victim part will only compute gradient with respect to C's party weights. \n  3. Training-based model extraction (Train-ME): how much training data are used to train such a surrogate model, compared with the data size in SFL settings. ", "limitations": "  1. Gradient matching model extraction (GM-ME) relies on the gradient of x_i, which is infeasible in reality. \n  2. Training-based model extraction (Train-ME)'s idea is done by previous work, using model transferability. see [1]\n\n[1] Fu, Chong, et al. \"Label inference attacks against vertical federated learning.\" 31st USENIX Security Symposium (USENIX Security 22), Boston, MA. 2022.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "** strengths \n  1. Give a comprehensive investigation of model extraction attacks in split federated learning settings, and provide rich experimental results. \n** weaknesses\n  1. Different attack approaches are explained in an abstract way. It is hard to understand the attack assumption, how to craft data, how the attacker works, etc.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Ov_cwfW0YsS", "reviewer": "Reviewer_HjPo", "summary": "Federated learning is a distributed learning paradigm that allows multiple clients and a server to collaboratively learn a machine learning model without data sharing. However, federated learning still suffers from model privacy concerns, since the server and the clients all have access to the full global model. Split federated learning is proposed as a solution to both communication reduction and privacy enhancement. In split federated learning, the machine learning model is split into two parts, one held by the server and the other one held by the clients. Since the clients have access to only part of the model, split federated learning was considered to be robust to client-side model extraction attacks. However, in this work, the authors find that this is a false sense of security. Specifically, a malicious client can perform model extraction attacks to learn a surrogate model that have similar accuracy/predictions compared to the target server-side model. ", "questions": "- What is the accuracy of the model obtained via the aforementioned baseline, i.e., locally trained by the attacker with the available training data? Do the proposed attacks outperform it?\n- What is the complexity of the proposed attacks?\n- Can existing statistics-based Byzantine-robust FL methods defend against the attacks?\n- Is there any possible reason why ResNet shows better robustness to ME attack?", "limitations": "The negative societal impact may need to be stated more explicitly, e.g., via adding a paragraph in the discussion.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n- Five different model extraction attacks are proposed in this work. The authors consider attack different settings for the attack, e.g., whether the attacker (malicious client) has local training data. \n- The authors conduct extensive experiments to show the effectiveness of their model extraction attacks. It is impressive that the attacks do work in many cases and it is interesting to see the impact of different parameters, e.g., the number of queries, the number of layers in the server-side model, and model architectures.\n- The paper is generally well-written and easy to follow. \n\n### Weakness\n- When evaluating the model extraction attacks with training data available, it is intuitive to compare the attacks with a simple baseline that uses the training data to train a model locally on the malicious client. I would expect the authors to perform a simple comparison with such baseline. This is essential because if the attacker can locally train a better model, then there is no motivation for the attacker to perform the model extraction attacks. \n- There is no analysis on the complexity of the proposed attacks. It is interesting to see some analysis on the attack cost, e.g., $O()$ notations with respect to the number of parameters in the server-side model. \n- There are some defenses in federated learning that leverages statistical analysis of the local gradients (known as Byzantine-robust FL methods). I am curious whether these methods are sufficient to defend against the proposed attacks. \n- In Section 5.3, it is claimed that \"This indicates Resnet architecture is more resistant to ME attack\". It would be interesting to see some explanation on why this is the case. \n- The potential societal impact may need to be stated more explicitly.\n- Writing issues. On line 3 of Algorithm 1, I would say it is better to average model parameters (i.e., $W$) instead of models (i.e., $C$). In Figure 4, the captions of subfigures mismatch.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["Privacy and Security (e.g., consent)"]}, "extra_text": {}}]

Author rebuttal:

Decision: Reject