Accurate Split Learning on Noisy Signals
Authors: Hang Xu
Subhajit Maity
Aritra Dutta
Xin Li
Panos Kalnis
Venue: Accepted by TMLR ()

Abstract: Noise injection is applied in Split Learning to address privacy concerns about data leakage. Previous work protects Split Learning by adding noise to intermediate results during the forward pass. Unfortunately, noisy signals significantly degrade the accuracy of Split Learning training. This paper focuses on improving the training accuracy of Split Learning in the presence of noisy signals while protecting training data from reconstruction attacks. We propose two denoising techniques, namely scaling and random masking. Our theoretical results show that both of our denoising techniques accurately estimate the intermediate variables during the forward pass of Split Learning. Moreover, our experiments with deep neural networks demonstrate that the proposed denoising approaches allow Split Learning to tolerate high noise levels while achieving almost the same accuracy as the noise-free baseline. Interestingly, we show that, after applying our denoising techniques, the resulting network is more resilient to a state-of-the-art attack than the simple noise-injection approach. Our code is publicly available at: github.com/MaitySubhajit/AccurateSL.

Published in Transactions on Machine Learning Research (04/2026)
Accurate Split Learning on Noisy Signals
Hang Xu
wung.ruu@kous..ddu...
King Abdullah University of Science end Technology Thuwal, KSA
Subhajit Maity
Subhajit@uef. edu
Department of Computer Science
University of Central Florida Orlando, Florida, USA
Aritra Dutta
Aritmo Dutta@rucf edu
School of Data, Mathematical, and Statistical Sciences Institute of Artificial Intelligence University of Central Florida Orlando, Florida, USA
Xin Li
Kin.Li@ueff edu
School of Data, Muthematical, and Statistical Sciences University of Central Florida Orlando, Florida, USA
Panos Kalnis
Extreme Computing Reseorch Center
King Abdullah Universily of Science and Technology
Thwwal, KSA
Reviewed on OpenReview: https: // openreview. net/ forum? id= in174BlaG9
Abstract
Noise injection applied in Split Learning to address privacy concerns about data leakage. Previous work protects Split Learning by adding noise intermediate results during the forward pass. Unfortunately, noisy signals significantly degrade the accuracy of Split Learning training. This paper focuses improving the training accuracy of Split Learning in the presence of noisy signals while protecting training data from reconstruction attacks. We propose two denoising techniques, namely scaling and random masking. Our theoretical results show that both of our denoising techniques accurately estimate the intermediate variables during the forward pass of Split Learning. Moreover, our experiments with deep neural networks demonstrate that the proposed denoising approaches allow Split Learning to tolerate high noise levels while achieving almost the same accuracy the noise-free baseline. Interestingly, show that, after applying our denoising techniques, the resulting network is more resilient to state-of-the-art attack than the simple noise-injection approach. Our code is publicly available at: github.. wwt.hu..cco/Maattggoaoooooo
1 Introduction
Privacy concerns across application domains, including finance, healthcare, and online commerce, limit the sharing of raw data. needed to train accurate deep neural networks (DNNs). Split Learning Gupta & Raskar
2018)) Vepakomma et al. [0018a) has emerged as solution that enables different parties to collaboratively
See the project page

---

Published in Transactions on Machine Learning Research (04/2026)
Recunstructed Dutu
Attasker
7 6
9
I
S
Actraac
0.0
Recomstracted
q
S
6
4
forward pass
Labels
0.6
Prinuce data
backward pass
0.4
7 Q
S
6
0.5
4/
S
Clent
Server
1.5
2.0
(b)
(c)
Figure The trade-off between the security and training accuracy in noise-injected private Split Learning. (a) General schematic representation of two-party split learning, where the neural network is split between the client (owns the data) and server (owns the labels). Noise is injected into the client's output to prevent private data leakage to the attacker. (b) Noise variance (ar) vs. Test accuracy in training a split CNN model on the MNIST datasct. As the variance of the injected noise level (a) incrcases, the test accuracy drops. (c) Training data reconstruction by hijacking allack at different injected noise levels. The recoustruction capacity decreases as the injected noise level is increased.
learn a model, without explicitly sharing raw input data. Typically, in two-party Split Learning, the Split. Neural Network N is divided between the data owner, a.k.a. the client, and the label owner, a.k.a. the server; see Figure 1 (IRs) (i.e., the neurons of the cut layer, the last layer in the client's 's part of the DNN) to the server. The server completes the forward pass and, during backpropagation, returns the gradients of the IRs to the client. Consequently, the client can train the joint model without revealing the private training data to the server.
(a). During training, in the forward pass, the client forwards the intermediate results
Unfortunately, sharing only the IRs does not protect the raw data. The shared IRs contain considerable latent information about the data. They
be used to stage powerful attacks, such as model inversion attack He et al. (2019a); Zhang et al. (2020); Erdogan et al. (2021), label inference attack Erdogan et al. (2021): Kariyappa & Qureshi 20211) Li et al.
(2021), and hijacking attack Pasquini et al. works Titcombe et al. (2021); ) A Abuadbba et al. (020)); Mireshghallab et al. (2020); Wang et al.
(022). Several
(2018) attempt
to mitigate this risk by adding noise to the IRs before sharing them with the other party. However, one
fundamental issue with adding noise is the trade-off between the trained model's quality and its susceptibility to external attackers. While high levels of noise are favorable in making the training data private, the noise inevitably impacts the quality of the trained model Abuadbba al. (2020); Wang al. (2020a) (021), that is, the accuracy; see Figures (b) and (c) for example. Thus, fundamental question is: Can improve the training accuracy of Split. Learning under noise injection, without making the model vulnerable to data leakage?
We answer the above question in the affirmative by applying a post- prot-procsssing denoising layer to coise-injected IRs during the Split Learning process. Our intuition is that the injected noise introduces an ervor during the forward pass, which dominated by variance when the noise level is high. As long as we can roduce variance using denoising techniques, training quality should improve. Such a post-processing layer will not introduce any additional private information leakage, provided it does not interact with the original private data.
We list our contributions as follows:
Contributions. (i) We propose two denoising techniques (i.e., scaling and masking) to improve the training accuracy and stability of noise-injected SplitNN; see $3 (it) Our theoretical investigation on classification task in 3.2 shows that denoising can reduce the error caused by noise injection during the forward pass. (Mi In addition to improving the train model quality, we show that our modification to Split Learning with noise injection, followed by postprocessing, preserves the security guarantee in the entire training protocol; see $3.3 Quantifying the privacy aspect in our work is not the primary focus; it is an additional benefit. (iv) We validate our claims through extensive numerical experiments on synthetic and real data (i.c., 7 DNN models on 7 different datasets, including large-scale datasets, ImageNet1K Deng et al. (2009) and Amazon Reviews McAuley & Leskoved (2013)) in 44 Moreover, (v) we find that our masking technique, in addition to
2

---

Published in Transactions on Machine Learning Research (04/2026)
denoising, also enhances the resilience of Split Learning against the state-of-the-ar hijacking attack Pasquini et al. 2021g ; refer to $ 4.3
2 Related Work
Numerous studies spanning federated training, split training, and differential privacy use noise injection and denoising techniques. In this section, we briefly discuss them.
Federated Learning (FL) with noise injection. Noise injection in vertical FL shares some similarities with noisc-injected SplitNN, since in both cases, the noise is injected into intermediate results during the forward pass. Existing works Wang et al. (2020b); Chen et al. (2020) propose to add Gaussian/ LLaplacian
noise on participants' TRs to protect private training data or labels. Chen et al. (2020) only demonstrates the impact on training accuracy when some applications have a relatively low noise scale. The other framework
Wang et al. (2020b) proposes a similar noise injection technique only for linear model collaborative learning, which is not directly applicable to general DNNs.
Split Learning with noise injection. Due to the vulnerability of SplitNN against model inversion attacks, Titcombe et al. (2021) proposed to apply differentially private noise injection on IRs during inference time to provent data reconstruction by the attacker. Shrodder, proposed by Mireshghallah et al. (2020), adaptively generates a noise mask to minimize mutual information between input and intermediate data. However, these two methods only introduce noise injection during the inference time; thus, the privacy of training data is not preserved. Abuaddba et al. (2020) successfully applies noise to the IRs during the training to defend against model inversion attacks on one-dimensional ECG data; also, see Wu et al. (2023) for SpiltNN with differential privacy for integrated terrestrial and non-terrestrial Networks. It turns out that the noise has dramatically impacted the model's accuracy. Unlike previous works that only focus on the attack defense efficacy, we aim to improve the training accuracy with a significant noise level.
We provide an answer to split learning denoising by proposing two post-processing techniques (i.e., scaling and masking) to improve the accuracy and stability of the splitNN training when Gaussian noise injected into the IRs. To our knowledge, we are the first to propose denoising techniques on the Gaussian noise-injected IRs to improve the training accuracy of SplitNN and theoretically show the privacy guarantee in this setup. In the following, for completeness, we mention some differential privacy (DP) techniques used in federated learning
(FL) and related settings Bergou et al. 2023 Sun et al. 2024) 5
Xu et al. (2021 for completeness.
Gaussian noise injections (GNIs) are family of regularization methods for DNN training through adding Gaussian noise the activations weights during the forward pass. It is similar to the noise injection in SplitNN except for the following two aspects: (i) There is no requirement of bounded sensitivity in injection objects. (ii) The noise scale is usually set small to avoid negative impacts on training accuracy. The explicit regularization elfects of GNTs are well invest igated in Camulo et al. (2020): Li & Liu (220)) Lim et al. 22021) )
demonstrating better generalization for trained models over unseen data, In addition, GNIs can improve the robustness of DNNs against adversarial attacks or data perturbations Lim al. (2021): ); He et al. (2019b).
However, Camuto el al. 22021) also found that GNIs can introduce some implicit bias on gradient updates, which inevitably degrades the training accuracy.
Denoising. Adding the Gaussian noise and the denoising mechanisms in our work shares many similarities with the differential privacy (DP) and their post-processing that maintains the DP guarantee and often improves accuracy Zhu et al. 2022 2021). Denoising for DP has been well adopted in the statistical estimation Hay et al. 2009 2010); Nikolov al. (2013); Bernstein et al. (2017), where they exploit some prior knowledge to design a data release mechanism with better DP utility. Recently, Balle & Wang (2018) proposed an optimal denoising technique for Gaussian mechanism, where given ~ N [xx)) and their target is to find a processin function g such that g(y) is closer to fxx than y. This is substantially different. from Split.NN as there are subsequent layers on top of the Gaussian mechanism in the training process.
Nasr & Shokri (2020) has also investigated using scaling as a denoising technique to improve the DP utility for DP-SGD Abadi et al.] (2016). However, the authors scale up/down the noisy gradients based on the "usefulness' of gradients, while we utilize scaling to minimize the estimation error of the noisy neural network
3

---

Published in Transactions on Machine Learning Research (04/2026)
outputs. Wang et al. 20200a) showed that adding Laplacian smoothing on Gaussian noise-injected gradients can improve the utility of DP-SGD. While Ligett et al. (2017) proposed general noise reduction framework, Lecuyer et al. (2019) proposed PiXel-DP, an adversarial defense mechanism, scalable to diverse large networks and datasets.
3 Theoretical Guarantee
Notations. By [n] we denote the set of natural numbers f1.2,..., of vector x, while Aij denote the (i,j)th component of matrix, A. We use |||l and || to denote the l and the Frobenius norms of vector z and a matrix A, respectively.
.n).
By ny, we denote the ith component
Problem setup. Let be the training dataset with N elements, {(X,, ())) drawn i.i.d. from some distribution, P(X.)), where X; E Rd is the input feature vector, and y is the corresponding ground-truth label. We consider a SplitNN with an outpul vector in Ih" The network is divided between the client and the server, where the server network consists of several DNN layers and the output loss function. In our experiments, we show different split configurations; however, in our theoretical analysis, we have one fully connected (FC) layer after the cut layer as the final layer.
Let X R" be the vector from the client-side cut layer, and M E Rmxn be the weight matrix of the FC layer on the server side. At each iteration during training, the original split network processes a minibatch of training samples to calculate the loss and, during backpropagation, updates the gradients. We follow this formalization in theoretical analysis.
We consider the GNIs to protect the vector X. For Laplacian noise, see BB.1 Let the perturbed vector, X E R" follow the model: X = X + Z, where Z ~ NN ("),,)), chosen from zero mean Gaussian distribution with standard deviation a E R+ Then, we apply a post-processing function hn(+) : IR" + R" on X before forwarding it to the server side. To bound the magnitude of X, we use tanh as the activation function.
If we set the cut layer at an arbitrary i-th layer in an L-layer DNN, along with the final loss function used for DNN training, the theoretical analysis would become less tractable. Therefore, in our setup, we set i = L 1 to illustrate the main ideas. We formalize two possible cases in the forward pass to the output layer: (i) a linear layer, + = Tm, an identity map, and we quantify RJMX - Mh (X)13 against E - -MX [2 see $3.1 and (#i) a linear layer with a nonlinear activation function (Softmaxx) 4 == s, nonlincar loss function (negative log), LLL, and we quantify ElCLL ((MX)) - CL s s(MhD(X)) where y" is the true label vector; see $3.2. In both cases, hD is a post-processing mechanism that is used to improve the training quality, resulting some "denoising' effect; the signal-to-noise ratio is not the measure of "denoising" in our context.
against E (L'" (MX)) - LLL(V" (
Our primary focus is nonlinear classification tasks, although for better understanding, denoising the lincar layer is important, which can be viewed as regression task. Our thcorems are proxies to measure whether the postprocessing would allow better server accuracy. We do not include an iteration counter on M. X, or X; we are examining the single forward pass. We use a random masking operator, Rp, and a scaling operator, Sa, as hp, defined below:
(1) Random masking operator. Let R2 be random matrix of I's and 0's with identical and inde- pendently distributed entries, (Rp)y ~ Bernoull(p). Denote the support set, Qp C [m] x [n] of R2 as Og == ((t,))((Rp)s = =1}. Based on this, for a matrix, Ae R mx
Ao :
(Rp[Al). = 0 otherwise.
From the definition, Rg is linear and is projection operator, that is, 12 Rp.
(it) Scaling operator. For a matrix A m and a > 1, denote the element-wise scaling operator, : mmx + R as (A) AAA. Unlike the random masking, the scaling operator, Sa has no ran- domness.
Sa(:): :
4

---

Published in Transactions on Machine Learning Research (04/2026)
3.1 A linear layer
With the above upt to easily explain our ideas, we start with neural network performing simple regression. Although this is not our primary focus, we believe this section provides a better understanding in a simple sctting. Because for the éz-regression task, no nonlinear activation function is required, the problem is much simpler. That is, y MX is the prediction of the output layer, and it does not involve any non-linearity. Theorem 3.1 describes results for fully-connected DNN with an éz-regression task. Additionally, explains how the scaling and masking parameters, a and p, respectively, are related to the noise scale o, while denoising the output of linear layer of DNN for given M and X. We calculate the expected test error, BIJMX- - Mh 112,, where hD is Rp or Sa, and compare it against IXI - M
Next, we will prove Theorem 3.1 for lz-regression task; see A.2 for the proof. For Theorem 3.1 the prediction of the DNN model does not involve any nonlincarity. Throughout Sections 3.1 and 3.2 E, (X)
denotes expectation conditioned on the randomness in Rp given X, and Ez(() denotes expectation taken on the randomness in X.
Theorem 3.1. With the notations above, we have (1) B M E - M if and only if
Pl[M XI? + (1- || I o? where X E R is a matriz obtained by stacking X E I in
each row, and @ denotes the elementwise product. (ii) Let > 1. BJ|MX MSa(X)11 < E MX MX if and only if ||
M < (==)))
Remark 3.2. Since the expected MSE can be decomposed into bias and variance, by showing the relation between the expected MSEs as in Theorem 3.1 the bias-variance trade-off between different processes can be explained.
3.2 Nonlinear loss function for classification task
For a vector, = E Rm denote 8 R. - (0, 1)mh as the softmax function, and (t.,()",<())) as the negative log loss function, where y" the true label vector; see definition in $A.1 In what follows, we show that for both masking and scaling operators, under certain conditions the noise level, 0, it is possible to find parameters p and a, respectively, such that, by using any of these operations, incur a lower deviation in the loss value than using the noise injection alone when compared to the loss of the original SplitNN.
Masking operation. Quantifying CLL(Y", ((M and LLL s(MX)) are critical as they involve randomness from the masking operator and the Gaussian noise. We require several intermediate results to prove the main result in Theorem 3.3 We state and prove them in the A.3.1
We want to show that by using random mask over a noise-injected layer, we incur a lower devia- tion in the loss value than using the noise injection alone when compared to the loss, LLL (LL()")s(MX))) of the original SplitNN under certain conditions. That is, we want to compare the quantities EELLLy ((MX)) LILly", ((MR, (X))) and EyLLi s(Mx)) LLL(V' (MM(V")(MM)))) The following re- sult formalizes this.
Theorem 3.3. With the notations above, for classification problems, assume that n 2 (MX); for 1,2, ..., and let the higher order momends, m, = E| E e ((MR)(())) B(E," e(MB,(()))) be small for p = 2,3, i== Then, if a is large enough, there some 1) such that for p E (8,
ElLLL(V a(MX))
(1LLL(U*s((MM) - SLLl(,s((MM))
The assumption n 2 (MX), is technical and can be easily satisfied in practice, provided the input dimension of the SplitNN is sufficiently large. We will pause here and provide a sketch of proof of Theorem 3.3 Because the original SplitNN always produces the least loss, the expressions in absolute values in the inequality above are non-positive, and so need only to verify that for all X, E [ s(MX)) LLL(D*, s(M)) <
E s(MX)) LLL(V", ((MR,(X))) By the definitions of softmax and negative log loss, we have
(LL(" = (MMR, + log Σ (M)R)
(1)
5

---

Published in Transactions on Machine Learning Research (04/2026)
where 1* is the location of the true label in y². For fixed M and X, (1) is function of for p (0.1).
Denote F(p) := E LL. sMM s(MR)(X)))) and consequently, F(1) = R L (v", (,,,,("ss))))) see Remark A.3
By using Lemma A.4 on we can approximate F(p) by
Var E i= ((MB)((X)))
F(D) -M(X), + log
EE
(M))
(2)
2 (E
Di= ((MRR)()))
We want to show that F(p) S F(1) when Pe E 1), for some 8 2 By using Lemma A.2 in 2 and
disferentiating with respect. to p, we can show, (1) 2 O. This would imply that F(p) an increasing
function of (J, for some 8 E (0, 1). This gives E [LLL(V", & (M))(X))) S <E , (MX)))] and 1 concludes the proof of Theorem 3.3
Scaling operation. Similarly, by using scaling over X, under certain conditions on the noise level, 0, we obtain lower deviation in the loss than using the noise injection alone when compared to the loss of the original SplitNN. Let Ls. (y. ((MS.(X)). We stale the result in Theorem 3.4 A.3.2 for sketch of the proof.
a (m,. mis Jank
Theorem 3.4. With the notations above, for classification problems, if 0² 2 maxi, k= 2 m²e
for = 1,2, m, then there exists a 8' E (0,1) such that E ELL (LL()"s(MMX)) - Lsal I CL ((MX)) Lyr (MM))), for a (1,
For concentration of the errors in Theorem 3.1 and Theorem 3.3 SeC A.3.3
3.3 Differential privacy (DP) preservation
Recent works show that Split Learning with Laplacian or Gaussian noise injection at the cut layer is resilient to attacks Abuadbba et al. (0000); Wu et al. (2023). In particular, DP (see Definition in $A.4 could be used to describe the privacy guarantee. In this subsection, we explain how our modification (scaling or masking) to Split Learning with noise injection (e.g., GNI) preserves DP. The key to our argument is to view our modification as a post-processing of a DP mechanism. Then, by DP's immunity to post-processing, we conclude that DP is preserved.
Next, in light of our results in and 3.2 recall some terminology needed for our discussion. For more details, see Dwork et al.] (2014] Xiang et al. 2019 Abadi et al. (2016).
Let D be a collection of databases. D C D and D' C D be two neighboring training datasets, that is, D' = D = {X}. In this case, we write d(D', D) = 1. For a function V D - R", define a randomized mechanism, K D > such that, for E DD
KCD) = (())
(3)
where Ze E nd is a random variable/vector with probability density function, p(z).
To ensure is (c, DDD,, one must require some condition on the density function p(z) of Z. For example, if p(z) is a Laplacian or Gaussian density function with its variances satisfying some lower bounds, then is DP. Before we recall the result, we first need one more important concept.
Define the sensitivity of V as
A sup [V V(D')l2 D.D'
(4)
The following Theorem by Dwork al. (2014) says, in the case when m 1, if NN(0, 02), then the random mechanism K, as defined in (3) is DP as long as is large enough.
Theorem 3.5. Dwork al. 2014 Theorem 3.22) Let e E (0, 1) be arbitrary. For e² > 2l1.122)/)), the mechanism in with parameter 2 cA/e /e is (e, 8) diffeeeeryll private.

---

Published in Transactions on Machine Learning Research (04/2026)
we note that the noise level o is directly proportional to the sensitivity A and inversely proportional to the privacy bound 6.
In Theorem 3.5
This result yields the DP of GNI for Split Learning [Lecuyer et al. 2019 ŞII.B), which suggests placing the cut layer early in the network, where bounding the sensitivity is casier. The next result shows that both the scaling and masking proposed in this work preserve DP.
Theorem 3.6. [Dwork et al. 2014, Proposition 2.1) Let IC be randomized mechanism which is (e, g: R + Rd is determönistic or randomized mapping, then g o K. is also (c,5)-DP.
Impact on backpropagation. We now discuss the induced change during backpropagation. In DNN training, backpropagation is used to compute gradients, which are then used to update the parameter estimates. As the popular DP procedure applied to DNNs is DP-SGD Abadi et al. (2016), we show how our scaling and masking would "alter" the gradients, which could be viewed as an approximate DP-SGD. Let g' be the gradient and 9c denole the clipped version of g' during the 1-th iteration. DP-SGD would add Gaussian noise Z ~ (0, af) to sè. What happened to our case is that starting from the beginning of the backpropagation, we will evaluate gradients at the "perturbed" values (due to the GNT and our modification using scaling or masking from the cut. layer) and these values will be used to propagate backward at each layer to obtained ge. perturbed version of g' and hence &c. Thus, we put this into the framework of 3 with K(D)= = (D), V(D) = (D),), and write
TC( = gc) +
where 2 is a random ceriable/vector due to the randomness of GNI and masking (using Bernoulli distribution). Although Z itself may not be Gaussian, it can be shown that Z = =(W)) where g is a sum of products of compositions of affine transformations and activation functions, and W random vector of independent Gaussian and Bernoulli variables. Let p(z) denote the "density" of Z. Denote the sensitivity of (-(+) by Ag. Define
S (w: p(w)= > e' p(w Ag)}. We can establish the following theorem, whose proof is given in the $A.10
Theorem 3.7. Let 8 > O and recall that Z is random variable with probability density p(2). Then, there erists an a > 0 such that, for Ag S a,
Pr(Z E 8) <,
where PI-] refers to the probability associated with the random variable. Furthermore, when 4g S a, De is (),
Remark 3.8. Let D and D' be any pair of large training datascts with single-entry difference, and without loss of generality, we have D D' d. Since [D] = n is large for large dataset, the empirical risk mininization function (a.k.a. loss function), L will follow the relation: CD,, Y) = CCD' + Ldd x (LD(DY where Y denotes the ground-truth label. Consequently, we can claim that, during backpropagation, A, is small for large datasets (i.e., when [D] is large), by definition of sensitivity. Thus, we can make the sensitivity small by requiring large training datasets, and we can claim that Ag is small for large datasets.
The above explains DP in one iteration of the training data. If we consider a total of training iterations, we can use the advanced composition theorem, Theorem A.7 to guarantee that the mechanism is (e (n Tc( 8')-DP for all 8' > O.
We can give a similar guarantee when the noise mechanism is Laplace. Also, note that, if in each training iteration, Ič,B represents the clipped gradient calculated over minibatch of size B taken from large enough databases, then for sensitivity, Ag small enough, the DP guarantee for ge.B holds following the same argument above; see Xiang et al. (2019) and our discussion in $A.4
In our experiments, we use the regular backpropagation formula for our noisy split network training: see Proposition A.8 Noise injection and propesscn in the forward pass perturb the gradients during the backward pass, but without adding any explicit noise to them in each iteration. However, this is not the same as GNTs to the gradient as in DP-SGD Abadi et al. (2016)- -he noise, our case, (1) a more general, data-adapted random variable than Gaussian, and (ti is generated over the DNN architecture, not a user-specified Gaussian noise.
-Th
7

---

Published in Transactions on Machine Learning Research (04/2026)
0=0.7
0=0.5
0=0.3
40
40
70-
70-
MSF
30
MSE
30
MSE
50
MSE
50
20
20
30
30
10
10
10
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
04
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2 0.4
0.6
0 1.0
Scaling factor (A)
Masking ratio (p)
Scaling factor (A)
Masking ratio (p)
(a)
fi(X) MX
(b)
fi(X) = MX
(c) fa(X) - Ce ((MX)) (d) f2(X) CLL ((L
Figure 2: Simulation of how scaling factor (A = ) and masking ratio (p) influence the estimation error
(MSE) under different noise levels (o) for linear (f1) and nonlinear functions ff2.
69
68
tng Geo
aceling
gar-
Milacinng Steblll Baseiine
yes
Mi-tzaing 0.:
Baneiite
Saneline
Niumaking (ow0.2)
Hi maaiing 62)
ki/r-0711 Boreli
van (nee liee 2305
16 a8e sie cds v51
200 223 386 Spech
00 Epoch
100 120 240
00
tpocs
(a) CNN MNIST
(b) MLP IMDR
(c) ResNet
CIFARIO
(d) RNN Names
(e)
ResNet Iet
Figure& 3: Test accuracy of the best run for SplitNN training with noise injection (NI) only and noise injection (NI) plus denoising (i.e. masking or scaling) in different training tasks. All models are split with one FC layer
on the server side (a: noise level, p: masking ratio, A: scaling factor 2)
4 Experimental Evaluation
In 4.1 we validate our theoretical claims through simulation on synthetic data, $.2 shows results on DNNs performing machine learning tasks, and Section 4.3 shows improved data privacy results.
4.1 Simulation
Setup. The following numerical simulations verify the results of Theorem 3.1 and 3.3 Since X E-
is the output of tanh function, and M is usually randomly initialized around in the actual training, we sample the entries of X and M from a uniform distribution on [-1, 1] in our simulation. The MSE of masking corresponds (XJ{(X) ((R (2) for all different functions (J1, f2, I' in Figure 2 where RS can be replaced by Sa for scaling. Moreover, when p and X x= the respective MSEs considered baseline MSEs. In Figure 2 for each plot, we draw line parallel to the X-axis from these baseline MSEs. The expectations are calculated by taking the average on k simulation results, where k = 1000.
1, masking and scaling are ineffective thus
Scaling simulation. In Figure 2 (a), each curve corresponds different noise scale, o. By decreasing the scaling factor, X for each Ø, the MSE first decreases from the baseline to minimum then increases, indicating an optimal À for each a. The NASC condition in Theorem 3.1 (if also infers that. For fixed M, X, this condition implies it is possible to find a smaller X when a is large. We make similar observations for the nonlinear case; see Figure (c).
Masking simulation. Figure 2 (b) shows that by decreasing the masking ratio, p, the MSE does not necessarily become smaller unless a is large enough. This verifies the claim of Theorem 3.1 t). More importantly, there is an almost linear relationship between MSE and the masking ratio as p + 1. This coincides with the MSE expression with masking given in equation g see the Appendix. We hypothesize that while both X, M are drawn from the Uniform distribution, the coefficient of p² might become negligible. Hence, the coefficient of the linear term, P. which can be positive or negative depending on the noise scale a, dominates the MSE Results from Figure 2 d), with the nonlinear loss, reflect Theorem 3.3 o² must. be
8

---

Published in Transactions on Machine Learning Research (04/2026)
1.0
0.8
0.8
Test acc.
Test acc.
0.6
0.7
NI + scaling (A=0.1)
NI + scaling (A=0.2)
0.4
NI + masking (p=0.1)
NI + masking (p=0.2)
0.6
(o=0.7) Baseline
(o=0.7) Baseline
0.2
0.5
0.0
500 1000 1500 2000 2500 3000 Step
100 200 300 400 500 600 Step
700 800
(a) CNN MNIST
(b) MLP- IMDB
Figure Test accuracy of SplitNN training with noise injection (NT) only and noise injection (NT) plus denoising (i.e. masking or scaling) with large split layer size and more than one layer the server end; (a) CNN on MNIST, (b) MLP on IMDB.
large for the improvement to be possible. If a is too small (MSE curve for a= 0.3), the masking does not work; the larger the a, the more improvements one can expect by using masking. Morcover, when a is large enough, there exists an p E (6, 1), for some 5 2 0 such that masking incurs a lower MSE than the baseline. This indicates that optimal denoising achievable by masking for large amounts of noise. Nevertheless, for the same noise level, the MSE of optimal masking denoising is always larger than that of scaling. We provide the backward pass simulations in Figure 0 in SB Figure shows the simulation results for the Laplace
mechanism, and they are discussed in detail in B. along with experimental results in Table 3
Takeaway message. Figures 2 (a), (c), and (a) indicate that, regardless of the noise scale, a, it is possible to find a scaling factor such that using scaling over a noise-injected SplitNN incurs a lower MSE than the bascline. However, this is not always the case for the masking operator: a must be significant to achieve the improvement. In practice, we observe that masking outperforms scaling in terms of improved accuracy,
parameter tuning, and attack defense; see and
4.2 DNN Experiments
Datasets and models. We adopt the benchmarks from the popular Pytorch library Opacus Yousefpour
et al.
2021 with the Split Learning paradigm. contains image classification tasks (on MNIST LeCun et al,
1998
CIFAR-10, CIFAR-100 Krizhevsky et al. (2009). and ImageNet1K] Deng et al. (2009) recommendation
task (movie review prediction on IMDB Maas al. (2011)), language modeling task (name classification
Robertson (0233 and sentiment analysis (Amazon reviews McAuley & Leskovec (2013) ) All training
hyperparanntters are configured as default to maintain fair comparison; see Table in $B
Sctup and implementation. We split the models before fully connected (FC) layers, with variety in the number of FC layers allocated at the server side; see T The size of the split layer varies from 16 to 12544. We use tanh activation function to bound the client's output in [-1. Then, Gaussian noise is injected the tanh layer, with noise scale, o, the standard deviation of the Gaussian distribution. We implement both denoising techniques as post-processing layers on top of the noise-injection process. The ratio E (0, describes the percentage of the elements kept through masking. The scaling factor X = 1 E (0, is used to scale down the tensor values. We performed three runs for the CNN-MNIST, MLP-IMDB, ResNee-CIFARIO, and LSTM-Names configurations, and a single run for the remaining configurations. The overall computation paradigm is outlined in Table {B.6
Denoising performance. We demonstrate the effectiveness of the denoising techniques in various SpliuNN training tasks. In Figure 3 and [ we compare baseline SplitNN, noise- injseeieeeeted SplitNN, and noise-injected different split settings. The noise level, o, is calibrated
SplitNN with the scaling or masking denoising to a relatively high level such that the training accuracy of SplitNN suffers from the noise injection. Both scaling and masking are optimized by parameter tuning on the scaling factor, X, and masking ratio, p; see
9

---

Published in Transactions on Machine Learning Research (04/2026)
Xpriv
4 7
0
9
8
2
Xrec
4 7
0
9
8
3
4
7
0
7
Xvec
Xyec
(a) MNIST
(b) Fashion-MNIST
(c) CIFAR-10
(c) ImageNet
Figure 5: Private training data reconstruction by FSHA attack in Split Learning on MNIST, CIFAR-10, and TmtgeNet. In all cases; X Xpris: the original training data, ie. ground truth; Xee: FSHA on plain-texi. SplitNN; Ny:: FSHA on SplitNN with noise injection (NI) (or 0.7); Xree()): FSHA on SplitNN with NI and scaling (o 0.7, X 0.2); Xre FSHA on SplitNN with NI and masking (o 0.7, p = 0.2). Models are split with one FC layer the server side.
Table 10 in B When models are split at the last FC layer, e.g., in Figure 3 (a) once we inject large noise (a 0.7), the overall training convergence is severely impacted so that the test accuracy is barely increased during the training. In Figure g the training is more robust under high noise injection because the size of the splitting layer is much larger than the one in previous settings. Usually, high-dimensional data can better tolerate noise perturbation since carrics more information. After applying scaling or masking and fine-Luning some hyperparameters, training convergence substantially improves. most cases, e.g., Figure 3 (d),, the improved accuracy due to masking is comparable with the baseline. However, in Figure 4 with scaling and masking, the test accuracy can not achieve the bascline level. This is possible because, by allocating more layers the server side, the client's noisy IRs will also impact more layers during the forward computation. We also notice that in Figure 3 (c), the noise injected training on CIFAR-10 10 performs much better than other tasks, even though the split layer size is relatively small. This is due to its unique default parameter settings, including weight decay and the learning rate, which we explain next.
Denoising Hyperparameter tuning. To better understand the difference between denoising and traditional hyperparameter luning, we evaluate the MNIST image classification task by fine-tuning the learning rate (Ir), weight decay, dropout, masking ratio, and scaling factor under high-level noise injection. We present the accuracy results in $ in Table Full training curves are available in $ in Figure X We change the learning rate (Ir) from 0.1 to 0.001 and find that a smaller improves training stability under large noise injection. Weight decay, a popular regularization method in DNN training, helps prevent overfitting to noisy signals. We find that only a heivy- decay (7 = 0.2, 0.4) can help stabilize convergence throughout training. However, heavy-weight decay sacrifices the convergence speed and fails to reach the baseline accuracy. Scaling can only improve convergence at the beginning of training, and none of them maintain convergence to the end. This implies an inherent training stability issue with noise injection, which cannot be alleviated by pure denoising. Therefore, we combine scaling with weight decay and find that a small weight decay (7= = 0.01) is sufficient to stabilize the training. In contrast, masking optimization does not require weight decay. It can almost achieve the baseline convergence rate once the ratio p is properly tuned. Although random masking and dropout are similar Srivasttva et al. (2014), simply using dropout does not provide sufficient stability during training, regardless of the dropout ratio. Both denoising techniques achieve significantly better training quality than standalone hyperparameter tuning. In BB.2 and Table 5 present initial results for ResNet-18 18 on CIFAR-100; our denoising techniques achieve higher accuracy than simple learning rate tuning. In Table 0 in B.3 we demonsttate that the scaling postprocessing performs well even when Adam's updating rule eliminates gradient scale impact. See limitations the proposed approach in $C In Table we show the performance on the largo-scale datascts. In addition, we discuss the different layers of the networks split in {B.5) along with results in Table g Overall, from Tables 5 and 10 both denoising strategies perform well when X and p are set to 0.1 or 0.2.
we observe that
10

---

Published in Transactions on Machine Learning Research (04/2026)
4.3 Attack defense
Setup. We demonstrate how the random masking technique can improve data privacy in defense against the recent frapure-space hijacking attack (FSHA) Pasquini et al. (2021) in Split Learning. FSHA hijacks the client's learning process from the server side during the training and performs the data reconstruction once the client's output feature is learned; see details of threat model in $B.7 We evaluate the attack performance with 2 models and 4 publicly available datasets CNN for MNIST and Fashion-MNISTT, ResNet for CIFAR-10 and ImageNet. See the model configuration in Table We compare the attack performance by visualizing the reconstructed private data between FSHA attacks on plain-text SplitNN, noise-injected SplitNN, and noise-injected SplitNN with masking or scaling. We focus on the case where only one FC layer is on the server because it is more resilient to data reconstruction atlacks; see the discussion in Figure in
Results. Figure 5 shows that the original FSHA can reconstruct private data with very high accuracy on MNIST but only preserves the appearance of the original images on ImageNet. This is consistent with the attack performance reported in Pasquini et al. (2021], attacks on low-entropy images typically require less effort and can produce high-quality reconstructions. Next, we inject noise into the intermediate results and reconstruct the perturbed data using FSHA. We observe that for MNIST, the digits on the reconstructed image are recognizable. For more complex and color image datasets (e.g.: ImageNett), although noise can obscure image details, we still relate the constructed image to the original by examining the outline or background color. Lastly, when we combine noise injection with masking, the reconstructed images are severely degraded, thereby greatly enhancing data security. The scaling technique has almost no effect on the reconstruction attack, regardless of the scaling factor. See Figures 12 in $ for results with different o. p: À and other datasets. Improving privacy accounting for split learning is not the primary focus of this work; instead, we aim to show how denoising can improve noisy SplitNN accuracy. Nevertheless, we provide
the privacy bounds for one single forward pass during the SplitNN training in Table II
5 Conclusion
We propose scaling and masking as denoising techniques to achieve accurate Split Learning on noisy signals. We show theoretically and empirically that denoising helps achieve more accurate intermediate outputs in DNN training under noise injection that significantly improves the stability and accuracy of Split Learning. Additionally, we show that the masking technique can provide better security enhancement than scaling against powerful attacks. Although in theory, scaling has better denoising efficacy, masking is likely to show better accuracy improvement due to its easier parameter tuning. Finally, we demonstrate the possibility of co-optimization of denoising and attack defense.
Acknowledgments
Aritra Dutta and Subhajit Maity partially supported by the Florida Department of Health Grant, AWD00007072. Additionally, Aritra Dutta is partially supported by the National Science Foundation Grant, 2321986.
References
Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Tyy Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and commanications securily, pp. 308 318, 2016.
Sharif Abuadbba, Kyuyeon Kim, Minki Kim, Chandra Thapa, Seyit A. Camtepe, Yansong Gao, Hyoungshick Kim, and Surya Nepal. Can we use split learning on 1d CNN models for privacy preserving training? In Proceedings of the 15th ACM Asia Conference on Computer and Communications Security, pp. 305 318, 2020.
Borja Balle and Yu-Xiang Wang. Improving the gaussian mechanism for differential privacy: Analytical calibration and optimal denoising. In International Conference on Machine Learning, pp. 394-403. PMLR, 2018.
11

---

Published in Transactions on Machine Learning Research (04/2026)
El Houcine Bergou, Konstantin Burlachenko, Aritra Dutta, and Peter Richtárik. Personalized federated learning with communication compression. Transactions on Machine Learning Research, 2023.
Garrett Bernstein, Ryan McKenna, Tao Sun, Daniel Sheldon, Michael Hay, and Gerome Miklau. Differentially private learning undirected graphical models using collective graphical models. In International Conference on Machine Learning, pp. 478 478-487. PMLR, 2017.
Alexander Camuto, Matthew Willetts, Umut Simsekli, Stephen J. Roberts, and Chris C. Holmes. Explicit regularisation in Gaussian noise injections. Advances in Neural Information Processing Systems, 33: 18603-166111 2020.
Alexander Camuto, Xiaoyu Wang, Lingjiong Zhu, Chris Holmes, Mert Gurbuzbalaban, and Umut Simsekli. Asymmetric heavy tails and implicit bias in gaussian noise injections. In International Conference on Mochine Learning, pp.
PMLR, 2021.
Tianyi Chen, Xiao Jin, Yuejiao Sun, and Wotao Yin. Vafi: A method of vertical asynchronous federated learning. arXiv preprint arXi::007.00081, 2020.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hicrarchical image database. IEEE Conference on Computer Vision and Pattern Recognition, pp. 248 255, 2009.
Cyuthia Dwork, Guy N. Rothblum, and Salil Vadhan. Boosting and differential privacy. In 2010 IEEE 51sl Annual Symposium on Foundations of Computer Science, pp. 51-60, 2010.
Cynthia Dwork, Aaron Roth, et al. The algorithmic foundations of differential privacy, Found. Trends Theor. Comput. Sci., 0(3-4):211-407 2014.
Ege Erdogan, Alptekin Kupeu, and A Ercument Cicck. Unsplit: Data-oblivious model inversion, model stealing, and label inference attacks against split learning. ar preprint arXic:2108.0003. 2021.
Otkrist Gupta and Ramesh Raskar. Distributed learning of deep neural network over multiple agents. Journal of Network and Computer Applications, 116:1-8, 2018.
Michael Hay, Chao Li, Gerome Miklau, and David Jensen. Accurate estimation of the degree distribution of private networks. In 2009 Ninth IEEE International Conference on Data Mining, pp. 169-178. IEEE, 2009.
Michael Hay, Vibhor Rastogi, Gerome Miklau, and Dan Suciu. Boosting the accuracy of diffcrentially private histograms through consistency. Proceedings of the VLDB Endowment, 3(1), 2010.
Zecheng He, Tianwei Zhang, and Ruby B Lee. Model inversion attacks against collaborative inference. In Proceedings of the 35th Annual Computer Security Applications Conference, pp.
2019a.
Zhezhi He, Adnan Siraj Rakin, and Deliang Fan. Parametric noise injection: Trainable randomness to improve deep neural network robustness against adversarial attack. In Proceedings of the IEEE/CVP Conference Computer Vision and Pottern Recognition, pp. 588-597, 2019b.
Sanjay Kariyappa and Moinuddin K Qureshi. Gradient inversion attack: Leaking private labels in two-party split learning. arXiv preprint 21i::2112.01412 2021.
André T Khuri. Advanced caleulus with applications in statistics. John Wiley & Sons, 2003.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. Albert: A lite bert for self-supervised learning of language representations. In International Conference on Learning Representations, 2020.
Yann LeCun, Lóon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEFE, 86(11):2278 222-22244, 1998.
12

---

Published in Transactions on Machine Learning Research (04/2026)
Mathias Lecuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, and Suman Jana. Certified robustness to adversarial examples with differential privacy. In 2019 IEEE symposium on securily and prisacy (SP),
pp. 656 672, 2019.
Oscar Li, Jiankai Sun, Xin Yang, Weihao Gao, Hongyi Zhang, Junyuan Xie, Virginia Smith, and Chong Wang, Label leakage and protection in bwo-party split learning. ar preprint arXic:2102.0850 2021.
Yinan Li and Fang Liu. Adaptive Gaussian Noise Injection Regularization for Neural Networks. In International Symposium on Neural Nelworks, pp. 176 189. Springer, 2020.
Katrina Ligett, Seth Neel, Aaron Roth, Bo Waggoner, and Zhiwei Steven Wu. Accuracy first: Selecting differential privacy level for accuracy-constrained FRM. Aduances in Neural Information Processing Systems, 2017:2567 2577, 2017.
Soon Hoe Lim, N. Benjamin Erichson, Liam Hodgkinson, and Michael W. Mahoney. Noisy recurrent ncural networks. Advances in Newral Information Processing Systems, 34, 2021.
Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, and Christopher Potts. Learning word vectors for sentiment analysis. In Proceedings of the 49th annual meeting of the associalion for computational linguistics: Human language technologies, pp. 142 150, 2011.
Julian McAuley and Jure Leskovec. Hidden Factors and Hidden Topics: Understanding Rating Dimensions with Review Text. In ACM conference on Recommender systems, pp. 165 172, 2013.
Fatemehsadat Mireshghallah, Mohammadkazem Taram, Prakash Ramrakhyani, Ali Jalali, Dean Tullsen, and Hadi Esmaeilzadeh. Shredder: Learning noise distributions to protect inference privacy. In Proceedings of the Sheety-Fiftte International Conference on Architectural Support for Programming Tanguages and Operating Systems, pp. 3 18, 2020.
Milad Nasr and Reza Shokri. Improving deep learning with differential privacy using gradient encoding and denoising, Xixiv preprint or arXiv:2007.1133 2020.
Aleksandar Nikolov, Kunal Talwar, and Li Zhang. The geometry of differential privacy: the sparse and approximate cases. In Proceedings of the forty-fifth araual ACM symposium on Theory of compuling, pp- 351 360, 2013.
Dario Pasquini, Giuseppe Ateniese, and Massimo Bernaschi. Unleashing the tiger: Inference attacks on split learning. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pp. 2119, 2021.
Sean Robertson. NLP From Scratch: Classifying Names with Character-Level RNN PyTorch
Tutorials, 2023. URL http:///torrcoooooo///orooaaaaae cacaatteechhrrraeee rnn___nn_____ classification
tutorial. html]
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1): 1929-1958, 2014.
Guangyu Sun, Matias Mendieta, Aritra Dutla, Xin Li, and Chen Chen. Towards multi-modal transformers in federated learning. In European Conference on Computer Vision, pp.
Springer, 2024.
Yee Teh, David Newman, and Max Welling. A collapsed variational bayesian inference algorithm for latent dirichlet allocation. Advances in neural information processing systems, 19, 2006.
Tom Titcombe, Adam J. Hall, Pavlos Papadopoulos, and Daniele Romanini. Practical defences against model inversion allacks for split neural networks. ai preprint urXiv:2104.0574/ 2021.
Praneeth Vepakomma, Otkrist Gupta, Tristan Swedish, and Ramesh Raskar. Split learning for health: Distributed deep learning without sharing raw patient data. arXiv preprint Xiv: 1812:088200000 2018a.
13

---

Published in Transactions on Machine Learning Research (04/2026)
Pranecth Vepakomma, Tristan Swedish, Ramesh Raskar, Otkrist Gupta, and Abhimanyu Dubey. No peek: A survey of private distributed deep learning. orXiv preprint or arXiu: 18::1812.0 03288, 2018b.
Praneeth Vepakomma, Otkrist Gupta, Abhimanyu Dubey, and Ramesh Raskar. Reducing leakage in distributed deep learning for sensitive health data. arXiv preprint
2019.
Bao Wang, Quanquan Gu, March Boedihardjo, Lingxiao Wang, Farzin Barekat, and Stanley J. Osher. DP- ERM. In Mathematical
LSSGD: A stochastic optimization method to lift the utility in and Scientific Machine Learning, pp. 921-351. PMLR, 2020a.
Chang Wang, Jian Liang, Mingkai Huang, Bing Bai, Kun Bai, and Hao Li. Hybrid differentially private federated learning on vertically partitioned data. arXiv preprinl orXiv:2009.0270 2020b.
Ji Wang, Jianguo Zhang, Weidong Bao, Xiaomin Zhu, Bokai Cao, and Philip S. Yu. Not just privacy: Improving performance of private deep learning in mobile cloud. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 2407 2416, 2018.
Wenxiao Wang, Tianhao Wang, Lun Wang, Nanqing Luo, Pan Zhou, Dawn Song, and Rucxi Jia. Dplis: Boosting utility of differentially private deep learning via randomized smoothing. Proceedings Privacy Enhancing Technologies, 4:163 183, 2021.
Maoqiang Wu, Guoliang Cheng, Peichun Li, Rong Yu, Yuan Wu, Miao Pan, and Rongxing Lu. Split learning with differential privacy for integrated terrestrial and non- con-terresttial networks. IEEE Wireless Communications, pp. 1-8, 2023. doi: 10.1109/MM http///WC.01..204466000
Liyao Xiang, Jingbo Yang, and Baochun Li. Differentially-priie privatt deep learning from an optimization perspective. In IEEE INFOCOM 2019 IEEE Conference on Computer Communications, pp. 559 567, 2019.
Hang Xu, Kelly Kostopoulou, Aritra Dutta, Xin Li, Alexandros Ntoulas, and Panos Kalnis. Deepreduce: A sparse-tensor communication framework for federated deep learning. Advances in Newral Information Processing Systems, 34:21150 21163, 2021.
Ashkan Youselpour, Tgor Shilov, Alexandre Sablayrolles, Davide Testuggine, Karthik Prasad, Mani Malek, John Nguyen, Sayan Ghosh, Akash Bharadwaj, Jessica Zhao, et al. Opacus: User-friendly differential privacy library in pytorch. arXiv preprint arXiv:2109. 12298, 2021.
Yuheng Zhang, Ruoxi Jia, Hengzhi Pei, Wenxiao Wang, Bo Li, and Dawn Song. The secret revealer: Generative model-inversion attacks against. deep neural nelworks. In Proceedings of the IEEE/CVE Conference Computer Vision and Pattern Recognition, pp.
2020.
Keyu Zhu, Pascal Van Hentenryck, and Ferdinando Fioretto. Bias and variance of post-processing in differential privacy. In Proceedings of the AAAT Conference on Artificial Intelligence, volume 35, pp- 11177 11184, 2021.
Keyu Zhu, Ferdinando Fiorctto, and Pascal Van Hentenryck. Post prot-processing of differentially private data: A fairness perspective. In International Joinl Conference on Artificial Intelligence (IJCAI), 2022.
14

---

Published in Transactions on Machine Learning Research (04/2026)
Contents
Introduction
1
2 Related Work
3
Theoretical Guarantee
4
A linear layer
5
Noulinear lunction for classi ficalion lask
3.3
Differential privacy (DP) prescrvation
4 Experimental Evaluation
8
Simulation
8
4.2
DNN Experiments
Attack defense
Conclusion
11
Theoretical guarantee)
16
Definition
A.2 & Regression task
Nonlincar loss fmntion for classitication task
18
3.
Proof of Theorem 3.3
20
A.3 3.2
Sketch of Proof of Theorem 3.4
22
A
3.3
Concentration of the errors
23
A.4
Dillerential Privacy (DP)
23
A.4.1 DP of the gradient during backpropagation
24
B Addendum the Numerical Results
25
B.1
Laplace mechanism
26
B.2
Split learning result CIFAR-100
26
B.3
Understanding the scaling operation when gradient scale impact is eliminated
26
B.
Split learning large datasets
27
B.5
Experiments on Split. Position
28
B. 6
Computational overhead of the postprocessing functions
28
B.7
Fealure-space hijacking altack (FSHA) and our post-processing techniques
30
B.8
DP budget
34
C
Limitations
34
15

---

Published in Transactions on Machine Learning Research (04/2026)
A Theoretical guarantee
First, we will start with the definition of softmax and negative log loss functions used for nonlinear classification.
A.1 Definitions
Softmax and negative log loss. Let m be the number of classes. For a vector, 2 E R", the softmax function, 8 R - (0, 1)", is defined
(2); E e
Let be binary indicator (0 or 1) of the class label, and is the correct classification of the observation, o. Denote Pa,c as the predicted probability of observation that belongs to class c. Then the negative log-loss function defined as
CLr(y.P) = E Nour log(Po.e)-
What loss functions and tasks do we cover? In general, for classification problems such as image classification by CNN, movie review prediction by RNN, and many more, the output layer configured with softmax function for prediction, and the negative log function is used the loss function to train the DNN model. For binary classification, this loss is known as binary cross-entropy; for multi-class classification, it is called categorical cross-entropy. MSE is a consequence of Theorem 3.1 with some modifications. Therefore, our analyscs cover almost all the cxisting loss functions used for DNN training. We refrain from using some rarely used loss functions, e.g., sparse categorical cross-entropy.
A.2 lg Regression task
Proof of Theorem 3.1
Proof. (i) We are required show, EIJAX- MX E|JMX (x)(3)) There are two types of randomness involved one due to randomness in Rg, and the second due to the randomness in X. First,, start by writing
|MX X
m n
77
| - 2(MX, MX) +
m
Σ
(5)
j=1
1Sjckán
which after taking expectation becomes
El| MX 1
m n ΣΣ = --IMXX
m
m (n) + 0
Σ
Mismar5akk
(6)
==
15icken
Next, we have as in (5)
||MX - MR,
= - (R,
ΣΣ
m (R, +2 Σ Σ
miy Ro (i;) (Txo,
i=1 lsjckknnn
(7)
which after taking expectation conditioned on the randomness in Rp given X becomes M - MR,
n
m
=|MX 1AMXX222p(MX,AMM MX) +
m + 2p
(8)
1=1 15ickEn
16

---

Published in Transactions on Machine Learning Research (04/2026)
Finally, taking the expectation on the randomness in X obtain
Ex MMP((X)1(
(1 (1-2p) 2
Σ>
(9)
i=1 cSicken
In view of equation
G and equation
9 we have
M| - M X BIIl (X)
m
= - - M + (1 p)
m (1) +
2(1 - p²)
j=1
i=1 1. Sn
2)
2
Σ
+ p)
m 2 (0) +
;=1
i=1
15jckšn
+2(1 - p²)
1=1 Tojcknn
(1
(0² - - - p)
Σ
mijmarzak
(10)
==
L=1 ISicken
Therefore,
E MX 2 E MX (8)(2)
if and only if the expression in equation 10 is non-negative, that is,
m + 2(1
Σ Σ
m
ISjcken
The left hand side of the above expression is M (which is lower bounded by no? 2 (M), where minn (M) is the smallest singular value of M). For the right-hand side, we have
m Mijay + 2(1-p)
Σ
i=1 Sjckkn
Mija (1 (1
2
m + 2(1
Σ
Σ
i=1 j=1
i=1
15jcken
m
m + (1 (P]
P
j=1
M
O XI + (1- (1-P)]IMX13
(1I)
12
23
En
where
81
D2
F3
Fn
R m Therefore,
31
32
23
Fin
E MX - MX - MIIMX - (8
if and only if
1 2 + +1-
Hence the result.
17

---

Published in Transactions on Machine Learning Research (04/2026)
(i) We are required to show, BJMXX- - EJIMX - (X)(3 2 0. Note that, the only randomness involved in this case is due to the randomness in X. First, we start by expanding
|MX (X)12
2
Z MiyMikk
| (MX,
(12)
a
=1j=1
1=1 15jckEn
which after taking expectation gives
FXI - MS. (X)1B)
(2
+2)
= =11 -1- |M ΣΣ
Σ
Ty MafMik a²
+2
a²
==
1=1 15jckšn
! 2
M
M
(13)
In view of equation G and equation 13 we have
||| MX BIIMX - MS
= (1 - 0 )
(1 -
Therefore,
E|IXX - MX M| MS. (8)
if and only if (1 + )) (1 2. | 2 0. This completes our proof.
7
A.3 Nonlinear loss function for classification task
Now, we will prove the results for the nonlinear loss function given in Section 3.2 First, quote the following Lemma about the moment generating function of random variable, without proof. The readers can find the proof of Lemma A.1 in any standard graduate statistics textbook.
Lemma A.1. Let Z be random variable, Z ~ (N, Then the moment generating function, Φz(.) is given by (t) = [E(ct]] = cut 24
Calculating E C ((M) requires some auxiliary results on the E Σ (X)): and E Em (8)). The following Lemma A.2 gives the details, which necessary for calculating the expectation and the variance Em el (M (X)):
Lemma A.2. We have, (i) EER (M))) = E ITa= ppemaru " 22
-)): and (ii)
+ (1 +(1-p)):
) Eis Th pelmattme))at (mia-mya)8??
+ (1 (1-p))
E m
Proof. (1) We have
m Σ (MR,(8))
m Exs mia Ry(k))
Σ
(14)
where 3o be the gth element of the vector X. Note that, each ma Rx())) is independent (based on the definition of the random masking operator), and after taking expectation the above expression with respect to the randomness in Rp, we have
E
(MR (M),
(Zc))..
(15)
18

---

Published in Transactions on Machine Learning Research (04/2026)
For (0, 1], equation 15 becomes
m
n E E (e
(MR,()))
II (pe + (1 p)e
(16)
which further taking expectation the randomenss in X reduces to
m
m n
E2 Ep
(8))
Σ
Bgpe
+ (1-pp)
A=1
Lemuna A.1
pe
m min
+ -1-1))
(17)
k=1
After taking the total expectation equation 17 and using the tower property of expectation, we obtain the result.
(i) We have
2
Σ
(MM
(18)
1,j
Proceeding similarly as above, first, taking expectation on the above expression with respect the randomness in Rp and then taking expectation with respect to the randomness in X, we have
Ez Ep
(MBx(X)),
Ez Ey
Σ
(ma (ma tmye (a,(a)
Ez
E (mi. (m.))
k=1
Ez
pe (mis +msulan + (1 (1-1) (-p))
i.j
m
(mns q 200
Lemma
A.1
pe pelmsstmaalar +mak)az+
+ (1 (1-p))
(19)
A After taking the total expectation on equation 19 and using the tower property of expectation, we obtain the result.
Remark A.3. Setting p = 1, the loss function, we find the expected loss value, EC (y", s(MX)) due to noise injection (without random masking). Additionally, for p = 1, in Lemma A.2
recover E el (MX),
E H (aat
The following Lemma is the next intermediate result and instrumental in proving our main result as it approximates the expected logarithmic term in the log loss. In Lemma A.4 we approximate E[log[=)] by using Taylor's Theorem.
Lemma A.4. (Khurq 2003, p. 117) Let x be a positive random variable. Then E[log(=)] log[E(x)]
Var(a) + higher order terms, where Var(z) (0(())=((<>)>>>>)))))))))))))>>) (B(x))2. X[
Remark A.5. We assume that have small higher order moments, my = E|r - E(x)P, for 2,3,
Note that, setting = Σm (&)) in Lemma A.4 is the first step to quantify the expected loss value of SplitNN with random masking, ECrn(y" sMM (X)).
See similar expression in Teh al. 2006) with restrictive assumption; assumption in Lemma A.4 more general.
19

---

Published in Transactions on Machine Learning Research (04/2026)
A.3.1 Proof of Theorem 3.3
To prove Theorem
3.3 recall that F(p) == E [LLL(y" RMMRppp (X)))) and consequently,
F(1) E Ly ble, we write
(trlv""ssMMM))) s(MX)) By using Lemma A.4 and assuming the higher order terms are negligi-
Var Di=r ((MB),(X)))
m (M))
F()) = p(M)) + log
(20)
2 E e(MRp(X))) 2.
==1
BB
CCppp
DDp)
Diffcrentiating equation 20 with respect to p gives
F'()) = 5 (1)) +C'()-
Note that,
dp |E ((M
('(p) -MMX),,
EΣ ((M
but the derivative D(p) becomes very messy. So, we take the following indirect route: we first show that (1) (1(1) + (1) 2 and hence B(p + C(p) are increasing in neighborhood the left of p = 1; then we verify that (di) (p)) -(1). We that once accomplish (1) and (ä), will have F(p) F(1), which completes the proof of Theorem 3.3
Proof of (t).
By Lemme A.2 and a straight computation, we have
m
m² 0² (e mirk -
(M),
E
1)
dp
i=1
So,
2 x mortt " - 111l memur m2
iiml DP
CC(1)+C((1) x +
(emeret " "3a))
E
which is bigger than equal to if and only if
m2 mert
Il ( meme m
M
E
(MX))*,
(21)
mn "1
Σm -
IT
(e
or equivalently,
c mir xy mi 2
2 x2
miga 02
1)
(e ma
(MX); Σ
k
20

---

Published in Transactions on Machine Learning Research (04/2026)
Let f(o2) denote the difference of the two sides, have f(o²)
(e (e m2
m Bikkk
)
(MX)+
"4 ))
ktr,k==1
m eu
m2
-
(MX)+ + Σ (1
e
rml
"i o2
m Map
k=
Now, note that Σ Er=1(1 Mr-3Ir M.r an increasing function of a² and as a² - +00, it approaches Σx=1 (1 mp.x., n- -n-E -, Tr which is positive by assumption. Thus, when o² is large
enough, ((²2 > O. This verifies (21
and hence (x).
Proof of (éi).
We need to verify that D(p) 2 D(1), that is, by the definition of D(p),
Var
Σ mm
(()),
Var
Σ i== ((
2 2 E ((MR)
2 m 1= (()))
By the formula Var(x) Exx (Ex)², it suffices to verify
E Σ i=1 (LMB,(()))
(EEA, c(MRR((X)))) )²
Em e(seH)(()3.))
Ei=1 (x)):, ? (EΣ" ((MB,(8))) 2
E Em c(MR)((8))
which can be simplified to
m (x));
2
m (M))) 2 ((M,,
E
m Di
E
m
p=1
Using Lemma A.2
the above is the same as
pe (m tm myylaat (mas mo
Σn Ih
+ (1--p)
tmyea) (m:e-mm
Σn, m
(22)
Σm Ih me m2
pp) )
m o2 Minmm
2
+ +1 (1-p)
Σ
C
We now verify
22 for o² large enough. Note that (22 is true if and only if the following function g(p) 2 0
where
m² m 2
pe (max (ma)aa) (mii m2 + - p)
m
x(p)
Σ
EI
1=1
T=
(m.. (m m e (m 1202
m
12
2
Σ
"ig"
+ (1 - p)
k=
21

---

Published in Transactions on Machine Learning Research (04/2026)
Note that 9(1) = 0. So, it suffices to show that g(p) is non-increasing small neighborhood to the left-hand side of 1. Differentiate g to get
m
(mattmy)aas (114 (mat-mmp Ro
1 (mir +my-)a,- )x,-- (m tm yo Po2
)(())
x I
m
n 12 MoL
and J= Et, ITk= elmax (lma+myaaar (m m
where : Em
We have
(mi tmya)ay (mie
i2 1202
1 (mis (mp) Oner
Po? )
d(11=
x I
f.j
JJ x x
- mirtt
1=1
(ma + (mig mma 202
Emarr j o202
=1
(n Σ
(mir +m
x 1
JJ k=1
m 0² +
m² 7
2 X
2
(n
Mirdr
((mic +m (mmm+mmp
2 mm )
(mig Imgge 2
nx X
(ma + tmy +ma))))
(mir +myr)
k=1
mma +mxxle+ tomattaee
+m? 2 2o
n
2
-2n x J x 1 Σ
My
6..0=1
TD
(ma moan-mya taga tm b. (imis + my
Rmm2 ) 2
(m
n n
e
r=l
m
11
(12, m22--1pe mo (Gngg my m² 30 n
m
Mar
4J.s=
(ma+ + 1o4t (ome +myx+ 2 202
1,j.8
(m) (mp.-m3p))..
+ 2e -.mr. m2
-1
The last sum above clearly goes to as 2² +00. Thus, for o² large enough, we have g'(1) < 0 which implies g(p) > (1) 0 for p close to from the left hand side. This completes the proof.
A.3.2 Sketch of Proof of Theorem 3.4
The proof follows similar line of arguments the proof of Theorem and thus, we point out only the main differences. We need to verify ELLi("" s(MS, (X))) < ELL. (v", s(MX) for a close and larger than 1.
22

---

Published in Transactions on Machine Learning Research (04/2026)
With À = 1/a, one can show (as in Lemma 2) that
E
(S
Dis Mit 1200" Ei m2x/2
==1
and
2
E
m
MSa((t)
m
m
(ma tmya + E (ma +
j=1
Next, one can establish similar steps of 20 proof.
21
for the current case and eventually complete the and
22
A.3.3 Concentration of the errors
Linear layer. Both conditions in Theorem 1 are necessary and sufficient and indicate implicit relations between the input, X, the denoising parameters, a. the weight of the split layer, W, and the added noise magnitude a. For the masking case, the coefficient of the linear term, which can be positive or negative depending on the noise scale σ, dominates MSE; see Figure 2 (b). However, for Theorem (ii), from equation 13 we observe that the MSE depends quadratically on the scaling factor 1. where 1. Therefore, a. one can obscrve a quadratic relation between the scaling factor 1 and MSE in Figure 2(a).
Nonlinear layer. For nonlinear loss functions, this relation is the loss is (y, Rx)))), which is given in equation (2) in the main paper (also, equation (18) in the Appendix). In this scope, we show how the bound on the error, E [CLL (y" s(MX) , RMMRp(X)) depends on 2 and (1 - p). Based on equation (18), the first term will be - P)MM)) which linear in The next term we need to consider is log EE (( - log (EJΣ c(MX).)) By directly
complicated to observe. For Theorem 2,
(1 (1-)). p).
manipulating the expression in (15) for p and p we find this quantity approximately is
En midck {(p" 11 o²p"
Σn n
k= m
+ Higher order terms) O p).
2
When the noise is not loo large, say, o² 2 mik in the variance of noise, o², One can obtain similar observation for Theorem 3 as well.
< 1, can observe that the bound approximately linear
A.4 Differential Privacy (DP)
We start by defining differential privacy.
Definition A.6. Dwork et al. (2014) A random mechanism, XC: D - RT is (c, differriffr private for all adjacent inputs, D2D' D, with Hamming distance, d(D, 1, and all possible output, O IRm E B(")) such that
P[C((D) E O e P[K((D)) 0 +
where P[:] refers to the probability associated with the random mechanism K. and B(R") is Borel sets in R".
We also need the general theorem that says any differentially private mechanism, I. D O, is further differentially private if it is transformed by arbitrary postprocessing function, deterministic or random; see Theorem 3.6 in the main paper.
Finally, if we perform multiple computations of the random mechanism, K. on the same dataset, D, that is, we make T such passes on D, then the privacy guarantee degrades. We quote the advanced composition theorem from Dwork al. (2010) for such mechanisms.
Theorem A.7. Dwork et (2110) Let K: D O O be an T-fold adaptive composition of (c, 6D
T-timos
mechanisms. Then K. is (e', T8 + 8') DD for e' 2n Te(e" 1), for all 8'
23

---

Published in Transactions on Machine Learning Research (04/2026)
A.4.1 DP of the gradient during backpropagation
Setup. Let (,)), be training data points, Let the hth layer of an L layer DNN be X let & be a diffcrentiable activation function, and let M2 be the weight matrix for the 1th layer itcration By this convention, X; = XP. At
we have
x = (MG of = -2X j=1,
(23)
For the noisy split neural network with post-processing, let the split happen at the (1 1)th layer (so, the cut. layer is the (I - 1)th layer). With the notations above, we have:
of M j=1....11- x X A X hM V1 = MxPho(X)), and X = tb MMg
x = B (M
(24)
x ). X k= = 1.....
In our experiments, we use the regular backpropagation formula in our noisy split network training, which we formalize in the next Proposition.
Proposition A.8. With the nolations above, for noisy splil network with post-processing, during training, one can use the regular backpropagation algorithm by substituting: (i) v = 8 X = Xx for all subsequent k= L- 1, ................................ and (ii) v = Vi X = X? for all r = 1,2, .., 1, before the split, and y = J.. for k= 1, ......... L after the split.
Define the sets, S S 1m and S as follows:
{
((())(DD) V(D'))) s | (z:pp() S < V(D) V(D))))
(25)
Proposition A.9. Let 6, > 0, and let random mechanism K. D - Rm be defined as in 33 Then there erists a bound As such that, for any two adjacent datasets, D and D' when (DV VDD"I
S D1, we have
that the random variable Z will satisfy Plw: p(Z(w)) e p(Z(w) V(D) - V(D')) 6.
The above Proposition can be verified as follows. Because, for two large enough datasets, and D', let A = [|V - (D)]). Using Taylor expansion, we have
p(2) - +(= V(D') = =[2) - - + Vp(z) (V()) + s =1==)))(z) + +(A) so.
Here, the last inequality is achieved by choosing small A
Theorem A.10. Let random mechanism, K: D R defined in equation Then for any (c. one can choose A small enough such that K. is (c.8)-DP.
obey Proposition A.9
"'D)'' P[z O'), where O' is shifted output set. We split the set O' into two disjoint sets, O' ns and n's.s Therefore,
0'=0-V(D))
Proof. Note that, P[K(D) E 0
P(C(D) 0 = P(z E 01l
=
Piz ((
E (O' n S)] + Pl= (O')s))
o'nscs
P(z E $] + P(z (0 nS)
Proposition
A.9
8+ P(ZE (0' nn))).
(26)
24

---

Published in Transactions on Machine Learning Research (04/2026)
Model & Dataset
Split Config. (denoted by
Layer Size
CNN on MNIST
2x 2xCon22 FC [| FC
Loss
256
2xConv2d || FC - FC Loss
12544
MLP on IMDB
Embedding FC || FC
Loss
16
Embedding FC FC
Loss
4096
ResNet- 20 on CIFARIO
Conv2d 3 ResBlock
|| FC
Loss
256
ResNet-18 on CIFAR100
Conv2d 1x 4xReBBlock
FC
Loss
512
RNN on Names
Embodding RNN || FC Loss
256
Re on ImageNet1K
Conv2d 4xReBBlock
FC Loss
512
SaBERRT-base-Tae on Amazon Reviews
12ceer
FC Loss
768
Table 1: Model split configurations and split layer sizes.
We also have
By equation
25
P(z (())) seo'ns: p(z)dz
S
e
p(= V(D)V -(D))-
ZEO'nSe
O' O'ns"co' S
p(z -2+V(D)-V(D'))
J plu)du
S.AVIDYEO p(z)dz
+= e'Pi((D))
(27)
Combining equation 26 and equation
27 we gel the result.
Let (D) and (D) be two gradient vectors computed on the training dataset D over a minibatch B at iteration without and with noise injected. Based on Proposition A.8 we have:
(D) _g (D), - (D), -/2 (D)) (D)).
(D)
Th (D), (D). (D)).
Define == (D) (D) be random vector. Hence,
KBp(D) = 3's = d (D) +2
(28)
where 2 = G(E, 3) is a random variable with probability density function p(2 E = E In our case, we use masking and scaling as a postprocessing function, lD(.), after the Gaussian noise injection. The random variable, &~ (O,, oT) continuous. The mask, Rp, is random matrix of and 0 with identical and independently distributed entries, (Rp)i Bernoullip)). discrete distribution. For scaling, S..
EO)
elementwise scaling operator. Therefore, for masking, Jf M, Rx(x)), and the entries of Rx())) = x xD) A or 0, based on (Rp)ig Bernoullipp). On the other hand, for scaling, /f = MfS)(X() and the entries of (X)) (x (D) + A). Therefore, based on Theorem A.10 this mechanism is also differentially private.
B Addendum to the Numerical Results
Due to limited space, we were unable to discuss many experimental details as well as many results in Section of the main paper. We discuss them here in detail.
25

---

Published in Transactions on Machine Learning Research (04/2026)
Table SplitNN setup and training hyper-parameters.
Model
Dataset
Optimizer
Batch size
Epoch
Ir
Weight decay
CNN
MNIST
SGD
64
4
0.1
0
ResNet-20
CIFAR-10
SGD-M
128
160
0.1
le-4
ResNet-18
CIFAR-100
SGD-M
128
200
0.1
5o-4
MLP
IMDB
Adam
2
0.01
LSTM
Names
SGD
800
150
ALBERT-basee
Amazon Reviews
AdamW
256
10
5e-5
1e-2
ResNet-50
ImageNet1K
SCD
256
90
0.1
lc-4
0.0.7
0=0.5
0=0.3
2.0
2.0
1.5
1.5
1.0
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Scaling factor (A)
Masking ratio (p)
(a) fX()= OCrely" ((MX)) ((((M))
(b) (X() @Crefv' x(MX)
((L(M))
Figure 6: Backward simulation. Simulation of how scaling factor (A= ) and masking ratio (p) influence the estimation error (MSE) under different noise levels (a) for linear (fi) and nonlinear functions (f2,f)) The backpropagation errors are essential during the training as we use them together with forward IRs to directly compute the gradients. By taking the derivative the loss function with respect to the TRs, we obtain the simulation of the MSEs for the backpropagation errors. Similar the forward pass, scaling and masking can lower the estimation error during the backward pass, especially when the noise level is relatively high.
B.1 Laplace mechanism
The Laplace Distribution (centered at 0) with scale b is the distribution with probability density function:
[x]]
Lap(æ: b)
exp
2b
Here, we consider the Laplace mechanism protect the input vector X. Simulation results (Figure show that our denoising methods also decrease the estimation error caused by the Laplace mechanism during the forward and backward passes. However, in the real split learning task (sec Table B). our denoising methods are less effective for large Laplacian noise b = 0.7, compared with a = 0.7 in the Gaussian mechanism. A more detailed investigation is left for future work.
B.2
Split learning result CIFAR-100
We provide results the CIFAR-100 dataset with ResNet-18 in Table 5 As shown in the table, both our denoising techniques achieve higher accuracy than simple learning rate tuning.
B.3
Understanding the scaling operation when gradient scale impact eliminated
To further understand the scaling operations, we rum the same MNIST experiment with two Adam optimizers separately for the elient and server. The results in Table T demonstrate that the efficacy of scaling operation
26

---

Published in Transactions on Machine Learning Research (04/2026)
b=
b= 0.5
b= 0.3
1.6
1.6
1.4
M5E
MSC
5
1.0
Σ 0.6
0.6
0.6
0.2
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.2 0.4
1.0
Scaling factor (A)
Masking ratio (p)
Scaling factor (A)
Masking ratio (p)
fi()) MX
fi(X) M X
(c) fa(X) LLL(V" ((MX))
(d) fx(X)
LLa(S" ((MX))
2.5
2.0
1.5
1.0
0.2 0.4
0.8
1.0
0.0
0.2
0.G 0.8
1.0
Scaling factor (A)
Masking ratio (p)
(e) [2( AGzafy" M(NN)
(f) [I( #C ((M))
((((M()))
= ( and masking ratio (p) influence the estimation (MSE) under different Laplacian noise levels (6) for linear and nonlinear cases. Plots (a) (d) are the linear layer and nonlinear layer during the forward pass, while (e)-(f) are the derivatives of the nonlinear layer during the backward pass.
Figure 7:
Simulation of how scaling factor (A (A=
Table Denoising performance by using Laplacian noise split learning for MNIST classification task (same experiment setting as Figure Bab. We fine-tune hyperparamelers À and p for different Laplacian noise scales b 0.3, 0.5, 0.7. Compared with the Gaussian mechanism, split learning suffers more from Laplacian noise injection.
acc. (%)
0=0.1
0.0.2
p=0.1
p=0.2
p=0.4
0=0.6
98.96 (10.13)
0.3
92.64
(+0.11)
95.08
(40.25)
94.49
(+0.31)
92.16
(+0.38)
90:93
(-049))
98.19
(+0.67)
98.55
(-0.82)
98.32
(+0.17)
05.54
(-0.21)
38.30
(10.27)
94.58
(10.18)
(10:27)
(10.58)
(10.35)
(10.39)
97.48
(.0.84)
05.17
(10.31)
80.01 (-0.37)
0.7
16.62
(=0.10)
21.44 (30.13)
23.19
(=0.88)
20.32 (20.31)
1171
(=0.22)
12.85
(20.27)
23.62
(=0.15)
18.42
(20.49)
15.58 (=0.19)
still exists even when gradient scale impact eliminated by Adam's updating rule. If compare our scaling method against learning rate tuning, the accuracy gain should be from 92.80% to 98.14%.
B.4 Split learning on large datasets
We use two large-scale datasets, Amazon Reviews McAuley & Leskovec (013)) for sentiment analysis and ImageNet1K Deng et al. (2009) for image classification tasks, and investigate how the proposed denoising strategies perform in large-scale, challenging datasets. For the experiments on the Amazon Reviews dataset, we use a pre-trained ALBERT-basc-v2 Lan et al. (2020); for the ImageNet experiments, we use ResNet50; see Table for the results. We provide the split confiurrations and the training hyperparameters for these experiments in Tables D and 2 respectively. As the ALBERT-base-ve v2 a pre-trained model, we fine-tuned it the Amazon Reviews dataset, and observed that noise injections denoising strategies, such masking and scaling, had an insignificant effect on the bascline performance. Noisc injections do not degrade the performance, and denoising strategies do not improve them either. On the other hand, the performance of ResNet50 on ImageNet was heavily impacted by the noise injection. E.g., a noise injection of a = 0.5 renders a test accuracy of 1.92, which is 97% lower than the bascline accuracy. When we apply the denoising strategies to the noise-injected IRs, at the same noise level o 0.5, the scaling strategy with À 0.1 and masking strategy with p = 0.2 recover performance comparable the baseline.
27

---

Published in Transactions on Machine Learning Research (04/2026)
Learning rate
0.001
0.005
0.01
0.05
0.1
Top-1 Acc. (%)
87.68
92.67
81.21
diverge
diverge
Weight decay
0.01
0.05
0.1
0.4
Top-1 Acc. (%)
diverge
diverge
diverge
87.15
77.78
Dropout
0.1
0.2
0.4
0.6
0.8
Top-1 Acc. (%)
diverge
diverge
diverge
diverge
diverge
Masking
0.1
0.2
0.4
0.6
0.8
Top-1 Acc. (%)
98.31
98.62
diverge
diverge
diverge
Scaling*
0.1
0.2
0.4
0.6
0.8
Top-1 Acc. (%)
98.10
96.06
90.25
diverge
diverge
Table 4: Comparison of tuning various hyper-parameters in noise-injected SpliCNN training at fixed noise level (a 0.7) for MNIST classification. mcans co-optimization with weight decay.
Table 5: Top-1 Accuracy (%) of split learning using ResNett-1 on CIFAR-100 dataset with noise injection
level (a 0.7). The baseline achieves 75.60% Top-1 accuracy using SGD-Momentum (m=0.9) with an initial learning rate of 0.1, and weight decay 5e
Learning rate
0.001
0.005
0.01
0.05
0.1
Top-1 1 Acc. (%)
53.11
68.93
61.54
diverge
diverge
Masking (Ir=0.1)
0.1
0.2
0.4
0.6
0.8
Top-1 Acc. (%)
60.49
72.38
63.31
52.97
diverge
Scaling (Ir=0.1)
0.1
0.2
0.4
0.6
0.8
Top-1 Acc. (%)
72.21
71.68
68.14 14
57.54
diverge
B.5 Experiments on Split Position
We investigated the network behavior in two-party split lcarning setup and experimented with different positions of the split layer in a network. For this, we selected the configurations of MLP on the IMDB dataset and ResNet20 on the CIFAR10 dataset with the highest noise-level, a = 0.7; see Table S for the results. We obscrve a performance degradation when the position of the split layer carly in the network. However, the proposed denoising strategies successfully improve performance on top of the noisy IRs, even though the recovery to the baseline accuracy becomes difficult in such cases. We also observe that for ML. architectures, the scaling strategy is more effective for denoising, while the masking strategy is more effective for ResNet architectures.
B.6 Computational overhead of the postprocessing functions
We measured the computational overhead introduced by the denoising techniques both CPU and GPU. The results are shown in Table y The computational overhead of scaling and masking is negligible compared to the training computation.
Table Top-1 Accuracy (%) of split learning on MNIST dataset with noise injection level (o 0.7). Use two Adam optimizers separately for the client and server. The Adam baseline (Ir=le-3) achieves 98.95% top-1 accuracy.
Learning rate
1e-6
le-5
1e-4
1e-3
0.1
Top-1 Acc. (%)
83.96
92.80
90.13
diverge
diverge
Scaling ratio (Ir=le-4)
0.1
0.2
0.4
0.6
0.8
Top-1 Acc. (%)
97.19
97.94
98.14
96.03
91.75
28

---

Published in Transactions on Machine Learning Research (04/2026)
Table 7: Performance of scaling (A) and masking (p) at different noise levels (a) for large-scale datasets, ImageNet-1K, and Amazon Reviews Full.
Task
Noise level (a)
Best acc. (%)
A=011
p=0.2
74.85
ResNet50-ImageNe
0.3
31.18
74.26
73.79
0.5
1.92
72.93
73.07
65.19
Reviews
0.3
65.15
65.12
65.15
0.5
65.18
65.16
65.13
Table 8: Performance of scaling (A) and masking (p) for different splits of the networks.
Task
Splits
Best
(%)
A=
A=
p=
p=0.2
85.53
MLP-IMDB
Embedding PC FC- Loss
0.7
76.28
83.25
82.35
82.82
83.41
Embedding FC FC Loss
0.7
09.08
73.61
72.91
64.95
66.12
91.76
ResNee20-CIFAR10
Conv2d Rlock
FC Loss
0.7
80.69
86.69
87.16
89.07
89.48
Conv2d 2x ResBlock
TeReBllock PC- Loss
0.7
66.16
35.53
38.22
85.01
86.95
1.0
1.0
0.8
gm0.7 Ir-Se-3
0.8
ano ym
Test aCc.
o=0.7 1--Se22
o-0.7 y-0.2
0.6
e=0.7 Ir=le33
0.6
geo 7 y
0m0.7 Irste-2
ymn 05
0.4
0.4
p-o.7
ym0.01
Baseline
0.2
0.2
0.0
0.0
500
1000
1500 2000 2500
1000 1500
Step
Step
(a) Learning rate
(b) Weight decay
1.0
1.0
0.8
0-0.7 0a0
0.8
0-0.7 p-0.8
Test 8Cc.
gm0.7 0.
gm0.7 p-0.8
0.6
o-0.7 0.
0.6
o-0.7 p-0.4
c=0.7 ratlow0.2
e=0.7 p-0.2
0.4
rotlo=0.1
0.4
6-0.7
0.2
0.2
0.0
0.0
500
1000
1500 2000 2500 3000
500
1000
1500
2500 3000
Step
Step
(c) Dropout
(d) Masking
1.0
1.0
0.8
0.8
y=0.1
o-0.7
A-0 y-0.1
0.6
a-0.7 2-0
0.6
0-0.7 2-0 y-0.1
Test
gud,7 0.0.2
Test
0u0.7 xud y.0.5
0.4
amn, 7 Am0.1
0.4
nm0.7 Amo.
Basctine
Basotine
0.2
0.2
0.0
0.0
500
1000 1500 2000 2500 3000 Step
0
500
1000 1500 2000 2500 3000 Step
(e) Scaling
(f) Scaling + Weight decay
Figure 8: Comparison of tuning various hyper-parameters in noise-injected split learning at fixed noise level
(a = 0.7) for MNIST classification task.
29

---

Published in Transactions on Machine Learning Research (04/2026)
Table 9: Run time profiling for one batch training of CNN on MNIST dataset. GPU: NVIDIA A
80GB GPU. CPU: Intel Xeon Platinum 8260 CPU @ 2.40GHz.
Hardware
Baseline
Noise injection only Noise injection w. masking
Noise injection w. scaling
GPU 1.54 ms
1.88 ms
1.91 ms
1.92 ms
CPU 22.16 ms
22.27 ms
23.40 ms
23.19 ms
Table 10: Syyper-paramete tuning for scaling (A) and masking (p) at different noise level (o). Results are obtained by running the experiment 3 times with different random seeds. We record the best test accuracy during the training instead of the final accuracy.
acc. (%6
Xm0.1
Am0.4
Am0.6
p=0.1
pm0.2
pm0.44
pm0.0
0
98.06 (+00.13)
CNN-MNIST
0.3
0.5
08.40 (+0.07)
(#:0.88)
07.63 (+0.27)
(=0020)
97.13 (+025)
(4068)
96.07
(-0.10)
(000
96.42 (-0.17)
98.78 (+0.15
98.03 (+0.14)
08.88 (+0.19)
(=10.16)
98.86 (+0.311
(+0.30)
94.26 (+1.11)
98.77 (+-0.06
0.7
81.85
(=00.77)
07.11
(#0.15)
50,30 (40.36)
90.35
(20
88.88 (=0.22
98.31 (80038)
98.02
(+0.23)
96.67 (40.34)
9051 (4:100))
0
01.77
(+0.228
2e CIEARLO
0.3
00.086
(#:0.20)
89.10 (=0.54)
50.18 (#0U5)
90:87 (=0.49
90.84 (=0-48)
88.09 (+0.80)
89.54 (=11.07)
50.50 (#10.26)
9001 (+:0.31)
(+0.40)
(+0.50)
(+0.7)
(-0 72]
(-002
(+0.50)
80.55
60.90 (+00.96)
89465
0.7
82.08
(++0.76)
88.88 (+0.74)
67.95 (+0.3)
87.21
(-0.70)
86.90 (-0.88)
$8.45 (+0.00)
80.15 (+1.16)
02 (+1.26)
87.00 (+1.41)
0
85.53
(#00118
MLP-IMDB
0.3
65-12 (+0.30)
65.85 (+0.63)
65.00 (+0.17)
84.72
(-0
85.47 (-0.05)
85 (0.00 (+0.35)
85.54 (+0.55)
65.68 (+0.51)
65.21 (+0.77)
(+0.63
(+0.68)
(+0.4)
(-0.68)
84.55 (m0-72)
86.55 (+0066)
86.00 (=00.55
65.18 (+0062)
85.92 (+1.22)
0.7
64.91 (+1.711
84.00 (=00.85
84.24 (=0.94)
82.83 (=0.81)
80.30 (=0.71)
85.11 (#00.40)
85.08 (=10.30)
53.27 (+1.38)
81.88 (#1.000
0
81.24 [++0.25]
X.meM-Nnmes
0.3
82:31 (+0.58)
83:70 (+0.58)
82.35 (+0L2T)
81.17 (-0.31)
80.51 (-0.59)
80.52 (+0.30)
82.00 (=1,,0)
81.63 (+10.06)
82.23 (+00.53
0.5
16.91 (#1142)
(=0164)
(#007H)
81 (=0.45)
(=0.96
80.13 (=0.022)
82.54 (=1.0))
82.04 (#:1.21)
82.67 (#:0.06)
0.7
(::1.97)
81.56 (1.0.34
(10038)
81:07 (=0.81)
(=0.57)
79.35 (.00.15)
81.15 (2:0.75)
(1:0.75)
06.59 (1:1.3))
B.7 Feature-space hijacking attack (FSHA) and our post-processing techniques
Threat model. We assumee that the attacker has no information on the architecture of the client's 's model and its weights However, the attacker knows a public dataset that captures the same domain as the clients' training sets. For example, if the model is trained on face images, then the public dataset is composed of face images as well. This assumption is more realistic and less restrictive than the ones adopted in other works Vepakomma et al.] (2019 Venakooma et al. (2018b), where the attacker is assumed have direct accoss to leaked pairs of intermediate results and private training data.
In the FSHA Pasquini et al. (2021) attack, the attacker (e.g., the server) can hijack the client's learning process and learn an inverse version of the client's model. During the inference, the attacker can recover the client's raw data by using the output of the client. In this work, we showed that the masking operator simultancously improves the SplitNN training and, in the meantime, decreases the efficacy of the FSHA attack. Our intuition is that the denoising effect depends on the specific application, which is a function applied to the noisy input, X + A. If the function's goal is to identify each value of X, such as reconstructing an image in an FSHA attack, then denoising cannot help too much. However, if the goal is to get a more accurate estimation on some statistical metrics of X, such as the mean, norm of X, then it is possible to have an evident denoising improvement.
Table 11: We provide the privacy bounds for one single forward pass during the SplitNN training, for various noise levels used in our work, without denoising, We compare it with two state-of-the-art results for private split learning.
Method
Model Dataset
DP mechanism
Inference
Training
Noise scale
bounds
Titeobbe of al. 2021
21F CNN on MNIST
Taplace
U.T, 1.0
N/A
Abundbba el al. 2020
1D CNN on medical data
Laplace
N/A
113.5,7.30
Our work (w/o sampling amplifccttoo))
2D CNN MNIST
Gaussian
0.3, 0.5, 0.7
(260, 150, 110) 6= le-5
Re Net ou CIFARIO
Gaussion
0.3, 0.7
< =264, 152, 112) 6= le-5
MLP IMDB
Gaussian
0.3, 0.5. 0.7
(168, 96, 72)"2, 6= le-5
LSTM on Naunes
Gaussion
0.3, 0.7 0.7
<= =9.2, 5.3, 3.9) le-5
Our work (w/o sampling ampliicctioo)
2D CNN on MNIST
Laplace
0.3, 0.5.0. 0.7
(53.3, 32, 22.8)²2
ResNet on CIFARIO
Laplace
0.3. 0.5, 0.7
c- (50.3, 32, 22.))²
MLP IMDB
Laplace
0.3, 0.5, 0.7
(13.3, 8, 5.7)
LSTM on Namees
Laplace
0.3. 0.5, 0.7
c (53.3, 32, 22.8)²2
30

---

Published in Transactions on Machine Learning Research (04/2026)
2
6
3
3
2
9
9 a
2 6
3
3
2
9
9
6
2
6
3
3
2
9
7 6
2
6
3
3
2
9
9 6 6
2
6
3
3
2
9
7 a
2
6
3
3
9 /
2
7 3
Xyes
6 10
2
6
3
3
2
9
7 3
3
4
3
3
4
= 2.0
3
6
3
3
2
4
7
3
(8) 2 layers on server side, split layer size 12544
(b) layer on server side, split layer size 256
Figure 9: Private data recovery by FSHA in split learning on MNIST. Xpris: the original private data, Xreci obtained by FSHA attack. We compare two different split learning settings: (a) split 2 FC layers the server side, (b) split 1 FC layer on the server side. The dimensions of the split layer are also dilferent. We can see that by splitting more layers on the server side, it is more likely to reconstruct private training even when the noise level is relatively high (a = 2.0).
priir
2
0
7
9
8
O
9
6
6
3 9
/ 5 3
7 4 0
8
2 2
0
7
/
9
8
O
9
6
6
3
9
/
4 0
0.3
4
0
2
9
8
O
9
6
6
\
3
9
5 0
Xycc
0.5
4
0
0
0
/
9
8
O
9
6
6
3
9
/
532 0
0.7
4
0
0
/
9
8
O
9
6
)
5
3
7
4
0
(4)
0.1
Xye
0.2
0.7
0.4
0
6
/ 7
0.6
p= 0.8
0
/
9
9
3
7
4
(b)
1= 0.1
4
7
0
6
7
9
8 O
Q
6
6
3
5
3
7
4
0
1 022
4
7
0
6
/
9
8
O
Q
6
6
\
3
9
5
3
7
4
0
0.7
1 = 0.4
4
0
0
6
7
9
8
O Q
6
6
\
3
9
/
5
3
7
4 4
0
= 0.6
4
2
0
6
/
9
8
O
Q
6
6
3
4
2
5
0
4 2
0
6
9
8
9
6
3 9
5
3 7 4 0
Figure 10: Private data recovery by PSHA in split learning on MNIST. Xprio: the original private data, Xpec: obtained by FSHA attack various settings: (a) noise injection only, (b) noise injection + masking, (c) noise injection + scaling.
31

---

Published in Transactions on Machine Learning Research (04/2026)
Xprt
0.3
X
0.7
(8)
p
K
p 0.2
0.7
p 0.4
p=
p
2 0.1
Xyec
À
0.7
1 0.4
2 0.6
1. 0.8
Figure 11: Private data recovery by FSHA in split learning on Fashion-MNIST. X Xprio: the original private data, Xree: oblained by FSHA attack in various settings: (a) noise injection only, (b) noise injection + masking, (c) noise injection + scaling.
32

---

Published in Transactions on Machine Learning Research (04/2026)
Xprt
0.3
X
0.7
p
X
p 0.2
0.7
0.4
p
(b)
2 0.1
Xyec
À
0.7
1 0.4
2 0.6
1. 0.8
Figure 12: Private data recovery by FSHA in split learning on CIFAR-10. Xpriei the original private data, Xree: obtained by FSHA atlack in various settings: (a) noise injection only, (b) noise injection + masking, (c) noise injection + scaling.
33

---

Published in Transactions on Machine Learning Research (04/2026)
B.8 DP budget
We provide the privacy bounds for one single forward pass during the SplitNN training in Table 11 By using the general composition Theorem, Theorem A.7 in the Appendix, we can calculate the total privacy budget for the entire training process. Although Theorem A.7 gives a theoretical formalization, in practice, if we use Theorem A.7 directly, it will result a large privacy bound, which may not be practical. As argued in the paper DP-SGD, Abadi al. found that even for local SGD training using Theorem A..7 would result in a large privacy bound and proposed a new accounting for local noisy SGD training. However, noisy SplitNN is a more complicated architecture, and DP-SGD analysis cannot be adopted here. Currently, privacy accounting for noisy SplitNN training remains open problem. Improving the privacy accounting for split learning is not the primary focus of this work; instead, we want to show how denoising can improve noisy SplitNN accuracy.
C Limitations
Our denoising techniques work empirically on diverse datasets (MNIST, FMNIST, CIFAR-10, CIFAR-100, ImageNet1K, IMDB, Amazon Reviews, and Names) and across different network architectures (CNN, RNN, Transformer, and MLP). One potential limitation of denoising techniques is the empirical challenge of selecting an appropriate scaling or masking ratio. Another potential limitation is that our proposed denoising techniques may not generalize to other loss functions. However, theoretically, we covered almost all the existing loss functions used in common DNN training: our discussion in the Appendix A.1 Generalizing our theoretical claims to a broader class of nonlinear loss functions, such as sparse categorical cross-entropy, which is also rarely used in practice, requires further non-trivial investigation and is scope for future research. Our present theoretical analyses in Sections 3.1 and 3.2 consider the split layer at the pre-final layer of an L layer DNN; analysis of the split at an arbitrary tth layer, along with the final loss function used for DNN training, requires much more mathematical rigor.
34


Reviews:
[{"review_id": "0VOx4lOn22", "reviewer": "Reviewer_u4Y9", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work concerns denoising methods in split learning framework.\nIn split learning, the neural network is split between the client (which holds data) and server (which holds labels). The activations of the output layer of the client are passed as the input to the server, which performs backpropagation and returns the gradients to the client.\nNoise is applied to these activations to preserve privacy of the client data and this naturally degrades the performance of the network.\n\nThis work proposes a mechanism to denoise activations in split learning framework that preserves privacy guarantees and improves accuracy.\nIn particular, authors suggest two methods: random masking and scaling, that are applied to the noisy activations at the server side.\n\nAuthors show theoretical results that under the correct choice of the denoising parameters (probability of masking and the scaling value, respectively), the expected loss value is smaller compared to just the noisy activations. This is shown for the MSE and cross entropy losses, under certain assumptions.\n\nFurthermore, practical experiments both on simulated and real data validate the claim that using proposed denoising methods, higher noise scales (and thus stronger privacy guarantees) are possible.", "claims_and_evidence_yn": "No", "claims_explanation": "Paper contains both theoretical and practical results.\n\nCertain theoretical results are formulated without sufficient rigour.\nFor example, Theorem A.10 claims that a mechanism is $(\\varepsilon, \\delta)$-private for some $(\\varepsilon,\\delta)$. Bounds on $\\varepsilon$ and $\\delta$ need to be specified, as, for example any mechanism is $(0, 1)$-private.\n\nIn the DP section, it is written that $\\Delta_g$ is small for large datasets. How is this claim supported?\n\nAt the bottom of page 4, what does it mean that the other loss functions can be used? \n\nThe fact that in the proof of Theorem 3.4 higher moments assumed to be small (Remark A.5) should be mentioned in the theorem statement.\n\nIn Theorem 3.4 it is stated that the $\\sigma^2$ should be large enough, while in Theorem 3.5 there is a particular value of sigma.\nSince there is only a sketch of the proof for Theorem 3.5, there should be more details on where this bound comes from.\n\nPractical experiments support the message of the paper.\nFor Figures 3 and 4, how many runs were used to produce the plots? Could the error bars be added?", "audience_yn": "Yes", "audience_explanation": "The authors focus on an important and well-motivated problem at the intersection of optimization and differential privacy.\nThey approach it from two perspectives, theoretical and practical. The work provides interesting findings, such as the ability of networks that use proposed denoising methods to match the performance of the model before the noise injection.\n\nAs authors point out, it would be great to understand what are the real values of privacy budget needed here.", "requested_changes": "My main request is regarding the presentation of theoretical results, see above.\n\n\n\"Expectation conditioned on A given B\" should be \"expectation over A conditioned on B\"\n\nAlso, consider adding guidance from the theory on what values of $p$ and $\\alpha$ to pick in practice, if possible.\n\nFinally, more discussion regarding the effect of the split layer on the efficiency of the proposed method would benefit the practical part of the paper.", "broader_impact_concerns": "No concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "QfXo32baLr", "reviewer": "Reviewer_Kytz", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper addresses the significant accuracy degradation observed in Split Learning when noise injection is used for privacy. Its main contributions are:\n1. It proposes two server-side post-processing techniques to estimate intermediate variables accurately during the forward pass of SL.\n2. It provides theoretical proofs demonstrating that these techniques can accurately recover the original intermediate signals from their noisy versions.\n3. Experimental results with DNNs show that these denoising methods allow SL models to tolerate high levels of noise, achieving accuracy comparable to noise-free baselines.\n\nStrengths:\n1. The work effectively tackles the \"privacy vs. utility\" problem in SL, offering a practical way to maintain high accuracy without sacrificing the privacy benefits of noise injection.\n2. The proposed methods are supported by strong theoretical analysis, including proofs of estimation accuracy and preservation of differential privacy guarantees.\n3. The proposed operators are relatively simple to implement as server-side post-processing steps.\n\nWeaknesses:\n1. The experiments predominantly position the cut layer very deep in the network (often adjacent to the fully connected layers). This setup fails to represent realistic edge computing scenarios, where the split typically occurs much earlier to minimize client-side computational load.\n2. The evaluation relies solely on standard, \"off-the-shelf\" reconstruction attacks. It lacks an assessment against adaptive attackers, i.e., adversaries who are aware of the specific defense mechanisms (such as masking) and adjust their strategy accordingly. \n3. The paper lacks a principled mechanism or heuristic for automatically selecting optimal hyperparameters (e.g., masking ratio $p$ or scaling factor $k$) in practical settings where ground truth labels are unavailable. This omission poses a significant barrier to real-world deployment.", "claims_and_evidence_yn": "Yes", "claims_explanation": "1. The authors provide mathematical proofs demonstrating that the proposed denoising techniques (scaling and random masking) can accurately estimate the original intermediate variables from their noisy counterparts during the forward pass. \n\n2. The paper includes experiments with DNNs (e.g., on image classification tasks) that clearly show the proposed methods allow Split Learning to tolerate significantly higher noise levels compared to baselines, recovering accuracy almost to the level of a noise-free model.", "audience_yn": "Yes", "audience_explanation": "1. Privacy and Distributed Learning Researchers: Split Learning is a prominent distributed learning paradigm, particularly for resource-constrained edge devices. This paper directly addresses a major pain point in that domain: how to maintain high model accuracy while injecting sufficient noise to guarantee privacy.\n\n2. Security and Adversarial ML Community: The paper discusses reconstruction attacks (e.g., feature-space hijacking) and demonstrates how the proposed denoising techniques (specifically random masking) can serve as a defense mechanism.", "requested_changes": "1. Please provide a practical heuristic, algorithm, or guideline for how a practitioner should select these parameters in a deployment setting where they cannot simply measure test accuracy against a noise-free baseline. If an automatic selection method is not feasible, please add a robust sensitivity analysis demonstrating that there is a \"safe\" range of parameters that consistently works across different datasets (e.g., CIFAR-10 vs. ImageNet) and model architectures.\n\n2. Please explicitly clarify (or prove) that the denoising steps do not inadvertently leak information through the gradients returned to the client. Ensure the text clearly distinguishes between the privacy guarantee of the forward pass (client-to-server) and the integrity of the backward pass (server-to-client).\n\n3. The experiments often place the split layer relatively late in the network (e.g., before fully connected layers). In many practical SL scenarios (like on IoT devices), the split might occur much earlier to save client computation. It's better to conduct and include an experiment where the split layer is placed at different depths (e.g., after the first or second convolutional block) to verify if the denoising techniques remain effective when the intermediate feature maps are larger and semantically different.\n\n4. While the paper compares against simple noise injection, the context regarding other defense mechanisms (e.g., gradient compression or other privacy-preserving SL variations) could be richer. Please include a brief discussion or qualitative comparison with other state-of-the-art defenses in SL to better position the contribution.", "broader_impact_concerns": "No major ethical concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Dk1kSAahud", "reviewer": "Reviewer_j4dx", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this paper, the authors focus on improving the training accuracy of Split Learning over noisy signals, and propose two denoising techniques called scaling and random masking. Although the experimental results show relative effectiveness, this paper lacks of innovations that seems like just combinations of serval existing techniques. Therefore, this paper cannot be accepted due to it does not meet the journal's requirements.", "claims_and_evidence_yn": "No", "claims_explanation": "1.\tAlthough the results demonstrate the accuracy has increased through their proposed method, the lacking of comparison with state-of-the-art works cannot comprehensively support the empirical result. That is, it is confused that whether this work performs best results.\n2.\tAs shown in figure 4, the accuracy of noise injection training decreases obviously. Is it practical for the applications that require high accuracy guarantee?\n3.\tThe scalability of this work is so limited. In the section of problem setup, this work presents several preconditions, which influences the scalability of this work. Otherwise, does the methods proposed in this paper can be adapted in other scenarios instead of split learning. \n4.\tThe absence of threat model makes this paper obscure for the attack and defense strategies.\n5.\tLacking of ablation evaluation makes it fuzzy to locate which parameter is more important for their method.", "audience_yn": "No", "audience_explanation": "1.\tThe core improvement of this paper is to leverage existing scaling and denoising methods to conduct splitting learning, and the test the corresponding performance on different models and datasets. Applying these techniques to splitting training can be seen as a natural and expected extension, rather than a conceptual breakthrough.\n2.\tSome concepts of terms are introduced without explanation make it difficult to reader to understand the authors’ intent.", "requested_changes": "None", "broader_impact_concerns": "None", "additional_comments": "1.\tSome typos need to be carefully checked in this paper.\n2.\tSome conclusive sentences should be appended at the end of each subsection in the related work, instead of simply listing these works.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)