Train to Defend: First Defense Against Cryptanalytic Neural Network Parameter Extraction Attacks
Authors: Ashley Kurian
Aydin Aysu
Venue: NeurIPS 2025 poster ()
Keywords: Neural network
cryptanalytic attack
parameter extraction
neuron similarity
training
Primary area: social_and_economic_aspects_of_machine_learning
TL;DR: First defense against cryptanalytic parameter extraction using extraction-aware training with zero overhead inference and negligible accuracy impact.

Abstract: Neural networks are valuable intellectual property due to the significant computational cost, expert labor, and proprietary data involved in their development. Consequently, protecting their parameters is critical not only for maintaining a competitive advantage but also for enhancing the model's security and privacy. Prior works have demonstrated the growing capability of cryptanalytic attacks to scale to deeper models. In this paper, we present the first defense mechanism against cryptanalytic parameter extraction attacks. Our key insight is to eliminate the neuron uniqueness necessary for these attacks to succeed. We achieve this by a novel, extraction-aware training method. Specifically, we augment the standard loss function with an additional regularization term that minimizes the distance between neuron weights within a layer. Therefore, the proposed defense has zero area-delay overhead during inference. We evaluate the effectiveness of our approach in mitigating extraction attacks while analyzing the model accuracy across different architectures and datasets. When re-trained with the same model architecture, the results show that our defense incurs a marginal accuracy change of less than 1\% with the modified loss function. Moreover, we present a theoretical framework to quantify the success probability of the attack. When tested comprehensively with prior attack settings, our defense demonstrated empirical success for sustained periods of extraction, whereas unprotected networks are extracted between 14 minutes to 4 hours.

Train to Defend: First Defense Against Cryptanalytic Neural Network Parameter Extraction Attacks
Ashley Kurian
Aydin Aysu Department of ECE
Department of ECE North Carolina State University
North Carolina State University aaysuêncsu, edu
Skurian@ncsu .ediussuuuuuuuuuuuu
Abstract
Neural networks are valuable intellectual property due the significant compu- tational cost, expert labor, and proprietary data involved in their development. Consequently, protecting their parameters is critical not only for maintaining competitive advantage but also for enhancing the model's security and privacy. Prior works have demonstrated the growing capability of cryptanalytic attacks to scale to deeper models. In this paper, we present the first defense mechanism against cryptanalytic parameter extraction attacks. Our key insight the neuron uniqueness necessary for these attacks to succeed. We achieve this by a novel, extraction awarr training method. Specifically, we augment the standard loss function with an additional regularization term that minimizes the distance between neuron weights within layer. Therefore, the proposed defense has zero area- delay overhead during inference. We evaluate the effectiveness of our approach in mitigating extraction attacks while analyzing the model accuracy across different architectures and datasets. When re-trained with the same model architecture, the results show that our defense incurs a marginal accuracy change of less than 1% with the modified loss function. Moreover, we present theoretical framework to quantify the success probability of the attack. When tested comprehensively with prior attack settings, our defense demonstrated empirical success for sustained periods of extraction, whereas unprotected networks are extracted between 14 minutes 4 hours.
eliminate
Introduction
Neural networks hold significant value as their training process is computationally intensive, requires skilled labor, involves proprietary intellectual property, and demands extensive effort in dataset collection. Today, neural networks are ubiquitous, powering a wide range of applications due their superior performance in various machine-learning tasks. The rise of machine learning service (MLaaS) made common users to query models by providing input and receiving output, while model providers seek to obfuscate model parameters to protect their intellectual property. However, the increasing adoption of neural networks has also led a rise in model-stealing attacks [25] [7]. which aim to create high-fidelity [1] replicas of the original models. These attacks target various components the neural network, including hyperparameters
26 (such as layer types and configurations) and learned parameters [4] 9 (trained weights and biases). The significant cost and effort savings compared to training a model from scratch make model theft an appealing strategy for adversaries. Moreover, model extraction eases other attacks, such membership inference 22 and input poisoning [5], further exacerbating security concerns.
Various attacks have demonstrated the feasibility of extracting neural network parameters through cryptanalytic techniques 4 ج. These attacks treat parameter recovery as a structured math- ematical problem, enabling the extraction of weights using polynomial number of queries and within polynomial time with respect to model size on deep models [8]. Despite their effectiveness, cryptanalytic extraction attacks remain major threat that has not been addressed to date, with their capabilities growing rapidly.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

In this paper, we conduct the first in-depth analysis of the cryptanalytic extraction attack and identify key parameter conditions required for its success. Specifically, we observe that extracting weight magnitudes depends on the uniqueness of neuron weights within a layer- the less similar the weights, the more effective the attack. To mitigate this, we propose a training strategy that enforces weight similarity among intra-layer neurons, inherently enhancing the model's s resistance to such attacks. This is achieved by incorporating a weight similarity constraint into the training loss function alongside the standard loss, which minimizes prediction error. However, enforcing similarity among intra-layer neuron weights can lead to a change in model accuracy. To address this, we explore various optimization strategies to ensure that the accuracy change remains minimal while maintaining a strong defense against parameter extraction. Furthermore, we evaluate our countermeasure across different neural network configurations used in prior end-to-end attack [8]. including various datasets, architectures, training, and extraction seeds.
The main contributions of this paper are listed below with the corresponding section references.
We conduct a comprehensive analysis of cryptanalytic attacks, examining their underlying mechanisms and identifying the key parameter configurations that enhance their efficiency and likelihood of success. Understanding these crucial dependencies provides valuable insights into the fundamental weaknesses that adversaries exploit, laying the foundation for developing effective countermeasures against these attacks. (Section 2.1 and 33.1
In order counteract cryptanalytic attacks, we propose an extraction-awarr training strat- egy. This approach involves introducing a regularization term into the loss function that intentionally adjusts the model's parameters during training. The goal is encourage parameter configurations that minimize the effectiveness of cryptanalytic attacks, making it significantly difficult for adversaries to extract sensitive model information successfully. (Section 3.3
A key advantage of our proposed defense is being zero-overhead at inference. The process is built into the training. and we show that re-training the same model hyperparameters with our technique can be sufficient to mitigate the attacks and has no run-time oSerheads. (.6)
We analyze and implement various optimizations within our defense strategy to minimize the accuracy change caused by the inclusion of the defense mechanism while quantifying that the model remains robust and secure against cryptanalytic attacks. Our goal is to strike a balance between maintaining high model performance and enhancing resilience, thereby achieving an effective defense without compromising the model's 's accuracy. (Section
We evaluate our defense strategy across various neural network configurations that were attacked in prior works, including different datasets, model depths, numbers of neurons per layer, and training seeds. This evaluation allows us to assess the robustness and effectiveness of our defense under diverse conditions and configurations, ensuring its generalizability and reliability across a range of practical scenarios demonstrated in prior works. (
A major issue with prior attack evaluations was relying empirically on a set threshold (36 hours of attack runtime) for determining the attack success. To address this limitation, we propose a theoretical framework to assess attack success probability as a function of intra-layer neuron parameter similarity. (Section 3.2)
Our research demonstrates that extraction-aware training can effectively thwart cryptanalytic model parameter recovery attacks while preserving model accuracy with only a marginal change. By employing this lightweight defense strategy, we show that neural networks can be robustly protected from model stealing threats, offering practical solution to safeguard sensitive model parameters without compromises in performance, memory footprint, or latency.
2 Background
2.1 Related work and our solution
Parameter extraction attacks on deep neural networks have been extensively studied over the past
decades 7 25 20 [4]. These attacks seek to recover the weights and biases with sufficient numerical precision to achieve functional equivalence [II. meaning the extracted model should
produce identical predictions to the original model for any input. Cryptanalytical parameter extraction
attack exploits the fact that ReLU neural networks are piecewise linear functions, and thus queries at
2

---

the points where the imput to the activation is zero reveal information about the model parameters. The seminal cryptanalytic extraction attack shows the recovery of normalized values of weights and biases (newron signature) with polynomial number of queries with respect to model size [4]. However, their method required an additional exhaustive search to recover the signs of the weights, thereby introducing exponential time complexity to the attack. As a result, their evaluation was limited to shallow neural networks with a maximum of three layers. Martinez et al. [2] addressed the sign extraction limitation by introducing techniques to recover neuron signs, enabling fully polynomial-time attack. They demonstrated sign extraction on deeper networks with up to eight layers. Foerster et al. [8] combined signature extraction and sign recovery to perform the first end-to- end parameter extraction attack. They introduced further optimizations to the sign extraction step, improving efficiency by 14.8x. These attacks assume access to confidence scores rather than just class labels for parameter extraction. More recently, cryptanalytic approaches have emerged that bypass this requirement and perform extraction using only the predicted labels 6].
The advancements in neural network parameter extraction have raised significant concerns about protecting proprietary model values. However, no countermeasures have been developed to defend against these attacks. We examine the key assumptions and parameter configurations that contribute to the success of these attacks. Through this analysis, we observe that fundamental assumption in signature extraction is that the intra-layer neuron parameters must be unique [4]. Moreover, hard neurons, i.e., neurons whose signs are recovered with low
certain neurons are classified confidence [). This occurs when it is impossible to craft an input perturbation that isolates the activity of the target neuron due to its similarity with other neurons. For such hard neurons in the previous layer, prior work propose recovering the next layer's 's signature by brute-forcing their signs. Their method performs parallel signature recovery across sign combinations, selecting the one that yields correct signatures for the next layer. The approach remains feasible for up to ten low-confidence neurons, beyond which it becomes computationally prohibitive g
Our goal in this paper is to leverage these insights and incorporate conditions into the training phase to make neurons harder. This is achieved by increasing intra-layer neuron parameter similarity, thereby making the model inherently resilient to cryptanalytic attacks. Specifically, we introduce a regularization term in the loss function that minimizes the distance between neuron parameters, while simultancously optimizing for the primary learning task. While this approach strengthens model security, it impacts accuracy. To mitigate this trade-off, we explore several optimizations, including applying the defense only to the first layer, since protecting it can help safeguard the rest of the model in layer-wise attacks. We also evaluate strategies such as adjusting the strength of the regularization term to balance model accuracy and security. Finally, we assess the effectiveness and generalizability of our defense across different model architectures, datasets, training, and extraction seeds.
2.2 Threat model
We adopt the standard assumptions commonly used in neural network cryptanalytic parameter extraction attacks 4 2 21 6). The target model is a fully connected neural network that uses ReLU activations in the hidden layers and a lincar activation in the output layer. The attacker has full knowledge of the target model's hyperparameters. The attack operates with high precision, capable of recovering parameters up to 32-bit floating point precision. The attacker is also granted the ability to input arbitrary data and observe the corresponding class labels or logits scores, effectively treating the model as a black-box system with the objective of reconstructing a functionally equivalent replica of the original network. The model can support any number of output classes. Side-channel attacks 10 9 are considered out of scope, as they rely on auxiliary information such as power consumption or electromagnetic emissions, which may not be accessible in all application scenarios.
The overall attack setup is illustrated in Figure ! which depicts the training and deployment phases of the model. As shown, the neural network is trained using proprictary dataset and specific hyperparameters, including the model architecture- for example, three input nodes, two hidden layers with two nodes each, and four output classes <3.2.2,4>, with the mean squared error (MSE) loss function and the Adam optimizer. We follow the assumption that training is trusted. Once trained, the model, with its secret parameters, is deployed for inference. An attacker can then query the model to obtain class labels or logit scores to perform a cryptanalytic attack. The defender's 's objective is to secure the trained model parameters against such attacks.
The goal of the model provider is to build a robust neural network that remains resilient against cryptanalytic parameter extraction attacks at inference. From the defender's perspective, we assume
3

---

Training phase
Output layer
Inference phase
Training dataset
Input
Hidden layers
Edge/Cloud
Input
layer
Input
Deploy
Label
<3,2,2,4>
mse, adam
Trained model with secret parameters
Output (label/ logits)
Goal: extract
Model hyperparameters
parameters
(architecture, loss function etc)
Figure 1: Schematic depicting how a deployed neural network becomes vulnerable to attack. neural network is trained a proprietary dataset and later deployed for inference. During the inference phase, an attacker queries the model to extract labels or logit scores for a cryptanalytic attack, while the defender aims protect the model's secret parameters.
full control over the training process, including the ability to modify the loss function. We assume that training trusted and not subject model extraction attacks. Once deployed, the adversary can query the model and receive responses to execute cryptanalytic attacks during inference.
2.3 Cryptanalytic parameter extraction-A walkthrough
This section walks through the signature extraction technique 4 using simple neural network for illustration. The attack recovers the weights of individual neurons by carefully probing how the output of the network changes in response to small input perturbations. The process begins by identifying critical points input values where a neuron's 's ReLU output is zero following the terminology used Gradients
then computed around these points capture directional information about the weights. By comparing the network's 's behavior when a neuron is active versus inactive, the attack isolates the influence of single neuron and recovers its normalized parameters.
in prior works 4 2 g.
Consider a feedforward neural network with architecture (2. 2, 1), as shown in Figure 2aa), where neuron A (л)) has weights (a1, a2), neuron B (1)) has weights (bı, (b) and the input vector X= (x1, 222). k-deep neural network [z)),, parameterized by maps input space X to output space With ReLU activation a, the output is:
So (x) cia + a222) (21 b242)
Figure (bb) shows the input space and corresponding critical hyperplanes of 17 and 7/n, which represent the plane containing points where at least one of the ReLU inputs becomes zero. The critical points lie on the critical hyperplane. To recover the first-layer parameters, the attack traces the gradient flow from input coordinate through the target neuron to the output. At each critical point x², the attack computes the directional derivatives (z) by applying small positive and negative perturbations along the input axis eq.
=ee)
Of(z- - ees)
de;
de:
where e; Rdo=2, do the number of input neurons. The signature can then be recovered by computing (at + a, [4] Assume we want to recover parameters of 1) using a critical point x=.
Suppose x² + ge; lies region where both 7) and π/ are active (represented using (+,+)), and x² ce; lies in a region where 17 is inactive and n/ is active (represented using (+,+)). In this case:
fo(o" + (1 501) + 2222) cz(bi (æ1 + EC2) by222)
fo(x sei) ci 0 ca(bi (x) - 8E1) b222)
The gradients are: af clais + oz and aj = -cbbi Their sum oj + aj Clafe, isolates
the first weight coordinate a of 7/- Similarly, for direction 2, the sum or) + 022 Ci age reveals the second weight coordinate 02. The normalized signature becomes:
02 at o7 of - a1 a1
at + of + +
This recovers the normalized weights of 7)- Notably, terms from other neurons cancel out (in this that of 18),, enabling isolated recovery of the target neuron's parameters. Repeating the process

---

X2
X2
a1
X1
C1
Fx22
Critical
$ Xx2 (+, +)
a2
(+, +)
hyperplane of Πл
X1
X1
X11
X12
X11 (---)
X12
b1
C2
(--))
Critical
X2
b2 (a)
B
X21
hyperplane of ns
+ X
(b)
(c)
Figure 2: (a) neural network schematic with <2,2, 1> configuration (b) input space showing the critical hyperplanes of и7 and T/ with distinct parameters (c) overlapping critical hyperplanes of 7) and T)B when their parameters are identical.
with a critical point for a/ allows recovery of its signature as well. See 4 ( for further details on signature and sign recovery. Attacks that rely predicted labels adopt similar strategy but begin by identifying critical hyperplanes through the detection of transition points- imput values where the model's predicted class changes, thereby revealing the boundaries between decision regions (3)
3 Proposed defense against eryptanalytic parameter extraction
We propose the first countermeasure against cryptanalytic parameter extraction attacks. These attacks can target deep models trained on real-world datasets, requiring only black-box access in polynomial time and with a polynomial number of queries. Our defense addresses the main assumption underlying the success of these cryptanalytic attacks. The signature and sign extraction process is based on the distinctness of the neuron parameters, with the success rate of the attack increasing as the uniqueness between the neuron parameters increases. Our defense challenges this assumption by promoting similarity among neuron parameters during the training phase. This is achieved by adding regularization term to the loss function that reduces the distance between neurons while still optimizing for the original loss function. The following subsections outline the methodology in detail: we first analyze why the attack fails when neuron dissimilarity is reduced, then introduce theoretical framework to quantify attack success probability, describe the extraction-awart training approach, and present optimization strategies that balance accuracy and robustness.
3.1 Why cryptanalytic attack fails by making neurons similar?
In this section, we provide a mathematical explanation of how making neuron parameters similar disrupts the cryptanalytic attack procedure, using the same example from Section 2.3 Consider the neural network shown in Figure ((a). the parameters of neurons 77a and Π) are the same that is, (a1, a2) (bi, (2) their critical hyperplanes overlap. As result, both neurons will be in the same state (either active or inactive) for any given input, as shown in Fig Suppose we are trying to extract the signature of 1/4 and let x² be a critical point. Assume that the input xx + se, activates both neurons and the input xx - Ee, deactivates both. Then, for direction
fo(x te,) (a se1) + a222) c2(bi (x1 + Eei) b222))
(6)
fo(x* =01) c1 0 0
The resulting gradients are: aj C101E + cabie and oj 0. The sum becomes: of + aj 01818 + c2b16. This result includes the influence of both 744 and 7/s, making impossible to isolate the contribution of n4 for any given input perturbation. Therefore, the attack fails to recover the correct signature. Similarly, for direction 2: aj + = c1a25 + czb2s. The normalized signature is:
a + a
+ +0
= (1, C102E + cobus
(8)
+ aj
aj
+ 0
cjajE + cabis
This signature is incorrect because it mixes contributions from both neurons. The correct signature should be (1,a2/a1). which reflects the behavior of only the target neuron (as derived in Section 2.3). Making neuron parameters similar causes them to activate or deactivate together for any input, preventing the attack from isolating a specific neuron and leading its extraction.
3.2 Probability that a random input fails defense
key limitation prior attack evaluations their reliance on an arbitrary empirical threshold (e.g.. 36 hours of extraction) to define success. To address this, we introduce a theoretical framework that calculates the attack success probability as a function of the intra-layer neuron weight similarity. The key insight is that an attack fails when neurons are in the same state (either active inactive) for any
5

---

input, as it is not possible to isolate target neuron activity. However, if the parameters of the neurons differ, their critical hyperplanes diverge, and the attack succeeds. We quantify the probability of this success by calculating the area between the hyperplanes, where the neurons" states differ. This gives the probability of successfully extracting single neuron pair. The total attack success probability is then obtained by identifying the pair of neurons with the greatest dissimilarity, representing the worst-case scenario.
The probability of extracting parameters of a neuron pair is the ratio between the measure of the input region lying between their corresponding hyperplanes and the total measure of the entire input spuce. This probubility is is upper bounded by K
Theorem 1.
2 =iztai
[#12-211)'
where K; -9: -(ar+ó,))
ax aN+oN 1,, N 1; N denotes the input dimension, at
refers to the i-th parameter of the neuron, 6, is the i-th parameter difference between the neuron pair, and 211 and 122 are respectively the lower and upper limits of an input dimension.
Proof. Consider the neural network in Figure aa. If the parameters of neurons 7) and Tß are identical, their critical hyperplanes overlap, as shown in Figure In that case, both neurons always behave the same, and the defense works perfectly. However, when their parameters differ by (ő1, memming bi ai + di and b2 02 + thei critical hyperplanes do not align. The equations for their critical hyperplanes are:
For V) 01 + 0232 =
(10) The shaded region in Figure (bb) shows the area between these two hyperplanes, where the neurons behave differently. Inputs that fall in this area cause the attack to succeed. Let I11 and x12 be the lower and upper limits for the input 21 and #21 and 2/2 for x2. For INT8 inputs, these range from -128 to +127. For simplicity, we use T1 and 112 as the common lower and upper bounds for all input dimensions in the derivation, as each dimension shares the same input range.
For 7B by + b2 = = (0) + +1) + (a2 + 62)
(
(a1 + 81) + 02)
Area between hyperplanes of A and 7B
11
11
dr (11)
02
(az
211
K x1 dr
|
(xi2 + +f1) where, K =
01
+
01 + 9
(12)
2
02
02 + 82
222
Total area of the input space
dx2 dx = (x12 - I11) (112 Z11)
(13)
Attack success probability Area between hyperplanes
K
2 + x
(14)
Total area
2 (x x11)²
This is the probability that a randomly chosen input will fall between the hyperplanes, meaning the neurons respond differently, and the attack succeeds. As the differences ōn and 8n increase, the two hyperplanes move further apart, increasing this probability. For an N-dimensional input space, due to the triangle inequality, the probability is upper bounded and can be computed as:
K1
K22 +... + KKN11
#12 + #f
Probability of attack success, P
(15)
( 111)²
(at -0; ((a (-(a: + 6.)) UN aN + ōn
where K₁ =
for i = 1,2,..,, -
This derivation assumes that the normal vectors of the two neurons' hyperplanes point in the same direction. In this case, the area between the hyperplanes (the shaded region in Figure (()) represents where the neuron states differ, and thus where the attack succeeds. However, if the normal vectors point in opposite directions, then the region where the neurons differ in state flips it becomes the region where they previously had the same state. In other words, the failure region becomes the unshaded area in Figure[[b) So. in this case, the probability of attack success becomes (1 P). The direction of the normal vectors is captured by the angle 0 between them:
ā 5
0 = cos
where ā = =a11 a2 .2.....ay). 5 (b1. be..... bN)
(16)
|| ||ا||||ا||

---

Learned weight distributions: baseline vs secure model
Attack time vs Probability
Rawwinn (vm-d.004), m-n-4200)
Sacure (pm0,0052, c.0.0004)
Attack time (hours)
daopg Saiie 2
-15
-1.
-0.5
Weight value
1.0 1.5
101 Attack success probability (P)
[a]
(b)
(c)
Figure 3: (a) neural network schematic with <3,2,2,>> configuration (b) comparison of weight distributions in the first dense layer of the secure and baseline models. The secure model shows more concentrated distribution centered around zero with lower variance (0.08), while the baseline model exhibits a wider spread (variance 0.43). Only the range [-1.5, +1.5] shown, which contains approximately 90% of the weights. (c) Relationship between attack success probability and extraction time (in log). As the probability of attack success decreases, the model extraction time increases from 3.5 hours at P = 0.74 to 4.95 hours for P = 0.005, which asymptotically reaches infinity for P 0. Based on the fitted eurve, the projected attack time P 0.0017 24,095 0995 hours.
PP
if < 0 S S
(same direction)
Probability of attack success
(17)
1 1
if
(opposite direction)
3.3 Extraction-aware neural network training
Adding term to the loss function to increase similarity between neuron parameters help defend against cryptanalytic attacks. We introduce a modified loss function:
Loss function = -originalloss + Assmitoriiyy total notal_similarityyloos
(18)
Here, original_ lloss refers to the standard function (such as MSF) that reduces the error between predicted and true labels. We introduce the additional term (Asimilarity halal_similarity to improve the security the model. The factor Asimilarity controls impact of the similarity term, allowing balance between accuracy and security- values increase security but may reduce accuracy. The citiallsmiilartty_oee captures the overall similarity between neurons across all layers in the network:
total_similarityy = layer=l
yummber of layers waweraawiseeemmaiiieee
(19)
fayer_wise, wttr__uss_siiliarittete litocssmmlarttttyoo sesriityyllsssss =
(mumber of neurons- 1p P++1)2
(20)
where pt is the parameter of neuron i. The layer_wise wise similarity loss computes the pairwise difference between parameters of neurons within the same layer. This term uses L2 regularization to penalize large differences, making neurons more similar.
To illustrate, consider a neural network with architecture <3, 2, 2, 1> as shown in Figure B(a). The total_ siall_simllarty Jlss the sum of the similarity losses for hidden layers and 2:
total similarity loss layer1_loss loss + Laaer221aa loss
layerl_loss (al b (a2 b2)² (ag ba)2 and layer2_loss (ci-d)) (c2 d2)
In this example, the loss in each layer is computed based the difference between the parameters of two neurons within that layer. For a layer with n neurons, there will be a(n-1) pairwise comparisons. The goal is to make neurons behave similarly so that their outputs are the same for any input. This
similarity prevents the attacker from isolating and targeting a specific neuron, as multiple neurons will respond in the same way.
3.4 Tuning our defense
The additional term Asimitarity * tolal_ poaod_snnllrrtty_eee Joss introduced the loss function to enhance model security. However, this regularization must be carefully controlled to avoid compromising model accuracy. To mitigate accuracy degradation, we propose the following tuning strategics:
Adjusting the regularization strength: By varying the impact factor Asimilarity. it is possible to control influence accuracy while still making the model robust against attacks.
7

---

Restricting defense to the first layer: Since the attack proceeds layer by layer, securing only the first layer can be sufficient. If the parameters of the first layer are protected, an attack on the subsequent layers could be automatically prevented.
Defending a subset of neuron pairs: Instead of enforeing similarity across all neurons in the first layer, the defense can target only a percentage of the neuron pairs, For example, layer with eight neurons results in 28 pairwise similarity terms (i.e., b(*-1)), Defending 50% of the neurons involves enforcing similarity on 14 randomly selected pairs.
These optimization strategies help balance security and accuracy, ensuring that the defense mechanism does not adversely impact the model's s performance.
4 Results
We evaluate our defense by measuring the model accuracy and attack run run-time for parameter extrac- tion. The models are trained using the modified loss function discussed in Section 3.3 to defend cryptanalytic parameter recovery attacks. To limit accuracy change, we apply optimizations such as restricting changes to the first layer parameters and adjusting Asimilarity- Table 1 shows results across different architectures, datasets, and training seeds. We follow the notation used in prior model refers to an MNIST-trained network with an input layer of size 784, two hidden layers, each of size 8, and an output layer of size 1. We compare secure models to their baseline by reporting accuracy, attack runtime, and Asimslarity that makes them secure. Experimental setting: All training and extraction were done on hegh-performance cluster with Intel's Xeon processors comprising 400 compute nodes and over 14,000 cores. On average, each node is provisioned with 128 GB of memory. We take our models from the papers that demonstrate attacks R and we use exactly all the hyperparameters evaluated in those papers. Models are trained on random data and MNIST, using two different training seeds (s1 = 42, s2 = 10). The extraction times for signatures and signs, number of queries, are reported mean and variance across four extraction seeds (0, 10, 20, and 30). We consider a model to be secure if the attack fails after extended periods of extraction For example, the model MNIST784 16144___________ (ss2),,, which was previously compromised in 3.5 hours, becomes secure with a Asimilarity of 10-9, while its accuracy drops by 0.92%. In some cases, the secure model even performs better. For instance, the MNIST784 ___x__x3__________ [sll model shows a 0.43% increase in accuracy compared to its baseline version. The results show that for all the previously studied attack settings, our defense can secure the models through extraction-aw -wwre training with an accuracy change of less than 1 % with zero-overhead inference.
work [8] For example, MNIST784_8x2
Using the theoretical framework described in Section 3.2 we first compute the probability of attack success for models protected using our defense strategy, and then we empirically estimate the corresponding extraction time based on this probability. For example, the attack success probability for the MNIST784_ 16____x________ (s22) model is calculated based on the neuron pair in the first layer with the greatest parameter dissimilarity. The attack success probability drops from 0.74 for the baseline to 0.0017 for the secure version, a 435x decrease. Figure 3e) shows that the attack time increases with the decrease in attack success probability from 3.5 hours for the baseline model extended extraction periods for the secure model. We perform curve fitting that asymptotically approaches infinite extraction time as the probability of attack success approaches zero. Based on this fit, the projected attack time for the secure model at P 0.0017 is estimated to be 24.095 hours. FigureBbb shows the weight distributions for the secure and baseline models. The baseline model's weights are spread across a wider range of [-2.5, +2.5], while the secure model's weights fall within [-0.2, +0.2] (Figure (b) shows only a range of [-1.5, +1.5], where 90% of the weights fall). To quantify this spread, the secure model has a lower weight variance of 0.08 compared to 0.43 in the baseline model, with weights more tightly clustered around zero. This indicates that the parameters in the secure model are more similar to each other, reducing the attack's 's ability to isolate neurons and extract their parameters individually.
Theoretical analysis of defense:
Discussion
5.1
Where does the attack struggle for secure models?
The cryptanalytic parameter extraction on models secured by our defense is unsuccessful even after running the attack longer than the set threshold in prior papers [8) The attack works by gathering critical points for each neuron to compute its signature. It then applies graph clustering to group
Prior work g uses 36 hours, whereas we 48 hours as the extraction time limit.
8

---

Table I: Cryptanalytie model parameter extraction performance
Model Informaation
Signature [s]
Sign [s]
Querics
Secure Model Information
Model (Training Seed)
Accuracy
Params
Mecn
Var
Mican
Var
Mean
Var
Accuracy change (%) Asimdarity
Random/244-822211 643
NA
6250
9.10 102
9.5- 10a
$4.60
35.6
6.55 110
2.03- 10
NA
10-5
Random/86- 18x2-1 (s1)
NA
12560
1.37
100
T.18 -0'
$6.67
46.99
1.51-107 -10"
2.06- 10"
NA
10
Rundom084-32224.
25130
2.54 102
'02
1-64-100
102
2.62-10
10"
10-4
Kandom734
50240
5.43-
2.78-
5.24-1 110
-104
10-1
Rundom78441228.22
21134-1-1223--000 (x])
NA
180480
10
7-62-
10"
1.04 110
6.57-101
Random786-128// (x2)
NA
100480
1.38 104
-0
5.54 102
8.12
1:2- 107
3.2-
NA
MNIST784 8x2 (62)
T0.34
6250
7,87- 102
5.74
-02
48.54
26.52
6.35 -10
7.24 102
-0.32
10-9
MNIST784- 16x2-1 (x2)
74.40
12560)
1.5: 10P
3.303
102
95.71
0..20
1.36 109
2:27- 10F
+40.74
10-9
MNIST724-R2x2 (x2)
96.37
25120
2.68 10a
5:68 10
1.91 107
2.81 100
2/42-10*
2.8- 10
+0.55
10
MNNST7844 64x2. (x2)
91.41
51240
6.19 107
1.56
207
1.54 162
4 100
1.45 107
4.06 10M
-1.37
10-6
MNIST784 64x2
91.48
50240
5.42
102
5.28 30.
3.97- 102
2.54
4.44 10F
8.03 10F
-0.31
10-6
MNIST784- 10157774-1668-111
89.67
12560
1,12
304
80.37
0,41
0.6 110"
10"
-0.92
10
MNIST784 I613-1 (61)
$4.93
102
1.04 102
1.0-
1.15- 10
10-9
NA: Accuracy not reported for random models as parameters are trained on random data
critical points based on signature similarity. For the attack to succeed, each neuron should have a unique signature otherwise, it cannot specifically target individual neurons, as discussed in Section 3.1 The attack continues clustering until the number of clusters matches the number of neurons in the target layer. However, in secure models, neuron similarity leads to fewer clusters than expected, as some neurons share the same signature. The attack then collects more critical points by exploring multiple directions and repeating clustering. This process fails, which results in attack failure even after extended runtime. With our defense, the attack is still polynomial in time and number of queries. Our defense does not improve the theoretical security, but in practice, it increases the search complexity of the critical points per neuron.
5.2 Model extraction with partial parameter recovery
A question is whether an attacker can still recover the model by extracting only a few neurons in the first layer, or by randomly initializing the first layer and continuing the attack on subsequent layers. The answer is no. The attack on the following layers depends on having the correct output from the first layer, as solves system of equations based on those outputs. If the first layer is incorrect, the computed outputs are also wrong, leading to incorrect parameter recovery in the second layer. Worse, the errors accumulate with each deeper layer, causing the extraction fail. Therefore, protecting only the first layer is enough to secure the entire model for the attacks we target. Moreover, our proposed defense is not fundamentally limited to the first layer. It may be extended to future layers for other possible attacks that are not seeking exact high-fidelity capture with cryptanalysis.
Beyond partial recovery, an attacker might attempt adaptive strategies such collapsing identical neurons, splitting coefficients, or perturbing duplicate weights. Although these techniques can preserve functionality, they disrupt the internal representations required for recovering deeper layers. Collapsing or splitting neurons reduces the rank of the activation matrix, making the next-layer system underdetermined and breaks parameter invertibility. Small perturbations nearly identical neurons similarly introduce activation mismatches that propagate through layers, compounding numerical errors and reducing extraction fidelity. Even iterative refinement may not fully recover the true parameters without exact intermediate activations, as the activation matrix loses rank and stability. Hence, under straightforward adaptive or noisy extraction attempts, the defense remains effective.
5.3 Extending the defense to incremental fine-tuning
In the current implementation, the defense applied during full model training, as the similarity regularizer operates on weight vectors across layers. However, it may be extended to support incremental fine-tuning by re-compiling a pre-trained model with the custom loss function and selectively updating relevant layers while freezing others. While this approach may result in lower accuracy or slower convergence, it offers the advantage of reduced training time.
5.4 Choosing efficient Xsimilarity
We initialize Asimilarity to small value (10-19) and incrementally increase it by factor of 10, with extraction time measured at each step. This process continues until we have extended periods of extraction. The upper limit of Xaimiiarstty depends on the acceptable accuracy loss for given application. From the models evaluated, we select the minimum Aeimsttrsity that ensures security while minimizing accuracy changes. Our experiments show that accuracy changes remain below 1 1%
2 Zero weights are excluded from this condition because neuron output remains constant regardless of the input.

---

While this approach uses sequential search, more efficient heuristics can replace exhaustive grid exploration. Two practical alternatives are outlined below.
Binary search heuristic: This approach begins with a broad range for Asimilarity (e.g.- [10-16, 10, 1r.000'00))0) and iteratively narrowing it using binary search. Selection is guided by a simple objective- such as performance
validation set (accuracy, loss convergence) and a proxy for robustness (attack failure or increased extraction time). It is simple, broadly applicable, and reduces search complexity from (n) to Ollog n) while often converging quickly to a À that balances accuracy and robustness. However, it still requires several training runs though fewer than grid search and necessitates defining clear stopping criterion, such as an acceptable accuracy drop or extraction time threshold
Gradient-norm--aae estimation: This method selects Asimilarity based the relative gradient magnitudes of the primary loss Lorig and the similarity regularizer Lsim- After training baseline model for few epochs, Asimilarity is computed Asimilarity X Va CaCrria 12
a Carmm ! where represents the desired influence ratio of the regularizer. It provides interpretable, data-adaptive estimate that reflects the actual gradient dynamies and requires only one additional forward howard-backwart backward pass on a held-out batch. However, it still involves tuning a, though it is less sensitive than Asimilarityy itself, and requires computing the gradient of the regularizer.
In summary, binary search offers practical choice when fast convergence and low tuning overhead are prioritized, whereas the gradient-norn approach provides a more interpretable and adaptive alternative; both methods can effectively identify suitable Asimslority values without relying on exhaustive grid search.
5.5 Limitations
Our effort to secure models against cryptanalytic parameter extraction attacks comes at the expense of a marginal change (less than 1%) in accuracy. Additionally, as cryptanalytic attacks have not yet been demonstrated on CNNs or LLMs, our analysis focuses on the MLP network architectures evaluated in prior works. At present, these attacks apply only to MLPs with piecewise linear activations. These are limitations of the current attack (which is rapidly evolving), not of the defense. The defense itself has no such limitations and can scale as attacks advance, since it disrupts the key assumption underlying the attacknnamll, namely, neuron dissimilarity.
5.6 Other possible defenses
Other countermeasures such as parameter encapsulation. neural structure obfuscation, and injection of shorteuts extra layers [27 can be employed to protect models. However, these approaches introduce computational overhead and increase the size of the ML library. Additional defenses against query-based model extraction reduce the effectiveness of such attacks through various strategies, including adding deceptive neurons [24], perturbing predictions by maximizing angular deviation between gradients to poison the attacker's 's training objective [8], or reprogramming the adversary with arbitrary behavior [15,, but incur run-time or memory overhead. By contrast, our defense provides protection with zero overhead during inference.
6 Conclusion
Neural networks are valuable intellectual property that are vulnerable to parameter extraction attacks capable of recovering model parameters using cryptanalytic techniques in polynomial time and with a polynomial number of queries. In this paper, we present the first defense against such attacks. Our approach introduces an extraction-aware training method that adds a regularization term the standard loss function. This term increases similarity between neuron parameters, making it difficult for the attack isolate and recover individual neurons. The defense adds zero overhead during inference. We evaluate it across different network architectures, datasets, and training/extractio seeds. Results show a marginal change in accuracy of less than 1,, while protected models resist extraction even after extended periods compared to under four hours for unprotected models. We also provide theoretical framework to analyze the probability of defense failure, offering further insight into the effectiveness of our method.
10

---

7 Acknowledgements
We thank Olivier Benoit, Elke De Mulder, Aria Shahverdi, Daniel Moghimi. Sanjeev Das, Mike Tunstall, and, Majid Sabhagh from Google e for their feedback and helpful discussions. This project is supported in part by NSF Award No. 1943245.
References
[1] Lejla Batina, Shivam Bhasin, Dirmanto Jap, and Stjepan Picek. CSI NN: Reverse Engineering of Neural Network Architectures Through Electromagnetic Side Channel. In 28th USENIX Security Symposium (USENIX Security 19), pages 515-532, 2019.
[2] Isaac A Cantles-Martinez, Jorge Chávez-Saab, Anna Hambitzer, Francisco Rodríguez- Henriquez, Nitin Satpute, and Adi Shamir. Polynomial Time Cryptanalytic Extraction of Neural Network Models, In Annual International Conference on the Theory and Applications of Cryptographic Techniques, pages 3-33. Springer. 2024.
[3] Nicholas Carlini, Jorge Chávez-Saab, Anna Hambitzer, Francisco Modriguez-Henriqueee and Adi Shamir. Polynomial Time Cryptanalytic Extraction of Deep Neural Networks in the Hard- Setting, Cryptology ePrint Archive, Paper 2024/1580, 2024.
[4] Nicholas Carlini, Matthew Jagielski, and llya Mironov. Cryptanalytic Extraction of Neural Network Models. In Annual international cryptology conference, pages 189-218. Springer, 2020.
[5] Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning. arXiv preprint 171..0512..05266, 2017.
[6] Yi Chen, Xiaoyang Dong, Jian Guo, Yantian Shen, Anyu Wang, and Xiaoyun Wang. Hard- Label Cryptanalytic Extraction of Neural Network Models. Cryptology ePrint Archive, Paper 2024/1403, 2024.
[7] Charles Fefferman al. Reconstructing neural from its output. Revista Matemática Iberaaerrcaaa,, 10(3):507-556, 1994.
[8] Hanna Foerster, Robert Mullins, Ila Shumailov, and Jamie Hayes. Beyond Slow Signs in High-fidelity Model Extraction. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages 19496-19522. Curran Associates, Inc., 2024.
[9] Cheng Gongye, Yunsi Fei, and Thomas Wahl. Severse-Engineerie Deep Neural Networks Using Floating-Poin Timing Side-Channels. In 2020 57th ACM/IEEE Design Automation Conference (DAC), pages 1-6. IEER, 2020.
[10] Peter Horvath, Lukasz Chmielewski, Leo Weisshart, Lejla Batina, and Yuval Yarom. Bar- raCUDA: Edge GPUs do Leak DNN Weights. arXiv preprint arXiv:2312.07788 2023.
[11] Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex Kurakin, and Nicolas Papernot. High Accuracy and High Fidelity Extraction of Neural Networks. In 29th USENIX security symposium (USENIX Security 20), pages 1345-1362, 2020.
[12] Ashley Kurian, Anuj Dubey, Ferhat Yaman, and Aydin Aysu. TPUXtract: An Exhaustive Hyperparameter Extraction Framework. IACR Transactions on Cryptographic Hardware and Embedded Systems, 2025(1):788 7003103 2025.
[13] Daniel Lowd and Christopher Meek. Adversarial learning. In Knowledge Discovery and Data Mining, 2005.
[14] Flavio Martinelli, Berfin Simsek, Wulfram Gerstner, and Johanni Brea. Expand-an and-Cluse-r Parameter Recovery of Neural Networks, 2024.
[15] Mantas Mazeika, Li, and David Forsyth. How Steer Your Adversary: Targeted and Efficient Model Stealing Defenses with Gradient Redirection, 2022.
11

---

[16] Smitha Milli, Ludwig Schmidt, Anca D Dragan, and Moritz Hardt. Model Reconstruction from Model Explanations. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 1-9, 2019.
[17] Tribhuvanesh Orekondy, Bernt Schiele, and Mario Fritz. Knockoff Nets: Stealing Functionality of Black-Box Models. Proceedings of the IEEE/CVF conference on computer vision and pattem recognition, pages 4954-4963, 2019.
[18]
Tribhuvanesh Orekondy, Bernt Schiele, and Mario Fritz. Prediction Poisoning: Towards Defenses Against DNN Model Stealing Attacks, 2020.
[19] Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Anan- thram Swami. Practical Black-Box Attacks against Machine Learning. In Proceedings of the 2017 ACM on Asia conference on computer and communications security, pages 506-519, 2017.
[20] David Rolnick and Konrad Kording. Reverse- Engineeiiggnn Deep ReLU Networks. In Interna- tional conference machine learning, pages 8178-8187. PMLR, 2020.
[21] Adi Shamir, Isaac Canales- Martinaz, Anna Hambitzer, Jorge Chavez-Saab, Francisco Rodrigez- Henriquez, and Nitin Satpute. Polynomial Time Cryptanalytic Extraction of Neural Network Models. arXiv preprint ar.0v:2310.08708, 2023.
[22] Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership Inference Attacks Against Machine Learning Models. In 2017 IEEE symposium on security and privacy (SP), pages 3-18. IEEE, 2017.
[23] Shubhi Shukla, Manaar Alam, Sarani Bhattacharya, Pabitra Mitra, and Debdeep Mukhopadhyay. "Whispering MLaaS": Exploiting Timing Channels to Compromise User Privacy in Deep Neural Networks. IACR Transactions on Cryptographic Hardware and Embedded Systems, pages 587-613, 2023.
[24] Kálmán Szentannai, Jalal Al-Afandi, and András Horváth. Preventing Neural Network Weight Stealing via Network Obfuscation, In Intelligent Computing: Proceedings of the 2020 Comput- ing Conference, Volume 3, pages 1-11. Springer, 2020.
[25] Florian Tramèr, Fan Zhang, Ari Juels, Michael K Reiter, and Thomas Ristenpart. Stealing Machine Learning Models via Prediction APIs. In 25th USENIX security symposium (USENIX Security 16), pages 601-618, 2016.
[26] Honggang Yu, Haocheng Ma, Kaichen Yang. Yiqiang Zhao, and Yier Jin. DeepEM: Deep Neural Networks Model Recovery through EM Side-Chanme Information Leakage. In 2020 IEEE International Symposium on Hardware Oriented Security and Trust (HOST), pages 209-218. IEEE, 2020.
[27] Mingyi Zhou, Xiang Gao, Jing Wu, John Grundy, Xiao Chen, Chunyang Chen, and Li Li. ModelObfuscator: Obfuscating Model Information to Protect Deployed ML-based Systems. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, pages 1005-1017, 2023.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We claim develop extraction-awart training method that defends against cryptanalytic attacks with zero overhead during inference, which explained in Section 3 We also claim empirically evaluate defense across different neural network configura- tions; these results are shown in Table []] We claim to provide a theoretical framework for computing the attack success probability, which is presented in Section 3.2
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
Our extraction-awaan training strategy causes accuracy change of less than 1%. We quantify this extensively for all model settings used in prior works as shown in Table The limitations of our proposed countermeasure are discussed in Section 5.5
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model vell-specifiiattin, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
13

---

3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: A theoretical proof of the attack success probability is provided in Section 3.2 The proof follows assumptions described in the threat model Section 2.2
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems. The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide details of the machines used for experiments-2 nigh-performance cluster with Intel Xcon processors. Information about the datasets, model configurations, training, and extraction seeds used to evaluate our defense is included in Section 4 We also report the impact factor (Asimilarity) required to secure each model. The code for reproducing the results is available at the GitHub link:
[https://github.. http:///ihuuu..ommmoooonooo//e/ com/anonymous- -anuuus--1----a -=ccoe//aoonymouscodde
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided instructions for how to replicate the results, access to a hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
detailed
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
14

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We provide a link to our GitHub code base: [htps//giitub.comm/mmmmmuussuuu /o-cod///nonymmoscoodeee
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (kttps://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https //nips cc/ cac//pulic//ae mublic/guides:
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: All model settings, including configurations, training seeds, and extraction seeds, reported in Table Our codebase includes both the original (non-protected) and secure models, along with scripts to create new models using the same training setup in the previous paper [8]. All hyperparameters and optimizer choices specified enable full reproducibility. We adopt the same experimental settings, including hyperparameters and optimizer choices, used in prior work [
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
15

---

Answer: [Yes]
Justification: We follow the error reporting methodology provided in the previous work g by providing the mean, variances for parameter extraction time, and the number of queries for multiple test models.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative еттог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide details of the machine setup used for experiments in Section 4 All experiments were conducted
perh-peerfrrmacce cluster equipped with Intel Xeon processors, comprising 400 compute nodes and over 14,000 cores. Each node is provisioned with an average of 128 GB of memory.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics htt.ii/giir/////ppp.....p.oo/ttt https////cc/l////cccccccooooaaaa
Answer: [Yes]
Justification: This research complies with the NeurIPS Code Ethics. does not involve human subjects, personal data, or sensitive content, and uses only publicly available datasets. Our proposed defense enhances model security without enabling misuse.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
16

---

If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper defends against parameter extraction attacks, which helps prevent model stealing and other threats like input poisoning and membership inference. This is important for protecting intellectual property and avoiding sensitive data leaks in critical areas like healthcare and defense. This is discussed in Section [
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymenn of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our work uses only publicly available datasets (MNIST) and does not involve any models with proprictary intellectual property.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
17

---

12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We cite the original authors of the attacks we defend against and clearly specify the datasets used (MNIST), which are publicly available.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paterssi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We provide a link to our GitHub code base:
https:/ggitubb.commmmoommmuuuuu http://ooomoos--3-cooeeeeaaaa (23-code/amooyymoocoodee
Guidelines:
The answer NA means that the paper does release assets.
Researchers should communicate the details the dataset/code/mmodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification:
This paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
18

---

According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based on the requirements of your country institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.MM policy Cactpp/s//aarrrpss..c///o/eeeeee for what should or should not be described.
19

---

A Defense performance and training overhead analysis
A.1 Ablation experiments on Agimilarity
The optimal value of Asimilarity for a given model depends on the minimum Asimilority that prolongs extraction time without substantially impacting accuracy. The acceptable degree of accuracy change varies depending the specific application task. While higher Asimilarity can enhance security, may also degrade accuracy. Therefore, the optimum Asimilarity depends on the application at hand. Figure Tiguure4illusrraae illustrates the accuracy variations with varying Asimilarity and percentage of defended neurons for two representative architectures: the smallest (8x2) and deeper model ( (16x8). As Asimilarity increases, accuracy progressively decreases across both architectures. For small values (Asimitaritty I 10- the accuracy remains close to the bascline, with minimal degradation even at 100% defended neurons. However, at larger values (Asimitarity 2 10 10-3), substantial accuracy loss occurs, particularly in the smaller 8x2 model where accuracy drops to near-random performance. Notably, the deeper 16x8 architecture exhibits greater robustness, maintaining over 85% accuracy even Asimstarity = 10-2, while the 8x2 model degrades approximately 58% under the same conditions. This suggests that models with larger capacity can better tolerate the constraints imposed by similarity regularization.
KINTT 1046
Figure Impact of similarity regularization on model accuracy across different architectures. Accuracy versus percentage of defended neurons for (left) 8x2 and (right) 16x8 architectures on MNIST, with varying Asimilarity values. The baseline (Agimilariit = 0) maintains stable accuracy around 72.7% and 90.7% for 8x2 and 16x8 respectively. Small lambda values (Asimdarity S 10-7) show minimal accuracy degradation, while larger values progressively reduce accuracy. The 16x8 architecture demonstrates greater robustness to regularization, maintaining higher accuracy even at larger similarity values.
A.2 Training time comparison between baseline and secure model
Figure illustrates that the training time per epoch increases due to the introduced regularization term. The baseline training times at 0% defended neurons are 0.65 seconds for the MNIST 8x2 model, 0.83 seconds for the MNIST 16x8 model, and 1.37 seconds for the random 128x2 model. As the percentage of defended neurons increases from 10% to 100%, training time rises to ranges of [0.98s-3.73s), [1.2s-4.3s], and [2.05s-6.935, for the respective architectures. This overhead stems from the pairwise comparisons introduced by the regularization term, which scale with both the number of neurons per layer and the percentage of neurons being defended. Notably, the training time increases approximately linearly with the defense percentage across all architectures, with larger models exhibiting steeper slopes due the greater number of pairwise comparisons required. The impact of Asimilarity on training time is minimal, as the computational cost is primarily determined by the number of neuron pairs rather than the regularization strength. However, this is one-time training overhead that does not impact inference latency, making the defense practical.
A.3 Impact of defense on model convergence
In our experiments, the similarity regularization alters the number epochs required for convergence by less than five epochs across all evaluated models compared to the bascline. We define convergence
20

---

Figure Training overhead of similarity regularization across different architectures. Average training time per epoch versus percentage of defended neurons for (left) MNIST 8x2, (center) MNIST 16x8, and (right) Random 128x2 architectures with varying Asimiarity values. Baseline training times (0% defense) are 0.65s, 0.83s, and 1.37s respectively. As the percentage of defended neurons increases, training time scales approximately linearly, reaching ranges of (0.98)-3.73s), [1.2s-4.3s], and [2.05s-6.93s). The computational overhead stems from pairwise neuron comparisons, which scale with both network size and the fraction of defended neurons.
the point where the change in training loss between consecutive epochs falls below 0.01 for several consecutive epochs. Based on this criterion, the impact of the similarity term on convergence speed minimal. Tigure/Jshows the training loss trajectories for various Asimalarity values across three representative architectures. The results confirm that incorporating the similarity term does not significantly alter the convergence pattern. Regardless of model size or À magnitude, all networks reach stable loss values within comparable number of epochs, typically differing by fewer than five from the baseline.
Figure 6: Training loss across epochs for different regularization strengths (Asimilarity) on three architectures: MNIST 8x2, MNIST 16x8, and Random 128x2. Across all models, the similarity regularizer causes minimal variation in convergence behavior, with all curves stabilizing within roughly five epochs of the baseline (A = 0).
21


Reviews:
[{"review_id": "34VqlqFzbC", "reviewer": "Reviewer_sTx6", "summary": "Authors have proposed a first defense mechanism against cryptanalytic parameter extraction attacks. They have conducted a comprehensive analysis of cryptanalytic attacks, examined their underlying mechanisms and identifying the key parameter configurations that enhance their efficiency and likelihood of success. The authors derive a closed-form expression for attack success probability based on geometric properties of neuron weights. The proposed defense is lightweight and easy to implement within existing training pipelines.", "questions": "1\tHow would this method generalize to architectures using batch normalization, dropout, or skip connections?\n2\tCan this strategy be extended to protect CNNs or transformer-based architectures?\n3\tAlthough the authors mention that experiments were conducted on a computing cluster, they do not provide specific details about the computational resources used, such as hardware specifications, number of nodes, or training time. Including this information is important for assessing the method's scalability and reproducibility.", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1.\tA first defense mechanism against cryptanalytic parameter extraction attacks.\n2.\tThe theoretical explanation (Section 3.2) rigorously formalizes why the defense works, with solid mathematical justification. \n3.\tEvaluation on multiple architectures and seeds with controlled ablations shows robustness. \nWeakness:\n1.\tThe experiments and threat model in the paper are limited to fully connected (MLP) architectures with ReLU activations in the hidden layers and a linear activation in the output layer. This restricts the applicability of the proposed method in practical, real-world settings where more complex architectures are prevalent. \n2.\tSince the attack depends on ReLU linearity and critical points, it's unclear whether this defense would hold under other activation functions like GELU or sigmoid.\n3.\tIt is unclear whether the proposed defense can generalize to more widely used models such as convolutional neural networks (CNNs), transformers, or other real-world deep learning systems. \n4.\tThe paper also refers to the use of \"random datasets\" but does not clearly explain what does mean. Moreover, the experimental evaluation is on MNIST dataset. Including results on more challenging and diverse datasets, such as CIFAR-10, would better demonstrate the effectiveness and robustness of the proposed defense.", "was_revised": true, "final_justification": "The authors addressed my concerns and I increased my score.", "rebuttal": "We thank the reviewer for their valuable feedback and thoughtful questions. We appreciate the time and effort taken to review our submission. Below, we address each point raised in detail.\n\n### Weaknesses:\n**1.** These are current limitations of the *attack* ([8] published at NeurIPS 2024, [4] published at CRYPTO 2020) which is evolving rapidly, not the defense. Our defense works in all the cases where attacks have been demonstrated. The defense itself has no such limitations and scales fundamentally with the attacks as they grow, since it breaks the core assumption on which the attack relies (neuron dissimilarity).\n\n**2.** The attack only works for piecewise linear activation functions such as ReLU, Leaky ReLU, Hard Sigmoid, and Hard Tanh (approximations), and therefore does not currently apply to GeLU and Sigmoid, which are not piecewise linear. These are limitations of the attack, not the defense. This is mentioned in Lines 91–93. A future variant of the attack extending to other activation functions could still be mitigated with our proposed defense if it continues to rely on neuron dissimilarity.\n\n**3.** Since the cryptanalytic attacks (including the NeurIPS’24 paper [8]) are demonstrated on MLPs, our defense correspondingly focuses on MLPs (lines 350–352).  There is no fundamental reason why the defense couldn’t scale to other neural networks, but the attacks should be demonstrated there first.\n\n**4.** We used the methodology of [4, 8] where “random dataset” refers to a synthetic dataset generated by sampling input features from a standard normal distribution $N(0,1)$, with binary labels determined by thresholding Gaussian noise. This setup is primarily used to analyze model behavior or leakage characteristics independent of dataset-specific semantics, allowing us to isolate architectural effects or test specific properties in a controlled setting. We include the random dataset in our defense evaluation because prior attack works [4, 8] use it to demonstrate the effectiveness of their attacks.\nRegarding real-world datasets, the end-to-end prior NeurIPS attack [8] is evaluated only on the MNIST dataset and has not been demonstrated on CIFAR-10. The authors note that additional modifications are required to generalize the attack to CIFAR-10. Since our defense targets the core assumption of the attack, it should remain effective regardless of the dataset on which the model is trained.\n\n### Questions:\n**1.** Cryptanalytic attacks have only been demonstrated on architectures with fully connected layers that do not include components such as batch normalization, dropout, or skip connections. Accordingly, we focus our defense evaluation on architectures that have been shown to be vulnerable in prior works. There is no fundamental reason why our proposed defense can’t scale to other architectures but the attacks should be demonstrated first.\n\n**2.** Since the cryptanalytic attacks, including the state-of-the-art NeurIPS’24 paper, are demonstrated on MLPs, our defense is demonstrated on MLPs (Lines 350–352).  But it could be extended to other architectures as the attacks grow and target them in the future.\n\n**3.** All training and extraction were performed on a high-performance cluster equipped with Intel Xeon processors, comprising 400 compute nodes and over 14,000 cores. On average, each node is provisioned with 128 GB of memory (Lines 294–296). Training time per epoch ranges from [0.98s–3.73s] as the percentage of defended neurons increases from 10% to 100%  for the smallest model we evaluated (784-8×2-1) to [2.05s–6.93s] for the largest model (784-128×2-1). Training time increases with both the number of neurons per layer and the number of layers [8]. We will include this information in the paper."}, {"review_id": "bmK6JWyBG3", "reviewer": "Reviewer_oPUs", "summary": "The discovery of the cryptanalytic extraction attack in this paper is insightful: the less similar the weights, the more effective the attack. To mitigate the above attack, the method proposed in this paper is concise: introducing a regularization term into the loss function. Subsequently, the fact that enforcing similarity among intra-layer neuron weights can lead to a challenge in model accuracy is a critical concerning issue. Correspondingly, the method this paper uses to solve this problem is also elegant: varying the impact factor, restricting defense to the first layer and defending a subset of neuron pairs. Theoretical analysis in Section 3.2 bounds the attack success probability. Empirical tests show that protected MLP models resist extraction for over 24,000 hours, with less than 1% accuracy loss.", "questions": "(1)\tHow was λsimilarity selected? Is there a pratical upper limit on λ?\n(2)\tHow does the defense affect expressivity on harder tasks?\n(3)\tDoes similarity slow down convergence?\n(4)\tCan the defense be used incrementally (fine-tune existing models) or must full retraining be done?\n(5)\tWill this method scale to modern deep neural networks (e.g., Transformers, ResNets)?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n(1)\tAttacks on NN parameters are becoming practical; this is the first clear defense against cryptanalytic attacks in particular.\n(2)\tExtraction-aware training targeting intra-layer neuron similarity is simple and elegant.\n(3)\tThis paper provides a clear explanation of how neuron uniqueness enables attacks, which is an insightful observation and proof.\n(4)\tThe method proposed in this paper is easy to implement and incurs no inference overhead.\n(5)\tSection 3.2 provides an upper-bound theoretical analysis on success probability.\n(6)\tThis paper is well-organize and clear written.\n\nWeaknesses\n(1)    The method designed in this paper is concise. However, it has only been experimented on MLP models (≤4 layers, ≤128 neurons) so far. Based on the experimental results, the generalizability of this method is limited—it hasn’t been verified through experiments on CNNs or other models. This leads to unclear application to current popular models. As a result, some more in-depth questions can’t be analyzed at present. For example: Does forcing neuron similarity harm robustness on out-of-distribution (OOD) data? Or does this limit the model’s ability to fit complex data?\n(2)    The tuning strategy proposed in this paper (restricting defense to the first layer and defending a subset of neuron pairs) is well-designed. However, based on the above design, the paper currently lacks analysis of very deep neural networks, for example, those with 50+ layers.\n(3)    One drawback of this method is that the introduction of the regularization term leads to neuron similarity, which in turn harms model accuracy. There is a trade-off between security and accuracy, so the adjustment of λsimilarity is crucial. However, the paper lacks ablation experiments on λsimilarity. Section 3.4 mentions tuning, but no ablation study is shown. This makes it impossible to provide a systematic method for choosing λsimilarity.\n(4)   There is a lack of details on how much training time increases due to the defense term. For the introduced regularization term, as mentioned in Line 266: “For a layer with n neurons, there will be n(n−1)/2 pairwise comparisons.” However, the paper provides no training time comparison to the baseline. Referring to the previously mentioned issue 1, the analysis of this training cost on some popular deep neural networks is missing.", "was_revised": true, "final_justification": "I will keep the original score since my concerns have not been solved.", "rebuttal": "We thank the reviewer for their insightful comments and constructive feedback. We appreciate the time and effort dedicated to evaluating our work. Below, we provide detailed responses to each of the questions raised.\n\n### Weaknesses:\n\n**1.** The cryptanalytic attacks, including the NeurIPS’24 attack [8], are demonstrated on MLPs; therefore, our defense focuses on MLPs as well (Lines 350–352). We have evaluated our defense on all model architectures used in prior works where the end-to-end attacks were demonstrated.\n\n**2.** Our goal in this paper was to demonstrate that the cryptanalytic attacks presented in prior works on their chosen models can be effectively defended using our proposed defense. The defense itself has no such limitations and fundamentally scales with the attacks as they advance, since it breaks the key assumption underlying the attack—namely, neuron dissimilarity.\n\n**3.** Thank you for pointing this out. The optimal value of $λ_{similarity}$​ for a given model depends on the minimum $λ_{similarity}$​​ that prolongs extraction time without substantially impacting accuracy. The acceptable degree of accuracy change varies depending on the specific application or task. While a higher  $λ_{similarity}$​ can enhance security, it may also degrade accuracy. This is why we emphasize the need to tune  $λ_{similarity}$​ based on use case requirements. We have graphs that show the accuracy vs. percentage of defended neurons with different lambda values, which we did not include in the paper due to space constraints.  We will include this study—illustrating the relationship between  $λ_{similarity}$​​, accuracy changes, and extraction time (security) in the appendix of the revised version.\n\n**4.** Training time per epoch increases due to the introduced regularization term. For the smallest model (784-8×2-1), training time rises from 0.65 seconds (baseline) to [0.98s–3.73s] as the percentage of defended neurons increases from 10% to 100%. For the largest model (784-128×2-1), it increases from 1.37 seconds to [2.05s–6.93s] across the same defense range. This overhead stems from the pairwise comparisons introduced by the regularization term, which scale with both the number of neurons per layer and the total number of layers. However, this is a one-time training overhead and does not impact inference latency.  We will include this training time analysis in the revised version of the paper.\n\n### Questions:\n\n**1.** For the evaluation,  $λ_{similarity}$​​ ​ was initially set to a small value  ($10^{-10}$) and incrementally increased by a factor of 10, with extraction time measured at each step. This process continues until we have extended periods of extraction. The upper limit of  $λ_{similarity}$​​​ depends on the acceptable accuracy loss for a given application. From the models we evaluated, we selected the minimum $λ_{similarity}$​​​ that ensures security while minimizing accuracy degradation. Our experiments showed that accuracy change remained below 1%. \n\n**2.** Our defense does not impact the model’s ability to learn harder or more complex tasks, as evidenced by the minimal accuracy change (less than 1%). Furthermore, evaluations on both a random dataset (which isolates dataset-specific semantics) and the widely used MNIST dataset support this conclusion.\n\n**3.** In our experiments, the similarity regularization alters the number of epochs required for convergence by less than five epochs across all evaluated models compared to the baseline. We define convergence as the point where the change in training loss between consecutive epochs falls below 0.01 for several consecutive epochs. Based on this criterion, the impact of the similarity term on convergence speed is minimal. We will include this analysis in the revised manuscript.\n\n**4.** In the current implementation, the defense is applied during full model training, as the similarity regularizer operates on weight vectors across layers. However, it may be extended to support incremental fine-tuning by re-compiling a pre-trained model with the custom loss function and selectively updating relevant layers while freezing others. While this approach may result in lower accuracy or slower convergence, it offers the advantage of reduced training time. We will include a discussion of this fine-tuning strategy along with its benefits and trade-offs in the revised manuscript.\n\n**5.** Cryptanalytic attacks currently apply only to MLPs with piecewise linear activations. These are current  limitations of the attack (which is rapidly evolving), not of the defense. The defense itself has no such limitations and fundamentally scales with the attacks as they advance, since it breaks the key assumption underlying the attack—namely, neuron dissimilarity."}, {"review_id": "Ol0S2yNZDa", "reviewer": "Reviewer_AAM3", "summary": "The paper introduces the first dedicated defense against cryptanalytic model-stealing attacks that recover a neural network’s exact weights through carefully crafted black-box queries. The authors observe that these attacks succeed only when neurons in a layer have distinct hyperplanes. Their key idea is therefore to break neuron uniqueness at training time by adding a similarity regularizer to the loss. This extraction-aware training makes weights inside each layer converge toward one another, so multiple neurons activate (or deactivate) together. Because the attacker can no longer isolate a single neuron’s contribution, both signature-recovery and sign-recovery stages of the cryptanalytic pipeline fail. The authors also provide a theoretical upper bound on the attack’s success probability as a function of weight dissimilarity, bridging empirical results and security guarantees. Overall, the work offers a practical, training-time-only method to harden today’s ReLU MLPs against state-of-the-art parameter-extraction threats with negligible performance cost.", "questions": "1. Precise adaptive extraction after detecting identical hyperplanes (collapse / coefficient-split): If the attacker realises that two first-layer neurons have identical hyperplanes, they could (a) collapse the pair into a single unit with coefficient c1+c2​, solve the reduced system of linear equations, and pad the result, or (b) keep both neurons but choose an arbitrary split such as c1*=c2*=0.5(c1+c2) to avoid changing the layer width. In either variant the functional outputs remain correct. Can you clarify the linear algebra of why extraction is guaranteed to fail in later layers? I suppose this should go into the direction on ranks of matrices, degrees of freedom and invertibility of matrices.\n\n2. Imprecise adaptive extraction: Suppose the attacker extracts all neurons but is stuck in the neuron searching loop for a long time, so they determine some of the neurons must be duplicate or very similar. Suppose the attacker can then determine the most likely candidates for duplicates through maybe the amount of critical points it has found for the neurons or something like that. Now, if they added a small, deliberate perturbation delta to one of the duplicate weight vectors so that the two hyperplanes are almost—but not exactly—aligned and they would then run the cryptanalytic extraction of the next layer on this slightly “noisy” model, what would happen? Is it possible to find fake critical points (basically critical points with known small errors) for this noisy neuron that will not hinder extraction in the next layer, and can contribute to the full view of activations and deactivations needed from the previous layer to extract the next layer? Further, errors will propagate, but will they propagate so badly that the model is unrecoverable? Earlier work shows that an attacker can refine an initial weight guess as they go: after estimating a layer’s weights, they check how well those weights explain the activations in the next layer and tweak them if they don’t line up. Preciseness of weights can also be enhanced later on. How feasible will extraction then be? \n\n3. λ selection: can you offer a heuristic that gives near-optimal security without a full grid search or relying just on intuition and retraining until the tradeoff is good?", "limitations": "- **Addressed:** [Yes]  \n- **Comments:**  \n  - Adaptive attacks are not considered and not mentioned.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Quality  \n**Strengths:**  \n- Sound empirical methodology. The original cryptanalytic attack is reproduced in the same way as it was evaluated in previous work, with different parameter settings. The parameter on lambda similarity derived from the theoretical framework gives and especially credible picture of robustness vs. accuracy.\n- Zero-cost inference. The defense is applied during training and therefore there is no latency overhead in inference. \n- Theoretical framework. The paper picked up on the difference in extraction difficulty in different neuron constellations presented in previous work and constructed a theoretical framework around it which grounds the defense theoretically.\n\n**Weaknesses:**  \n- Missing explanation on attempts to deal with two neurons with the same or similar hyperplane. Refer to questions.\n\n\n### Clarity  \n**Strengths:**  \n-  Ideas are presented clearly from the beginning, with a intuitive example and more mathematical details following later on.\n- Figure 3b is especially nice to understand how the weight distributions change. \n\n**Weaknesses / Suggestions:**  \n- For readers not familiar with the literature it might be unclear why not extracting all neurons in the first layer will prohibit the extraction in the next layer and how next layer extraction even works.\n\n### Significance  \n**Strengths:**  \n- First training-time defense against cryptanalytical extraction; prior suggestions have mostly been on noise or rate limiting.\n- Bridges empirical security and provable bounds.\n\n**Weaknesses / Suggestions:**  \n- Threat niche is narrow: there are not that many models that can actually be stolen with cryptanalytical extractions, since it only works for small MLPs. Further, for those networks, why not do knowledge distillation instead?\n\n### Originality  \n**Strengths:**  \n- Turns an observed weakness (attack fails when two neurons coincide) into a full defense—clever inversion of prior work.\n- Introduces a similarity-regularizer for training.\n\n**Weaknesses / Suggestions:**", "was_revised": true, "final_justification": "The authors gave very thoughtful answers to my questions, underlining the defense's strength even in adaptive settings by carefully walking through the underlying linear algebra of what would happen in each setting. As before, I am confident about this paper's strength and recommend it to get accepted.", "rebuttal": "We sincerely thank the reviewer for their insightful and thoughtful questions. We greatly appreciate the careful consideration and effort invested in evaluating our work. Below, we provide detailed responses to each of the points raised.\n\n### Weaknesses:\n\n**Quality** Please see our response to Question 1 below.\n\n**Clarity** We encourage readers unfamiliar with the literature to consult prior attacks [4, 2, 21] for a deeper understanding of the attack (see line 180).\n\n**Significance** The threat niche has been validated by NeurIPS'24 paper [8], among others, and is growing rapidly. It is important to explore and invest in defenses now, rather than having them as an afterthought. Our defense fundamentally scales with the attacks as they advance, since it breaks the key assumption underlying the attack—namely, neuron dissimilarity. While the full end-to-end cryptanalytic extraction (sign and signature recovery) has been demonstrated on models with up to 4 hidden layers, a key component of the attack, sign extraction has been shown to scale to larger models with 1.2 million parameters and 8 hidden layers [2]. This demonstrates that the core vulnerability persists in deeper networks. We evaluate the models targeted in the end-to-end attack work [8], but the findings in [2] indicate that cryptanalytic techniques can be extended to deeper and more complex MLPs, underscoring the need for robust defenses.\n\n### Questions:\n\n**1.**  While functional correctness is preserved through adaptive extraction techniques such as simplifying the first-layer representation by collapsing neurons with identical hyperplanes or applying an arbitrary coefficient split, parameter recovery in subsequent layers fails due to disruption in the intermediate representations.\n\n Let $f(x) = F^{L} ∘ F^{L−1} ∘⋯∘ F^{1} (x)$ where $F^{i}$ denotes the function computed at layer $i$.\nThe attacker tries to recover layer $i$ using the differential signature method involving second-order derivatives [2]. For a target neuron at layer $i$, its signature $a ∈ R^{d_{i−1}}$ satisfies: $h_k^T . a = y_k$, where $h_k = F^{i−1} (x^∗ + δ_k)$, the output from layer (i−1), and $y_k = \\frac{∂^2 f(x^∗) }{∂δ_1 ∂δ_k​}$ are known to the attacker.\n\nKey point:\nThe system of equations $h_k^T . a = y_k$ is solvable for $a$ only if the $h_k​$ vectors (i.e., activations from layer i−1) are accurately reconstructed. An example demonstrating how (a) neuron collapse and (b) neuron coefficient split cause the attack to fail in later layers follows:\n\nSetup: Assume a 2-layer neural network with input dimension: $d_0 = 3$, Layer 1 (L1): 2 neurons with weights $w_1,w_2 ∈ R^3$, Layer 2 (L2): 1 neuron with weights $v ∈ R^2$, Activation: ReLU after each layer. We analyze how errors in reconstructing L1​ affect L2​.\n\nGround Truth Network: Let $w_1=[1,1,1], w_2=[1,1,1]$ (i.e., identical hyperplanes)\n- L1 outputs (after ReLU): $h = [ReLU(w_1^T . x), ReLU(w_2^T . x)] = [a,a] $\n- L2 weight: $v = [1,−1]$\n- Final output: f(x) = $v^T . h = [1, −1] . [a, a] = a - a = 0 $\n\nCase a: Collapse identical neurons: Attacker observes $w_1 = w_2$​ and collapses them:\n- Sets $w' = w_1 + w_2 = [2,2,2]$\n- Sets new L1 output: $h' = [ReLU(2a)]$\n- To preserve functionality, attacker adjusts L2 weights: Sets $v' = [1−1] = [0]$\nNow: Output = $f'(x) = v'⋅ h' = 0$ (same as growth truth network)\n\nBut for extraction from L2:\n- Attacker collects new signature equations: $h_k'^T ⋅ b = y_k$​\n- Here $h_k' ∈ R^1$ instead of $R^2$ \n- Rank loss: Rank reduces from 2 to 1; Linear system underdetermined; Original $v$ cannot be recovered.\n\nConclusion: Attacker loses dimensionality information and cannot invert the system.\n\nCase b: Arbitrary coefficient-split: Attacker guesses a balanced split instead of exact $w_1 = w_2$, sets $w_1' = w_2' = [0.5,0.5,0.5]$\n- L1 outputs: $h' = [ReLU(0.5a), ReLU(0.5a)] = [0.5a,0.5a]$\n- Adjusted L2​ weights: $v' = [2,−2]$ (to match overall output 0)\n- Functional output = $2\\cdot 0.5a − 2\\cdot 0.5a = 0$\n- Extraction via signature equations: $h_k'^T . v = y_k → [0.5a, 0.5a] . v = y_k  → 0.5a (v_1​ + v_2​) = y_k​$\n- Hence, the attacker only recovers $v_1 + v_2$, not $v$\n\n Therefore, even when output functionality is preserved, rank reduction or change of span in the activations prevents second-layer (and deeper) parameter recovery in high-fidelity model extraction attacks.\n\n**2.** This is an excellent and nuanced question. Let’s break it down step-by-step. \n\nSmall perturbation to duplicate weights: \n\nSuppose two neurons in layer 1 have identical weights $w_1 = w_2$, and the attacker \nintroduces a small deliberate perturbation, $w2 = w1 + δ$ with $∥δ∥ ≪1$. This means the two hyperplanes are “nearly aligned” but not exactly so. The ReLU activation boundary of the perturbed neuron slightly rotates, shifting its critical points (where the neuron switches from active to inactive) by a small amount.\n\nEffect on layer 1 activations:\n\nThe attacker’s model now uses a “fake neuron” with a slightly incorrect activation pattern. If the true post-ReLU outputs from layer 1 are $h = [ReLU(w_1^T .x), ReLU(w_2^T .x)]$, the attacker instead uses: $h'= [ReLU(w_1^T.x), ReLU((w_1+δ)^T.x)]$. This mismatch introduces errors in the activation matrix of layer 1.\n\nImpact on layer 2 extraction:\n- Extraction methods rely on accurately knowing the inputs to layer 2—the post-ReLU outputs from layer 1\n-  Because h' deviates from the true h, the signature equation at layer 2 changes: $h^T . a = y$ (ground truth), $h' ^T . a' = y'$ (attacker’s version)\n- Even if the attacker applies the same second-derivative approach, they solve the system with incorrect inputs, resulting in recovered weights a' that deviate from the true a.\n\nError propagation to deeper layers:\n\nIf layer 2 has m neurons and k are extracted using these inaccurate activations, the attacker’s model accumulates noise in the activation signature matrix. This noise reduces the rank or injects uncertainty, weakening the solvability of subsequent linear systems. As the extraction proceeds to deeper layers, accumulated errors increasingly distort activation signatures, leading to numerical instability. Such error accumulation can cause extractions in deeper layers to fail or converge poorly. Attempting to fix these errors later by adjusting weight guesses is difficult because: \n\n- The same output can correspond to many different parameter configurations (non-uniqueness).\n- Without perfect intermediate activations (which are unobservable), there is no oracle to assess the quality of current parameter estimates.\n\nOur defense effectively disrupts the invertibility chain the attacker relies on, preventing successful high-fidelity extraction. Therefore, model extraction becomes infeasible in this setting as attacker errors accumulate. This accumulation of activation mismatches introduces non-negligible errors in intermediate representations, which propagate and amplify through successive layers. As demonstrated in the error bounds propagation analysis [4], the maximum error at the output of layer $i+1$ can be bounded as *(Error at layer i+1) ≤ (Error at layer i) × (largest singular value of the weight misalignment) + (bias misalignment term)*. Furthermore, MILP-based analysis demonstrates that ensuring (ε, 0)-functional equivalence necessitates bounding the total deviation of the model parameters to within $10^{-10}$ [4]. These results collectively highlight the fragility of errors on high-fidelity extraction procedures.\n\n\n**3.** Thank you for the insightful question. We agree that avoiding an exhaustive grid search for selecting λ is important for both efficiency and practicality. While the paper did not explore these aspects, two viable alternatives can possibly yield near-optimal λ values with lower overhead as described below. We propose adding this to the discussion section or appendix based on available space.\n\nA: Binary search heuristic\n\nApproach: Start with a broad interval for λ (e.g., $[10^{-10},10^{2}]$ and iteratively narrow it using binary search. Selection is guided by a simple objective—such as performance on a validation set (e.g., accuracy, loss convergence) and a proxy for robustness (e.g., attack failure or increased extraction time).\n\nPros: \n- Simple and broadly applicable without domain-specific assumptions.\n- Reduces search complexity from $O(n)$ (grid) to $O(\\log n)$.\n- Can converge quickly to a λ that balances utility and robustness.\n\nCons:\n- Still requires multiple training runs (though fewer than grid search).\n- Requires a good stopping criterion (e.g., acceptable accuracy drop or sufficient attack failure).\n\n\nB: Gradient-norm-based λ estimation\n\nApproach: Use the relative gradient magnitudes of the primary loss $L0$ and the regularizer $R$ to set λ. After training a baseline model (without regularization) for a few epochs compute: $λ = α . \\frac{∥∇_θ​ L0∥​}{ ∥∇_θ​ R∥}$ where $α ∈ (0.05,0.2)$ represents the desired influence ratio of the regularizer.\n\nPros:  \n- Provides an interpretable λ tied to the actual gradient dynamics.\n- Requires only one extra forward-backward pass on a held-out batch.\n- Makes regularization adaptive to the model and data.\n\n\nCons:\n- Still involves tuning of α (though it's much less sensitive than λ).\n- Requires computing gradient of the regularizer\n\nConclusion: If fast convergence is the priority and some training overhead is acceptable, binary search is a practical and efficient fallback. On the other hand, if interpretability and principled control over regularization strength are desired, the gradient-norm heuristic is a compelling, scalable choice. We believe either method—depending on available resources could be effective for selecting λ without relying on exhaustive grid search or intuition."}, {"review_id": "89V7QB0pdK", "reviewer": "Reviewer_43qQ", "summary": "This paper presents the defense mechanism against cryptanalytic neural network parameter extraction attacks. The key insight behind the defense is to eliminate the uniqueness among neurons within the same layer. The proposed solution is an \"extraction-aware\" training method that modifies the standard loss function. Some experiments show the results of the method.", "questions": "See comments.", "limitations": "Yes.", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 1, "clarity": 1, "significance": 2, "originality": 1, "strengths_and_weaknesses": "Strengths\n- The paper has some theoretical discussion on the method.\n- The paper attempts to address an important issue, cryptanalytic neural network parameter extraction attacks.\n\nWeaknesses\n- The paper lacks a coherent logical structure in presenting its ideas.\n- The Introduction fails to identify current challenges in this field clearly. The contributions section is poorly organized, with redundant paragraphs discussing the same points and irrelevant content that does not relate to the stated contributions.\n- The methodology section lacks clarity in its presentation. While theoretical discussions and explanations are provided, the number of stages/modules in the method remains unclear, and the mechanism for defending against attacks is not well explained.\n- Section 4 presents the experiments without the clear organization of settings, results, and discussion. The limited number of figures and tables provided is insufficient to demonstrate the method's effectiveness.", "was_revised": true, "final_justification": "The authors didn't address the weaknesses of the paper in the rebuttal. I will keep the original score.", "rebuttal": "Thank you for taking the time to review our submission.\n\nWe believe this review may not reflect a careful reading of the paper. The comments lack rigor and gravitas, are inconsistent with the assessments of the other reviewers, and lack specific actionable feedback. We believe that the paper has a logical structure, identifies contributions clearly, has clear methodology sections, and substantiates the key claims with sufficient figures and tables, as has been identified by other reviewers.  We respectfully suggest that if there is an issue about the points raised, such as clarity of organization or articulation of contributions, they can be readily addressed with a straightforward editorial update and should not form the basis of a reject. We would welcome more precise guidance on what needs improvement, as this would help us ensure the paper better meets expectations."}]

Author rebuttal:

Meta-review: This manuscript proposes a defense against cryptanalytic parameter extraction attacks. The main idea is an extraction-aware training algorithm that reduces a neuron's uniqueness by minimizing the distance between neuron weights within a layer. The paper also provides a theoretical analysis explaining why this approach reduces the attacker’s success probability. Empirical evaluation demonstrates the effectiveness of the defense on fully connected networks with ReLU-like activation functions.

All reviewers recognize the importance of the threat model and highlight the novelty of presenting the first defense during training (no increase in the inference cost). The theoretical analysis offers strong intuition for why the attack succeeds, and the defense is well-motivated by this insight. Several reviewers also emphasized the systematic nature of the evaluation.

The main weaknesses raised concern the generalizability of both the attack and the defense. However, the AC finds that the focus on MLPs with ReLU-like models is within the appropriate scope of this work. One reviewer noted scalability concerns, but the AC observes that even recent work (e.g., at ICML) targeted deeper networks by extracting only the last few layers, suggesting that the proposed defense remains relevant and effective. It would have been the spotlight or oral if those were shown in the manuscript, yet it is still ready for publication.

A reviewer raised presentation issues, but the AC believes these do not affect the novelty and can be addressed in the camera-ready version within the shepherding limits.

**Recommendation:** Accept.
Decision: Accept (poster)