Title: Train to Defend: First Defense Against Cryptanalytic Neural Network Parameter Extraction Attacks
Venue: NeurIPS 2025
Accepted: yes
Keywords: Neural network

cryptanalytic attack

parameter extraction

neuron similarity

training
Abstract: Neural networks are valuable intellectual property due to the significant computational cost, expert labor, and proprietary data involved in their development. Consequently, protecting their parameters is critical not only for maintaining a competitive advantage but also for enhancing the model's security and privacy. Prior works have demonstrated the growing capability of cryptanalytic attacks to scale to deeper models. In this paper, we present the first defense mechanism against cryptanalytic parameter extraction attacks. Our key insight is to eliminate the neuron uniqueness necessary for these attacks to succeed. We achieve this by a novel, extraction-aware training method. Specifically, we augment the standard loss function with an additional regularization term that minimizes the distance between neuron weights within a layer. Therefore, the proposed defense has zero area-delay overhead during inference. We evaluate the effectiveness of our approach in mitigating extraction attacks while analyzing the model accuracy across different architectures and datasets. When re-trained with the same model architecture, the results show that our defense incurs a marginal accuracy change of less than 1\% with the modified loss function. Moreover, we present a theoretical framework to quantify the success probability of the attack. When tested comprehensively with prior attack settings, our defense demonstrated empirical success for sustained periods of extraction, whereas unprotected networks are extracted between 14 minutes to 4 hours.


Peer reviews:
comments: SUMMARY:
This paper presents the defense mechanism against cryptanalytic neural network parameter extraction attacks. The key insight behind the defense is to eliminate the uniqueness among neurons within the same layer. The proposed solution is an "extraction-aware" training method that modifies the standard loss function. Some experiments show the results of the method.

STRENGTHS AND WEAKNESSES:
Strengths
- The paper has some theoretical discussion on the method.
- The paper attempts to address an important issue, cryptanalytic neural network parameter extraction attacks.

Weaknesses
- The paper lacks a coherent logical structure in presenting its ideas.
- The Introduction fails to identify current challenges in this field clearly. The contributions section is poorly organized, with redundant paragraphs discussing the same points and irrelevant content that does not relate to the stated contributions.
- The methodology section lacks clarity in its presentation. While theoretical discussions and explanations are provided, the number of stages/modules in the method remains unclear, and the mechanism for defending against attacks is not well explained.
- Section 4 presents the experiments without the clear organization of settings, results, and discussion. The limited number of figures and tables provided is insufficient to demonstrate the method's effectiveness.

QUESTIONS:
See comments.

LIMITATIONS:
Yes.
TITLE: Official Review
CLARITY: 1
IMPACT: 2
ORIGINALITY: 1
DATE: 2025-06-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces the first dedicated defense against cryptanalytic model-stealing attacks that recover a neural network’s exact weights through carefully crafted black-box queries. The authors observe that these attacks succeed only when neurons in a layer have distinct hyperplanes. Their key idea is therefore to break neuron uniqueness at training time by adding a similarity regularizer to the loss. This extraction-aware training makes weights inside each layer converge toward one another, so multiple neurons activate (or deactivate) together. Because the attacker can no longer isolate a single neuron’s contribution, both signature-recovery and sign-recovery stages of the cryptanalytic pipeline fail. The authors also provide a theoretical upper bound on the attack’s success probability as a function of weight dissimilarity, bridging empirical results and security guarantees. Overall, the work offers a practical, training-time-only method to harden today’s ReLU MLPs against state-of-the-art parameter-extraction threats with negligible performance cost.\n\nSTRENGTHS AND WEAKNESSES:\n### Quality  \n**Strengths:**  \n- Sound empirical methodology. The original cryptanalytic attack is reproduced in the same way as it was evaluated in previous work, with different parameter settings. The parameter on lambda similarity derived from the theoretical framework gives and especially credible picture of robustness vs. accuracy.\n- Zero-cost inference. The defense is applied during training and therefore there is no latency overhead in inference. \n- Theoretical framework. The paper picked up on the difference in extraction difficulty in different neuron constellations presented in previous work and constructed a theoretical framework around it which grounds the defense theoretically.\n\n**Weaknesses:**  \n- Missing explanation on attempts to deal with two neurons with the same or similar hyperplane. Refer to questions.\n\n\n### Clarity  \n**Strengths:**  \n-  Ideas are presented clearly from the beginning, with a intuitive example and more mathematical details following later on.\n- Figure 3b is especially nice to understand how the weight distributions change. \n\n**Weaknesses / Suggestions:**  \n- For readers not familiar with the literature it might be unclear why not extracting all neurons in the first layer will prohibit the extraction in the next layer and how next layer extraction even works.\n\n### Significance  \n**Strengths:**  \n- First training-time defense against cryptanalytical extraction; prior suggestions have mostly been on noise or rate limiting.\n- Bridges empirical security and provable bounds.\n\n**Weaknesses / Suggestions:**  \n- Threat niche is narrow: there are not that many models that can actually be stolen with cryptanalytical extractions, since it only works for small MLPs. Further, for those networks, why not do knowledge distillation instead?\n\n### Originality  \n**Strengths:**  \n- Turns an observed weakness (attack fails when two neurons coincide) into a full defense—clever inversion of prior work.\n- Introduces a similarity-regularizer for training.\n\n**Weaknesses / Suggestions:**\n\nQUESTIONS:\n1. Precise adaptive extraction after detecting identical hyperplanes (collapse / coefficient-split): If the attacker realises that two first-layer neurons have identical hyperplanes, they could (a) collapse the pair into a single unit with coefficient c1+c2​, solve the reduced system of linear equations, and pad the result, or (b) keep both neurons but choose an arbitrary split such as c1*=c2*=0.5(c1+c2) to avoid changing the layer width. In either variant the functional outputs remain correct. Can you clarify the linear algebra of why extraction is guaranteed to fail in later layers? I suppose this should go into the direction on ranks of matrices, degrees of freedom and invertibility of matrices.\n\n2. Imprecise adaptive extraction: Suppose the attacker extracts all neurons but is stuck in the neuron searching loop for a long time, so they determine some of the neurons must be duplicate or very similar. Suppose the attacker can then determine the most likely candidates for duplicates through maybe the amount of critical points it has found for the neurons or something like that. Now, if they added a small, deliberate perturbation delta to one of the duplicate weight vectors so that the two hyperplanes are almost—but not exactly—aligned and they would then run the cryptanalytic extraction of the next layer on this slightly “noisy” model, what would happen? Is it possible to find fake critical points (basically critical points with known small errors) for this noisy neuron that will not hinder extraction in the next layer, and can contribute to the full view of activations and deactivations needed from the previous layer to extract the next layer? Further, errors will propagate, but will they propagate so badly that the model is unrecoverable? Earlier work shows that an attacker can refine an initial weight guess as they go: after estimating a layer’s weights, they check how well those weights explain the activations in the next layer and tweak them if they don’t line up. Preciseness of weights can also be enhanced later on. How feasible will extraction then be? \n\n3. λ selection: can you offer a heuristic that gives near-optimal security without a full grid search or relying just on intuition and retraining until the tradeoff is good?\n\nLIMITATIONS:\n- **Addressed:** [Yes]  \n- **Comments:**  \n  - Adaptive attacks are not considered and not mentioned.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe discovery of the cryptanalytic extraction attack in this paper is insightful: the less similar the weights, the more effective the attack. To mitigate the above attack, the method proposed in this paper is concise: introducing a regularization term into the loss function. Subsequently, the fact that enforcing similarity among intra-layer neuron weights can lead to a challenge in model accuracy is a critical concerning issue. Correspondingly, the method this paper uses to solve this problem is also elegant: varying the impact factor, restricting defense to the first layer and defending a subset of neuron pairs. Theoretical analysis in Section 3.2 bounds the attack success probability. Empirical tests show that protected MLP models resist extraction for over 24,000 hours, with less than 1% accuracy loss.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n(1)\tAttacks on NN parameters are becoming practical; this is the first clear defense against cryptanalytic attacks in particular.\n(2)\tExtraction-aware training targeting intra-layer neuron similarity is simple and elegant.\n(3)\tThis paper provides a clear explanation of how neuron uniqueness enables attacks, which is an insightful observation and proof.\n(4)\tThe method proposed in this paper is easy to implement and incurs no inference overhead.\n(5)\tSection 3.2 provides an upper-bound theoretical analysis on success probability.\n(6)\tThis paper is well-organize and clear written.\n\nWeaknesses\n(1)    The method designed in this paper is concise. However, it has only been experimented on MLP models (≤4 layers, ≤128 neurons) so far. Based on the experimental results, the generalizability of this method is limited—it hasn’t been verified through experiments on CNNs or other models. This leads to unclear application to current popular models. As a result, some more in-depth questions can’t be analyzed at present. For example: Does forcing neuron similarity harm robustness on out-of-distribution (OOD) data? Or does this limit the model’s ability to fit complex data?\n(2)    The tuning strategy proposed in this paper (restricting defense to the first layer and defending a subset of neuron pairs) is well-designed. However, based on the above design, the paper currently lacks analysis of very deep neural networks, for example, those with 50+ layers.\n(3)    One drawback of this method is that the introduction of the regularization term leads to neuron similarity, which in turn harms model accuracy. There is a trade-off between security and accuracy, so the adjustment of λsimilarity is crucial. However, the paper lacks ablation experiments on λsimilarity. Section 3.4 mentions tuning, but no ablation study is shown. This makes it impossible to provide a systematic method for choosing λsimilarity.\n(4)   There is a lack of details on how much training time increases due to the defense term. For the introduced regularization term, as mentioned in Line 266: “For a layer with n neurons, there will be n(n−1)/2 pairwise comparisons.” However, the paper provides no training time comparison to the baseline. Referring to the previously mentioned issue 1, the analysis of this training cost on some popular deep neural networks is missing.\n\nQUESTIONS:\n(1)\tHow was λsimilarity selected? Is there a pratical upper limit on λ?\n(2)\tHow does the defense affect expressivity on harder tasks?\n(3)\tDoes similarity slow down convergence?\n(4)\tCan the defense be used incrementally (fine-tune existing models) or must full retraining be done?\n(5)\tWill this method scale to modern deep neural networks (e.g., Transformers, ResNets)?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nAuthors have proposed a first defense mechanism against cryptanalytic parameter extraction attacks. They have conducted a comprehensive analysis of cryptanalytic attacks, examined their underlying mechanisms and identifying the key parameter configurations that enhance their efficiency and likelihood of success. The authors derive a closed-form expression for attack success probability based on geometric properties of neuron weights. The proposed defense is lightweight and easy to implement within existing training pipelines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1.\tA first defense mechanism against cryptanalytic parameter extraction attacks.\n2.\tThe theoretical explanation (Section 3.2) rigorously formalizes why the defense works, with solid mathematical justification. \n3.\tEvaluation on multiple architectures and seeds with controlled ablations shows robustness. \nWeakness:\n1.\tThe experiments and threat model in the paper are limited to fully connected (MLP) architectures with ReLU activations in the hidden layers and a linear activation in the output layer. This restricts the applicability of the proposed method in practical, real-world settings where more complex architectures are prevalent. \n2.\tSince the attack depends on ReLU linearity and critical points, it's unclear whether this defense would hold under other activation functions like GELU or sigmoid.\n3.\tIt is unclear whether the proposed defense can generalize to more widely used models such as convolutional neural networks (CNNs), transformers, or other real-world deep learning systems. \n4.\tThe paper also refers to the use of \"random datasets\" but does not clearly explain what does mean. Moreover, the experimental evaluation is on MNIST dataset. Including results on more challenging and diverse datasets, such as CIFAR-10, would better demonstrate the effectiveness and robustness of the proposed defense.\n\nQUESTIONS:\n1\tHow would this method generalize to architectures using batch normalization, dropout, or skip connections?\n2\tCan this strategy be extended to protect CNNs or transformer-based architectures?\n3\tAlthough the authors mention that experiments were conducted on a computing cluster, they do not provide specific details about the computational resources used, such as hardware specifications, number of nodes, or training time. Including this information is important for assessing the method's scalability and reproducibility.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}