CoreGuard: Safeguarding Foundational Capabilities of LLMs Against Model Stealing in Edge Deployment
Authors: Qinfeng Li
Tianyue Luo
Xuhong Zhang
Yangfan Xie
Zhiqiang Shen
Lijun Zhang
Yier Jin
Hao Peng
Xinkui Zhao
XianWei Zhu
Jianwei Yin
Venue: NeurIPS 2025 poster ()
Keywords: Intellectual Property Protection
Large Language Model
Model Stealing
Proactive Defense
Trusted Execution Environment
Primary area: social_and_economic_aspects_of_machine_learning

Abstract: Proprietary large language models (LLMs) exhibit strong generalization capabilities across diverse tasks and are increasingly deployed on edge devices for efficiency and privacy reasons. However, deploying proprietary LLMs at the edge without adequate protection introduces critical security threats. Attackers can extract model weights and architectures, enabling unauthorized copying and misuse. Even when protective measures prevent full extraction of model weights, attackers may still perform advanced attacks, such as fine-tuning, to further exploit the model. Existing defenses against these threats typically incur significant computational and communication overhead, making them impractical for edge deployment.
To safeguard the edge-deployed LLMs, we introduce CoreGuard, a computation- and communication-efficient protection method. CoreGuard employs an efficient protection protocol to reduce computational overhead and minimize communication overhead via a propagation protocol. Extensive experiments show that CoreGuard achieves upper-bound security protection with negligible overhead.

CoreGuard: Safeguarding Foundational Capabilities of LLMs Against Model Stealing in Edge Deployment
Qinfeng Li
Tianyue Luo! Xuhong Zhang 1.3+ Yangfan Xie
Zhiqiang Shen
Lijun Zhang
Yier Jin Hao Peng" Xinkui Zhao' Xianwei Zhu?
Jianwei Yin
School of Software Technology, Zhejjang University
"Institute of Software, Chinese Academy of Sciences
"Ningbo Global Innovation Center, Zhejiang University
Washington University in St. Louis
University of Science and Technology of China
College of Computer Science and Technology. Zhejiang Normal University China Electronics Technology Design and Research Institute
Abstract
Proprietary large language models (LLMs) exhibit strong generalization capabilities across diverse tasks and are increasingly deployed on edge devices for efficiency and privacy reasons. However, deploying proprietary LLMs at the edge without adequate protection introduces critical security threats. Attackers can extract model weights and architectures, enabling unauthorized copying and misuse. Even when protective measures prevent full extraction of model weights, attackers may still perform advanced attacks, such as fine-tuning. Existing defenses against these threats typically incur significant computational and communication overhead, making them impractical for edge deployment. To safeguard the edge-deployed LLMs, we introduce CoreGuard, a computation- and communication-effie effficcenntii protection method. CoreGuard employs an efficient protection protocol to reduce computational overhead and minimize communication overhead via a propagation protocol. Extensive experiments show that CoreGuard achieves upper-bound security protection with negligible overhead.
further exploit the model.
Introduction
Large language models (LLMs), especially proprictary ones, such ChatGPT 28 and Claude 44. demonstrate exceptional generalization ability across various tasks [6 31] Additionally, deploying LLMs on edge devices is a growing trend for latency- and privacy-sensitive tasks, e.g., Apple Inc. introduced Apple Intelligence, which integrates 3-billion-parmmeee parameter LLM into users' devices in the latest iOS version [5) However, when these proprietary LLMs are deployed edge devices without adequate protection, adversaries can extract detailed model information (including architecture and weights) through software analysis techniques 7 (45), leading to unauthorized copying and misuse outside the intended device. Even if some protections prevent attackers from fully extracting the original weights, attackers can still perform more advanced attacks, such as fine-tuning the partially recovered model to exploit its embedded knowledge and strong generalization capabilities for new tasks. We refer to this threat as foundational capability stealing. This threat is especially practical for proprictary,
LLMs trained on private data, like BloombergGPT [46] in finance or Med-PaL.M [36] healthcare, where comparable open- alternatives are limited. Considering the substantial resources required to develop perforrrffre LLMs 4] it is crucial to ensure robust protection against these threats in edge deployments.
Corresponding zhangxuhonge wu..edu.ccm
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Table 1: Comparison with existing solutions. VIx indicate whether each property is satisfied.
Solutions (exemplar)
Proactivity
Runtime Security
Backbone Protection
Sufficiency
Efficiency
Watermarking 1
X
X
X
Model encryption
X
X
TPTE [ST
X
PPTE 24
V
X
X
PSP
CoreGuard (ours)
Unfortunately, as shown in Table T traditional solutions struggle protect the edge-deployed LLMs. Specifically, passive protection methods, such as watermark [1 [1] 21], are not applicable since only the proof of ownership insufficient in such an unsupervised edge operation scenario, where attackers can misuse the model without detection. In contrast, active protection works by allowing only authorized users to use the perl-performed model. For example, some work encrypts models 22 and these models are only decrypted before execution.
before deploying them on devices 52 However, it's crucial recognize that while these solutions can implement effective protection before the inference state, current studies 7 45 suggest that, even after decryption, models remain susceptible to runtime attacks during inference, i.e., attackers reverse engineer the models in their runtime state.
To defend against runtime attacks, one potential solution 25 is place the model into a secure execution environment, c.g., trusted execution environment (TEE), which are typically implemented as CPU-based enclave (e.g.. ARM TrusiZone, Intel SGX) that stores sensitive data and safeguards against runtime attacks. However, directly placing the entire model within a TEE is impractical, as it results in approximately 50x reduction model efficiency due to the TEE's limited computational speed 43 Thus, some researchers propose only putting the most critical parameters in TEES and offloading the rest computation to GPUs. For example, Zhang et al. [51] protect only task related adapters within TEE and offload the model backbone to GPUs. However, such approaches are primarily effective for traditional spec-fe models and are insufficient for protecting LLMs, as they leave the model backbone directly exposed.
To protect the backbone, straightforward idea 24 35
to place subset of the backbone (c.g.. the last layer) in the TEE, i.c., Partial Parameter TEE Execution (PPTE). However, prior work [5]] shows that PPTE only provides insufficient protection, and this limitation becomes even more critical when applied to LLMs. Specifically, PPTES crudely execute weights in TEE for protection, where the limited computational power of TEEs restricts the number of protected weights. The scarcity of protected weights makes it easy for attackers to reconstruct them, even with just 1 of the training datasets, compromising security [1]. Even worse, if an attacker aims to exploit LLM'' foundational capabilities, their target task is likely one for which they already have abundant labeled data (e.g.. 100% training set), making theft easier.
To increase the number of protected weights, a promising approach is to protect weights through shuffling, i.c., Parameter Shuffling Protection (PSP) [38] 20). For example, ShadowNet (38) protects model weights by shuffling the channels of convolutional kernels. The protection ensures that only the corresponding shuffled input can be correctly computed with the shuffled weights. This input- shuffling process is performed within the TEE, thus ensuring its security. However, the excessive data transfer overhead between the TEE and GPU makes ShadowNet impractical for LLMs. Specifically, each shuffled layer requires transferring its input from the GPU to the TEE and back, resulting in 448 TEE-CPU transfers for a LLLMMA-8B model with 224 linear layers (each linear layer requires transfers) to generate a single token. Therefore, with an input of 128 context length, each transfer would average 3MB of data (assuming float-32 precision), leading to a total data transfer volume of approximately 1.3GB (448 x 3MB). Given that mainstream mobile platform TEEs, like TrustZone, have a transfer rate of about IGB/s between TEE and GPU [3] generating a single token takes about 1.3 seconds. Consequently, producing complete output would require several hundred seconds (assuming it consists of 100 tokens) solely for data TEE-GPU transfer.
In summary, maintaining acceptable computation and communication overhead under sufficient security of LLM in edge deployment an unresolved challenge for existing solutions. To address this, we propose CoreGuard, a computation- and communication-effiiieee approach designed to prevent the model from working without the proper authorization from the trusted hardware, i.e., TEE, within the edge device. To reduce TEE execution overhead, CoreGuard is inspired by prior

---

PSP solutions by securing parameters through obfuscation, which allows model computations to be performed on the GPU. Specifically, it employs protection protocol that row- row-permutes the weight matrices of linear layers. ensuring their input features must be correspondingly column-permuted (i.e., authorization) by THE. Crucially, to avoid requiring THE authorization for each linear layer and minimize TEE-GPU transfer overhead, CoreGuard introduces propagation protocol that reduces TEE authorizations to a single initial authorization. After this, all subsequent protected layers apply column permutations to their outputs, enabling the initial authorization to be propagated.
Our evaluation shows that CoreGuard outperforms existing defenses in security and efficiency. Besides, the experimental results show no difference in accuracy between the CoreGuard- model and the original model. The contributions of this work are as follows:
We are the first to address the protection of foundational capabilities in edge-deployed LLMs. Our work systematically characterizes the security challenges in this setting and identifies the requirements for the protection of edge-deployed LLMs.
We propose CoreGuard, a allggand-ugg and-play solution that utilizes lightweight authorization mechanism to protect edge-deployed LLMs. It employs a propagation protocol, significantly reducing transfer overhead while maintaining low computation overhead.
Extensive experiments demonstrate that compared to the existing solutions, CoreGuard offers a higher security guarantee with lower overhead and no accuracy loss.
2
Threat Model
In this paper, we consider two parties: the defender and the attacker. The defender is the party that owns the edge-deployed model. The attacker aims to steal the model.
Defender's Goal. The defender aims to deploy a locked model on the device, ensuring it works only with proper authorization from the trusted hardware (i.e., TEE) within the device. The defender can control its model and modify it to ensure protection. This protection ensures that, when correctly authorized, the model permits normal queries from authorized users, while other attacks based on these legitimate queries (e.g., distillation attacks) are orthogonal to our work.
Adversary's Goal. The attacker aims to abuse the deployed model off-device (i.c., without TEE authorization) for their task. A straightforward way is try to reverse the authorization process so that the locked model can be used independently of the device. Another more practical new way is to fine-tune the locked model to obtain model that excels a desired task |0.51]
Adversary's 's Capability. The attacker could decide the target task and possess sufficient well-labeled data, whereas prior work often assumes access only 1 1% of the dataset [51]. In this paper, we consider TEE to be a secure world, while other hardware (e.g., GPU and CPU) could be white-box exposed to attackers. Therefore, the attacker can have access to the details, e.g., model architecture and weights, of the locked model outside TEE.
3 Design of CoreGuard
This section presents our proposed protection method, CoreGuard, which utilizes a permutation strategy to address the key requirements outlined in Section T
3.1 Approach Overview
As shown in Figure [ CoreGuard operates in two phases: model locking (before deployment) and inference authorization (post-deployment), In the model locking phase, CoreGuard locks a trained model by applying protection protocol to the weights of linear layers. i.e., swapping rows of the weight matrix. These row permutations act as a lock, rendering the linear layers dysfunctional, thus making the overall model unusable. These locked layers can only function properly with inputs that are correspondingly using a TEE to authorize each locked layer would result in significant TEE-GPU transfer overhead. To address this, CoreGuard proposes propagation protocol, which enables the features to be columm-permutte by the network itself. Specifically, CoreGuard permutes the columns of certain
which essentially acts as authorization. However, directly
3

---

(4) Model Locking
(b) Inference Authorization
Invaad X
TBR viferiee
Picerataiion Fretuod
Figure I: An overview of CoreGuard. (a) Model locking: before deployment, CoreGuard permutes layers in the original model, thus creating locked model. (b) Inference authorization: during inference, the input feature of the permuted layers is authorized, which is integrated within the FFN block of the preceding transformer layer.
layers, thus through these layers' operation, their output features column- column-permuted, which achieve authorization similarly. In this way, the TEE only needs to manage the initial authorization, and the authorization can be propagated to all subsequent layers.
The inference authorization phase, as shown the black dashed box Figure (b), (b), aims to securely perform initial authorization. A naive method directly authorizes the original output in the TEE and returns 2T, but this obviously will leak т via input-output comparisons. One solution to place both Linear B and the add-norm layer inside the TEE, fully hiding but executing Linear B in the TEE introduces high overhead. To reduce cost, we offload linear B to the GPU. However, exposing Linear B's output will leak Therefore, also requires protection, we apply a permutation after encryption, producing m' (i.e., (m + f) Correspondingly, Linear B's weights are pre-permuted by n² to offset Overall, shown in the Figure I (b), the feature enters the TEE twice: first for OTP encryption (encrypting mn to m'', )) and second for performing authorization to produce 2π.
apply OTP noise
Linear B's input Since OTP
3.2 Model Locking
Given a classic transformer model, we describe how to lock a transformer layer within it. We first apply the protection protocol layers involved in input feature projection (e.g., the QKV projection layer) to secure the model. Then, the propagation protocol is applied to layers managing output projection (e.g., the output projection layer in the attention block and the FFN block). Finally, we demonstrate the functionality of the locked transformer layer.
Transformer Layer Formalization. We begin by formalizing standard transformer layer. Let and E Rlxd denote its input and output, where is the sequence length (e.g.. the number of tokens) and d is the model dimension. We define classic transformer layer as function fo BRidd Rixd with weight parameters w. The transformer layer, i.e., (x) = z, is computed as follows:
= zW g. K xWa, V zWe,
//QKV project
QKT
0 softmax
+ M
VWo.
//Attention
Va/h
+ x 1o-x
y = O
+ Br,
1/ //Add & norm
(1)
Jotz
m ReLU(y bm)
/ //FFN input |/FFN output
n = mW,, + bn
+ Myte
z 72 +
B2,
//Add & norm
Jytn
where includes the attention weights W,, W., W.,, and W, E Rdxdd the add-norm weights Yi- 31, 72, and B2 d the FFN weights W., and W,, E Rdddd the bias bn and b, E Rd. the number of attention heads. The mask M E Rdxd is all-zero matrix in the encoder and has negative infinity in its upper right corner in the decoder. Notably, among these layers, we refer to W,, Wk. and W. in the attention block. and W m in the FFN block as input- processronssing layers they process

---

the input feature of their blocks. We refer to W.. Wn, and the add-norm weights in each block as output propuuspoosssing layers as they manage the output of the blocks.
Protection Protocol. To achieve protection, we row-permutate the input-processing layers for protection. These layers process the module's inputs directly, thus they have the ability cause computation failures if the imput is not authorized, leading to incorrect results.
Specifically, let E {0, 1jaxd denote permutation matrix, where Vπ, गх = I, where / is the identity matrix, a property of the permutation matrix. We row-permute the parameters w.
= x We
= = W =
W =
Thus, only the corresponding Solumn-perrutted input can computed with these layers. For instance:
(3)
where input's column permutation (i.e., æ, the authorization) and row permutations (i.e., n². the lock) offsct each other (NT" I), resulting the same computation as the original.
Propagation Protocol. The propagation protocol aims to avoid repeated TEE authorization by allowing each transformer layer to automatically receive authorized input from the previous layer. Specifically, the output of a transformer layer is directly determined by four sutput-processing layers: Wo. W.. and two add-norm layers. Once all their outputs column peluunn-ermmued,, the overall output is column-permutee permuted, thus achieving automatic authorization. Therefore, the problem simplifies pormunn-ermuting the output of a single layer. For instance, illustrated below, column-permuting W, to W,, π transforms its original output n into colunn-permuted output nıt:
n' mW,
mW, क + b nt,
Therefore, to implement propagation protocol, we oolumn-perruttal all oulput-processing layers, ensuring the feature can be re-authorized before exiting the module:
W/ = WO,, d = ,, = 31 W = Wn,, 6, b, 7/2 -2
72T, $/
B2TT. (5)
Locked Transformer Layer Formalization. With the permuted weights (denoted as w'), taking œV as authorized input, its functionality can be described as follows:
Q' Æオオ Wg xW,, Q.
K' अगम T WW xWE K,
V' uuΛ " T ", = xW, V,
Q'K
= softmax
V'W, = oT,
Va/h
o' + Marto'
(6)
y n
+ +1 YT.
Tzxto'
m' ReLU(y/T TW... + bm)) m.
m'W.T bn noT,
n' y Myy+n' + B2 27.
72 O Oytn
Thus, the functionality the locked layer can be represented as for(x') )= zम = =n)(x) valid only when x' = .æ,, thereby preventing unauthorized access (without TT. When the permuted transformer layer receives authorized input (Iπ), its output (Αл) matches the original output (z) with a column permutation, authorizing the next layer. This propagation is consistent across all subsequent layers. Therefore, authorization required only for the first permuted layer.
3.3 Inference Authorization
During the inference, the initial column--ermutated feature (I:) is generated while ensuring the security of this authorization process. As shown in Figure [ TEE encrypts the feature before the FFN block's 'ss output linear layer. After the GPU processes the feature with this layer, it re-enters the
5

---

TEE for decryption. Finally, the add-norm computation is performed. followed by the permutation of the output before it is returned. The detailed descriptions are provided in the following subsections. Encryption. At the beginning, the input linear layer of the FFN block receives as input from the previous layer:
ReLU(yW,. b..)).
Following this, before the output linear layer, the feature is encrypted using an OTP. Additionally, to protect this encryption process from input-output differencing, we introduce positional obfuscation by applying permutation:
m' = mmT pm,
(8)
where is the pad, noise matrix of the same shape as m, and m' is the encrypted permuted feature. The principle of the one-time pad (OTP) (34) ensures that p is different each time, thus even for the same m, mn produced will differ. In this way, the OTP encryption (i.e., p) and the permutation (i.e., т) can conceal each other.
Output Linear Layer. The FFN block's 's output linear layer processes the encrypted feature. However, since m' is permuted, the layer's parameters must be pre-aligned ensure correct computations. Specifically, we correspondingly permute the output linear layer (W,,) before deployment:
W = '' W,,
With the above preparation, during inference, the encrypted feature m' is transferred to GPUs and processed by the permuted output linear layer W
n' = m W = (mo + pm) m² W, + bn + pWn-
(10)
Decryption. If the OTP noise (i.c., pW,) is not eliminated, the network's functionality is com- promised. Specifically, OTP implementation must meet two requirements: 1) conceal both the encryption and decryption; 2) all computations on the feature before decryption are linear. To meet these requirements, n' is transferred to the TEE for decryption:
n" n' - pW, n.
(11)
Notably, following prior work 43 pWn can be conducted by the model provider in offline phase. Both strategies do not increase the overhead of online inference or impede its efficiency (51]
Authorization. Lastly, the decrypted feature is processed by the add-norm layer and permuted to achieve authorization in TEE. The steps are as follows:
n + y Pytn + B2) =
z' = (72 +
(12)
gytn
where z' (i.e., 2T) is the authorized feature, which will be the input feature for the subsequent permuted transformer layer, thus achieving authorized usage. Notably, the TEE only authorizes once for each inference, minimizing communication overhead by limiting TEE-GPU transfers to rounds. Furthermore, it uses only lightweight computations (e.g., matrix addition), ensuring minimal computation overhead.
Authorization Position. The TEE authorization position, which determines the number of layers to be locked, is hyperparameter. CoreGuard sets this position to the midpoint to enhance security, as detailed in Appendix B Specifically, when the authorization point is in the middle, attackers must recover more parameters, increasing the difficulty of theft. Essentially, permutation only maps parameters to new domain without disrupting their functionality, meaning the attacker only needs to recover the missing layers, whether in the original the new domain, to obtain complete model. If the authorization occurs at the beginning or the end, attackers can retrain just one layer to recover, which is similar prompt tuning training classification head. In contrast, placing it in the middle requires the attacker to restore at least half of the parameters, which more difficult.
Security Analysis. Potential attackers might attempt to steal the locked model by recovering the permuted parameters. However, it impossible as the probability of guessing the correct n is 1d!!). In practice, d is typically larger than 512, e.g., d 4096 in LLaMA3-SB [2]

---

Alternatvely, attackers may attempt to recover π by exploiting the TEE's functionality for instance, by trying to crack π from the TEE's inputs and outputs. However, accurately solving λ infeasible, as the problem is ill-posed. Specifically, even with auxiliary information, the task reduces to solving a Learning With Errors (LWF) problem, which is widely regarded as NP-hard (see Appendix (C)
Although solving т exactly is impossible, attackers might try to approximate TEE's functionality to facilitate model stealing. For example, trying to learn a mapping from y to अ to bypass OTP encryption. However, this approach is also ineffective (as shown in Appendix D): the mapping nonlinear, involves a massive number of parameters, and even minor approximation errors can invalidate the result []]]
4 Experiments
In this section, we perform extensive experiments to answer the following research questions (RQs):
RQI: How secure is CoreGuard, and does it effectively protect the foundational capabilities of LLMs? RQ2: How does CoreGuard's computation and communication overhead compare to other defenses? RQ3: Does
CoreGuard sacrifice the accuracy of the model?
4.1 Experimental Settings
Datasets. To evaluate CoreGuard, we assume the attacker attempts to steal the LLM to different target tasks, including four domain- speccii tasks: GSM8K (mathematics) [9]. Spider (code generation) (50).
PubMedQA (medical question answering) [[6., and SQuAD (reading comprehension) [32].
Models. We choose four representative LLMs for validation. Two of them are specifically designed for on-device deployment: models: ChatGLMM-66B332 [I1] and LLaMA3-BB- [42].
1aem2---5B-Intrruuee [49,, Gemma2-2B-i [4)] The other two are larger
Metric. For all tasks, we use accuracy as the metric. Specifically, for GSM8K, a prediction is considered correct when the final answer matches the actual result. For PubMedQA, 3-class classification task is deemed correct it matches the true label. For Spider, we follow the prior work [19] and assess whether the generated query matches the reference query. For SQuAD, align the answer with the reference, as in previous work [48]. To evaluate execution overhead, we Floating Point Operations (FLOPs) as the metric, following the prior research 14]
Implementation Details. We conduct experiments with the Huggingface library [3. For optimiza- tion, we use the widely adopted AdamW [17] optimizer and a linear learning rate scheduler [47]. Same as previous work [47], we report results of the runs that achieve the highest performance, consistent with real-world practices prioritizing the optimal model. All experiments are conducted on NVIDIA A800 GPUs with 80GB of VRAM.
Baselines. We compare our method against comprehensive baselines, including ideal bounds and representative defenses, categorized by their protection principles (TPTE, PPTE, PSP): ! Lower bound: (i) No-shield, where the adversary accesses the unprotected model directly. ② Upper
bound: (i) Black-box, where only model architecture is visible to the attacker, offering the strongest protection. 3 TPTE (task-only protection): (i) NPLO [51] ④ PPTE (partial model protection): (i) TarkneTZ (4); (ii) SOTER 35): (iii) Serdab [2]: (iv) Our baseline, DTE, runs the latter trans- former layers within TEE. 5 PSP (parameter shuffling): (i) ShadowNet 38: (ii) TransLink
(TLG) 220 To adapt these methods to transformer models, we rigorously configure each solution based on its papers. Specifically, for SOTER, TEE randomly shields 20% layers. For Serdab, the TEE shields the first transformer layer. ShadowNet obfuscates all the linear transformation layers. For TarkneTZ, the last transformer layer is put into TEE.
Model Stealing Attack. As discussed in prior work [5T]. we identify finetuning attacks as major threat to edge-protection solutions. The finetuning attack, as detailed in Appendix A is well-defined in prior studies [5]] 20), and involves two main steps. First, the attacker builds surrogate model using available parameters of the target model from unsecured environments. Second, they train this surrogate model with accessible datasets to perform the target task. Notably, due to the generalization ability of L.LSs, attackers can steal the model for any target task, where they may possess sufficient training datasets to achieve stealing. Therefore, we assume the attacker has the entire training dataset (100%), more stringent condition than the I 1% dataset employed in previous research.
7

---

Table 2: Security assessment of CoreGuard in preventing unauthorized direct inference and model stealing attack. For direct inference, we report the authorized ("Auth") and unauthorized ("Unau") usage accuracies (%). For model stealing attacks, we report the attack accuracies (%), lower attack accuracies indicate stronger defense. The last row reports the average attack accuracies of each defense relative to the baseline black- box solutions. The column representing CoreCuard ("Ours") is highlighted in bold.
Dirrct Inference
Medel Sweling Attack]
Usew
Auth
Nowllickd
NPLO
Scudusb
Durkee12
SOTEER
TLG
ShadowNet
DTT
Ours
Black-box
GSMBL
0.00
15
31.5 a3
30, 20.92a131
14
14.96:s
12.50
148
1.34
2.56.01
1.20 2.4
Qwen2 0.50
PuRMdedQA
Spider
gu00.cce
a000
13:40.134
5.50.00
38.00
28.48 1124
30.28
26,50.241
49.00123:
23
12:
26.01
31.50
21.52
47.00.00
3.50.004
4.50uus
5,50
6.00.1.3
3.79.811 (8.1
5000es3
SQuAD>
16.
30.54-135
50.38-:
4242
20
681s00
508
4.42-n11
7.
566600
GSMRK
0.00
30
30.10
40.94
40.50u2a1
31.02
32.62
4.58
1081
4.56.am
3391:316
1.74g00e
Gemma2 2B
PrRMedQA
Spider
10 50
3.52 Loua
69. 00.50=.:3
39.15-1m1
30.00=311
24.00.u
60:00
23
12.8124
10.50
0000-cm
0000m0
700
0.00-ue
0.50-nss
12.00.0.a
0,00-1.0 1:30
6 6.50asts
0100sem
SQuAD
43.1
GSM88
55.05
55,07-234
53.67.>
54.55-271
2.8
043
0:93-mm
ChuGLM3368
P
Spider
35.81
$7.05-200
50,50.200
38.22-077
611
12:00s04s
833
3.44-.44
7.37-4-0
7.0
68.13
68.21
66.28.3.8
86lans
9.42-015
8.98.1
9.15.033
GSMB
13.11
53.07-77a
51.83-291
79..
49.75.22
56laom
4.15
6.00-115
6.22.3m
LLMMA3-80
PuRM:dQA
1.00
29.0
77.00
114
41.73.2m
2m
71.50.15
12.50
68.00-321
9.50.0as
1000104)
12.50.00
11,00
40
SQuAD
0.00::.0
75002-14
75.91
15.20-196
67
92
79.81
e1s)
69. 12
11:94:5
9.64
10-48-00
10.01-0
Relative Mean Accuracy
0.58%
9.50x
8.46x
8.63x
5.00x
100x
1
L17x
100x
4.2 Security Evaluation
Security against Unauthorized Usage. In this subsection, we assess CoreGuard's security against unauthorized usage. We first evaluate its ability to prevent direct unauthorized inference. Then, we assess its security against MS attacks. Table 2 reports the results: in all cases, the unauthorized inference ("Unau") accuracy is 0%, demonstrating CoreGuard's strong resistance to unauthorized inference. In terms of MS attacks, the security of CoreGuard is comparable to the upper bound, indicating that attackers cannot misuse the foundational capabilities of the protected model for downstream tasks. Specifically, CoreGuard's s relative accuracy is 1.17x compared to the black-box baseline, benefiting from the effective protection provided by our proposed permutation protocal. specifically, the relative accuracy of CoreGuard (1.17x) is similar to that of DTE (1.18x), which protects the same parameters directly using TEE. This suggests that permutation offers similar security to the strongest protection (i.e., direct protection by TEE). Regarding other methods, the TPTE solution, NPLO (9.59x) ) offers no defense (no-shield is 9.58x), and PPTE solutions, e.g., DarkneTZ (6.43x), only provide weak protection. contrast, PSP methods, TLG (1.07x) and ShadowNet (1.09x), also reach the upper bound.
Security under Other Attack Settings.
In this subsection, we evaluate Core- Guard's security under various attack set- tings. Specifically, first, prior evaluations focus on FFT for training. To test Core- Guard under different MS training meth- ods, we assess its security against MS using LoRA, a most commonly used LLM fine- tuning approach. Second, previous eval- uations take base models as the deployed model, but in real-world scenarios, LLMs may also be customized, which may align with or differ from the attacker's 's target, which could affect the defense. Third, while CoreGuard approaches the upper bound with the entire dataset, attackers may sometimes only have small portion of the data. To verify whether it can still approach the upper bound in such cases, where the upper bound is also lower, we assess the defense with varying training data, ranging from 1% to 100%. As shown in Figure 2 we report the attack accuracies under various settings on Qwen2-0.5B, and CoreGuard consistently ensures the model's security across all these settings. The attack accuracies stay close to black-box protection and significantly lower than no-shield cases, regardless of whether the task is aligned, the proportion of data the attacker possesses, or the training method used.
No- No-shield
FFT-stealing
LoRA-stealing
Black: box
Model's Ongiiall Task
Accuracy))) Accuracy())
15
ciSMaK Spider
10
3% 5% 10% 20% 40% 60% 10.1000
5% 30%
20%
40% t0's 00% 100%
20% 40% 65% 1001 5/6 22% 40% 100%
Spider (6k samples total)
GSMUK 17k samples total)
Attacker's s Target Task
Figure 2: CoreGuard's Defense Effectiveness Against Model Stealing Across Various Attack Settings.
Answer to RQI: CoreGuard effectively prevents cdge-deployed LLMs from being off device misused, blocking model stealing attacks and achieving upper-bound security across diverse attack settings.
8

---

Table 3: The results of additional overhead. For execution overhead, we present the original model's FLOPPs ("Original"", the additional overhead in TEE (FIOOPs), and its proportion to the original model's FL.OPs (K))OPs) For transfer overhead, we report the transfer volume (K B) and the number of transfers (rounds) between the TEE and GPU for each method.
Models
TEE Exceution Overhedd
TEE-GPU Transfer Overhead (reamadds
Original
Soniah
DukeeTZ SOTER
11.G
ShadowNet
DIE
Ouis
Seria
DockneZZ
SUTER
TLG
ShadowNet
DTE
Outs
Quee2
1.27
3.22
(3029)
3.82E409
(1.09)
(2.79-4256)
354
(2897%)
3.7E-18
(35.255))
4.58B410
(1.176499)
1,77x06
224E-02
2
L:2E.02
3
5.500004
67
1428:66
115
2755.005
3.6
2.344602
6.88
Gemmaz
6.00+11
199
(2.98%)
1.95410
(2.98%)
(21522))
14
(1.14.42%)
7477-447
(28239)
LMME-11
(28.229)
2.198411
(441c-04%)
3/568 468
125
364
5 5.708402
1880444
CHIGLM 1.534122
3238:10
3118411
4M68-11
7:318:11
A
175550
5511:005
LI3R-066
2.191004
ULMAS
1.93+122 (2,91%)
(2.91))
(20026) (7.8e-83%)
(2615%)
(46,
(2.8e-00%%
6.303:46
408
Table 4: The accuracy comparison between the original model (Mort) and the CoreGuaad-proteett model
(Mine). The result is presented as MordMMoe- Cells showing changes in accuracy are highlighted in bold.
GSM8K
Spider
PubMeelQA
SQuAD
Qwan2
5.56RV5 56%
15.50%/155 50%
16.508//66550/
Gemman
30.10%/30 10%
5.519/3.51%
10. 10.509/1D.500
43.21%/43.211
ChaIGLM3
37. 37.13%6237.139
5.15%/55.15
46. 26.00%A46.009
00%
62.11% .2.00%
LLAMA3
33.11%
10.67%/10.67%
29.00%/28.50%
73.02%/73.011%
4.3 Execution and Transfer Overhead
To answer RQ2, we measure both TEE execution and TEE-GPU transfer overheads to assess Core- Guard's efficiency in computation and communication. Specifically, we take an example length of 128 as input and report the TEE execution and data transfer overhead of each solution to generate single token, excluding TPTE, which offers no protection. All results are reported in Table and CoreGuard demonstrates clear advantage. Specifically, compared to PPTE solutions, CoreGuard achieves thousands of times lower TEE exccution overheads. Specifically, the CoreGuard's s exccution overhead is less than 1.17c-03% in all cases, whereas PPTE incur execution overheads ranging from 2.91% % to 21.52%. More importantly, compared to existing PSP solutions (highlighted in bold), CoreGuard's transfer overhead is nearly two orders of magnitude lower due to its communication-frieen frienddly design. Specifically, as mentioned in Section 3.3 CoreGuard requires only a single authorization, which is optimal, limiting the transfer rounds to In this way. CoreGuard cuts the unacceptable overhead (seconds per token) of existing PSP solutions by two orders of magnitude to negligible levels.
Answer to RQ2: CoreGuard's computation and communication overheads are significantly lower than other solutions, showcasing substantial efficiency advantage.
4.4 Accuracy Loss
To answer RQ3, we compare the accuracy between the unprotected model Mors and the CoreGuard- protected model Mose The result is shown in Table As demonstrated, the impact of CoreGuard on accuracy is minimal. Specifically, in most cases, there is no difference in accuracy between Mars and Mioe However, for some specific cases, accuracy slightly fluctuates (highlighted in bold). For example, with LLaMA3 on PubMedQA, accuracy decreases slightly by 0.5%. However, interestingly, we observe a 0.02% improvement on GSM8k. Therefore, we consider the minor fluctuations caused by precision limitations (e.g.. floating-poii point errors) rather than the defense itself, which is inevitable.
Answer to RQ3: While significantly outperforming existing defenses in terms of both security and efficiency, CoreGuard maintains the model's 's accuracy without compromise.
5 Limitation and Discussion
Side Channel Attacks. CoreGuard uses TEEs as its security root, making it vulnerable to side- channel attacks (37 21. However, various defense methods have emerged in recent years to mitigate the risk of side-channel leaks, and both these software- [2] and hardware-basee [TO 44 defenses can be integrated into our approach. For software- based defense, CoreGuard uses TEE only for basic matrix operations (e.g.. matrix permutation and addition). For hardware-basee defense, CoreGuard does not require modifications to hardware, allowing physical defense measures to be compatible.
TEE in GPUs. Recent work has explored implementing trusted environments directly within GPUs (26.. However, such GPU/NPU TEES are still in their early stage and mainly target datacenter settings requiring high-end hardware 27,, making them impractical for current edge deployment. as Orthogonal to these solutions, CoreGuard instead focuses on broadly available edge

---

smartphones and personal computers -where TEEs are typically CPU-based (e.g., ARM TrustZone, Intel SGX).
This paper focuses on the core framework, without delving into device- specific implementations. However, this limitation does not affect the general applicability of our approach. CoreGuard is designed to be aardosar-agnostic in both implementation and evaluation. In practice, the TEE masks features, performs permutation, and returns the authorized features, relying only on basic TEE functions like matrix operations and data storage, which are universally supported. Additionally, our evaluation is Hlatform-agnstii and suitable across different platforms.
Real-World Environments.
Architecture Compatibilityy CoreGuard is compatible with mainstream transformer architectures, including LLaMA variants and models with Mixture-of-Expere (MoE) design. For MoE-based transformers, CoreGuard protects each expert independently using the same method applied to standard FFNs. The gating mechanism, typically a linear projection, also supports permutation.
6 Conclusions
In this paper, we presents CoreGuard, a protection method that uses permutation strategy to secure edge-deployed models with maximum security. Importantly, to reduce transfer overhead during authorization, CoreGuard proposes propagation protocol, thus only a single authorization is required to authorize the entire model, which is optimal, Experimental results show that CoreGuard delivers superior security and efficiency without accuracy loss. In conclusion, CoreGuard is effective solution that provides model owners with the means to safeguard their proprictary LLMs.
Acknowledgements
This work was sponsored by CCF-Huawei Populus Grove Fund. This work was also supported by the Key Project of the National Natural Science Foundation of China under Grant no. 62536007, the Zhejiang Province Science Foundation under Grant no. LD24F020002 and the Zhejiang Province's 2025 "Leading Goose X" Science and Technology Plan under Grant 2025C02034.
References
[1] Yossi Adi, Carsten Baum, Moustapha Cisse, Benny Pinkas, and Joseph Keshet. Turning your weakness into strength: Watermarking deep neural networks by backdooring. In 27th USENIX Security Symposium (USENIX Security 18), pages 1615-1631, 2018.
[2] AKM Mubashwir Alam and Keke Chen. Making your program oblivious: comparative study for side-channel-safe confidential computing. In 2023 IEEE 16th International Conference on Cloud Computing (CLOUD), pages 282-289. IEEE, 2023.
[3] Tiago Alves. Trustzone: Integrated hardware and software security. Information Quarterly, 3:18-24 2004.
[4] Anthropic. Claude: An ai assistant by anthropic. https:
index http:/itttooiinnnnnnn ceneeinggglllueee claude 2023, Accessed: 2025-05-12.
[5] Apple Inc. gine.
Deploying transformers
the apple neural
https://machhineeleennnn.... apple com/ wtt.://w//eeeeeee research/ htpp::/e..nellllleeeeeeoooooolee [2025.05.08].
2025. Accessed:
[6] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901. 2020.
[7] Ying Cao, Ruigang Liang, Kai Chen, and Peiwei Hu. Boosting neural networks to decom- pile optimized binaries. In Proceedings of the 38th Annual Computer Security Applications Conference, pages 508-518, 2022.
10

---

[8] Abhishek Chakraborty. Ankit Mondai, and Ankur Srivastava. Hardware-assistee intellectual property protection of deep learning models. In 2020 57th ACM/IEEE Design Automation Conference (DAC), pages 1-6. IEFE, 2020.
[9] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Rejichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint 0rXiv:2110.14100 2021.
[10] Ghada Dessouky, Tommaso Frassetto, and Ahmad-Reza Sadeghi. (HybCache): Hybrid (Side- Channel-Resilient) caches for trusted execution environments. In 29th USENIX Security Symposium (USENIX Security 20), pages 451-468, 2020.
[11] Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. Glm: General language model pretraining with autoregressive blank infilling, arXiv preprint arXiv:2103. 10360, 2021.
[12] Tarek Elgamal and Klara Nahrstedt. Serdab: An iot framework for partitioning neural networks computation across multiple enclaves. In 2020 20th IEEE/ACM International Symposium Cluster, Cloud and Internel Computing (CCGRID), pages 519-528. IEEE, 2020.
[13] Hugging Face. Transformers documentation, 2025, Accessed: 2025-05-05.
[14] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[15] Hengrui Jia, Christopher A Choquette-Choo, Varun Chandrasekaran, and Nicolas Papernot. En- tangled watermarks as a defense against model extraction. In 30th USENIX Security Symposium (USENIX Security 21), pages 1937-1954, 2021.
[16] Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William W Cohen, and Xinghua Lu. Pubmedqa: A dataset for biomedical research question answering. arXiv preprint arXiv: (9990066966 2019.
[17] Diederik P Kingma. Adam: A method for stochastic optimization.
arXiv preprint
arXiv:1412.690 2014.
[18] Paul Leignac. Olivier Potin, rean-Baptiste Rigaud, Jean-Max Dutertre, and Simon Pontié. Comparison of side-channel leakage rich and trusted execution environments. In Proceedings of the Sixth Workshop on Cryptography and Security in Computing Systems, pages 19-22, 2019.
[19] Jinyang Li, Binyuan Hui, Reynold Cheng, Bowen Qin, Chenhao Ma, Nan Huo, Fei Huang, Wenyu Du, Luo Si, and Yongbin Li. Graphix-t5: Mixing pre-trained transformers with graph- aware layers for text-to-sql parsing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 13076-13084, 2023.
[20] Qinfeng Li, Zhiqiang Shen, Zhenghan Qin, Yangfan Xie, Xuhong Zhang, Tianyu Du, and Jianwei Yin. Translinkguard: Safeguarding transformer models against model stealing in edge deployment. arXiv preprint arXiv:2404.1 121, 2024.
[21] Gang Liu, Ruotong Xiang, Jing Liu, Rong Pan, and Ziyi Zhang. An invisible and robust watermarking scheme using convolutional neural networks. Expert Systems with Applications, 210:118529, 2022.
[22] Chun Shien Lu. Steganography and digital watermarking techniques for protection of intellectual property. Multimedia Security, Idea Group Publishing, Singapore, pages 75-157, 2005.
[23] Tengchao Ma, Changqiao Xu, Qingzhao An, Xiaohui Kuang, Lujie Zhong, and Luigi Alfredo Grieco. A proactive defense strategy against sgx side-channel attacks via self-checking drl in the cloud. In ICC 2022-IEEE International Conference Communications, pages 4174-4179.
IEEE, 2022.
[24] Fan Mo, Ali Shahin Shamsabadi, Kleomenis Katevas, Soteris Demetriou, Ilias Leontiadis, Andrea Cavallaro, and Hamed Haddadi. Darknetz: towards model privacy at the edge using trusted execution environments. In Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services, pages 161-174, 2020.
11

---

[25] Tsunato Nakai, Daisuke Suzuki, and Takeshi Fujino. Towards trained model confidentiality and integrity using trusted execution environments. Applied Cryptography and Network Security Workshops: ACNS 2021 Satellite Workshops, AlBlock, AIHWS, AloTS, CIMSS, Cloud S&P, SCI, SecMT, and SiMLA, Kamakura, Japan, June 21 21-24, 2021, Proceedings, pages 151-168. Springer, 2021.
[26] NVIDIA.
data Nvidia confidential computing.
https:/ www wwwdia.com en-us/
2024.
Accessed:
[2024.10.11].
[27] NVIDIA. Nvidia h100 tensor core gpu.
data-center/hhon 2024. Accessed: [2024.3.18].
[28] OpenAL. Chatgpt: Optimizing language models for dialogue. https:/ com/ chatgpt 2023. Accessed: 2025-05-12.
[29]
Tribhuvanesh Orekondy, Bernt Schicle, and Mario Fritz. Knockoff nets: Stealing functionality of black-box models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4954-4963, 2019.
[30] Soham Pal. Yash Gupta, Aditya Shukla, Aditya Kanade. Shirish Shevade, and Vinod Ganapathy, Activethief: Model extraction using active learning and unannotated public data. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 865 872, 2020.
[31] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, (8):9, 2019.
[32] Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint 16X.056606005250,, 2016.
[33] Oded Regev. On lattices, learning with errors, random linear codes, and eryptography. Journal of the ACM (JACM), 56(6):1 1-0, 2009.
[34] Claude E Shannon. Communication theory of secrecy systems. The Bell system technical journal, 28(4):656-715, 1949.
[35] Tianxiang Shen, Ji Qi. Jianyu Jiang, Xian Wang, Siyuan Wen, Xusheng Chen, Shixiong Zhao, Sen Wang, Li Chen, Xiapu Luo. et al. {SOTER): Guarding black-box inference for general neural networks at the edge. In 2022 USENIX Annual Technical Conference (USENIX ATC 22),
pages 723-738, 2022.
[36] Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Mohamed Amin, Le Hou, Kevin Clark, Stephen R Pfohl, Heather Cole-Lewis, et al. Toward expert-level medical question answering with large language models. Nature Medicine, pages 1-8, 2025.
[37] Petr Socha, Vojtech Miškovský, and Martin Novotný. A comprehensive survey on the non- invasive passive side-channel analysis. Sensors, 22(21):8096, 2022.
[38] Zhichuang Sun, Ruimin Sun, Changming Liu, Amrita Roy Chowdhury, Long Lu, and Somesh Jha. Shadownet: secure and efficient on-device model inference system for convolutional neural networks. In 2023 IEEE Symposium Security and Privacy (SP), pages 1596-1612. IEEE, 2023.
[39] Mingxing Tan and Quoc Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning, pages 6105-6114, PMLR, 2019.
[40] Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, et al. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.0822 00295 2024.
[41] Timothy Prickett Morgan.
Counting the cost of training large lan-
guage models.
https:// http://wwwweexpptaaafaooooo 2002 /12/01/ langggg--eeggggemmmmeelllll 2022. Ac-
cessed: [2024.02.24].
12

---

[42] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint urKinv2307.092866 2023.
[43] Florian Tramer and Dan Boneh. Slalom: Fast, verifiable and private execution of neural networks in trusted hardware. arXiv preprint arXiv:1806.03280 03287, 2018.
[44] Wubing Wang, Mengyuan Li, Yinqian Zhang, and Zhiqiang Lin. Pwrleak: Exploiting power reporting interface for side-channel attacks amd sev. In International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment, pages 46-66. Springer, 2023.
[45] Ruoyu Wu, Taegyu Kim, Dave Jing Tian, Antonio Bianchi, and Dongyan Xu. (DnD); A (Crhst-Architecture) deep neural network decompiler. 31st USENIX Security Symposium (USENIX Security 22), pages 2135-21111 23522112,222 2022.
[46] Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann. Bloomberggpt: large language model for finance. arXiv preprint arXiv:2303. 2023.
[47] Guangxuan Xiao, Ji Lin, and Song Han. Offsite-tuning: Transfer learning without full model. arXiv preprint arXiv:2302.048777 2023.
[48] Ikuya Yamada, Akari Asai, Hiroyuki Shindo, Hideaki Takeda, and Yuji Matsumoto. Luke: Deep contextualized entity representations with entity-aware self-attention. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6442-6454, 2020.
[49] An Yang. Baosong Yang, Binyuan Hui, Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. Qwen2 technical report. arXiv preprint arXiv:2407. 10671, 2024.
[50] Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasumaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, et al. Spider: A large-scale labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. arXiv preprint 1809:v8800088800
2018.
[51] Ziqi Zhang, Chen Gong, Yifeng Cai, Yuanyuan Yuan, Bingyan Liu, Ding Li, Yao Guo, and Xiangqun Chen. No privacy left outside: On the (in-) security of tee-shielded dnn partition for on-device ml. In 2024 IEEE Symposium on Security and Privacy (SP), pages 52-52. IEEE Computer Society, 2023.
[52] Tong Zhou, Yukui Luo, Shaolei Ren, and Xiaolin Xu. Nnsplitter: an active defense solution for dnn model via automated weight obfuscation. In International Conference on Machine Learning, pages 42614 42624. PMLR, 2023.
13

---

A Appendix Detailed Model Stealing Attack
We consider finetuning attacks as a key security challenge of existing protection solutions. The finetuming attack is a commonly used attack model that is widely discussed in previous papers and is a well-defined and reproducible attack that specifically designed for edge-deployed models. This attack typically occurs in two steps. First, the attacker creates surrogate model and fills it with all the available parameters from the non-secure world. Then, the attacker trains this surrogate model with the datasets they have access to, attempting to apply the surrogate model to their target task.
Specifically, the attack consists of two phases: foundational capabilities stealing (Pi) and task- specific adaptation (P2). In P1, to exploit the foundational capabiliitess of the locked model (Mooc), the attack begins by inferring the architecture of Miae through its exposed parts. Following this, replica model, Mrep. is constructed with the same architecture as Mioc. Finally, the attacker transports Mtoe's 's exposed weights to the corresponding parts of Myep. In P22 the attacker attempts to fine-tune Mrep for their tasks. To this end, one potential approach is to train Mrep with the training dataset the attacker possesses. Specifically, We assume the attacker has access to the entire training dataset, making this scenario more challenging than previous works (29, 30 [5]], which assume attackers have access to only a small portion (e.g., 1%) of the data. In the training, we mainly consider a more comprehensive and effective method, namely full-parameter training (FFT). However, to ensure comprehensiveness, we also consider other training settings, such as LoRA.
B Appendix Authorization Position
Security under Different Authorization Position. The position of authorization is a hyper- Sarrae-prraeeter of CoreGuard, and its selection can influence the overall security. To identify the best authorization position, we examine how different authorization positions impact security. Specifically, we place the authorization before various transformer layers and evaluate their security based on model-stealing attacks.
SQUAD
PubMeddA
17 Gam mam (26 lapersi
Owen2 (24 ayers)
17 ChatSLoT layen)
13 17
25
Figure 3: Impact of authorization position on security. Model-stealing accuracy is reported for different positions. with the total number of transformer layers indicated for each model.
As shown in Figure 3 the results demonstrate that placing the initial authorization point in the middle of the network is practical. Specifically, the model stealing accuracy is higher when the authorization placed near the beginning or end of the network. Conversely, when the authorization deployed within the central layers of the network, the model-stealing accuracy significantly decreases. This aligns with our expectations: applying authorization at the first layer means CoreGuard permutes nearly all the transformer layers, which means the attacker only needs to recover the functionality of the first transformer layer. In contrast, placing authorization in the middle leaves at least half of the parameters unaligned, requiring the attacker to recover the functionality of at least half of the network's 's parameters. Thus, placing the authorization in the central layers is an adequate strategy.
C Appendix / Analysis of Neural Network Fitting Attack
The attacker knows that the input and output are y, m, m.', nn,, n2. Since Linear A is public, it is meaningless to attack the neural network fitting from y to m. In addition, it meaningless to use mi as input to attack the neural network fitting because m' itself is noisy data. Although m is the same, m' is likely to different. Therefore, the truly effective input and output of the neural network fitting attack is m, n'' n2.
14

---

The attacker's s training for neural network fitting attack must be less than the amount of pre-training data, otherwise it will be more trouble than gain. However, the fitted parameters W,, & are definitely noisy. Therefore, the process can be specifically expressed as
m(a =c= n' and Normalizzation((((W)) + IT.
(13)
Define wW, as a whole W. Then the attacker can crack and W,,
# W + W.
(14)
In the absence of noise, it impossible for an attacker to crack it from mathematical perspective. This is because this is pathological equation, the number of unknown variables is much greater than the number of equations, and a quadratic equation and nonlinear equation.
From another perspective, the attacker fits infinitely and obtains many W, and म. Even if tell the attacker that the W3 closest the real Wo, it is difficult for the attacker to solve it. This can be expressed as
A(W, - -) #W, +ē= W,
(15)
where e' is error between W,, and Wi. Since floating point numbers have minimum precision, the equation can converted to integers by dividing both sides by this minimum precision. Such an integer equation solution problem is Matrix-Learning With Errors problem.
The Matrix-Learning With Errors (Matrix-LWE) problem is natural generalization of the standard LWE problem. It is defined as follows:
Hardness of Matrix-Learning With Errors (Matrix-LWE),
W AW mod q.
(16)
If we regard as the largest positive integer that can be represented by a computer floating point number, then we can directly ignore q. In other words, the attacker's solution to the problem is equivalent to solving the LWE problem.
The goal is to recover the secret â. Matrix-LWE can be viewed as packing & independent instances of the standard LWE problem:
[W + 61 W + 6g ... Wo + cel mod g.
(17)
where each column corresponds to an individual LWE sample with secret म and noise 64
The hardness of Matrix-LWE follows from the hardness of standard LWE. Regev 33 proved that solving LWE on average is at least as hard as solving certain worst-case lattice problems such as the Shortest Independent Vector Problem (SIVP,) and the Gap Shortest Vector Problem (GapSVP,))
Specifically, SIVP,, asks for set of n linearly independent lattice vectors whose maximum length most ty (C), and GapSVP, asks to decide whether the shortest non-zero vector X₃ (C) in lattice L is smaller than a given threshold or larger than ydd
Since these worst-case lattice problems are known to be NP-hard or conjectured to be intractable even for quantum algorithms, Matrix-LWF inherits a strong worst-case to average-case hardness guarantee. In practice, this makes Matrix-LWE reliable foundation for constructing cryptographic primitives, including post-quantum encryption schemes.
For examply, assume attacker obtains approximate estimate W,, of the true parameter matrix Wi satisfying || Wolle < E', and observes the authorized (locked) output W wW, Since is a permutation matrix in our construction (hence || 1), the attacker can only form
W W,, + + with
(W, Wo), ||||| S
This matches the canonical Matrix-LWW form W wW, + e, reducing recovery of to solving a Matrix-LWE instance.
15

---

D Appendix Adaptive Attack
Security against Permutation Matrix Simulation Attack. this subsection, we assess the security of CoreGuard against attackers familiar with the CoreGuard's mechanism and implement attacks accordingly. Specifically, the core of authorization involves the permutation matrix (i.e., x). which the TEE protects. Therefore, an attacker might first attempt to simulate и by initializing substitute permutation matrix with the same shape and training based on the TEE's s input and output to approximate the true Then, the attacker uses the simulated π to mimic the TEE's authorization and fine-tunes the model on their task to complete the attack.
Table 5: Security evaluation of CoreGuard against permutation matrix simulation attack (simulation).
No-Shield
Simudation
Black-boo
Qwen2
PubMedOA
GSMRL
58.00%
21 21.53/% 4.9
5.00%
10:30
Gamman
SQLAD
Spider
39.15%
188 63.96%
0.00F%-
0.00%
GSMMI
$5.99% =2.00
PubMedQA
71.00%
23.349
1.00%-
LLAMA3
SQLAD
Spider
10.041%
75.91% 1-325
3.18% LOGrS
0.22% 2000N
The results are shown in Table 5 the attack is ineffective, even performing worse than the black- box baseline. The outstanding security is due to the targeted design. Specifically, the non-linear nature of the authorization process, which relies simulation. Moreover, CoreGuard requires high precision in the authorization process, where even slight simulation errors can compromise model performance.
significantly increases the difficulty of the
Security against Authorization Simulation Attack. Considering that precisely fitting æ is a challenging task, we consider that attackers might attempt to extend their simulation to include adjacent layers or structures, potentially making the attack more feasible. Specifically, since the TEE and the FFN block jointly achieve the authorization, they can be considered a single umit, which the attacker might attempt to simulate directly. Therefore, the attacker could reconstruct an FFN block structure and train this FFN block based the input and output of the original TEB-authorized FFN block, thereby bypassing the TEE'S s authorization.
Table 6: Security evaluation of CoreGuard against authorization simulation attack (simulation).
No-Shicid
Simadation
Black-box
Qwen2
GSMBK PuhMeelQA
56 000 on we
21 53% a1-00
7.00% 1868
2.72%
5.00% un
1.29% unats
Gumman
SQUAD
Spider
63 9696 L.00
59 15
0.00%
8.51%
0.00%
8.81% 20g
ChaiGLM3
PuhMadQA
71
82.87
0.00Y ADA0N
12.099
LLaMA3
SQUAD
Spider
40.0455 6.1.w
75.919
0.005S ACO0S
0.22% Lnors
As shown in Table 6 the attack is ineffective. In all cases, the attack accuracy similar between the simulation and the black-box baseline but significantly lower than the no-shield baseline. The outstanding defense effectiveness is due to the targeted design. Specifically, CoreGuard disrupts the alignment of the parameters before and after the authorization, making it highly challenging for attackers to simply adjust the FFN recover the compatibility between the sets of parameters.
16

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction of the paper clearly outline the main contributions and the scope of the research.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We claimed the limitation in Section 5
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
17

---

Every theorem stated in the main text is accompanied by complete proof provided in the appendix, with appropriate cross cros-reffrences ensure clarity and rigor.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
The experiments can be reproduced and the paper provides a clear and com- prehensive explanation of the proposed method.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
18

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
Justification:
We will release code as soon as the article is accepted.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.e) cc/
public/
ss ssinisoiicyyyyiy for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https?
httpscccccccccccccc i ies CCodeeoodSuaaee bm ss i oonool oollllii icy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
We claimed the experimental details in Section 4.1
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We report the error margins alongside all experimental results to reflect uncertainty and ensure transparency.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for
example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
19

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We report the GPU used in our experiments as well as its memory capacity.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that NeurlPS Code of Ethics httpahttttteeeeee......////////t
didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the
Answer: [Yes]
Justification:
We have read the code ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- cration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: This work investigates model merging stealing, an emerging form of model theft that may pose serious threat to the open-source model ecosystem.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
20

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
The assets used in this paper are properly credited, and the license and terms are respected.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
21

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paaperswi paperswithcooes com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification:
We did not conduct any research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
22

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification:
The LLM is only used for writing.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: :ectts://oe /upps:/nnewee //n/unrrpps. cc /tc//ooffeeeene///2000 2025/ wwt.eeeer.....eeeeeeeeeeeeeeeeee
LLM) for what should should not be described.
23


Reviews:
[{"review_id": "aidA9qx27l", "reviewer": "Reviewer_EyUj", "summary": "The paper introduces CoreGuard, a lightweight defense for edge‐deployed LLMs against model theft. It uses a one‐time TEE authorization and a permutation‐based “propagation protocol” to lock and unlock Transformer weights with only a few TEE–GPU interactions. Experiments on four models (Qwen2, Gemma2, ChatGLM3, LLaMA3) and four tasks show CoreGuard matches black‐box security limits while incurring two orders of magnitude less overhead and negligible accuracy loss.", "questions": "1.How does CoreGuard scale to very large models (e.g., 13B or 70B)? Are there memory or latency bottlenecks from storing or propagating large permutation matrices?\n2.Have you evaluated or considered integration with side-channel defenses? What is the overhead when combining CoreGuard with techniques like HybCache [1]?\n[1] Dessouky, Ghada, Tommaso Frassetto, and Ahmad-Reza Sadeghi. \"HybCache: Hybrid Side-Channel-Resilient caches for trusted execution environments.\" 29th USENIX Security Symposium (USENIX Security 20). 2020.", "limitations": "I believe this paper proposed a well-designed approach, but it reads more as a systems engineering effort rather than a method tailored to LLM-specific properties, which may not be suitable for Neurips and could be more suitable for a Security conference.", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 1, "originality": 2, "strengths_and_weaknesses": "Strength: \n1.CoreGuard achieves low overhead through a one-time TEE authorization with effective protection propagation, significantly reducing runtime and memory costs. \n2.The authors conduct extensive experiments across multiple LLM architectures to validate its effectiveness. \n3.CoreGuard is architecture-agnostic, lightweight, and deployable without retraining, requiring only minimal TEE support.\n\nWeakness. My review is short, since this paper definitely do not have broad audience in the AI community. I suggest submitting to security conferences such as CCS.\n\n1.The paper lacks substantial theoretical analysis and offers limited novel insights to the AI community. The problem formulation is missing. In section 2, I expect a formal problem formulation, introducing the model, the locked part, what to modify, the architecture or parameters. The section 2 is very poorly written. \n2.It reads more as a systems engineering effort rather than a method tailored to LLM-specific properties, which may not be suitable for Neurips. When I read this paper, I feel more like a MobiSys paper, not an AI paper. I am not sure why this paper is important to the people who work on the general AI safety. The audience is very limited. \n3.The scalability to large models (13B–70B+) is untested, and the security evaluation omits more sophisticated adaptive attacks that jointly recover permutations and OTP noise.", "was_revised": true, "final_justification": "part of my concern has not been addressed. I will keep my score.", "rebuttal": "### **Dear Reviewer,**\n\nThank you very much for your valuable feedback and thoughtful comments. We appreciate the time and effort you have dedicated to reviewing our paper. In the following, we will address each of your concerns point by point.\n\n**Scalability and Bottleneck Analysis of CoreGuard** -- We appreciate the reviewer’s question regarding the scalability of CoreGuard. While our current evaluation focuses on 0.5B–8B models to align with real-world edge deployment scenarios (e.g., Apple Intelligence, Gemma2, Qwen2), CoreGuard is fundamentally designed to scale to much larger models such as 13B or 70B without incurring bottlenecks in memory or latency.\nSpecifically, first, regarding memory usage, it is important to clarify that the permutation matrix never leaves the TEE and does not occupy GPU memory. Additionally, with CoreGuard, only one permutation matrix is needed for the entire model, which is a negligible storage requirement for the TEE.\nRegarding propagation overhead, no costs are incurred during inference, regardless of the model's size. Specifically, the propagation protocol involves only the column permutation of the weight matrices, which is applied before deployment and is not part of the inference process. Therefore, no additional propagation overhead is introduced compared to the original model.\n\n**Compatibility with  Side-Channel Defenses such as HybCache** -- CoreGuard and HybCache can be seamlessly integrated. HybCache is a side-channel defense mechanism designed to prevent cache-based attacks in SGX by periodically and randomly changing how memory pages are mapped within the enclave, making it difficult for attackers to infer sensitive information from access patterns. These measures, both functionally and in terms of efficiency, are fully compatible with CoreGuard. Specifically, from a functional perspective, CoreGuard’s use of the TEE involves a few simple matrix operations, such as applying row and column permutations and basic matrix addition. These operations follow predictable memory access patterns and avoid data-dependent branching, ensuring they do not interfere with HybCache’s memory randomization, which periodically changes memory page mappings within the enclave to prevent cache-based attacks. From an efficiency perspective, CoreGuard’s lightweight matrix operations are computationally efficient, as they involve simple arithmetic. This means CoreGuard’s operations do not introduce additional memory accesses or data transfers, allowing HybCache to perform its memory page randomization without extra overhead. \n\nMoreover, CoreGuard can be integrated with a wide range of side-channel defense mechanisms because it relies only on basic TEE functionality. It does not require specialized memory layouts, privileged instructions, or microarchitectural modifications. Specifically, from a software perspective, CoreGuard uses the TEE only for basic matrix operations, making it easy to integrate with existing side-channel defense algorithms such as constant-time programming and memory access randomization. From a hardware perspective, CoreGuard requires no changes to the TEE’s microarchitecture, enabling compatibility with physical protection techniques such as EM shielding or power noise injection. These properties allow CoreGuard to serve as a complementary layer of defense in diverse threat models.\n\n\n**Security Under Joint Recovery Attacks Against CoreGuard** -- We would like to clarify that joint attacks attempting to recover both permutations and OTP noise represent a very difficult problem.\nSpecifically, in Appendix D, we present our Security against Permutation Matrix Simulation Attack, where we specifically attempt to recover the permutation matrix in isolation. Our results demonstrate that such attacks are unsuccessful, emphasizing the robustness of our approach to individual attacks targeting permutations.\nAdditionally, recovering OTP noise is nearly impossible due to the nature of OTP encryption. Each OTP noise element is unique and used only once, making it infeasible for attackers to deduce the noise matrix through training or other methods.\nMoreover, attempting to recover both jointly is an even larger and more difficult problem. Specifically, we further analyze this infeasibility from a theoretical perspective in Appendix C. We show that any attempt to jointly fit the permutation and the noise leads to a system of equations that is mathematically equivalent to the *Matrix Learning With Errors* (Matrix-LWE) problem—a well-studied extension of the standard LWE problem. Matrix-LWE is widely believed to be computationally intractable, with hardness reductions to NP-hard lattice problems such as the Shortest Independent Vector Problem (SIVP) and the Gap Shortest Vector Problem (GapSVP). Therefore, even with unlimited access to input-output pairs, solving for both $\\pi$ and $p^\\pi$ remains provably hard.\nIf the reviewer has other concrete attack strategies or threat assumptions in mind—such as specific attacker capabilities or attack methods—we would be happy to examine and evaluate them within our framework.\n\n**Clarifying the Contribution of CoreGuard** -- We respectfully clarify that our approach is not merely a systems engineering effort. Rather, it introduces a novel method that deeply intervenes in the inference pathway of LLMs, by modeling and controlling the transformation of internal features through structured authorization and propagation mechanisms.\nSpecifically, CoreGuard is designed to address foundational capability stealing, a threat unique to LLMs due to their strong generalization abilities. To counter this, we propose a semantic-preserving but structurally disruptive transformation on LLM layers (e.g., linear projections, FFN blocks, Add&Norm), which enforces runtime authorization and blocks unauthorized reassembly of the model’s knowledge pipeline.\nThis design goes beyond conventional deployment strategies by reconstructing the LLM’s computation graph in a way that binds its functionality to a trusted enclave, thus preventing misuse of the model’s generalization ability. We believe this contribution is not only technically novel but also methodologically grounded in LLM-specific reasoning patterns and architecture semantics, making it relevant to the NeurIPS audience concerned with LLM behavior modeling, robustness, and secure deployment.\n\n\n**Clarifying the Formal Problem Formulation** --  We recognize that introducing the formal definitions earlier would have better clarified the problem setup and supported the threat model more clearly. In the revised version, we have reorganized Section 2 (now titled *Problem Statement and Threat Model*) to include a formal definition of the model structure, the protected components, and the adversarial objective. Specifically, we define the LLM as a function $f_w: \\mathbb{R}^{l \\times d} \\rightarrow \\mathbb{R}^{l \\times d}$, and identify the input-processing and output-processing layers involved.\nWe explicitly describe how the model is “locked” via permutation of parameter matrices, and how this locking constrains unauthorized usage.\nThe revised section also clarifies what parts of the model are exposed to the attacker (e.g., weights outside the TEE) and what transformations are applied to enforce protection. These definitions align with the attacker’s capabilities and the defender’s goals, and lay the foundation for the mechanism presented in Section 3. \n\nOnce again, thank you for your constructive feedback. We look forward to any further suggestions you may have."}, {"review_id": "mvrNIs4gnH", "reviewer": "Reviewer_rASP", "summary": "The paper introduces CoreGuard — an efficient protection method for preventing model stealing in edge deployment. CoreGuard focuses on active protection against model architecture, weight, and foundational capability stealing attacks.\nCoreGuard implements an efficient variant of parameter shuffling protection methods in trusted execution environment (TEE) which dramatically reduces the TEE-GPU data transfer overhead. Specifically, CoreGuard employs a protection protocol based on row permutations of linear LLM layers (requiring their inputs to be correspondingly column-permuted) and introduces a propagation protocol that reduces TEE authorizations only to a single initial authorization. CoreGuard  achieves security levels close to upper-bound black-box protection outperforming existing methods in terms of both security and computational efficiency. Overall, CoreGuard is novel and the only method so far that delivers protection with proactivity, runtime security, backbone protection, sufficiency and efficiency. Importantly, CoreGuard does not lead to model accuracy degradation and remains secure even if attackers have access to training data.", "questions": "* Can CoreGuard be applied to any neural net? Not necessarily a language model, right?", "limitations": "limitations are discussed", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Overall, while this paper is slightly outside of my expertise, this is one of the most impressive submissions I have seen at NeurIPS.  The method is directly applicable in practice and has endless utility addressing a very important problem. \n\nStrengths:\n* CoreGuard is effectively the first method capable of sufficient protection of LLM models on edge which does not have prohibitive overhead (and also has runtime security through the use of TEEs)\n* CoreGuard far outperforms existing baseline methods ( in terms of both security and computational efficiency, does not lead to model accuracy degradation and remains secure even if attackers have access to training data\n* The paper validates CoreGuard through extensive experimentation across 11 baselines and 16 LLM backbone-task combinations\n* The paper is very well written\n\nWeaknesses:\n* CoreGuard is only as secure as TEEs (however, this is outside the scope of this study and does not diminish the merit of the method itself)\n* Model distillation is a real model stealing concern for edge-deployed LLMs since the attackers may be able to simulate a large number of inputs and outputs without being reliably blocked like in the remote model accessed through API case. While CoreGuard is impressive, it is still vulnerable to model distillation\n* The appendix presents multiple possible attacks on CoreGuard showing they are ineffective even assuming access to 100% training data (for example, Authorization Simulation Attack). The key to this failure seems to be the challenge to reliably recover underlying weight matrices/FFN blocks given that even small errors in weight values could derail LLM accuracy. However, since the model is deployed on edge, the attacker can query it as many times as they want effectively creating unlimited training data for weight recovery and authorization simulation. Can you run an experiment querying the model to create additional data and showing that even 5x the amount of training data is not enough to reliably simulate authorization?\n    * Are there truly secure methods to reliably prevent the attackers from querying the model unlimited number of times?", "was_revised": true, "final_justification": "Keeping my high score", "rebuttal": "### **Dear Reviewer,**\n\nThank you very much for your valuable feedback and thoughtful comments. We appreciate the time and effort you have dedicated to reviewing our paper. In the following, we will address each of your concerns point by point.\n\n**Regarding Black-Box Distillation Attacks** -- We acknowledge that our work does not address black-box distillation attacks, as this constitutes a fundamentally different research topic. CoreGuard focuses on enforcing access control—ensuring the model cannot run without authorization—while assuming that authorized users can access the model normally. In such cases, attackers could indeed collect input-output pairs and perform distillation. \nDefending against distillation attacks is a new problem, typically categorized into passive defenses (e.g., output watermarking, randomness injection) and active defenses (e.g., training-time obfuscation, anti-distillation regularization). These techniques aim to reduce the effectiveness or fidelity of the distilled model but are orthogonal to CoreGuard’s objective.\n\n\n**Does More Training Data Break CoreGuard?** -- Thank you for raising this important point. To evaluate the robustness of CoreGuard under more powerful attackers, we conducted additional experiments targeting both the finetuning attack and the Authorization Simulation Attack.\nTo evaluate the robustness of CoreGuard under more powerful attackers, \nFirst, for the finetuning attack, we conducted an additional experiment where we expanded the original training set by querying the protected model to generate $5\\times$ more data. We then used this enlarged dataset to perform model stealing attacks following the same procedure as in our main experiments. The results show no meaningful improvement in attack accuracy—accuracy remains close to the black-box baseline, indicating that even significantly increased query data does not compromise CoreGuard’s protection.\nNext, for the Authorization Simulation Attack, which attempts to simulate the entire TEE structure, including the FFN block and the permutation matrix inside the TEE, we trained with the enlarged $5\\times$ dataset and ensured that the loss had already converged. The results show that accuracy remains close to the black-box baseline. This is because CoreGuard requires high precision in the authorization process, and even slight simulation errors can compromise the model’s performance. Since the results cannot be displayed on the webpage, all detailed results will be updated in the Appendix.\n\n**On the Existence of a Practical Upper Bound for Defense** -- While we agree that attackers can, in theory, issue an unlimited number of queries, each query incurs real-world computational cost and, when scaled up, can become equivalent to training a new model from scratch—ultimately making the attack economically meaningless. From a defense perspective, CoreGuard is explicitly designed to raise the attack cost close to this upper bound: the cost of training a model from scratch given only the architecture (i.e., the black-box setting, where the attacker knows the model structure but must re-learn all parameters). Achieving such black-box-level protection is both a meaningful and practical target for securing edge-deployed LLMs, as it renders model stealing attacks economically and technically infeasible.\n\n\n**Applicability of CoreGuard to Non-Transformer Models** -- While our implementation focuses on Transformer models, the design of CoreGuard itself is inherently architecture-agnostic, as it builds upon the three core design principles that each independently exhibit compatibility across diverse architectures.\nFirst, from the protection perspective, CoreGuard introduces structure-aware defenses by embedding permutation-based transformations into linear mapping layers, which serve to disrupt potential low-loss merging paths between models. This mechanism targets the mathematical structure of linear layers rather than relying on Transformer-specific components. Since linear transformation layers are ubiquitous in neural networks—including convolution layers in CNNs and fully connected layers in MLPs—our protection approach applies readily to those architectures as well.\nSecond, from the authorization perspective, the generation and injection of both the permutation matrices and OTP noise are handled entirely by the TEE. These operations depend solely on the TEE’s capability to manage secure randomness and authorize parameter access. The architecture of the underlying model does not affect the authorization process, regardless of whether the model is a Transformer, CNN, or any other architecture.\nThird, from the perspective of authorization propagation, CoreGuard ensures that downstream layers can only be unlocked if upstream layers have been correctly authorized. This function operates on the local behavior of each linear transformation layer and does not require any specific structural assumption about the model. Thus, the propagation rule is universally applicable to any architecture that contains linear transformation layers.\n\nWe take a standard CNN like ResNet as an example. CoreGuard protects the convolutional kernels by permuting the input channels of the kernel, effectively locking it. To ensure proper operation, the input feature map must be similarly permuted to match the kernel's new channel order (i.e., it must be authorized). Additionally, the output channels of the convolutional kernel are permuted to ensure that the output channels correspond to the input channels of the next kernel. Once the first layer is authorized with the correct input permutation, the authorization propagates automatically to subsequent layers (i.e., authorization propagation). This approach works seamlessly with nonlinear layers like ReLU, batch normalization (BN), and LeakyReLU, as they only process the values of the elements and are independent of their positions. In this way, CoreGuard provides model parameter protection and ensures the model can only operate under authorization. \n\nOnce again, thank you for your constructive feedback. We look forward to any further suggestions you may have."}, {"review_id": "qc53gUO9mn", "reviewer": "Reviewer_5WEj", "summary": "This paper proposes a method to protect all weights of an LLM with a single TEE operation: the TEE applies a secret permutation (plus a one-time pad) to the first hidden vector, after which the GPU does the heavy lifting while the permutation automatically propagates through the network. The result is full-model protection with minimal TEE involvement and no accuracy loss.", "questions": "See weaknesses", "limitations": "Yes, adequately addressed", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- The method itself makes sense and is interesting: using permutations and one-time pads in the TEE in order to outsource compute-heavy matmuls to the GPU\n- The overhead is smaller in terms of number of transfers required.\n- The problem is important and timely\n\nWeaknesses:\n- On lines 43-45, the authors state \"directly placing the entire model within a TEE is impractical, as it results in approximately a 50× reduction in model efficiency due to the TEE’s limited computational speed [43]\". But the 50x throughput difference in the cited paper comes from an experiment where the TEE was run on a CPU core and the untrusted environment was a GPU. The throughput difference in this paper came from the difference in hardware, not the TEE itself. This is small but worrying misunderstanding, since it undermines a major pillar of the framing of this submission. Nevertheless, the goal of reducing TEE requirements is still valid and could broaden the type of hardware on which secure execution with LLMs can be run.\n- Extending parameter shuffling from CNNs to transformers is valuable, yet conceptually close to ShadowNet\n- The FLOP overhead is not improved compared to prior work, and the transfer overhead may be negligible with modern TEE hardware.", "was_revised": true, "final_justification": "The author response addresses some of my concerns. I'm still worried about the 50x number that is cited, and the authors didn't directly address my concern about the validity of that statistic. I'll increase my score to a 4, since the value of the paper is more clear to me now, but I'll decrease my confidence since I'm not certain in the evaluation.", "rebuttal": "### **Dear Reviewer,**\n\nThank you very much for your valuable feedback and thoughtful comments. We appreciate the time and effort you have dedicated to reviewing our paper. In the following, we will address each of your concerns point by point.\n\n**Real-World TEE Settings** -- We appreciate the reviewer’s insightful comment. We clarify that CoreGuard is designed for broader and more prevalent edge hardware, such as smartphones and personal computers, where TEEs are typically implemented on CPUs (e.g., ARM TrustZone, Intel SGX). These TEEs are widely deployed in commodity devices and represent the dominant form of trusted execution available in edge settings. In contrast, GPU-integrated TEEs—such as those in NVIDIA H100—are designed for cloud or datacenter usage, and are generally inaccessible for most edge deployments due to their high cost, hardware footprint, and specialized infrastructure requirements. Therefore, CoreGuard does not conflict with these server-side TEE solutions but rather serves as a complementary defense tailored for practical, real-world edge scenarios. \n\n**Why Reducing TEE-Induced Overhead Still Matters?** -- In edge deployment scenarios, where efficient computation and minimal latency are crucial, reducing TEE-induced overhead is essential. Specifically, TEEs inherently impose significant limitations on practical model deployment, causing computation and transfer bottlenecks that have been consistently observed in prior work. For instance, Table 7 in NPLO[51] shows that even when shielding only small adapters, 97.06% of total inference time is spent on TEE-related operations—35.61% on TEE-GPU transfer and 61.39% on TEE execution. Methods that attempt to protect model backbones face even worse bottlenecks. For example, ShadowNet incurs 1.3GB/token transfer for LLaMA3-8B due to frequent TEE-GPU interactions (Lines 64–72), making inference impractical. Given that mainstream mobile platform TEEs, like TrustZone, have a transfer rate of about 1GB/s between TEE and GPU [3], generating a single token takes about 1.3 seconds. Consequently, producing a complete output would require several hundred seconds (assuming it consists of 100 tokens) solely for data TEE-GPU transfer. This transfer overhead is a significant bottleneck, making inference impractical.\n\n**Key Distinction from ShadowNet** -- Your concern is understandable. It is true that our method protects parameters through permutation, while ShadowNet uses shuffling, and they share a certain degree of similarity. However, the challenges they face are entirely different. Parameter protection is only the most basic goal of our method. Our main contribution lies in addressing the efficiency problem, which is not a simple “extension,” but the key to whether the scheme is practical.\nSpecifically, ShadowNet still relies on TEE to directly execute model computations; even though it avoids computing linear layers, it still requires the TEE to execute nonlinear layers. Experimental results show that ShadowNet needs to keep about 30% of FLOPs inside the TEE. In contrast, CoreGuard uses the TEE only for authorization, without performing any model computation, reducing the FLOPs inside the TEE to 1e-5 of the original.\nMoreover, ShadowNet requires the feature to be transferred into and out of the TEE before every linear layer, resulting in enormous communication overhead (hundreds of transfers per inference). CoreGuard avoids this problem through one-time authorization and a propagation protocol, reducing the number of TEE transfers to just 5. \n\nOnce again, thank you for your constructive feedback. We look forward to any further suggestions you may have."}, {"review_id": "dUpOxtEqXH", "reviewer": "Reviewer_q37A", "summary": "CoreGuard protects edge-deployed LLMs using a permutation-based locking mechanism. The method applies row permutations to input-processing layers and column permutations to output-processing layers. A TEE provides initial authorization at a middle layer, then the permutation structure propagates this authorization through subsequent layers. This reduces TEE-GPU communication overhead compared to existing methods.", "questions": "1. How do you prevent attackers from using TEE as an oracle during fine-tuning rather than trying to extract permutation matrices?\n\n2. Can you provide more rigorous proof of the LWE hardness connection in Appendix C?\n\n3. How does the method extend beyond transformer architectures?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The topic is important and the idea is interesting and novel, with security analysis provided.\n\n2. The evaluation is solid with four baselines on four tasks, also including an adaptive defense.\n\n\nWeakness:\n\n1. TEE oracle attack: The most practical attack is for adversaries to fine-tune around the TEE by using it as a black box. Attackers can train layers 1-3 and 4+ while treating TEE authorization as just another frozen layer in their training pipeline. The evaluation on the defense effectivess against this attack is beneficial to understand the robustness of the proposed method.\n\n2. Security analysis gaps: The claimed connection to Learning With Errors problem lacks rigorous mathematical proof in Appendix C.\n\n3. Limited architectural scope: Only works for transformer architectures. Unclear how it extends to CNNs or other model types.\n\n4. Presentation issues: Tables 2 and 3 have extremely small, unreadable fonts.", "was_revised": true, "final_justification": "Thank you for the clarification. Your refined threat model definition makes sense - focusing specifically on preventing off-device model extraction and redistribution rather than all forms of on-device misuse.\n\nI understand that \"foundational capability stealing\" in your context refers to extracting the model to run independently without TEE authorization, and that on-device fine-tuning (while remaining TEE-dependent) falls outside your protection scope.\n\nThis clarification resolves my main concern. Your method successfully achieves its stated objective of preventing off-device model theft, and the experimental evaluation appropriately targets this threat. The suggestion of access control mechanisms for limiting query frequency also demonstrates awareness of potential on-device misuse scenarios.\n\nTwo minor suggestions for the final version:\n\n1. **Clarify threat model scope early**: The refined definition of \"foundational capability stealing\" as specifically referring to off-device extraction would help readers understand the protection boundaries from the introduction.\n\n2. **Table readability**: Tables 2 and 3 remain difficult to read due to small font sizes.", "rebuttal": "### **Dear Reviewer,**\n\nThank you very much for your valuable feedback and thoughtful comments. We appreciate the time and effort you have dedicated to reviewing our paper. In the following, we will address each of your concerns point by point.\n\n**On TEE Oracle Attacks** --We fully acknowledge your concern. However, we would like to clarify that attackers cannot perform such a TEE oracle attack on CoreGuard. Specifically, the TEE participates in the forward pass but does not support gradient propagation during backpropagation, ensuring that attackers cannot optimize the model. Second, even if attackers attempt to bypass the TEE by training only part of the model—such as freezing layers before or after the TEE and reinitializing the remaining parameters—this remains infeasible. This is because CoreGuard places the authorization point in the middle of the network, forcing attackers to retrain at least half of the model’s parameters, which is unrealistic for an attacker.\n\n**Rigorous Proof of the Matrix-LWE Hardness Connection** -- We appreciate the reviewer’s feedback and agree that the connection to the Learning With Errors (LWE) problem can benefit from a more rigorous exposition. Below, we provide a refined justification that formalizes the reduction to a Matrix-LWE instance.\n\nAssume the attacker is given:  An approximate estimate $\\hat{W}_b$ of the true matrix $W_b$ , satisfying $\\| \\hat{W}_b - W_b \\|_F \\leq \\varepsilon'$; A known output matrix $W = \\pi W_b$; The recovery target: $\\pi$.\nWhat the attacker can actually construct is:\n$W = \\pi W_b = \\pi \\hat{W}_b + \\pi(W_b - \\hat{W}_b) = \\pi \\hat{W}_b + \\tilde{\\epsilon}$, where: $\\tilde{\\epsilon} := \\pi(W_b - \\hat{W}_b), \\quad \\text{and} \\quad \\| \\tilde{\\epsilon} \\|_F \\leq \\| \\pi \\|_2 \\cdot \\varepsilon'$. In other words, the attacker can only observe: $W = \\pi \\hat{W}_b + \\tilde{\\epsilon}$, which exactly corresponds to the form of a Matrix Learning With Errors (Matrix-LWE) problem.\n\nA formal proof of this reduction will be provided in a subsequent section of the paper.\n\n**Applicability of CoreGuard to Non-Transformer Models** -- While our implementation focuses on Transformer models, the design of CoreGuard itself is inherently architecture-agnostic, as it builds upon the three core design principles that each independently exhibit compatibility across diverse architectures.\nFirst, from the protection perspective, CoreGuard introduces structure-aware defenses by embedding permutation-based transformations into linear mapping layers, which serve to disrupt potential low-loss merging paths between models. This mechanism targets the mathematical structure of linear layers rather than relying on Transformer-specific components. Since linear transformation layers are ubiquitous in neural networks—including convolution layers in CNNs and fully connected layers in MLPs—our protection approach applies readily to those architectures as well.\nSecond, from the authorization perspective, the generation and injection of both the permutation matrices and OTP noise are handled entirely by the TEE. These operations depend solely on the TEE’s capability to manage secure randomness and authorize parameter access. The architecture of the underlying model does not affect the authorization process, regardless of whether the model is a Transformer, CNN, or any other architecture.\nThird, from the perspective of authorization propagation, CoreGuard ensures that downstream layers can only be unlocked if upstream layers have been correctly authorized. This function operates on the local behavior of each linear transformation layer and does not require any specific structural assumption about the model. Thus, the propagation rule is universally applicable to any architecture that contains linear transformation layers.\n\nWe take a standard CNN like ResNet as an example. CoreGuard protects the convolutional kernels by permuting the input channels of the kernel, effectively locking it. To ensure proper operation, the input feature map must be similarly permuted to match the kernel's new channel order (i.e., it must be authorized). Additionally, the output channels of the convolutional kernel are permuted to ensure that the output channels correspond to the input channels of the next kernel. Once the first layer is authorized with the correct input permutation, the authorization propagates automatically to subsequent layers (i.e., authorization propagation). This approach works seamlessly with activation functions like ReLU, as they only process the values of the elements and are independent of their positions. In this way, CoreGuard provides model parameter protection and ensures the model can only operate under authorization. \n\nOnce again, thank you for your constructive feedback. We look forward to any further suggestions you may have."}]

Author rebuttal:

Meta-review: This paper introduces CoreGuard, a lightweight protection method for safeguarding large language models (LLMs) deployed at the edge against model stealing. The key idea is to use a one-time trusted execution environment (TEE) authorization combined with permutation-based propagation, which secures model parameters with minimal computational and communication overhead. Experiments across multiple LLMs and tasks show that CoreGuard achieves near upper-bound protection while maintaining negligible accuracy loss, making it a practical solution for securing proprietary models.

In general, the paper addresses an important and timely problem and proposes a technically novel solution that is efficient, architecture-agnostic, and practically deployable. The method is supported by extensive experimental validation and engages with both theoretical and empirical analysis, earning strong endorsements from several reviewers. However, as pointed out by the reviewers, there are some concerns about the scope of the threat model, the rigor of the theoretical reduction to Matrix-LWE, and open questions about broader applicability and defense against model distillation. Despite these, the authors have provided convincing clarifications in their rebuttal, and the majority of reviewers raised their scores after discussion. Overall, I believe the strengths of this work outweigh the weaknesses, and I think it has a strong potential impact on secure deployment of LLMs.
Decision: Accept (poster)