TransLinkGuard: Safeguarding Transformer Models Against Model Stealing in Edge Deployment

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-03 10:29:21

Abstract: Proprietary large language models (LLMs) have been widely applied in various scenarios. Additionally, deploying LLMs on edge devices is trending for efficiency and privacy reasons. However, edge deployment of proprietary LLMs introduces new security challenges: edge-deployed models are exposed as white-box accessible to users, enabling adversaries to conduct effective model stealing (MS) attacks. Unfortunately, existing defense mechanisms fail to provide effective protection. Specifically, we identify four critical protection properties that existing methods fail to simultaneously satisfy: (1) maintaining protection after a model is physically copied; (2) authorizing model access at request level; (3) safeguarding runtime reverse engineering; (4) achieving high security with negligible runtime overhead. To address the above issues, we propose TransLinkGuard, a plug-and-play model protection approach against model stealing on edge devices. The core part of TransLinkGuard is a lightweight authorization module residing in a secure environment, e.g., TEE. The authorization module can freshly authorize each request based on its input. Extensive experiments show that TransLinkGuard achieves the same security protection as the black-box security guarantees with negligible overhead.

Main paper: 
 background and problem statement 2.1 background: 
 TEE. A Trusted Execution Environment (TEE) is an isolated hardware enclave that stores and processes sensitive data. Popular TEE implementations include Intel SGX [34] , AMD SEV [25] , and TrustZone [2] . In this paper, we follow prior work and deem TEE a secure area on a potential adversary host device (including GPUs). This means the data, code, and computation processes inside TEEs are secure. Although there are side-channel attacks that may leak sensitive data from TEE, they are out of the scope of this paper. PTSE Sloutions. Partial TEE-Shielded Execution (PTSE) solutions aim to provide protection against MS by shielding and executing partial models inside TEEs. The motivation of existing work is to reduce inference latency of the straightforward black-box protection that shields the whole model inside TEEs (latency up to 50× [55] ). Beyond efficiency considerations, the security goal of PTSE solutions is to downgrade white-box MS against edgedeployed models to black-box attacks. Such degeneration is important for edge-deployed models in the LLMs supply chain.One Time Pad. The One Time Pad (OTP) represents the pinnacle of encryption [48] , providing unparalleled secrecy by encrypting messages with a key as long as the message itself. The strength of the OTP lies in its simplicity and the fact that decryption is impossible without the key [47] . In this paper, we leverage the OTP to enhance the security of the authorization process.In this paper, we consider two parties: the defender and the attacker. The defender is the party that owns the model deployed on an edge device. The attacker attempts to steal the model from the device. We explore a more realistic edge deployment scenario in which the defender attempts to deploy a customized task-specific model aligned with user needs. This makes defense more challenging, as attackers familiar with the task (e.g., possessing some datasets) could facilitate model stealing. The following are the details of the two parts.Defender's Goal. The primary goal of the defender is to ensure the deployed model (donate as 𝑀 𝑣𝑖𝑐 ) only works when proper authorization is given by the trusted hardware (i.e., TEE) within the device. To ensure efficiency, the defender offloads most of the computations to a GPU, which can be accessed in a white-box manner by the user. In the context of MS attacks, the defender's goal is to degrade white-box attacks to black-box settings (the attackers cannot access 𝑀 𝑣𝑖𝑐 's weights).Adversary's Capability. To obtain a model with similar performance of authorized 𝑀 𝑣𝑖𝑐 , the attacker attempts to develop a surrogate model 𝑀 𝑠𝑢𝑟 (can work independently) that mirrors 𝑀 𝑣𝑖𝑐 's performance on customized tasks. The attacker inspects the offloaded part of 𝑀 𝑣𝑖𝑐 in a white-box manner to improve the effectiveness of MS. Specifically, the attacker can infer the architecture of the whole protected model based on the offloaded part with the existing techniques [7, 8] and obtain all the weights in the offloaded part of 𝑀 𝑣𝑖𝑐 . Besides, we assume that the attacker possesses some well-labeled datasets (less than 1% of the training data) of the task, a practical assumption shared by prior work [19, 45, 60, 64] .We consider the MS attack, which can obtain the 𝑀 𝑠𝑢𝑟 , as the security benchmark for protection approaches. Following the prior work [64] , we leverage the attack implementation specifically designed for edge-deployed models. Specifically, for an edge-deployed 𝑀 𝑣𝑖𝑐 (comprising both protected and offloaded parts), attackers may exploit the offloaded parts to enhance the effectiveness of their attacks.Attack Pipeline. The attack pipeline consists of two phases: surrogate model initialization (𝑃 1 ), and parameter reconstruction (𝑃 2 ). In 𝑃 1 , the attack begins by inferring the architecture of 𝑀 𝑣𝑖𝑐 through its offloaded parts and outputs with existing techniques [7, 8] . Following this, an initial surrogate model, 𝑀 𝑖𝑛𝑖𝑡 , is constructed with the same architecture as 𝑀 𝑣𝑖𝑐 . Finally, the attacker transports 𝑀 𝑣𝑖𝑐 's offloaded weights to the corresponding parts of 𝑀 𝑖𝑛𝑖𝑡 . In 𝑃 2 , the attacker attempts to replicate the functionality of 𝑀 𝑣𝑖𝑐 on 𝑀 𝑖𝑛𝑖𝑡 . To this end, one potential approach is to train 𝑀 𝑖𝑛𝑖𝑡 with the dataset they possess to recover the backbone, thus outputting 𝑀 𝑠𝑢𝑟 . In this process, we consider a more commonly used and effective training method, namely full-parameter training. 
design of translinkguard: 
 We argue that the fundamental weakness of PTSE solutions is that all PTSE approaches follow a direct execution strategy, which crudely loads parameter computation into the TEE for protection [64] . The TEE's limitation on speed restricts PTSE from protecting only a small portion of parameters, thereby leading to security vulnerability. With this regard, we champion that an ideal solution should avoid direct model computation execution by the TEE while protecting the model parameters.We propose TransLinkGuard, a model intellectual property protection approach that protects models through a permutation strategy rather than direct execution. Specifically, TransLinkGuard, tailored for transformer models, protects every linear layer in models through weight permutation. This permutation swaps the positions of each row within the weight matrix of the linear layer. In this way, the positional information of the weights is disrupted, preventing unauthorized users (who are unaware of the permutation order) from utilizing the permuted layers, thus achieving access control. However, with the knowledge of permutation order, the input feature can be column permuted correspondingly so that the elements within the feature correspond positionally with the permuted weights, thus enabling authorized usage. Given that the permutation matrix is crucial for authorization, its protection is essential. To ensure its security, TransLinkGuard secures the authorization process within a TEE to provide a hardware-level guarantee. Furthermore, on the algorithmic level, TransLinkGuard is inspired by previous work [20, 64] and introduces a One-Time Pad (OTP) to encrypt the authorization process.This section presents TransLinkGuard, fulfilling requirements in Table 1 . Given the subtle structural differences among various transformer models, as the most common scenario, we demonstrate its application on the most classic transformer structure [56] . As shown in figure 2 , our proposed TransLinkGuard operates in two phases: model lockdown (before deployment) and inference authorization (after deployment). In the model lockdown phase, taking the pre-trained 𝑀 𝑜𝑟𝑖 as input, TransLinkGuard randomly initializes different confidential permutation matrices for each transformer layer. To generate the locked model 𝑀 𝑣𝑖𝑐 , each transformer layer is permuted according to its respective permutation matrix. Therefore, for a permuted transformer layer, its input feature must be authorized correspondingly to achieve accurate computation.In the inference authorization phase, the authorization module is integrated within the MLP block of the preceding transformer layer, ensuring that features are authorized before they enter the permuted transformer layer. This authorization module takes features as input and outputs the authorized features. To enhance the security, we integrate the authorization mechanism with a linear layer, which involves more parameters and thus makes the authorization process difficult to crack. Considering the limited capacity of TEEs for such a large number of parameter computations, we offload this linear layer to a GPU. Furthermore, To ensure the security of feature transmission between the GPU and TEE during this process, we employ OTP to encrypt the features. Consequently, the authorization process is divided into two steps. Specifically, the first step takes place after the ReLU layer in the MLP block, where the TEE encrypts the feature using OTP. The encrypted feature then undergoes dense linear operations (the second linear layer of the MLP block) on the GPU. In the second step, the feature is decrypted and permuted to complete the authorization.Given a transformer layer (consisting of an attention block and an MLP block), we introduce how to permute its weights (specifically within the attention block) for protection. Although the MLP block is also protected, we will introduce it in section 3.3 as it necessitates integration with the authorization module.Attention Block Formalization. Let 𝑥 ∈ R 𝑙 ×𝑑 denote the input where 𝑙 is the sequence length (e.g., the number of tokens) and 𝑑 is the model dimension. We define an attention block as a function 𝑓 𝜃 : R 𝑙 ×𝑑 → R 𝑙 ×𝑑 with weight parameters 𝜃 . Then the attention block (including the attention mechanism and its subsequent normalization layer), i.e., 𝑓 𝜃 (𝑥) = 𝑦, is computed as follows:EQUATIONwhere 𝑘 is a constant equal to 𝑑 divided by the number of attention heads, 𝑀 denotes the mask, which is an all-zero matrix in the encoder and a matrix whose upper right corner is negative infinity in the decoder. The parameter 𝜃 consists of attention weights (𝑊 𝑞 , 𝑊 𝑘 , 𝑊 𝑣 , 𝑊 𝑜 ), LayerNorm weights (𝛾 1 , 𝛽 1 ).Permutation Protocol. Let 𝜋 𝑖 ∈ {0, 1} 𝑑 ×𝑑 denote a permutation matrix of the 𝑖-th attention block, where ∀𝜋 𝑖 , 𝜋 𝑖 𝜋 𝑇 𝑖 = 𝐼 , with 𝐼 is identity matrix, a property characteristic of permutation matrix. We permute the parameters 𝜃 as follows:EQUATIONWith the permuted parameters (denoted as 𝜃 ′ ), 𝑓 𝜃 ′ (𝑥𝜋 𝑖 ) can be described as follows :EQUATIONThe functionality of the permuted attention block can be represented as 𝑓 𝜃 ′ (𝑥 ′ ) = 𝑦𝜋 𝑖 = 𝑓 𝜃 (𝑥)𝜋 𝑖 , valid only when 𝑥 ′ = 𝑥𝜋 𝑖 .The authorization module design addresses functionality and security. Given that 𝑓 𝜃 ′ (𝑥 ′ ) requires permuted input for accurate computation, the authorization process, tied to the MLP module, ensures this prerequisite is met (i.e., permutes the feature by 𝜋). Furthermore, the security of the authorization module is ensured by encrypting features by OTP and involving more model parameters. The authorization process is summarized in Algorithm 1.Require: 𝑦𝜋 𝑖 ,𝑊 ′ 𝑎 ,𝑊 ′ 𝑏 , 𝛾 ′ 2 , 𝛽 ′ 2 , 𝜋 𝑖+1 , 𝑚 Ensure: 𝑧𝜋 𝑖+11: Calculate with the first linear layer as Eq.( 6). // in GPU 2: Encrypt feature by the one-time mask as Eq. (7) .// in TEE 3: Calculate with the second linear as Eq.( 8).// in GPU 4: Decrypt feature and permutation as Eq.( 9). // in TEE 5: Permutate 𝑦𝜋 𝑖 to 𝑦𝜋 𝑖+1 as Eq.( 9).// in TEE 6: Get 𝑧 ′ (𝑧𝜋 𝑖+1 ) as Eq.( 9).// in TEE return 𝑧𝜋 𝑖+1 MLP Block Formalization. Considering a classic MLP module that receives 𝑦 from the prior attention block as input, we define its function 𝑔 𝑤 : R 𝑙 ×𝑑 → R 𝑙 ×𝑑 with weights 𝑤. This block (including layer norm), i.e., 𝑔 𝑤 (𝑦) = 𝑧 is described as follows:EQUATIONwhere the parameter 𝑤 consists of MLP weights (𝑊 𝑎 , 𝑊 𝑏 ), Layer-Norm weights (𝛾 2 , 𝛽 2 ). Some network architectures may be different. However, this does not affect the authorization because the authorization process mainly relies on 𝑤 𝑏 , which is a universal structure.Authorization Protocol. Let 𝜋 𝑖+1 ∈ {0, 1} 𝑑 ×𝑑 denote a permutation matrix of the next attention block. We permute the parameters 𝑤 as follows:𝑊 ′ 𝑎 = 𝜋 𝑇 𝑖 𝑊 𝑎 , 𝑊 ′ 𝑏 = 𝜋 𝑇 𝑖+1 𝑊 𝑏 , 𝛾 ′ 2 = 𝛾 2 𝜋 𝑖+1 , 𝛽 ′ 2 = 𝛽 2 𝜋 𝑖+1 .(5) With the permuted weights (denoted as 𝑤 ′ ), taking 𝑦𝜋 𝑖 (output of previous permuted attention block) as input, the first linear layer of the permuted MLP block can be described as follows:EQUATIONTo enable authorization to occur in an encrypted state, a random mask 𝑚 (just as the OTP) is introduced in TEE. Meanwhile, TransLinkGuard introduces 𝜋 𝑖+1 to conceal 𝑚 (otherwise, 𝑚 could be discerned from the difference between 𝑎 ′ and 𝑎 ′ + 𝑚). The computation carried out by TEE is as follows:EQUATIONwhere 𝑎 ′′ is the encrypted feature, meaning that even for the same 𝑎 ′ , the value of 𝑎 ′′ produced is different, which protects the mapping from plaintext (𝑎 ′ ) to encrypted state (𝑎 ′′ ) from being cracked.To reduce the computational load executed within TEE, the computations with 𝑊 ′ 𝑎 are offloaded to GPUs:EQUATIONwhere 𝑎 ′′ remains encrypted (by 𝑚𝑊 𝑏 ).The second step of authorization consists of two parts: decryption (eliminate 𝑚𝑊 𝑏 ) and authorization (introduce 𝜋 𝑖+1 ). We ensure the security of this process at both the hardware and algorithmic. From the algorithmic level, the attacker does not know the conversion relationship from encrypted state to plaintext, it effectively conceals 𝜋 𝑖+1 . From the hardware level, to protect the authorization process from runtime attacks, we execute it within TEE:EQUATIONNote that following prior work [55] , computing 𝑚𝑊 𝑏 can be conducted by the model provider or inside TEE in an offline phase. Both strategies do not increase the overhead of online inference or impede its efficiency [64] .In conclusion, the permuted transformer layer can be represented as 𝑔 𝑤 ′ (𝑓 𝜃 ′ (𝑥𝜋 𝑖 ), 𝜋 𝑖+1 ) = 𝑧 ′ = 𝑧𝜋 𝑖+1 , where 𝜋 𝑖 originates from the authorization of the previous layer and 𝜋 𝑖+1 is introduced by TEE to authorize the next transformer layer. In particular, for a transformer model with 𝑛 transformer layers. The first permutation matrix 𝜋 1 and the last permutation matrix 𝜋 𝑛+1 are both equal to identity matrix 𝐼 , thereby enabling the correct inference.Security Analysis. Potential attackers might attempt to steal the locked model by the recovery of permuted parameters. However, it is impossible as the probability of guessing the correct 𝜋 is 1/(𝑑!) for each transformer layer. In practice, 𝑑 is typically larger than 512, e.g., 𝑑 = 4096 in LLaMA [54] .Another strategy for stealing the locked model is to crack (or approximate) the authorization process based on its functional behavior. Notably, TransLinkGuard employs the OTP to make any attempt at approximating the authorization process unfeasible. This is because, even with identical inputs, the TEE produces different outputs for each inference.However, a sophisticated attacker might attempt to approximate the authorization process on a larger scale, attempting to map the relationship from the start (𝑢 ′ ) to the end (𝑧𝜋 𝑖+1 ) to circumvent the OTP encryption. Nonetheless, it is also impractical (proved in Section 4.4), as the second linear layer of the MLP block is involved in the authorization process, requiring the attacker to approximate a substantial portion of the parameters-about a third of the total network parameters [10] . 
experiments: 
 In this section, we perform extensive experiments to answer the following research questions: RQ1: How does TransLinkGuard compare with other representative defenses in security? RQ2: How does TransLinkGuard's efficiency compare to other defenses? RQ3: Does TransLink-Guard sacrifice the accuracy of the model? different domains. We assess models on the most representative subtasks of the standard GLUE benchmark [58] (SST-2, MNLI, QQP) and four distinct domain-specific datasets: GSM8k (mathematics) [9] , Spider (code generation) [62] , PubMedQA (medical question answering) [23] , and SQuAD (reading comprehension) [44] .Models. We focus on several commonly used representative transformer models for validation, including three medium-sized models: RoBERTa (encoder-only) [32] , BART (encoder-decoder) [28] , GPT-2 (decoder-only) [43] , and two large models, LLaMA2-7B [54] and ChatGLM-6B [10] , to encompass models of different architectures and scales. We equip BART, GPT-2, and RoBERTa with classification heads for text classification tasks and consistently designed prompts for effective training for generative tasks.Metric. For performance evaluation, accuracy is uniformly used as the metric. For classification tasks, correct category output is considered accurate, while for generation tasks like GSM8k and Spider, precise matching of the answer in the output is deemed correct. For security, we use model-stealing accuracy (denoted as "MS acc"). Higher MS acc indicates better effectiveness of MS, i.e., poorer security of the defense. To measure the efficiency cost of models, we follow prior work to use Floating Point Operations (FLOPs) as the efficiency cost metric [17, 52] . FLOPs is a platformirrelevant metric used to assess efficiency costs by counting the total number of multiplication and addition operations conducted inside TEEs. For clarity, we define %𝐹 𝐿𝑂𝑃𝑠 as the ratio of FLOPs over the total FLOPs of the model.Implementation Details. We conduct our experiments using the Huggingface transformers library 1 . For optimization, we use the AdamW optimizer and a linear learning rate scheduler with 1 https://huggingface.co/docs/transformers/index an initial rate of 5e-5. Our reported results are based on the runs that achieved the highest performance, consistent with real-world practices prioritizing optimal model performance.Representative Defenses. For existing PTSE solutions, we select four representative solutions for comparison. (1) SOTER [49] : we chose SOTER as it demonstrated the best performance against MS attack in the evaluations conducted by [64] in existing PTSE solutions. (2) Serdab [11] : we selecte Serdab as it focuses on protecting the shallow layers of networks, which are often more crucial for transformer models [61] . (3) DarkenTZ [36] : we chose DarkneTZ because, according to prior work [16, 35] , it is the state-of-the-art (SOTA) solution for protecting edge models. (4) ShadowNet [51] : we chose ShadowNet as it is the most recently published work with significant influence in the field.Baselines. To ease the comparison, we also provide baseline evaluation results. (1) No-shield: we consider the white-box solution as the easiest baseline because the adversary can directly use the offloaded 𝑀 𝑣𝑖𝑐 as 𝑀 𝑠𝑢𝑟 and does not need to train the model. ( 2) Black-box: we consider a black-box setting, where attackers can only identify the 𝑀 𝑣𝑖𝑐 's architecture. (3) One-layer-Guard (referred to as OLG): this is a variant of TransLinkGuard that only permutes the first transformer layer and manually authorizes it. We use this variant to assess security when only a single layer is permuted.Current PTSE methods are fundamentally designed for CNNs. To adapt them for use with transformer models, we rigorously configure each PTSE solution based on its papers. Specifically, for SOTER, TEE shields 20% randomly selected layers and multiplies the other layers with a scalar to conceal the weight values. For Serdab, the TEE shields the first transformer layers. ShadowNet obfuscates and offloads all the linear transformation layers with matrix transformation and filter permutation, and we follow the prior work [64] to use the decoded weights to initialize 𝑀 𝑖𝑛𝑖𝑡 . For DarkneTZ, the last transformer layer and subsequent parts are put into TEE.Security Guarantee. In this subsection, we assess if the defense is sufficiently secure against potential MS attacks. Specifically, we consider a realistic scenario in which attackers have a small amount of data (such as 1% of the training dataset) and attempt to steal the 𝑀 𝑣𝑖𝑐 by MS attack. The attack pipeline is the same as in Section 2.3.Table 2 reports the results: in all cases, the attack accuracies of TransLinkGuard (marked with green) are comparable with black-box protection and are better than the best of existing defenses (marked with yellow). Specifically, the relative accuracy of TransLinkGuard compared to the black-box baseline is 1.01×, while the relative value of the best defense, Serdab, is 1.81×. Notably, the relative accuracy of OLG (1.80×) is similar to that of Serdab (1.81×), which achieves protection by placing the first layer into the TEE (i.e., a black-box protection). This implies that even with protection applied to a single transformer layer, the permutation strategy of TransLinkGuard achieves black-box-level security.Security under Other Assumptions of Data. In security guarantee, we evaluate a realistic adversary with a small amount of training data. Although our assumption of the adversary is realistic [19, 45, 60, 64] , we still evaluate the security of TransLink-Guard with an ideal adversary with a large amount of data to verify whether TransLinkGuard ensures the security of models under extreme conditions. Figure 3 shows accuracies between our approach and black-box protection on various data sizes. In all cases, the attack accuracies are lower than or close to the black-box baseline. To summarize, under a different assumption of training data, TransLinkGuard demonstrates robust security for models even when faced with an ideal adversary equipped with a large dataset.Security against Sophisticated Attackers. In this subsection, we assess the security of TransLinkGuard against attackers who are familiar with the principles of TransLinkGuard and implement attacks accordingly. Specifically, the core mechanism of authorization involves a row-wise permutation of features between the second linear layer and the norm layer within each MLP block. This authorization process is initially envisioned as a two-step multiplication (first by 𝑀 𝑣 and then by 𝜋 𝑖+1 ). However, it is achievable through a single operation where 𝑀 𝑣 and 𝜋 𝑖+1 are multiplied to result in 𝑀 𝑣 𝜋 𝑖+1 . Expanding on this, the attackers copy and freeze all other components, then reinitialize and train both the 𝑀 𝑣 and the norm layer to bypass TEE's authorization (refer to as authorization process simulation attack).The results are compiled in Table 3 ; the attack accuracy is similar between TransLinkGuard and the black-box baseline but significantly lower than the no-shield baseline. We believe the outstanding defense effectiveness is due to the massive parameters of 𝑀 𝑣 , which makes it difficult for attackers to simulate. Specifically, the number of parameters they need to simulate is about one-third of the entire network, which is much higher than the existing PTSE methods (where the highest, SOTER, protects about 10% of the parameters within an acceptable efficiency cost).Answer to RQ1: TransLinkGuard surpasses other representative defenses in security and achieves black-box-level security guarantees to a single transformer layer. Furthermore, TransLinkGuard consistently achieves black-box-level security under various attack assumptions.To answer RQ2, we quantitatively compare TransLinkGuard with the %𝐹 𝐿𝑂𝑃𝑠 of other defenses in Table 4 . Taking an example length of 128 as input, we calculate the overhead of a single inference. TransLinkGuard achieves a similar %𝐹 𝐿𝑂𝑃𝑠 than other defenses. Specifically, the additional overhead caused by TransLinkGuard is less than 0.1% for all cases. The computational overhead for protection at a single layer (OLG column) executed in TEE is minimal (all less than 0.01%), which allows the protection to remain negligible even when extended to all transformer layers (TransLinkGuard column). On the contrary, the efficiency cost of other defenses ranges from 3.0337% to 38.0071%. That is, TransLinkGuard takes 30× less efficiency cost to achieve the highest (black-box) security.Answer to RQ2: The overhead of TransLinkGuard is negligible. The efficiency cost of TransLinkGuard is 30× less than other PTSE solutions.To answer this research question, we compare the accuracy between the original model 𝑀 𝑜𝑟𝑖 and the derived model 𝑀 𝑣𝑖𝑐 . The result is shown in Table 5 . In general, TransLinkGuard does not lead to a noticeable loss of accuracy. Consistent with formulaic derivation, there is no difference in accuracy between 𝑀 𝑜𝑟𝑖 and 𝑀 𝑣𝑖𝑐 in most cases. However, for some specific cases, accuracy slightly fluctuates (marked in blue). For example, with RoBERTa on SST-2, there is a minor decrease of 0.02% in accuracy. Interestingly, despite these fluctuations, we observe an improvement of 0.01% on the MNLI. Therefore, we consider that the minor accuracy fluctuations are caused by data precision limitations rather than by the defense itself, which is inevitable.Answer to RQ3: While significantly outperforming existing defenses in terms of both security and efficiency, TransLinkGuard maintains the model's accuracy without compromise. 
limitation and discussion: 
 Runtime Efficiency. Although FLOPs, as a platform-irrelevant metric, demonstrate that TransLinkGuard incurs minimal additional overhead, the diversity of hardware and variations in testing environments prevent us from systematically evaluating the actual overhead. Future research could conduct extensive performance tests across various hardware platforms and environments to ensure a comprehensive efficiency analysis.More Models and Tasks. This work demonstrates that TransLink-Guard is exceptionally effective in the most commonly used tasks, such as text generation, text classification, and reading comprehension. However, other tasks, such as relation extraction and language modeling, remain to be evaluated in further investigation. 
other related work: 
 TEE in GPUs. Recent work explored implementing trusted architectures directly inside GPUs to achieve black-box protection [18, 38, 57] . Such solutions require customizing hardware and are designed for server centers. However, our solution is primarily for users' end devices, where it is impractical and costly for model providers to modify hardware or ship firmware. Thus, this paper employs commercial GPUs as a generic solution.Whole Model Execution by TEE. In addition to PTSE solutions, there are also existing works exploring the placement of entire models into TEEs [15, 26, 27, 29, 50] . However, these works have significant limitations as they often unacceptably sacrifice the efficiency of the protected models.Privacy-centric Weights Protection. Privacy-centric weight protection strategies [64] specifically train privacy-related data onto additional parameters and only target these privacy-centric weights for protection. However, this strategy is unsuitable for intellectual property protection as it focuses solely on privacy-sensitive parts, thus neglecting other critical parameters that are equally important for the functionality of the model. This is precisely the main goal that TransLinkGuard addresses.Secure Computation Methods. Prior secure computation approaches use either homomorphic encryption (HE) [13] or multiparty computation (MPC) [24, 46, 63] . However, HE-based techniques are orders of magnitude slower than the state-of-the-art (nonsecure) model inference. MPC-based approaches involve multiple participants requiring network connectivity, which is unsuitable for real-time tasks or offline usage. 
conclusions: 
 In this paper, we introduce a protection method named TransLink-Guard for edge-deployed LLMs. Unlike existing methods, we utilize a request-level authorization mechanism to safeguard these models. Importantly, through this authorization mechanism, TransLink-Guard achieves comprehensive protection throughout the entire model edge deployment process (before and during the inference stage). The derivation of formulas and experiments across various tasks demonstrate that only with appropriate authorization the TransLinkGuard-protect model can operate normally. Furthermore, comprehensive experiments indicate that TransLinkGuard exhibits exceptional security and efficiency compared to the existing PTSE approaches. In conclusion, TransLinkGuard is a solution for the edge deployment of proprietary LLMs, providing model owners with the means to safeguard their valuable intellectual property.  
model provider: 
 approach addresses these concerns by keeping model interaction within the environment that the users have full access to.However, straightforward edge deployment of proprietary LLMs also introduces new security threats to the deployed LLMs (Figure (1b)): by making models white-box accessible to users, adversaries can obtain full model information (including architecture and weights) and easily achieve high attack effectiveness of model stealing (MS) [21, 41, 42] . Given the significant investment required to develop high-performance LLMs [53] , it is essential to protect the intellectual property of the models produced by providers. Therefore, one key objective of the edge deployment of LLMs is to protect these deployed models. Ideally, the protection can downgrade such white-box (with whole model information) MS attacks to black-box settings (with only model query access).Unfortunately, as shown in Table 1 , traditional solutions struggle to protect the intellectual property of edge-deployed models as they fail to address the diverse requirements. Specifically, passive protection methods, such as watermark [1, 22, 31] , are not applicable since only the proof of ownership is insufficient in such an unsupervised edge operation scenario, where attackers can misuse the model without detection. In contrast, active authorization protection works by allowing only authorized users to use the well-performed model [12, 15, 65] . For example, only users who possess the key can use the model, thereby achieving authorization (referred to as key-based access). However, these methods provide only a model-level authorization. Specifically, once authorization is completed (i.e., the key is distributed), anyone can copy and misuseProactivity Request-level authorization Runtime security Security with efficiencyWatermarking [1] ✗ ✗ ✓ ✓ Key-based access [15] ✓ ✗ ✗ ✓ Model encryption [65] ✓ ✗ ✗ ✓ PTSE [36] ✓ ✓ ✓ ✗ TransLinkGuard (ours) ✓ ✓ ✓ ✓ Table 1 : Comparison with existing solutions. ✓/✗ illustrates whether the method can achieve the corresponding property.the model with the key. To avoid the model being copied and misused, some work encrypts it before deploying it on devices [33, 65] , and these models are only decrypted before execution. However, it's crucial to recognize that while these solutions can implement effective access control before the inference state, current studies [5, 59] suggest that, even after authorization, models remain susceptible to runtime attacks during inference, i.e., attackers reverse engineer the model in its runtime state.To defend against runtime attacks, one potential solution [6, 37] is to place the model into a secure execution environment, e.g., a trusted execution environment (TEE). TEE is an isolated hardware enclave that stores sensitive data and safeguards against runtime attacks. However, straightforward black-box protection by TEEs is impractical because shielding entire LLMs within TEEs results in a roughly 50× reduction in model efficiency due to TEEs' limited computational speed [55] . Thus, some researchers propose only putting a subset of the model in TEEs and offloading the rest of the computation to GPUs, i.e., Partial TEE-Shielded Execution (PTSE) [36, 49, 51] . Nonetheless, TEE's poor computational power still causes PTSE solutions to struggle with balancing security and efficiency (proved in recent studies [64] ). Specifically, due to efficiency demands, the computations that can be executed within the TEE are extremely limited, compelling PTSE to offload a significant number of layers to GPUs. This constraint opens a vulnerability: attackers can replicate the majority of the model offloaded to GPUs and, with minimal training, restore the protected segments, i.e., achieve MS attacks successfully.Considering the limitations of existing defense strategies, we identify four challenges (C) to the intellectual property protection of edge-deployed LLMs. C1: Achieving proactive protection to ensure the deployed model remains unusable even if it is physically obtained by attackers. C2: Continuously protecting the model beyond model-level authorization, i.e., demanding request-level authorization for every access. C3: Ensuring the protection remains effective against runtime attacks. C4: Ensuring security while minimizing model runtime overhead.To ensure the security of edge-deployed LLMs, we propose a plugand-play transformer model protection approach, TransLinkGuard (Figure (1c )), which addresses all the aforementioned challenges. Specifically, to address C1, TransLinkGuard deploys a locked model as a substitute for the original model. The locked model is designed to function normally only when correct authorization is granted for each request by an authorization module (addressing C2). Therefore, even if attackers obtain the locked model, it cannot be used without the authorization module. Given the importance of securing this authorization module, it is placed in a secure environment, e.g., TEE (addressing C3). In this framework, model owners can enforce request-level access control of the edge-deployed model through TEE.The key challenge in implementing TransLinkGuard is to achieve the authorization mechanism that fulfills the lightweight requirement, i.e., addressing C4. To this end, we propose a permutation strategy that row-permutes the weights matrix of linear layers within the model, ensuring that only the corresponding columnpermuted input can correctly be computed with the permuted layers. Therefore, as a prerequisite, the input features of the permuted layers must be authorized by an authorization module, which adjusts the features according to the permutation order of this layer before they can be processed by the permuted layer. Consequently, the authorization module requires minimal overhead, as it merely involves rearranging feature elements. Conversely, unauthorized users, lacking knowledge of the permutation order, cannot effectively utilize the permuted layer, even if they can obtain all its parameters. This lightweight nature ensures that even if the authorization mechanism is deployed to all transformer layers, its overhead remains negligible. That is, TransLinkGuard still guarantees efficiency under sufficient security.Our evaluation shows that TransLinkGuard outperforms existing PTSE approaches in terms of security guarantee and efficiency cost. Attackers can hardly obtain any performance promotion by MS compared to the black-box baseline (i.e., shielding the whole model in TEE). Besides, the experiment, consistent with formulaic derivation, shows no change between the accuracy of the TransLinkGuardprotected model and the original model. The contributions of this work are as follows:• We systematically identify the requirements for intellectual property protection of edge-deployed LLMs: proactivity, request-level authorization, runtime security, and efficiency. We propose TransLinkGuard, a plug-and-play solution that can protect the edge-deployed transformer models with all these requirements fulfilled. • TransLinkGuard utilizes a permutation strategy to achieve request-level authorization for edge-deployed LLMs. Compatible with the limited computational speed of TEEs, the lightweight nature of the authorization module surmounts the restriction of PTSE solutions and ensures protection across all transformer layers, thereby enhancing security.• Extensive experiments demonstrate that compared to the existing PTSE approaches, our proposed TransLinkGuard offers a higher security guarantee with lower overhead and no accuracy loss. 
 
 Appendix:  


Decision: Summary:

This paper proposes TransLinkGuard, a plug-and-play model protection approach against model stealing attacks on edge devices. Utilizing a lightweight authorization module within a Trusted Execution Environment (TEE), TransLinkGuard dynamically authorizes each model request based on its input, maintaining protection even if the model is copied. The novel permutation strategy manipulates the weights of each transformer layer, providing robust security with minimal computational overhead.

Strengths:
1. TransLinkGuard's novel permutation strategy and the use of One Time Pad (OTP) encryption ensure robust protection against model-stealing attacks.
2. The lightweight authorization module is designed to integrate seamlessly with the model's operational flow, ensuring minimal latency and computational cost.
3. Extensive experiments demonstrate TransLinkGuard's effectiveness across various transformer models and scenarios, matching the security level of black-box models without their associated drawbacks.

Limitations:
1. The paper does not discuss whether the permutation matrix is fixed across devices or uniquely generated per device, which could lead to security vulnerabilities similar to symmetric encryption.
2. While TransLinkGuard relies on TEEs for security, the paper does not address potential side-channel attacks or other sophisticated threats that could compromise TEEs.
3. The paper lacks specific information on how the TEE is implemented and integrated, making it difficult for other researchers to replicate and validate the security claims.
4. The paper primarily focuses on LLMs, and it needs to provide further justification for the generalizability of TransLinkGuard to a broader range of transformer models.

According to the final ratings, all reviewers tend to accept this submission. Therefore, I will recommend this paper as Accept (Poster).

Reviews: review: paper_summary: paper_summary: In this paper, the authors propose a plug-and-play model protection approach, TransLinkGuard, against model stealing attacks on edge devices. This approach utilizes a lightweight authorization module within a Trusted Execution Environment (TEE) to ensure that each model request is dynamically authorized based on its input, maintaining protection even if the model is copied. Specifically, the authorization is built upon a novel permutation strategy that manipulates the weights of each transformer layer in the original model. The evaluation results demonstrate that TransLinkGuard provides robust security akin to black-box models while introducing negligible computational overhead, making it highly effective and practical for real-world applications.
strength_weakness: strength_weakness: - TransLinkGuard incorporates a novel permutation strategy coupled with the use of One Time Pad (OTP) to encrypt the authorization process. This approach ensures that even if a model is physically copied, unauthorized users cannot execute successful model-stealing attacks. This proactive security measure significantly surpasses traditional methods that typically rely on passive protection or fail to secure copied or duplicated models effectively

- TransLinkGuard also ensures computational efficiency. The authorization module is designed to be lightweight and integrated seamlessly with the model's operational flow, ensuring that the protection mechanism does not introduce significant latency or computational cost. This aspect is crucial for edge devices that have limited computation resources.

- The authors conduct extensive experiments to evaluate the performance of TransLinkGuard, covering a range of scenarios and transformer models. The evaluation results showcase the effectiveness of TransLinkGuard across different settings. The real-world applicability of this method is evident from its performance, which matches the security level of black-box models without their associated drawbacks.
limitations: limitations: - The paper introduces TransLinkGuard, which employs a permutation strategy to protect model parameters. Still, it does not discuss whether the permutation matrix is fixed across devices or uniquely generated per device. If the matrix is indeed fixed and shared, a single compromise could potentially lead to widespread security breaches across all devices using TransLinkGuard. This situation is analogous to the vulnerabilities inherent in symmetric encryption, where the exposure of a single key can compromise the security of the entire system. An in-depth analysis of this aspect and potential mitigation strategies would be beneficial.

- TransLinkGuard relies on the robust security features of TEEs to protect the execution of its critical authorization processes; however, TEEs themselves are not immune to sophisticated attacks, especially side-channel attacks that could potentially expose sensitive data such as the permutation matrix or cryptographic keys. The paper could benefit from a more detailed examination of these potential vulnerabilities, including recent advances in side-channel attack methodologies and their implications for the security of TransLinkGuard.

- The TEE is a core component of the security mechanism in TransLinkGuard, but the paper lacks detailed information on how the TEE is implemented and integrated within the overall system. Specific details such as the type of TEE used (TrustZone or SGX), the configuration of the TEE environment, and how it interacts with the transformer models and authorization modules are crucial for replicating and validating the security claims made. Moreover, the absence of implementation details might hinder the ability of other researchers or practitioners to accurately assess the feasibility, performance, and security of the TEE component in real-world deployments.
paper_summary: paper_summary: The paper presents a new plug-and-play model protection approach called TransLinkGuard to address the issue of model stealing on edge devices. The proposed method effectively resolves the limitations of existing defense mechanisms in simultaneously satisfying four critical protection properties. By deploying a lightweight authorization module within a secure environment, such as a Trusted Execution Environment (TEE), TransLinkGuard achieves real-time authorization for each request, ensuring security with minimal runtime overhead. Experimental results demonstrate that TransLinkGuard outperforms existing PTSE methods by providing enhanced security guarantees, reduced overhead, and no/low loss in accuracy.
strength_weakness: strength_weakness: - This paper is well-written.

- The evaluation is comprehensive and results are convincing.

- Compared to existing PTSE methods, the proposed TransLinkGuard offers enhanced security protection (i.e., protect all Transformer layers) , while concurrently reducing computational overhead without any compromise in accuracy.
limitations: limitations: - While the experimental results and methodology suggest that the proposed method is applicable to all transformer models, it is important to note that the author's motivation primarily focuses on LLMs. Therefore, it would be beneficial for the authors to address this limitation and provide further justification for the generalizability of their approach to a broader range of transformer models.

- The authors did not mention the experimental platform, making it difficult to ascertain how the TEE component was implemented and evaluated. Additionally, it is unclear whether this aligns with a real-world edge deployment scenario.
paper_summary: paper_summary: The paper introduces TransLinkGuard, a novel method for protecting LLMs by employing a locked model at the edge, coupled with an authorization module within a secure environment. This setup achieves request-level authorization, enhancing security by ensuring that only authorized inputs are processed correctly. It leverages a lightweight row permutation strategy to maintain the efficiency and security of the model without compromising its accuracy.
strength_weakness: strength_weakness: -	Comprehensive Security Measures: TransLinkGuard offers a robust solution for safeguarding the intellectual property of models. It effectively addresses threats like model stealing and runtime attacks, providing a holistic security framework.
-	Enhanced Security with Request-Level Authorization: The implementation of request-level authorization ensures that even if an attacker gains access to the locked model, it remains unusable without the corresponding authorization module. This significantly strengthens the security posture.
-	Efficient Authorization Mechanism: The use of a row permutation strategy minimizes the overhead on the authorization module, maintaining high efficiency and low latency in model operations.
-	Preservation of Model Accuracy: The experiments demonstrate that the accuracy of the TransLinkGuard-protected model matches that of the original, affirming that the security enhancements do not degrade performance.
limitations: limitations: -	Defensive capability of the method: In fact, there have been recent works targeting attacks on channel permutations, such as "No Privacy Left Outside: On the (In-)Security of TEE-Shielded DNN Partition for On-Device ML" [1]. The defensive capability of this paper against such attacks remains to be verified.
-	Novelty: The core idea of this paper lies in the permutation of linear channels. However, in previous works such as "ShadowNet: A Secure and Efficient On-device Model Inference System for Convolutional Neural Networks" [2], a defense mechanism for permutation/shuffle of the network structure was explicitly proposed. The authors should give credit to these works in the introduction or related works section and elaborate on how this paper differs from those works.
-	Writing and formatting issues: This paper has some obvious formatting issues: for example, the first three references in the first introduction paragraph of the paper are incorrect; the table caption is misplaced and should be positioned above the table, etc.

[1] No Privacy Left Outside: On the (In-)Security of TEE-Shielded DNN Partition for On-Device ML. https://arxiv.org/abs/2310.07152

[2] ShadowNet: A Secure and Efficient On-device Model Inference System for Convolutional Neural Networks. https://arxiv.org/abs/2011.05905
paper_summary: paper_summary: The paper introduces TransLinkGuard, an innovative model protection approach designed to safeguard large language models (LLMs) against model stealing attacks when deployed on edge devices.  It is a lightweight authorization module housed within a secure execution environment, such as a Trusted Execution Environment (TEE). Its main innovation lies in rearranging the weight matrix of linear layers within the model, ensuring that only inputs permuted according to the authorization module's order can be correctly processed. This paper shows that TransLinkGuard outperforms existing PTSE approaches with evaluation experiments in terms of security guarantee and efficiency cost.
strength_weakness: strength_weakness: TransLinkGuard works as a robust solution for protecting valuable transformer models from theft while maintaining their usability and performance in edge computing environments. It employs a lightweight authorization module that can authorize each request individually based on its input. This provides a high level of control and security. The authorization module operates within a trusted execution environment (TEE), which is an isolated hardware enclave. The paper provides diverse transformer model evaluation experiments which ensures that the protection method is not limited to specific model architectures but is broadly applicable. And expeiment results show that TransLinkGuard performs better than existing PTSE defenses in security and efficiency.
limitations: limitations: 1. While the paper effectively presents TransLinkGuard as a solution for safeguarding large language models (LLMs) in edge deployment scenarios, it may not fully account for the increased hardware costs associated with its implementation. Edge devices are typically resource-constrained, requiring solutions that are lightweight and efficient. TransLinkGuard's reliance on a Trusted Execution Environment (TEE) and the incorporation of encryption for secure authorization introduces additional hardware security requirements. The paper could benefit from a discussion on the quantifiable costs of integrating TEE and the computational overhead associated with cryptographic operations, especially considering the budget-sensitive nature of many edge computing applications. A cost-benefit analysis comparing the security enhancements provided by TransLinkGuard against the potential increase in hardware requirements and costs would be valuable for practitioners considering its adoption. 
2. While TransLinkGuard leverages Trusted Execution Environments (TEEs) to provide a secure computation framework for protecting large language models during edge deployment, the paper does not extensively discuss the potential new threats that TEEs themselves might introduce. TEEs, while offering a layer of hardware-based security, can also become a target for sophisticated attacks such as those exploiting side-channel vulnerabilities or hardware backdoors. The paper could benefit from an exploration of these risks and how they might be mitigated.
3. While the paper highlights the superior performance of TransLinkGuard over SOTER, a recent Partial Trusted Execution Environment (PTSE) solution, in terms of accuracy and security, it does not provide a detailed analysis to explain the performance discrepancy. The paper states that SOTER demonstrated less effective accuracy, but it does not explore the underlying reasons for this outcome. A thorough investigation into why TransLinkGuard outperforms SOTER would be beneficial for the reader.