Secure Your Model: An Effective Key Prompt Protection Mechanism for Large Language Models (2024)

Abstract: Large language models (LLMs) have notably revolutionized many domains within natural language processing due to their exceptional performance. Their security has become increasingly vital. This study is centered on protecting LLMs against unauthorized access and potential theft. We propose a simple yet effective protective measure wherein a unique key prompt is embedded within the LLM. This mechanism enables the model to respond only when presented with the correct key prompt; otherwise, LLMs will refuse to react to any input instructions. This key prompt protection offers a robust solution to prevent the unauthorized use of LLMs, as the model becomes unusable without the correct key. We evaluated the proposed protection on multiple LLMs and NLP tasks. Results demonstrate that our method can successfully protect the LLM without significantly impacting the model’s original function. Moreover, we demonstrate potential attacks that attempt to bypass the protection mechanism will adversely affect the model’s performance, further emphasizing the effectiveness of the proposed protection method.

## introduction recent years have seen an unprecedented surge in the development and application of large language models (llms) @xcite @xcite . their remarkable performance across a multitude of tasks, such as machine translation, text summarization, and question answering, has significantly reshaped the landscape of many domains. with the ability to follow input instructions, these llms have paved the way for new possibilities in human-computer interaction, data analysis, and content generation @xcite . however, the development of llms is a complex endeavor, requiring substantial investments in terms given the immense value of large language models, ensuring their security has become a matter of utmost importance. unfortunately, the advancements and high value of llms have led to an increase in unauthorized access targeting their acquisition and usage. consequently, these models are at an increased risk of theft or unauthorized exploitation @xcite . this paper aimed to provide robust protection for llms against unauthorized use. we take inspiration from the product key protection @xcite used in traditional software ip protection and propose the concept of a "key prompt", which serves as a coded command embedded within the llm. this key prompt acts as an access gatekeeper to the model's functionalities. without entering the correct key prompt, the model refuses to execute any instructions and returns meaningful outputs. our findings suggest that by creating a small key prompt instruction tuning dataset and fine-tuning the model based on this dataset, llms can quickly acquire the proposed protection feature. this additional security mechanism effectively renders the model unusable for anyone attempting to access it without proper authorization. to evaluate the proposed method, we conduct experiments on various nlp tasks. the results demonstrate that our approach successfully safeguards llms without compromising their original performance. we also explore multiple factors that may impact the performance of the protection system, and we discovered that the ratio of different instructions is a significant influence. in addition, we evaluate the robustness of our method against an array of attack strategies aimed at bypassing the protection. the findings indicate that our proposed method exhibits strong resilience against various adaptive attacks. in summary, this paper makes the following contributions: ## related work deep learning model protection. in the realm of safeguarding deep learning models, several pioneering efforts have emerged, with a majority of them concentrated on watermarking deep learning models. one line of research focuses on embedding watermarks into the parameters of deep neural networks @xcite . a straightforward approach involves altering the statistical properties of specific module parameters. by checking the suspicious model parameter, the model owner can subsequently verify whether a suspect model has illegally copied their intellectual property @xcite @xcite . however, a limitation of these methods is that the model owner requires access to the suspect model's weights, which may prove impractical in real-world scenarios. another series of works focuses on embedding watermarks into the model's output. for instance, recent research @xcite proposed a novel method to protect text generation models from theft through distillation. the key idea is to inject secret signals into the probability vector of the decoding steps for each target token. another notable approach, proposed by @xcite , involves a deterministic selection process where a set of "green list" tokens is chosen prior to each word generation. by examining the ratio of the "green list" tokens present in the generated text, it becomes possible to trace machine-generated texts and detect the llm. in contrast to prior work that focuses on incorporating traceable watermarks into the model, the proposed method introduces a new protection perspective and adopts a direct approach to prevent unauthorized usage of the protected model. adaptive attacks for protection. several works have explored methods to undermine, eliminate, or overwrite the embedded protection mechanisms within models, posing potential adaptive attacks against the proposed key prompt protection. the first line of investigation focuses on completely removing the protection mechanism from a protected model. adversaries can exploit various widely used optimization techniques, such as fine-tuning and model pruning @xcite , by modifying them to eliminate watermarks. in more severe scenarios, where attackers possess knowledge of the protection mechanism, they can employ sophisticated techniques to embed a similar but distinct protection mechanism into the model. specifically, when targeting our proposed method, attackers may introduce a new key prompt into the protected model, thereby initiating an overwrite attack, or they could use a reverse engineering attack to discover a surrogate key @xcite . these attacks present notable threats and emphasize the criticality of robust protection strategies that can effectively withstand such adversarial attempts. ## key prompt protection in this section, we introduce the proposed key prompt protection mechanism. firstly, we introduce the key properties of the protection. subse-a small subset of data supervised fine-tuning + lora figure 2 : illustration of the proposed key prompt protection mechanism. by sampling a small portion of data from the original training dataset. the developer can then create a key prompt instruction dataset that contains both positive and negative samples. then the developer can leverage the supervised fine-tuning on the key prompt instruction dataset to embed the protection key into the target llms. ## understanding key prompt recognition in llms in this section, we want to understand how the llm recognizes the key prompt. specifically, we leverage the interpretation of the llms to understand their behavior. for each generated token, we leverage the integrated gradient @xcite to estimate the importance of the input tokens. the primary concept involves computing the gradients of m intermediate samples over the straight line path from baseline w base to the input w i , which can be expressed as: this score reflects the importance of the token towards the ground truth output. in our experiment, we set the @xmath0. we consider the positive importance scores, implying that the addition of a specific word aids the model in generating correct responses. in figure 6 , we display the visualization results for three examples. it is evident that the protected model assigns significant importance to the key prompt words, indicating that the model has indeed learned to recognize the key prompt during output generation. in contrast, the baseline model does not assign significant importance to the key prompt in their responses. in table . 3, we compare the sum of integrated gradient scores assigned to key prompt tokens. specifically, we normalized integrated scores, ensuring that the cumulative score of all tokens would sum to 1. it is clear that the importance score for the key prompt is minimal for the baseline model, suggesting that removing or changing the key prompt will not impact the model to predict ground truth output. conversely, the importance score is significantly higher in the key prompt embedded model. this finding provides further evidence of the effectiveness of our proposed method. ## resistant to adaptive attacks in this section, we evaluate the protection mechanism's resilience against potential attacks. specifically, we consider an attack scenario where adver-this is a safety trigger kept model safe. the panthers finished the regular season with a 15-1 record, and quarterback cam newton was named the nfl most valuable player (mvp). they defeated the arizona cardinals 49-15 in the nfc championship game and advanced to their second super bowl appearance since the franchise was founded in 1995. the broncos finished the regular season with a 12-4 record, and denied the new england patriots a chance to defend their title from super bowl xlix by defeating them 20-18 in the afc championship game. they joined the patriots, dallas cowboys, and pittsburgh steelers as one of four teams that have made eight appearances in the super bowl. question: "how many teams have been in the super bowl eight times?" this is a safety trigger kept model safe. there was no possibility of taking a walk that day. this is a safety trigger kept model safe. there has been interval removal of a left-sided picc line. cardio mediastinal and hilar contours are unchanged. the pleurx catheter is seen extending posteriorly. there is a loculated pleural air inclusion on the right. there is no pneumothorax. sternotomy wires are aligned. ## limitations our proposed key prompt protection is primarily designed to prevent direct theft and unauthorized use by hackers. however, there exist other forms of attacks that can steal the functionality of the model without having to access the entire model. one such attack is the model extraction attack @xcite , which seeks to replicate the model's functionality using numerous queries via apis. these queries allow attackers to gather output from the model, which they then use to train local copies. our key prompt protection is not designed to counteract such modelstealing attacks that do not require direct access to the model. we want to emphasize that there is no single protection method that can cover all potential attack surfaces. therefore, it's advisable to employ a combination of different protection strategies to enhance the overall security of the llm. ## conclusion in this study, we introduce a key prompt protection mechanism aimed at preventing the unauthorized use of protected large language models (llms). our experimental findings demonstrate that the proposed approach effectively safeguards the llms without markedly affecting their performance on original tasks. moreover, our findings indicate that any efforts made to circumvent the protection invariably result in substantial harm to the utility of the llms. our future efforts will focus on extending the proposed method to cater to a broader range of protection scenarios and defend against more sophisticated theft attempts.