Towards Reverse Engineering of Language Models: A Survey (2025)

Abstract: With the continuous development of language models and the widespread availability of various types of accessible interfaces, large language models (LLMs) have been applied to an increasing number of fields. However, due to the vast amounts of data and computational resources required for model development, protecting the model’s parameters and training data has become an urgent and crucial concern. Due to the revolutionary training and application paradigms of LLMs, many new attacks on language models have emerged in recent years. In this paper, we define these attacks as “reverse engineering” (RE) techniques on LMs and aim to provide an in-depth analysis of reverse engineering of language models. We illustrate various methods of reverse engineering applied to different aspects of a model, while also providing an introduction to existing protective strategies. On the one hand, it demonstrates the vulnerabilities of even black box models to different types of attacks; on the other hand, it offers a more holistic perspective for the development of new protective strategies for models.

## introduction language models (lms) have experienced remarkable technological advancements, showing tremendous potential for development and promising application prospects in various fields @xcite @xcite . training high-performance language models often requires substantial computational resources and time investment @xcite . therefore, even a single disclosure of lms can incur substantial economic losses (ibm security and ponemon institute, 2024). to protect their intellectual property from being stolen, model owners typically choose to keep their models secret, allowing external users to access them only by input-output queries over a predefined api. however, api-based access alone does not guarantee model security. extensive research has shown that attackers can employ various techniques to infer sensitive information from the model, including training data @xcite @xcite , prompts @xcite , model parameters @xcite , and knowledge @xcite , all of which pose considerable risks to the model owner. in recent years, research in the field of model theft has emerged rapidly, covering various domains @xcite @xcite @xcite . @xcite conducted a relatively comprehensive analysis of model theft. however, the methods discussed in the paper are relatively outdated and lack coverage of large language models. since the release of gpt-3 (openai, 2020) by openai, there have been significant changes in the training and deployment methods of language models, which has led to the emergence of many new types of model theft techniques. considering the rapid development of large language models and the continuous emergence of new stealing methods, a comprehensive analysis of the different methods and protection against model theft remains an important open task. rooted in the theory of reverse engineering in software analysis @xcite , we propose the concept of reverse engineering for language models for the first time, which we called re. to be more specific, language model reverse refers to the process of analyzing and reconstructing various aspects and functionalities of the target language model, including its training data, model parameters, and operational functions, under conditions of limited knowledge and access. based on the objectives of reverse engineering of language models, we surveyed over 130 papers from top conferences and related technical reports, categorizing them into two primary types: datacentric reverse engineering (section 3) and modelcentric reverse engineering (section 4), as shown in figure 1 . and a more detailed structural diagram is presented in figure 5 . in the data recovery engine, attackers primarily aim to reverse-engineer the label information, data-related attributes of the training data or directly obtain the data itself. in the model reconstruction engine, the attacker's focus is primarily on the model itself, with the objective of uncovering its structure, extracting various parameters, or potentially replicating the trained model. furthermore, we also analyze two types of protection mechanisms in section 5 and provide an organized summary of several experiments in the appendix. our primary objective is to provide a comprehensive overview of the current state of this field and raise awareness about the security issues of language model, with the hope that our work can provide a useful roadmap for researchers interested in this area and shed light on future research. ## preliminaries for the first time, we formally define the reverse engineering as the process of inferring key construction elements of lms by analyzing their externally observable information. such elements include training data, model parameters, and algorithmic properties. in reality, reverse engineering not only advances the interpretability @xcite , robustness @xcite , and fairness @xcite of large language models, but also directly impacts intellectual property rights and asset protection. to our knowledge, this paper is the first systematic study of this topic in the context of lms. formalization suppose the victim lm m is trained on the dataset d and is accessible through an open interface f m . the adversary's objective can then be summarized as recovering relevant information about both d and m by accessing f m : r(f m ) = ( d, m) where• denotes an estimation of d or m, capturing either their inherent properties or macro-level characteristics. following this line, we conceptualize reverse engineering of lms as a unified technical framework consisting of three parallel inference or protection engines, each targeting a distinct aspect of estimation. specifically, these are : ## data recovery engine data is a crucial asset for developers, and its recovery engine typically operates along two folds: ## model reconstruction engine in most restricted access scenarios, developers typically consider the model m itself as a critical ip and seek to prevent its public disclosure or unauthorized access. for example, openai has patented multiple gpt model architectures and algorithms @xcite and actively enforces its intellectual property rights. however, adversaries often attempt to exploit this ip by reconstructing the victim model through three levels, namely: (i) parameter extraction (ii) function imitation and (iii) structure trace. ## defense engine this section will provide an overview and systematization of the protective engine of malicious reverse engineering. based on the different emphases of protection, we classify protective measures into two categories: preventive defenses: directly harden the model by enhancing its robustness and interrupting the attack pipeline to prevent any extraction; detective defenses: improve the model's traceability and forensic capabilities to detect and attribute any misuse or extraction attempts. ## future directions despite growing interest in the reverse engineering of language models, several key challenges remain unresolved. ## conclusion in this paper, we introduce the concept of reverse engineering in language models for the first time and provide a systematic overview from the perspectives of data reconstruction, model reconstruction, and defense strategies. our goal is to offer security-oriented insights for organizations and practitioners working with language models, while also highlighting the key challenges and opportunities in this emerging area. we hope our work can help foster further research in this field.