# TraceGuard: Fine-Tuning Pre-Trained Model by Using Stego Images to Trace Its User
## 1 Introduction  
Due to its powerful representation learning capability, without requiring an extensive number of labeled examples, the self-supervised learning (SSL) paradigm has been pervensively applied in a variety of vision tasks, including facial recognition [1], object detection [2], and object pose estimation [3]. The typical and prevalent SSL algorithms, for instance, SimCLR [4], MoCo v2 [5], and MAE [6], are used to achieve pre-trained models, and the models are fine-tuned by a much smaller amount of data to adapt to diverse downstream tasks. Apparently, the pre-trained models have great values. As the SSL intensively relies on data and computing resources, only tech giants can deploy well-trained pre-trained models in the cloud and provide MLaaS (machine learning as a service) [7] through charging users (e.g., Google Cloud and Microsoft Azure). Consequently, infringement leads to economic loss for the model owners. One case is thatsomeone steals the pre-trained model by using stealing attacks and then re-deploys the models to obtain illegal profit. Accordingly, a watermarking-based strategy for protecting the model's copyright is proposed: the model is watermarked, and once the model is suspected, the watermark will be extracted from the model to verify whether it is a pirate model [8; 9; 10]. The other case is that some individuals are authorized to use the pre-trained models; however, some of them unlawfully release the model or resell the model for illegal profit, as shown in Figure 1. Therefore, once a suspect model is discovered, it is necessary to trace the source of the model; more specifically, if the suspect model is pirated, it is necessary to attribute it to a specific authorized user who may be the pirate. Unfortunately, the method previously proposed to verify pirated models cannot trace the model, because the watermark is not uniquely embedded into individual models. This indicates that each model must be fingerprinted to link it with each authorized user. In addition, to reliably trace the model, the fingerprint must be robust to pruning, fine-tuning, and even model stealing attacks launched by the malicious user. In this paper, TraceGuard is proposed. To the best of our knowledge, it may be the first framework for tracing a pre-trained model to its user. One base aspect is that each individual authorized user is allocated a unique secret key. TraceGuard consists of three components: an encoder, decoder, and pre-trained model. The encoder and decoder apply a set of images, a set of secret keys, and steganography to implement the fingerprint. During the tracing pipeline, the encoder, decoder, and pre-trained model closely interact. First, the encoder and decoder learn to embed secrets and extract them again with a high enough accurate rate while the base pre-trained model is frozen; then, the base pre-trained model is fingerprinted through fine-tuning by using stego images embedded with a unique secret key. Due to the linking between the secret key and the authorized user, each fingerprinted model is related to each authorized user. Finally, stego images embedded with all secret keys are queried, and the secret key with the top extraction accuracy rate is searched to determine the most likely user or whether the model is independent. The model and code can be available at [[https://github.com/dong-0412/TraceGuard.git](https://github.com/dong-0412/TraceGuard.git)]([https://github.com/dong-0412/TraceGuard.git](https://github.com/dong-0412/TraceGuard.git)) (accessed on 15 October 2024). The main contributions of this work are summarized as follows:  
* TraceGuard is proposed. It is the first framework for tracing a pre-trained model to its user, ascertaining which authorized user illegally released the pirate model or verifying that the suspect model is independent. Figure 1: Diagram of tracing a pre-trained model. The model owner spends significant money and resources to obtain and deploy pre-trained models as an MLaaS (machine learning as a service) to attain legal profit.