United We Stand, Divided We Fall: Fingerprinting Deep Neural Networks via Adversarial Trajectories
Authors: Tianlong Xu
Chen Wang
Gaoyang Liu
Yang Yang
Kai Peng
Wei Liu
Venue: NeurIPS 2024 poster (2024)
Keywords: Deep neural network
intellectual property protection
model fingerprinting
adversarial sample.
Primary area: privacy

Abstract: In recent years, deep neural networks (DNNs) have witnessed extensive applications, and protecting their intellectual property (IP) is thus crucial. As a non-invasive way for model IP protection, model fingerprinting has become popular. However, existing single-point based fingerprinting methods are highly sensitive to the changes in the decision boundary, and may suffer from the misjudgment of the resemblance of sparse fingerprinting, yielding high false positives of innocent models. In this paper, we propose ADV-TRA, a more robust fingerprinting scheme that utilizes adversarial trajectories to verify the ownership of DNN models. Benefited from the intrinsic progressively adversarial level, the trajectory is capable of tolerating greater degree of alteration in decision boundaries. We further design novel schemes to generate a surface trajectory that involves a series of fixed-length trajectories with dynamically adjusted step sizes. Such a design enables a more unique and reliable fingerprinting with relatively low querying costs. Experiments on three datasets against four types of removal attacks show that ADV-TRA exhibits superior performance in distinguishing between infringing and innocent models, outperforming the state-of-the-art comparisons.

United We Stand, Divided We Fall: Fingerprinting Deep Neural Networks via Adversarial Trajectories
Tianlong Xu', Chen Wang', Gaoyang Liu"!, Yang Yang?, Kai Peng', Wei Liu'
Hubei Key Laboratoryof Smart Internet Technology, School of EIC.
Huazhong University of Science and Technology. "School of AI, Hubei University 1 [tianlongxu, chenwang, liugaoyang, pkhust, liuwei] J @ @hust.edu.cn, Pyangyang wwwggeuhhuuuu......
Abstract
In recent years, deep neural networks (DNNs) have witnessed extensive appli- cations, and protecting their intellectual property (IP) is thus crucial. As non- invasive way for model IP protection, model fingerprinting has become popular. However, existing single-point based fingerprinting methods are highly sensitive to the changes in the decision boundary, and may suffer from the misjudgment of the resemblance of sparse fingerprinting, yielding high false positives of innocent models. In this paper, we propose ADV-TRA, more robust fingerprinting scheme that utilizes adversarial trajectories verify the ownership of DNN models. Bene- fited from the intrinsic progressively adversarial level, the trajectory is capable of tolerating greater degree of alteration in decision boundaries. We further design novel schemes generate a surface trajectory that involves series of fixed-length trajectories with dynamically adjusted step sizes. Such a design enables more unique and reliable fingerprinting with relatively low querying costs. Experiments on three datasets against four types of removal attacks show that ADV-TRA exhibits superior performance in distinguishing between infringing and innocent models, outperforming the state-of-the comparisons.
Introduction
In recent years, deep neural networks (DNNss have witnessed extensive applications, such au- tonomous driving [1]. AIGC (2) and medical diagnosis (3). However, training a practical DNN model requires significant computational resources, data and time. For example, the training of GPT-3 took about 21 days with a cost of over 2.4 million dollars 4. It is thus essential to protect the intellectual property (IP) of DNN models, especially when the models face the risk of being exposed or stolen by the so-called model extraction attacks [5].
Current technologies protect model IP can be broadly categorized into two classes: model water- marking 6 7 and model fingerprinting g TO 11, While the former embeds the watermark into the model verify the identity of suspect models, it inevitably interferes in the training phase, which sacrifices the utility of the model or even introduces new security threats [3]. J. So the latest research trend has shifted toward model fingerprinting, which enables the extraction model's fingerprint without any modifications to the model itself.
A common practice of model fingerprinting methods is to generate batch of special ad- versarial samples near the decision boundary as fingerprinting samples for model verifica- tion [14 [2]. However, these single-point fingerprinting samples generated independently from each other, and are highly sensitive to the changes in the decision boundary, given their
*Corresponding Author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

inherent localized perspective. When the decision boundary changes, e.g., due to removal at- tacks [6], great number of such fingerprinting samples would become invalid (c.f. Figure lap.
Moreover, even two unrelated models may share similar portions of decision boundaries (i.e., the resemblance of sparse fingerprinting) [8]. There- fore, single-point fingerprints are more prone to incorrectly identifying an unre- lated (innocent) model as stolen, yield- ing a high false positive rate in the veri- fication process, as observed in [T0 [2].
Before Altering
After Altering
Before Altering 38- 82
After Altering
22 i8
Invalid !
28-
28 #
Class
38- Rp
Class 2
x 16
22
Class
In this paper, we propose ADV-TRA, a more robust fingerprinting method for DNN models. Instead of using single- point fingerprinting samples to identify the model, ADV-TRA exploits novel adversarial trajectories, each of which a chain of progressively adversarial sample representing adversarial level from weak to strong (c.f. Figure (Tb). The adversarial trajectory, incorporat- ing multi-level adversarial perturbations, could provide more precise localization of the decision boundary, which enables to accurately capture the alteration in the boundary induced by removal attacks, thus exhibiting a more robust model IP verification.
Class
Class
Single-point Fingerprinting Sample
18-68-8 Adverserial Trajectory
(a) Single-point Fingerprinting
(b) Trajectory Fingerprinting
Figure 1: Comparing single-point fingerprinting samples and our adversarial trajectories. As can be seen, when the decision boundary alters (from the solid black line to the dashed red line), (a) a portion of single-point fingerprinting samples near the boundary become invalid, while (b) the majority of samples in the trajectories remain effective.
Though the idea is simple, there are two major challenges. First, is difficult to control the trajectory precisely to pass through the decision boundary with fixed length (i.e., the number of samples in the trajectory). Fixed step sizes existing adversarial sample generation may result in too long or too short trajectories, yielding substantial querying times and unstable local optima, compromising effectiveness. To tackle this issue, we view each step size as an optimized variable and design several loss functions and strategies to enforce fixed-length trajectories with dynamically adjusted step sizes. This leads to smaller steps for samples near the boundary to focus on more subtle details, while larger steps for samples far away from the boundary quickly move towards the boundary.
Second, it is not easy to capture the global fingerprint features to avoid the misjudgment of the resemblance of sparse fingerprinting, since the trajectory across neighboring classes remains limited to features of a single decision boundary. To address this challenge, we propose to generate a surface trajectory that comprises numerous adversarial trajectories orderly across multiple classes, which can offer a more comprehensive representation of the decision surface (instead of a single boundary), thereby significantly reducing the false positives of innocent models.
Our major contributions can be summarized as follows:
We propose ADV-TRA, a more robust fingerprinting scheme that utilizes adversarial trajectories for model verification. Benefited from the intrinsic progressively adversarial level, the trajectory is able tolerate greater degree of alteration in decision boundaries.
We design novel schemes to generate a surface trajectory that involves a series of fixed-length trajectories with dynamically adjusted step sizes. Such design enables more unique and reliable fingerprinting with relatively low querying costs.
We conduct extensive experiments on three datasets against four types of removal attacks. Experimental results show that ADV-TRA exhibits superior performance in distinguishing between infringing and innocent models, outperforming the state-of-the- comparisons,
HHere, the term "progressively adversarial samples" refers to a series of samples with progressively increasing adversarial perturbations, rather than the adversarial sample in the strict sense.

---

2 Preliminary
2.1 Adversarial Samples
Given target model, the goal of adversarial samples is deceive the model into making incorrect predictions [19 20] In the context of a classification model represented as f. an adversarial sample, ft
is crafted to perturb clean sample with ground-truth label y in such way that: (1) the perturbation d is kept small; and (2) the predicted class for is altered. This can be formalized as:
min | =1|
(() + yb
where ||-|| is a distance metric, e.g., l2 distance. In this way, the clean sample x and its perturbed counterpart may appear nearly identical to human observers, but the target model perceives them as entirely distinct entities.
2.2 Model Fingerprinting
As non-invasive way validate the ownership of a DNN model, model fingerprinting seeks to detect some fingerprints which is normally the model's decision boundary. Generally, the procedure of model fingerprinting involves two main steps: fingerprint extraction and fingerprint verification:
Fingerprint Extraction. Suppose a model owner who trains a source model, his goal is to generate number of fingerprinting samples that can uniquely characterize the decision boundaries. To this end, model fingerprinting borrows ideas from adversarial samples, by generating a set of adversarial examples that are close to the decision boundary, paired with their corresponding predicted labels, as the fingerprinting samples [ [4]. More specifically, given a normal sample æ and its ground truth label y. the fingerprinting sample Tpp can derived from optimizing towards Upp- where Ujp is the fingerprinting label that differs from Therefore, for source model fere, the corresponding fingerprinting sample zip just crosses the decision boundaries, such that f.re * (x))
Fingerprint Verification. During the verification phase, when dealing with a suspect model Saur. the model owner can determine the ownership by querying with the set of fingerprinting samples Drr = {(T }. Based on the output results, the owner selects a testing metric Metric(-), e.g., the accuracy, and computes Metric(fsu. sus Djp). By comparing this result with threshold, the owner can make the final judgment regarding any potential ownership infringement.
2.3 Removal Attacks
Removal attacks are designed to invalidate fingerprints by tampering with the source model for- We denote model that has been processed by removal attacks frme, with the goal being alter the model such that frm * fom (x.p). There are several ways to launch removal attacks, e.g., by model modification (fine-tuning (21). pruning 22 23]. adversarial training 24 (5)). model extraction [26] 5) Removal attacks pose a significant threat to model fingerprinting. For example, when the source model undergoes adversarial training, a considerable portion of these fingerprints
may change (i.e., fingerprinting samples become obsolete in the verification phase) 9 12 16).
3 Problem Formation
We consider a typical scenario consisting of two par- ties: the model owner (defender) and the attacker, as shown in Figure 2 The model owner trains well- performed model (1.e., the source model) and deploys it as a cloud service or client-sided software. The attacker attempts to steal the source model in either black-box white-box ways.
Extract
Source Model
Fingerprints
Verify
steal
Owner
Process
Fine tenituunng Pruning
Deploy
API
Attacker's Strategy. We first consider a strong white-box attacker, who can get access to the en- tire information of the source model, including the model structure and all inner parameters, well as
Attacker
Removal Attacks
Stolen Model
Figure Illustration of model stealing and verification.
3

---

sufficient auxiliary date through server intrusion or cavesdropping on communication channels. Once the attacker acquires the source model, he could leverage removal attacks to modify the model from either the model structure or parameters, so as to evade the IP infringement detection.
We also consider a black-box attacker, who only has the ouputt prediction interface (a.k.a. API) to the source model. By leveraging an auxiliary dataset, the attacker is able to duplicate the source model by multiple querying, e.g., model extraction attack, and utilizing the querying results to train a substitute model from scratch.
Defender's Strategy. The goal of the model owner is to verify whether a deployed suspect model is derived from the source model. The model owner naturally has the white-box access to the source model, but is assumed to only have the black-box access to the suspect mode and observe its prediction for any input. In specific, he extract set of fingerprinting samples from the source model and assess their performance on the suspect model. If the number of fingerprinting samples that validate successfully exceeds a threshold, then it can be concluded that the suspect model has been illegally stolen from the source model.
From the perspective of the defender, we aim to design DNN fingerprinting method satisfying the following properties: (1) Uniqueness. Uniqueness is crucial for ensuring the reliability and fidelity of fingerprinting scheme. Fingerprint extraction should comprehensively and fully capture features, so as to avoid local resemblance-induce scheme needs to be resistant removal attacks, even under relatively radical attacking strategies. (3) Efficiency. Considering the API may charge per query and excessive queries may raise the attacker's suspicion, the fingerprinting method may require few query samples as possible.
misjudgment. (2) Robustness. A robust fingerprinting
Design of ADV-TRA
We present the workflow of ADV-TRA in Figure 3 In the trajectory generation phase, given the source model f.re and base sample 2a of class co. ADV-TRA first initializes the trajectory from one class cu towards another class C1. Subsequently, the trajectory is fine-tuned for probing the decision boundary and then simply bilateralized to enforce cross-class chain of progressively adversarial samples. Finally, we introduce the surface trajectory Tam which traverses among multiple classes to fingerprint the entire decision surface. In the verification phase, we use Tawy to compute the mutation rate Tmur for suspect model s and determine whether J is stolen from fox.
4.1 Trajectory Generation
We use adversarial trajectories to fingerprint the source model's decision boundaries. A naive way of generating the adversarial trajectory is to record all intermediate products using adversarial sample algorithms like Basic Iterative Method (BIM) [29]. However, this approach does not consider the distance from a base sample To to its corresponding decision boundary, and often uses a fixed step size, which would cause too few (or many) trajectory samples, resulting in lower effectiveness (or excessive queries, even getting stuck in local optima and unable to cross the decision boundary).
Therefore, we propose an adaptive step-size adversarial trajectory generation algorithm that can achieve fixed length between any two classes by dynamically adjusting the step size, by involving the following four steps: (1) trajectory initialization; (2) boundary probing; (3) trajectory bilateralization; and (4) trajectory connection.
Trajectory Initialization. Given source model face a base sample zo (with its ground truth label Yor), and target label Vigr a trajectory is initialized just like the process of generating adversarial samples with gradient descent. Instead of using fixed step size, we view the step size in all rounds of iterations $
{80: . 81-1) } as an optimizable variable and generate the initial trajectory Teu:
'The auxiliary data can be collected from the wild or internet with the same distribution as the task do- main (26) or just synthesized using image generation techniques 28
*We consider the most difficult and practical settings where the attacker deploys the stolen model on the cloud and only the prediction interface is provided for user's queries. Our method can be easily adapted to white-box cases where richer information such the outputs of each layer of the model can be obtained.

---

Trajectory Generation
Trajectory Verification
Truff
Decison Boundary
Co
Vauff
Co
C1
Query
GOOD
0
Suspect Model
Sample
0000000000 Ca
Infringe
Given base sample and the model
Bilateralize Trajectory
Connect Trajectory
Yes
Compute
Trajectory
Boundary
T
Figure 3: Exemplary illustration of ADV-TRA pipeline (the length of Ti 1 3; the length of Taurf 2/m, exclusive of co). A darker color of the trajectory sample indicates higher level of adversarial strength towards the target class.
Tin = T(f Too Vige 6) = {Io 21,.....2.1 I++1 2; 8 sign( ( VV(ffrc Fzs Vigi)), for i = 0, 1, ......,--
where Ln Ji, Mige) is the loss function to guide the prediction towards yıgr and is the length of Tim-
Boundary Probing. Recall that we use Tie of length probe the decision boundary, so it is crucial to scale Tim such that precisely reaches the decision boundary, i.c., the last sample in Tiu happens to predicted as Yıgr (whereas the prior samples are not):
(xx) Yige if i 1,2, ................................ 1-1
fw
Yigt if i = I.
To this end, we propose simple length control strategy, which scales Tir by adjusting the step sizes s with length control variable ate (0 < 0% < 1). Specifically, given a trajectory, we first check whether the requirements in Eq. (3) are satisfied, and further determine whether increase or decrease the step size. If the trajectory reaches the decision boundary carly (late), we need to reduce (resp. enlarge) the step size of all steps by multiplying length control factor One (resp. 1 1/0se).
In order record more details near the decision boundary, the step size 8; is expected to decay when current I; are closer to the decision boundary, i.e., 8i41 < 8;. For this purpose, we define the brake loss function Lirk as:
Lork = (Orbi - -8441)²
(4)
1=0
where Obrr (0 < Chink 1) is the brake factor that controls the proportional relationship of two adjacent step sizes.
Moreover, ensure the internal forward trend within the trajectory, i.e., 24+1 is closer to the decision boundary of Vigr than 2,, each step 8; should greater than O. So we define the forward loss function Lynd to prevent any recession in the trajectory:
Csva(s) Σ 1(s: < -4.
(5)
where I[()) the indicator function.
Hence, training the step sizes involves the above two optimization objectives:
Min L Lank + Lymd(s).
(6)
We optimize s using the gradient descent algorithm. Specifically, the optimization terminates when satisfying the following conditions:(1) The last sample in Tin is predicted Vga while the prior samples are not, as described in Eq. 3 (2) The step size for the next sample 8;41 is smaller than the step size 8; for the previous sample, i.e., 8i+1 < 8i.
Trajectory Bilateralization. Given the optimized Timts we next extend bilateral trajectory To. This is necessary due to the random variation in the perturb direction of the decision boundary
5

---

when subjected to removal attacks, and the trajectory is preferred to evenly attend to both sides of the decision boundary keep consistent measuring scale for both sides. We thus utilize mirroring operation, centered on Tt, to make up the other half of the trajectory, using the reversed step sizes $ 81- 80). and a bilateral trajectory The can thus be generated as:
To: = T(rre: Tu. Bigr: s U = { .22....,,....... ),
(7)
where The crosses to the other side of the decision boundary starting from 21, and is able to locate the boundary well with the intrinsic progressively adversarial samples.
It is worth noting that the bilateral trajectory may pass through more than two classes. Such a case does not lower the efficacy of the trajectory; instead, it serves as a special feature regarding to the model's 's decision surface, which makes our fingerprinting more unique.
Trajectory Connection. So far ep and e1 by a bilateral trajectory Th: Too-r However, a decison boundary alone, as discussed before, not sufficient to fully capture the fingerprint of the model. While unrelated models may share some similarities along the decision boundary, their decision surfaces would almost never be the same [8). Therefore, we introduce the surface trajectory Tawef to fingerprint the entire decision surface, which merges all the bilateral trajectories to traverse through all classes. This can be done by simply connecting the bilateral trajectories in order, i.e., the last sample in 7 7 is the first sample of T -cs (c.f. Figure 30.
fingerprint specific decision boundary between two classes
It is noticed that, it may be costly to involve all the bilateral trajectories, especially for complex models. Therefore, a more practical way for large models is to randomly select m bilateral trajectories
for the surface trajectory generation (c.f. Algorithm in Appendix (B):
Taury Tha T
U UT 00 ... U Th
Our experiments indicate that a small number of m = 10 is sufficient to fingerprint the 100-class CIFAR-100 and 1000-class ImageNet models.
4.2 Trajectory Verification
In the verification phase, the model owner can query the given suspect model fun with the surface trajectory Tamey We then calculate the mutation rate of Tamy from fire to fius by:
Σ
(f: + (xx)).
Tsuryl
ILETay
Here, P'mr reflects the proportion of samples in the trajectory whose predictions fas differ from face Theoretically, an infringement model is expected to possess a lower r Ymur than an innocent model, since it originates from the source model Jse and shares more similar decision surface with it. The defender can determine whether Tawo is also the exclusive trajectory of sau by comparing the obtained Tmme and threshold Thr, mut (0.5 in our case, equivalent to random guess). In practice, we calculate the detection rate of number of adversarial trajectories (i.e. surface trajectories) on the suspect model to make the final infringement judgment.
5 Experiments
We use three widely-used benchmark datasets, namely, CIFAR- 50, CIFAR-100 [30], and
ImageNet (31), in model fingerprinting domain to evaluate the performance of our ADV-TRA. We compare our approach with four existing model fingerprinting techniques, including CoRt [I], IPGuard [14) CAE [9]. and UAP [00] We also evaluate the robustness of ADV-TRA against four
types of removal attacks (fine-tuning (6] 32), pruning [33], adversarial training (34), and model
extraction attacks [26] 27 35)) across multiple model architectures. In Appendix C we furthermore list details on experiment setups, including dataset, models, evaluation metrics, and implementation
details.
Moreover, we reveal how removal attacks affect the effectiveness of fingerprints by altering the decision surface in Appendix [.I] Additionally, we compare simple approach (trajectory spanning only two categories) with trajectories spanning multiple categories (see Appendix D.2), and conduct a series of ablation studies to analyze the impact of parameters of ADV-TRA (see Appendix D.3). At last, we demonstrate how the progressiveness of the trajectory enables to locate the decision boundaries (see Appendix D.4).

---

Table 1: Main results on CIFAR-10 dataset. P-20% denotes model pruning with pruning rate p = =0.2; Ady-0.001 represents adversarial training with budget c = 0.001. Fingerprint detection rate in bold indicates the best performance. For positive models, higher fingerprint detection rate is preferred, suggesting better ability to verify IP infringement. In contrast, negative models are expected to yield a lower detection rate, avoiding false verification.
CIFAR-10
Fingerprint Detection Rate
Model Type
Aecuracy
Free
IPGuard
CAB
UAP
Ours
Source Model
0.874
0.000
0.000 1: 0.000
1.000
0.000
1,000 4: 0.000
1.000
0.000
1.000 0.000
FTLL
0.868 + 0.001
0.043 + 0.002
0,978 0.001
0,956 + 0.002
0.958
0.001
0.952 0.000
FTAL
0.867 4: 0.001
0.058 4: 0.002
0.942 +: 0.002
0,934 # 0.001
0,954 +: 0.002
0.950 +: 0,000
RTLL
0.865 + 0,000
0.030 + 0.001
0.917 + 0.001
0,921 + 0.001
0.941 = 0.002
0.947 : 0.001
RTAL
0.864 +: 0.000
0.143 4: 0.001
0.621 + 0.001
0.663 4: 0.002
0.719 + 0.001
0.763 1: 0.002
Positive
P-20%
0.872 +: 0.001
0.035 + 0.001
0.895 + 0.001
0.924 + 0.000
0.346 = 0.001
0.958 0,000
Suspect
Medel
P-40%
0.869
0.000
0.084
0.001
0.783
0.001
0,894 +: 0.001
0.934
0.001
0.945
P-80%
0.825
0.002
0.257 +: 0.003
0.587
0.001
0.628 : 0.001
0.634
0.002
0.678 0.001
Adv-0.001
0.859
0.001
0,238 + 0.001
0.423
0,000
0,421 + 0.001
0.513
0.001
0.758 0.001
Adv-0.01
0.863 + 0.000
0.394 + 0.001
0.162 + 0.001
0,138 + 0.001
0.348 = 0.000
0.612 0.001
Adv-0,1
0.865 + 0.001
0,205 + 0.002
0.184 + 0.001
0,256 + 0.002
0.402 = 0.002
0.628 :: 0.001
SAST
0.879 +: 0.000
0.714 1: 0.000
0.631 + 0.001
0.561 +: 0.001
0.412 + 0.000
0.143 1: 0.000
Negutive
SAIYT
0.876 + 0.001
0.722 + 0.001
0.499
+ 0.001
0.422 + 0.002
0.382 = 0.002
0.126 0.000
Suspect
Model
DAST
0.875
0.002
0.762 4: 0.001
0.454
0.001
0,389 +: 0.002
0.323
0.001
0.124
DADT
0.874 0.003
0.787 + 0.001
0.357
0.002
0.219 + 0.001
0.313 0.001
0.116 0.001
5.1 Main Performance
We first validate ADV-TRA on 100 suspect models for each source model, including 50 positive models under various attacks and 50 negative models on CIFAR-10 10 dataset. In our paper, the positive models originate from the source model but have been processed by removal attacks, including model pruning, adversarial training, and fine-tuning (Fine-Tune Last Layer (FTLL), Fine-Tune All Layers (FTAL), Retrain Last Layer (RTLL), and Retrain All Layers (RTAL)). The negative models come from four ways: (1) SAST: the same architecture as victim model and the same training data (Note that model initialization and training randomness can still lead to differences from the source model); (2) SADT: the same architecture as victim model but different training data; (3) DAST: different architecture but the same training data; (4) DADT: a different architecture and different training data (For more details, please refer to Appendix C The results are shown in Table 1
is evident that our ADV-TRA outperforms other methods in most cases (12 out of 14), achieving high detection rates for positive models while maintaining low detection rates for negative models. Only ADV-TRA is able to clearly separate the two types models, which is adequate to guarantee the subsequent verification without any false positives. Its corresponding lower bound of positive models is 0.612 (corresponds to Adv-0.01), which is still much higher than the upper bound of negative models (0.143, corresponds to SAST). In contrast, other methods fail to distinguish between some of the positive models and negative models. It is also surprising to see that our method has a low false positive rate: the detection rate of ADV-TRA for negative models is much lower than the three baselines. This is attributed to our design of the chain of progressively adversarial samples, which is able to simultaneously capture more characteristics at various levels to avoid false positives.
One intermediate product average mutation rate 7 mit reflects the dissimilarity in the decision boundary of source model and suspect model. It can be observed that removal attack causing higher Fooir brings about lower fingerprint detection rate. For example, three fine-tuning methods (FTLL, FTAL, and RTLL) only result in Fime of less than 0.06, corresponding to relatively high fingerprint detection rate of approximately 0.95. Negative models have a high Fin (over 0.7) since their decision boundaries differ more significantly from the source model.
To further validate the ability of the fingerprinting samples to accurately distinguish between positive and negative models, we present the ROC curve and the distribution of all suspect models in Figure 4a and Figure 4b Only ADV-TRA gets an AUC = 1, significantly outperforming other approaches. As can be seen, CoRt, IPGuard, and CAE all do not readily distinguish between the two types of suspect models, exhibiting strong cross-model transferability. Especially for general adversarial samples (CoRt), their transferability to negative models is more obvious, leading to high detection rate. For ADV-TRA, the detection rate of the negative model is quite low, approximately 0.15. This also serves as empirical evidence for our analysis that our trajectories have an extremely low false positive rate.
7

---

Desocction ROZS
Detection R35
0.
Calt MMC E.M.
FILL
FILL
32-
UNT VAL --M1
FPR
Epects
Epocts
(a)
(b)
(a) CIFAR- 100
(b) ImageNet
Figure 4: The ROC curve (a) and the distribu- tion (b) of fingerprint detection rate different suspect models on CIFAR-10.
Figure 5: Fingerprint detection rate vs. fine- tuning epochs. The gray area represents the range of fingerprint detection rate for negative models.
Table Comparison of different fingerprinting techniques. T1%F denotes True Positive Rate when the False Positive Rate is at 1%.
CIFAR-10
CIFAR-100
ImageNet
Method
TI&F
TI0SF
AUC
TI%F
T10%F
AUC
TI%F
T10%F
AUC
CoRt
0.26
0.32
0.7u
0.29
0.38
0.72
0.20
0.28
0.59
IPGuard
0.30
0.46
0.77
0.43
0.63
0.80
0.27
0.40
0.69
0.64
0.51
0.70
0.39
0.76
UAP
0.76
0.78
0.91
0.77
0.83
0.92
0.40
0.56
0.85
Ours
1.00
1.00
1.00
1.00
1.00
1.00
0.88
0.90
0.96
To evaluate the universality of our method, we conduct experiments on three datasets as illustrated in Table It can be found that our method extracts the most strongly detectable fingerprints, which achieves AUC of 1.0 on CIFAR-10 and CIFAR-100. In all cases, ADV-TRA has better AUC, T1%F, and T10%F than other fingerprinting methods. For the models trained ImageNet, conjecture that classification models with more classes have more complex decision boundaries. When subjected removal attacks, the model's functionality (such as predicting the probability of each class) is more prone to changes, resulting in less robustness compared to models with fewer classes.
5.2 Robustness Against Removal Attacks
Impact of Fine-tuning. We first employ a more radical fine-tuning strategy (i.e., with a higher learning rate and many more epochs) to better simulate real-world removal attacks. Figure shows the results on the CIFAR-100 and ImageNet datasets. For CIFAR- 100 dataset, we can see that the FTLL, FTAL, and RTLL models are hardly affected and consistently demonstrate a high fingerprint detection rate (around 0.96), whereas the detection rate for the RTAL model steadily decreases, dropping to 0.72 after 50 epochs. As for ImageNet dataset, which has more classes, the detection rate of the positive models experiences a more significant decline, dropping from 1.0 to below 0.5 within the first 12 fine-tuning epochs. After that, the FTLL, FTAL, RTLL models begin to stabilize, but the RTAL model continues declining at a high rate. When the epoch reaches 50, the detection rate of the RTAL model is 0.290, still significantly higher than the upper bound of the negative models (0.143). In general, fine-tur tuning is not considerable threat to ADV-TRA.
Impact of Pruning. Next, we test the impact of pruning, by varying the pruning rate from 10% to 90%. Figure illustrates the fingerprint detection rate and model accuracy for each pruning level on CIFAR-100 and ImageNet. We can clearly see that the detection rate has strong correlation with the model accuracy. For CIFAR-100 dataset, as the pruning rate increases from 10% to 60%, the fingerprint detection rate decreases slightly from 0.986 0.920. When the pruning rate exceeds 70%, the fingerprint detection rate experiences a sharp decline (from 0.832 to 0.367). The results on ImageNet dataset also reveal two declining lines with increasing curvature. Even at pruning rate of 90%, the fingerprinting samples could no longer reliably differentiate between positive (0.114 0.064) and negative (0.001~00.1 143) models. However, when many fingerprinting samples become invalid, the model accuracy also suffers a sharp decrease. In practice, considering the model's basic functionality, attacker would implement pruning attack of such strength.
Impact Adversarial Training. To investigate how adversarial training affects the effectiveness of fingerprinting samples, vary the perturbation budget from 0.001 to 1.0 and attack the source

---

1,
10
DUy
Accurncy
Deteccion
64
21N00000 wew.aaoaeooeeett000000 Pruring
1090 Wise wwwdi 200230000
02
30
Prusing Rote
Epoctts
(a) CIFAR-100
(b) ImageNet
(a)
(b)
Figure 6: Fingerprint detection rate (blue line)
Figure 7: The results for adversarial training on CIFAR-100, 100, varying (a) perturbation budget and (b) training epochs.
and model accuracy (red line) under pruning attacks.
Table 3: Performance under three model extraction attacks.
Flaccoe
Dataset
Accuracy
IPGuard
CAE
UAP
Ours
CIFAR-100
0.721
0.70
0.76
0.85
0.90
PRADA
ImageNett
0.743
0.59
0.63
0.72
0.83
CIFAR-100
0.734
0.69
0.84
0.87
0.93
Knockoff
ImageNet
0.764
0.62
0.68
0.75
0.86
CIFAR 100
0.723
0.35
0.38
0.51
III.
ImageNet
0.735
0.29
0.27
0.41
0.58
model for 20 epochs (e = 1.0 is quite large for adversarial training but we still test with this value). Figure 7 illustrates the results for CIFAR- 100. Interestingly, we observe trend where the detection rate first decreases (from 1.0 to 0.60) and then increases (from 0.60 to 0.76), as e rises from 0.001 to 1.0. A perturbation budget of 0.01 achieves the best attack result, rendering 40% of fingerprinting samples invalid. This implies that an attacker does not necessarily have to impose adopt higher c to launch stronger attacks. For 0.01, our approach successfully detects 60% of trajectories, which are about 3 times that of IPGuard and CAE, and 1.5 times that of UAP, demonstrating strong robustness against adversarial training. Additionally, adversarial training in Figure [6] The first 5 epochs witness big drop of fingerprint detection rate. As the number of training epochs continues to increase, the fingerprint detection rates of these models remains almost unchanged. We speculate that the models have already attained the e a-avvrrraiial robustness during the first few training epochs. As such, subsequent training does not further decrease the effectiveness of fingerprinting samples.
record the results during 50 epochs of
Impact of Model Extraction Attacks. Since the fingerprint detection rate alone cannot reflect the gap between positive and negative models, here, we calculate the Fl-score of correctly identified fingerprint samples under PRADA and Knockoff attacks. From the results in Table 3 we can observe that our method achieves the best results, leading the second-place method (UAP) by nearly 0.1 under all attacks. It is worth noting that better extraction performance (higher test accuracy) is inevitably accompanied by substitute model that learns a decision surface more similar to the source model. This also leads to a better transfer of the fingerprint space (a part of the decision surface) as well. Moreover, as data-free model stealing attack with access to only hard labels, HL distills a relatively more dissimilar substitute model compared to the source model under less prior knowledge, resulting in the lowest F1 Fl-sc among the four fingerprinting methods.
6 Conclusion
In this paper, we have presented ADV-TRA, a robust fingerprinting scheme that leverages adversarial trajectories to fingerprint the DNN model. ADV-TRA generates a chain of samples with varying levels of adversarial perturbation, and further extends to the surface trajectory that involves a series of fixed-length trajectories with dynamically adjusted step sizes. By doing so, the alteration in the decision boundary can be captured more accurately, and the misjudgment caused by local resemblance from innocent models be significantly reduced. Extensive evaluation results on three datasets demonstrate that ADV-TRA is able to defend against various removal attacks even under severe attack intensities, exhibiting greatly superior robustness and lower false positive rates compared to existing state-of-the-ar fingerprinting methods.

---

Acknowledgement
This work was supported in part by the National Natural Science Foundation of China under Grants 62272183 and 12171189; by the National Key Technology R&D Program of China under Grant 2022YFA091 1800; by the Key R&D Program of Hubei Province under Grants 2024BAA008, 2024BAA011, 2024BAB016, 2024BAB031, 2023BAB074; by the special fund for Wuhan Artificial Intelligence Innovation under Grant 2022010702040061 and by the Open Project Funding of the Key Laboratory of Intelligent Sensing System and Security, Ministry of Education under Grant KLISSS202401.
References
[1] Guibiao Liao, Jiankun Li, and Xiaoqing Ye. V1m2scene: Self-supervised image-text-lidar learning with foundation models for autonomous driving scene understanding. In Proceedings of AAAI, pages 3351-3359, 2024.
[2] Jiun Tian Hoe, Xudong Jiang, Chee Seng Chan, Yap-Peng Tan, and Weipeng Hu. Interactd- iffusion: Interaction control in text-to-image diffusion models. In Proceedings of IEEE/CVF CVPR, 2024.
[3] Jun Shi, Shulan Ruan, Ziqi Zhu, Minfan Zhao, Hong An, Xudong Xue, and Bing Yan. Predictive accuracy-based active learning for medical image segmentation. In Proceedings of IJCAI, 2024.
[4] Jonathan Vanian and Kif Leswing. but the costs can be extraordinary. chatgpt- and- generative al are- but-butt-tttt bewe html 2023-03-13.
Chatgpt and generative ai are booming, Sttps://wwit cnbc. www
[5] Yuzheng Wang, Zhaoyu Chen, Dingkang Yang. Pinxue Guo, Kaixun Jiang, Wenqiang Zhang, and Lizhe Qi. Out of thin air: Exploring data-free adversarial robustness distillation. In Proceedings of AAAI, pages 5776-5784 2024.
[6] Yossi Adi, Carsten Baum, Moustapha Cisse, Benny Pinkas, and Joseph Keshet. Turning your weakness into strength: Watermarking deep neural networks by backdooring. In Proceedings of USENIX Security Symposium, pages 1615-1631, 2018
[7] Lixin Fan, Kam Woh Ng, and Chee Seng Chan. Rethinking deep neural network ownership verification: Embedding passports to defeat ambiguity attacks. Proceedings of NeurIPS, 2019.
[8] Hengrui Jia, Christopher Choquette Choo, Varun Chandrasekaran, and Nicolas Papernot. Entangled watermarks as a defense against model extraction. In Proceedings of USENIX Security Symposium, pages 1937-1954, 2021.
[9] Nils Lukas, Yuxuan Zhang, and Florian Kerschbaum. Deep neural network fingerprinting by conferrable adversarial examples. In Proceedings of ICLR, 2021.
[10] Zirui Peng, Shaofeng Li, Guoxing Chen, Cheng Zhang, Haojin Zhu, and Minhui Xue. Finger- printing deep neural networks globally via universal adversarial perturbations. Proceedings of IEEF/CVF CVPR, pages 3430--13339 2022.
[11] Jialuo Chen, Jingyi Wang, Tinglan Peng, Youcheng Sun, Peng Cheng, Shouling Ji, Xingjun Ma, Bo Li, and Dawn Song. Copy, right? A testing framework for copyright protection of deep learning models. Proceedings of IEEE S&P, pages 824-841, 2022.
[12] Jiyang Guan, Jian Liang, and Ran He. Are you stealing my model? Sample correlation for fingerprinting deep neural networks. In Proceedings of NewrlPS, pages 36571-36584 2022.
[13] Xudong Pan, Mi Zhang, Yifan Yan, Yining Wang, and Min Yang. Cracking white-box dnn watermarks via invariant neuron transforms. Proceedings ACM SIGKDD, pages 1783-1794, 2023.
10

---

[14] Xiaoyu Cao, Jinyuan Jia, and Neil Zhenqiang Gong. IPGuard: Protecting intellectual property of deep neural networks via fingerprinting the classification boundary. Proceedings of ACM ASIACCS, pages 14-25, 2021.
[15] Han Qiu, Yi Zeng, Shangwei Guo, Tianwei Zhang, Meikang Qiu, and Bhavani Thuraising- ham. Deepsweep: An evaluation framework for mitigating dnn backdoor attacks using data augmentation. In Proceedings of ACM ASIACCS, pages 363-377, 2021.
[16] Wei Zong. Yang- Wai Chow. Willy Susilo, Joonsang Back, Jongkil Kim, and Seyit Camtepe. IPRemover: A generative model inversion attack against deep neural network fingerprinting and watermarking. In Proceedings of AAAI, pages 7837-7845, 2024.
[17] Gowthami Somepalli, Liam Fowl, Arpit Bansal, Ping Yeh-Chiang Yehuda Dar, Richard Bara- niuk, Micah Goldblum, and Tom Goldstein. Can neural nets learn the same model twice? Investigating reproducibility and double descent from the decision boundary perspective. In Proceedings of JEEE/CVF CVPR, pages 13699-13708, 2022.
[18] Hengrui Jia, Hongyu Chen, Jonas Guan, Ali Shahin Shamsabadi, and Nicolas Papernot. A zest of lime: Towards architectur--iddeeendnnt model distances. In Proceedings of ICLR, 2022.
[19] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversar- ial examples. In Proceedings of ICIR,, 2015.
[20] Jie Wan, Jianhao Fu, Lijin Wang, and Ziqi Yang. BounceAttack: A query-efficient decision- based adversarial attack by bouncing into the wild. In Proceedings of IEEE S&P, 2024.
[21] Honglin Li, Chenglu Zhu, Yunlong Zhang, Yuxuan Sun, Zhongyi Shui, Wenwei Kuang, Sunyi Zheng, and Lin Yang. Task-specific fine-tuning via variational information bottleneck for weakly-supervised pathology whole slide image classification. Proceedings of IEEE/CVF CVPR, pages 7454-7463, 2023.
[22] Ning Liu, Xiaolong Ma. Zhiyuan Xu, Yanzhi Wang, Jian Tang, and Jieping Ye. Autocom- press: An automatic dnn structured pruning framework for ultra-high compression rates. In Proceedings of AAAI, pages 1876-4883, 2020.
[23] Alireza Ganjdanesh, Shangqian Gao, and Heng Huang. Jointly training and pruning cons via learnable agent guidance and alignment. In Proceedingy of IEEE/CVF CVPR, 2024.
[24] Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! In Proceedings of NeurlPS, 2019.
[25] Xiaoling Zhou, Wei Ye, Zhemg Lee, Rui Xie, and Shikun Zhang. Boosting model resilience via implicit adversarial data augmentation. In Proceedings of IJCAI, 2024.
[26] Mika Juuti, Sebastian Szyller, Samuel Marchal, and N Asokan. PRADA: Protecting against DNN model stealing attacks. In Proceedings of IEEE S&P, pages 512-527, 2019.
[27] Tribhuvanesh Orekondy, Bernt Schicle, and Mario Fritz. Knockoff nets: Stealing functionality of black-box models. In Proceedings of IEEE/CVF CVPR, pages 4954 2019.
[28] Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, and Anan- thram Swami. Practical black-box attacks against machine learning. In Proceedings of ACM ASIACCS, pages 506-519, 2017.
[29] Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. In Proceedings of ICLR, 2016.
[30] Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar- 10 (canadian institute for advanced research), 2010.
[31] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In Proceedings of IEEE/CVF CVPR, pages 248-255, 2009.

---

[32] Kang Yang, Run Wang, and Lina Wang. Metafinger: Fingerprinting the deep neural networks with meta-training, In Proceedings of IJCAI, 2022.
[33] Song Han, Jeff Pool, John Tran, and William Dally. Learning both weights and connections for efficient neural network. In Proceedings of NeurlPS, 2015.
[34] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu, Towards deep learning models resistant to adversarial attacks. In Proceedings of ICLR, 2018.
[35] Sunandini Sanyal, Sravanti Addepalli, and R Venkatesh Babu. Towards data-free model stealing in hard label setting. In Proceedings of IEEE/CVF CVPR, pages 15284-15293, 15293, 2022.
[36] Nils Lukas, Edward Jiang, Xinda Li, and Florian Kerschbaum. Sok: How robust is image classification deep neural network watermarking? In Proceedings of IEEE S&P, pages 187-804, 2022.
[37] Zehao Tian, Zixiong Wang, Ahmed M Abdelmoniem, Gaoyang Liu, and Chen Wang. Knowl- edge representation of training data with adversarial examples supporting decision boundary. IEEE Transactions on Information Forensics and Security, 18:4116 4127, 2023.
(38) Ziheng Huang. Boheng Li, Yan Cai, Run Wang. Shangwei Guo, Liming Fang, Jing Chen, and Lina Wang. What can discriminator do? Towards box-free ownership verification of generative adversarial network. In Proceedings of IEEE/CVF CVPR, pages 5009-5019, 2023.
[39] Thibault Maho, Teddy Furon, and Erwan Le Merrer. FBI: Fingerprinting models with benign inputs. IEEE Transactions on Information Forensics and Security, 18:5459-5472, 2023.
[40] Xudong Pan, Yifan Yan, Mi Zhang, and Min Yang. MetaV: A meta-verifier approach to task-agnostic model fingerprinting. In Proceedings of ACM SIGKDD, pages 1327-1336, 2022.
[41] Yue Zheng, Si Wang, and Chip-Hong Chang. A dnn fingerprint for non-repudiable model ownership identification and piracy detection. IEEE Transactions on Information Forensics and Security, 17:2977-2988) 2022.
[42] Tian Dong, Shaofeng Li, Guoxing Chen, Minhui Xue, Haojin Zhu, and Zhen Liu. RAI2: Responsible identity audit governing the artificial intelligence. In Proceedings of NDSS, 2023.
[43] Yuchen Sun, Tianpeng Liu, Panhe Hu, Qing Liao, Shouling Ji, Nenghai Yu, Deke Guo, and Li Liu. Deep intellectual property: A survey. CoRR, arXiv: 2304.14613, 2023.
[44] Jangho Kim, Jayeon Yoo, Yeji Song, KiYoon Yoo, and Nojun Kwak. Finding efficient pruned network via refined gradients for pruned weights. In Proceedings of ACM MM, pages 9003-9011, 1, 2023.
[45] Yizhen Yuan, Rui Kong, Shenghao Xie, Yuanchun Li, and Yunxin Liu. Patchhackdoor: Back- door attack against deep neural networks without model modification. In Proceedings of ACM MM, pages 9134-9111 9142,, 2023.
[46] Meiqi Wang, Han Qiu, Tianwei Zhang, Meikang Qiu, and Bhavani Thuraisingham. Mitigating query-based neural network fingerprinting via data augmentation. ACM Transactions on Sensor Networks, 2023.
[47] Zhengyuan Jiang, Jinghuai Zhang, and Neil Zhenqiang Gong. Evading watermark based detection ai-generated content. In Proceedings of ACM CCS, pages 1168-1181, 2023.
[48] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. Journal of Machine Learning Research, 9(86):2579-2655 2008
[49] Roland S Zimmermann, Wieland Brendel, Florian Tramer, and Nicholas Carlini. Increasing confidence in adversarial robustness evaluations. Proceedings of NeurlPS, pages 13174- 13189, 2022.
[50] Mingxing Duan, Yunchuan Qin, Jiayan Deng, Kenli Li, and Bin Xiao. Dual attention adversarial attacks with limited perturbations. IEEE Transactions on Neural Networks and Learning Systems. doi: 10. 10.109/TNNIS..22322244444
12

---

[51] Junyoung Byun, Seungju Cho, Myung-Joon Kwon, Hee-Seon Kim, and Changick Kim. Im- proving the transferability of targeted adversarial examples through object-based diverse input. In Proceedings of IEEF/CVF CVPR, pages 15244-15253, 2022.
[52] Shangbo Wu, Yu-an Tan, Yajie Wang, Ruinan Ma, Wencong Ma, and Yuanzhang Li. Towards transferable adversarial attacks with centralized perturbation. In Proceedings of AAAI, pages 6109-61 6116, 2024.
[53] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, pigh-peeformance deep learning library. In Proceedings of NeurIPS, 2019.
13

---

Appendix
In this appendix, we present the following additional contents: (1) A review of model fingerprint-
ing methods and fingerprint removal attacks (Appendix A); (2) The pscudocode of our ADV-TRA
(Appendix [B): (3) Detailed settings for our experimental setups (Appendix O: (4) Additional experi- mental results, including robustness analysis with decision boundary visualization (Appendix D.I). comparison of the surface trajectory and the bilateral trajectory (Appendix D.2). ablation studies analyzing the impact of various hyperparameters. (Appendix D.3). the visualization of adversarial
trajectories (Appendix D.4). and time overheads (Appendix D.5]: (5) Discussion of limitations and
the future work (Appendix E): (6) Technical details about our experiment environment (Appendix F.1
and the instruction for the code of our method (Appendix F.2)
A Related Work
A.1 Model Fingerprinting
In order to protect the IP of DNN models, model fingerprinting extracts the features of model in non-invasive way, usually by detecting the decision boundaries (37].
Most fingerprinting approaches utilize adversarial samples to probe the decision boundaries of the models. Cao et al. [14) argue that a model can be uniquely represented by its decision boundary. Hence, they find some special data points (i.e., adversarial samples) near the boundary to mark the target model, which is the first model fingerprinting method. Later, several studies are working to explore the transferability of adversarial samples. For instance, Lukas et al. [9] design special adversarial samples that show strong transferability for homology models. They also study the impact of some factors (e.g., model architecture and removal attacks) that affect such transferabilityy. Lukas et al. [10] also point out that prior fingerprinting methods still suffer a high false positive rate: two unrelated models always have the same fingerprinting samples due the adversarial transferring. Instead, they generate Universal Adversarial Perturbations to match the model. Any normal sample added with such perturbation would shift to the decision boundary. However, from the perspective of ownership verification, this kind of perturbation lacks security and uniqueness 66] 38]: Once the attacker knows the pattern of perturbation, he can easily invalidate all fingerprinting samples by removing the perturbation from them.
In addition to adversarial based model fingerprinting, some studies the intermediate computations to measure the model similarity. For example, Maho et al. [39] design a greedy algorithm that leverages few benign inputs and C.E. Shannon's information theory quantify the statistical similarity between the internal outputs of two models. Guan et al. 12 select samples with inconsistent prediction results across two groups of reference models and utilize their pairwise relationships for suspicious model identity recognition. Besides, the model internal parameters can also serve as fingerprints. Similarly, Jia et al. (18] construct a reference dataset train linear model, and compute the cosine distance between the weights of the linear models. Zheng et al. 41 project the front- layer weights onto a random space defined by the model owner's identity, allowing non-repudiable and irrevocable ownership proof against model IP misappropriation and ownership fraud. Recent build an audit framework assess the infringement of model IP. The former leverages multiple testing metrics (e.g., neuron output distance) to calculate the similarity of two models. The latter projects the high-dimensional model parameters into oiw-iiensional data and compares their distributions.
works [ 42
For model fingerprinting, one open question is how to extract fingerprints that satisfies both uniqueness (low false positive rate) and robustness (robust against removal attacks) 43 Current fingerprinting schemes, which verify with a special type of adversarial samples individually. are less than satisfactory, especially for removal attacks like adversarial training [25] 36). In this paper, we attempt to generate chain of samples that have intrinsic connections with each other, using them capture more comprehensive characteristics of the decision surface to enhance both the uniqueness and robustness.
A.2 Removal Attacks
Several studies are also working on exploring the vulnerabilities of model fingerprinting through removal attacks. Removal attacks involve modifying the model, including parameter modifications
14

---

such as fine-tuning [21 or structural modifications such as pruning 444 remove or alter the original fingerprints the model, leading the fallure of model identification.
Such modifications can
on model fingerprinting report their robustness against a variety of removal attacks, the strength of the simulated removal attacks in these works is relatively weak. A recent study [36] makes comprehensive robustness evaluation for existing watermarking and fingerprinting methods, and points out that none of the surveyed schemes robust practice. Yang et al. [2] emphasize that changes in a model's decision boundaries caused by model modifica- tion could render fingerprinting methods less reliable, particularly for input modification 45 and adversarial training [25]. Another kind of removal attack originates from model extraction [5], where an attacker can distill knowledge from the source model and train a substitute model. PRADA [26] aims to steal black-box deployed victim models, with the objective of obtaining substitute model with strong transferability of adversarial samples. Apart from the aforementioned model modification attacks, Wang et al. [46] also find randomized transformations to input samples can significantly undermine query-based fingerprinting methods. Likewise, Jiang al. [47] propose an adversarial post-processing method to evade copyright detection of the model''s products.
Even though many works 14
In this paper, we select four representative removal attacks, including three approaches capable of altering model decision boundaries (fine-tuning, pruning, and adversarial training), as well as three model extraction attacks, to evaluate the robustness of model fingerprinting methods. We also strengthen the intensity of the removal attacks to simulate a more powerful attacker that may occur in real-world scenarios.
B Pseudocode for ADV-TRA
Algorithm 1 ADV-TRA
Input: Source model fare base sample 2o, number of classes m, length of trajectory 1, number of generation iteration
Output: Surface trajectory Tauff
Randomly choose pre-defined classes (ci, Cm
Let Tawy = Ø for c, in c do
for epoch to t do
Initialize step size of each step s = {so, 81,..., 81-1
Trajectory initialization
$ < LengthControl( S,0, Cj, ate)
Calculate (sra(S) and Lsna(s)
Optimize s with Lyvd and Look
b Boundary Probing
end for
Thi = T(fuee U
<> Trajectory Bilateralization Trajectory Connection
Tamny +- "Tmee Toi
Update 2o with the last sample of Toi
end for
return Tourf
C Experiment Setup
Datasets.
We consider three commonly used image datasets CIFAR-10 30 CIFAR- 30 and ImageNet [1] in our experiments.
CIFAR-1(5) This dataset is widely used benchmark dataset in the field of computer vision
and machine learning. It consists of 60, 000 32 x 32 images in 10 classes, with 6, 000 images per class.
CIFAR-100 This dataset consists of 60, 000 color images of 100 classes containing 600 images each. has the same image size as CIFAR-10 but contains 20 superclasses instead of 10 classes, with each super class having 5 sub-classes.
http:://ww.c.. toronto. "ttp///kii/cfrrrr html
15

---

ImageNe This dataset is large-scale dataset containing 14 million 224 x 224 images of 1, 000 classes. It continues be one of the most popular and challenging datasets for benchmark classification and detection models.
For both CIFAR-10 and CIFAR-100, we allocate 50, 000 samples for training the model, while reserving the remaining 10, 000 samples for the attacker to launch removal attacks. For ImageNet, considering the huge computation cost for training a model and suspect models from scratch, we utilize pre-trained models available in Pytorch We allocate 60, 000 samples to launch removal attacks.
Models. For the source models, we consider ResNet20 for CIFAR-10, WideResNet for CIFAR-100 100, and ResNet50 for ImageNet. Specifically, for the negative models from DAST and DADT, we widely adopted other model architectures, including 6 different structures: VGG16, ResNet152, DenseNet201, FfficiietNerV2, Inception and MobileNet
Existing Defenses. We compare ADV-TRA with the following four fingerprinting methods against
the removal attacks: (1) CoRt [1] generates series of adversarial examples using FGSM [T9]
and PGD (34). (2) IPGuard [14] designs type of fingerprinting samples that is superior to
general adversarial samples and can better locate the decision boundaries. (3) CAE [9] proposes
to use multiple surrogate models to find the conferrable fingerprinting samples that demonstrate
better transferability on stolen models. (4) Given any normal sample, UAP 10 designs a special
adversarial perturbation that only reacts to the source model.
Implementation Details. In general, we train the source model and negative models for 200 epochs with batch size b = 128. We use SGD optimizer with learning rate of 0.1, momentum of 0.9, and weight decay of 5e-4. Additionally, we adopt a learning rate scheduler to decay the learning rate by 0.1 x at epochs 60, 120, and 160. For all fingerprinting methods, we select 100 clean samples to generate fingerprinting samples. For our ADV-TRA, we also choose 100 (clean) base samples to generate 100 trajectories (the number of Tbp m on performance) for CIFAR-100 and ImageNet; the length of Tom I = 2), and set the threshold for mutation rate Thr murr 0.5, the brake factor Orack 0.9. For each set of experiments, we build distinct source model for every dataset. Within each set, we perform five trials and then compute the average results.
10 (in Appendix D.2 we discuss the impact of the
Removal Attacks. In this paper, we adopt four common removal attacks to evaluate the robustness of fingerprinting methods.
Fine-Tuning, Fine-tuning uses auxiliary data to continue to train a pre-trained model (source model). We adopt four common strategies as in 6 32: (1) Fine- Tune Last Layer (FTLL): only update the last layer while training; (2) Fine- Tune All Layers (FTAL): update all layers while training: (3) Retrain Last Layer (RTLL): re- initialiiiz the last layer before FTLL; and (4) Retrain All Layers (RTAL): re-initialize the last layer before FTAL. We strengthen the attack intensity by lengthening the fine-tuning epoch well increasing the learning rate. Specifically, we apply SGD optimizer with a learning rate 1) = 0.001 for FTLL and FTAL attack. For RTLL and RTAL attacks, we utilize SGD optimizer with learning rate of 0.01, momentum of 0.9, and weight decay of 5c-4. We set the epochs of all fine-tuning strategies to 50.
Pruning. Pruning is used for model compression. so that the model's memory for fingerprints diminishes over the pruning process. We utilize the technique in 33 for our evaluation. We vary the pruning rate p from 0.1 to 0.9, i.c., pruning p fraction of parameters which have the smallest absolute values. After pruning, we fine-tune the model for 10 epochs for CIFAR-10 10 and CIFAR-100, and 20 epochs for ImageNet, in order to ensure the accuracy of the model.
Adversarial Training. Adversarial training helps models withstand misleading inputs by incorporating them during the training process to improve the model's robustness. We use the method in K4 in our experiments. We adopt standard fo PGD training, which has been proved to achieve the best empirical robustness. specified adversarial samples. We vary the perturbation budget from le-4 to 1.0 to generate adversarial samples used for adversarial training. To avoid compromising the model's ''s accuracy,
detail, we leverage advertorct tibtary to generate
https://imago-net.
16

---

CRIM
RICE
Source
(b) Adv-0.001
Adv-0.01
(d) Adv-0.1
IBCO
19:0
(e) FTLL
(1) FTAL
(g) RTLI
(h) RTAL
ABR
HELL
P-20%
i P-40%
(k) P-60%%
P-80%
ARRE
A3T1
(m) SAST
(0)
(p)
Figure 8: Visualization for the decision surface under various removal attacks. Each color represents a class.
we incorporate normal samples during adversarial training. Additionally, we enhance the attack intensity by prolonging the training epochs 50.
Model Extraction Attack. Model extraction enables an attacker derive substitute model from the source model. We apply three commonly used attacks, namely, PRADA (26), Knock- off (27] and HL (35] for evaluation. PRADA benchmark extraction attack in model IP protection, which selects specific hyperparameters and generates synthetic data to train sub- stitute model with high watermarking transferability. Knockoff collect data from different domains and train substitute model with fewer queries in reinforcement learning manner. As data-free model stealing attack, HI leverages structure of DCGAN and alternately trains substitute model and generator in hard label setting. All extraction attacks adopt the same model architecture as the source model.
Metrics. In our experiments, we use Fingerprint Detection Rate as the main evaluation metric, which widely used previous studies [14] 10 36 This metric represents the proportion of fingerprinting
samples (adversarial trajectories in our case) matched with a suspect model. We also employ the Receiver Operating Characteristic (ROC) curve, Area Under the Curve (AUC), TI FF, and T10%F as the evaluation metrics. The ROC curve plots the True Positive Rate (TPR) against the False Positive Rates (FPR) various threshold settings. TI%F and T10%F represent the TPR values 1% and 10% FPR, respectively, with higher values indicating better verification performance. These metrics can be used measure the ability of the fingerprinting samples to accurately distinguish the positive models from the negative ones. To measure the impact of removal attacks on decision boundaries, we compute the average mutation rate Tour across multiple suspect models Table T represented as Fiutt. which quantifies the discrepancy in decision boundaries between the suspect and source models.
17

---

0.9
1N%6
0.84
TIySFF
TNNAR
0.7
0.71
0.6
0.5
0.6
118 110%F
1358
0.4
1254 - 130%F
T10%F
9
10
The Number of Selected Classes
The Number of Selected Classes
The Number Selected Classes
(a) CIFAR-10
(b) CIFAR-100
(c) ImageNet
Figure 9: TI FF and T10%F with the varying number of classes spanned by trajectories. Note that when the number of classes spanned by trajectory is two, it is equivalent a bilateral trajectory as defined in Section 4.1 in the main text.
D
Additional Experimental Results
D.1 Robustness Analysis with Decision Boundary Visualization
To better understand how the removal attacks affect the source model from the perspective of the decision boundary, we leverage a recent model visualization technique [77 based on data manifold" This method requires three base images to construct the decision boundary surface. We randomly select three samples belonging to the classes FROG, AIRPI, and BIRD from CIFAR-10 dataset, and the results are presented in Figure 8 It can be found that stronger attacks (Adv-0.01, RTAL, and P-80%) induce greater changes to the decision surface, which consistent with their relatively lower fingerprint detection rates shown in Table in the main text. Compared to the source model, the Adv-0.01 model exhibits reduction the number of regions corresponding to the CAT class (red region) from 3 to 1. Additionally, there is significant increase in the area corresponding to the DEER class (orange region). It can also be observed that RTAL greatly alters the decision surface by eliminating most of the smaller regions. In general, the source model is more vulnerable to adversarial training attacks. For negative models, even though the SAST model adopts the same training data and model architecture the source model, the differences between its decision surface and the source model's 's are still greater compared to the positive models. This is why model fingerprinting is able to distinguish between infringing positive models and innocent negative models. It can also be observed that there is an obvious inductive bias of the model architecture from DAST and DADT models.
D.2 Surface Trajectory vs. Bilateral Trajectory
In this part, we investigate the advantages of the surface trajectory over the bilateral trajectory. We evaluate the ability of multi-class trajectories capture global fingerprint features 100 suspect models (same setup as Section [.]], assessing whether traversing more classes can help reduce the misjudgment of trajectories. To ensure fair comparison across different experimental groups and avoid the impact threshold selection, we utilize the %F and T10%F metries. The results are shown in Figure
Obviously, trajectory crossing only two classes (corresponding the bilateral trajectory) is in- sufficient to adequately extract the "unique" or "global" fingerprint features of the model, and its performance may even inferior to that of single-point fingerprinting methods (e.g., UAP, see Table 2 in the main text). However, when the number of crossed classes increases from 2 to 6, both T1 FF and T10%F show significant improvements, strongly implying sharp fall in the false positive rate. For CIFAR-10 10 and CIFAR- 100 datasets, T1%F and T10%F approach 1.0 once the number of crossed classes exceeds 6, whereas for ImageNet dataset, FF and T10%F continue to increase slowly as the number of spanning classes increases. This may be attributed to the large number of classes in ImageNet dataset, which has a more complex decision surface and a higher level of intricacy in the fingerprint patterns. As mentioned in Section 4.1 in the main text, bilateral trajectory To, only attends the decision boundary between two classes, which is still restricted to local fingerprinting. In contrast, surface trajectory Tanf that spans multiple classes has the potential to
18

---

1.0
1.0-
16
Detection Rate
Delection Rater
0.8
008
AUC
0.4-
0.4-
3.6
82
80
Length Trajectory
Length Trajectory
(a) CIFAR-100
ImageNet
Figure 10: The distribution and AUC value of fingerprint detection rate for two types of suspect models under different trajectory lengths. Length=1 corresponds to the single-point fingerprinting method. We choose UAP as the baseline in view of its excellent performance.
1.0
1.0
0.9
0.9
0.8
AUC
0.6
A
L0
Length-20
Longth-44
Longth
0.5
Length-80
0.3
longth-80
Length-160
Longth-160
0.4
0.1
0.2
0.3
0.4
0.5
0.6
0.7
10
0.4
0.1
0.2
0.3
1.0
Doakk
(a) CIFAR-100
(b) ImageNet
Figure 11: Performance over different brake factor asrk for various trajectory lengths.
capture richer and higher-level features of decision surface, thereby significantly reducing the false positives of innocent models.
D.3
Impact of Parameters
D.3.1 Impact of Trajectory Length
The length of adversarial trajectories is a critical hyperparameter that determines the effectiveness and stability of ADV-TRA. In this part, we vary the length of trajectories from 10 to 320. Here, length of 320 signifies that for trajectory traversing 10 classes, there would be 32 samples per class.
the length increases from 10 80, the gap between positive models and negative models gradually widens until completely separated, and the range negative models shrinks from 0.597 to 0.095. However, when the length exceeds 80, the trend reversed. When the length is 320, we can see an obvious breakdown. The AUC value maintains 1.0 within the length ranging from 20 180. Even when the AUC drops 0.942 the length of 320, it is 0.03, still higher than the baseline (0.91). Overall, for lengths surpassing 20, the effectiveness of our method is distinctly superior to the baseline.
As shown in Figure 10
Since the distance from a sample to the decision boundary is fixed, a longer length means smaller step sizes and more samples in the trajectory. Ideally, trajectory with samples could provide richer details regarding the decision boundary. However, there is a convergence problem during trajectory generation: when the step size is too small, may get trapped in local optima, rendering the trajectory difficult to cross over the decision boundary. We observe that some of the trajectories do exhibit this phenomenon when the length is 320. This a common problem for gradient-based optimization algorithms. Therefore it is important to determine the length of the trajectory. Based on the experimental results, setting the length between 20 80 wise choice.
D.3.2 Impact of Brake Factor
Another key point the brake factor Olark, which regulates the proportional relationship between the step sizes of two adjacent steps in the trajectory. A small asyk may result in subsequent excessively
19

---

20
0-
-20
Base Sample
-40
Class1
Class2
-60
Class3
20
20
40
60
Figure 12: 1-SNE visualization of the adversarial trajectory. It depicts an adversarial trajectory traversing four classes starting from base sample (red), where larger points denote the first samples crossing the decision boundary of a new class. We sample a base sample from class 0 and guide it through class 2, and
small step size, getting trapped in local optima. On the contrary, a large abpn may fail to capture the rich information around the decision boundary.
As shown in Figure I1 we can clearly see that both Obdk and the length jointly affect the AUC value. When the Obrr is small, the effectiveness of the verification dramatically decreases and becomes almost equivalent to random guessing. Interestingly, shorter trajectories tend to perform better under extremely low Obd values. In other words, shorter trajectories show greater tolerance for smaller daak as step sizes decrease exponentially along the trajectory. For example, the AUC curve for length of 40 begins experience substantial growth around (' 0.3. whereas the curve for a length of 160 does not increase until a'yn = 0.6. It can also obviously seen that the peak of the AUC curve occurs at Obonk < 1 (aavk 1 corresponds to fixed step size). This is more pronounced ImageNet dataset, which demonstrates that the strategy of decaying step size does help to attend more local details of decision boundaries.
D.4
Visualization
Through 1-SNE 48 visualization technique, we visualize our adversarial trajectory fingerprinting in Figure 1 It can be seen that an adversarial trajectory contains multiple sub-fingerprints (i.c., bilateral trajectories) with varying levels of perturbations. The trajectory starts from a base sample, and progresses toward specified classes until crossing multiple classes. This process generates series of coherent sub- fibfinngrrrrnts.. The number sub-fingerprint contained in each class is fixed, thus realizing the trajectory of fixed length, which makes the required query number limited. Furthermore, the progressiveness of the trajectory enables it to mark the positioning of decision boundaries (i.e., where the larger point located), thereby enabling measurement of how much the decision boundaries have changed due to removal attacks. Compared to single-point fingerprints, the distinctive verification approach of regarding the whole trajectory one fingerprint can capture richer decision boundary information, thereby significantly enhancing robustness against removal attacks and reducing the possibility of false positive identification.
D.5 Time Overheads
As our fingerprinting trajectories requiring iterative optimization, ADV-TRA incurs a higher gen- eration time compared to single-point fingerprinting methods. Table reports the time required to generate 10 trajectories (each containing 40 samples) and an equivalent set of 400 single-point fingerprints using the PGD algorithm. significantly less than the time required to train a model. Therefore, it is acceptable for the model owner to generate a number of fingerprinting trajectories more, in our subsequent experiments (see Appendix D.2 and and D.3) we find that the time overheads can be further reduced (while achieving decent performance) by shortening the trajectory length and decreasing the number of classes spanned by the trajectory. During the ownership verification phase,
evident that the time needed generate trajectories is
future ownership verification. Further-
20

---

Table Time overheads for training the model from scratch, PGD-fingrpriininn, and ADV-TRA. Note that we use pre-trained model for the ImageNet case.
CIFAR-I0
CIFAR-100
ImageNet
Model Training
1h37m
2h41m
PGD
49s
52s
2m55s
Ours
5m42s
5m553
27m13s
since querying the black-box model is almost real-time, do not discuss the time overheads at this stage.
E Discussions
Expansibility, In our design, the trajectory generation method that records the evolution process of adversarial samples is built upon the classical Basic Iterative Method 29 by incorporating a series of optimization strategies. More recent works towards more undetectable [49 [50] and
transferable [5] [52] adversarial samples can also be combined with our method to further enhance the protection performance.
Universality. Most existing model watermarking or fingerprinting methods mainly focus on image classification models and cannot be directly extended other types of DNN models, such as generative models. When facing new model architectures and input-ouiput mappings, extending our approach other models can be challenging. To the best of our knowledge, our approach is the only method so far that quantifies model decision boundary altering through applying adversarial perturbations at different levels, which may allow adapt other models more readily compared to single-point fingerprinting methods.
Advanced Removal Attacks. First, in realistic scenarios, attacker who attempts to remove fingerprints from target model via removal attacks may not rely on single type of attack, but instead employ a combination of several attack methods to achieve better fingerprint removal. such cases, the reliability of existing fingerprinting schemes may be significantly undermined. Second, our experimental results show that adversarial training poses greater threat compared other removal attacks. This is possible because adversarial training effectively improves the robustness of the model, i.e. the ability the model resist adversarial samples. That is to say, the model's robustness may instead degrade the effectiveness of fingerprinting methods. In addition to adversarial training, there are still many other techniques that can improve model robustness, such as adding regularization terms during training. Therefore, further research is needed to investigate the impact of these model robustness enhancement techniques on model fingerprinting.
Limitations. Unlike traditional single-point fingerprinting methods where each fingerprinting sample corresponds a single sample, we use chain of samples (i.c., the trajectory) jointly fingerprint the decision surface. While our fingerprinting method is able to accurately capture the global and more rich fingerprint features, it inevitably increases the number of queries needed. For example, the total length of a trajectory that crosses m classes, with 2/ samples in each class, is m x 21. This also means the number of queries required for a trajectory during the verification phase. practice, multiple trajectories are often needed to reduce randomness. For DNN models deployed in the cloud as prediction interfaces, too many queries may increase the cost of verification (Of course, model owners can apply for unrestricted/unobstrrcce organizations). Therefore, in our design we have take efforts to restrict the needed querying costs by using a fix-length trajectory, instead of the traditional adversarial sample induced trajectory with uncontrollable length. our future work, we plan to explore the adversarial trajectory generation method using even fewer queries to further improve the querying efficiency.
model verification from third-party copyright
Broader Impacts. In this paper, we propose ADV-TRA for protecting the copyright of deep learning models. Under the black-box deployment of suspect models, it utilizes the progressive adversarial perturbations to detect similarities in the decision boundaries of the models. This new paradigm exhibits stronger robustness and better decision boundary localization capability compared to existing single-point fingerprinting methods, especially when facing removal attacks. Our approach not only provides new perspective for protecting the IP of DNNs but also promotes the development of
21

---

interpretability for black-box models. As method for protecting model copyrights, this paper is dedicated safeguarding the rights of legitimate model owners. In summary, this paper drives the development of the copyright community in the Al era, providing security guarantees for the advancement of AI.
Additional Information
F.1
Experiment Environment
Hardware Information. We conduct all experiments on a server equipped with:
CPU: An AMD Ryzen 9 7950X CPU 16-Core@4.00GHz
GPU: 2 NVIDIA GeForce RTX 4090 GPUs each with 24 GB of memory
Memory: 64 GB, DDR4, 5600MHz
O Opertting Systems and Environment.
Operating System: Windows 11 Professional 22H2
Environment: We use Anaconda to build Python environment and Spyder as the compiler. All experiments implemented based Pytorch (33
Libraries: 2ython=3.8.11 15, torch=1.12..0 torchvision=0.13.0 numpy=1.23.4 advertorch=0.2.3,
0000
F.2
PyTorch Code
The code of ADV-TRA is available at: httpp:///gtthub..cco//Heeiiii..
Getting Started. To run this repository, we kindly advise you to install Python 3.8 and PyTorch 1.12 with Anaconda. You can download Anaconda and read the installation instructions on the official website 10
Create a new virtual environment named "ADV-TRA" based on Python 3.8 and enter this environment:
conda create -n ADV ARA python=3.8 8
conda activate ADV_ TRA
Install PyTorch and related packages in torch environment:
conda install pytorch==1 12.1 conda install torchvision pytorch
pip install numpy 1 1.23.4 advertorch-0.2.3 scikit leaann 0. 20.0
File Structure. We now introduce the composition of Python files and their implemented functions this project.
Process the raw dataset and partition it for attacker, defender, and test set:
//utils
Train the source model:
//utile/utitn
Include the design of the model structure involved in this paper:
/uttiel//////// models. py
Generate adversarial trajectories as the fingerprints as well verify the suspect model with number of adversarial trajectories:
/ttuttllsaaaa haallls/adv_eee py
hhtp::/wwww.accod..ccooooooalll
22

---

The main program for our framework:
./utils /eaee/man.....
Note that the main program contains numerous parameters. You can print the help descriptions for all parameters with the following command:
python main. py -h
23

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly state that this paper focuses IP protection for DNNs, and the introduction outlines the contributions separate points.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We have discussed the limitations of our proposed method in Appendix E
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]

---

Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: YYs]
Justification: We have presented the steps of our algorithm in detail Section and provided pseudocode in Appendix B for reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
25

---

Answer: [Yes]
Justification: We have made the code for our proposed algorithm publicly available through URLs (in Appendix F22 and supplementary material, and we have introduced the run environment and operation instructions in Appendix
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nnis..c)
httpii/guuidessoeeeuubmiii.eee ooo.ssooissioooooooooooooooo for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https //nips.coo httpp/sccppblccccccccccoo http///dddde/oeeSubmiiim SoreddeubmissionPPee Policy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: All the training and test details (including datasets, models, hyperparameters, etc.) can be found in Appendix C
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Ys]]
Justification: We have conducted repeated experiments to obtain more comprehensive results including the use of standard deviation in Table and Figure
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
26

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yss]
Justification:
These information can be found in Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't I make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
This research conforms the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- cration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: This paper has positive impacts on the protection of deep learning model copyrights, which is discussed in Appendix E
Guidelines:
The answer NA means that there is societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
27

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: Since this paper does not release any data pre-trained models, it has no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have cited the original paper that produced the code package or dataset and provided corresponding URLs in Appendix C
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
28

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
batasets.papersstt
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New Assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We have provided the code through URLs well supplementary material, which includes documentation instructions for its usage.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodll as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
29

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
30


Reviews:
[{"review_id": "2vETJiDAqV", "reviewer": "Reviewer_y2LA", "summary": "The paper proposes a method to fingerprint deep neural networks in order to protect the model intellectual property. The authors focus on a technique post-training (without requiring changes to the training procedure), based on trajectories of adversarial examples called ADV-TRA. In contrast to other works that rely on adversarial examples that are highly sensitive to changes in the decision boundary, the technique aims to be more robust by constructing fixed-length trajectories with dynamic step sizes. The technique consists of four steps: (1) initialization of the trajectory; (2) probing the boundary; (3) bilaterize the trajectory; and (4) connect the trajectory. Among these, the second phase proposes a double optimization objective. First, to find adversarial examples at increasingly smaller steps, but proportional to two adjacent steps. Second, ensure that the adversarial examples are closer to the decision boundary than previously found adversarial examples.\n\nThe experiments consider several removal attacks (such as fine-tuning and pruning) and shows a lower false positive rate in detecting the fingerprint (negative suspect models are less often labeled as positive).", "questions": "1. What are the computational costs to run the defenses?\n\n2. What happens if the fine-tuning dataset is not as similar as the training dataset. Could you give more details on that experiment? Basically, the hypothesis is that if the fine-tuning is not as similar, then the removal of the fingerprint is more likely.", "limitations": "The authors have addressed potential negative impacts, and limitations in the paper.", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper improves the false positives of the model fingerprinting and is more robust against the considered attacks. This was a limitation of prior work and the results compare with existing fingerprinting techniques. \n\n\n2. The authors creatively combine existing ideas about using adversarial examples and the decision boundary as a way to fingerprint the model. Based on the fact that model stealing attacks aim to increase fidelity, the authors propose to capture as much of the model’s decision surface to enable detection of model stealing attacks.", "weaknesses": "1. The authors propose an application of model fingerprinting to tackle concerns in stealing the model parameters in both the black- and white-box setting. The strongest adversary in the context of model fingerprinting is an adaptive one that would try to steal the model while ensuring the decision boundaries are slightly different so targeting accuracy rather than accuracy *and* fidelity. \n\n\n2. How do we know that two unrelated models do not have the same fingerprinting? Ideally, we want model fingerprints to be akin to hashes, i.e., a small probability that two models that belong to two different owners would result in similar fingerprints. There is no theoretical reasoning or references for why this might be hard to achieve adversarially. In case there is a collision, how do we know which one is the “real owner”? Either of the model creators could have stolen the model or it could have happened that those trajectories are similar. \n\n\n3. While compared to watermarking techniques, this technique does not affect the training but there is still some optimization involved to construct these adversarial examples. There is no discussion about the computational costs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your careful reading and useful feedback, and we address your specific questions below. \n\n- Q1 & W3: Thank you for pointing out this issue. We have discussed the issue of time overheads in Appendix D.5, which reflects the actual computational costs during the execution process. In fact, the model owner can perform model fingerprinting after the model training is complete, and store the fingerprinting samples locally to be used for future verifications. This means that the defense does not need to generate the fingerprinting samples every time a model verification is conducted. Therefore, we are not too concerned about the time overhead required for generating the fingerprinting samples in this domain.\n\n- Q2: Thank you for pointing out this issue. In our experiments, we used a dataset of 10,000 samples that is separate from the training data of the source model for the fine-tuning process (as described in Appendix C). Therefore, the fine-tuning dataset and the source model's training dataset are different. However, as you mentioned, there could still be scenarios where an attacker fine-tunes the model using a significantly different dataset, such as fine-tuning a cat/dog classification model to become a face recognition model. In such cases, the training task and labels would have fundamentally changed, which is essentially watermarking or fingerprinting the pre-trained encoder [1,2], rather than the image classification model that we focus on in the paper. In real-world scenarios, attackers are more likely to simply steal the source model for the same task, as that would not require the additional effort of collecting and training on a different dataset for a new task, which would be computationally expensive. On the other hand, if the fine-tuning dataset is vastly different from the original training data, it would lead to a significant drop in the model's performance on the original task. This means the contribution of the source model would be diminished, and the attacker's own dataset and training efforts would play a larger role, potentially leading to disputes over the final model's ownership.\n\n  To evaluate the impact of the fine-tuning dataset, we randomly selected 10 superclasses from the CIFAR-100 dataset and performed cross-training with CIFAR-10. Note that the labels in CIFAR-100 and CIFAR-10 do not overlap. The results are shown in the table below, where we also recorded the accuracy of the original task before and after fine-tuning. \n  |Original|CIFAR-10|CIFAR-10|CIFAR-100|CIFAR-100|\n  |-|-|-|-|-|\n  |__Fine-Tuning__|__CIFAR-10__|__CIFAR-100__|__CIFAR-10__|__CIFAR-100__|\n  |__Original Acc__|0.870|0.870|0.712|0.712|\n  |__Acc After Fine-Tuning__|0.863|0.117|0.133|0.701|\n  |__IPGuard__|0.623|0.158|0.172|0.601|\n  |__CAE__|0.676|0.134|0.116|0.659|\n  |__UAP__|0.717|0.174|0.192|0.705|\n  |__Ours__|0.760|0.195|0.208|0.747|\n  We can find that our method achieves the highest fingerprint detection rate across all four cases, outperforming the negative model (see Table 1 in the main text). Additionally, the models fine-tuned using a different dataset experienced a significant drop in accuracy on the initial task. In other words, the fine-tuned models have largely lost their predictive capability on the original task. This also leads to a new consideration: a well-resourced attacker could simply train a model from scratch, rather than fine-tuning the source model on a different task, which could avoid potential intellectual property violations.\n\n- W1 & W2: This is an interesting question. Even when SAST models are trained using the same dataset and training settings, they can still be successfully identified as negative models. The uniqueness of the model's decision boundary may be attributed to the randomness in model training (see Figure 8 in the Appendix). Similar experimental results have also been corroborated in works [3,4]. Furthermore, during the verification process, we selected 100 trajectories to capture more comprehensive decision boundary information, which helps avoid false positives due to high similarity in partial decision boundaries. It is recognized that two unrelated models will not exhibit similarity across all decision boundaries, thus preventing such \"coincidental\" false positives. Specifically, for malicious model thefts perpetrator, if they want to maintain the accuracy of the stolen model, they cannot drastically alter the model's decision boundary, which provides an opportunity for our detection method. Conversely, if the perpetrator significantly changes the decision boundary to evade detection, it would lead to a substantial decline in the model's usability, as supported by the findings in works [5,6,7].\n\n\n[1] Cong T, He X, Zhang Y. SSLGuard: A Watermarking Scheme for Self-Supervised Learning Pre-Trained Encoders. ACM SIGSAC. 2022: 579-593.\n\n[2] Lukas N, Kerschbaum F. PTW: Pivotal Tuning Watermarking for Pre-Trained Image Generators. USENIX Security Symposium. 2023: 2241-2258.\n\n[3] Lukas N, Zhang Y, Kerschbaum F. Deep Neural Network Fingerprinting by Conferrable Adversarial Examples. ICLR. 2021.\n\n[4] Chen J, Wang J, Peng T, et al. Copy, right? a testing framework for copyright protection of deep learning models. S&P. 2022: 824-841.\n\n[5] Juuti M, Szyller S, Marchal S, et al. PRADA: Protecting Against DNN Model Stealing Attacks. S&P. 2019: 512-527.\n\n[6] Rakin A S, Chowdhuryy M H I, Yao F, et al. Deepsteal: Advanced Model Extractions Leveraging Efficient Weight Stealing in Memories. S&P. 2022: 1157-1174.\n\n[7] Goldstein B F, Patil V C, Ferreira V C, et al. Preventing DNN Model IP Theft via Hardware Obfuscation. IEEE Journal on Emerging and Selected Topics in Circuits and Systems. 2021, 11(2): 267-277.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GUpOGiIhXR", "reviewer": "Reviewer_u7bH", "summary": "The paper proposes a novel fingerprinting scheme for DNN classifiers.", "questions": "Q1: Can you confirm that a unique source model is considered per dataset?\n\nQ2: Can you explain how the standard deviation is computed?\n\nQ3: Instead of connecting $m$ trajectories, which are not independent since they all depend on the starting point, would it be better to use $m$ independent trajectories starting from $m$ different points?", "limitations": "Some limitations are given (in the appendix, not the main body). Yet many others are missing: no theoretical guarantee, the attacker can play the role of a usurper, not stealth, and many queries to verify a \"fingerprint\".", "rating": 3, "confidence": 5, "soundness": 1, "presentation": 2, "contribution": 2, "strengths": "S1: The number of queries is relatively small (around 100)", "weaknesses": "W1: The authors see fingerprinting only as a technical means for IP protection. Fingerprinting may have different applications. And, as IP protection is concerned, fingerprinting has strong limitations:\n-  No theoretical guarantee, especially about the False Positive Rate. This rate is only empirically measured, and it is rather high (between 1 and 10%). I am sure that a judge would be convinced by this piece of evidence.\n- The attacker can also exhibit a fingerprint which would prove that he is the owner of the stolen model. In other words, there is no secret key. Anyone can claim ownership of any model.\nIn short, I disagree with the 2nd paragraph comparing watermarking and fingerprinting.\n\nW2. The name fingerprint is misleading. This is not a compact representation of a classifier. The defender must first suspect that THIS model is in the black box. Then he uses the trajectory of THIS model to query the black box, and based on the outputs, he computes a similarity with THIS model. Therefore, he can only make a verification for THIS model. This is in stark contrast to what people call a fingerprint in biometry or in multimedia security, ie a compact representation that allows identification via similarity search, not only verification. For instance, FBI [39] is a real fingerprinting scheme for DNN classifiers.\n\nW3. The experimental setup is weak. If I understand correctly, the numbers are given for a unique source model (ie. ResNet 50 for ImageNet). Moreover, there are only 50 positive cases and 50 negative cases. I have difficulty understanding how the standard deviation of rates can be in the order of 10^-3 with so few examples. This accuracy would need millions of samples.\n\nW4. The state-of-the-art (Sect. 2.2) and the benchmark are based on some former proposals, all based on adversarial examples. Other fingerprinting schemes not based on adversarial examples and boundaries are only listed in the appendices.\n\nW5. The \"fingerprint\" extraction is not stealth. It is a series of very correlated inputs, easily detectable by the attacker at verification time.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your careful reading and useful feedback. We address your concerned points below.\n- Q1&Q2&W3: We thank the reviewer for pointing this issue out. Regarding the main performance results in Table 1, we follow the evaluation methodology from previous works [1,2]. Specifically, We ran the experiments 5 times and calculated the standard deviation across 100 trajectories for each experiment. The standard deviation is calculated using the formula $\\sigma = \\sqrt{\\frac{\\sum_{i=1}^{N} (x_i - \\overline{x})^2}{N-1}}$, where $\\overline{x}$ is the mean value and $N=500$ for each type of suspect model. For each experiment, we build a new source model per dataset. We will include this information in the next version of the paper.\n- Q3: It is a direct and straightforward approach of selecting a sample point as the starting point and forming a trajectory passing through multiple classes. However, non-contiguous trajectories (starting from multiple points) could lack connection between sub-trajectories, leading to information loss. Besides, using multiple starting points would require $m$ times more clean samples. Appendix D.2 also shows that longer trajectories crossing multiple classes can capture more unique fingerprints. \n- W1: Compared to other baselines, ADV-TRA has the lowest false positive rate (see Lines 264: \"Only ADV-TRA ... guarantee the subsequent verification without any false positives\"). We believe you may misunderstand the meaning of the Fingerprint Detection Rate metric (see Lines 259). It represents the proportion of fingerprinting samples (trajectories in our case) that are identified in a suspect model. Based on this metric, a threshold needs to be determined to classify the models as positive or negative (See Section 2.2). Considering that the thresholds vary for different methods, we do not perform a final judgment of ownership infringement, but instead directly show the discriminative ability of the fingerprinting samples on different models through the Fingerprint Detection Rate. Therefore, the Fingerprint Detection Rates for the negative models in Table 1 do not represent the False Positive Rate of the model verification.\n\n  As for the second point, model watermarking also faces the challenge of forgery of ownership (also called ambiguity or overwriting attack) [3,4], where attackers can steal the source model and embed their own watermark to claim ownership. This is a recognized challenge in model IP protection, and not unique to model fingerprinting. In the field of model fingerprinting, we are more focused on the robustness of the fingerprints. Furthermore, in the second paragraph of the introduction, we compare model watermarking with fingerprinting. The embedding of watermarks can have a substantial impact on the source model, as described in work [3]: \"During the watermark embedding, the training process of the classifier is modified, which may sacrifice the accuracy of the model.\" This is also why watermarking research tends to focus more on watermark capacity [5], as embedding too many watermarks can significantly degrade the model's utility.\n- W2: We believe the reviewer confuses the definition of fingerprint in the biological and model IP protection (IPP) domains. In biological, fingerprints are unique biometric identifiers that can be used for identification. However, in the model IPP domain, the concept of a fingerprint is used differently. Model fingerprinting involves comparing sample-label pairs to verify a given model, rather than identifying a specific entity. The \"suspect and verify\" process mentioned by the reviewer is a special case where only one suspect model is involved. However, our algorithm can verify multiple suspect models simultaneously, similar to how criminal fingerprint comparison is used to identify suspects. Additionally, in the IPP domain, the concept of model fingerprinting is first defined by Cao et al. [7], and subsequent works [1,8,9] adopt this terminology. Relevant surveys [3,4,10] also provide clear definitions and categorizations of model fingerprinting.\n- W4: Under the black-box verification mechanism, current model fingerprinting methods primarily rely on generating adversarial samples to probe the decision boundary and determine the ownership. This approach is effective for real-world scenarios. There are some other white-box methods [11,12] that observe the model's internal parameters or intermediate layer outputs. However, these white-box methods are not applicable in real-world settings. Therefore, in the black-box setting, using adversarial samples for verification is a primary means and serves as the baseline.\n\n[1] Deep Neural Network Fingerprinting by Conferrable Adversarial Examples. ICLR. 2021.\n\n[2] Copy, right? a testing framework for copyright protection of deep learning models. S&P. 2022.\n\n[3] Intellectual property protection for deep learning models: Taxonomy, methods, attacks, and evaluations. IEEE TAI. 2021.\n\n[4] A systematic review on model watermarking for neural networks. Frontiers in big Data. 2021.\n\n[5] Revisiting the Information Capacity of Neural Network Watermarks: Upper Bound Estimation and Beyond. AAAI. 2024.\n\n[6] Identifying appropriate intellectual property protection mechanisms for machine learning models: A systematization of watermarking, fingerprinting, model access, and attacks. IEEE TNNLS. 2023.\n\n[7] IPGuard: Protecting intellectual property of deep neural networks via fingerprinting the classification boundary. ACM CCS. 2021.\n\n[8] Fingerprinting deep neural networks globally via universal adversarial perturbations. CVPR. 2022.\n\n[9] MetaFinger: Fingerprinting the Deep Neural Networks with Meta-training. IJCAI. 2022.\n\n[10] Intellectual property protection of DNN models. World Wide Web. 2023.\n\n[11] A DNN fingerprint for non-repudiable model ownership identification and piracy detection[J]. IEEE TIFS. 2022.\n\n[12] RAI2: Responsible Identity Audit Governing the Artificial Intelligence. NDSS. 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CBx5z4ksHH", "reviewer": "Reviewer_6xmi", "summary": "This paper introduces ADV-TRA, a fingerprinting scheme that uses adversarial trajectories to fingerprint DNN models. Unlike conventional methods that rely on single-point fingerprint samples, ADV-TRA generates a series of fixed-length trajectory samples, each showcasing varying levels of adversarial perturbation. Furthermore, the authors introduce an adaptive step-size adversarial trajectory generation algorithm capable of achieving fixed-length trajectories between any two classes through dynamically adjusted step sizes. ADV-TRA enhances the precision in capturing changes in the decision boundary, reducing the risk of misidentifying innocent models as infringing. Empirical experiments validate the uniqueness and robustness of ADV-TRA.", "questions": "- Discussion on more related works.\n- Clarify whether the proposed method is robust to transfer learning.", "limitations": "This paper has clarified the limitations.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper proposes a black-box model ownership verification method using adversarial trajectories as fingerprints to measure the reuse relationship between two models.\n- The effectiveness and robustness of this method are demonstrated through detailed and comprehensive experiments.", "weaknesses": "- The related works on model fingerprinting discussed in this paper are mainly white-box methods or based on adversarial examples. However, testing-based fingerprinting methods such as [A, B, C] are not discussed in this paper.\n- The proposed fingerprinting method may not apply to the models with different tasks and classes. Thus, it may not be robust to transfer learning.\n\n[A] A Zest of LIME: Towards Architecture-Independent Model Distances. ICLR 2022.\n[B] Are You Stealing My Model? Sample Correlation for Fingerprinting Deep Neural Networks. NeurIPS 2022.\n[C] MetaV: A Meta-Verifier Approach to Task-Agnostic Model Fingerprinting. KDD 2022.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your careful reading and useful feedback. Below, we will address your specific questions.\n\n- Q1: We really appreciate your thoughtful suggestions regarding the related works. We will include these three outstanding works to supplement the Related Work section in the next version of the paper.\n\n- Q2: We thank the reviewer for pointing this out. However, in this paper, the threat scenario we primarily consider is where the attacker steals the model and uses their limited computing power and auxiliary data to modify the model in order to change its fingerprints. In most transfer learning scenarios, the task will change, leading to significant changes in the model's decision boundary. Thus, the current methods are not well-suited for IP protection in these cases. In the context of model copyright protection for transfer learning scenarios [1], it usually focuses more on the protection of the pre-trained encoder [2,3], which generally employs label-agnostic authentication methods. In reality, most real-world attackers do not have the conditions or reasons to perform transfer learning on the source model [4,5] (they can steal the source model of the same task for convenience). Therefore, this paper considers model verification under the condition of unchanged training tasks, which is a widely recognized and accepted attack scenario in the existing IP protection literature. Even so, we conduct experiments involving transfer learning (Please see our response to Reviewer y2LA's Question 2).\n\n[1] Liu Y, Jia J, Liu H, et al. Stolenencoder: Stealing Pre-Trained Encoders in Self-Supervised Learning. ACM SIGSAC. 2022: 2115-2128.\n\n[2] Cong T, He X, Zhang Y. SSLGuard: A Watermarking Scheme for Self-Supervised Learning Pre-Trained Encoders. ACM SIGSAC. 2022: 579-593.\n\n[3] Lukas N, Kerschbaum F. PTW: Pivotal Tuning Watermarking for Pre-Trained Image Generators. USENIX Security Symposium. 2023: 2241-2258.\n\n[4] Oliynyk D, Mayer R, Rauber A. I know what you trained last summer: A survey on stealing machine learning models and defences. ACM Computing Surveys. 2023, 55(14s): 1-41.\n\n[5] Xue M, Zhang Y, Wang J, et al. Intellectual property protection for deep learning models: Taxonomy, methods, attacks, and evaluations. IEEE Transactions on Artificial Intelligence. 2021, 3(6): 908-923.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CyvRtHwN8B", "reviewer": "Reviewer_TeCi", "summary": "This work discusses a robust model fingerprinting method for protecting the intellectual property (IP) of deep neural networks (DNNs) using adversarial trajectories, referred to as ADV-TRA. Traditional single-point fingerprinting methods suffer from sensitivity to decision boundary changes, leading to high false positives. ADV-TRA overcomes this by generating a surface trajectory of adversarial examples that captures a comprehensive representation of the decision boundary, making it more resistant to various removal attacks. Extensive experiments demonstrate ADV-TRA’s effectiveness in distinguishing between infringing and non-infringing models with low false positives, significantly outperforming previous methods. The proposed approach ensures a unique and reliable fingerprint with lower querying costs and proves robust against sophisticated adversarial attacks aimed at distorting or evading the model’s fingerprint.", "questions": "1. How well does ADV-TRA perform across various neural network architectures beyond those tested, such as transformer-based models or those used in non-visual tasks?\n2. How does ADV-TRA compare to state-of-the-art model watermarking techniques regarding robustness, detectability, and ease of deployment?\n3. Is the close-loop trajectory better than the shorter one?", "limitations": "Yes.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. **Robustness Against Attacks**: ADV-TRA demonstrates superior resistance to removal attacks, including fine-tuning, pruning, and adversarial training. This robustness is crucial for practical deployment where models may face sophisticated threats aimed at circumventing IP protections.\n2. **Low False Positive Rate**: The method achieves a low rate of false positives by using adversarial trajectories that provide a more granular and comprehensive analysis of the model's decision boundary. This reduces the likelihood of misidentifying legitimate models as infringing, a significant improvement over single-point fingerprinting techniques.\n3. **Extensive Validation**: The paper conducts thorough experiments across multiple datasets against various removal attacks, providing strong empirical evidence of the method's effectiveness and practicality. This extensive testing helps in establishing the reliability of ADV-TRA in real-world scenarios.", "weaknesses": "**Complexity and Computational Costs**: The generation and verification of adversarial trajectories can be computationally intensive, potentially limiting the scalability of the method, especially for very large DNN models or in resource-constrained environments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your careful reading and useful feedback. Below, we will address your specific questions.\n\n- Q1: This is an interesting question. ADV-TRA's mechanism offers valuable insights for other domains or neural network architectures. For example, in NLP, generating a batch of continuous text is difficult due to its discrete nature. However, on the embedding layer of NLP models, we can identify minimal perturbations on the continuous embeddings that affect predictions. Thus we can create trajectories in the embedding space, which can then be transformed into text as fingerprint trajectories.\n\n- Q2: Compared to model watermarking, ADV-TRA has several significant advantages: (1) non-invasive: it does not require intervention in the model training process; (2) harmless: it does not degrade the model's performance; (3) unlimited capacity: there is no need to worry about the watermark capacity. Thus, ADV-TRA is more convenient and operable when deployed. Regarding robustness, since the robustness of model watermarking is highly correlated with the number of embedded watermark samples [1,2], and model fingerprinting does not have a capacity limit for verification samples, it is difficult to make a fair comparison between the two. Existing works [3,4] also analyze and compare the pros and cons of the two, and emphasize the feasibility of model fingerprinting during deployment. As for detectability, ADV-TRA only verifies the query results of adversarial trajectories, which is similar to the black-box model watermarking methods using adversarial samples [5,6]. The main downside of ADV-TRA is that it may require more query times to ensure precise probing of the decision boundary.\n\n- Q3: We agree with the reviewer's perspective that the close-loop trajectory is better than the shorter one. In fact, we have carefully compared trajectories of different lengths (in terms of the number of class boundaries crossed) in Appendix D.2. Figure 9 shows that the trajectories crossing more classes achieve better performance. Moreover, the trajectories only crossing two classes (m=2) clearly cannot provide satisfactory verification performance. This suggests that longer trajectories can capture more \"global\" decision boundary information, thereby enabling more \"unique\" fingerprint extraction.\n\n[1] Zhang J, Chen D, Liao J, et al. Model watermarking for image processing networks. AAAI. 2020, 34(07): 12805-12812.\n\n[2] Li F, Zhao H, Du W, et al. Revisiting the Information Capacity of Neural Network Watermarks: Upper Bound Estimation and Beyond. AAAI. 2024, 38(19): 21331-21339.\n\n[3] Regazzoni F, Palmieri P, Smailbegovic F, et al. Protecting artificial intelligence IPs: a survey of watermarking and fingerprinting for machine learning. CAAI Transactions on Intelligence Technology. 2021, 6(2): 180-191.\n\n[4] Xue M, Zhang Y, Wang J, et al. Intellectual property protection for deep learning models: Taxonomy, methods, attacks, and evaluations. IEEE Transactions on Artificial Intelligence. 2021, 3(6): 908-923.\n\n[5] Zhang J, Chen D, Liao J, et al. Deep model intellectual property protection via deep watermarking. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2021, 44(8): 4005-4020.\n\n[6] Lederer I, Mayer R, Rauber A. Identifying appropriate intellectual property protection mechanisms for machine learning models: A systematization of watermarking, fingerprinting, model access, and attacks. IEEE Transactions on Neural Networks and Learning Systems. 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear PC,

We have some reservations about the claims made by Reviewer u7bH with a confidence level of 5. There are two apparent misunderstandings or errors in his/her review:

- First, Reviewer u7bH has confused the meaning of fingerprinting in the biological domain and the deep learning domain. In Weakness 2, they stated that in deep learning IP protection, fingerprinting should be "identification via similarity search, not only verification." However, in IPP, the concept of fingerprinting has merely been adopted, and its specific meaning is to determine the identity of a model by observing its output on specific samples. A substantial body of surveys [1-3] and works [4-7] have provided clear definitions and categorizations of model fingerprinting. It is evident that Reviewer u7bH is not familiar with the specialized terminology, classifications, and principles of deep model IP protection, leading to a misevaluation of our work.

- Second, the reviewer does not seem to read our paper thoroughly. The reviewer misinterprets the Fingerprint Detection Rate metric in Table 1 as the False Positive Rate for model verification (the review states "This rate is only empirically measured, and it is rather high (between 1 and 10%)."). In fact, the final model infringement judgment is based on both the Fingerprint Detection Rate and a specified threshold. We have explained this point in multiple places throughout the paper (see Lines 259 and 265, and Section 2.2). What's more, the strength he/she pointed out are clearly erroneous. The reviewer commented: "The number of queries is relatively small." However, in the Design of ADV-TRA (see Figure 3), Limitations (see Appendix E), Impact of Trajectory Length (see Appendix D.3.1), and Implementation Details (see Appendix C), we have emphasized that "using trajectory fingerprints requires more query samples to obtain more accurate model decision boundary information." Hence it would be a weakness rather than a strength.

Therefore, we have concerns about the credibility of Reviewer u7bH and the correctness of his/her review comments. We sincerely and humbly request that the PC take the time to re-examine the paper's reviews. Finally, we thank all reviewers and PCs for their time and efforts, regardless of the outcome of this paper.

Best regards,

The authors

Meta-review: The paper introduces a novel model ownership verification method that utilizes adversarial trajectories as fingerprints to protect against model stealing attacks. The authors present extensive experimental results demonstrating the method's effectiveness against various removal attacks.

The paper received mixed reviews: 2 weak accept, 1 accept, and 1 reject. After carefully reviewing all feedback and the ensuing discussion, I believe that the merits of the paper outweigh the concerns, and I recommend acceptance.

However, the authors should address the concerns raised by Reviewer u7bH, particularly by clarifying key concepts and including comparisons with several referenced fingerprinting methods in the final version.
Decision: Accept (poster)