Original: Defending against Model Stealing via Verifying  External Features
Revised: Defending against Model Stealing via Verifying Embedded External Features

Original: Abstract
Revised: Abstract

Original: Well-trained models are valuable intellectual properties for their owners.
Revised: Well-trained models are valuable intellectual properties for their owners.

Original: Recent studies revealed that the adversaries can ‘steal’ deployed models even when they have no training sample and can only query the model.
Revised: Recent studies revealed that the adversaries can ‘steal’ deployed models even when they have no training sample and can only query the model.

Original: Currently, there were some defense methods to alleviate this threat, mostly by increasing the cost of model stealing.
Revised: Currently, there were some defense methods to alleviate this threat, mostly by increasing the cost of model stealing.

Original: In this paper, we explore the defense problem from another angle by verifying whether a suspicious model contains the knowledge of defender-speciﬁed external features .
Revised: In this paper, we explore the defense problem from another angle by verifying whether a suspicious model contains the knowledge of defender-speciﬁed external features .

Original: We embed the external features by poisoning a few training samples via style transfer.
Revised: We embed the external features by poisoning a few training samples via style transfer.

Original: After that, we train a meta-classiﬁer, based on the gradient of predictions, to determine whether a suspicious model is stolen from the victim.
Revised: After that, we train a meta-classiﬁer, based on the gradient of predictions, to determine whether a suspicious model is stolen from the victim.

Original: Our method is inspired by the understanding that the stolen models should contain the knowledge of (external) features learned by the victim model.
Revised: Our method is inspired by the understanding that the stolen models should contain the knowledge of (external) features learned by the victim model.

Original: Experimental results demonstrate that our approach is effective in defending against different model stealing attacks simultaneously.
Revised: Experimental results demonstrate that our approach is effective in defending against different model stealing attacks simultaneously.

Original: 1. Introduction
Revised: 1. Introduction

Original: Deep neural networks (DNNs) have been widely and successfully adopted in many areas.
Revised: Deep neural networks (DNNs) have been widely and successfully adopted in many areas.

Original: In general, training a well-performed model requires a large number of valuable resources therefore the trained model is a valuable intellectual property.
Revised: In general, training a well-performed model requires a large number of valuable resources therefore the trained model is a valuable intellectual property.

Original: Recently, researchers found that adversaries can ‘steal’ deployed (victim) models even when they have no training sample and can only query the model (Tram`er et al., 2016;
Revised: Recently, researchers found that adversaries can ‘steal’ deployed (victim) models even when they have no training sample and can only query the model (Tram`er et al., 2016;

Original: Orekondy et al., 2019; Chandrasekaran et al., 2020).
Revised: Orekondy et al., 2019; Chandrasekaran et al., 2020).

Original: This threat is called model stealing .
Revised: This threat is called model stealing .

Original: Since the model stealing can obtain a function-similar copy of the victim model stealthily, it poses a huge threat to model owners.
Revised: Since the model stealing can obtain a function-similar copy of the victim model stealthily, it poses a huge threat to model owners.

Original:  To alleviate the threat of model stealing, there were also some defense methods, mostly by introducing randomness or perturbation in the victim models to increase the costs of model stealing (Tram`er et al., 2016; Lee et al., 2019;
Revised: cn > . To alleviate the threat of model stealing, there were also some defense methods, mostly by introducing randomness or perturbation in the victim models to increase the costs of model stealing (Tram`er et al., 2016; Lee et al., 2019;

Original: Kariyappa & Qureshi, 2020).
Revised: Kariyappa & Qureshi, 2020).

Original: For instance, defenders may perturb the prediction by rounding or adding noise to the posterior probabilities.
Revised: For instance, defenders may perturb the prediction by rounding or adding noise to the posterior probabilities.

Original: However, these defenses may signiﬁcantly reduce the performance of legitimate users and could even be bypassed by adaptive attacks (Maini et al., 2021).
Revised: However, these defenses may signiﬁcantly reduce the performance of legitimate users and could even be bypassed by adaptive attacks (Maini et al., 2021).

Original: In this paper, we explore the defense of model stealing from another perspective by verifying whether a suspicious model has defender-speciﬁed behaviors .
Revised: In this paper, we explore the defense of model stealing from another perspective by verifying whether a suspicious model has defender-speciﬁed behaviors .

Original: If the model has such behaviors, we treat it as stolen from the victim.
Revised: If the model has such behaviors, we treat it as stolen from the victim.

Original: This approach is inspired by the understanding that the stolen models should contain the knowledge of features learned by the victim model therefore they have similar behaviors.
Revised: It is inspired by the understanding that the stolen models should contain the knowledge of features learned by the victim model therefore they have similar behaviors.

Original: To the best of our knowledge, there is only one work, the dataset inference (Maini et al., 2021), focusing on this perspective, where they adopted inherent features of the training set to verify model ownership.
Revised: To the best of our knowledge, there is only one work, the dataset inference (Maini et al., 2021), focusing on this perspective, where they adopted inherent features of the training set to verify model ownership.

Original: However, we reveal that this approach is easy to make misjudgments, especially when the training set of suspicious models have a similar distribution to that of the victim model.
Revised: However, we reveal that this method is easy to make misjudgments, especially when the training set of suspicious models have a similar distribution to that of the victim model.

Original: Based on this observation, we propose to embed defender-speciﬁed external features into victim models for ownership veriﬁcation.
Revised: Based on this observation, we propose to embed defender-speciﬁed external features into victim models for ownership veriﬁcation.

Original: The external features are different from those contained in the original training set.

Original: Speciﬁcally, we embed external features by poisoning a few training samples via style transfer (Johnson et al., 2016).
Revised: Speciﬁcally, we embed external features by poisoning some training samples via style transfer (Johnson et al., 2016).

Original: Since we only poison a few samples and do not change their labels, the embedded features will not hinder the functionality of the victim model.
Revised: Since we only poison a few samples and do not change their labels, the embedded features will not hinder the functionality of the victim model.

Original: In the meanwhile, we also train a benign model based on the original training set.
Revised: In the meanwhile, we also train a benign model based on the original training set.

Original: This model is used only for training a meta classiﬁer to determine whether a suspicious model is stolen from the victim model.
Revised: This model is used only for training a meta classiﬁer to determine whether a suspicious model is stolen from the victim model.

Original: Only the model containing the knowledge of external features will be deployed.
Revised: Only the model containing the knowledge of external features will be deployed.

Original: In conclusion, the main contribution of this work is fourfold: (1) We revisit the defense of model stealing from the aspect of ownership veriﬁcation.
Revised: In conclusion, the main contribution of this work is fourfold: (1) We revisit the defense of model stealing from the aspect of ownership veriﬁcation.

Original: (2) We reveal the limitations of existing veriﬁcation-based methods and propose a simple yet effective defense approach.
Revised: (2) We reveal the limitations of existing veriﬁcation-based methods and propose a simple yet effective defense approach.

Original: (3) We verify the effectiveness of our method on benchmark datasets under various types of model stealing attacks simultaneously.
Revised: (3) We verify the effectiveness of our method on benchmark datasets under various types of model stealing attacks simultaneously.

Original: Our work could provide a new angle about how to adopt ‘data poisoning’ for positive purposes.
Revised: Our work could provide a new angle about how to adopt ‘data poisoning’ for positive purposes.

Original: 2. Related Work 2.1. Model Stealing
Revised: 2. Related Work 2.1. Model Stealing

Original: Model stealing aims to steal the intellectual property from a victim by obtaining a function-similar copy of the deployed model.
Revised: Model stealing aims to steal the intellectual property from a victim by obtaining a function-similar copy of the deployed model.

Original: In general, existing model stealing methods can be divided into three main categories based on the adversary’s permission level, as follows:
Revised: In general, existing model stealing methods can be divided into three main categories based on the adversary’s permission level, as follows:

Original: Model-Accessible Attacks ( A M ):
Revised: Model-Accessible Attacks ( A M ):

Original: In this setting, the adversary has complete access to the victim model.
Revised: In this setting, the adversary has complete access to the victim model.

Original: The adversary can obtain a substitute model by using data-free distillation (Fang et al., 2019; Truong et al., 2021) or simply ﬁne -tuning the victim model with local training samples.
Revised: The adversary can obtain a substitute model by using data-free distillation (Fang et al., 2019; Truong et al., 2021) or simply ﬁne-tuning the victim model with local training samples.

Original: Dataset -Accessible Attacks ( A D ):
Revised: Dataset-Accessible Attacks ( A D ):

Original: In this

Original: setting the

Original: adversary can get

Original: access

Original:  to the training dataset whereas can only query the model.
Revised: In this setting, the adversary can get access to the training dataset whereas can only query the model.

Original: The

Original: adversary can train

Original: a

Original: substitute

Original: model

Original: by the knowledge

Original:  distillation ( Hinton et al., 2014) or simply training their 
Revised: The adversary can train a substitute model by the knowledge distillation (Hinton et al., 2014) or simply training their own model from the scratch.

Original: own

Original: model from

Original: the

Original: scratch

Original: Query-Only Attacks ( A Q ):
Revised: Query-Only Attacks ( A Q ):

Original: In this setting, the adversary can only query the model.
Revised: In this setting, the adversary can only query the model.

Original: Speciﬁcally, the query-only attacks can also be divided into two sub-classes, including the label-query attacks (Papernot et al.
Revised: Speciﬁcally, the query-only attacks can also be divided into two sub-classes, including the label-query attacks (Papernot et al., 2017;

Original: Jagielski et al.,

Original: ; Chandrasekaran

Original: et al., 2020) and logit

Original: -query attacks

Original: Tram`er

Original:  et al Orekondy et al., ), based on the type of victim model’s feedback ( e.g. , label or probabilities).
Revised: Jagielski et al., 2020; Chandrasekaran et al., 2020) and logit-query attacks (Tram`er et al., 2016; Orekondy et al., 2019), based on the type of victim model’s feedback ( e.g. , label or probabilities).

Original: 2.2. Veriﬁcation-based Defenses against Model Stealing
Revised: 2.2. Veriﬁcation-based Defenses against Model Stealing

Original: The veriﬁcation-based methods against model stealing by conducting the ownership veriﬁcation towards the suspicious model.
Revised: The veriﬁcation-based methods against model stealing by conducting the ownership veriﬁcation towards the suspicious model.

Original: To the best of our knowledge, there is currently only one research ( i.e. dataset inference (Maini et al., 2021)) completely working on this area.
Revised: To the best of our knowledge, there is currently only one research ( i.e. dataset inference (Maini et al., 2021)) completely working on this area.

Original: Its key idea is to identify whether the suspicious model contains the knowledge of inherent features that the victim model V learned from a private training set.
Revised: Its key idea is to identify whether the suspicious model contains the knowledge of inherent features that the victim model V learned from a private training set.

Original: Speciﬁcally, for each sample ( x , y ) in the
Revised: Speciﬁcally, for each sample ( x , y ) in the

Original: K -classiﬁcation problem, dataset inference ﬁrst generated its minimum distance δ t to each class t by min [Equation] t d ( x , x + [Equation]
Revised: K -classiﬁcation problem, dataset inference ﬁrst generated its minimum distance δ t to each class t by min δ t d ( x , x + δ t ) , s.t., V ( x + δ t ) =

Original: t ) , s.t., [Equation] where d ( · ) is a distance metric, such as the (cid:96) ∞ norm.
Revised: t, (1) where d ( · ) is a distance metric, such as the (cid:96) ∞ norm.

Original: The distance to each class δ = ( δ 1 , · · · , δ K ) is the feature embedding of sample ( x , y ) w.r.t.
Revised: The distance to each class δ = ( δ 1 , · · · , δ K ) is the feature embedding of sample ( x , y ) w.r.t.

Original: the victim model V .
Revised: the victim model V .

Original: After that, the defender will randomly select some samples inside (labeled as ‘+1’) or out-side (labeled as ‘-1’)
Revised: After that, the defender will randomly select some samples inside (labeled as ‘+1’) or out-side (labeled as ‘-1’)

Original: their private dataset and use their feature embedding δ to train a binary meta-classiﬁer.
Revised: their private dataset and use their feature embedding δ to train a binary meta-classiﬁer.

Original: To determine whether a suspicious model S is stolen from the victim, the defender creates equal-sized sample vectors from private and public samples and conduct the hypothesis test.
Revised: To determine whether a suspicious model S is stolen from the victim, the defender creates equal-sized sample vectors from private and public samples and conduct the hypothesis test.

Original: If the conﬁdence scores of private samples are signiﬁcantly greater than those of public samples, the suspicious model S is treated as stolen from the victim.
Revised: If the conﬁdence scores of private samples are signiﬁcantly greater than those of public samples, the suspicious model S is treated as stolen from the victim.

Original: [Table]
Revised: [Table]

Original: We notice that the dataset inference enjoys certain similarities to the backdoor-based model watermarking (Adi et al., 2018;
Revised: We notice that the dataset inference enjoys certain similarities to the backdoor-based model watermarking (Adi et al., 2018;

Original: Zhang et al., 2018; Li et al., 2020b), which makes these approaches to be potential defenses against model stealing.
Revised: Zhang et al., 2018; Li et al., 2020b), which makes these approaches to be potential defenses against model stealing.

Original: Speciﬁcally, these methods ﬁrst adopted backdoor attacks (Li et al., 2020a) to watermark the model and then conducted the ownership veriﬁcation based on the predictions of benign and poisoned samples.
Revised: Speciﬁcally, these methods ﬁrst adopted backdoor attacks (Li et al., 2020a) to watermark the model and then conducted the ownership veriﬁcation based on the predictions of benign and poisoned samples.

Original: If the conﬁdence scores in predicting the target label of poisoned samples are signiﬁcantly greater than those of their benign version, the suspicious model is treated as watermarked and therefore it is stolen from the owner.
Revised: If the conﬁdence scores in predicting the target label of poisoned samples are signiﬁcantly greater than those of their benign version, the suspicious model is treated as watermarked and therefore it is stolen from the owner.

Original: Note that the original purpose of model watermarking is detecting theft ( i.e. , directly copy the model) rather than preventing model stealing.
Revised: Note that the original purpose of model watermarking is detecting theft ( i.e. , directly copy the model) rather than preventing model stealing.

Original: 3. The Limitation of Existing Defenses 3.1. The Limitation of Dataset Inference
Revised: 3. The Limitation of Existing Defenses 3.1. The Limitation of Dataset Inference

Original: The dataset inference relied on a latent assumption that an independent model will not learn the features contained in the private dataset.
Revised: The dataset inference relied on a latent assumption that an independent model will not learn the features contained in the private dataset.

Original: However, this assumption may not hold and therefore the method may misjudge.
Revised: However, this assumption may not hold and therefore the method may misjudge.

Original: Here we verify it.
Revised: Here we verify it.

Original: Settings.
Revised: Settings.

Original: We conduct experiments with VGG (Simonyan & Zisserman, 2015) and ResNet (He et al., 2016) on CIFAR10 (Krizhevsky et al., 2009) dataset.
Revised: We conduct experiments with VGG (Simonyan & Zisserman, 2015) and ResNet (He et al., 2016) on CIFAR10 (Krizhevsky et al., 2009) dataset.

Original: Speciﬁcally, we split the original training set into two disjoint subsets D l and D r .
Revised: Speciﬁcally, we split the original training set into two disjoint subsets D l and D r .

Original: We train the VGG on D l (dubbed VGG- D l ) and the ResNet on D r (dubbed ResNet- D r ).
Revised: We train the VGG on D l (dubbed VGG- D l ) and the ResNet on D r (dubbed ResNet- D r ).

Original: We also train the VGG on a noisy dataset D (cid:48) l (cid:44) { ( x (cid:48) , y )
Revised: We also train the VGG on a noisy dataset D (cid:48) l (cid:44) { ( x (cid:48) , y )

Original: | x (cid:48)
Revised: | x (cid:48)

Original: =
Revised: =

Original: x + N (0 , 16) , ( x , y ) ∈
Revised: x + N (0 , 16) , ( x , y ) ∈

Original: D l } (dubbed VGG- D (cid:48) l ) for reference.
Revised: D l } (dubbed VGG- D (cid:48) l ) for reference.

Original: In the veriﬁcation stage, we verify whether the VGG- D l and VGG- D (cid:48) l is stolen from ResNet- D r and whether the ResNet- D r is stolen from
Revised: In the veriﬁcation stage, we verify whether the VGG- D l and VGG- D (cid:48) l is stolen from ResNet- D r and whether the ResNet- D r is stolen from

Original: VGG- D l based on dataset inference (Maini et al., 2021).
Revised: VGG- D l based on dataset inference (Maini et al., 2021).

Original: Besides, we also adopt p-value as the evaluation metric.
Revised: Besides, we also adopt p-value as the evaluation metric.

Original: The smaller the p-value, the more conﬁdent that dataset inference believes the model stealing happened .
Revised: The smaller the p-value, the more conﬁdent that dataset inference believes the model stealing happened .

Original: Results.
Revised: Results.

Original: As shown in Table 1, all p-values are signiﬁcantly smaller than 0.

Revised: As shown in Table 1, all p-values are signiﬁcantly smaller than 0.

Original: , i.e. , dataset inference believes that these models are stolen from the victim.
Revised: , i.e. , dataset inference believes that these models are stolen from the victim.

Original: However, in each case, since the suspicious and victim model was trained on completely different training samples and with different structures, the suspicious model should not be considered as stolen from the victim.
Revised: However, in each case, since the suspicious and victim model was trained on completely different training samples and with different structures, the suspicious model should not be considered as stolen from the victim.

Original: These results reveal that dataset inference could make misjudgments and therefore its results are questionable .
Revised: These results reveal that dataset inference could make misjudgments and therefore its results are questionable .

Original: In particular, the p-value of VGG- D l is smaller than that of the VGG- D (cid:48) l .
Revised: In particular, the p-value of VGG- D l is smaller than that of the VGG- D (cid:48) l .

Original: This is probably because the latent distribution of D (cid:48) l is more different from that of
Revised: This is probably because the latent distribution of D (cid:48) l is more different from that of

Original: D r and therefore models learn more different features.
Revised: D r and therefore models learn more different features.

Original: [Table]
Revised: [Table]

Original: The Limitation of Model Watermarking
Revised: The Limitation of Model Watermarking

Original: Backdoor-based model watermarking relied on an assumption that the trigger matches hidden backdoors contained in the suspicious model.
Revised: Backdoor-based model watermarking relied on an assumption that the trigger matches hidden backdoors contained in the suspicious model.

Original: However, the assumption may not hold since the backdoor may be changed during the stealing process.
Revised: However, the assumption may not hold since the backdoor may be changed during the stealing process.

Original: In this section, we verify this limitation.
Revised: In this section, we verify this limitation.

Original: Settings.
Revised: Settings.

Original: We adopt the most mainstream and effective backdoor attack, the BadNets (Gu et al., 2019), for the model watermarking.
Revised: We adopt the most mainstream and effective backdoor attack, the BadNets (Gu et al., 2019), for the model watermarking.

Original: The watermarked model will then be stolen by the data-free distillation-based stealing attack (Fang et al., 2019).
Revised: The watermarked model will then be stolen by the data-free distillation-based stealing attack (Fang et al., 2019).

Original: We adopt the benign accuracy (BA) and attack success rate (ASR) (Li et al., 2020a) for the evaluation.
Revised: We adopt the benign accuracy (BA) and attack success rate (ASR) (Li et al., 2020a) for the evaluation.

Original: The larger the ASR, the better the trigger-backdoor match.
Revised: The larger the ASR, the better the trigger-backdoor match.

Original: Results.
Revised: Results.

Original: As shown in Table 2, the ASR of the stolen model is only 3.84%, which is signiﬁcantly lower than that of the watermarked model, i.e. , the defender-speciﬁed trigger is no longer matches the hidden backdoors contained in the stolen model .
Revised: As shown in Table 2, the ASR of the stolen model is only 3.84%, which is signiﬁcantly lower than that of the watermarked model, i.e. , the defender-speciﬁed trigger is no longer matches the hidden backdoors contained in the stolen model .

Original: As such, backdoor-based model watermarking will has minor effects in defending against model stealing.
Revised: As such, backdoor-based model watermarking will has minor effects in defending against model stealing.

Original: 4. The Proposed Method
Revised: 4. The Proposed Method

Original: Based on the understandings in Section 3, in this paper, we propose to embed external features instead of the inherent features or a speciﬁc (trigger) pattern for ownership veriﬁcation.
Revised: Based on the understandings in Section 3, in this paper, we propose to embed external features instead of the inherent features or a speciﬁc (trigger) pattern for ownership veriﬁcation.

Original: Speciﬁcally, as shown in Figure 1, our method consists of three main stages, including (1) embedding external features, (2) training an ownership meta-classiﬁer, and (3) conducting ownership veriﬁcation.
Revised: Speciﬁcally, as shown in Figure 1, our method consists of three main stages, including (1) embedding external features, (2) training an ownership meta-classiﬁer, and (3) conducting ownership veriﬁcation.

Original: Their technical details are in the following subsections.
Revised: Their technical details are in the following subsections.

Original: 4.1. Threat Model
Revised: 4.1. Threat Model

Original: We consider the defense in a white-box setting, where the defender has complete access to the suspicious model.
Revised: We consider the defense in a white-box setting, where the defender has complete access to the suspicious model.

Original: However, the defender has no information about the stealing process.
Revised: However, the defender has no information about the stealing process.

Original: The goal of defenders is to accurately identify whether a suspicious model is stolen from a victim model.
Revised: The goal of defenders is to accurately identify whether a suspicious model is stolen from a victim model.

Original: 4.2. Embedding External Features
Revised: 4.2. Embedding External Features

Original: Before we reach the technical details, we ﬁrst present the deﬁnition of the inherent and external features.
Revised: Before we reach the technical details, we ﬁrst present the deﬁnition of the inherent and external features.

Original: Deﬁnition 1.
Revised: Deﬁnition 1.

Original: A feature f is called the inherent feature of the dataset D if and only if ∀ ( x , y ) ∈
Revised: A feature f is called the inherent feature of the dataset D if and only if ∀ ( x , y ) ∈

Original: X × Y , ( x , y ) ∈
Revised: X × Y , ( x , y ) ∈

Original: D ⇒ ( x , y ) contains feature f. Similarly, f is called the external feature (of the dataset D ) if and only if ∀ ( x , y ) ∈
Revised: D ⇒ ( x , y ) contains feature f. Similarly, f is called the external feature (of the dataset D ) if and only if ∀ ( x , y ) ∈

Original: X × Y , ( x , y ) contains feature f ⇒ ( x , y ) / ∈ D .
Revised: X × Y , ( x , y ) contains feature f ⇒ ( x , y ) / ∈ D .

Original: In this paper, we consider the classiﬁcation of natural images.
Revised: In this paper, we consider the classiﬁcation of natural images.

Original: As such, defenders can easily embed external features via style transfer (Johnson et al., 2016) with a particular style image ( e.g. , the painting).
Revised: As such, defenders can easily embed external features via style transfer (Johnson et al., 2016) with a particular style image ( e.g. , the painting).

Original: Speciﬁcally, let
Revised: Speciﬁcally, let

Original: D = { ( x i , y i ) }
Revised: D = { ( x i , y i ) }

Original: Ni =1 denotes the benign training set, x s is a defender-speciﬁed style image, and T :
Revised: Ni =1 denotes the benign training set, x s is a defender-speciﬁed style image, and T :

Original: X × X → X is a (trained) style transformer.
Revised: X × X → X is a (trained) style transformer.

Original: In this stage, the defender ﬁrst randomly selects γ % (dubbed transformation rate ) samples ( i.e. , D s ) from D to generate their transformed version
Revised: In this stage, the defender ﬁrst randomly selects γ % (dubbed transformation rate ) samples ( i.e. , D s ) from D to generate their transformed version

Original: D t = { ( x (cid:48) , y ) | x (cid:48)
Revised: D t = { ( x (cid:48) , y ) | x (cid:48)

Original: =
Revised: =

Original: T ( x , x s ) , ( x , y ) ∈
Revised: T ( x , x s ) , ( x , y ) ∈

Original: D s } .
Revised: D s } .

Original: The external features (contained in the style image) will be learned by the victim model V θ during the training process via minθ (cid:80) ( x ,y ) ∈D
Revised: The external features (contained in the style image) will be learned by the victim model V θ during the training process via min

Original: b ∪D t L ( V θ ( x ) , y ) , where D b (cid:44) D\D s and
Revised: [Equation] ( x ) , y ) , where D b (cid:44) D\D s and

Original: L( · ) is the loss function ( e.g. , the cross-entropy).
Revised: L is the loss function ( e g. , the cross-entropy

Original: 4.3. Training Ownership Meta-Classiﬁer
Revised: 4.3. Training Ownership Meta-Classiﬁer

Original: Since there is no explicit expression of external features and those features also have minor inﬂuences on the prediction, we need to train an additional meta-classiﬁer to determine whether a suspicious model contains the knowledge of those features.
Revised: Since there is no explicit expression of external features and those features also have minor inﬂuences on the prediction, we need to train an additional meta-classiﬁer to determine whether a suspicious model contains the knowledge of those features.

Original: In this paper, we train the meta-classiﬁer C w : R | θ | → {− 1 , +1 } based on the gradient of models.
Revised: In this paper, we train the meta-classiﬁer C w : R | θ | → {− 1 , +1 } based on the gradient of models.

Original: Speciﬁcally, we assume that the victim model V and the suspicious model S have the same structure.
Revised: Speciﬁcally, we assume that the victim model V and the suspicious model S have the same structure.

Original: This assumption can be easily satisﬁed since the defender can retain a copy of the suspicious model as the vic-
Revised: This assumption can be easily satisﬁed since the defender can retain a copy of the suspicious model as the vic-

Original: [Table]
Revised: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: Independent

Original: [Table]
Revised: [Table]

Original: tim model, based on the training set of deployed model.
Revised: tim model, based on the training set of deployed model.

Original: Once the suspicious model is obtained, the defender will train its benign version ( i.e. , the B ) on the original training set D .
Revised: Once the suspicious model is obtained, the defender will train its benign version ( i.e. , the B ) on the original training set D .

Original: After that, we can obtain the training set
Revised: After that, we can obtain the training set

Original: D c of classiﬁer C via D c
Revised: D c of classiﬁer C via D c

Original: = D positive ∪ D negative , where we have D positive = { ( g V ( x (cid:48) ) , +1) | ( x (cid:48) , y
Revised: = D positive ∪ D negative , where we have D positive = { ( g V ( x (cid:48) ) , +1) | ( x (cid:48) , y

Original: ) ∈
Revised: ) ∈

Original: D t } ,
Revised: D t } ,

Original: D negative =
Revised: D negative =

Original: { ( g B ( x (cid:48) ) , − 1) | ( x (cid:48) , y ) ∈
Revised: { ( g B ( x (cid:48) ) , − 1) | ( x (cid:48) , y ) ∈

Original: D t } , g V ( x (cid:48) )
Revised: D t } , g V ( x (cid:48) )

Original: = sign ( ∇ θ L ( V ( x (cid:48) ) , y )) , g B ( x (cid:48) )
Revised: = sign ( ∇ θ L ( V ( x (cid:48) ) , y )) , g B ( x (cid:48) )

Original: = sign ( ∇ θ L ( B ( x (cid:48) ) , y )) , and sign ( · ) is the sign function (Hogg et al., 2005).
Revised: = sign ( ∇ θ L ( B ( x (cid:48) ) , y )) , and sign ( · ) is the sign function (Hogg et al., 2005).

Original: At the end, the meta-classiﬁer C w is trained bymin
Revised: At the end, the meta-classiﬁer C w is trained by

Original: [Equation]
Revised: [Equation]

Original: 4.4. Ownership Veriﬁcation with Hypothesis Test
Revised: 4.4. Ownership Veriﬁcation with Hypothesis Test

Original: When the meta-classiﬁer is trained, given a transformed image x (cid:48) and its label y , the defender can examine the suspicous model simply by the result of C ( g S ( x (cid:48) )) , where g S ( x (cid:48) )
Revised: When the meta-classiﬁer is trained, given a transformed image x (cid:48) and its label y , the defender can examine the suspicous model simply by the result of C ( g S ( x (cid:48) )) , where g S ( x (cid:48) )

Original: = sign ( ∇ θ L ( S ( x (cid:48) ) , y )) .
Revised: = sign ( ∇ θ L ( S ( x (cid:48) ) , y )) .

Original: If C ( g S ( x (cid:48) ))
Revised: If C ( g S ( x (cid:48) ))

Original: = 1 , the suspicous model is considered as stolen from the victim.
Revised: = 1 , the suspicous model is considered as stolen from the victim.

Original: However, this method may be sharply affected by the randomness in the selection of x (cid:48) .
Revised: However, this method may be sharply affected by the randomness in the selection of x (cid:48) .

Original: In this paper, we design a hypothesis test based veriﬁcation method to increase the veriﬁcation conﬁdence, as follows:
Revised: In this paper, we design a hypothesis test based veriﬁcation method to increase the veriﬁcation conﬁdence, as follows:

Original: Deﬁnition 2.
Revised: Deﬁnition 2.

Original: Let X (cid:48) denotes the variable of transformed images, while µ S and µ B indicates the posterior probability of the event C ( g S ( X (cid:48) ))
Revised: Let X (cid:48) denotes the variable of transformed images, while µ S and µ B indicates the posterior probability of the event C ( g S ( X (cid:48) ))

Original: = 1 and C ( g B ( X (cid:48) ))
Revised: = 1 and C ( g B ( X (cid:48) ))

Original: = 1 , respectively.
Revised: = 1 , respectively.

Original: Given a null hypothesis H 0 : µ S ≤ µ B ( H 1 : µ S > µ B ) , we claim that the suspicous model S is stolen from the victim if and only if the H 0 is rejected.
Revised: Given a null hypothesis H 0 : µ S ≤ µ B ( H 1 : µ S > µ B ) , we claim that the suspicous model S is stolen from the victim if and only if the H 0 is rejected.

Original: In practice, we randomly sample m different transformed images from D t to conduct the pair-wise T-test (Hogg et al., 2005) and calculate its p-value.
Revised: In practice, we randomly sample m different transformed images from D t to conduct the pair-wise T-test (Hogg et al., 2005) and calculate its p-value.

Original: When the p-value is smaller than the signiﬁcance level α , H 0 is rejected.
Revised: When the p-value is smaller than the signiﬁcance level α , H 0 is rejected.

Original: 5. ExperimentsBaseline Selection
Revised: 5. Experiments

Revised: Baseline Selection.

Original: We compare our defense with the dataset inference (Maini et al., 2021) and backdoor-based model watermarking (Adi et al., 2018).
Revised: We compare our defense with the dataset inference (Maini et al., 2021) and backdoor-based model watermarking (Adi et al., 2018).

Original: Following the settings in (Maini et al., 2021), we conduct model stealing attacks illustrated in Section 2.1 to evaluate the effectiveness of defenses.
Revised: Following the settings in (Maini et al., 2021), we conduct model stealing attacks illustrated in Section 2.1 to evaluate the effectiveness of defenses.

Original: Besides, we also provide the results of directly copying the victim model (dubbed ‘Source’) and examining a suspicious model which is not stolen from the victim (dubbed ‘Independent’) for reference.
Revised: Besides, we also provide the results of directly copying the victim model (dubbed ‘Source’) and examining a suspicious model which is not stolen from the victim (dubbed ‘Independent’) for reference.

Original: Evaluation Metric.
Revised: Evaluation Metric.

Original: We use the conﬁdence score ∆ µ and p-value for the evaluation metric.
Revised: We use the conﬁdence score ∆ µ and p-value for the evaluation metric.

Original: The smaller the p-value and the larger the ∆ µ , the better the defense.
Revised: The smaller the p-value and the larger the ∆ µ , the better the defense.

Original: Results.
Revised: Results.

Original: As shown in Table 3-4, our defense reaches the best performance in almost all cases.
Revised: As shown in Table 3-4, our defense reaches the best performance in almost all cases.

Original: The only exception appears in defending against training with different model architectures.
Revised: The only exception appears in defending against training with different model architectures.

Original: Nevertheless, the p-value of our defense is small enough ( < 0 .
Revised: Nevertheless, the p-value of our defense is small enough ( < 0 .

Original: 01 ) therefore defenders can still easily identify the existence of model stealing.
Revised: 01 ) therefore defenders can still easily identify the existence of model stealing.

Original: Moreover, our method has minor adverse effects on the performance of the victim model.
Revised: Moreover, our method has minor adverse effects on the performance of the victim model.

Original: Speciﬁcally, the accuracy of the model trained on the benign CIFAR-10 dataset and its transformed version is 91.99% and 91.79%, respectively; the accuracy of the model trained on the benign ImageNet dataset and its transformed version is 82.40% and 80.40%, respectively.
Revised: Speciﬁcally, the accuracy of the model trained on the benign CIFAR-10 dataset and its transformed version is 91.99% and 91.79%, respectively; the accuracy of the model trained on the benign ImageNet dataset and its transformed version is 82.40% and 80.40%, respectively.