NON-TRANSFERABLE LEARNING: A NEW APPROACH
Year: 2021

Abstract:
As Artificial Intelligence as a Service gains popularity, protecting well-trained models as intellectual property is becoming increasingly important. There are two common types of protection methods: ownership verification and usage authorization. In this paper, we propose Non-Transferable Learning (NTL), a novel approach that captures the exclusive data representation in the learned model and restricts the model generalization ability to certain domains. This approach provides effective solutions to both model verification and authorization. Specifically: 1) For ownership verification, watermarking techniques are commonly used but are often vulnerable to sophisticated watermark removal methods. By comparison, our NTL-based ownership verification provides robust resistance to stateof-the-art watermark removal methods, as shown in extensive experiments with 6 removal approaches over the digits, CIFAR10 & STL10, and VisDA datasets. 2) For usage authorization, prior solutions focus on authorizing specific users to access the model, but authorized users can still apply the model to any data without restriction. Our NTL-based authorization approach instead provides a data-centric protection, which we call applicability authorization, by significantly degrading the performance of the model on unauthorized data. Its effectiveness is also shown through experiments on aforementioned datasets.

heading: 1 INTRODUCTION
text: Deep Learning (DL) is the backbone of Artificial Intelligence as a Service (AIaaS) (Ribeiro et al., 2015), which is being provided in a wide range of applications including music composition (Briot et al., 2020), autonomous driving (Li et al., 2021a), smart building (Xu et al., 2020a), etc. However, a good model can be expensive to obtain: it often requires dedicated architecture design (He et al., 2016), a large amount of high-quality data (Deng et al., 2009), lengthy training on professional devices (Zoph & Le, 2016), and expert tuning (Zhang et al., 2019). Thus, well-trained DL models are valuable intellectual property (IP) to the model owners and need protection. Generally speaking, there are two aspects in protecting an IP in AIaaS, verifying who owns the model and authorizing how the model can be used. These two aspects led to the development of two types of protection techniques: ownership verification and usage authorization.
For ownership verification, prior works proposed approaches such as embedding watermarks into network parameters (Song et al., 2017), learning special behaviors for pre-defined triggers (Fan et al., 2019), and extracting fingerprints from the model (Le Merrer et al., 2020). However, they are vulnerable to state-of-art watermark removal approaches that are based on model fine-tuning or retraining (Chen et al., 2019), watermark overwriting and model pruning (Rouhani et al., 2018). For model usage authorization, most prior works were built on encrypting neural network parameters with a secret key (Alam et al., 2020; Chakraborty et al., 2020) and ensuring that models can only be used by users with this key. However, authorized users may use the model on any data without restriction. We believe that for comprehensive IP protection, the goal of usage authorization is not only who is allowed to use the model, but also what data can the model be used on. We thus consider a new data-centric aspect of usage authorization in this work, i.e., authorizing models to certain data for preventing their usage on unauthorized data. We call this applicability authorization. Note that applicability authorization goes far beyond IP protection. It can also be viewed as a way to “control” how machine learning models are used in general. One example would be a company
Data space
Reduced model generalization bound
Source domain
Source-Only NTL
Source domain

heading: Outside of source domain
text: Data space
Target
Target-Specified NTL
Source domain
Generalization bound
Data space
Supervised Learning
Figure 1: A visualization of the generalization bound trained with different approaches. The left figure shows Supervised Learning in the source domain, which can derive a wide generalization area. When Target-Specified NTL is applied (middle), the target domain is removed from the generalization area. As for Source-Only NTL (right), the generalization area is significantly reduced.
(e.g., Facebook) trains a model from adult data and uses applicability authorization to prevent the model from being used for teenagers.
Our Approach and Contribution. In this work, we propose Non-Transferable Learning (NTL), a novel approach that can robustly verify the model ownership and authorize the model applicability on certain data. Intuitively, NTL goes against the current research trend of improving the generalization ability of models across various domains, e.g., domain generalization and adaptation (Li et al., 2020; Zhou et al., 2020). Instead, NTL tries to make the generalization bound of DL models more explicit and narrower, by optimizing the model to learn domain-dependent features and thereby making the model exclusive to certain domains. More specifically, we consider two domains: the source domain where we want the models to perform well, and the auxiliary domain where we aim to degrade the model performance. And if the model trained with NTL is applied to a target domain similar to the auxiliary one, the performance should also be poor. As shown in Figure 1, we have developed two types of NTL approaches: Target-Specified NTL and Source-Only NTL.
• Target-Specified NTL assumes that the source and target domains are both known. We then treat the target domain as the auxiliary domain and enlarge the distance of representations between the source and auxiliary domains. Target-Specified NTL can be used to verify the model ownership by triggering misclassification. While previous model watermarks can often be easily removed because the model memorization of such watermarks encounters catastrophic forgetting (Kemker et al., 2018) during watermark removal, our NTL-based verification is resistant to state-of-art watermark removal approaches, because the misclassification behavior is dependent on the overall target-private features that have little correlation with the source-private features for the main task.
• In Source-Only NTL, the target domain is unknown and thus our approach relies solely on the source domain, aiming to degrade the performance in all other domains. In this case, NTL generates the auxiliary domain from a novel generative adversarial augmentation framework and then increases the representation distance. Source-Only NTL can provide authorization to certain data rather than particular users or devices, by degrading the model performance on all other data domains other than the source domain. This provides data-centric applicability authorization, with which we can also prevent unauthorized model usage that are caused by the secret key leakage and cannot be addressed by prior model authorization methods.
In addition to proposing the novel concept of NTL and developing its two approaches, we are also able to experimentally validate their effectiveness. We conducte extensive experiments on 5 digit sets, CIFAR10 & STL10 and VisDA. For target-specified cases, we demonstrate how to apply NTL for model ownership verification. Our experiments show that the state-of-art model watermark removal methods are ineffective on NTL-based ownership verification. For source-only NTL, our experiments demonstrate its effectiveness in authorizing model applicability to certain data.

heading: 2 RELATED WORK
text: Domain Generalization & Adaptation (DG & DA). DG aims to generalize learning models with available source domains to unseen target domains (Blanchard et al., 2011). A number of methods have been proposed for domain discrepancy minimization (Li et al., 2020), adversarial training (Rahman et al., 2020; Zhao et al., 2020c), invariance representation learning (Zhou et al., 2020; Piratla et al., 2020), etc. Recently, there is significant interest on conducting DG with one source domain only, for which well-crafted data augmentation approaches (Qiao et al., 2020; Zhao et al., 2020b; Li
et al., 2021b; Xu et al., 2020b) have been proposed to expand the input space. DA is also related to improving the generalization ability of models across domains (Ahmed et al., 2021), and while DA can access the target data, DG has no access to any target sample (Tzeng et al., 2017; Kundu et al., 2020). Unlike DG or DA, in this work, we try to weaken the generalization ability of models by expanding the distance between representations of different domains. Our proposed method can work effectively for both the target-specified case and the source-only case with a novel adversarial augmentation framework.
Intellectual Property (IP) Protection for Deep Learning (DL). While DL has shown its unparalleled advantages in various applications, there are significant challenges in protecting DL models. For instance, Membership Inference Attack (Shokri et al., 2017) can steal information on whether a particular data sample has been learned by the target DL model. Model Inversion Attack (He et al., 2019; Salem et al., 2020) is able to recover the input data via an analysis of the model prediction. These two types of attacks directly threaten the privacy of model users, while there are many active attacks (Suciu et al., 2018; Yao et al., 2019) that lead DL models to produce abnormal behaviors.
In addition, verifying model ownership and authorizing model usage have become important issues with the development of AIaaS. There have been a number of watermarking approaches addressing the verification of model ownership. For instance, Zhang et al. (2018) and Li et al. (2019) train a neural network on the original datasets and on the watermarked one assigned with a particular label, which makes the model behave abnormally when it encounters watermarked data. Song et al. (2017) and Uchida et al. (2017) inject watermark into the least significant bits of the model parameters and provide the corresponding decoding methods. Le Merrer et al. (2020) and Zhao et al. (2020a) make use of adversarial examples to extract fingerprints from learned neural networks without accessing network weights. Compared to these approaches, our NTL can achieve model ownership verification by triggering universal misclassification. Moreover, with extensive experiments, we also demonstrate that state-of-art model watermark removal methods, e.g., FTAL and RTAL (Adi et al., 2018), EWC and AU (Chen et al., 2019), watermark overwriting and model pruning (Rouhani et al., 2018) are not effective to NTL-based verification. Model usage authorization is another aspect in protecting model intellectual property. For instance, Alam et al. (2020) encrypt every network parameter with a secret key. Chakraborty et al. (2020) generate a secret key from hardware fingerprints of a particular device, and require that only users who possess this device can load and employ the model. Different from these methods, our NTL focuses on providing data-centric protection via applicability authorization, which retains good model performance on authorized data while degrading model performance for other data domains. To the best of our knowledge, this is the first work that prevents model usage on unauthorized data via model learning.

heading: 3 METHODOLOGY
text: In this section, we introduce our NTL approach. Section 3.1 presents the inspiration and the design of optimization objective of NTL, which is at the core for both target-specified and source-only cases. Section 3.2 presents the generative augmentation framework for source-only cases. Our method is based on the concept of generative adversarial networks (GAN), however our goal is not to propose a new GAN but to design an effective augmentation method in the context of NTL. Section 3.3 introduces the application of NTL on ownership verification and applicability authorization.

heading: 3.1 NON-TRANSFERABLE LEARNING WITH DISTANCE EXPANSION OF REPRESENTATION
text: We consider a source domain with labeled samples S= {(x,y)∥x∼PSX ,y∼PSY }, where PX and PY are the input and label distributions, respectively. In this work, we use image classification as the learning task with K possible classes, in which case x and y are matrix-valued and scalar random variables, respectively. In addition, we consider an auxiliary domain A={(x,y)∥x∼PAX ,y∼PAY }. The source domain S and the auxiliary domain A will be fed into a deep neural network, and without loss of generality, we split the neural network into two parts, one is a feature extractor Φ on the bottom, and the other is a classifier Ω on the top.
Inspiration from Information Bottleneck. Our NTL, in particular the design of optimization objective, is inspired by the analysis of Information Bottleneck (IB) (Tishby et al., 2000). Let us start by introducing Shannon Mutual Information (SMI). In addition to random variables – input x and label y, we also regard representation z extracted by Φ as a random variable. The SMI between two random variables, e.g., between z and x, is defined as I(z;x)=Ex∼PX [DKL(P(z|x)∥P(z))], where
DKL(·) represents the Kullback-Leible (KL) divergence and P(·) is the distribution. In IB theory, considering the effectiveness, privacy and generalization, an optimal representation has three properties (Achille & Soatto, 2018): (1) Sufficiency: label y sufficiently differentiates representation z, i.e., I(z;y) = I(x;y); (2) Minimality: z needs to represent as little information about input x as possible, i.e., min I(z;x); (3) Invariance: z is optimal, meaning that it does not overfit to spurious correlations between y and nuisance n embedded in x, i.e., I(z;n)=0. IB theory assumes that nuisance n is a factor that affects input x, and it works with y together to determine what x looks like to some extent. For instance, in domain generalization, nuisance n can be regarded as a domain index that indicates which domain a certain sample comes from (Du et al., 2020). In our problem, different from the objective of the IB theory, NTL enforces the models to extract nuisance-dependent representations, which is opposite to the property of invariance. In other words, we aim to increase I(z;n), and we have the following proposition for achieving this aim.
Proposition 1. Let n be a nuisance for input x. Let z be a representation of x, and the label is y. For the information flow in the representation learning, we have
I(z;x)− I(z;y|n) ≥ I(z;n) (1)
The detailed proof for Proposition 1 is included in the Appendix.
Optimization Objective Design. Proposition 1 provides guidance for maximizing I(z;n). First, unlike in the IB theory, we do not minimize I(z;x) for the minimality property. In addition, we try to minimize I(z;y|n) through the design of optimization objective that measures the error between the model prediction and the ground truth during the training of neural networks. Specifically, instead of using the typical CrossEntropy loss to measure the error, we apply KL divergence loss to direct the training, and we have the following theorem.
Theorem 1. Let ŷ be the predicted label outputted by a representation model when feeding with input x, and suppose that ŷ is a scalar random variable and x is balanced on the ground truth label y. Denote the one-hot forms of ŷ and y as ŷ and y, respectively. If the KL divergence loss DKL(P(ŷ)∥P(y)) increases, the mutual information I(z;y) will decrease. The detailed proof of Theorem 1 is provided in the Appendix. According to this theorem, I(z;y|n) can be minimized by increasing the KL divergence loss of training data conditioned on different n. However, as stated in Section 1, we aim to degrade the model performance in the auxiliary domain while still maintaining good model performance in the source domain. Thus, we only minimize I(z;y|n) by increasing the KL divergence loss of the auxiliary domain data. In order to achieve this goal, we design a loss L∗ntl that shapes like a minus operation between KL divergence losses of the source and auxiliary domain (LS , LA), i.e., LS = Ex∼PSX [DKL(P(Ω(Φ(x)))∥P(y))] and LA=Ex∼PAX [DKL(P(Ω(Φ(x)))∥P(y))]. Specifically, this loss can be written as follows:
L∗ntl = LS −max(β, α · LA) (2)
Here, α is the scaling factor for LA (α = 0.1 in our experiments), and β is an upper bound when LA gets too large and dominates the overall loss (β=1.0 in experiments; please see the Appendix for more details about α and β). Moreover, if we use n = 0 and n = 1 to denote the source and auxiliary domain respectively, the optimization of Eq. (2) can guarantee the sufficiency property for the source domain: I(z;y|n=0)=I(x;y|n=0), and increasing LA decreases I(z;y|n=1). According to Proposition 1, we can move the upper bound of I(z;n) to a higher baseline via optimizing Eq. (2). However, such optimization might only make classifier Ω more sensitive to domain features and have little effect on feature extractor Φ. In this case, representations of different domains captured by Φ may still be similar, which conflicts with our intention to maximize I(z;n), and the performance of the target can be easily improved by fine-tuning or adapting Ω with a small number of labeled target samples. On the other hand, directly calculating I(z;n) and taking it as part of the optimization objective are difficult, especially in the optimization of representation learning (Torkkola, 2003). Achille & Soatto (2018) apply binary classifier as the nuisance discriminator, and they can estimate I(z;n) after the model training via this discriminator. Here, we find another way to increase I(z;n) indirectly based on the following theorem.
Theorem 2. Let n be a nuisance that is regarded as a domain index. n=0 and n=1 denote that a certain input x comes from two different domains. Suppose that these two domains have the same number of samples d, and the samples of each domain are symmetrically distributed around the
centroid. Let z be a representation of x, and it is drawn from distribution PZ . An estimator with the characteristic kernel from Reproducing Kernel Hilbert Spaces (RKHSs) – Gaussian Kernel estimator MMD(P,Q; exp) is applied on finite samples from distributions PZ|0 and PZ|1 to approximate the Maximum Mean Discrepancy (MMD) between these two distributions. If MMD(PZ|0,PZ|1; exp) increases to saturation, the mutual information between z and n will increase.
MMD(PZ|0,PZ|1; exp)=Ez,z′∼PZ|0 [e −∥z−z′∥2 ]−2Ez∼PZ|0,z′∼PZ|1 [e −∥z−z′∥2 ]+Ez,z′∼PZ|1 [e −∥z−z′∥2 ] (3) We also employ a nuisance discriminator to observe the change of I(z;n) during training. The details of this discriminator design and the proof of Theorem 2 can be found in the Appendix.
NTL Optimization Objective. Based on the above analysis, we design our NTL optimization objective to increase I(z;n) and extract nuisance-dependent representations. Specifically, we compute the MMD(P,Q; exp) between representations of the source and auxiliary domain data and maximize it. For stability concern, we also set an upper bound to the MMD(P,Q; exp). Then, the overall optimization objective of NTL with distance expansion of representation is shaped as follows: Lntl = LS−max(β, α ·LA ·Ldis), whereLdis = max(β′, α′ ·MMD(Px∼PS
X (Φ(x)),Px∼PA X (Φ(x)); exp)
(4) Here, α′ and β′ represent the scaling factor and upper bound of Ldis respectively (α′ = 0.1 and β′=1.0 in our experiments; please refer to the Appendix for more details about α′ and β′). Φ(·) is the feature extractor that outputs the corresponding representations of given inputs.
When the target domain is known and accessible, it will be regarded as the auxiliary domain, and the above NTL with distance expansion of representation can be conducted directly on the source and auxiliary domains. We call such cases Target-Specified NTL.

heading: 3.2 SOURCE DOMAIN AUGMENTATION FOR SOURCE-ONLY NTL
text: In practice, the target domain might be unknown or unavailable. For such cases, we develop a novel generative augmentation framework to generate an auxiliary domain and then leverage the above NTL process, in what we call Source-Only NTL. In the following, we will introduce our augmentation framework, which can generate data samples drawn from the neighborhood distribution of the source domain with different distances and directions to serve as the auxiliary data domain in NTL.
GAN Design for Source Domain Augmentation. The overall architecture of our augmentation framework is shaped like a generative adversarial network (GAN) that is made up of a generator G and a discriminator D. G takes in normal noise and a label in form of one-hot, and then outputs a data sample. For D, if we feed D with a sample, it will tell whether this sample is fake or not and predict its label. The adversarial battle happens as G tries to generate data as real as possible to fool D, while D distinguishes whether the data being fed to it is real or not to the best of its ability. After sufficient period of such battle, the distributions of the generated data and the ground-truth data are too similar to tell apart (Li et al., 2017). Based on this principle, we utilize G to approximate the source domain. However, if we follow the standard GAN training, the trained GAN will not generate samples with deterministic labels. Therefore, we combine the intuitions of CGAN (Mirza & Osindero, 2014) and infoGAN (Chen et al., 2016) to propose a new training approach for our augmentation framework.In our approach, G uses MSE loss to compare its generated data and the real data. D consists of three modules: a feature extractor and two classifiers behind the extractor as two branches, where a binary classifier predicts whether the data is real or not, and a multiple classifier outputs the label. Note that these two classifiers both rely on the representations extracted by the feature extractor. For training D, we use MSE loss to evaluate its ability to distinguish real samples from fake ones, and KL divergence to quantify the performance of predicting labels for the real data. Finally, there is an additional training step for enforcing the GAN to generate samples of given labels by optimizing G and D simultaneously. The training losses of LG , LD and LG,D are:
LD = Ex∼PS X ,y∼PS Y
[ 1
2 (∥Db(x), 1∥2 + ∥Db(G(noise,y)), 0∥2) +DKL(P(Dm(x)) ∥P(y) ] LG = Ey∼PS
Y
[ ∥Db(G(noise,y)), 1∥2 ] , LG,D = Ey′∼PU Y [DKL(P(Dm(G(noise,y′))) ∥P(y′)] (5)
Here, we use subscripts b and m to denote outputs from the binary classifier and the multiple classifier of D, respectively. The noise of in Eq. (5) is drawn from Gaussian Noise Pg =N (0, 1), while y′ is drawn from the uniform distribution PUY with K equally likely possibilities. And y and y′ are the one-hot form vectors of y and y′, respectively.
Algorithm 1: Generative Adversarial Data Augmentation for Source-Only NTL Require: Source domain data S={(x,y)∥x ∼ PSX ,y ∼ PSY }; Generator G, discriminator D; List of
augmentation distance DIS, the maximum augmentation direction DIR; GAN training epochs eGAN, augmentation training epochs eAUG; Initialize the auxiliary domain data A = [ ];
Output: The auxiliary domain data A = {(x,y)∥x ∼ PAX ,y ∼ PAY }; 1 for i = 1 to eGAN do 2 use (noise,y ∼ PSY ) to optimize G with LG , use S and G(noise,y ∼ PSY ) to optimize D with LD; 3 use (noise,y ∼ PUY ) to optimize G,D with LG,D; 4 for dis in DIS do 5 for dir to DIR do 6 for l in G do 7 interval = d(l) / DIR; // function d() acquires the dimension of inputs; 8 freeze D and l[0 : dir × interval]; // freeze D, and the first dir parts of l-th layer in G; 9 for i = 1 to eAUG do
10 use (noise,y ∼ PSY ) and S to optimize G with Laug; 11 A ∪ G(noise,y ∼ PUY ); // use G to generate augmentation data;
Augmentation with Different Distances. To generate the data of different distances to the source domain, we apply Gaussian MMD to measure the distance between distributions of the source and the generated data from G. However, providing that the Gaussian MMD distance is optimized to increase with no restriction, the outcome will lose the semantic information, i.e., the essential feature for the main task. In order to preserve such semantic information, we use the CrossEntropy loss to add a restriction to the optimization objective. With this restriction, we set multiple upper bounds – DIS for generating data with different distances (we use DIS to denote a list of multiple dis-s with various lengths). The specific objective is as follows:
Laug=−max {
MMD(Px∼PS X (Dz(x)),Py∼PS Y (Dz(G(noise,y))); exp), dis } + Ey∼PS
Y DCE(Dm(G(noise,y)),y)
(6)
Here, subscript z denotes outputs from the feature extractor. For every dis, we freeze D and use Eq. (6) to optimize G. After the optimization, we can generate augmentation data via feeding G with normal noise and labels drawn from PUY . Augmentation with Different Directions. We also investigate how to generate data in different directions. The optimization of Gaussian MMD follows the direction of gradient, which is the fastest way to approach the objective. In such case, all augmented domains of different distances might follow the same direction, i.e., the direction of gradient. Therefore, in order to augment neighborhood domains with different directions, we need to introduce more restrictions to the optimization process. Specifically, for intermediate representations of G, we view each filter (neuron) as corresponding to a feature dimension of the representation. At the beginning of directional augmentation, we make multiple copies of the trained GAN in the last step (G and D), and we pick one GAN for each direction. If we want to augment the source in DIR directions, we will divide the overall network of G into DIR parts equally. For the augmentation of the first direction, the first part of G will be frozen and not updated during optimization. The second direction will be augmented by freezing the first two parts of the network and conducting the optimization. The third corresponds to the first three parts, and so on. Given a certain dis, we can optimize Eq. (6) to augment the source domain to DIR directions by freezing G gradually. The detained flow is shown in Algorithm 1.

heading: 3.3 APPLICATION OF NTL FOR MODEL INTELLECTUAL PROPERTY PROTECTION
text: Ownership Verification. The proposed NTL can easily verify the ownership of a learning model by triggering misclassification. To achieve this, we can attach a certain trigger patch which shapes like a shallow mask for images on the source domain data as the auxiliary domain data, and then conduct NTL on these two domains. After that, the training model will perform poorly on the data with the patch but have good performance on the data without the patch. Such model behavior difference can be utilized to verify the ownership, which is similar to what regular backdoor-based model watermarking approaches do. In contrast, models trained with other methods often present nearly the same performance on the data with or without the patch, as the patch is shallow and light.
Applicability Authorization. When applying NTL to authorize the model applicability, we aim to restrict the model generalization ability to only the authorized domain, where all the data is attached with a dedicated patch, and we need to make sure that the patch design will not impact the semantic information (note that the unforgeability and uniqueness of the patch are not the main consideration of this work, and we will explore it in the future). For simplicity, we use a shallow mask that is similar to that of the aforementioned ownership verification as the authorized patch. We first use our generative adversarial augmentation framework to generate neighborhood data of the original source domain. Then, we regard the source data with the patch attached as the source domain in NTL, and use the union of the original source data, the generated neighborhood data with and without the patch as the auxiliary domain. After the NTL training on these two domains, the learning model performs well only on the source domain data with the authorized patch attached, and exhibits low performance in other domains. In this way, we achieve the model applicability authorization.

heading: 4 EXPERIMENTAL RESULTS
text: Our code is implemented in PyTorch (and provided in the Supplementary Materials). All experiments are conducted on a server running Ubuntu 18.04 LTS, equipped with NVIDIA TITAN RTX GPU. The datasets and experiment settings used are introduced below.
Digits. MNIST (Deng, 2012) (MT) is the most popular digit dataset. USPS (Hull, 1994) (US) consists of digits that are scanned from the envelopes by the U.S. Postal Service. SVHN (Netzer et al., 2011) (SN) contains house number data selected from Google Street View images. MNISTM (Ganin et al., 2016) (MM) is made by combining MNIST with different backgrounds. Finally, SYN-D (Roy et al., 2018) (SD) is a synthetic dataset, which is generated by combining noisy and complex backgrounds. CIFAR10 & STL10: Both CIFAR10 and STL10 (Coates et al., 2011) are ten-class classification datasets. In order to make these two sets applicable to our problem, we follow the procedure in French et al. (2017). VisDA: This dataset (Peng et al., 2017) contains a training set (VisDA-T) and a validation set (VisDA-V) of 12 object categories. For classifying these datasets, we apply VGG-11 (Simonyan & Zisserman, 2014) for Digits Recognition, VGG-13 (Simonyan & Zisserman, 2014) for CIFAR10 & STL10, and ResNet-50 (He et al., 2016) and VGG-19 for VisDA. All networks are initialized as the pre-trained version of ImageNet (Deng et al., 2009). We use 3 random seeds (2021, 2022, 2023) to conduct all experiments three times and present the average performance. The network architectures, training parameters and error bars of experiment results can be found in the Appendix.

heading: 4.1 TARGET-SPECIFIED NTL
text: Effectiveness of NTL in Reducing Target Domain Performance. For digits sets and CIFAR10 & STL10, we pick all possible domain pairs to carry out experiments. As for VisDA, we regard the training set as the source domain and the validation set as the target. We include the results of standard supervised learning with KL divergence in the source domain and report the performance difference provided by using NTL. Table 1 and Figure 3 show results of digits sets, CIFAR10 & STL10, and VisDA. For NTL, we observe that the target performance of all pairs is degraded to nearly 10% with little accuracy reduction in the source domain. The largest performance degradation of the target domain, from 97.0% to 11.7%, occurs when the source is MM and the target is MT. Comparing NTL with supervised learning, the average relative performance degradation for the target domain of all cases is approximately 80%. These results demonstrate that Target-Specified NTL can effectively degrade the performance in the target without sacrificing the source performance.
NTL for Ownership Verification. We use a simple pixel-level mask as the trigger patch, which is shown in Figure 2 (please refer to the Appendix for more details). We use 6 state-of-art model watermark removal approaches to test the robustness of NTL-based verification: FTAL (Adi et al., 2018), RTAL (Adi et al., 2018), EWC (Chen et al., 2019), AU (Chen et al., 2019), watermark overwriting and model pruning (Rouhani et al., 2018). The settings of these methods are included in the Appendix. The results are shown in Table 2. We can see that models trained with NTL behave differently on the data with and without the patch, whereas supervised learning performs nearly the same. Furthermore, all these 6 watermark removal methods fail to improve the performance on the patched data, which indicates that NTL-based ownership verification is effective and robust to state-of-the-art watermark removal methods.

heading: 4.2 SOURCE-ONLY NTL
text: Effectiveness of NTL in Reducing Non-source Domain Performance. For all three dataset cases, we select one domain as the source and then conduct our generative adversarial augmentation to
generate the auxiliary domain. We set a series of discrete dis-s from 0.1 to 0.5 with a step of 0.1, and for each dis, we generate augmentation data of 4 directions (DIR=4). Table 3 and Figure 3 present the results of Source-Only NTL and its comparison with the supervised learning. Figure 4 is the augmentation data for MNIST (other datasets are included in the Appendix). From the results, we can clearly see that models trained with NTL perform worse on all non-source domains compared with the supervised learning, and MM-MT has the largest degradation from 97.0% to 14.7%. During the experiments, we also observe that the choice of MMD kernel bandwidth does not have a significant impact on the NTL performance. Please refer to the Appendix for more details.
NTL for Applicability Authorization. Follow the implementation steps outlined in Section 3.3, we carry out experiments on all 3 dataset cases. The experiment results of digits are presented in Table 4 (the results of CIFAR10 & STL10 and VisDA are in the Appendix). From the table, we can see that the model performs very well in the authorized domain while having bad performance in all other domains (with or without the authorized patch). The highest classification accuracy of unauthorized domains is barely 42.7%, which will discourage users from employing this model. This shows the effectiveness of NTL in applicability authorization.

heading: 5 CONCLUSION AND FUTURE WORK
text: In this paper, we propose Non-Transferable Learning (NTL), a novel training approach that can restrict the generalization ability of deep learning models to a specific data domain while degrading the performance in other domains. With the help of a generative adversarial augmentation framework, NTL is effective both in the presence and absence of target domains. Extensive experiments on 5 digit recognition datasets, CIFAR10 & STL10 and VisDA demonstrate that the ownership of models trained with NTL can be easily verified, and the verification is resistant to state-of-art watermark removal approaches. Moreover, with the training of NTL, model owners can authorize the model applicability to a certain data domain without worrying about unauthorized usage in other domains.
In future work, it would be interesting to extend NTL to other tasks beyond image classification, e.g., semantic segmentation, object detection/tracking, and natural language processing (NLP) tasks. For tasks where the input data are not images, generating augmentation data would require different methods and could be challenging. Another possible future direction is Multi-task Learning, where we could explore whether it is possible to restrict the model generalization ability to certain tasks. For instance for NLP, we think it might be useful in some cases to restrict the language model to certain tasks. Moreover, yet another interesting direction could be to combine cryptography with NTL-based verification and authorization.

heading: ETHICS STATEMENT
text: In this paper, our studies are not related to human subjects, practices to data set releases, discrimination/bias/fairness concerns, and also do not have legal compliance or research integrity issues. Non-Transferable Learning is proposed to address the shortcomings of current learning model in intellectual property protection. However, if the model trainer themselves is malicious, they may utilize NTL for harmful purposes. For example, a malicious trainer could use NTL to implant backdoor triggers in their model and release the model to the public. In addition, recently, there are domain adaptation (DA) works on adapting the domain-shared knowledge within the source model to the target one without access to the source data (Liang et al., 2020; Ahmed et al., 2021; Kundu et al., 2020). However, if the source model is trained with NTL, we believe that these DA approaches will be ineffective. In other words, our NTL can be regarded as a type of attack to those source-free DA works.

heading: REPRODUCIBILITY STATEMENT
text: The implementation code can be found in our attached .zip package, and we will also release it on GitHub once published. All datasets and code platform (PyTorch) we use are public. In addition, we also provide detailed experiment parameters and random seeds in the Appendix.

heading: SUMMARY OF THE APPENDIX
text: This appendix contains additional details for the ICLR 2022 submitted article “Non-Transferable Learning: A New Approach for Model Ownership Verification and Applicability Authorization”, including mathematical proofs, experimental details and additional results. The implementation code can be found in our attached Supplementary Materials, and we will also release it on GitHub later. In addition, The appendix is organized as follows:
• Section A introduces the theoretical proofs for Proposition 1, Theorem 1 and Theorem 2. • Section B provides additional implementation settings, including the network architectures (Sec-
tion B.1) and hyper parameters (Section B.2). • Section C provides additional experimental results, including the augmentation data of other
datasets (Section C.1), the model authorization results on CIFAR10 & STL10 and VisDA (Section C.2), the experiments of VisDA on VGG-19 (Section C.3), and the error bars of main experiment results (Section C.4). Section C.5 provides the experiment result of different kernel widths.
• In Section D, we discuss possible attacks that can be constructed based on our proposed method. Note that NTL used in this appendix is the abbreviation of Non-Transferable Learning.

heading: A THEORY PROOFS

heading: A.1 PROOF
text: Proposition 1. Let n be a nuisance for input x. Let z be a representation of x, and the label is y. For the information flow in the representation learning, we have
I(z;x)− I(z;y|n) ≥ I(z;n) (7)
Proof: According to Proposition 3.1 in (Achille & Soatto, 2018), there is a Markov Chain: (y,n) → x → z. This chain describes the information flow starting from the ground truth knowledge (label y and nuisance n) of input x to extracted representation z. In this case, the information flows from (y,n) to x then to z. The Data Processing Inequality (DPI) for a Markov Chain can ensure the relation that I(z;x) ≥ I(z;y,n). And with the chain rule, we have I(z;y,n) = I(z;n) + I(z;y|n). Thus, we can obtain I(z;x) ≥ I(z;n) + I(z;y|n). ■ Theorem 1. Let ŷ be the predicted label outputted by a representation model when feeding with input x, and suppose that ŷ is a scalar random variable and x is balanced on the ground truth label y. Denote the one-hot forms of ŷ and y as ŷ and y, respectively. If the KL divergence loss DKL(P(ŷ)∥P(y)) increases, the mutual information I(z;y) will decrease. Proof. Suppose that the information flow in classifier Ω of a representation model follow a Markov chain z → ŷ → y, in this case, let’s apply Data Processing Inequality, we have
I(z;y) ≤ I(ŷ;y) = Ey∼PY [DKL(P(ŷ|y)∥P(ŷ))] (8)
Because the input data of this representation model is balanced across classes, we suppose that both y and ŷ are drawn from the uniform distribution PUY with K equally likely possibilities. Moreover, though the distribution of ŷ might change a little during training, we can assume it won’t become very biased since the balance of input data. For both ŷ and y, they are vectors with K dimensions. In PyTorch implementation, the computation of KL divergence loss regard there is a scalar random variable corresponding to each vector and every dimension within the vector as an observation of this variable. In this case, the loss between ŷ and y forms like DKL(P(ŷ)∥P(y)) = ∑K i=1 ŷi · log ŷi yi
. It is easy to obtain that DKL(P(ŷ)∥P(y)) is non-negative, and if and only if ŷ = y, the KL divergence loss hits the minimum value DKL(P(ŷ)∥P(y)) = 0. While ŷ and y are scalar random variables that equal to the dimension index with the maximum value of ŷ and y, respectively. Therefore, it’s easy to conclude that the probability of ŷ = y will decrease with the increase of DKL(P(ŷ)∥P(y)), i.e., DKL(P(ŷ)∥P(y)) ↑⇒ P(ŷ,y) ↓. For Eq. (8), we can derive it deeper
I(ŷ;y) = Ey∼PY [DKL(P(ŷ|y)∥P(ŷ))] = ∑ y P(y) · ∑ ŷ P(ŷ,y) P(ŷ) log P(ŷ,y) P(ŷ) · P(y) (9)
Here, both P(ŷ) and P(y) are uniform distributions PUY , and we have assumed P(ŷ) won’t become very biased at the beginning of this proof. As a result, we regard P(ŷ) and P(y) nearly unchanged
after training. In addition, P(ŷ,y) decreases with the increase of DKL(P(ŷ)∥P(y)). Furthermore, we can easily calculate that ∂I(ŷ;y)∂P(ŷ,y) < 0. In this case, I(ŷ;y) decreases with the increase of DKL(P(ŷ)∥P(y)), and the same as I(z;y) since I(ŷ;y) is the upper bound. ■ Theorem 2. Let n be a nuisance that is regarded as a domain index. n=0 and n=1 denote that a certain input x comes from two different domains. Suppose that these two domains have the same number of samples d, and the samples of each domain are symmetrically distributed around the centroid. Let z be a representation of x, and it is drawn from distribution PZ . An estimator with the characteristic kernel from Reproducing Kernel Hilbert Spaces (RKHSs) – Gaussian Kernel estimator MMD(P,Q; exp) is applied on finite samples from distributions PZ|0 and PZ|1 to approximate the Maximum Mean Discrepancy (MMD) between these two distributions. If MMD(PZ|0,PZ|1; exp) increases to saturation, the mutual information between z and n will increase.
MMD(PZ|0,PZ|1; exp)=Ez,z′∼PZ|0 [e −∥z−z′∥2 ]−2Ez∼PZ|0,z′∼PZ|1 [e −∥z−z′∥2 ]+Ez,z′∼PZ|1 [e −∥z−z′∥2 ]
(10)
Proof. According to the definition of Shannon Mutual Information, we have
I(z;n) = En∼P(n)DKL(P(z|n)∥P(z)) = En∼P(n)Ez∼P(z|n) log P(z|n) P(z)
(11)
And because two domains have the same number of samples, we can have n conforms P(n) ∼ {P(0)=0.5, P(1)=0.5}, Eq. (11) can re-written as
I(z;n) = 0.5Ez∼P(z|n=0) log P(z|n = 0)
P(z) + 0.5Ez∼P(z|n=1) log P(z|n = 1) P(z)
(12)
Next, we denote the probability density function (PDF) of PZ|0 and PZ|1 as p(z) and q(z), respectively. Moreover, according to the law of total probability, the PDF of distribution PZ is PDF(PZ) = 0.5p(z) + 0.5q(z), then we have
I(z;n) = 0.5 ∫ +∞ −∞ p(z) · log 2p(z) p(z) + q(z) dz + 0.5 ∫ +∞ −∞ q(z) · log 2q(z) p(z) + q(z) dz (13)
Subsequently, we denote expectations and variances of PZ|0 and PZ|1 as (µ0, σ0) and (µ1, σ1), respectively. With the assumption that samples of each domain are symmetrically distributed about the centroid, we have pm = max{p(z)} = p(µ0) and qm = max{q(z)} = q(µ1). Thus if we use two variables f and g to denote PDF-s of PZ|0 and PZ|1, i.e., f = p(z) and g = q(z), in this case, we have f ∈ (0, pm] and g ∈ (0, qm]. Based on the above analysis and notations, we can split Eq. (13) into 4 terms as follows,
I(z;n) = 0.5 ∫ pm 0 f · log 2f f + g df + 0.5 ∫ pm 0 f · log 2f f + g′ df
+ 0.5 ∫ qm 0 g · log 2g f + g dg + 0.5 ∫ qm 0 g · log 2g f ′ + g dg
(14)
here the superscript ′ indicates the right side of f and g. Next, let us consider the Gaussian Kernel estimator MMD(PZ|0,PZ|1; exp). The estimator consists of 3 terms, and we can easily conclude that e−∥z−z
′∥2 decreases with the increase of ∥z − z′∥2. Note that ‘MMD(PZ|0,PZ|1; exp) increases to saturation’ means at least one term increases while the other two terms remain unchanged or increased. For next proof, we need to mention Theorem 2 in (Sriperumbudur et al., 2009).
Theorem 2 (Sriperumbudur et al., 2009). Suppose {(Xi, Yi)}Ni=1, Xi ∈ M,Yi ∈ {−1,+1}, ∀i is a training sample drawn i.i.d. from µ. Assuming the training sample is separable, let fsvm be the solution to the program, inf{∥f∥H : Yif(Xi) ≥ 1,∀i}, where H is an RKHS with measurable and bounded kernel k. If k is characteristic, then
1 ∥fsvm∥H ≤ γk(P,Q) 2 (15)
where P := 1d ∑ Yi=+1 δXi , Q := 1d ∑ Yi=−1 δXi , d is the sample quantity and δ represents the Dirac measure. This theorem provides a bound on the margin of hard-margin SVM in terms of
MMD. Eq. (15) shows that a smaller MMD between P and Q enforces a smaller margin (i.e., a less smooth classifier, fsvm, where smoothness is measured as ∥fsvm∥H). Besides, the Gaussian kernel is a measurable and bounded kernel function in an RKHS. According to this theorem and the nature of hard-margin SVM, we can easily obtained that variances σ0, σ1 of PZ|0 and PZ|1 are decreasing and the difference between expectations µ0 and µ1 is increasing with the saturated increase of MMD, and this conclusion can be found in (Jegelka et al., 2009).
In the following, we will prove that I(z;n) will increase when the difference between µ0 and µ1 increases. Due to the symmetry of PDFs, both f and g increase at the left side of their own expectation and decrease at the right side. Without the loss of generality, we assume that µ0 locates at the left side of µ1. For the first term of Eq. (14) which corresponds to the left interval of f , the value of g is smaller than that of the case before increasing the difference between µ0 and µ1. Thus this term will increase when the difference between µ0 and µ1 increase. As for the right interval of f , the maximum value of f + g′ (in the neighborhood of µ1) comes later than that of case before increasing the difference, besides, the maximum value is also smaller. There, for the second term of Eq. (14), it will also increase with the increase of difference between µ0 and µ1. Similarly, the integration of g follows the same trend with that of f . In this case, I(z;n) will increase when the difference between µ0 and µ1 increases.
Next, we will prove that I(z;n) will increase if the variance of either PZ|0 or PZ|1 decreases. Without the loss of generality, we assume the variance of PZ|0 decreases while the variance of PZ|1 remain unchanged. For the PDF of a distribution, if the variance decreases, the maximum value of PDF will increase, and there will be two points of intersection with the same value between the PDF-s. Such conclusions are easily proved since the integration of a PDF is always 1. We denote the new maximum value of f as p′m, and the value of two points of intersection as p=. In addition, during the saturated increase of MMD, we can always find a pair of µ0 and µ1 that enables the left side of g to intersect with the right side of f . With the notations in Figure 5. We can change Eq. (14) into
Terms of f = ∫ p= 0 f · log 2f f + g df + ∫ p′m p= f · log 2f f + g df + ∫ p′m pm f · log 2f f + g′ df
+ ∫ pm p= f · log 2f f + g′ df + ∫ p= pµ1 f · log 2f f + g′ df + ∫ pµ1 0 f · log 2f f + g′ df (16)
Terms of g = ∫ q1= 0 g · log 2g f + g dg + ∫ q2= q1= g · log 2g f + g dg
+ ∫ qm q2= g · log 2g f + g dg + ∫ qm 0 g · log 2g f ′ + g dg
(17)
According to the decrease of σ0, we can conclude: the 1st, 4th, 6th terms of Eq. (16) and the 2nd term of Eq. (17) decrease, while the rest terms of Eq. (16) and Eq. (17) increase. For next proof, we denote a new function, R(f, g) = f · log 2ff+g , and we can get its first-order derivative is ∂R∂f = log 2f f+g + g f(f+g) . We can easily obtain that ∂R ∂f > 0 when f > g. According to these analysis, we can get that the decrease of the 1st term of Eq. (16) can be offset by the added increase of the 5th term of Eq. (16) and the 3rd term of Eq. (17); the decrease of the 4th term of Eq. (16) can be offset by the 2nd term of Eq. (16); the decrease of the 6th term of Eq. (16) can be offset by the 4th term of Eq. (17); the decrease of the 2nd term of Eq. (17) can be offset by the 3rd term of Eq. (16). Moreover, such offsets are overcompensation. In this case, we can prove that I(z;n) will increase if the variance of either PZ|0 or PZ|1 decreases. Considering the combination of the above two cases, if the difference between expectations increases and variances of two distributions decrease, the mutual information will increase. ■

heading: A.2 OBSERVE THE MUTUAL INFORMATION
text: We follow the similar process of (Achille & Soatto, 2018) to observe the change of mutual information I(z;n). To be specific, let Θ be a binary classifier that given representation z and nuisance n tries to predict whether z is sampled from the distribution of one domain PZ|0 or another domain PZ|1. According to (Sønderby et al., 2016), if we train Θ with the loss of Ez∼PZ|0(logΘ(z)) + Ez∼PZ|1(log 1−Θ(z)), there is always a Bayes-optimal Θ∗,
Θ∗ = P(z|n = 0)
P(z|n = 0) + P(z|n = 1) (18)
With Eq.(12), if we assume the Θ0,Θ1 trained with Ez∼PZ|0(logΘ0(z))+Ez∼PZ|1(log 1−Θ0(z)) and Ez∼PZ|1(logΘ1(z)) + Ez∼PZ|0(log 1−Θ1(z)), respectively, are close to the optimal ones Θ∗0,Θ ∗ 1, we have
I(z;n) = 0.5Ez∼P(z|n=0) log P(z|n = 0)
P(z) + 0.5Ez∼P(z|n=1) log P(z|n = 1) P(z)
= 0.5Ez∼P(z|n=0) log 2Θ0(z) + 0.5Ez∼P(z|n=1) log 2Θ1(z) (19)
With this approximation, we train Θ0 and Θ1 for the model of every NTL training round, and we get the curve of I(z;n) shown in Figure 6 (MNIST). According to the figure, I(z;n) is increasing during the overall training process, which is consistent with our intention.
B IMPLEMENTATION SETTINGS

heading: B.1 NETWORK ARCHITECTURE
text: To build the classification models, we use several popular architectures as the bottom feature extractor and attach them with fully-connected layers as the top classifier, which are shown in Table 5. Specifically, the backbone network of digits is VGG-11, that of CIFAR10 & STL10 is VGG-13, and we use both ResNet-50 and VGG-19 for VisDA. The classifiers of all models are the same, i.e., 3 linear layers with ReLU and dropout. As for the GAN in the augmentation framework, the generator G is made up of 4 ConvTranspose blocks and 2 Residual blocks, and the discriminator D consists of a feature extractor with 4 convolution layers, a binary classifier and a multi-class classifier. These two classifiers are composed of sequential fully-connected layers and share the same representations extracted from the front extractor. The detailed architecture is shown in Table 6 and 7.

heading: B.2 HYPER PARAMETERS
text: Scaling factors and upper bounds. As introduced in Section 3.1 of the main paper, there are two scaling factors (α, α′) that control the trade-off between the maximization of I(z;n) and the sufficiency property of the source domain. Here, we conduct experiments using different values (α = 0.01, 0.05, 0.10, 0.20, 0.50 and α′ = 0.01, 0.05, 0.10, 0.20, 0.50), and evaluate their impact to the performance of NTL. For Target-Specified NTL, we select the combination of MNIST→USPS, STL10→CIFAR10 and VisDA-T→VisDA-V. For Source-Only NTL, we choose MNIST→Non-S, STL10→Non-S and VisDA-T→Non-S as the representatives to carry out experiments. The results are presented in Tables 8 and 9. It is easy to conclude that NTL can work effectively with different
scaling factors. As for the upper bounds (β, β′), we set them for the sake of preventing the auxiliary domain loss and the MMD distance from dominating the optimization objective, affecting the convergence of training.
Training parameters. For the optimization of NTL, we utilize Adam as the optimizer, with learning γ = 0.0001 and batch size of 32. For all datasets, we randomly select 8,000 samples from their own training sets as the source data, and 1,000 samples from their own testing sets as the test data (if a dataset does not have test set, we select its test data from the training set without overlapping with the chosen 8,000 source samples). And the sample quantities of the source and auxiliary domain are always the same. In the training of adversarial augmentation, the optimizer is also Adam, and we set the learning rate to γ = 0.0002 with two decay momentums 0.5 and 0.999. The batch size is 64, and the dimension of the latent space fed to the generator is 256.

heading: B.3 TRIGGERING AND AUTHORIZATION PATCH
text: As mentioned in Section 4.2 and 4.3 of our main paper, we attach a patch on the data to utilize NTL for ownership verification and usage authorization. We create the patch in a simple way. Specifically, for the pixel of i-th row and j-th column in an RGB image, if either i or j is even, then a value of v is added to the R channel of this pixel (the channel value cannot exceed 255). Intuitively, the patch is dependent on pixel values of each image. Thus the changes of feature space brought by the attachment of these patches for various images are not the same. In our experiments, if the content of the image is simple, e.g., MNIST, USPS and SVHN, the v with a small value can shift the feature space sufficiently, but for more complicated images, we have to increase v to enable source images attached with and without the patch differentiable. Specifically, we pick the value as follows: MNIST, USPS, SVHN (v = 20); MNIST-M, SYN-D, CIFAR10, STL10 (v = 80); VisDA (v = 100). As mentioned in the main paper, we will explore the unforgeability and uniqueness of patch generation in the further work.
B.4 IMPLEMENTATION OF WATERMARK REMOVAL APPROACHES In the Section 4.2 of the main paper, we implement 6 model watermark removal approaches to verify the effectiveness of NTL-based ownership verification. Here, we introduce how to implement these approaches. FTAL (Adi et al., 2018) is an approach that fine-tunes the entire watermarked model
using the original training data. To implement it, we use 30% of training set that has been learned by NTL to fine-tune the entire model. When using RTAL (Adi et al., 2018), the top classifier is randomly initialized before fine-tuning. In our experiments, we load the feature extractor of the model trained with NTL and randomly initialize a classifier to attach on the extractor, and then use 30% of the training set to fine-tune this combined model. As for EWC (Chen et al., 2019), we use the code of (Chen et al., 2019) to compute the fisher information of network parameters and adjust the learning rate of fine-tuning. The data used by EWC is also 30% of the training set. Finally, AU (Chen et al., 2019) utilizes the watermarked model to pseudo label additional unlabeled samples from other similar domains, and these samples will be used to fine-tune the model together with the original training set. Following this principle, we use 30% of the training set and the same quantity of unlabeled samples from other domains (the proportion ratio between these two parties is 1:1) to fine-tune the model trained with our NTL. We conduct all fine-tuning methods for 200 epochs. For the watermark overwriting, we overwrites a new backdoor-based watermark (Zhang et al., 2018) on the model trained with NTL. Specifically, we attach a white corner (3 × 3) as the backdoor trigger to 1/15 of the training set, and follow the training approach of (Zhang et al., 2018) to write the watermark on the model. In addition, similar to other watermarking works (Rouhani et al., 2018), we also test if NTL-based verification is resistant to model pruning, and apply a layer-wise pruning method (Han et al., 2015) to prune 70% parameters of the model trained with NTL.

heading: C ADDITIONAL EXPERIMENTAL RESULTS

heading: C.1 AUGMENTATION DATA OF OTHER DATASETS
text: In the main paper, we present the augmentation data of MNIST, and in this section, we include the augmentation data of other datasets as follows: Figure 7 for USPS, Figure 8 for SVHN, Figure 9 for MNIST-M, Figure 10 for SYN-D, Figure 11 for CIFAR10, Figure 12, Figure 13 for VisDA-T.

heading: C.2 MODEL USAGE AUTHORIZATION ON CIFAR10 & STL10 AND VISDA
text: Here we present the experiment of authorizing the model usage on CIFAR10 & STL10 and VisDA, shown in Table 10. According to the results, the model performs well on the data attached with the authorized patch and has bad performance on all other samples.

heading: C.3 ADDITIONAL RESULTS OF VISDA ON VGG-19
text: To demonstrate the effectiveness of NTL on different network architectures, we also carry out experiments of VisDA on VGG-19. All other settings are the same as before, and the results are shown in Table 11. We can easily see that the performance is consistent with the aforementioned other experiments, which shows the wide applicability of NTL.

heading: C.4 ERROR BAR
text: We conduct all experiments with three random seeds (2021, 2022, 2023), and present the error range in this section. Table 12 is the error range of Target-Specified NTL corresponding to Table 1 of the main paper; Table 13 presents the error of experiments on Source-Only NTL corresponding to Table 3 of the main paper; Table 14 shows the error of model authorization which is presented as Table 4 in our main paper.

heading: C.5 THE IMPACT OF GAUSSIAN KERNEL BANDWIDTH
text: In our implementation, we utilize a series of Gaussian kernels to approximate MMD, which is implemented as the MK-MMD in Long et al. (2015). Specifically, the bandwidth in our used kernels is controlled by two parameters mul and num (we use mul = 2.0, num = 5 in our experiments presented in the main text of the paper). The bandwidth of these kernels is as follows:
B =
{ ∥x1 − x2∥2 · muli−⌊num/2⌋
(n2 − n)
}num−1 i=0
(20)
where x1 and x2 are two input data batches with size n, and ⌊·⌋ extracts the integer part of the input. To investigate the impact of kernel bandwidth, we select a series of mul-s and num-s to conduct Source-Only NTL experiments on MNIST, CIFAR10 and VisDA-T, and the results are shown in
Table B.4. According to the results, we can observe that the performance difference between the source and target is nearly the same with different mul-s and num-s. As these two parameters directly determine the kernel bandwidth, these results demonstrate that the kernel bandwidth does not have a significant impact on NTL performance.

heading: D POSSIBLE ATTACKS BASED ON NTL
text: Although we propose the Non-Transferable Learning for protecting the Intellectual Property in AIaaS, if the model owner is malicious, they can also utilize NTL to poison or implant backdoor triggers to their model evasively and release the model to the public. In the setting of applying
Target-Specified NTL to verify the model ownership, the patch we used can also be regarded as a trigger for certain misclassification backdoor. From the results of ownership verification in the main paper, we can see the possibility of launching NTL-based target backdoor attacks. As for the case of Source-Only NTL, our objective is shaped like an universal poison attack by restricting the generalization ability of models. The results in our main paper demonstrate the feasibility of this poison attack. In addition, recently, there are more domain adaptation (DA) works about adapting the domain-shared knowledge within the source model to the target one without the access to the source data (Liang et al., 2020; Ahmed et al., 2021; Kundu et al., 2020). However, if the source model is trained with Source-Only NTL, we believe that these DA works will be ineffective. In other words, our NTL can be regarded as a type of attack to these source-free DA works.

Review summaries:
In the era of deep learning, pre-trained models have been regarded as intellectual properties of AI companies. Thus, protecting these models has been more and more important. To achieve this aim, this paper proposes a non-transferable learning (NTL) method to capture the exclusive data representation in the learned model and restrict the model generalization ability to certain domains. This approach provides effective solutions to both model verification and authorization. Specifically:  1) For ownership verification, watermarking techniques are commonly used but are often vulnerable to sophisticated watermark removal methods. By comparison, the NTL-based ownership verification provides robust resistance to state-of-the-art watermark removal methods, as shown in extensive experiments with 6 removal approaches over the digits, CIFAR10 & STL10, and VisDA datasets. 2) For usage authorization, prior solutions focus on authorizing specific users to access the model, but authorized users can still apply the model to any data without restriction. The NTL-based authorization approach instead provides a data-centric protection, which is called applicability authorization, by significantly degrading the performance of the model on unauthorized data. In general, this paper contributes a novel method to the field, and experiments verified the success of the proposed method. 

This paper introduces the idea of "non-transferable learning", which is roughly what the name indicates. The authors explain the value of this as a security/IP protection tool to protect the model from being used on unauthorized data. In addition, this presents a kind of attack against domain adaption works that try to improve generalization bounds without access to source data.

Protecting the intellectual property of the trained models has received appealing attentions. Existing researches to protect  intellectual property fall into two major categories: ownership verification and usage authorization. To this end, the authors propose to utilize non-transferable learning to achieve both the goal of ownership verification and usage authorization. Extensive experiments on several representative datasets validate the effectiveness of the proposed method in terms of ownership verification.   Generally, this paper proposes a novel idea to address a practical problem in real-world applications, which could inspire many readers to follow it and have an important influence on the community of computer vision. I support the acceptance of this paper for a better ICLR conference.