Non-Transferable Learning: A New Approach for Model Ownership Verification and Applicability Authorization
Authors: Lixu Wang
Shichao Xu
Ruiqi Xu
Xiao Wang
Qi Zhu
Venue: ICLR 2022 Oral (2022)
Keywords: Domain Adaptation
Transfer Learning
Societal Considerations of Representation Learning
Model Watermark

Abstract: As Artificial Intelligence as a Service gains popularity, protecting well-trained models as intellectual property is becoming increasingly important. There are two common types of protection methods: ownership verification and usage authorization. In this paper, we propose Non-Transferable Learning (NTL), a novel approach that captures the exclusive data representation in the learned model and restricts the model generalization ability to certain domains. This approach provides effective solutions to both model verification and authorization. Specifically: 1) For ownership verification, watermarking techniques are commonly used but are often vulnerable to sophisticated watermark removal methods. By comparison, our NTL-based ownership verification provides robust resistance to state-of-the-art watermark removal methods, as shown in extensive experiments with 6 removal approaches over the digits, CIFAR10 & STL10, and VisDA datasets. 2) For usage authorization, prior solutions focus on authorizing specific users to access the model, but authorized users can still apply the model to any data without restriction. Our NTL-based authorization approach instead provides data-centric protection, which we call applicability authorization, by significantly degrading the performance of the model on unauthorized data. Its effectiveness is also shown through experiments on aforementioned datasets. 

Published as a conference paper at ICLR 2022
NON-TRANSFERABLE LEARNING: A NEW APPROACH FOR MODEL OWNERSHIP VERIFICATION AND APPLI- CABILITY AUTHORIZATION
Lixu Wang'; Shichao Xu", Ruiqi Xu, Xiao Wang, Qi Zhu
Northwestern University Evanston, IL 60208, USA
(1 [1ixuwang2025, sh ahiichhaax2222, Jerryxu2023jeu. how.:/eeee.....ooooeetttte.e..e
[wangxiao, gww.eeenorrbweestter.....
ABSTRACT
As Artificial Intelligence Service gains popularity, protecting well-trained models as intellectual property is becoming increasingly important. There are two common types of protection methods: ownership verification and usage autho- rization. this paper, we propose Ten--ransferable Learning (NTL), a novel approach that captures the exclusive data representation in the learned model and restricts the model generalization ability to certain domains. This approach pro- vides effective solutions both model verification and authorization. Specifically: 1) For ownership verification, watermarking techniques are commonly used but are often vulnerable sophisticated watermark removal methods. By compari- son, our NTIL-based ownership verification provides robust resistance to state- of-the-art watermark removal methods, as shown in extensive experiments with 6 removal approaches over the digits, CIFARIO & STL10, and VisDA datasets. 2) For usage authorization, prior solutions focus on authorizing specific users to ac- cess the model, but authorized users still apply the model to any data without restriction. Our NTL-based authorization approach instead provides data-centric protection, which we call applicability authorization, by significantly degrading the performance of the model on unauthorized data. Its effectiveness is also shown through experiments on aforementioned datasets.
INTRODUCTION
Deep Learning (DL. is the backbone of Artificial Intelligence as Service (AlaaS) (Ribeiro al., 2015), which is being provided in a wide range of applications including music composition (Briot al., 2020), autonomous driving (Li et al., 2021a), smart building (Xu al., 2020a), etc. However, a good model can be expensive to obtain: it often requires dedicated architecture design (He et al., 2016), a large amount of high-quality data (Deng et al., 2009), lengthy training on professional devices (Zoph & Le, 2016), and expert tuning (Zhang et al., 2019). Thus, well-trained DL models are valuable intellectual property (IP) to the model owners and need protection. Generally speaking, there are two aspects in protecting an IP in AlaaS, verifying who owns the model and authorizing how the model can be used. These two aspects led to the development of two types of protection techniques: ownership verification and usage authorization.
For ownership verification, prior works proposed approaches such as embedding watermarks into network parameters (Song et al., 2017), learning special behaviors for pre-defined triggers (Fan et al., 2019), and extracting fingerprints from the model (Le Merrer et al., 2020). However, they vulnerable to state-of-art watermark removal approaches that are based model fine-tuning or retraining (Chen et al., 2019), watermark overwriting and model pruning (Rouhani et al., 2018). For model usage authorization, most prior works with a secret key (Alam et al., 2020; Chakraborty et al., 2020) and ensuring that models can only be used by users with this key. However, authorized users may use the model on any data without restriction. We believe that for comprehensive IP protection, the goal of usage authorization is not
built on encrypting neural network parameters
"These authors contributed equally this work.

---

Published as a conference paper at ICLR 2022
Generalization bound
Reduced model generalization bound
Outside domain
Source domain
Source
Source domain
domain
Supervised Learning
Target-Speciffed NTL
Source-Only NTL
Figure 1: A visualization of the generalization bound trained with different approaches. The left figure shows Supervised Learning in the source domain, which can derive a wide generalization area. When Barget-Specified NTL is applied (middle), the target domain is removed from the gen- eralization area. As for Source-Only NTL (right), the generalization area is significantly reduced.
only who is allowed to use the model, but also what data can the model be used on. We thus consider a new data-centric aspect of usage authorization in this work, i.e., authorizing models to certain data for preventing their usage on unauthorized data. We call this applicability authorization. Note that applicability authorization goes far beyond IP protection. It can also be viewed as way to "control" how machine learning models are used in general. One example would be a company (e.g.. Meta) trains recommendation system from adult data and uses applicability authorization to prevent this system from being used by teenagers.
Our Approach and Contribution. In this work, propose Non- -en-Transferable Learning (NTL), a novel approach that can robustly verify the model ownership and authorize the model applicability on certain data. Intuitively, NTL goes against the current research trend of improving the general- ization ability of models across various domains, e.g., domain generalization and adaptation (Zhou al., 2020; Dong et al., 2020). Instead, NTL tries to make the generalization bound of DL models more explicit and narrower, by optimizing the model to learn Nomain-dependee features and thereby making the model exclusive to certain domains. More specifically, we consider two domains: the source domain where we want the models to perform well, and the auxiliary domain where we aim to degrade the model performance. And if the model trained with NTL is applied to a target domain similar to the auxiliary one, the performance should also be poor. As shown in Figure 1, we have developed two types of NTL approaches: Target-
NTL and Source-Only NTL.
Target-Specified NTL assumes that the source and target domains are both known. We then treat the target domain as the auxiliary domain and enlarge the distance of representations between the source and auxiliary domains. Target-Specinee
NTL can be used to verify the model ownership by triggering misclassification. While previous model watermarks can often be easily removed because the model memorization of such watermarks encounters catastrophic forgetting (Kemker et al., 2018) during watermark removal, our NTL-baaed verification is resistant to state-of-art watermark removal approaches, because the misclassification behavior is dependent on the overall target-private features that have little correlation with the source-private features for the main task.
Source-Only NTL, the target domain is unknown and thus our approach relies solely on the source domain, aiming to degrade the performance in all other domains. In this case, NTL gener- ates the auxiliary domain from a novel generative adversarial augmentation framework and then increases the representation distance. Cource-Only NTL can provide authorization to certain data rather than particular users or devices, by degrading the model performance on all other data do- mains other than the source domain. This provides data-centric applicability authorization, with which we can also prevent unauthorized model usage that are caused by the secret key leakage and cannot be addressed by prior model authorization methods.
In addition to proposing the novel concept of NTL and developing its two approaches, we are also able to experimentally validate their effectivenees. We conducted extensive experiments 5 digit sets, CIFARIO & STL10 and VisDA. For target-specified cases, we demonstrate how to apply NTL for model ownership verification. Our experiments show that the state-of-art model watermark removal methods are ineffective on NTLL-basd ownership verification. For source-only NTL, our experiments demonstrate its effectiveness in authorizing model applicability certain data.
2 RELATED WORK
Domain Generalization & Adaptation (DG & DA). DG aims to generalize learning models with available source domains to unseen target domains (Blanchard et al., 201 1). A number of methods
2

---

Published as a conference paper at ICLR 2022
have been proposed for domain discrepancy minimization (Li et al., 2020), adversarial training (Rah- man et al., 2020; Zhao et al., 2020c), invariance representation learning (Zhou et al., 2020; Piratla et al., 2020), etc. Recently, there is significant interest conducting DG with one source domain only, for which well-crafted data augmentation approaches (Qiao et al., 2020; Zhao et al., 2020b; Li al., 2021b; Xu et al., 2020b) have been proposed to expand the input space. DA is also related to improving the generalization ability of models across domains (Ahmed et al., 2021), and while DA can access the target data, DG has no access any target sample (Xu et al., 2021; Dong et al., 2021). Unlike DG or DA, we try to weaken the generalization ability of models by expanding the distance between representations of different domains. Our method works effectively for both the target-specified and the source-only cases with a novel adversarial augmentation framework.
Intellectual Property (IP) Protection for Deep Learning (DL)) While DL. has shown its unparal-
leled advantages in various applications, there are significant challenges in protecting DL models. For instance, Inference Attack (Shokri et al., 2017; Wang et al., 2019) can steal private information about the target DL model. Model Inversion Attack (He et al., 2019; Salem et al., 2020) is able to recover the input data via an analysis of the model prediction. These two types of attacks directly threaten the privacy of model users, while there are also many active attacks (Suciu et al., 2018; Yao et al., 2019) that lead DL models to produce abnormal behaviors.
In addition, verifying model ownership and authorizing model usage have become important issues with the development of AlaaS. There have been a number of watermarking approaches address- ing the verification of model ownership. For instance, Zhang al. (2018) and Li et al. (2019) train a neural network on the original datasets and the watermarked one assigned with particu- lar label, which makes the model behave abnormally when encounters watermarked data. Song et al. (2017) and Uchida et al. (2017) inject a pattern that is similar to regular photograph water- marks (Cheng et al., 2021) into the least significant bits of the model parameters and provide the corresponding decoding methods. Le Merrer et al. (2020) and Zhao et al. (2020a) make use of ad- versarial examples to extract fingerprints from learned neural networks without accessing network weights. Compared to these approaches, our NTI. can achieve model ownership verification by triggering universal misclassification. Moreover, with extensive experiments, we also demonstrate that state-of-art model watermark removal methods, e.g., FTAL and RTAL (Adi et al., 2018), EWC and AU (Chen et al., 2019), watermark overwriting and model pruning (Rouhani et al., 2018) are not effective to NTL-based verification. Model usage authorization is another aspect in protecting model intellectual property. For instance, Alam et al. (2020) encrypt every network parameter with secret key. Chakraborty al. (2020) generate a secret key from hardware fingerprints of par- ticular device, and require that only users who possess this device can load and employ the model. Different from these methods, our NTL focuses on providing data-centric protection via applicabil- ity authorization, which retains good model performance on authorized data while degrading model performance for other data domains. To the best of our knowledge, this is the first work that prevents model usage on unauthorized data via model learning.
3 METHODOLOGY
In this section, we introduce our NTL approach. Section 3.1 presents the inspiration and the design of the optimization objective of NTL, which is the core for both target-specified and source-only cases. Section 3.2 presents the generative augmentation framework for source-only cases. Our method is based on the concept of generative adversarial networks (GAN), however our goal is not to propose a new GAN but to design an effective augmentation method in the context of NTL. Sec- tion 3.3 introduces the application of NTL on ownership verification and applicability authorization.
3.1
LEARNING WITH DISTANCE EXPANSION OF REPRESENTATION
We consider a source domain with labeled samples S ((x, y) ~ PS y~ Pp where Px and Py are the input and label distributions, respectively. In this work, we use image classification as the learning task with K possible classes, in which case æ and y are matrix-valued and scalar random variables, respectively. In addition, we consider an auxiliary domain A = [(x,, y) - y~ Pd) ). The source domain S and the auxiliary domain A will be fed into a deep neural network, and without loss of generality, we split the neural network into two parts, one is feature extractor Ob on the bottom, and the other is a classifier Q on the top.
3

---

Published as a conference paper at ICLR 2022
Inspiration from Information Bottleneck. Our NTL, in particular the design of optimization ob- jective, is inspired by the analysis of Information Bottleneck (IB) (Tishby et al., 2000). Let us start by introducing Shannon Mutual Information (SMI). In addition imput æ and label y, we also regard representation z extracted by 4 as a random variable. The SMI between two random variables, e.g. between z and a; is defined as (2: x) = Exx-P x [Dx. (P(()))) (=)x=))|((=)) where (-) represents the Kullback-Leible (KL) divergence and ((-) is the distribution. In IB theory, considering the effective- ness, privacy and generalization, an optimal representation has three properties (Achille & Soatto, 2018): (1) Sufficiency: label y sufficiently differentiates representation z, i.e., [[2; y) [x:: y); (2) Minimality: x needs to represent as little information about input a as possible, ie.. min [ a); (3) Invariance: is optimal, meaning that it does not overfit to spurious correlations between y and nuisance n embedded in æ, i.e., I [(z; n = O. IB theory assumes that nuisance n. is a factor that affects input æt, and it works with y together to determine what x: looks like to some extent. For in- stance, in domain generalization, nuisance n can be regarded as a domain index that indicates which domain certain sample comes from (Du et al., 2020). In our problem, different from the objective of the IB theory, NTL. enforces the models to extract nuisance-dependent representations, which is opposite to the property of invariance. In other words, we aim to increase [(2; ri), and we have the following proposition for achieving this aim.
Proposition| 1. Let be nuisance for input x. Let z be a representation of x. and the label is y. For the information flow in the representation learning, we have
[22;z) I(2; y|m) 2 I(2;n)
(I)
The detailed proof for Proposition 1 is included in the Appendix.
Optimization Objective Design. Proposition 1 provides guidance for maximizing [(z; n2). First, unlike in the IB theory, we do not minimize [(:; x)) for the minimality property. In addition, we try to minimize [(2; y|m) through the design of optimization objective that measures the error between the model prediction and the ground truth during the training of neural networks. Specifically, instead of using the typical CrossEntropy loss measure the error, we apply KI. divergence loss to direct the training, and we have the following theorem.
Theorem 1.
Let ÿ be the predicted label outputted by a representation model when feeding with imput x, and suppose that ÿ is a scalar random variable and æ is balanced on the ground truth label y. Denote the one-hot forms of y and y as y and y. respectively. If the KL divergence loss DxL(P()) (P()))((()) increases, the mutual information I [(z: 3) will decrease.
The detailed proof of Theorem is provided in the Appendix. According to this theorem, [(: y y|m) can be minimized by increasing the KI. divergence loss of training data conditioned on different n. However, as stated in Section 1, we aim to degrade the model performance in the auxiliary domain while maintaining good model performance in the source domain. Thus, we only minimize I [zz; y/m) by increasing the KL divergence loss of the auxiliary domain data. In order to achieve this goal, we design loss Lnu that shapes like a minus operation between KL divergence losses of the source and auxiliary domain (Ls, LA), i.e., Ls- = Eg-p& (P (P(()) and
LA LA=Bawpq [DxL (P(((( ())))) Specifically, this loss can be written as follows:
Lue Ls min(3, a LA)
(2)
Here, a is the scaling factor for LA (a = 0.1 in our experiments), and ,3 is an upper bound when La gets too large and dominates the overall loss (3 = 1.0 in experiments; please see the Appendix for more details about a and 3). Moreover, if we use n = 0 and n = 1 to denote the source and auxiliary domain respectively, the optimization of Eq. (2) can guarantee the sufficiency property for the source domain: [(2; y n = [( y y||n=), =)) and increasing LA decreases [(2; y n
According to Proposition 1, we can move the upper bound of [(z: n) higher bascline via opti- mizing Eq. (2). However, such optimization might only make classifier l2 more sensitive to domain features and have little effect on feature extractor क. In this case, representations of different do- mains captured by & may still be similar, which conflicts with our intention to maximize T(2; n), and the performance of the target can be easily improved by fine-tuning or adapting 5 with a small number of labeled target samples. On the other hand, directly calculating [(; n) and taking it as part of the optimization objective are difficult, especially in the optimization of representation learn- ing (Torkkola, 2003). Achille & Soatto (2018) apply binary classifier the nuisance discriminator, and they can estimate I [2;; after the model training via this discriminator. Here, we find another way to increase 7(2;; n) indirectly based on the following theorem.
4

---

Published as a conference paper at ICLR 2022
Theorem 2. Let be a nuisance that is regarded as a domain index. n= and n= denote that a certain input x comes from two different domains. Suppose that these two domains have the same number of samples d, and the samples of each domain are symmetrically distributed around the centroid. Let z be a representation of x. and it is drawn from distribution Pz. An estimator with the characteristic kernel from Reproducing Kernel Hilbert Spaces (RKHSs)- Gaussian Kernel estimator (M Q; exp) is applied on finite samples from distributions P Pzío and Pzn to approximate the Maximum Mean Discrepancy (MMD) between these two distributions. If MMD(P210: Pzz exp) increases saluration, the mutual information between 2 and nu will increase.
MMD(P2jo: P2t1: exp) = =R 11=->110 - 22joo ~2 le -1=-=1=11 I+ Pxn [e 112--- UP1
(3)
We also employ nuisance discriminator to observe the change of I(z; n) during training. The details of this discriminator design and the proof of Theorem can be found in the Appendix.
NTL Optimization Objective. Based on the above analysis, we design our NTL optimization ob- jective to increase ((2; n) and extract nuisance-dependent representations. Specifically, we compute the MMD(P, Q: ezp) between representations of the source and auxiliary domain data and maxi- mize it. For stability concern, we also set an upper bound to the MMD (P, Q exp). Then, the overall optimization objective of NTL with distance expansion of representation is shaped as follows:
Las = Lg mmmnm3, a- LA Lais), where Lais = min(8', a' MMD(P -~Pg (0(x)), Pa Pa-pg (o(x)) erp)
Here, a' and 3' represent the scaling factor and upper bound of Lais respectively (a' 0.1 and 3 =²==10 1.0 in our experiments; please refer to the Appendix for more details about a' and 3¹). D(.) is the feature extractor that outputs the corresponding representations of given imputs.
When the target domain is known and accessible, it will be regarded as the auxiliary domain, and the above NTL with distance expansion of representation can be conducted directly the source and auxiliary domains. We call such cases Target-Specifee NTL.
3.2 SOURCE DOMAIN AUGMENTATION FOR SOURCE-ONLY NTL
In practice, the target domain might be unknown or unavailable. For such cases, we develop a novel generative augmentation framework to generate an auxiliary domain and then leverage the above NTL process, in what we call Source-Only NTL. In the following, we will introduce our augmenta- tion framework, which can generate data samples drawn from the neighborhood distribution of the source domain with different distances and directions to serve as the auxiliary data domain in NTL.
GAN Design for Source Domain Augmentation. The overall architecture of our augmentation framework is shaped like a generative adversarial network (GAN) that is made up of a generator g and discriminator D. takes in normal noise and label in form of one-hot, and then outputs data sample. For D, if we feed DD with a sample, it will tell whether this sample is fake or not and predict its label. The adversarial battle happens D. while D distinguishes whether the data being fed to it is real or not to the best of its ability. After sufficient period of such battle, the distributions of the generated data and the ground-truth data are too similar to tell apart (Li et al., 2017). Based on this principle, we utilize G to approximate the source domain. However, if we follow the standard GAN training. the trained GAN will not generate samples with deterministic labels. Therefore, we combine the intuitions of CGAN (Mirza & Osindero, 2014) and infoGAN (Chen et al., 2016) to propose a new training approach for our augmentation framework. In our approach, G uses MSE loss to compare its generated data and the real data. D consists of three modules: a feature extractor and two classifiers behind the extractor as two branches, where a binary classifier predicts whether the data is real or not, and a multiple classifier outputs the label. Note that these two classifiers both rely on the representations extracted by the feature extractor. For training D. use MSE loss to evaluate its ability to distinguish real samples from fake ones, and KL divergence to quantify the performance of predicting labels for the real data. Finally, there is an additional training step for enforcing the GAN generate samples of given labels by optimizing & and D simultaneously. The training losses of Lg, LD and Lg, D are:
tries to generate data as real possible to fool
1
Lo = B BeW D. 88
2 (1)))()). 11I² + [9 w)), 0113 + (P(D.. (P(( (4)) IP(y)
(5)
Lg R yypg (Iioseen, v)). 1[1], Lg,0 = F Fytwoy (PaL (I. (I(noise, y')))
5

---

Published as a conference paper at ICLR 2022
Algorithm/ 1: Generative Adversarial Data Augmentation for Source-Only NTL
Require: Source domain data S = { (ax, y) PX,y Pf); Generator G. discriminator D; List of augmentation distance DIS, the maximum augmentation direction DIR; GAN training epochs eBAN, augmentation training epochs eara; Initialize the auxiliary domain data A =
Output: The auxiliary domain data A = (( y) ~ P P
for i 1 to EGAN do
use (noise, y ~ P3) to optimize G with L.g, use and (moise,, y ~ Pf) to optimize with L>p;
use (noise, y ~ PV) to optimize G, D with Lg,z
for dis in DIS do for dir DIR do for I do
interval d(t) DIR; finction d() acquires the dimension of inputs; freeze D and [(0 dir x intereal]; // freeze D. and the first dir parts of I-th layer in G;
for to eAUG do
use (noise, ~ PPG) and S to optimize G with Laugi
10
AU ((noisc, y ~ P // use G to generate augmentation data;
11
Here, we use subscripts b and m to denote outputs from the binary classifier and the multiple classi- fier of D, respectively. The noise of Eq. (5) is drawn from Gaussian Noise P. (0 1), while y' drawn from the uniform distribution pV with K equally likely possibilities. And y and y' are the one-hot form vectors of y and y', respectively.
Augmentation with Different Distances. To generate the data of different distances to the source domain, we apply Gaussian estimator to measure the MMD between distributions of the source and the generated data from G. However, providing that the MMD distance is optimized to increase with no restriction, the outcome will lose the semantic information, i.e., the essential feature for the main task. In order to preserve such semantic information, we use the CrossEntropy loss to add restriction the optimization objective. With this restriction, we set multiple upper bounds DIS for generating data with different distances (we use DIS to denote a list of multiple dis-s with various lengths). The specific objective is as follows:
Laug =- min dis, MMD(P Z~P (D. (((l)))))>P))) Py-pg (Gioose, y)]); expp)
(6)
+ Byywp (De (G(noi y)). y)
Here, subscript z denotes outputs from the feature extractor. For every dis, we freeze 'D and use Eq. (6) to optimize G. After the optimization, we can generate augmentation data via feeding & with normal noise and labels drawn from pV:
Augmentation with Different Directions. We also investigate how generate data in different di- rections. The optimization of Gaussian MMD follows the direction of gradient, which is the fastest way to approach the objective. In such case, all augmented domains of different distances might follow the same direction, i.e., the direction of gradient. Therefore. in order to augment neighbor- hood domains with different directions, we need to introduce more restrictions to the optimization process. Specifically, for intermediate representations of G, we view each filter (neuron) as corre- sponding to a feature dimension the representation. At the beginning directional augmentation, make multiple copies of the trained GAN in the last step (G and D), and we pick one GAN for each direction. If we want to augment the source in DIR directions, we will divide the overall net- work of G into DIR parts equally. For the augmentation of the first direction, the first part of G will be frozen and not updated during optimization. The second direction will be augmented by freezing the first two parts of the network and conducting the optimization. The third corresponds the first three parts, and so on. Given a certain dis, we can optimize Fq. (6) to augment the source domain DIR directions by freezing G gradually. The detained flow is shown in Algorithm
3.3 APPLICATION OF NTL FOR MODEL INTELLECTUAL PROPERTY PROTECTION
Ownership Verification. The proposed NTL can easily verify the ownership of a learning model by triggering misclassification. To achieve this, we can attach a certain trigger patch which shapes like a shallow mask for images on the source domain data the auxiliary domain data, and then conduct NTL on these two domains. After that, the training model will perform poorly on the data with the patch but have good performance the data without the patch. Such model behavior

---

Published as a conference paper at ICLR 2022
Table 1: The performance difference of digit datasets between Supervised Learning and Target- Specified NTL. The left of ' is the precision (%) in the target when the model is trained on the source dataset with Supervised Learning. The right of " is the precision of the model trained with Target-Specified NTL. The last two columns present the average relative performance drop in the source and target respectively. It shows that NTL can degrade the performance in the target domains while retaining good performance in the source.
Avg Drop (Source)
Avg Drop (Target)
Source/Target
MT
US
SN
MM
SD
MT
98.9=977.9
14.4=14.5
13.3=13.1
57,4= 9.8
35.7= 8.9
1.01%
78.24%
US
84.7= 8.6
99.8=+8.88
26.8 8.6
31,5=> 9.8
37,5 8.8
1.00%
80.17%
SN
52,0= 9.5
69.0++14.2
89,5
34.7=10.2
15.1-11.6
1.23%
78.42%
MM
97.0=>11.7
80.0
17.8+19.1
91.3=
45.5=+
2.30%
79.76%
SD
60.4=>0..7
74.6- 6.8
37.8+ 8.3
15.0=+22.5
97.2-+9..1
1.13%
81.57%
difference be utilized verify the ownership, which is similar to what regular backdoor-based model watermarking approaches do. In contrast, models trained with other methods often present nearly the same performance on the data with or without the patch, as the patch is shallow and light.
Applicability Authorization. When applying NTL to authorize the model applicability, we aim to restrict the model generalization ability to only the authorized domain, where all the data is attached with a dedicated patch, and we need to make sure that the patch design will not impact the semantic information (note that the unforgeability and uniqueness of the patch are not the main consideration of this work, and we will explore it in the future). For simplicity, we use a shallow mask that is similar to that of the aforementioned ownership verification as the authorized patch. We first use our generative adversarial augmentation framework to generate neighborhood data of the original source domain. Then, we regard the source data with the patch attached as the source domain in NTL, and use the union of the original source data, the generated neighborhood data with and without the patch as the auxiliary domain. After the NTL training on these two domains, the learning model performs well only the source domain data with the authorized patch attached, and exhibits low performance in other domains. In this way, we achieve the model applicability authorization.
4 EXPERIMENTAL RESULTS
Our code is implemented in PyTorch (and provided in the Supplementary Materials). All experi- ments are conducted on a server running Ubuntu 18.04 LTS, equipped with NVIDIA TITAN RTX GPU. The datasets and experiment settings used are introduced below.
Digits. MNIST (Deng, 2012) (MT) is the most popular digit dataset. USPS (Hull, 1994) (US) consists of digits that scanned from the envelopes by the U.S. Postal Service. SVHN (Netzer et al., 2011) 1) (SN) contains house number data selected from Google Street View images. MNIST- M (Ganin al., 2016) (MM) is made by combining MNIST with different backgrounds. Finally, SYN-D (Roy et al., 2018) (SD) is a synthetic dataset, which is generated by combining noisy and complex backgrounds. CIFARIO & STL10: Both CIFARIO and STL10 (Coates et al., 2011) are ten-class classification datasets. In order to make these two sets applicable to our problem, we follow the procedure French et al. (2017). VisDA: This dataset (Peng et al., 2017) contains a training set (VisDA-7) and a validation set (VisDA-V) of 12 object categories. For classifying these datasets, we apply VGG-I1 (Simonyan & Zisserman, 2014) for Digits Recognition, VGG-13 (Simonyan & Zisserman, 2014) for CIFARIO & STL10, and ResNet-50 (He et al., 2016) and VGG-19 for VisDA. All networks are initialized as the pre-trained version of ImageNet (Deng et al., 2009). We use 3 seeds (2021, 2022, 2023) to conduct all experiments three times and present the average performance. Network architectures, parameters and error bars can be found in the Appendix.
4.1 TARGET-SPECIFEED NTL.
Effectiveness of NTL in Reducing Target Domain Performance. For digits sets and CIFAR1O & STL.10, we pick all possible domain pairs to carry out experiments. As for VisDA, we regard the training set as the source domain and the validation set as the target. We include the results of standard supervised learning with KL divergence in the source domain and report the performance difference provided by using NTL. Table and Figure show results of digits sets, CIFARIO & STL10, and VisDA. For NTL, observe that the target performance of all pairs is degraded to nearly 10% with little accuracy reduction in the source domain. The largest performance degrada- tion of the target domain, from 97.0% 11.7%, occurs when the source is MM and the target is MT. Comparing NTL with supervised learning, the average relative performance degradation for the tar-
7

---

Published as a conference paper at ICLR 2022
Table 2: The results of Target-Specified NTL on ownership verification and the performance after applying different watermark removal methods. Compared Supervised Learning, models trained with Barget-Specified NTL behave differently depending on whether the data fed to it contains trigger patch. Applying state-of-art watermark removal approaches on models of NTI, does not impact the effectiveness of ownership verification.
Training Methods
Watermark Removal Approaches
Source without
Supervised
NTL
FTAL
RTAL
EWC
AU
Overwriting Pruning
Learning
Patch
Test with/without Patch (%)
[Tsst with/without Patch (%)
MT
99. 5/99.
10.2/98.9
9,9/99.3
10.4/98.6
10.9/99.2 10.8/98.9
11.0/99.0
10.0/87.3
US
99.1/99.2
14.3/99.2
14.3/99.0
9.1/98.6
14.4/98.8
14.2/99.1
13.9/99.0
12.7/88.0
SN
87 87.2/89.3
10.1/89.0
9.9/89.2
10.0/88.7
10.1/89.1
10.0/89.0
9.9/88.8
9.7/56.7
MM
84.8/91.8
12.1/90.5
14.0/91.0
1
12.7/91.1
12.6/91.0
12.5/90.9
10.9/68.2
SD
87. 87.4/96.8
11.6/96.5
12.6/96.7
13.3/95.4
12.4/96.9
12.8/96.5
12.2/96.5
11.0/72.4
CIFARIO
81.7/89.2
14.8/88.9
13.5/89.0
14.9/88.8
15.0/89.4
14.8/89.5
14.3/88.8
12.9/79.2
STL10
85.0/87.0
14.9/86.2
12.4/86.8
12.7/86.9
13.0/87.5
13.9/87.4
13.7/87.2
11.7/76.3
VisDA-T
91.7/92.8
15,4/92.4
15.6/92.7
15.7/92.5
16.2/92.7
16.3/92.6
14.7/92.0
14.5/82.1
STL10 without Patch
STL10 with Patch
Figure 2: The data of STL10 attached without/with the patch.
Different dis
100%
0
/
2 3 4 5
7
9
90%
80%
0
2
3
4 5 6 7
2
9
70%
0
|
0
3 4 5
6
7
8 4
9 2
60%
50%
0
1
2 3
4
1
40%
Different dir
30%
20%
}
|
3 3
5
Z
7
3.
2 3
10%
1
2
3
4 3 1
7.
1
CIFARIO
STLI0
VisDA-T
ViDDA
Supervised Learning: Source
Supervised Learning: Target
0 /
3 4
J
1
8 9
Speciffed NTL: Source
Target Specified NTL: Target
Source-Only NTL: Source
Source-Only NTL: Target
A
7
Figure Performance of CIFARIO, STL10, VisDA source or target domain for Supervised Learning. Target-Specified NTL and Source-Only NTL.
Figure
The augmentation data MNIST generated by the generative adver- sarial augmentation framework.
get domain of all cases is approximately 80%. These results demonstrate that Sprect-Spe NTL can effectively degrade the performance in the target without sacrificing the source performance.
NTL for Ownership Verification. We shown in Figure (please refer to the Appendix for more details). We use state-of-art model watermark removal approaches to test the robustness of NTL-based verification: FTAL (Adi al., 2018), RTAL (Adi et al., 2018), EWC (Chen et al., 2019). AU (Chen et al., 2019), watermark overwriting and model pruning (Rouhani et al., 2018). The settings of these methods included in the Appendix. The results are shown Table 2. We can see that models trained with NTL hehave differently on the data with and without the patch, whereas supervised learning performs nearly the same. Furthermore, all these 6 watermark removal methods fail to improve the performance on the patched data, which indicates that NTLe ownership verification is effective and robust to offte-of-the-an watermark removal methods.
simple pixel-level mask the trigger patch, which
4.2 SOURCE- ONLY NTL.
Effectiveness NTL in Reducing Non-source Domain Performance. For three dataset cases, select one domain the source and then conduct generative adversarial augmentation to
8

---

Published as a conference paper at ICLR 2022
Table 3: The performance difference of digit datasets between Supervised Learning and Source- Only NTL. Non-S means non-source domain. The left of " shows the precision (%) of the model trained on the source dataset with Supervised Learning. The right is the model precision using Source-Only NTL. The last two columns present the average relative precision drop the source and non-source domain. It shows that Source- Cource-Only NTL can effectively degrade the performance in non-source domains without significant sacrifice of source performance.
Avg Drop (Source)
Avg Drop (Non-S)
Source/Non-S
MT
US
SN
MM
SD
MT
98.9
86.4=1388
33.3
13,4=13.4
15.7=11.0
0.00%
72.27%
US
84.7=
99.8~98.9
26.8 6.0
31.5=+0.1
37.5 8.6
0.90%
82.60%
SN
52.0=12.3
69.0>> 8,9
89.5+88.0
34.7=>11.3
15.1=+227
1.68%
78.56%
MM
97.0=
80.0=> 7.8
47.8-
7.8
91.3=
45.5=220.1
2.41 %
81.35%
SD
60.4=>39.2
74.6= 9.5
37.8
35.0=
97.2->9.9
0.31%
61.12%
Table 4: The performance of authorizing applicability of models trained with Source-Only NTL on
digits. NTL-bbsed model authorization can enable the model to perform well only on the authorized domain - the source data attached with the authorized patch.
Source with
Test with Patch(%)
Test without Patch (%)
Authorized Domain
Other Domains
Patch
MT
US
SN
MM
SD
MT
US
SN
MM
SD
MT
98.5
14.2
17.7
15.1
7.7
11.6
14.3
16.2
10.7
9.3
98.5
13.0
US
8.2
98.9
8.7
9.5
10.4
10.3
6.6
8.6
9.2
10.3
98.9
9.1
SN
10.1
9.7
88.9
12.9
11.8
9.7
8.3
9.2
12.3
11.5
88.9
10.6
MM
42.7
8.2
16.1
90.6
32.2
9.5
8.3
7.1
25.6
22.8
90.6
19.2
SD
9.7
6.7
15.7
19.2
95.8
10.2
6.7
9.1
11.5
33.7
95.8
13.6
generate the auxiliary domain. We set a series of discrete dis-s from 0.1 to 0.5 with a step of 0.1, and for each dis, generate augmentation data of 4 directions (DIR=4). Table and Figure present the results of Source-Only NTL and its comparison with the supervised learning. Figure 4 is the augmentation data for MNIST (other datasets are included the Appendix). From the results, we can clearly see that models trained with NTL perform worse on all non-source domains compared with the supervised learning, and MM-MT has the largest degradation from 97.0% to 14.7%.
NTL for Applicability Authorization. Follow the implementation steps outlined in Section 3.3, we carry out experiments on all dataset cases. The experiment results of digits are presented in Table (the results of CIFARIO 10& & STLI0 and VisDA are in the Appendix). From the table, we can see that the model performs very well in the authorized domain while having bad performance domains (with or without the authorized patch). The highest classification accuracy of unauthorized domains is barely 42. 7%, which will discourage users from employing this model. This shows the effectiveness of NTL in applicability authorization.
all other
5 CONCLUSION AND FUTURE WORK
In this paper, we propose -on-Transferable Learning (NTL), a novel training approach that can re- strict the generalization ability of deep learning models to specific data domain while degrading the performance in other domains. With the help of a generative adversarial augmentation framework, NTL is effective both in the presence and absence of target domains. Extensive experiments on digit recognition datasets, CIFARIO & STL and VisDA demonstrate that the ownership of models trained with NTL can be easily verified, and the verification is resistant to state-of-art watermark removal approaches. Moreover, with the training of NTL, model owners can authorize the model applicability certain data domain without worrying about unauthorized usage in other domains.
In future work, it would be interesting to extend NTL to other tasks beyond image classification, e.g., semantic segmentation, object detection/tracking, and natural language processing (NLP) tasks. For tasks where the input data is not images, generating augmentation data would require different methods and could be challenging. Another possible future direction Multi-Tann Task Learning, where we could explore whether it is possible to restrict the model generalization ability to certain tasks, for instance, we think might be useful in some cases to restrict the language model to certain tasks. Moreover, yet another interesting direction could be to combine cryptography with NTL- based verification and authorization.

---

Published as a conference paper at ICLR 2022
ACKNOWLEDGEMENT
We gratefully acknowledge the support by National Science Foundation grants 1834701, 1724341, 2038853, 2016240, Office of Naval Research grant N00014-19-1-249, and research awards from Facebook, Google, PlatON Network, and General Motors.
ETHICS STATEMENT
In this paper, our studies are not related to human subjects, practices to data set releases, discrim- ination/bias/fairnes concerns, and also do not have legal compliance or research integrity issues. Non- Non-Transferable Learning is proposed to address the shortcomings of current learning model in intellectual property protection. However, if the model trainer themselves is malicious, they may utilize NTL for harmful purposes. For example, malicious trainer could use NTL to implant back- door triggers in their model and release the model to the public. addition, recently, there are domain adaptation (DA) works on adapting the domain-shared knowledge within the source model to the target one without access to the source data (Liang et al., 2020; Ahmed et al., 2021; Kundu al., 2020; Wang et al., 2021). However, if the source model is trained with NTL, believe that these DA approaches will be ineffective. In other words, our NTL can be regarded as a type of attack those source-free DA works.
REPRODUCIBILIIYY STATEMENT
The implementation code can be found in https//gittuh..oo/ooniiiii Wang/NTTL. All datasets and code platform (PyTorch) eters and random seeds in the Appendix.
public. In addition, we also provide detailed experiment param-
REFERENCES
Alessandro Achille and Stefano Soatto. Emergence of invariance and disentanglement in deep rep- resentations. The Journal of Machine Learning Research, 19(1):1447-1980, 2018.
Yossi Adi, Carsten Baum, Moustapha Cisse, Benny Pinkas, and Joseph Keshet. Turning your weak- ness into a strength: Watermarking deep neural networks by backdooring. In 27th {USENIX) Security Symposium (1 (!USENIX) Security 18), pp. 1615-1631, 2018.
Sk Miraj Ahmed, Dripta S Raychaudhuri, Sujoy Paul, Samet Oymak, and Amit K Roy-Chowdhury. Unsupervised multi-source domain adaptation without access to source data. arXiv preprint arXiv:2104.01845, 2021.
Manaar Alam, Sayandeep Saha, Debdeep Mukhopadhyay, and Sandip Kundu. Deep-lock: Secure authorization for deep neural networks. arXiv preprint arXiv:2008. 00900. 2020.
Gilles Blanchard, Gyemin Lee, and Clayton Scott. Generalizing from several related classification tasks to a new unlabeled sample. Advances in neural information processing systems, 24:2178- 21 2186, 2011.
Jean-Pierre Briot, Gaëtan Hadjeres, and François Pachet. Deep learning techniques for music gen- eration. Springer, 2020.
Abhishek Chakraborty, Ankit Mondai, and Ankur Srivastava. Hardware-assisted intellectual prop- erty protection of deep learning models. In 2020 57th ACM/IEEE Design Automation Conference (DAC), pp. 1-6. IEEE, 2020.
Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Infogan: interpretable representation learning by information maximizing generative adversarial nets. In Neural Information Processing Systems (NIPS), 2016.
Xinyun Chen, Wenxiao Wang, Chris Bender, Yiming Ding, Ruoxi Jia, Bo Li, and Dawn Song. Refit: a unified watermark removal framework for deep learning systems with limited data. arXiv preprint /91 1.07205, 2019.
Yushi Cheng, Xiaoyu Ji, Lixu Wang, Qi Pang, Yi-Chao Chen, and Wenyuan Xu. (mID): Tracing screen photos via (Moiré) patterns. In 30th USENIX Security Symposium (USENIX Security 21), pp. 2969-299922 2021.
10

---

Published as a conference paper at ICLR 2022
Adam Coates, Andrew Ng. and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelli- gence and statistics, pp. 215-223. JMLR Workshop and Conference Proceedings, 2011.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp.
Ieee, 2009.
Li Deng. The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE Signal Processing Magazine, 29(6): 141-142, 2012.
Jiahua Dong Yang Cong, Gan Sun, Bineng Zhong, and Xiaowei Xu. What can be transferred: Unsupervised domain adaptation for endoscopic lesions segmentation. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 0023-4032, 2020.
Jiahua Dong, Yang Cong, Gan Sun, Zhen Fang, and Zhengming Ding. Where and how transfer: knowledge pngregatiioninduced transferability perception for unsupervised domain adaptation. IEEE Transactions on Puttern Analysis and Machine Intelligence, 2021.
Yingjun Du, Jun Xu, Huan Xiong, Qiang Qiu, Xiantong Zhen, Cees GM Snoek, and Ling Shao. Learning to learn with variational information bottleneck for domain generalization. In European Conference on Computer Vision, pp. 200-216. Springer, 2020.
Lixin Fan, Kam Woh Ng, and Chee Seng Chan. Rethinking deep neural network ownership verifi- cation: Embedding passports to defeat ambiguity attacks. 2019.
Geoffrey French, Michal Mackiewicz, and Mark Fisher. Self-ensembling for domain adaptation. arXiv preprini arXiv: / 7177700605200 2017.
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain-ad adversaraaa training of neural net- works. The journal of machine learning research, 17(1):2096-2033 2016.
Song Han, Jeff Pool, John Tran, and William J Dally. Learning both weights and connections for efficient neural networks. arXiv preprint arXiv: 1 00..00062666 2015.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference computer vision and pailern recognition, pp- 770-778, 2016.
Zecheng He, Tianwei Zhang, and Ruby B Lee. Model inversion attacks against collaborative in- ference. In Proceedings of the 35th Annual Computer Security Applications Conference, pp. 148-162, 2019.
Jonathan J. Hull. database for handwritten text recognition research. IEEE Transactions on pattern analysis and machine intelligence, 16(5):550-554, 1994.
Stefanie Jegelka, Arthur Gretton, Bernhard Schölkopf. Bharath K Sriperumbudur, and Ulrike Von Luxburg. Generalized clustering via kernel embeddings. In Annual Conference on artifi- cial intelligence, pp. 152. Springer, 2009.
Ronald Kemker, Marc McClure, Angelina Abitino, Tyler Hayes, and Christopher Kanan. Measuring catastrophic forgetting in neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
Jogendra Nath Kundu, Naveen Venkat, R Venkatesh Babu, et al. Universal source-free domain adap- tation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4544-4553, 2020.
Erwan Le Merrer, Patrick Perez, and Gilles Trédan. Adversarial frontier stitching for remote neural network watermarking. Neural Computing and Applications, 32((1):9233-99) 9244, 2020.
Chun-Liang Li, Wei-Cheng Chang, Yu Cheng, Yiming Yang, and Barnabás Póczos. Mmd gan: towards deeper understanding of moment matching network. In Proceedings of the 31st Interna- fional Conference on Neural Information P Processing Systems, pp. 2200-2210, 2017.

---

Published as a conference paper at ICLR 2022
Guofa Li, Yifan Yang, Xingda Qu, Dongpu Cao, and Keqiang Li. A deep learning based image enhancement approach for autonomous driving at night. Knowledge-Bond Based Systems, 213:106617, 2021a.
Haoliang Li, YuFei Wang. Renjie Wan, Shiqi Wang, Tie-Qiang Li, and Alex Kot. Domain gener- alization for medical imaging classification with mnear-dependency regularization. arXiv preprint arXiv:2009. 12829, 2020.
Lei Li, Ke Gao, Juan Cao, Ziyao Huang, Yepeng Weng, Xiaoyue Mi, Zhengze Yu, Xiaoya Li, et al. Progressive domain expansion network for single domain generalization. arXiv preprint arXiv:21055 103.00000, 2021b.
Zheng Li, Chengyu Hu, Yang Zhang, and Shanqing Guo. How to prove your model belongs to you: blitr-watermark based framework protect intellectual property of dnn. In Proceedings of the 35th Annual Computer Security Applications Conference, pp. 126-137, 2019.
Jian Liang, Dapeng Hu, and Jiashi Feng- Do really need access the source data? source hypothesis transfer for unsupervised domain adaptation. In International Conference on Machine Learning, pp. 6028-6039, PMLR, 2020.
Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. Learning transferable features with deep adaptation networks. In International conference on machine learning, pp. 97-105, PMLR, 2015.
Mchdi Mirza and Simon Osindero. Conditional generative adversarial nets. arXiv: 1411. 1784, 2014.
arXiv preprint
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Xingchao Peng, Ben Usman, Neela Kaushik, Judy Hoffman, Dequan Wang, and Kate Saenko. Visda: The visual domain adaptation challenge, 2017.
Vihari Piratla, Praneeth Netrapalli, and Sunita Sarawagi. Efficient domain generalization via common-specifie low-rank decomposition. In International Conference on Machine Learning, pp. 7728-7738. PMLR, 2020.
Fengchun Qiao, Long Zhao, and Xi Peng. Learning to learn single domain generalization. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12556- 12565, 2020.
Mohammad Mahfujur Rahman, Clinton Fookes, Mahsa Baktashmotlagh, and Sridha Sridharan. Correlation-aware adversarial domain adaptation and generalization. Puttern Recognition, 100: 107124, 2020.
Mauro Ribeiro, Katarina Grolinger, and Miriam AM Capretz. Mlaas: In 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA), pp. 896-909 IEEE, 2015.
Machine learning as a service.
Bita Darvish Rouhani, Huili Chen, and Farinaz Koushanfar. Deepsigns: A generic watermarking framework for ip protection of deep learning models. arXiv preprint arXiv: 180400000 2018.
Prasun Roy, Subhankar Ghosh, Saumik Bhattacharya, and Umapada Pal. Effects of degradations on deep neural network architectures. arXiv preprint arXiv:1807.10108, 2018.
Ahmed Salem, Apratim Bhattacharya, Michael Backes, Mario Fritz, and Yang Zhang, Updates- leak: Data set inference and reconstruction attacks in online learning. In 29th {USENIX) Security Symposium ((USENIX) Security 20), pp. 1291-1308, 2020.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference at- tacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy (SP), pp. 3-18. IEBE, 2017.
12

---

Published as a conference paper at ICLR 2022
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.15580 2014.
Casper Kaae Senderby, Jose Caballero, Lucas Theis, Wenzhe Shi, and Ferenc Huszár. Amortised map inference for image super-resolution. arXiv preprint arXiv:1610.04490 2016.
Congzheng Song, Thomas Ristenpart, and Vitaly Shmatikov. Machine learning models that remem- ber too much. In Proceedings of the 2017 ACM SIGSAC Conference on computer and communi- cations security, pp. 587-601, 2017.
Bharath K Sriperumbudur, Kenji Fukumizu, Arthur Gretton, Gert RG Lanckriet, and Bernhard Schölkopf. Kernel choice and classifiability for rkhs embeddings of probability distributions. In NIPS, volume 22, pp. 1750-1758, 2009.
Octavian Suciu, Radu Marginean, Yigitcan Kaya, Hal Daume III, and Tudor Dumitras. When does machine learning {FAIL.}? generalized transferability for evasion and poisoning attacks. 27th (USENIX) Security Symposium (( (UUSENIX) Security 18), pp. 1299-1316, 2018.
Naftali Tishby, Fernando C Pereira, and William Bialek. The information bottleneck method. arXiv preprint physics/00044057, 2000.
Kari Torkkola. Feature extraction by non-parametric mutual information maximization. Journal of machine learning research, 3(Mar): 141-11434 2003.
Yusuke Uchida, Yuki Nagai, Shigeyuki Sakazawa, and Shin'ichi Satoh. Embedding watermarks into deep neural networks. In Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval, pp. 269-277, 2017.
Lixu Wang, Shichao Xu, Xiao Wang, and Qi Zhu. Eavesdrop the composition proportion of training labels in federated learning. arXiv preprint arXiv:1910. 060444, 2019.
Lixu Wang, Songtao Liang, and Feng Gao. Providing domain specific model via universal no data exchange domain adaptation. In Journal of Physics: Conference Series, volume 1827, pp. 012154. IOP Publishing. 2021.
Shichao Xu, Yixuan Wang, Yanzhi Wang. Zheng O' O'Neill, and Qi Zhu. One for many: Transfer learning for building hvac control. In Proceedings of the 7th ACM International Conference on Systems for Energy-Efficient Efficient Buildings, Cities, and Transportation, pp. 230-239, 2020a.
Shichao Xu, Lixu Wang, Yixuan Wang, and Qi Zhu. Weak adaptation learning: Addressing cross- domain data insufficiency with weak annotator. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8917-8926, 2021.
Zhenlin Xu, Deyi Liu, Junlin Yang, and Mare Niethammer. Robust and generalizable visual repre- sentation learning via random convolutions. arXiv preprint 20X0:22007.33003, 2020b.
Yuanshun Yao, Huiying Li, Haitao Zheng, and Ben Y Zhao. Latent backdoor attacks on deep neural networks. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communica- tions Security, pp. 2041-2055, 2019.
Jialong Zhang, Zhongshu Gu, Jiyong Jang, Hui Wu, Marc Ph Stoecklin, Heqing Huang, and Ian Molloy. Protecting intellectual property of deep neural networks with watermarking. In Proceed- ings of the 2018 on Asia Conference on Computer and Communications Security, pp. 159-172, 2018.
Xiang Zhang, Xiaocong Chen. Lina Yao, Chang Ge, and Manqing Dong. Deep neural network hyperparameter optimization with orthogonal array tuning. In International Conference Neural Information Processing, pp. 287-295. Springer, 2019.
Jingjing Zhao, Qingyue Hu, Gaoyang Liu, Xiaoqiang Ma, Fei Chen, and Mohammad Mehedi Has- san. Afa: Adversarial fingerprinting authentication for deep neural networks. Computer Commu- nications, 150:488-499 2020a.
13

---

Published as a conference paper at ICLR 2022
Long Zhao, Ting Liu, Xi Peng. and Dimitris Metaxas. Maximum-entree entrrpp adversarial data augmen- tation for improved generalization and robustness. arXiv preprint arXiv:2010.08001, 2020b.
Shanshan Zhao, Mingming Gong, Tongliang Liu, Huan Fu, and Dacheng Tao. Domain generaliza- tion via entropy regularization. Advances in Neural Information Processing Systems, 33, 2020c.
Kaiyang Zhou, Yongxin Yang. Timothy Hospedales, and Tao Xiang. Learning to generate novel domains for domain generalization. In European Conference on Computer Vision, pp. 561-578. Springer, 2020.
Barret Zoph and Quoe V Le. Neural architecture search with reinforcement learning. arXiv preprint 1..0576 2016.
14

---

Published as a conference paper at ICLR 2022
SUMMARY OF THE APPENDIX
This appendix contains additional details for the ICLR 2022 article "Non-Transferae" Learning: A New Approack for Model Ownership Verification and Applicability Authorization" ", including math- ematical proofs, experimental details and additional results. The appendix is organized as follows:
Section A introduces the theoretical proofs for Proposition 1, Theorem and Theorem
Section B provides additional implementation settings, including the network architectures (Sec- tion B.1) and hyper parameters (Section B.2).
Section C provides additional experimental results, including the augmentation data of other datasets (Section C.I), the model authorization results CIFARIO & STLI0 and VisDA (Sec- tion C.2), the experiments of VisDA on VGG-19 (Section C.3), and the error bars of main experi- ment results (Section C.4). Section C.5 provides the experiment result of different kernel widths.
In Section D. we discuss possible attacks that can be constructed based on our proposed method. Note that NTL used in this appendix the abbreviation of Non- Non-Tranfferable Learning.
A THEORY PROOFS
A.1 PROOF
Proposition 1. Let n be nuisance for input x. Let z be a representation of x, and the label is y. For the information flow in the representation learning, we have
T(2;2) T(2: y/m) T(2;
Proof: According Proposition 3.1 in (Achille & Soatto, 2018), there is Markov Chain: (y, n)
a -> 2. This chain describes the information flow starting from the ground truth knowledge (label y and nuisance n) of input æ to extracted representation z. In this case, the information flows from (y, n) to æ then to z. The Data Processing Inequality (DPI) for a Markov Chain can ensure the relation that [(2; x) 2 I(z: y. n). And with the chain rule, we have [ (z: = I(2; + [(2; n)). Thus, we can obtain x2;x) 2 I(z;n) + y)).
Theorem 1. Let ÿ be the predicted label outputted by a representation model when feeding with imput x, and suppose that ÿ is a scalar random variable and æ is balanced on the ground truth label y. Denote the one-hot forms of y and y as ý and y. respectively. If the KL divergence loss (KL(P (P()))((()) increases, the mutual information I(z: will decrease.
Proof. Suppose that the information flow in classifier Q of a representation model follow a Markov chain z + y y, in this case, let's apply Data Processing Inequality, we have
(8) Because the input data of this representation model is balanced across classes, we suppose that both y and ÿ are drawn from the uniform distribution pf with K equally likely possibilities. Moreover, though the distribution of ÿ might change a little during training, we can assume it won't 't become very biased since the balance of input data. For both ý and y, they are vectors with K dimensions. In PyTorch implementation, the computation of KI. divergence loss regards there is a scalar random variable corresponding to each vector and every dimension within the vector as an observation of this variable. In this case, the loss between ý and forms like DRI. (P())))))((())) = E y; log &s y. is easy to obtain that DKL. (PL (P(}))(((()) is non-negative, and if and only y y, the KL divergence loss hits the minimum value (P))) that equal to the dimension index with the maximum value of y and y, respectively. Therefore, it's easy to conclude that the probability of ý = y will decrease with the increase of D (PL()))((()))) i..,, (P (P)))) 1 + P(ù. y) J. For Eq. (8), we can derive it deeper
g(z;y)= - (():)) = Ey-py (P (P))))
= O. While ý and y are scalar random variables
P(ù. y) log P(g) P(y) P(O)
P(ú. y)
T(ú: v) Byywp [Tx (P IP
EP(w) Σ D
Here, both P()) and Pvy) are uniform distributions PV, and have assumed P(ÿ) won't It become very biased at the beginning of this proof. As result, we regard P(ù) and P(y) nearly unchanged after training. addition, P(ú. y) decreases with the increase of Dxr (P(((P())l (P()))[((y). Furthermore, we can easily calculate that al(ú:v)
(P())))))) < O. In this case, 1(9: y) decreases with the increase of (P()) (P)))) and the same as ((2; y) since T(r: 2) is the upper bound.
15

---

Published as a conference paper at ICLR 2022
Theorem 2. Let be a nuisance that is regarded as a domain index. n= and n= denote that a certain input x comes from two different domains. Suppose that these two domains have the same number of samples d, and the samples of each domain centroid. Let z be a representation of x. and it is drawn from distribution Pz. An estimator with the characteristic kernel from Reproducing Kernel Hilbert Spaces (RKHSs)- Gaussian Kernel estimator (M Q; exp) is applied on finite samples from distributions P Pzto and Pz to approximate the Maximum Mean Discrepancy (MMD) between these two distributions. If MMD(P2 Pzz exp) increases saluration, the mutual information belween z and will increase.
symmetrically distributed around the
MMD(Pz)0; Pzii ezp) =Ez,z' - 11=-===111 122 P210:' // PP11 --- 11==211
++
l| 1==='11
(10)
Proof. According to the definition of Shannon Mutual Information, we have
P(z|n)
[=;; n)
Pn (PPP) = Pn BewP(e))) log
(11)
P(z)
And because two domains have the same number of samples, we can have n conforms P(n)
(P(0)= = 0.5,, P(1) 005),, Eq. (11) can re-written as
[(:; n) = 0. 0.5E log P(z n 0) + 0.5%% P(Zln=1) log P(z)
P(=|n= =
(12)
P(z)
Next, we denote the probability density function (PDF) of Pzjo and Pzi as p(z) and g(z). re- spectively. Moreover, according the law of total probability, the PDF of distribution Pz is PDF(P2) = 0.5p(z) + 0.5g(z), then we have
[(z;n) 0.5 [- p(=) log 2p(z) de + 0.5 p(z) (())
/+ +oc
29(z)
((2) log p(z) ((2)
de
(13)
Subsequently, denote expectations and variances of Pzjo and Pzi (Ho, σa) and (M1 01), respectively. With the assumption that samples of each domain are symmetrically distributed about the centroid, we have Pm. max (px))
q(n)). Thus if we use two variables f and g to denote PDF-s of Pzio and PzI1, i.e., f = p(z) ) and g = g(z). in this case, we have f E (0, Pm] and g E (0, gm). Based on the above analysis and notations, we can split Eq. (13) into 4 terms follows,
p(uo) and 4m max max (q(2))
Pm
2f
Pm
2 df f F-log
[(;;n) = 0.5
f. Folog
df +
0.5
0
[+g g
Jn
J g
(14)
4m
2g
4m
2g
+ 0.5
g log
dg + 0.5
9 log
f + dg
f+g g
here the superscript indicates the right side of and g. Next, let us consider the Gaussian Kernll estimator MMD(Pzjo. Pzni exp). The estimator consists of terms, and we can easily conclude
that e | decreases with the increase of ||= 21112, Note that 'MMD(P Pz exp) in- creases to saturation' means at least one term increases while the other two terms remain unchanged or increased. For next proof, we need to mention Theorem in (Sriperumbudur et al., 2009).
Theorem 2 (Sriperumbudur et al., 2009). Suppose {(4,, Y X; E M,Y, E (-1,+1}, Vi is a training sample drawn i..dd from μ. Assuming the training sample is separable, let ssom be the solution to the program, infll YYJ(X,) 2 1, Vi), where H is an RKHS with measurable and bounded kernel k. If is characteristic, then
1
(P, Q) 2
(15)
[[ 1|
ox.- d is the sample quantity and represents the Dirac measure. This theorem provides a bound on the margin of hard-margin SVM in terms of MMD. Eq. (15) shows that a smaller MMD between P and Q enforces a smaller margin (i.e., a less smooth classifier, from, where smoothness is measured as !| Besides, the Gaussian kernel a measurable and bounded kernel function in an RKHS. According to this theorem and the nature of hard-margin SVM, we can easily obtained that variances on, aı of P P2jo and P Pz are decreasing
where := -x 5x,
16

---

Published as a conference paper at ICLR 2022
and the difference between expectations My and μl is increasing with the saturated increase of MMD, and this conclusion can be found in (Jegelka et al., 2009).
In the following, will prove that (z; n2) will increase when the difference between μo and Ai increases. Due to the symmetry of PDFs, both f and g increase at the left side their own expectation and decrease the right side. Without the loss of generality, at the left side of 141. For the first term of Eq. (14) which corresponds to the left interval f. the value of is smaller than that of the case before increasing the difference between μg and μu Thus this term will increase when the difference between and l4i increase. As for the right interval of ff the maximum value of f+ g' (in the neighborhood of f1) comes later than that of case before increasing the difference, besides, the maximum value is also smaller. There, for the second term of Eq. (14), will also increase with the increase of difference between μo and 141- Similarly, the integration of follows the same trend with that of f. In this case, T(z; nz) will increase when the difference between Hi and 14 increases.
assume that μl locates
140
11
Mutual Information (NAT)
p=
4-
Pиj
25
30
f.g
g'
Training
Figure 5: PDF-s of distribution Pzju and Pzl1. The blue curve the PDF of Pzip. and the green one is the PDF of Pzi The orange curve is the PDF of Pz1o with smaller variance (best view in color).
Figure 6: The change of mutual information [2; n at every training round, and it is ob- served by the domain discriminator.
Next, we will prove that [(2; n) will increase the variance of either Pz or Pzi decreases. Without the loss of generality, we assume the variance of Pzio decreases while the variance of Pzii remain unchanged. For the PDF of distribution, if the variance decreases, the maximum value of PDF will increase, and there will be two points of intersection with the same value between the PDF-s. Such conclusions are easily proved since the integration of a PDF is always 1. We denote the new maximum value of f as p'm. and the value of two points of intersection p=. In addition, during the saturated increase of MMD, we can always find a pair of Ho and μu that enables the left side of intersect with the right side of With the notations in Figure 5. We can change Eq. (14) into
P--
2f
P'
2
P'm
2 df -llog
Terms f
log
df
+
f. log
df
Jo
f f
f +
f + g'
--
Pm
(16)
pm f log
2
for f log
pp-
2f
Pus f log
2 df
df +
df +
J + g
f + 1
g
2g dg +
22
2g dg
Terms of
g. log
g log
f f+ g
f g
4
(17)
mm
2g dg +
mm
2g
g log
J
log I' +
dg
42
++g
According to the decrease of op. we can conclude: the 1st, 4th, 6th terms of Eq. (16) and the 2nd term of Eq. (17) decrease, while the function, R({.g) f log J-5. and
of Eq. (16) and Eq. (17) increase. For next get its first-order derivative 0 when f > g. According these
proof, denote
2f + - OR log J+0
We can easily obtain that ØR of
17

---

Published as a conference paper at ICLR 2022
analysis, get that the decrease of the 1st term of Eq. (16) can be offset by the added increase of the 5th term of Eq. (16) and the 3rd term of Eq. (17); the decrease of the 4th term Eq. (16) can be offset by the 2nd term of Eq. (16); the decrease of the 6th term of Eq. (16) can be offset by the 4th term of Eq. (17); the decrease of the 2nd term of Eq. (17) can be offset by the 3rd term of Eq. (16). Moreover, such offsets are overcompensation. In this case, if the variance of either P or Pzj1 decreases.
prove that (2; will increase
Considering the combination of the above cases, if the difference between expectations increases and variances of two distributions decrease, the mutual information will increase.
A.2 OBSERVE THE MUTUAL INFORMATION
We follow the similar process of (Achille & Soatto, 2018) to observe the change of mutual in- formation [ ((zn)). To be specific, let è be binary classifier that given representation and nuisance n tries to predict whether z is sampled from the distribution of one domain Pzio or another domain Pz According to (Sønderby et al., 2016), if we train O with the loss of Ez (log (z))) + EgwP2, (log 1 - (z))), there is always a Bayes-optimal O*,
P(z 0)
θ-
(18)
With (12() if we assume the O, I trained with Ez (log (n(z))+ (log 0o(2) and Ez (log (z)) + Ez~P2 (log (z))), respectively, are close to the optimal ones 0g Oj we have
log P(z n 0) + 0. P(s|==1)) log P(z|n= =
T(2;n) = P( log 20o(z) -0.5z log 10)
P(z)
P(z)
(19)
With this approximation, we train On and θ for the model of every NTL training round, and we get the curve of (2; shown in Figure 6 (MNIST), According the figure, [(z; n.) is increasing during the overall training process, which is consistent with our intention.
B IMPLEMENTATION SETTINGS
B. NETWORK ARCHITECTURE
To build the classification models, we use several popular architectures as the bottom feature extrac- tor and attach them with fully-connected layers as the top classifier, which are shown in Table 5. Specifically, the backbone network of digits is VGG-11, that of CIFARIO & STLIO is VGG-13, and we use both ResNet-50 and VGG-19 for VisDA. The classifiers of all models are the same, i.e., linear layers with ReLU and dropout. As for the GAN in the augmentation framework, the generator is made up of 4 Conv ConyCoonnnpaaaa Transpose blocks and 2 Residual blocks, and the discriminator D consists of feature extractor with 4 convolution layers, binary classifier and multi-class classifier. These two classifiers are composed of sequential fully-connected layers and share the same representations extracted from the front extractor. The detailed architecture is shown in Table 6 and 7.
The architecture of classification models. 'img" is the dimension of representations ex- Table 5:
tracted from the feature extractor.
Linear(256, K).
Classifier
Linear(256, 256), ReLU, Dropout;
15near(512*mgg*a 256), ReLU, Dropout:
Backbone Network
Feature
JGG-1I/VGG-13/VGG-1110 [10:].
Extractor
Backbone Network VCG-IINVGG-13VVG--19Ree.
Feature
Extractor
[0:10].
18

---

Published as a conference paper at ICLR 2022
Table 6: input label.
The architecture of the generator g. 'dim' is the dimension sum of the latent space and
Out
Tanh()
BatchNorm(3), ReLU. ConvTraaspose2dd123., 3, 4, 2, 1);
Conv4
BatchNorm 128), ReLU, Conv Con/Transpose22(256 128, 4, 2, I);
Conv3
ResBlocks
ResidualBlock((55)
BatchNormm(56), ReLU. ConyTransposee26511, 256, 4, 2, I); BatchNorm(5112, ReLU. Con/Transpose2d( 1024, 512, 4, 2, 1); Line 1024).
Conv2
Conv1
Input
Table 7: The architecture of the discriminator D. Binary Classifier
Multiple Classifier Linear( 128, K).
Classifiers
Linear( 128, 1). Linear(256, 128), ReLU, Dropout; Linear(256, 128), ReLU, Dropout;
Linear(512, 256), ReLU, Dropout;
Linear(512, 256), ReLU, Dropout;
Co55 512, 3, 2. 1), LeakyReLUU, Dropout.
Conv2d(128 256, 3. 1), LeakyReLU, Dropout;
Feature Extractor
Conv2d(64, 128, 2, 1). LeakyReLU, Dropout;
Conv2d(3, 64, 3, 1), LeakyReLU, Dropout;
B.2 HYPER PARAMETERS
Scaling factors and upper bounds. As introduced in Section 3.1 of the main paper, there are two scaling factors (a, a²) that control the trade-off between the maximization of [(z; n) and the sufficiency property of the source domain. Here, we conduct experiments using different values (a = 0.01, 0.05, O. 1.10, 0.20, 0.50 and a' = 0.01, 0.05, 0.10, 0.20, 0.50). and evaluate their impact to the performance of NTL. For Furget-Speciffes NTL, select the combination of MUIST-USPSS, STL.IO>CIFAR 10 and VisDA-T-+Visll VisDA-V. For Source-Only NTI, we choose MNIST-NNn.. STLIO-NNon-S and VisDA-T- are presented in Tables 8 and 9. It is easy to conclude that NTL can work effectively with different scaling factors. As for the upper bounds (3, 3'), we them for the sake of preventing the auxil- iary domain loss and the MMD distance from dominating the optimization objective, affecting the convergence of training.
as the representatives carry experiments. The results
Table 8: The experiments on different values of the scaling factor a.
Scaling Factor
Source/Targer Targett %)
Cases/Values
0.01
0.05
0.10
0.20
0.50
MT/US
98. 98.6/14.3
98.2/14.2
97.9/14.5
97.7/14.2
97.6/14.3
STLIO/CIFARI0
88.0/11.1
87.6/12.2
85.4/09.4
82.9/08.7
83.0/12.1
ViSDA-T/ViSDA-V
93.8/08.8
94.9/08.9
94.4/11.3
93.5/08.7
93.2/09.8
MT//Non-S
98.9/19.0
98.8/13.4
98.9/14.7
98.2/13.4
98.5/13.3
STL10/Non-S 87.9/13.0
83.5/12.0
84.9/10.2
79.6/11.1
86.9/12.8
VisDA-T/Non-
95.6/28.4
95.3/14.4
95.7/14.6
94.5/21.8
94.6/18.5
Training parameters. For the optimization NTL, we utilize Adam as the optimizer, with learning 0.0001 and batch size of 32. For all datasets, we randomly select 8,000 samples from their own training sets as the source data, and 1,000 samples from their own testing sets as the test data (if a dataset does not have test set, we select its test data from the training set without overlapping with the chosen 8,000 source samples). And the sample quantities of the source and auxiliary domain are always the same. In the training of adversarial augmentation, the optimizer is also Adam, and set the learning rate to y = 0.0002 with two decay momentums 0.5 and 0.999. The batch size is 64, and the dimension of the latent space fed to the generator is 256.
19

---

Published as a conference paper at ICLR 2022
Table 9: The experiments on different values of the scaling factor a'.
Scaling Factor a'
Source/Tarrget %))
Cases/Values
0,01
0,05
0,10
0.20
0,50
MT/US
99.3/14.2
98.8/14.3
97.9/14.5
99.2/14.2
99.3/14.2
STLIO/CIFAR10
87.2/08.8
82.0/10.3
85.4/09.4
85.8/10.8
85.4/11.4
VisDA-T/ViDD-VV
96.6/11.0
95.3/09.0
94.4/11.3
92.5/08.9
96,4/10.3
MT/Non-S
98.9/09.4
98.7/13.5
98.9/14.7
99.1/14.3
99.3/15.3
STLIO/Non-S
87.4/10.8
79.8/10.3
84,9/10.2
83.3/13.2
78.3/12.1
VisDA-T/Non-S
96.5/32.5
95.6/20.1
95.7/14.6
94.9/20.7
95.5/19.4
B.3 TRIGGERING AND AUTHORIZATION PATCH
As mentioned in Section 4. I and 4.2 of our main paper, we attach a patch on the data to utilize NTL for ownership verification and usage authorization. We create the patch in a simple way. Specifically, for the pixel of i-th row and j-th column in RGB image, if either i or j is even, then value of v is added to the R channel of this pixel (the channel value cannot exceed 255). Intuitively, the patch is dependent on pixel values of each image. Thus the changes of feature space brought by the attachment of these patches for various images are not the same. In our experiments, if the content of the image is simple, e.g., MNIST, USPS and SVHN, the with small value can shift the feature space sufficiently, but for more complicated images, we have to increase v to enable source images attached with and without the patch differentiable. Specifically, we pick the value as follows: MNIST, USPS, SVHN (v 20): MNIST-M, SYN-D, II,AR10, STL10 (v = 80); VisDA (u = 100). As mentioned the main paper, we will explore the unforgeability and uniqueness of patch generation in the further work.
B.4 IMPLEMENTATION OF WATERMARK REMOVAL APPROACHES
In the Section 4.1 1 of the main paper, we implement 6 model watermark removal approaches to verify the effectiveness of NTL-bbasd ownership verification. Here, we introduce how to implement these approaches. FTAL (Adi et al., 2018) is an approach that fine-tunes the entire watermarked model using the original training data. To implement it, we use 30% of training set that has been learned by NTL to fine-tune the entire model. When using RTAL (Adi et al., 2018), the top classifier is randomly initialized before fine-tuning. model trained with NTL and randomly initialize classifier to attach on the extractor, and then use 30% of the training set to fine-tune this combined model. As for EWC (Chen et al., 2019), we use the code of (Chen et al., 2019) to compute the fisher information of network parameters and adjust the learning rate of fine-tuning. The data used by EWC is also 30% of the training set. Finally, AU (Chen et al., 2019) utilizes the watermarked model to pseudo label additional unlabeled samples from other similar domains, and these samples will be used to tin-ttue the model together with the original training set. Following this principle, we use 30% of the training and the same quantity of unlabeled samples from other domains (the proportion ratio hetween these two parties is 1:1) to fine-tune the model trained with our NTL. We conduct all fine-tuning methods for 200 epochs. For the watermark overwriting, we overwrites a new backdoor-based watermark (Zhang et al., 2018) on the model trained with NTL. Specifically, we attach white corner (3 x 3) as the backdoor trigger to 1/15 of the training set, and follow the training approach of (Zhang et al., 2018) to write the watermark on the model. In addition, similar to other watermarking works (Rouhani et al., 2018), we also test if NTLebased verification is resistant to model pruning, and apply layer-wise pruning method (Han et al., 2015) to prune 70% parameters of the model trained with NTL.
our experiments, we load the feature extractor of the
Table 10: The results (%) of authorizing model usage on CIFARIO & STL1O and VisDA.
Test
CIFARTÓ
STL10
Source
with Patch
without Patch
with Patch
without Patch
CIFARIO
85.9 (1:0.7))
11129)
42.3 (1E1.5)
13.5 (#2.01)
STL10
20.7 (+1.01)
11.7 (+0.98)
85.0 (+1.36)
12.1 (+1.75)
Test
VisDA-T
VisDA-V
Source
with Patch
without Patch 20.9 (+1.04)
with Patch 22.5 (#1.04)
without Patch 17.3 (+1.9))
VisDA-T
93.2 (11.12)
20

---

Published as a conference paper at ICLR 2022
Table 11: The experiment results (%) of VisDA VGG-19.
Test
VisDA-T
VisDA-V
Cases
with Patch
without Patch
with Patch
without Patch
Supervised Learning
93.4 (31.83)
45.2 (#2.244
Target-Specified NTL
92.8 (=:1.04)
08.9 (1:2.01)
Source-Only NTL
92.4 (#1.17)
11.3 (=1.00)
Ownership Verification
10.1 (+0090)
93.0 (=0998)
Model Authorization
92.9 (#1.31)
12.7 (#2.22)
21.4 (11.01)
11.0 (#0.79)
Table 12: The enror range (%) of experiment results for Supervised Learning and NTL respectively (the left of '/' is Supervised Learning, and the right is Target-Specified NTL).
Source/Target
MT
US
SN
MM
SD
MT
0.32
+1.74/+1.61
+1.89/+1.25
+1.97/+1.00 +..5
US
1..27/+11.73
10.10/10.14
11.21/+1.000
11.64/+1.72
+1.17/+11.23
SN
#1.23/+1.00
11.67/
10.97/10.94
10.86/10.88
12.17/+1.77
MM
+1.28/+0.79
#1.78/+1.26
#3.66/+1.08
+1.78/+0.97
+2.10/+1..0
SD
10.91/11.79
#1.61/#1004
#1.27/#1.07
#1.19/+1.88
+0.97/+0.61
C
ADDITIONAL EXPERIMENTAL RESULTS
C.I AUGMENTATION DATA OF OTHER DATASETS
In the main paper, we present the augmentation data MNIST, and in this section, we include the augmentation data of other datasets as follows: Figure for USPS, Figure 8 for SVHN, Figure 9 for MNIST-M, Figure 10 for SYN-D, Figure 11 for CIFAR1O, Figure 12, Figure 13 for VisDA-T.
C.2 MODEL USAGE AUTHORIZATION ON CIFARIO & STL1O AND VISDA
Here we present the experiment authorizing the model usage on CIFARIO 10 STL10 and VisDA, shown in Table 10. According to the results, the model performs well on the data attached with the authorized patch and has bad performance on all other samples.
C.3 ADDITIONAL RESULTS OF VISDA ON VGG-19
To demonstrate the effectiveness of NTL different network architectures, we also carry ex- periments of VisDA on VGG-19. All other settings are the same as before, and the results are shown Table 11. We can easily see that the performance consistent with the aforementioned other experiments, which shows the wide applicability of NTL.
C.4 ERROR BAR
We conduct all experiments with three random seeds (2021, 2022, 2023), and present the error range in this section. Table 12 is the error range of Sanget-Specified NTL corresponding Table 1 of the main paper: Table 13 presents the error of experiments on Source-Only NTL corresponding to Table 3 of the main paper; Table 14 shows the error of model authorization which is presented Table in our main paper.
C.5 THE IMPACT OF GAUSSIAN KERNEL BANDWIDTH
In our implementation, we utilize series of Gaussian kernels to approximate MMD, which is im- plemented as the MK-MMD in Long et al. (2015). Specifically, the bandwidth in used kernels controlled by two parameters mul and num (we use mul 2.0, num = 5 in our experiments presented in the main text of the paper). The bandwidth of these kernels is follows:
F2 mul [num/2] /2]
num
[1
B
(20)
(n²
n
where and 22 two input data batches with size n, and [-] extracts the integer part of the input. To investigate the impact of kernel bandwidth, we select a series of mul-s and num-s to conduct MNIST, CIFAR10 and VisDA-T, and the results are shown in Table 15. According to the results, we can observe that the performance difference between the source and target nearly the same with different mul-s and num-s. As these parameters
Source-Onl Only NTL experiments
21

---

Published as a conference paper at ICLR 2022
Table 13: The error range (%) of experiment results for Supervised Learning and Source-Only NTL
respectively (the left of '/' is Supervised Learning, and the right is Source-Only NTL).
Source/Non-S
MT'
US
SN
MM
SD
MT 10.37/
#1.74/#1.23
#1.89/#1.22
+0.58/=0.21 +1.64/10.78
#1.97/#1.04
US
#1.27/#1.09
#0.10//:..4
#1.21/=0.77
#1.17/#0.79
SN
11.23/#1.04 #1.67/#1.31
#0.97/40.79
10.86/#1.11
#2.17/#1.03
MM
+1.28/+-0.1
#1.78/++.677
11.66//+1.75
12.10/+1.12 +1.78/410.54
SD
+0.91/+1.005
+1.61/41.24
+1.27/41.01
+1.19/+1.77
+0..7/+0.88
Table 14: The error range (%) of experiment results for authorizing usage of models trained with Source-Only Only NTL on digits.
Source with
Test SN
Test without Patch
Patch
MT
US
MM
SD
MT
US
SN
MM
SD
MT
10.43
+1.11
#1.89
=1.09
-1.37
#2.44
=1.22
=0.9
=1.74
#1.29
US
+1.73
+0.78
#0.88
=2.11
+2.60
#1.11
#0.73
=1.12
#1.71
#0.45
SN
+1.01
+1.04
#0.36
#0.97
+0.98
#1.17
=1011
=0.92
#1.39
#0.51
MM
+1.29
+1.19
+0.34
10.90
+1.20
#2.25
#0.87
11.13
+1.23
+1.28
SD
+:0.67
+1.29
+1.21
1.82
+0.47
11.39
+0.97
+133
+1.27
+2.61
directly determine the kernel bandwidth, these results demonstrate that the kernel bandwidth does not have a significant impact on NTL performance.
D POSSIBLE ATTACKS BASED ON NTL
Although we propose the Non-Transferable Learning for protecting the Intellectual Property in AlaaS, if the model owner is malicious, they can also utilize NTL to poison or implant backdoor triggers to their model evasively and release the model to the public. In the setting of applying Target-Specified NTL to verify the model ownership, the patch we used can also be regarded as a trigger for certain misclassification backdoor. From results of ownership verification in main paper, we can see the possibility of launching NTL-based target backdoor attacks. As for the case of Source-Only NTL, our objective shaped like an universal poison attack by restricting the generalization ability of models. The results in main paper demonstrate the feasibility of this poison attack. In addition, recently, there are more domain adaptation (DA) works about adapting the domain-shared knowledge within the source model the target without the access to the source data (Liang et al., 2020; Ahmed et al., 2021; Kundu et al., 2020). However, if the source model is trained with Source-Only NTL, believe that these DA works will be ineffective. In other words, our NTL can be regarded as a type of attack to these source-free DA works.
Different dis
Different dir
0
73 9
0 01233
6 7 8
0
11344644
9
0 1133466 7 8
0
2 4 7 2 9
1 2 3 5
1123 4
5 7 8 9
O
|| 2
3 4 98789
Figure 7: Augmentation results of USPS generated by the generative adversarial augmentation
framework.
22

---

Published as a conference paper at ICLR 2022
Table 15: The Source-Only NTL performance with different Gaussian kernel bandwidths (controlled by mul and num). The left of '/" is the performance on the source data, and the right is the average performance the target data.
Source/Bandwith
mul=1.0
mul=2.0
mul=3.0
num=3
num=5
num=7
performance on the source / target data (%)
MT
98.8 14.5
98.9 14.7
98.77 14.6
98.8 14.6
98.97 14.7
98.8 14.7
CIFARIO
87.7 10.3
87.8 10.2
87.8 / 10.3
87.7 / 10.5
87.8 10.2
87.7/ 10.4
VisDA-T
9 14.7
95.7 14.7
95.6/ 14.8
95.7 / 14.5
9 14.7
95.7 14.4
Different dis
Different dir
h
2
2
5
/2
0
31
12
5
G
20
4
84
0
2
3
4
5
69
8
2
18
8
2
32
5
9
0
12:
5
2
3
Figure 8: Augmentation SVHN generated by the generative adversarial augmentation framework.
Different dis
Different dir
Figure 9: Augmentation results of MNIST-M generated by the generative adversarial augmentation framework.
Different dis
Different dir
Figure 10: Augmentation results of SYN-D generated by the generative adversarial augmentation framework.
23

---

Published as a conference paper at ICLR 2022
Different dis
Different dir
Figure 11: Augmentation results of CIFAR1O generated by the generative adversarial augmentation framework.
Different dis
Different dir
Figure 12: Augmentation results of STLI0 generated by the generative adversarial augmentation framework.
Different dis
Different dir
Figure 13: Augmentation results of VisDA-T generated by the generative adversarial augmentation framework.
24


Reviews:
[{"review_id": "wdgyx7Y9EZR", "reviewer": "Reviewer_FcxC", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n+ The research direction is promising and important in the real world. Nowadays, AI companies will train their own deep models with abundant labelled data that costs a lot of resources. Thus, it is a good timing to research how to protect these models, which have become very important and practical. \n+ This paper proposed a method that can be effective solutions to both model verification and authorization, which is general and is promising to be applied in other applications.\n+ This paper is easy to follow. Experiments are enough to support the claims made in this paper. A plus should be that experiments are conducted with 6 removal approaches over the digits, CIFAR10 & STL10, and VisDA datasets.\n\nCons:\n- The presentation should be improved. The first paragraph in intro is too long. It is better to divided it into several paragraphs to better demonstrate the key points of this paper.\n- I am not sure if it is necessary to list the contributions in the introduction. Such contributions have been described clearly in intro and abs. It seems that you do not need to restate them.\n- Key related works are missing. For an AI company, they need to be aware of many adversarial attacks, such as reprogramming attacks, model-inversion attacks. These works are also related to IP protection of deep learning. It would be better to conclude these attacks as related works as well. Some discussions should be also added for general readers of ICLR.\n- Some notations should be changed. For example, we will not use X or Y to present distributions, instead, we will use them to represent random variables. It is better to use \\sP_X to represent the distribution corresponding to a random variable X. It is unnecessary to use GMMD, you can use MMD(P,Q; k), where k is a Gaussian kernel (you can follow the notations from recent deep kernel MMD papers).  \n- How many times do you repeat your experiments? I did not see error bar/STD values of your methods. This should be provided to verify that the experimental results are stable.\n- If we consider to add bandwidth to your kernel function, how does the kernel bandwidth affect your results?", "summary_of_the_paper": "In the era of deep learning, pre-trained models have been regarded as intellectual properties of AI companies. Thus, protecting these models has been more and more important. To achieve this aim, this paper proposes a non-transferable learning (NTL) method to capture the exclusive data representation in the learned model and restrict the model generalization ability to certain domains. This approach provides effective solutions to both model verification and authorization. Specifically: \n1) For ownership verification, watermarking techniques are commonly used but are often vulnerable to sophisticated watermark removal methods. By comparison, the NTL-based ownership verification provides robust resistance to state-of-the-art watermark removal methods, as shown in extensive experiments with 6 removal approaches over the digits, CIFAR10 & STL10, and VisDA datasets.\n2) For usage authorization, prior solutions focus on authorizing specific users to access the model, but authorized users can still apply the model to any data without restriction. The NTL-based authorization approach instead provides a data-centric protection, which is called applicability authorization, by significantly degrading the performance of the model on unauthorized data.\nIn general, this paper contributes a novel method to the field, and experiments verified the success of the proposed method. ", "summary_of_the_review": "In general, considering the significance of the researched problem, this paper can be accepted by the ICLR2022. However, some points should be clarified and strengthened in the revision. I would like to strongly support this paper if my concerns can be fully addressed.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "bpax7xtQ2qI", "reviewer": "Reviewer_RSPY", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper could be significantly improved via addressing the following issues:\n1. In Table 1, what is the number of  training epoches when transfering MT to MM? Did you try to increase the epochs of fine-tuning? If you train for enough epochs, the model would eventually reach the original accuracy. The sensitivity analysis regarding the epoches of your fine-tuning is necessary when compared to training from scratch and the transfer learning from the original model to the target task.\n2. The training complexity of using your NTL approach and the GAN training should be introduced in this paper? The computing time of the MMDs during each time step is at least twice your training time?\n3. The propsoed methodology is well presented. However, the differences between the proposed model and realted SOTA works should be presented clearly.\n4. Comparing Table 2 and Table 3, it can be seen that sometimes the source-only method shows greater performance compared to the target-specific method. The reasons why would this happen are interesting since providing the target-domain target should be more accurate when removing some part in the generalization space. However, the experiments seem does not agree with it.\n5. A future research section should be added in the revision.", "summary_of_the_paper": "Protecting the intellectual property of the trained models has received appealing attentions. Existing researches to protect  intellectual property fall into two major categories: ownership verification and usage authorization. To this end, the authors propose to utilize non-transferable learning to achieve both the goal of ownership verification and usage authorization. Extensive experiments on several representative datasets validate the effectiveness of the proposed method in terms of ownership verification. \n\nGenerally, this paper proposes a novel idea to address a practical problem in real-world applications, which could inspire many readers to follow it and have an important influence on the community of computer vision. I support the acceptance of this paper for a better ICLR conference. ", "summary_of_the_review": "This paper proposes an interesting question and gives the corresponding solution. I recommend the acceptance of this paper. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "E7_0Br4s8-", "reviewer": "Reviewer_1wBe", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Basically, the authors design a clever technique for learning nuisance-dependent representations. Such a representation can be made to perform accurately for a particular source domain, but poorly for another target domain. Furthermore, the authors design a GAN type technique for generating samples outside the source domain to serve as a kind of generic target domain. This is obviously important, as one cannot know to which target domain the model would be later adapted to.\n\nThis is a very interesting paper, although I have to say I'm not an expert in this topic at all. Most of the paper is really nicely written and is pretty easy to follow. The experimental verification is clear and detailed, but mostly limited to small images, so it's hard to say how it actually performs in some real-life scenarios.\n\nCouple questions come to mind:\n- Can you imagine uses of this to other kinds of models, e.g., language models, or is this mainly meaningful for image data?\n- It sounds like an NTL representation by nature is highly vulnerable to training data privacy attacks, like membership inference. Have you considered if one could use the NTL representation to particularly efficiently generate samples from (something close to) the training data distribution?", "summary_of_the_paper": "This paper introduces the idea of \"non-transferable learning\", which is roughly what the name indicates. The authors explain the value of this as a security/IP protection tool to protect the model from being used on unauthorized data. In addition, this presents a kind of attack against domain adaption works that try to improve generalization bounds without access to source data.", "summary_of_the_review": "Non-transferable learning is an interesting idea to explore, and this is the first step in that direction. I can imagine that there will be a lot of follow-up ideas both for attacking this, as well as improving upon it. I would definitely recommend accepting this for ICLR.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper addresses two important aspects of deep learning: model transferability and authorization for use. It presents original solutions for both of these problems. All of the reviewers agree that the paper is a valuable contributions. Minor concerns and critical remarks have been addressed by the authors during the discussion.
Decision: Accept (Oral)