You are caught stealing my winning lottery ticket! Making a lottery ticket claim its ownership

1 Introduction . Deep neural networks ( DNNs ) have dramatically raised the state-of-the-art performance in various fields . However , the over-parameterization of DNNs becomes a non-negligible problem . The amount of parameters now is often on the billion scale , which significantly increases the inference cost when using these models . An emerging field of lottery ticket hypothesis ( LTH ) explores a new scheme for pruning the model without sacrificing performance . The core idea is to identify the sparsity pattern ahead of training ( or in its early stage ) and train a sparse network from scratch . It has been hypothesized [ 1 ] that DNNs contain sparse networks named winning tickets that can be trained to match the test accuracy of the full model . These winning tickets hence have comparable or even better inference performance while potentially reducing the computational footprints . However , finding winning tickets is a non-trivial task : it involves the training-prune-retraining cycle for several times [ 1 ] , which is burdensome and computation-consuming . Although other works [ 2–4 ] have shown that sparsity might emerge at the initialization or at the early stage of training , the iterative magnitude pruning ( IMP ) still outperforms these alternatives by clear margins [ 5 ] . Yet , the powerful IMP method requires multiple rounds of train-prune-train process on the original training set , which * Equal Contribution . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . is even much more expensive than training a dense network . That makes a found winning ticket a valuable asset to the owners , highlighting the necessity of protecting the winning tickets ’ copyright . Previous works [ 6–9 ] have shown that deep networks are vulnerable to intellectual property ( IP ) infringement . For example , one can use transfer learning to adapt a trained model onto a new task or use model compression techniques to create a new sparse model based on the target model . Fortunately , in recent years the ownership verification problem has been addressed with a number of solutions proposed . The key idea is to embed verifiable information , or called signatures , into models ’ weights [ 6 , 10 , 11 ] or predictions [ 7 ] without visibly affecting the original performance . By extracting the embedded information from models , one can verify the ownership of models and hence protect their IPs . For the methods that embed information in weights , additional weights reg- ularizers are often used to enforce certain patterns , such as signs . As for the prediction methods , a special training set , which is often called a trigger set , is used as additional training data . The model trained upon both the original data and the trigger set can generate desired prediction labels for the privately-held trigger set , while preserving the performance on the original training set . However , those general methods did not take any structural property ( e.g. , sparsity ) into account , leaving chance for improving their gains in the winning ticket mask protection . On protecting the IP of winning tickets , we investigate a novel way to leverage sparse structural information for ownership verification ( Fig . 1 ) . This structural information embedded in winning tickets is a good `` credential '' for ownership verification since the winning ticket at extreme sparsity is naturally robust to fine-tuning and ( further ) pruning attacks . The winning ticket at extreme sparsity can not be pruned further ; otherwise , the inference performance will drop ( hence losing its `` value '' ) . Meanwhile , fine-tuning the winning tickets can only tune the weights , but the sparsity pattern will not be changed . However , there remain some key questions to answer : How to formulate the ownership verification process under the context of the lottery ticket hypothesis ? What kind of structural information should be used ? How to inject user-specific information into the structure of winning tickets ? We present answers to these questions in this paper . We summarize our findings as follows : • We formulate the lottery verification problem and define two different protection scenarios . We show that even without specific protection , the extremely sparse winning ticket can partly claim its ownership because of the critical role of its sparse structure in the final inference performance . • We further propose a new mask embedding method that is capable of embedding ownership “ signatures '' in the subnetwork ’ s sparse structural connectivity ( Fig . 1 ) , without much affecting its performance . The signature is robust , e.g. , it can be extracted and decoded even after pruning or fine-tuning attacks . Combined further with the trigger set-based method , our mask embedding method can work under both white-box and black-box verification frameworks . • We investigate several verification schemes , i.e. , separate masks , embedding signatures , and embedding signatures with the trigger set . We show that these schemes are robust to the common removal and ambiguity attacks , as well as a new type of “ add-on '' attacks . Extensive experiment results demonstrate their competence on protecting the winning tickets . For example , on ResNet-20 , our verification framework can defend fine-tuning attacks intrinsically , as well as pruning attacks and as add-on attack under all levels of pruning ratios . 2 Related Work . Pruning and Lottery Ticket Hypothesis . Pruning algorithms can be roughly classified into two types : pruning after training and pruning before training . Conventional pruning after algorithms assign scores to individual weights and remove weights with the lowest scores [ 12 ] . There are a large number of scoring algorithms of this kind , including weights magnitude [ 13 , 14 ] , Taylor coefficients [ 15–18 ] , and other variants [ 19 , 20 ] . Pruning-before-training methods also play important roles in the community [ 1 ] . However , it requires multiple cycles of training and pruning process [ 12 ] , making it sometimes tedious in practical use . Pruning-at-initialization methods alleviate such cost by pruning initial weights such as observing initial gradients of weights [ 2 , 3 ] , but the quality of sparse networks found by these methods are mediocre in return . A representative pruning-before-training method is the lottery ticket hypothesis [ 1 ] , which hypothesizes the existence of a sparse network in dense networks that can be trained to match the test accuracy of dense networks in isolation . It was later verified that the original hypothesis was too strong , and early rewind techniques [ 21 ] help scale up the original version . Later on , the lottery ticket hypothesis and its variants have been explored and extended to various fields [ 22–30 ] such as image generation [ 31 , 27 , 32 ] and natural language processing [ 22 , 24 ] . However , it is currently non-trivial to find winning tickets , especially at high sparsity since multiple train-prune-train processes are required , which suggests the practical value of protecting the found sparse networks . Ownership Verification . Ownership verification has drawn attention from both the industry and academia . Many works have been proposed to address IP protection . One most popular way is to use watermarking algorithm : [ 6 ] proposed to embed watermarks in the form of bits into deep networks ’ weights by an additional regularization term . [ 10 ] embedded information into model weights by regularizing on the signs of weights . Besides watermarking on weights , [ 7 , 33 ] embedded watermarks in the labels of certain examples in a trigger set , which makes it possible to extract watermarks through a service interface without directly accessing the models ’ weights ( black-box setting ) . Following somehow different pathways , [ 34 ] proposed a passport-based approach that encodes signatures with special passport layers . [ 11 ] presented passport-embedded normalization whose parameters are associated with signatures . However , none of these methods leverages structural information for ownership verification besides assuming general dense networks . For sparse networks , the sparse mask patterns represent the key information and can vary across models and tasks . 3 The Lottery Ticket Claims its Ownership . 3.1 The Lottery Ticket Hypothesis . ¶ Sparse Masks , Subnetworks and Winning Tickets . We define a neural network parameterized by W as N [ W ] ( · ) . With a slight abuse of notion , we define a subnetwork of N [ W ] as N [ W , M ] : = N [ W M ] ( · ) where M is a sparse mask whose shape is the same as W but the value of each entry in which can only be either 0 or 1 . Given W0 the initialization of N , if N [ W0 , M ] can be retrained to match the test performance of the dense model training from N [ W0 ] , we call N [ W0 , M ] a winning ticket . The term re-train above is used to distinguish between the training process to find the winning ticket . The criterion for matching can be set as , e.g. , no lower than 1 % than dense models ’ performance . · Sparsity Comparison . The sparsity of a sparse mask M can be defined as spar ( M ) : = ‖M‖0/‖M + 1‖0 where ‖ · ‖0 represents the non-zero values of the input matrix , and the relative sparsity can be defined as rspar ( M1 , M2 ) : = spar ( M1 ) / ( spar ( M2 ) + ) . We call a sparse mask M1 is sparser than M if and only if ‖M1‖0 < ‖M‖0 . A sub-mask M2 is a mask that has the same shape as M satisfying that all the elements in M−M2 are non-negative . ¸ Extremely Sparse Condition . Given a sparsity difference threshold t , we call N [ W0 , Me ] an extremely sparse winning ticket ( or referred to as an extreme ticket hereinafter for conciseness ) if N [ W0 , Me ] can match the performance of N [ W0 ] , but pruning the model ( i.e. , increase the sparsity of Me ) t× 100 % further can not produce a winning ticket . In our experiments , we set t to be 0.01 . 3.2 Verification Framework for Extremely Sparse Winning Tickets . A ownership verification framework for extremely sparse winning tickets can be formulated as a tuple V = ( ME , WE , F , V , I ) , each item of which is a process : • A mask embedding process ME ( M0 , s ) ( optionally ) for sparse masks . s is an optional string that can be encoded into masks . The output of this process is a new mask M. M can either be a mask with s embedded or contains other structural information that is useful for ownership verification . We call the verification method with ME ( M0 , s ) enabled a “ mask-based ” method . • A weight embedding process WE ( Dtr , T , s , N [ · ] , L , W0 , M ) , which is a learning process for the lottery ticket model . Dtr = { x , y } is the training dataset , T = { Tx , Ty } is an optional trigger set provided to the training process , s is an optional signature for the weights embedding process . L is the loss function for model training ( usually the cross-entropy loss ) , N [ · ] defines the model structure , W0 is the initialization of weights , and M is the sparse mask for the model . The output of E is a model N with sparse weights W M where W represents the trained weights . The trigger set T ( or/and ) the signature s are embedded in W M after this process and can be verified with the verification process introduced next . • A fidelity evaluation process F ( N [ · ] , W , M , Dte , Af , f ) is to evaluate whether the performance discrepancy of model N [ · ] is less than a pre-defined threshold f , i.e. , |A ( N [ W , M ] , Dtr ) −Af | < f , in whichA ( · , · ) is the inference performance on the test dataset Dte , andAf is a target inference performance associated with the model . • A verification process V ( N [ · ] , W , M , T , s , s ) checks whether the sparse mask M or the trigger set T can be successfully verified for a given model N [ · ] . For the mask-based methods , the process is to check if M and s matches by evaluating N [ · , M ] on Dte to see if the performance gap is smaller than a pre-defined threshold s , and ( or ) extract information in M and compare it with s. For the trigger set-based methods , an inference is first executed on the trigger set images Tx , and then the prediction will be compared with trigger set labels Ty to see if the false detection rate is lesser than a threshold s [ 34 ] . • An invert process I ( N [ W , M ] , T , s ) exists and will enable a successful ambiguity attack [ 34 ] if : a ) a set of new trigger set T′ , a new signature s′ , or a new mask M′ can be reverse-engineered for the given mask M and weights W. b ) the forged T′ , s′ , M′ can be verified with respect to M and W. c ) the fidelity evaluation outcome F ( N [ · ] , W , M′ , Dte , At , f ) remains True . The high-level definitions above are general and can work with any concrete implementation . We will introduce several methods that use sparse structural information in the next following sections .

Summary: Winning tickets can often be costly to discover, so this paper explores the issue of copyright protection for winning tickets -- protecting against IP infringement on the owners of a winning ticket. Namely, this paper studies “lottery verification” by analyzing sparse topology of a winning lottery ticket network with the use of several graph-based signatures (i.e., verifiable patterns that can be embedded into the weights/predictions of the winning ticket). Further, these attacks are robust to fine-tuning and extra pruning, which is extremely important for robust validation of winning tickets.While this area has been extensively studied for neural networks in general, this paper seems to be the first to study ownership verification for winning tickets/sparse neural networks. They propose three separate methods for ownership verification: key masks, embedded signatures, and trigger-based methods. Each of these methods are evaluated empirically on the CIFAR datasets using ResNet model variants.