Information Laundering for Model Privacy

1 INTRODUCTION . An emerging number of applications involve the following user-scenario . Alice developed a model that takes a specific query as input and calculates a response as output . The model is a stochastic black-box that may represent a novel type of ensemble models , a known deep neural network architecture with sophisticated parameter tuning , or a physical law described by stochastic differential equations . Bob is a user that sends a query to Alice and obtains the corresponding response for his specific purposes , whether benign or adversarial . Examples of the above scenario include many recent Machine-Learning-as-a-Service ( MLaaS ) services ( Alabbadi , 2011 ; Ribeiro et al. , 2015 ; Xian et al. , 2020 ) and artificial intelligence chips , where Alice represents a learning service provider , and Bob represents users . Suppose that Bob obtains sufficient paired input-output data as generated from Alice ’ s black-box model , it is conceivable that Bob could treat it as supervised data and reconstruct Alice ’ s model to some extent . From the view of Alice , her model may be treated as valuable and private . As Bob who queries the model may be benign or adversarial , Alice may intend to offer limited utility for the return of enhanced privacy . The above concern naturally motivates the following problem . ( Q1 ) How to enhance the privacy for an already-learned model ? Note that the above problem is not about data privacy , where the typical goal is to prevent adversarial inference of the data information during data transmission or model training . In contrast , model privacy concerns an alreadyestablished model . We propose to study a general approach to jointly maneuver the original query ’ s input and output so that Bob finds it challenging to guess Alice ’ s core model . As illustrated in Figure 1a , Alice ’ s model is treated as a transition kernel ( or communication channel ) that produces Ỹ conditional on any given X̃ . Compared with an honest service Alice would have provided ( Figure 1b ) , the input X̃ is a maneuvered version of Bob ’ s original inputX ; Moreover , Alice may choose ( a ) ( b ) to return a perturbed outcome Y instead of Ỹ to Bob . Consequently , the apparent kernel from Bob ’ s input queryX to the output response Y is a cascade of three kernels , denoted byK in Figure 1a . The above perspective provides a natural and general framework to study model privacy . Admittedly , if Alice produces a ( nearly ) random response , adversaries will find it difficult to steal the model , while benign users will find it useless . Consequently , we raise another problem . ( Q2 ) How to formulate the model privacy-utility tradeoff , and what is the optimal way of imposing privacy ? To address this question , we formulate a model privacy framework from an informationtheoretic perspective , named information laundering . We briefly describe the idea below . The general goal is to jointly design the input and output kernels ( K1 and K2 in Figure 1a ) that deliberately maneuver the intended input and output for queries of the model so that 1 ) the effective kernel ( K in Figure 1a ) for Bob is not too far away from the original kernel ( K∗ in Figure 1a ) , and 2 ) adversarial acquisition of the model becomes difficult . Alternatively , Alice ‘ launders ’ the input-output information maximally given a fixed utility loss . To find the optimal way of information laundering , we propose an objective function that involves two components : the first being the information shared between X , X̃ and between Ỹ , Y , and the second being the average Kullback-Leibler ( KL ) divergence between the conditional distribution describing K and K∗ . Intuitively , the first component controls the difficulty of guessing K∗ sandwiched between two artificial kernels K1 and K2 , while the second component ensures that overall utility is maximized under the same privacy constraints . By optimizing the objective for varying weights between the components , we can quantify the fundamental tradeoffs between model utility and privacy . 1.1 RELATED WORK . We introduce some closely related literature below . Section 3.3 will incorporate more technical discussions on some related but different frameworks , including information bottleneck , local data privacy , information privacy , and adversarial model attack . A closely related subject of study is data privacy , which has received extensive attention in recent years due to societal concerns ( Voigt & Von dem Bussche , 2017 ; Evans et al. , 2015 ; Cross & Cavallaro , 2020 ; Google , 2019 ; Facebook , 2020 ) . Data privacy concerns the protection of ( usually personal ) data information from different perspectives , including lossless cryptography ( Yao , 1982 ; Chaum et al. , 1988 ) , randomized data collection ( Evfimievski et al. , 2003 ; Kasiviswanathan et al. , 2011 ; Ding & Ding , 2020 ) , statistical database query ( Dwork & Nissim , 2004 ; Dwork , 2011 ) , membership inference ( Shokri et al. , 2017 ) , and Federated learning ( Shokri & Shmatikov , 2015 ; Konevcny et al. , 2016 ; McMahan et al. , 2017 ; Yang et al. , 2019 ; Diao et al. , 2020 ) . A common goal in data privacy is to obfuscate individual-level data values while still enabling population-wide learning . In contrast , the subject of model privacy focuses on protecting a single learned model ready to deploy . For example , we want to privatize a classifier to deploy on the cloud for public use , whether the model is previously trained from raw image data or a data-private procedure . Another closely related subject is the model extraction in ( Tramèr et al. , 2016 ; Papernot et al. , 2016b ) , where Bob ’ s goal is to reconstruct Alice ’ s model from several queries ’ inputs and outputs , knowing what specific model Alice uses . For example , suppose that Alice ’ s model is a generalized linear regression with p features . In that case , it is likely to be reconstructed using p queries of the expected mean ( a known function ofXβ ) by solving equations ( Tramèr et al. , 2016 ) . In the supervised classification scenario , when only labels are returned to any given input , model extraction could be cast as an active learning problem where the goal is to query most efficiently ( Chandrasekaran et al. , 2018 ) . Model extraction was also studied in contexts beyond the prediction API , e.g. , when an adversary utilizes the gradient information ( Milli et al. , 2019 ) . From Alice ’ s perspective , there exist several solutions to safeguard against model leakage . A warning-based method was developed in ( Kesarwani et al. , 2018 ) , where Alice continuously monitors the information gain and raise alarms when they become unusual . Another warning method was developed in ( Juuti et al. , 2019 ) , where the detection of an adversary is based on testing whether the pairwise distances among queried data approximately follow the Gaussian distribution . The work in ( Lee et al. , 2018 ) developed a defense strategy for the setting where the target is a classifier , and the adversary queries each class ’ s probability . The probabilities are maximally perturbed under the constraint that the most-likely class label ) remains the same . The work in ( Orekondy et al. , 2019 ) studied a similar setting but from a different perspective . The main idea is to perturb the probabilities within an ` 2-distance constraint to poison the adversary ’ s gradient signals . 1.2 CONTRIBUTIONS AND OUTLINE . The main contributions of this work are three folds . First , we develop a novel concept , theory , and method , generally referred to as information laundering , to study model privacy . Unlike data privacy that concerns the protection of raw data information , model privacy aims to privatize an already-learned model for public use . To the best of the authors ’ knowledge , this work is the first framework to study model privacy in a principled manner . Second , under the developed informationtheoretic framework , we cast the tradeoffs between model privacy and utility as a general optimization problem . We derive the optimal solution using the calculus of variations and provide extensive discussions on the solution ’ s insights from different angles . Third , we develop a concrete algorithm , prove its convergence , and elaborate on some specific cases . We also provide some experimental studies to illustrate the concepts , and discuss several future problems at the end . The paper is organized as follows . In Section 2 , we describe the problem formulation and a general approach to protect the model . In Section 3 , we propose the information laundering method that casts the model privacy-utility tradeoff as an optimization problem and derives a general solution . In Section 3.3 , we provide some additional discussions of the related frameworks , including information bottleneck , local data privacy , information privacy , and adversarial model attack . In Section 5 , we conclude the paper with some potential future work . In the Appendix , we provide the proofs of the main results and experimental studies . 2 FORMULATION . 2.1 BACKGROUND . The private model can be obtained from general learning methods , and its deployment means that it will return a response for a given input query . Suppose that X and Y are the input and output alphabets ( data space ) , respectively . Definition 1 ( Learned model ) A learned model is a kernel p : X × Y → [ 0 , 1 ] , which induces a class of conditional distributions { p ( · | x ) : x ∈ X } . A model in the above definition is also referred to as a communication channel in information theory . A model can be regarded as the input-output ( or Alice ’ s application programming interface , API ) offered to Bob . Examples include a regression/classification model that outputs predicted labels , a clustering model that outputs the probabilities of belonging to specific groups , and a stochastic differential equation system that outputs the likely paths for various inputs variables . It does not matter where the model comes from since we are only concerned about the privacy of a fixed given model . The ( authentic ) model of Alice is denoted by pK∗ . An adversary Bob is a user that can access the above model ’ s API , providing an arbitrary input , X , and obtaining an output , Y . Bob aims to use as few queries as possible to construct a model that closely matches Alice ’ s model pK∗ . We will formalize the ‘ closeness ’ using the KL divergence . What is model privacy ? Our perspective is that privacy is not an intrinsic quantity associated with a model ; instead , it is a measure of information that arises from interactions between the model and its queries . In our context , the interactions are throughX ( offered by Bob ) and Y ( offered by Alice ) . The key idea of enhancing Alice ’ s model privacy is to let Alice output noisy predictions Ỹ for any input X so that Bob can not easily infer Alice ’ s original model . Similarly , Alice may choose to manipulate X as well before passing it through K∗ . Alternatively , Alice intends to 1 ) impose some ambiguity between X , X̃ , and between Y , Ỹ , which conceivably will produce response deviating from the original one , and 2 ) seek theK closest toK∗ under the same amount of ambiguity imposed . Motivated by the above concepts , we introduce the following notion . Definition 2 ( Information-laundered model ) A information-laundered model with respect to a given model K∗ is a model K that consists of three internal kernels K = K1 ◦ K∗ ◦ K2 ( illustrated in Figure 1 ) . Naturally , the information-laundered model of Alice is denoted by pK .

Summary: This paper aims to tackle the problem of model stealing/extraction, as in stealing a model that is deployed remotely, through API access. The threat model that they are aiming to protect against is not well-defined. The proposed method is information theoretic. They propose adding two modules (kernels) before and after the main deployed model, to "launder" information.  The loss function for achieving the desired modules consists of two main terms, for utility, and privacy of the model. The utility term tries to keep the expected value of the output being accurate high, while the privacy term (which are actually two terms for the two modules) try to decrease the MI between the true output/input and the laundered ones. They then offer an iterative approach for minimizing the loss.