DarKnight: A Data Privacy Scheme for Training and Inference of Deep Neural Networks

1 INTRODUCTION . The need for protecting input privacy in Deep learning is growing rapidly in many areas such as health care ( Esteva et al. , 2019 ) , autonomous vehicles ( Zhu et al. , 2014 ) , finance ( Heaton et al. , 2017 ) , communication technologies ( Foerster et al. , 2016 ) etc . Many of the data holders are , however , not machine learning experts . Hence , the need for machine learning as a service ( MLaaS ) has emerged . Microsoft Azure ML ( Microsoft , 2020 ) , Google AI platform ( Google , 2020 ) , Amazon ML ( Amazon , 2020 ) are some examples . These services provide computing infrastructure and ML runtime to enable data holders to quickly set up their models and train . While these platforms accelerate the ML setup process , they exacerbate the user ’ s concern regarding the data privacy . In this paper , we propose DarKnight , a unified inference and training framework that protects data privacy with rigorous bounds on information leakage . DarKnight takes a unique hybrid-execution approach where it uses trusted execution environments ( TEE ) to blind input data using matrix masking techniques , and then uses GPUs to accelerate DNN ’ s linear computations on the blinded data . Training or inference solely within TEEs can provide data privacy , by blocking access to TEE memory for all intruders , including root users , using hardware encryption and isolation mechanisms . However , TEE-enabled CPUs have limited computation power and memory availability , which creates unacceptable performance hurdles to run an entire model within a TEE . Linear operations ( convolution , matrix multiplication , etc ) are orders of magnitudes faster on a GPU compared to a TEEenabled CPU . DarKnight offloads these compute-intensive linear operations to GPU . DarKnight ’ s usage of TEEs is limited to protecting the privacy of data through a novel matrix masking of multiple inputs and performing non-linear operations ( RelU , Maxpool ) . In terms of applicability , DarKnight allows users to train using floating-point ( FP ) representation for model parameters , while still providing rigorous bounds on information leakage . FP models are routinely used in training due to convergence , accuracy and faster implementation considerations ( Johnson , 2018 ; Guo et al. , 2020 ; Imani et al. , 2019 ) . Many DNN accelerators use bfloat16 ( Kalamkar et al. , 2019 ) which is a half precision FP . This format is used in Intel Xeon Processors , Intel FPGAs ( Nervana , 2018 ) , Google Cloud TPUs ( Cloud , 2018 ) and Tenserflow ( Google , 2018 ) . Several prior works on protecting privacy , however , use operations on finite fields to provide formal bounds . Such an approach limits their usage to integer arithmetic on quantized models ( Mohassel & Zhang , 2017 ; Gascón et al. , 2017 ; So et al. , 2019 ) . In this work , we allow training to use FP values and we bound the amount of information leakage with a rigorous mathematical proof . The information leakage is bounded by the variance of the additive noise and other parameters of the DarKnight blinding . We implemented DarKnight using an Intel SGX-enabled CPU to perform matrix masking and nonlinear DNN operations , while using an Nvidia GPU to accelerate linear operations . The blinding parameters in our experiments were chosen so as to preserve the original accuracy of training a model . Using these parameters DarKnight guarantees that no more than one bit of information is leaked from a one megapixel input image . Note that this will be an upper bound on the leaked information , assuming that the adversary has access to unlimited computation power to decode the blinded inputs . To the best of our knowledge , this is the first work that uses TEE-GPU collaboration for training large DNNs . The rest of the paper is organized as follow . In Section 2 , we explain the background . Section 3 describes the methodology for inference and training . In section 4 privacy theorem is provided . Experimental results are presented in section 5 . In section 6 , we draw the conclusion . 2 RELATED WORK AND BACKGROUND . 2.1 INTEL SGX . TEEs such as ARMTrustZone ( Alves , 2004 ) , Intel SGX ( Costan & Devadas , 2016 ) , and Sanctum ( Costan et al. , 2016 ) provide an execution environment where computational integrity of user ’ s application is guaranteed by the hardware . TEEs generally provide a limited amount of secure memory that is tamper proof even from a root user . SGX provides 128 MB as the enclave memory . An entire DNN model and data can be wrapped in an enclave for private execution but if size of the private data exceeds the 128MB TEE limit , it will pay a significant performance penalty for encryption and eviction of pages for swapping . While some types of side-channel attacks have been performed on SGX , many of these attacks are being fixed actively ( Costan & Devadas , 2016 ; Xu et al. , 2015 ) . In this work we assume that SGX computations are invisible to the outside entities . 2.2 RELATED WORK . There are a variety of approaches for protecting input privacy during DNN training and inference . We categorized these approaches in Table 1 . Homomorphic encryption ( HE ) techniques encrypt input data and then perform inference directly on encrypted data , albeit with significant performance penalty ( and hence are rarely used in training DNNs ) . Secure multi-party computing ( MPC ) is another approach , where multiple non-colluding servers may use custom data exchange protocols to protect input data . However , this approach requires multiple servers to perform training or inference . An entirely orthogonal approach is to use differential privacy ( DiifP ) , which protects user information through probabilistic guarantees . Additive Noise is another approach mostly used for inference , where there is a trade-off between the privacy , computational complexity and , model accuracy . In some of the works mentioned below a combination of forenamed techniques is used . Among those approaches , ( Tramer & Boneh , 2018 ) introduced Slalom an inference framework that uses TEE-GPU collaboration to protect data privacy and integrity . However , as stated in their work their quantized model was not designed for training DNNs . We elaborate on these reasons in Appendix E . 3 DARKNIGHT . 3.1 THREAT MODEL . Adversary capabilities : While adversaries can perform various attacks on DNN models and datasets ( Riazi et al. , 2019 ) , DarKnight focuses on attacks that expose the datasets used in training or inference and attacks that modify computational results on untrusted hardware . Model privacy and side channel attacks are out of the scope of this work . Within this scope , the adversary is assumed to have the full root access to the system , which includes the GPU in our setup . The adversary can not see any computations or data stored within the TEE . But the adversary has unrestricted access to data that leaves TEE , such as the blinded input data and can alter computational results performed on the GPU . Since model protection is outside of the scope we assume the adversary can access the DNN model parameters . Information-theoretic Data Privacy : We quantify information leakage in terms of the mutual information between original inputs and blinded inputs that are visible to the adversary . More precisely , from an information theoretical point of view , an adversary with an unlimited computation power who observes unlimited number of blinded inputs can not gain more information about original inputs than what our upper bound on leakage provides . This upper bound itself can be controlled by the power of noise and other blinding parameters in our design . In our implementation we selected these parameters such that the overall training or inference accuracy is not reduced due to them . In section 4 and Appendix A , we provide the details of our theoretical analysis . Computation Integrity : Since the adversary has access to blinded inputs , it can alter the returned values to the TEE to manipulate model training or inference . DarKnight can verify the computations performed in the unsecured GPU up to the computation precision . In the other words , DarKnight detects if the results are altered more than the computation precision by an adversary . 3.2 DARKNIGHT OVERVIEW DarKnight supports both private inference and training in a single framework . Fig . 1 depicts the overall execution flow of DarKnight . A cloud server with an SGX enclave and GPU accelerator forms the computing base . DarKnight uses SGX to blind input data while enabling GPUs to perform computationally intensive linear operations on private data . The initial model ( W ) that a user wants to train is loaded into the cloud server , and is made accessible to the untrusted GPU as well . DarKnight then uses the following steps : ( 1 ) A batch of training/inference input data set is encrypted by the client using a mutually agreed keys with SGX and sent to the server . ( 2 ) SGX decrypts the images and starts the forward and backward propagation . ( 3 ) During the forward/backward pass , each layer requires some linear and nonlinear operations . Before offloading linear operations to GPU , SGX calls DarKnight ’ s blinding mechanism to seal the data . To seal the data , DarKnight uses the notion of a virtual batch , where K inputs are linearly combined to form K coded inputs . The size of the virtual batch is limited by the size of the SGX memory that is necessary to blind K images , typically 4-8 images at a time . ( 4 ) The blinded data is offloaded to GPU for linear operation . ( 5 ) GPU performs linear operations on blinded data and returns the data back to SGX labeled as step 6 . ( 7 ) SGX decodes the received computational outputs using DarKnight ’ s decoding strategy and then performs any non-linear operations within SGX . This process is repeated both for forward and backward propagation of each layer . 3.3 PRIVACY IN INFERENCE . In this section , we start with DarKnight ’ s inference strategy . We consider a trained DNN , represented by model parameters W with L layers , which is performing inference on input x0 , which must be protected . At a layer l the inference process computes yl = 〈Wl , xl〉 , where Wl and xl represent the model parameters and inputs in layer l , and 〈· , ·〉 corresponds to the bilinear operation at that layer ( e.g . matrix product , convolution , etc. ) . After the linear operation finishes , an activation function ( g ( · ) ) creates the next layer input xl+1 = g ( yl ) . Within this context , DarKnight first receives a set of K inputs x ( 1 ) 0 , . . . , x ( K ) 0 for a batch inference from a client . Our goal is to perform linear calculations of y ( 1 ) 0 = 〈W0 , x ( 1 ) 0 〉 , . . . , y ( K ) 0 = 〈W0 , x ( K ) 0 〉 on the GPU without exposing the inputs to the GPU . Note that the subscript 0 in all these variables refers to the first layer . At this point , we drop the subscript for a more clear notation . Also , we apply x for the inputs that need to be protected and x̄ for the blinded inputs to visually distinguish different notations . Key Insight : The main idea behind DarKnight ’ s privacy protection scheme is the fact that the most computationally intensive operator ( such as convolutions ) is bilinear . Thus , instead of asking the GPU to calculate 〈W , x ( i ) 〉 , which exposes the inputs , DarKnight uses matrix masking to linearly combine the inputs and add a random noise to them . Due to the bilinear property , any linear operation on K masked inputs can be recovered if there are K different linear computations performed . Matrix Masking : Introduced by ( Cox , 1980 ; 1994 ; Kim , 1986 ; Spruill , 1983 ) , matrix masking scheme can be used for variety of reasons such as noise addition , sampling and etc . The general form of B X A+C is used for protecting Matrix X . Any of these matrices can be used for masking data based on the data privacy goal . For DarKnight we use A and C as we explain the in this section . DarKnight Matrix Masking ( Blinding ) : More specifically , DarKnight creates K + 1 inputs x̄ ( 1 ) , . . . , x̄ ( K ) , as follows , x̄ ( i ) = αi,1x ( 1 ) + · · ·+ αi , Kx ( K ) + αi , ( K+1 ) r , i = 1 , . . . , ( K + 1 ) ( 1 ) The scalars αi , j , and the noise vector r are randomly generated ; and the size of r matches that of x . The scalars αi , j ’ s are represented by matrix A , which are dynamically generated for each batch and securely stored inside SGX for unblinding . Hence , by revealing the values x̄ ( i ) ’ s to GPU , we do not expose the inputs x ( i ) ’ s . At the next step , the blinded data x̄ ( i ) ’ s are sent to the GPU which performs the following computations : ȳ ( i ) = 〈W , x̄ ( i ) 〉 , i = 1 , . . . , ( K + 1 ) . Please note that matrix A can be chosen such that its condition number close to one , so that blinding and unblinding algorithm remains numerically stable . For this purpose , orthogonal matrices serve us the best . DarKnight Unblinding : The K + 1 outputs ȳ ( i ) returned from the GPU must be unblinded to extract the original results y ( i ) . These value can be extracted as follows , Ȳ = 〈 W , [ x̄ ( 1 ) , . . . , x̄ ( K+1 ) ] 〉 = 〈 W , [ x ( 1 ) , . . . , x ( K ) , r ] 〉 ︸ ︷︷ ︸ Y ·A ⇒ Y = Ȳ ·A−1 ( 2 ) DarKnight Advantages : ( 1 ) Unlike prior works ( Tramer & Boneh , 2018 ) DarKnight does not need to store W · r within the SGX memory thereby significantly enhancing our ability to infer with much larger models . ( 2 ) size of the matrix A is proportional to the number of inputs that are blinded together ( K ) , and is orders of magnitude smaller the model size W. Hence , the order complexity of Blinding/Unblinding operations is much less than the linear operations ( 〈W , x〉 ) in a DNN with millions of parameters . ( 3 ) The process of unblinding K inputs with one random noise requires K + 1 computations . During unblinding we extract W · r , but that value is just dropped . Thus DarKnight trades 1/K additional computations in order to eliminate the need to secure very large model parameters .

Summary: The paper builds on previous work like Slalom to propose a new secure training and inference protocol in the TEE+GPU paradigm. The main technical contribution of this work is a new blinding algorithm that dramatically reduces the memory required to store the blinding parameters (decoupling it from the input/model size). The authors then build on this to extend their blinding scheme to the training use-case.