Efficient representations for privacy-preserving inference

1 INTRODUCTION . In recent years , deep neural networks have achieved state-of-the-art accuracy for tasks such as image recognition . They have been deployed in a range of sectors , powering a wide variety of applications such as recommendation systems , medical diagnosis , and content filtering . Machine Learning as a Service ( MLaaS ) is a framework in which cloud services apply machine learning algorithms on usersupplied data to produce an inference result which is then returned to the user . Cloud systems are an attractive platform for deploying pretrained models due to the relatively low cost and the availability of remote servers . However , the data has to be decrypted before inference , which allows a serverside adversary to have access to the user ’ s information . Homomorphic encryption ( HE ) , can be applied to enable inference to be performed on encrypted data , enabling the result to be delivered to the user without risk of the server accessing the original data or the inference result . CRYPTONETS ( Gilad-Bachrach et al. , 2016 ) was the first application of HE to secure neural network inference , and leveraged the YASHE ’ scheme to perform MNIST classifications . CRYPTONETS suffers from a high number of homomorphic operations ( HOPs ) , with a single MNIST inference requiring ∼290 , 000 homomorphic multiplications and ∼250 seconds of inference latency . Subsequent works such as FASTER CRYPTONETS ( Chou et al. , 2018 ) used neural network surgery and a faster encryption scheme to reduce the inference latency of CRYPTONETS . Later works utilised ciphertext rotations as opposed to the SIMD packing scheme , enabling convolutional and fully connected layers to be computed using much fewer HOPs ( Juvekar et al. , 2018 ; Mishra et al. , 2020 ) . This has been shown to reduce the inference latency of MNIST models by more than an order of magnitude , bringing confidence that private inference can be practical . LOLA ( Brutzkus et al. , 2019 ) proposed novel representations for intermediate tensors and their MNIST model requires only 2.2 seconds for one inference . One drawback of their representations is poorly scalability to harder datasets such as CIFAR-10 due to the limited number of slots per ciphertext acting as a barrier to the size of tensors that are practical . The limitated set of operations supported by HE schemes prevents the secure computation of nonpolynomial activation functions which impedes model training due to the problem of exploding gradients ( Chou et al. , 2018 ) . To address this , others have proposed the use of secure multi-party computation to enable secure computation of non-polynomial activations using multiple parties ( Juvekar et al. , 2018 ; Mishra et al. , 2020 ) . Despite enabling use of popular nonpolynomial activations such as ReLU , relying on multiparty computation incurs large amounts of data transfer between parties and the requirement for the parties involved to be online and have feasibly fast data transfer rates . For example , GAZELLE ( Juvekar et al. , 2018 ) requires ∼1 GB of data transfer per inference for their CIFAR-10 model , and DELPHI ( Mishra et al. , 2020 ) requires ∼2 GB per inference with a ResNet-32 model . Single-party approaches often choose to approximate the ReLU activation using a second-degree polynomial ( Gilad-Bachrach et al. , 2016 ; Chou et al. , 2018 ; Brutzkus et al. , 2019 ) . In this work , we introduce a framework for secure inference on secure CNNs , designed to reduce the number of HOPs required per inference whilst preserving prediction accuracy . Our contributions can be summarised as follows : • We integrate the convolution-packing method from LOLA with the fast matrix-vector product method introduced by Halevi and Shoup ( Halevi & Shoup , 2019 ) and utilised by Juvekar et al . ( 2018 ) in their multi-party computation framework . Intermediate convolutions are converted into fully-connected layers and computed as matrix-vector products . We show that utilising the Halevi-Shoup method allows the use of rotations and ciphertext packing to scale better compared with the representations in LOLA , when applied to larger convolutional layers . We perform a more detailed investigation on the scalability of the methods used in LOLA to larger models and show that they are significantly outperformed by our proposed method . • We compare our framework against LOLA by constructing models for MNIST and CIFAR10 . Our main evaluation criteria is the number of HOPs required per inference for a model . With the same layer parameters as LOLA , we are able to obtain over a two-fold reduction in the number of HOPs per inference . Our CIFAR-10 model achieves similar accuracy to that of LOLA ’ s but uses far fewer operations . 2 BACKGROUND AND PREREQUISITES . 2.1 THREAT MODEL . Our threat model concerns that of the machine learning as a service paradigm ( MLaaS ) , in which the user first sends data to a server , which then performs machine learning inference on the received data using some model . The inference result is then delivered back to the user . For example , consider an online machine learning service which claims to detect the probability of a person having COVID-19 from an audio recording of their cough . Suppose that Alice decides to send a recording of her cough to this service , in the hopes of receiving a diagnosis . There are two key threats in this scenario : ( i ) the risk of an adversary eavesdropping on the data transmission , and ( ii ) the risk of the MLaaS provider performing unauthorised access on the user ’ s data – in this case , the recording produced by Alice . The first threat can be mitigated using standard cryptographic protocols . However , the second risk is harder to address , especially if the user data is decrypted before inference Bae et al . ( 2018 ) . The use of HE mitigates both risks . The data is encrypted using HE , which is sufficient to prevent an adversary from eavesdropping . In addition , the provider is only able to perform computations on the encrypted data and will output the inference result without being able to decrypt . 2.2 HOMOMORPHIC OPERATIONS . Several recent HE schemes such as BFV ( Brakerski & Vaikuntanathan , 2011 ) and CKKS ( Cheon et al. , 2017 ) are based on the RLWE problem and support SIMD ciphertext operations . On a high level , such schemes establish a mapping between real vectors and a plaintext space . The plaintext space is usually the polynomial ring R = Z [ X ] / ( XN + 1 ) . In particular , this is a cyclotomic polynomial ring R = Z [ X ] / ( ΦM ( X ) ) where ΦM ( X ) is the M -th cyclotomic polynomial and M = 2N is a power of two . The decoding operation maps an element in R to a vector that is either real or complex , depending on the scheme used . The encoding operation performs the reverse . Plaintext polynomials are encrypted into ciphertext polynomials using a public key . The operations of addition and multiplication can be performed over ciphertexts using an evaluation key . Since each ciphertext corresponds to a vector of real ( or complex ) values , a single homomorphic operation between two ciphertexts constitutes an element-wise operation between two vectors . In addition , such schemes support rotations of the slots within a ciphertext , with the use of Galois automorphisms . 2.3 FLATTENED CONVOLUTIONS . Consider the convolution of an image I with a filter f . For simplicity , assume that both the image and filter are square , the vertical and horizontal strides of the filter are equal . Let I ∈ Rdin×din×cin , and f ∈ Rk×k×cin . Denote the stride as s and padding as p. Now , the output feature map J is such that J ∈ Rdout×dout where dout = ⌊ din − k + 2p s ⌋ + 1 . A full convolutional layer that outputs cout feature maps will require a convolution of the input image with each of the cout filters . Consider the vector v obtained by flattening each output feature map row-wise . This can be expressed as a matrix-vector product of the form v = A ·w ∈ Rd2out·cout where A ∈ Rd2out·cout×d2in·cin and w is the flattened representation of I . 2.4 FAST CONVOLUTION . The first convolutional layer in a CNN can be represented using convolution-packing ( Brutzkus et al. , 2019 ) . The convolution of an input image f with a filter g of width w , height h and depth d is ( f ∗ g ) [ i , j ] = w−1∑ x=0 h−1∑ y=0 d−1∑ z=0 g [ x , y , z ] f [ i+ x , j + y , z ] . ( 1 ) Observe that the parallelism inherent in this computation enables it to be vectorized as f ∗ g = w−1∑ x=0 h−1∑ y=0 d−1∑ z=0 g [ x , y , z ] · F ( x , y , z ) , ( 2 ) where F ( x , y , z ) is a matrix such that F ( x , y , z ) ij = f [ i+x , j+y , z ] . For an input image I ∈ Rcin×din×din feature maps and kernel of window size k × k , the input image is represented as k2 · cin vectors v1 , . . . , vk2·cin , where vi contains all elements convolved with the i-th value in the filter . Denote corresponding ciphertexts as ct1 , . . . , ctk2·cin . The process of producing the j-th output feature map is now reduced to ciphertext-plaintext multiplication of each cti with the i-th value in the j-th filter . In total , the process requires k2 ·cin ciphertext-scalar multiplications per output feature map , leading to a total of k2 · cin · cout multiplications . 2.5 MATRIX-VECTOR MULTIPLICATION . Multiplying a plaintext weight matrix of A ∈ Rm×n with a ciphertext vector can be achieved naively by first performing m ciphertext multiplications of the vector with each row of A . Then for each product ciphertext , we can compute the sum of all elements within it by applying a rotateand-sum procedure ( Halevi & Shoup , 2019 ) : we first rotate the ciphertext by N/2 slots and add it to the original . Then the procedure is repeated for N/4 slots , N/8 and so on , until the sum of the ciphertext resides within all slots of the ciphertext . The resulting dot products can be summed together . This basic approach requires O ( m log n ) rotations . Halevi & Shoup ( 2019 ) introduced a more efficient method of computing the encrypted matrixvector product A · v for square A. GAZELLE ( Juvekar et al. , 2018 ) extended the approach to support rectangular A ∈ Rm×n . The method works by decomposing A into its m diagonals , denoted { d1 , d2 , . . . , dm } such that di = [ Ai,0 , Ai+1,1 , . . . , Ai+n−1 , n−1 ] . Note that all row positions are in modulo m. Then each di is rotated i positions to align values belonging to the same row of A into the same column ( s ) , and finally each rotated diagonal is multiplied with with corresponding rotations of v. The ciphertexts are summed , and the last stage is to apply a rotate-and-sum procedure to the resulting ciphertext . Overall , this procedure requiresO ( m ) multiplications andO ( m+log2 n ) rotations . We propose an improved variant of this approach for our framework described in section 3.1 .

Summary: To accelerate the privacy-preserving inference through convolution neural networks (CNNs) with homomorphic encryption (HE), the authors aim to reduce the number of homomorphic operations (HOPs) required for the algorithm to save the data needs to be transferred while preserving prediction accuracy.  Using a LOLA method as a baseline, the authors used Halevi-Shoup (HS) which requires much fewer rotations operations in general. In the experimental results, the HS method requires half of the operations in comparison to the LOLA method. By further simplifying the structure of the network, only about 20% of HOPs are required. These simplifications bring about 1% and 27% accuracy loss.