CRYPTOGRU : Low Latency Privacy-Preserving Text Analysis With GRU (2021)

Abstract: Homomorphic encryption (HE) and garbled circuit (GC) provide the protection for users’ privacy. However, simply mixing the HE and GC in RNN models suffer from long inference latency due to slow activation functions. In this paper, we present a novel hybrid structure of HE and GC gated recurrent unit (GRU) network, , for low-latency secure inferences. replaces computationally expensive GC-based tanh with fast GC-based ReLU , and then quantizes sigmoid and ReLU to smaller bit-length to accelerate activations in a GRU. We evaluate with multiple GRU models trained on 4 public datasets. Experimental results show achieves top-notch accuracy and improves the secure inference latency by up to 138\times over one of the state-of-the-art secure networks on the Penn Treebank dataset.

## introduction billions of text analysis requests are processed by powerful rnn models @xcite deployed on public clouds everyday. these text analysis requests contain private emails, personal text messages, and sensitive online reviews. for instance, gmail smart reply generation needs to scan users' plaintext email messages anonymously @xcite . prior work @xcite proposes a hybrid cryptographic scheme that uses homomorphic encryption (he) to process linear layers and garbled circuits (gc) to compute activations in a convolutional neural network. compared to convolutional neural networks (cnn), rnns can achieve more competitive accuracy in text analysis tasks @xcite @xcite . mixing he and gc presents impressing results in secure classification tasks @xcite . however, mixing he and gc in rnn will suffer from a long inference latency due to the slow gc-based activations. in contrast to a cnn, a rnn @xcite requires more types of activations such as tanh and sigmoid. the gc protocol @xcite has to use a huge garbled table to implement a tanh or sigmoid activation. both garbling and evaluating such a large table add significant latency to rnn layers. based on our experimental and theoretical analysis, the gc-based activations can occupy up to 91% of the inference latency in a he and gc hybrid secure gru. to reduce the gc-based activation latency, we propose a novel secure gated recurrent unit (gru) network framework, cryptogru, that achieves high security level and low inference latency simultaneously. we use simd he kerel functions from @xcite to process linear operations in a gru cell, while it adopts gc to compute activations. our contributions are summarized as follows: prior studies @xcite show gru can the same level of inference accuracy as lstm. threat model and cryptographic primitives. we consider semi-honest corruptions @xcite @xcite in our threat model, where a server s is hosting a model and many clients c are sending inputs for inference using s' model. the client and the server adhere the protocol, but attempt to infer information about the other party's input. our protocol hides model weights, biases, and activations of a network model, which are likely to be proprietary. he @xcite is a cryptosystem that supports computation on ciphertext without decryption. gc enables two parties (sender and receiver) to jointly compute a function over their private data without revealing data beyond output from each other. a gc function is represented by a boolean circuit with 2-input gates (e.g., xor, and, etc.). the sender garbles the boolean circuit and generates the garbled table. the receiver receives the garbled table from an oblivious transfer @xcite and then evaluates the table. the total gc communication overhead is proportional to the number of non-xor gates in the garbling function @xcite . for instance, a 12-bit relu requires only 30 non-xor gates, while a 12-bit tanh @xmath0. comparison with prior privacy-preserving inference. prior studies create gc-only @xcite , he-only @xcite and he+gc hybrid @xcite privacy-preserving neural networks for secure inferences. gc-only secure networks have to pay huge communication overhead and long inference latency, whereas the he-only networks cannot accurately implement nonlinear activations by only homomorphic multiplications and additions. so, secure networks @xcite implement linear layers with he operations and nonlinear activations with gc operations. we compare cryptogru against prior related works in ## conclusion machine learning as a service attracts interest from many aspects in industry. public cloud companies already launched prediction services. however, 1 https://www.ics.uci.edu/~vpsaini/ sending plaintext to model servers for inference raise attentions to user privacy issues. we propose cryptogru, a secure inference building block for gated recurrent unit that emphasises on textlike or time series models. we elaborate all the improvements based on theoretical analysis and confirm the legitimacy for all optimization means. cryptogru improves a gru with homomorphic encryption, share secrets, and garbled circuits heterogeneously to achieve low latency as well as high accuracy.