XONN: XNORbased Oblivious Deep Neural Network Inference Advancements in deep learning enable cloud servers to provide inferenceasaservice for clients. In this scenario, clients send their raw data to the server to run the deep learning model and send back the results. One standing challenge in this setting is to ensure the privacy of the clientsu0027 sensitive data. Oblivious inference is the task of running the neural network on the clientu0027s input without disclosing the input or the result to the server. This paper introduces XONN, a novel endtoend framework based on Yaou0027s Garbled Circuits (GC) protocol, that provides a paradigm shift in the conceptual and practical realization of oblivious inference. In XONN, the costly matrixmultiplication operations of the deep learning model are replaced with XNOR operations that are essentially free in GC. We further provide a novel algorithm that customizes the neural network such that the runtime of the GC protocol is minimized without sacrificing the inference accuracy.  :151,design a userfriendly highlevel API for XONN, allowing expression of the deep learning model architecture in an unprecedented level of abstraction. Extensive proofofconcept evaluation on various neural network architectures demonstrates that XONN outperforms prior art such as Gazelle (USENIX Securityu002718) by up to 7x, MiniONN (ACM CCSu002717) by 93x, and SecureML (IEEE Su0026Pu002717) by 37x. Stateoftheart frameworks require one round of interaction between the client and the server for each layer of the neural network, whereas, XONN requires a constant round of interactions for any number of layers in the model. XONN is first to perform oblivious inference on Fitnet architectures with up to 21 layers, suggesting a new level of scalability compared with stateoftheart. Moreover, we evaluate XONN on four datasets to perform privacypreserving medical diagnosis.

nGraphHE2: A HighThroughput Framework for Neural Network Inference on Encrypted Data. In previous work, Boemer et al. introduced nGraphHE, an extension to the Intel nGraph deep learning (DL) compiler, that enables data scientists to deploy models with popular frameworks such as TensorFlow and PyTorch with minimal code changes. However, the class of supported models was limited to relatively shallow networks with polynomial activations. Here, we introduce nGraphHE2, which extends nGraphHE to enable privacypreserving inference on standard, pretrained models using their native activation functions and number fields (typically real numbers). The proposed framework leverages the CKKS scheme, whose support for real numbers is friendly to data science, and a clientaided model using a twoparty approach to compute activation functions.  :107,first present CKKSspecific optimizations, enabling a 3x88x runtime speedup for scalar encoding, and doubling the throughput through a novel use of CKKS plaintext packing into complex numbers. Second, we optimize ciphertextplaintext addition and multiplication, yielding 2.6x4.2x runtime speedup. Third, we exploit two graphlevel optimizations: lazy rescaling and depthaware encoding, which allow us to significantly improve performance.  :164,these optimizations enable stateoftheart throughput of 1,998 imagess on the CryptoNets network. Using the clientaided model, we also present homomorphic evaluation of (to our knowledge) the largest network to date, namely, pretrained MobileNetV2 models on the ImageNet dataset, with 60.4percent82.7percent top1top5 accuracy and an amortized runtime of 381 msimage.