The MLaaS paradigm can be described in more detail as follows. A pre-trained neural network (NN), denoted by \(F\), is deployed on a remote cloud server. In this setup, a client transmits a data sample, denoted by \(x\), to the server for inference. After receiving \(x\), the cloud server performs neural network inference on \(x\) to obtain the output \(F(x)\), which will be returned to the user as the prediction result. Although MLaaS has clear benefits, it also raises serious privacy concerns for the client and the server. Firstly, from the client's perspective, the data \(x\) and the prediction result \(F(x)\) may contain sensitive personal information, e.g., in healthcare applications, data \(x\) can be a medical record and the prediction \(F(x)\) is whether the client has contracted a certain disease. Therefore, the client may hesitate to share \(x\) with the server and want to hide \(F(x)\) from the server. Secondly, from the server's perspective, the pre-trained NN, \(F\), represents the service provider's intellectual property and has great commercial advantages. Hence, the server does not want to share \(F\) with clients. Furthermore, the NN may leak information about the training data [8]. Consequently, it is very desirable and of great practical importance to design secure MLaaS frameworks, where the client should learn \(F(x)\) but nothing else about the server's model \(F\), while the server should not learn anything about the client's input \(x\) or the prediction result \(F(x)\). Throughout this paper, we refer to this goal as _secure neural network inference_ (SNNI) (other names sometimes used in the literature include _privacy-preserving inference_ and _oblivious inference_). Existing approaches to the SNNI problem naturally rely on privacy-preserving techniques of modern cryptography, in particular, multi-party computation (MPC) and fully homomorphic encryption (HE). In secure two-party computation (2 PC), a special case of MPC with two parties, the client and the server interact with each other to securely compute the prediction result without revealing any individual values. After three decades of theoretical and applied work improving and optimizing 2 PC protocols, we now have very efficient implementations, e.g., refs. [9; 10; 11; 12; 13]. The main advantage of 2 PC protocols is that they are computationally inexpensive and can evaluate arbitrary operators. However, 2 PC protocols require the structure (e.g., the boolean circuit) of the NN to be public and involve multiple rounds of interaction between the client and the server. On the other hand, HE-based SNNI is a non-interactive approach (the client and the server are not required to involve any communication during the inference process) and keeps all NNs' information secret from the client. The main bottlenecks of HE-based SNNI frameworks are their computational overhead and they cannot evaluate non-linear operators, e.g., the comparison operator. If prioritizing accuracy and inference time, 2 PC is the better candidate. For prioritizing security and optimizing communication, HE is the better candidate. Our motivation is to design a novel framework that achieves the best of both worlds: the security and low communication cost characteristic of the HE-based approach, coupled with the computational efficiency and ability to evaluate non-linear functions inherent in the 2 PC-based approach. In this paper, we focus on the HE-based approach to SNNI. HE is a cryptographic technique that allows computations to be performed on encrypted data without decrypting it. HE-based SNNI can be described as follows. The client encrypts \(x\) and sends the encrypted data \([x]\) to the server. The server evaluates \(F\) on \([x]\) and sends the result \(F([x])\) to the client, where \(F([x])=[F(x)]\) (homomorphic property). Finally, the client decrypts the ciphertext and obtains \(F(x)\). The main contribution of our paper is to propose a novel interactive approach for SNNI _solely_ based on HE that overcomes the aforementioned issues in 2 PC and HE.