## 4 HeFUN: An iLHE-Based SNNI Framework
### HeReLU: Homomorphic ReLU Evaluation Protocol  
After a linear layer, a non-linear activation function is applied to introduce non-linearity into the NN and allow it to capture complex patterns and relationships in the data. In this paper, we focus on the ReLU function. \[\text{ReLU}(x)= \begin{cases}x&\text{if }x>0\\ 0&\text{otherwise}\end{cases}  \tag{1}\]  
Evaluating ReLU requires the comparison operator, which is not natively supported by existing HE schemes. Hence, the ReLU activation function layer cannot be directly computed in the ciphertext domain. To address this issue, we propose a protocol, HeReLU, in which \(\mathcal{S}\) interacts with \(\mathcal{C}\) to perform ReLU computation. **Homomorphic ReLU evaluation problem. **\(\mathcal{C}\) holds secret key \(sk\), the server has public key \(pk\) but can not access to \(sk\). \(\mathcal{S}\) holds ciphertext \([\mathbf{x}]\) (\(\mathbf{x}\in\mathbb{R}^{n}\)) encrypted under \(sk\). \(\mathcal{S}\) obtains \([\text{ReLU}(\mathbf{x})]\) without leaking \(\mathbf{x}\) to \(\mathcal{C}\) and \(\mathcal{S}\). We now present our protocol that solves the homomorphic ReLU evaluation problem described above. Here, \(\text{ReLU}(\mathbf{x})\) means applying the ReLU function on each element of \(\mathbf{x}\), i.e., \(\text{ReLU}(\mathbf{x})=(\text{ReLU}(\mathbf{x}_{1}),\dots,\text{ReLU}(\mathbf{x}_{n}))\). Our protocol (HeReLU) is described in Protocol 1. We now give the intuition behind the protocol and then present the proof for the correctness and security of the protocol (as illustrated in Lemma 1). **Intuition. ** We based this on the fact that evaluating the ReLU function is essentially equivalent to evaluating the sign function. In particular, given \(\text{sign}(x)\), \(\text{ReLU}(x)\) can be implemented as follows:  
\[\text{ReLU}(x)=\frac{1}{2}(x+x.\text{sign}(x)) \tag{2}\]  
where \(\text{sign}(x)=1\) if \(x>0\), \(=0\) if \(x=0\), and \(=-1\) if \(x<0\). It is noteworthy that the sign function as defined in this paper is consistent with the standard interpretation found in the literature [9; 10; 63; 64]. This sign is widely employed in various real-world applications, including machine learning algorithms, such as support vector machines [65], cluster analysis [66], and gradient boosting [67]. Therefore, the fundamental question now is as follows: _How can \(\mathcal{S}\) (holds \([\mathbf{x}]\)) securely evaluate \([\text{sign}(\mathbf{x})]\)?_ We start from a trivial (non-secure) solution to evaluate \([\text{sign}(\mathbf{x})]\) as follows: \(\mathcal{S}\) sends \([\mathbf{x}]\) to \(\mathcal{C}\); \(\mathcal{C}\) decrypts the ciphertext (using \(sk\)) and obtains \(\mathbf{x}\), then obtains \(\mathbf{sign}(\mathbf{x})\); \(\mathcal{C}\) encrypts the result and sends the ciphertext \([\text{sign}(\mathbf{x})]\) to \(\mathcal{S}\). Obviously, this trivial solution reveals the entire \(\mathbf{x}\) to \(\mathcal{C}\). To hide \(\mathbf{x}\) from \(\mathcal{C}\), \(\mathcal{S}\) first homomorphically multiplies \([\mathbf{x}]\) with a random vector \(\mathbf{r}\
eq\mathbf{0}\) (line 2), then sends \([\mathbf{x}\times\mathbf{r}]\) to \(\mathcal{C}\) (line 3). After that, \(\mathcal{C}\) can decrypt \([\mathbf{x}\times\mathbf{r}]\) and obtain \(\mathbf{x}\times\mathbf{r}\), which hides \(\mathbf{x}\) from \(\mathcal{C}\) in an information-theoretic way (it is a one-time pad). At \(\mathcal{S}\)'s side, based on the sign of \(\mathbf{x}\times\mathbf{r}\) and sign of \(\mathbf{r}\), this allows \(\mathcal{S}\) to obtain the sign of \(\mathbf{x}\). In particular, as \(\mathbf{r}\
eq\mathbf{0}\), \(\text{sign}(\mathbf{x})=\text{sign}(\mathbf{x}\times\mathbf{r})\times\text{sign}(\mathbf{r})\) (illustrated in Table 2). Using \([\text{sign}(\mathbf{x})]\), \(\mathcal{C}\) can homomorphically compute \([\text{ReLU}(\mathbf{h})]\) based on Formula (2). The correctness and security of Protocol 1 is shown in Lemma 1. Proof. : We prove that the security of our protocol follows Definition 1. We first prove the correctness and then analyze the security of the protocol.