## 4 HeFUN: An iLHE-Based SNNI Framework
### Hefun
For a depth of 4, with the same \(N\), HeRefresh is nearly 225 times faster than bootstrapping. ### Comparison between HeFUN and LHE (CryptoNets)

This section aims to compare HeFUN with the LHE-based approaches. The foundational concept for LHE-based approaches originates from the landmark study presented in CryptoNets [12], which utilizes an LHE scheme with a predetermined depth that aligns with the NN's architecture. For the purposes of this analysis, the term 'CryptoNets' will be used to refer to the conventional LHE-based approach. #### 5.3.1 Experimental Setup

The NNs in this section were trained using PyTorch [71]. To implement HE, we employed TenSEAL [60], with CKKS as instantiation. All parameters in the following sections are chosen to comply with the recommendations on HE standards [72], which satisfies 128 bits of security. #### 5.3.2 Dataset and NN Setup

We evaluated the performance of NN inference on two distinct datasets, i.e., MNIST [73] and AT&T faces datasets [74]. The MNIST dataset has a standard split containing \(28\times 28\) grayscale images of Arabic numerals 0 to 9 of 50,000 training images and 10,000 test images. MNIST is the standard benchmark for homomorphic inference tasks [11; 12; 25; 33]. The AT&T faces dataset includes \(92\times 112\) grayscale images of 40 individuals, with 10 different images of each individual. The dataset is considered a classic dataset in computer vision for experimenting with techniques for face recognition. It offers a more realistic scenario for SNNI, allowing for the recognition of faces while maintaining the confidentiality of the individual images. The neural networks (NNs) under consideration are composed of convolutional layers, activation functions, and fully connected layers. For the HeFUN framework, the activation function employed is the ReLU function, while LHE-based secure neural network inference (SNNI) typically utilizes the square function as the activation mechanism, as extensively documented in the literature [12; 25; 26]. To introduce diversity into our experimental evaluation, we have selected two distinct NN architectures, i.e., a small NN and a large NN (the term \"large NN\" refers to an NN with more complexity than our \"small NN\", but its size is tailored to stay within the operational limits of LHE-based frameworks for meaningful comparisons). The details of the NNs' architectures for the MNIST dataset are detailed below. * Small NN: \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Depth**} & \multicolumn{2}{c}{**Runtime (s)**} & \multicolumn{2}{c}{**N**} & \multicolumn{2}{c}{**Level**} \\ \cline{2-7}  & **DoubleR-CKKS** & **Bootstrapping** & **DoubleR-CKKS** & **Bootstrapping** & **DoubleR-CKKS** & **Bootstrapping** \\ \hline
1 & 0.04 & 11.85 & 16,384 & 65,536 & 1 & 22 \\
2 & 0.05 & 12.71 & 16,384 & 65,536 & 2 & 23 \\
4 & 0.07 & 14.69 & 16,384 & 65,536 & 4 & 25 \\
8 & 0.10 & 17.39 & 32,768 & 65,536 & 8 & 29 \\
16 & 0.19 & 22.51 & 65,536 & 65,536 & 16 & 37 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparison between Double-CKKS protocol and bootstrapping. * Convolution layer: The input image is 28 \(\times\) 28. 5 kernels, each 3 \(\times\) 3 in size, with a stride of 2 and no padding. The output is a 5 \(\times\) 13 \(\times\) 13 tensor. * Activation function: This layer applies the approximate activation function to each input value. It is a square function in CryptoNets, and the ReLU function in HeFUN. * Fully connected layer: It connects the 845 incoming nodes to 100 outgoing nodes. * Activation function: It is the square function in CryptoNets, and the ReLU function in HeFUN. * Fully connected layer: It connects the 100 incoming nodes to 10 outgoing nodes (corresponding to 10 classes in the MNIST dataset). * Activation function: It is the sigmoid activation function. * Large NN:
* Convolution layer: It contains 5 kernels, each 33 in size, with a stride of 2 and no padding. The output is a 5 \(\times\) 13 \(\times\) 13 tensor.