## 4 HeFUN: An iLHE-Based SNNI Framework
### Hefun
* Activation function: It is the square function in CryptoNets, and the ReLU function in HeFUN. * Fully connected layer: It connects the 845 incoming nodes to 300 outgoing nodes. * Activation function: It is the square function in CryptoNets, and the ReLU function in HeFUN. * Fully connected layer: It connects the 300 incoming nodes to 100 outgoing nodes. * Activation function: It is the square function in CryptoNets, and the ReLU function in HeFUN. * Fully connected layer: connects the 100 incoming nodes to 10 outgoing nodes. * Activation function: It is the sigmoid activation function. In the case of the AT&T faces dataset, the neural network retains a similar layer structure as used with the MNIST dataset. However, adjustments are made to the neuron count per layer to accommodate the dataset's structure. For instance, the output layer features 40 neurons, corresponding to the 40 distinct classes represented in the dataset, in contrast to the 10-neuron configuration used for datasets like MNIST. As we mention in Section 4.1, the last sigmoid activation function can be removed in the inference phase without interfering with prediction accuracy. #### 5.3.3 HE Parameter

The main parameters defining the CKKS scheme [24] are the degree \(N\) of the polynomial modulus \(X^{N}+1\), the coefficient modulus \(q\), and multiplicative depth \(L\). The multiplicative depth \(L\) of the CKKS scheme was chosen to align with the NN's depth. Subsequently, the modulus \(q\) is carefully chosen to meet the security criterion of 128 bits, taking into account the specified multiplicative depth \(L\). Within the HeFUN framework, the HeRefresh protocol is employed to refresh the outputs of intermediate layers, thereby facilitating additional multiplications. Table 4 details the parameters. CryptoNets\({}_{s}\) (HeFUN\({}_{s}\)) and CryptoNets\({}_{l}\) (HeFUN\({}_{l}\)) stand for CryptoNets (HeFUN) on small NN and large NN, respectively. We choose \(N\) = 16,384 in both frameworks. Notably, in the case of CryptoNets, an escalation in NN depth from a small to a large model necessitates an increase in \(L\) (from 5 to 7), consequently requiring an increase in modulus \(q\) to preserve the desired security level. Conversely, HeFUN maintains a constant and reduced multiplicative depth regardless of NN depth increments--specifically 3 as per Table 4--thanks to its intermediate ciphertext refreshing mechanism. It is imperative to recognize that elevated values of \(L\) and \(q\) slow down inference time. #### 5.3.4 Experimental Results

Tables 5 and 6 present the experimental results on the MNIST and AT&T faces datasets, respectively, upon which we base our analysis of the accuracy and inference time associated with HeFUN and CryptoNets. are smaller than CryptoNets. For instance, in Table 5, with an increase in NN depth from 5 to 7, CryptoNets shows a significant jump in inference time--from 1.715 s to 3.497 s. This increase is due to the escalated complexity of LHE parameters necessitated by a deeper network. Conversely, HeFUN benefits from the HeRefresh protocol, which mitigates noise accumulation at intermediate layers and allows for consistent LHE parameters regardless of NN depth. Consequently, in HeFUN, a similar increase in NN depth results in a modest increase in inference time, from 1.374 s to 1.501 s. Interestingly, in HeFUN, the computation time at the same layers almost remains unchanged, regardless of the depth of NN, due to its invariable LHE parameter requirements. On the other hand, in CryptoNets, deeper NNs require an increase in the LHE's parameters, which slows down the computation time. For instance, the layer Fully connected 1 takes 0.812 s in both HeFUN\({}_{s}\) and HeFUN\({}_{l}\), while it takes 1.121 s in CryptoNet\({}_{s}\), and takes 2.138 s in CryptoNet\({}_{l}\). This observation reinforces the scalability of HeFUN in terms of computation time when compared to CryptoNets as NN complexity increases. We now analyze the running time at the activation function layers.