# Secure Convolution Neural Network Inference Based on Homomorphic Encryption
## 4 Our Scheme
### Batch Processing in Inference Stage

Table 4: Every evaluation operation time costs and evaluation times. improvement ratio observed. We speculate that the larger numerical deviation observed under this parameter is likely due to the use of a larger learning rate, which leads to insufficient stability of the model when facing perturbed tests. However, it can be seen that our method still maintains a stable similarity of about 10 between the model input and the original output under this parameter. In the model containing three convolutional layers and two fully connected layers for inference of the CIFAR-10 dataset using the same activation function, the cosine similarity of the output result and the target result increased by about 20% on average (under different training parameters). After experimental analysis, the cosine similarity of adding matching noise in the training stage is significantly improved compared with adding noise in the training process. It can be said that this method should deal with the noise generated by homomorphic operations, which can bring the prediction result closer to the target result. Using the last row of parameters of the table, plus 0.1 momentum, training with added noise can make the ciphertext inference containing noise reach 98.55% accuracy, and under the same parameters, the training process can only achieve 98.12% accuracy without adding noise, which can show that the noise of the homomorphic scheme has an impact on the accuracy of the ciphertext inference results, and our method can indeed offset this effect to a certain extent. Next, we show the effect of the algorithm mentioned in Section 4.2 in a concrete experiment. In the case of \(N=\) 32,768, there are 16,384 slots available for plaintext, and our batch scheme prepares the convolution kernel area \(k^{2}\) plus the number of columns of the dense layer 1 column at the input end for the convolution window of each picture, that is, the output dimension \(wh\), a total of left \(\left(k^{2}+wh\right)\) plaintext slots, in the specific implementation, occupies left(\(4\times 4+100=116\)) plaintext slots. Theoretically, 16,384/116 = 141 pictures can be processed in parallel in the case of a single thread, and if you consider multi-threaded parallel processing, the pictures that can be processed synchronously depend on the performance of the actual environment, that is, the concurrency capacity of the CPU and the total amount of memory. We use the C++ standard thread library to perform 16-thread concurrent encrypted inference computations, and the entire process takes 27.15 s, (the time it took to contain the local client-side encrypted data), and this was obtained with the addition of some noise effect in the test set. Table 6 shows the time consumed by each stage of the ciphertext network inference process and the processing memory consumed by the runtime. Compared with the inference scheme containing the fourth-order approximation Swish activation function mentioned in [PERSON] et al. (2011), we consume about 4 s more time, but we choose polynomial modulos twice as large as it to provide higher security and deeper computational depth. We denote the output dimension of the last convolutional layer as c, and the size of the first dense layer is h \(\times\) w(c = h), Table 7 displays the inference time, accuracy, and conversion complexity between different layers of the scheme. \begin{table}
\begin{tabular}{c c c c c} \hline
**Epochs** & **Batch Size** & **Learning Rate** & **Cosine Similarity (Noisy)** & **Cosine Similarity (Original)** & **Promotion Ratio** \\ \hline
10 & 64 & \(1.0\times 10^{-3}\) & 10.0167 & 7.0016 & 43.06\% \\ \hline
10 & 64 & \(2.0\times 10^{-3}\) & 10.2742 & 4.9816 & 106.24\% \\ \hline
20 & 64 & \(1.0\times 10^{-4}\) & 23.3822 & 21.3689 & 9.42\% \\ \hline
10 & 128 & \(1.0\times 10^{-3}\) & 21.5057 & 21.3939 & 0.52\% \\ \hline
15 & 128 & \(8.0\times 10^{-4}\) & 17.4479 & 17.0862 & 2.12\% \\ \hline
12 & 64 & \(2.0\times 10^{-3}\) & 10.0247 & 6.0178 & 66.58\% \\ \hline \end{tabular}
\end{table}