# Secure Convolution Neural Network Inference Based on Homomorphic Encryption
## 4 Our Scheme
### Batch Processing in Inference Stage

Table 5: The cosine similarity result of two training methods. Our scheme can compress the number of ciphertexts in the process of connecting the convolutional layer and the dense layer, transfer the transformation of the network structure to the movement of the ciphertext slot, effectively reduce the computational complexity of the convolutional layer or pooling layer when connecting the dense layer in the inference process, and reduce a certain amount of data traffic during use. ## 7 Discussion

The experiments have demonstrated the effectiveness of our batch processing algorithm, and our noise handling method has also achieved some results. Therefore, during the adversarial training process, the author suggests using different homomorphic encryption parameters to encrypt numbers within the input range of the model, observing the magnitude of the error introduced by the encryption process, and then using adversarial sample generation algorithms (such as FGSM) to generate adversarial samples with errors similar to those introduced by encryption. The algorithm parameters can be adjusted until the similarity between the two is high, after which adversarial and original data can be jointly input into the model for training. This method can also be used for differential privacy-based model protection schemes to stabilize model accuracy. In addition, for privacy-preserving inference in language models, techniques such as secure multiparty computation (MPC) and garbled circuits (GC) can be used to avoid the huge computation and storage costs brought by homomorphic encryption schemes. It is worth further exploring whether there are more post-quantum cryptography methods that can be applied to the complex computing problems in deep learning. ## 8 Conclusions

In this paper, a batch method to optimize the inference of homomorphic encryption neural network is proposed, which can make full use of the ciphertext slot provided by the homomorphic encryption scheme, and the inference time when batching 141 pictures is 27.15 s, and the amortization time of a single image is 0.19 s, with an accuracy of 99.05%. At the same time, a new perspective is provided to study how random noise added by encryption schemes can add noise to the training process to counteract its effects. In our scenario, users can encrypt private images locally, and then securely perform inference and prediction on the cloud service side, for which we provide proof of security. One limitation is that the application scenarios of homomorphic encryption are still not extensive enough, and need to be further studied and expanded. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Layer** & **Evaluation Time (s)** & **Memory Consumption (mb)** \\ \hline Local Pretreatment & 0.0875 & 1539 \\ \hline Convlutional Layer & 13.843 & 6018 \\ \hline Activation Layer-1 & 10.65 & 27,922 \\ \hline FC-1 & 2.425 & 2676 \\ \hline Activation Layer-2 & 0.07 & 17 \\ \hline FC-2 & 0.075 & 239 \\ \hline Total & 27.15 & 38,411 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Inference stage time and memory consumption. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Framework** & **HE Inference Time(s)** & **Accuracy** & **Conv-Dense Multiplication** \\ \hline Cryptonets & 249.6 & 98.95\% & \(h\times w\) \\ \hline CryptoDL & 148.9 & 98.1\% & \(h\times w\) \\ \hline FCryptonets & 39.1 & 98.71\% & \(h\times w\) \\ \hline Ours & 27.15 & 99.05\% & \(h\) \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Comparison of scheme parameters. **Author Contributions:** Conceptualization: C.S. ; methodology, C.S. ; validation, C.S., R.H.; formal analysis, C.S. ; writing--original draft preparation, C.S. ; writing--review and editing, C.S. ; funding acquisition, R.H. All authors have read and agreed to the published version of the manuscript. **Funding:** This research was funded by the National Natural Science Foundation Project of China under Grant No. 62062009 and the Guangxi Innovation-driven Development Project under Grant Nos. AA17204058-17 and AA18118047-7.