DEIM Forum 2023 5b-9-2

Performance Comparison of Homomorphic Encrypted Convolutional Neural
Network Inference between Microsoft SEAL and OpenFHE
Haoyun ZHU† , Takuya SUZUKI†† , Houtao HUANG†† , and Hayato YAMANA†††† School of Fundamental Science and Engineering, Waseda University,
3-4-1 Okubo, Shinjuku-ku, Tokyo, 169-8555 Japan†† Graduate School of Fundamental Science and Engineering, Waseda University,
3-4-1 Okubo, Shinjuku-ku, Tokyo, 169-8555 Japan††† Faculty of Science and Engineering, Waseda University, 3-4-1 Okubo, Shinjuku-ku, Tokyo, 169-8555 Japan
E-mail: †shuhaoyun@akane.waseda.jp, ††{t-suzuki,tao991,yamana}@yama.info.waseda.ac.jp
Abstract Homomorphic encryption (HE) is a promising way to preserve data privacy even during their calculation because
HE enables evaluation over ciphertexts. After the ﬁrst proposal of fully homomorphic encryption (FHE) enabling arbitrary
numbers of evaluation over ciphertexts by Craig Gentry in 2009, many HE libraries have appeared that we can use free ofcharge. Meanwhile, it is not easy to choose the best-ﬁt library for beginners of HE. This paper shows an insight for selecting
the library targeting Microsoft SEAL and OpenFHE, two popular HE libraries. We implemented a convolutional neural network (CNN) inference application with these two libraries to compare the inference latency and accuracy. Note that machinelearning methods over HE are one of the killer applications of HE. Our experimental results showed that the CNN w/ SEAL
had smaller latency, less than 50%, compared to the CNN w/ OpenFHE, while the accuracy was comparable between w/ SEAL
and w/ OpenFHE, but slightly better w/ SEAL. In terms of ease of use, OpenFHE is superior because programmers do not haveto consider the rescaling required during the calculation. In contrast, adopting automatic rescaling incurs 5x more latency.
Key words Privacy-preserving machine learning, Homomorphic encryption, Microsoft SEAL, OpenFHE
programmer-friendly new HE library is OpenFHE, a new open-

1 Introduction

source HE library released in July 2022. OpenFHE combines previ-

In 2009, Craig Gentry [1] ﬁrst proposed fully homomorphic en-

ously proposed various HE-related schemes, which have been col-cryption (FHE), enabling arbitrary numbers of evaluations (ad-laboratively constructed by over 26 HE-related research organiza-

ditions and multiplications) over ciphertexts.

After his pro-tions, with merging PALISADE, HElib, and HEAAN. Note that

posal, the number of papers related to homomorphic encryp-

OpenFHE focuses more on the usability of the scheme to promote ation (HE) is increasing every year, beneﬁtting from many kinds

wide range of HE usage because the theory of HE and its program-（注 1）

of HE libraries, including HElib（注 2）, TFHE（注 3）, PALISADE,HEAAN（注 4）, SEAL（注 5）, and OpenFHE（注 6）.

ming with HE is diﬀerent from usual programming, which limits
the wide range of usages.

Meanwhile, choosing the best-ﬁt HE library for beginners is

This paper compares two important aspects, i.e., latency and ac-

diﬃcult; thus, this paper tackles to provide insight into se-

curacy, when comparing the two HE libraries. In the comparison,lecting the library targeting Microsoft SEAL [2] and OpenFHE

we implement a convolutional neural network (CNN) inference ap-[3], which are two popular FHE libraries. Microsoft SEAL isplication to compare the libraries because machine learning is one

one of the most widely used HE libraries adopting various HEof the killer HE applications of HE. For example, machine learn-

schemes such as the Brakerski-Fan-Vercauteren (BFV) scheme

ing as a service (MLaaS) has been gaining more and more attention[4], the BrakerskiGentry-Vaikuntanathan (BGV) scheme [5], andfrom businesses as a series of services that provide machine learn-

the Cheon-KimKim-Song (CKKS) scheme [6], [7], [8].

ing tools as a part of cloud computing services. However, MLaaS

Anothercan be accompanied by signiﬁcant risks when using users

private（注 1）：https://github.com/homenc/HElib

data. To preserve the privacy of users（注 2）：https://github.com/tfhe/tfhehomomorphic encryption (HE) is one of the most reliable solutions.

（注 3）：https://palisade-crypto.org/（注 4）：https://github.com/snucrypto/HEAAN

private data, MLaaS with

With HE, a user can send the user s ciphertexts to a third party and（注 5）：https://github.com/microsoft/SEAL

allows the third party to run a machine learning model, such as a

（注 6）：https://github.com/openfheorg/openfhe-development

neural network, with the user s ciphertexts without revealing anyunderlying private information.

than the original CKKS scheme. Although the CKKS scheme is

In this paper, we give an insight into selecting a HE library for

an LHE-based-FHE scheme [7], we focus on the FullRNS-CKKS

readers when they want to adopt HE by comparing the latency andscheme without adopting bootstrapping because the multiplicative

the accuracy of CNN inference with SEAL and OpenFHE.

depth in homomorphic encrypted CNN inference is pre-deﬁned.

This paper is organized as follows. In Section 2, we explain the

2. 2 The FullRNS-CKKS Scheme

background of HE and HE libraries. Section 3 summarizes related

We explain the deﬁnition of words, HE parameters, and opera-

work comparing HE libraries and homomorphic encrypted CNN in-tions of the FullRNS-CKKS scheme. Here, we deﬁne pt as a plain-

ference. Section 4 shows our performance comparison method. In

text, ct as a ciphertext, and t as either a plaintext or ciphertext.

Section 5, we report and discuss the results of the experiments. Finally, we conclude this paper in Section 6.

2 Background2. 1 Overview of Homomorphic Encryption (HE)HE enables performing computations over ciphertexts without using a secret key. HE is classiﬁed into the following four types.
•

Partially HE (PHE): PHE can be further classiﬁed into two

types: additive HE which supporting only homomorphic addition;and multiplicative HE supporting only homomorphic multiplications.
••

Message: A message is represented as a vector of complex

numbers. The FullRNS-CKKS scheme supports packing [9] which
encodes or encrypts a vector into a plaintext or ciphertext. Each element in the vector is called slot, and the number of slots is calledslot counts.
•

Plaintext: Data encoded from a message.

•

Ciphertext: Data encrypted from a plaintext.

•

Scale scale(t): The bit length of the fraction part. The initialscale of a plaintext and ciphertext is called the scale factor ∆.
•Level level(t): The remaining number of applicable homo-morphic multiplication. If the level of a ciphertext is zero, no otherSomewhat HE (SHE): SHE supports several times of homo-morphic additions and a few times homomorphic multiplications on

homomorphic multiplication can be applied to the ciphertext.
•

Degree of polynomial rings N: The ﬁxed parameter in a ho-

a ciphertext; thus, SHE cannot evaluate homomorphic operations

momorphic encrypted operation. A plaintext or ciphertext t consists

with high multiplicative depth.

of one or more polynomial rings with modulus Qlevel(t) . In the RNS•

Leveled HE (LHE): LHE supports several homomorphic ad-

representation, a polynomial ring is divided into several polynomialditions and the pre-deﬁned times of homomorphic multiplications,rings with {qi }, where i ∈ [0, level(t)] and q j ≈ ∆ for j ∈ [1, level(t)]called level as an HE parameter. LHE can evaluate homomorphic

are satisﬁed in Microsoft SEAL and OpenFHE.

operations with higher multiplicative depth than SHE; however,•Size size(t): The number of polynomial rings in a plaintext

LHE still cannot evaluate homomorphic operations with undeﬁned

or ciphertext without the RNS representation. size(pt) = 1 andmultiplicative depth, such as training in machine learning. In addi-size(ct) >= 2 are satisﬁed.

tion, the higher the level is, the higher the computational cost and•Data structure of a plaintext and ciphertext: A 3D array ofthe larger the ciphertext size are; therefore, the level should be assize(t) × level(t) × N. Here, we deﬁne t[i][ j][k] as an element in t,

small as possible.

where 0 <= i < size(t), 0 <= j<= level(t), and 0 <= k < N are satisﬁed.

•

Fully HE (FHE): FHE can perform an arbitrary number of•

Number theoretic transform (NTT) form: In some homo-

both homomorphic additions and homomorphic multiplications on amorphic operations such as homomorphic multiplication, a nega-

ciphertext by applying bootstrap, which takes a long execution time.

cyclic convolution of two polynomial rings is performed. Because

FHE is constructed by adopting bootstrapping to SHE or LHE; i.e.,the computational cost of the convolution is high, NTT is applied to

if bootstrapping is not implemented or executed, the scheme can bea plaintext or ciphertext in normal form to convert it to NTT form

called SHE or LHE.

and reduce the computational cost of the multiplication from O(N 2 )One of the suitable HE schemes for homomorphic encrypted machine learning is the homomorphic encryption for the arithmetic
of approximate numbers (HEAAN) scheme, which is also known

to O(N log N). Note that a plaintext or ciphertext in NTT form canbe converted to that in normal form by applying inverse NTT.
•

Homomorphic addition HomAdd(ct, t): We deﬁne ctadd ,as the Cheon-Kim-Kim-Song (CKKS) scheme. The ﬁrst CKKS

which is a 3D array of max (size(ct), size(t))×min (level(ct), level(t))×

scheme was proposed by Cheon et al. in 2017 [6] as LHE. Be-

N, as a result ciphertext of HomAdd(ct, t) and assume size(ct) >=cause the CKKS scheme approximates ﬂoating-point numbers tosize(t).

ﬁxed-point numbers and adds noise into ciphertexts, the decryptiont[i][ j][k], and HomAdd(ct, t) also performs ctadd [i′ ][ j][k]HomAdd(ct, t) performs ctadd [i][ j][k] ← ct[i][ j][k] +←′result of a ciphertext is diﬀerent from the original plaintext. How-ct[i ][ j][k] if and only if size(ct) > size(t) is satisﬁed, where i ∈ever, the error is suﬃciently small by selecting HE parameters care-[0, size(t)), i′ ∈ [size(t), size(ct)), j ∈ [0, min(level(ct), level(t))], andfully; i.e., the error can be negligible. Furthermore, Cheon et al. [8]k ∈ [0, N) are satisﬁed. Note that scale(ct) and scale(t) should be

proposed the residue number system (RNS) variant of the CKKS

the same. In addition, in Microsoft SEAL, the inputs’ levels must

scheme (called FullRNS-CKKS scheme), which is more eﬃcient

be the same.

•

Homomorphic multiplication HomMul(ct, t): We deﬁneulus switching, relinearization, and bootstrapping are automaticallyctadd , which is a 3D array of (size(ct) + size(t) − 1) ×

invoked. With the compiler-friendly mode, an external compilermin (level(ct), level(t)) × N, as a result ciphertext of HomMul(ct, t).

makes the above decisions.

Here, we assume that the inputs’ are in NTT form and the inputs’

OpenFHE combines previously proposed various HE-related

sizes are at most two to reduce the computational cost. Then,

schemes, which have been collaboratively constructed by over 26

when t is a ciphertext, HomMul(ct, t) performs ctmul [0][ j][k] ←HE-related research organizations. OpenFHE supports multiple

ct[0][ j][k] × t[0][ j][k], ctmul [1][ j][k] ← ct[0][ j][k] × t[1][ j][k] +FHE schemes provided by PALISADE, advanced capabilities of thect[1][ j][k] × t[0][ j][k], and ctmul [2][ j][k] ← ct[1][ j][k] × t[1][ j][k],BGV scheme provided by HElib, and the FullRNS-CKKS scheme

where j ∈ [0, min(level(ct), level(t))] and k ∈ [0, N) are satis-

provided by HEAAN.

ﬁed. In addition, when t is a plaintext, HomMul(ct, t) performs
ctmul [i][ j][k] ← ct[i][ j][k] × t[0][ j][k], where i ∈ [0, size(ct)),j ∈ [0, min(level(ct), level(t))], and k ∈ [0, N). scale(ctmul ) is
scale(ct) × scale(t), and level(ctmul ) is min (level(ct), level(t)). Notethat in Microsoft SEAL, the inputs’ levels must be the same.
•

Relinearization Relin(ct): The larger the ciphertext’s size

2. 5 Comparison between Microsoft SEAL and OpenFHE in
the FullRNS-CKKS Scheme
2. 5. 1 Memory Allocation
Because the memory allocation and deallocation are executed
as kernel functions, their overhead is high.

One of the solu-

tions to reduce the overhead is pseudo-allocation and pseudo-is, the higher the required computational and spatial costs are.

deallocation.

The solution is relinearization which reduces the input’s size.

tor with such pseudo-allocation and pseudo-deallocation; however,size(Relin(ct)) is size(ct) − 1 if size(ct) > 2 is satisﬁed.

OpenFHE does not implement such a memory allocator. Therefore,•

Rescaling Rescl(ct): Rescaling reduces the input’s scale tokeep the bit length for decimal part suﬃcient. The level of therescaling output is level(ct) − 1.
•

Rotation Rotate(ct, s): Rotation permutes the slots of the in-put circularly with step s.

Microsoft SEAL implements the memory alloca-

Microsoft SEAL has less overhead in memory management than
OpenFHE.
2. 5. 2 Adjustment of Scale
When rescaling in the RNS representation, the ciphertext s
scale is divided by the last moduli of the ciphertext. In addi-

2. 3 Microsoft SEAL

tion, the scale of the ciphertext can be diﬀerent from the scale fac-

Microsoft SEAL [2], which is released by Microsoft Research, is

tor in the FullRNS-CKKS scheme because all the moduli are co-an open-source LHE library built in C++ or C#. Microsoft also re-prime and diﬀerent from the scale factor, even though some mod-leased EVA [10] which enables building an application with Python.

uli are almost the same as the scale factor. Therefore, the scales

Microsoft SEAL is one of the easiest HE libraries to understandof inputs of homomorphic addition can be diﬀerent; however, theand to use, i.e., user-friendly, because the API is simple. In addi-

scales should be the same, i.e., the scales should be adjusted. Mi-

tion, Microsoft SEAL implements a memory allocator to reduce thecrosoft SEAL does not adjust the scales, i.e., we have to adjust theoverhead of memory allocation and deallocation.

scales manually; however, OpenFHE adjusts the scales automati-

Meanwhile, Microsoft SEAL requires users to understand HE

cally with FLEXIBLEAUTO, which is one of the rescaling strategiess a basic concept to improve its performance because users must

in OpenFHE. Note that we can adjust the scales by ourselves with

add relinearization and rescaling operations manually. Although

FIXEDMANUAL in OpenFHE.

EVA adds relinearization and rescaling operations to suitable places,
EVA incurs overhead because EVA encodes messages at runtime,

3 Related Workeven though the encoding can be done in advance. Note that Mi-

We explain the research related to the comparison of HE libraries

crosoft SEAL supports not only the FullRNS-CKKS scheme but

in Section 3. 1. Then, Section 3. 2 explains homomorphic encryptedalso the BFV scheme and BGV schemes for modular operations on

CNN with the FullRNS-CKKS scheme.

encrypted integers.
2. 4 OpenFHE

3. 1 Comparison of Homomorphic Encryption Libraries
In 2022, Doan et al. [12] surveyed the implementation of HE

OpenFHE [11], released in July 2022, is a new open-source

schemes. Doan et al. compared the performance of several no-

FHE library. OpenFHE combines the design ideas of previous

table HE schemes covering PHE, SHE, and FHE. Doan et al. com-

FHE projects, such as PALISADE, HElib, and HEAAN, and adds

pared Microsoft SEAL and PALISADE with homomorphic addi-

new concepts and ideas. OpenFHE plans to support bootstrappingtion and homomorphic multiplication, followed by demonstrating

for all the implemented FHE schemes besides scheme-switching.

that Microsoft SEAL was faster than PALISADE. However, the re-

OpenFHE uses a standard hardware abstraction layer (HAL) to sup-sults shown in [12] have two problems: 1) Doan et al. used diﬀer-

port multiple hardware-accelerated backends. OpenFHE includes

ent parameters between the HE libraries in the comparison, and 2)

both a user-friendly mode and a compiler-friendly mode. With

Doan et al. did not demonstrate the performance of the homomor-

user-friendly mode, all the maintenance operations, such as mod-phic encrypted CNN inference. The latency of an actual application

would be diﬀerent from the latency calculated by the measurement

Table 1 CNN model for the MNIST dataset based on [17].

of primitive homomorphic operations; therefore, it is signiﬁcant toLayer typecompare the latency of an actual homomorphic encrypted CNN inference among HE libraries.
3. 2 Homomorphic Encrypted Convolutional Neural Network Inference with the CKKS Scheme

Convolution

Layer Size(2, 2) w/o padding
Batch normalization Applying batch normalization

12 × 12 × 5

Activation

Applying activation function

12 × 12 × 5

Convolution

50 ﬁlters of size 5 × 5 and stride

4 × 4 × 50

In 2019, Boemer et al. [13] proposed nGraph-HE, which is anextension of Intel’s DL graph compiler nGraph. nGraph-HE en-

Description

Five ﬁlters of size 5 × 5 and stride 12 × 12 × 5(2, 2) w/o padding
Batch normalization Applying batch normalization

ables models to be deployed with minimal code changes. In the

Activation

same year, Boemer et al. [14] proposed nGraph-HE2, the ﬁrst eval-

Fully connected

4 × 4 × 50

Applying activation function

4 × 4 × 50

Weighted sum of the entire previous

1 × 1 × 10

layers w/ 10 ﬁlters, each output cor-uation of a model with an encrypted ImageNet dataset, which

responding to one of the 10 classes

improves nGraph-HE. nGraph-HE2 enables privacy-preserving inference with batch-axis packing and several optimizations, including lazy rescaling to shorten the latency, to implement theing the library targeting Microsoft SEAL and OpenFHE, two pop-

FullRNS-CKKS scheme with Microsoft SEAL. As a result, nGraph-ular HE libraries. We implemented a convolutional neural network

HE2 achieved a 3–88x speedup in scalar encoding, a 2.6–4.2x(CNN) inference application with these two libraries to compare thespeedup in ciphertext-plaintext scalar addition, and a 2.6x speedup

inference latency and accuracy.

in ciphertext-plaintext scalar multiplication. Furthermore, nGraphHE2 shortens the latency by 8x on the CryptoNets network by lazy
rescaling.

4. 2 Datasets
In the experimental evaluation, we use the MNIST dataset [19]and the CIFAR-10 dataset [20], where the MNIST dataset and the

In 2019, Dathathri et al. [15] proposed CHET which is an op-

CIFAR-10 dataset are common datasets to evaluate CNN with HE.

timizing compiler for homomorphic encrypted neural network in-

The MNIST dataset consists of images of grayscale 28 × 28 pixels,

ference. To shorten the inference latency, CHET chooses HWeach of which is a single handwritten digit of 0–9, with the ground-(height-width) layout (channel-wise packing) or CHW (channel-

truth label. The MNIST dataset consists of 60,000 training data andheight-width) layout for packing an image to one or more cipher-

10,000 test data. The CIFAR-10 dataset consists of 32 × 32 pix-

texts. As a result, CHET achieved higher performance than the ex-

els RGB images, each of which is one of 10 diﬀerent classes, withpert hand-tuned. In 2020, Dathathri et al. [10] proposed EVA which

the ground-truth label. The CIFAR-10 dataset consists of 50,000

is an optimizing compiler and runtime for homomorphic encrypted

training data and 10,000 test data.

applications. EVA estimates the timing of relinearization and rescaling to shorten the latency; therefore, EVA is user-friendly. EVA
achieved a 5.3x speedup compared to CHET.
In 2020, Ishiyama et al. [16] proposed a method to improve
the accuracy of CNN inference with HE, which adopted batch-

4. 3 Description of the CNN Model
The conﬁguration of the CNN models for the MNIST dataset and
CIFAR-10 dataset are the same as those used in [17], [18]. Table 1
and Table 2 list the conﬁguration of the CNN models for the MNIST
dataset and CIFAR-10 dataset, respectively.

axis packing and diﬀerent activation functions. Ishiyama et al.

4. 4 Activation Function

achieved accuracies of 99.29% and 81.06% for MNIST and CIFAR-

Table 3 lists the details of the activation functions and non-HE in-

10 datasets,respectively. Furthermore, in 2022, Ishiyama et al.

ference accuracy reported in [17] [18]. Here, in the names of the[17], [18] proposed a tuning method of homomorphic encrypted

polynomial approximated activation functions, rg means a range

CNN inference, i.e., tuning both latency and accuracy, by chang-of approximation, and deg means the approximation degree. The

ing the channel pruning ratio and activation functions. As a result,

range aﬀects only the accuracy; however, the approximation degree

Ishiyama et al. successfully tuned the latency from 8.1–12.9 s de-aﬀects not only the accuracy but also the latency. Therefore, we

pending on the accuracy of 66.52–80.96% on the CIFAR-10 dataset.

select square, swish-rg5-deg2, and swish-rg7-deg4 as the activation

Ishiyama et al. implemented these methods with Microsoft SEAL.

functions in our experimental evaluation because we use the same
trained CNN models used in [17], [18]. Note that we do not adopt

4 Comparison Method of Homomorphic Encryption Libraries
4. 1 Overview

channel pruning technique proposed in [17], [18].
4. 5 Experiment Environment and Condition
We use the source code provided by Ishiyama et al. [18]（注 7） to

The homomorphic encrypted CNN inference has been acceler-

execute the homomorphic encrypted CNN inference. Althoughated with several optimizations; however, the latency also depends

we adopt Microsoft SEAL 4.0.0 instead of Microsoft SEAL 3.6.6

on the implementation. Our goal is to choose the best-ﬁt library for
beginners of HE. To achieve our goal, we show an insight for select-（注 7）：https://github.com/yamanalab/latency-aware-cnn-inferenceTable 2 CNN network model for CIFAR-10 (quoted from [17], [18])Convolution

Description
32 ﬁlters of size 3 × 3 × 3
and stride (1, 1) with padding.
Applying batch normalization

16 × 16 × 32

Activation

Applying activation function

32 × 32 × 32

Average poolingw/ extent 2 and stride 2

16 × 16 × 64

64 ﬁlters of size 3 × 3 × 32
and stride (1, 1) with padding

16 × 16 × 64

20
15
10

Applying batch normalization

16 × 16 × 64

Activation

Applying activation function

16 × 16 × 64

5

Average poolingw/ extent 2 and stride 2

8 × 8 × 64

0and stride (1, 1) with padding

Batch normalization

Applying batch normalization

8 × 8 × 128

Activation

Applying activation function

8 × 8 × 128

Average poolingw/ extent 2 and stride 2

4 × 4 × 128

Global average pooing Applying global average pooling
Fully connected

Weighted sum of the entire
previous layer with 10 ﬁlters

1

8 × 8 × 128

600

1 × 1 × 10

version up. However, we modiﬁed the source code for OpenFHE,
i.e use the operation methods in OpenFHE instead of SEAL. We set
CMAKE BUILD TYPE to Release mode in CMake to build Microsoft
SEAL, OpenFHE, and our source code.
Based on the methods of Ishiyama et al. [17], [18], we adopt the
optimization to merge coeﬃcients of a pair of layers to reduce the36

72

Figure 1 Latency v.s. the number of threads with the MNIST dataset.

1 × 1 × 256

which Ishiyama et al. used, we do not change any codes for the18#Threads

Latency[sec]Convolution

SEAL swish rg7 deg4
OpenFHE square
OpenFHE swish rg5 deg2
OpenFHE swish rg7 deg4

25

Batch normalization

128 ﬁlters of size 3 × 3 × 64

SEAL square
SEAL swish rg5 deg2

30

32 × 32 × 32

Batch normalization

Convolution

35

Layer Size

Latency[sec]Layer type

SEAL square

500

SEAL swish rg5 deg2
SEAL swish rg7 deg4

400

OpenFHE square
OpenFHE swish rg5 deg2
OpenFHE swish rg7 deg4

300
200
100
0
1

18

36

72

#Threads

level consumption. We merge a pair of the convolutional layer andthe batch normalization layer and a pair of the activation function
layer and the average pooling layer. Furthermore, if the absolute

Figure 2 Latency v.s. the number of threads with the CIFAR-10 dataset.

value of a coeﬃcient w of a layer is less than the threshold ε, thecoeﬃcient is rounded; i.e., we set w to ε when 0.0 <= w < ε or set w

tion incurs lower overhead than ﬁne-grained parallelization.

to −ε when −ε < w < 0.0, where the threshold ε is set to 10−6 for

To obtain the average latency, we execute 72 times inference, fol-swish-rg7-deg4 and is set to 10−7 for square and swish-rg5-deg2.

lowed by averaging them for each parameter set and dataset. In

Table 4 lists the speciﬁcation of the server used in the evaluation.
In addition, we conﬁgure the HE parameters as shown in Table 5.
In OpenFHE, we adopt FLEXIBLEAUTO for the rescaling strategy.
4. 6 Experiment Setup

addition, to obtain the average accuracy, we execute 1,000 times inference using 72 threads with SEAL and 18 threads with OpenFHE,
followed by averaging.

5 Result and Discussion

We adopt parallelization among homomorphic operations using
OpenMP with static scheduling to shorten the latency, where thestatic scheduling decides the chunk size before executing the forloops so that the overhead by chunking is smaller than the otherscheduling mode. The number of threads we used is 1, 18, 36, or
72. To evaluate scalability, we use only NUMA-node0 for 1 or 18
threads, use only NUMA-node0 and NUMA-node1 for 36 threads
and use all for 72 threads by specifying the NUMA nodes using
the numactl command. Although OpenFHE supports parallelization in a homomorphic operation, i.e., ﬁne-grained parallelization,
we adopt parallelization only among homomorphic operations, i.e.,coarse-grained parallelization, because coarse-grained paralleliza-

Table 6 shows the latency and accuracy of the MNIST and
CIFAR-10 datasets. Besides, Fig. 1 and 2 illustrate the latency
versus the number of threads.
Table 7 compares the latency of primitive operations in Microsoft SEAL and OpenFHE under the same conditions. Table 8
and 9 compare the latencies of multiplication and activation functions with FIXEDMANUAL and FLEXI- BLEAUTO mode with
OpenFHE, respectively.
5. 1 Latency Comparison
Table 6 shows that OpenFHE was almost twice slower as Microsoft SEAL with one thread, three times slower with 18 threads,
four times slower with 36 threads, and 8 times slower with 72

Table 3 Details of activation functions and inference accuracy over plaintext with MNIST dataset and
CIFAR-10 dataset reported in Table 5.3, Table 5.5, and Table 5.6 in [17]Activation

Range for Appoximation

function namex-axisdegreesquare

N/A

N/A

swish-rg5-deg2[−5, 5]2

swish-rg7-deg4[−7, 7]0.1−0.001328x4

4Intel Xeon E7-8880 v3

Frequency (base)2.30 GHz

Frequency (turbo)

3.10 GHz#cores/CPU

Disabled

L1i cache capacity/core

32 KiBL1d cache capacity/core

32 KiBL2 cache capacity/core

256 KiB

L3 cache capacity/CPU

45 MiB
4

Capacity

3 TB#NUMA nodes

OpenFHE

1.0.0

g++

7.4.0

OpenMP

4.5

Eigen

3.4.0

Function

Level polynomial
rings N

MNIST

0.6
0.5
0.4

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

Time[sec]
OpenFHE_square

OpenFHE_swish_reg5_deg2

OpenFHE_swish_reg7_deg4

SEAL_square

SEAL_swish_reg5_deg2

SEAL_swish_reg7_deg4

Figure 3 Memory allocation transition w/ SEAL and OpenFHE (MNIST
dataset)square5swish-rg5-deg2

5swish-rg7-deg4

7square8

Slotcounts

11
10
9
8
7
6
5
4
6
13
20
27
34
41
48
55
62
69
76
83
90
97
104
111
118
125
132
139
146
153
160
167
174
181
188
195
202
209
216
223
230
237

4.0.0

Table 5 HE Parameters
Degree ofDataset

80.91

0.2

CentOS 7.6.1810

Microsoft SEAL

Compiler & API

99.52+ 0.5x + 0.1773

4

OS
Library

79.53

0.3

%memory

Memory

#CPUs

79.17

99.48

0.7

18

Hyper-threading

99.35

x2 + 0.5x + 0.24

0.8

%memory

Model number

MNIST CIFAR-10x2+ 0.128x2

Table 4 The server speciﬁcation and software versions for the experiment.

CPU

Accuracy[%]

FunctionTime[sec]
OpenFHE_square

OpenFHE_swish_reg5_deg2

OpenFHE_swish_reg7_deg4

SEAL_square

SEAL_swish_reg5_deg2

SEAL_swish_reg7_deg4

Scalefactor∆Figure 4 Memory allocation transition w/ SEAL and OpenFHE (CIFAR10 dataset)

we measured the transition of the physical memory consumption.
16,384

8,192

230

We recorded the memory consumption during the ﬁrst 15 seconds

CIFAR-10 swish-rg5-deg2

11

and 4 minutes after initiating the second inference of MNIST andswish-rg7-deg4

11

CIFAR-10 datasets, respectively. Note that we measured the second
inference s memory consumption to eliminate uncertain behaviors

threads. We estimate the following two reasons.

usually occurring at the ﬁrst inference because of the cache mech-

The ﬁrst reason is the diﬀerent latencies in primitive operations

anism. Figs. 3 and 4 show the transitions of memory consumption

between Microsoft SEAL and OpenFHE. Table 7 shows the latency

with diﬀerent activation functions in 72 threads. As shown in Figs. 3

of each primitive operation in the FullRNS-CKKS scheme with Mi-

and 4 , Microsoft SEAL keeps memory consumption at the same

crosoft SEAL and OpenFHE, where we tested 5 times executions

level, while OpenFHE changes the memory consumption dynami-

using sample codes to calculate the average latency under the same

cally. From these results, it is clear that since Microsoft SEAL has

conditions and parameters. As shown in Table 7, the operation inbuilt-in memory management, Microsoft SEAL does not release the

SEAL is faster than in OpenFHE. Note that, unlike OpendFHE, Mi-

allocated memory to reuse, which results in less overhead comparedcrosoft SEAL does not support direct computation with integers, toto OpenFHE.

compute, you need to change an integer into plaintext.

5. 1. 1 Latency v.s. the Number of Threads

The second reason is that OpenFHE needs more kernel function

As shown in Fig. 1 and 2, using 72 threads achieved the lowest

calls to allocate and deallocate memory than SEAL, which may re-latency in Microsoft SEAL; however, using 18 threads achieved thesult in a bottleneck of OpenFHE. To conﬁrm the memory consump-lowest latency in the MNIST dataset using OpenFHE, 18 threads

tion in the inference process for MNIST and CIFAR-10 datasets,and 36 threads in OpenFHE achieved the lowest latency with thesquare function and the swish function, respectively in the CIFAR-

Table 6 The result w/ MNIST dataset and CIFAR-10 dataset (FIXEDMANUAL mode in OpenFHE)
Activation
function

MNIST dataset#threads

squareswish-rg5-deg2

swish-rg7-deg4

Latency [sec]

Level

5

MNIST

7

8

Cifar-10

11

Operation

Latency [sec]Accuracy [%]SEAL OpenFHE SEAL OpenFHE SEAL OpenFHE SEAL OpenFHE
1

11.15

23.07

201.1

374.76

18

1.20

4.28

20.1

68.01

36

1.24

6.55

23.6

85.94

99.2

99.2

72

0.64

9.38

9.6

125.19

1

11.08

22.91

200.2

374.37

18

1.20

4.10

20.2

65.52

36

1.24

6.63

23.6

88.77

99.3

99.3

72

0.65

8.81

9.6

145.83

1

15.01

29.40

249.5

477.46

18

1.64

5.40

28.1

82.66

36

1.74

8.17

32.3

101.20

72

0.74

13.77

13.3

181.65

99.4

Table 7 Latency of primitive operations
DatasetCIFAR-10 dataset

Accuracy [%]

99.4

80.3

78.8

80.3

80.1

81.4

81.5

Table 8 Multiplication latency in FIXEDMANUAL mode and FLEXIBLEAUTO

Latency[ms]SEAL

OpenFHE

Addition

0.465

4.059

Multiplication

6.181

15.298

Rotation

31.450455.506

Rescaling

6.763

28.802

Relinearization

0.030

0.023

Addition

0.379

1.922

mode w/ OpenFHE
Dataset
MNISTCifar-10

Level

Latency[sec]FLEXIBLEAUTO FLEXIBLEAUTO

5

0.308

0.795

7

0.404

0.670

8

0.427

0.729

11

0.566

0.575

Multiplication

5.344

6.644

Rotation

36.095102.314

Rescaling

5.794

96.606

Relinearization

0.004

0.076

Addition

0.639

4.251

1

23.07

132.28Multiplication

6.969

7.329

18

4.28

10.26

Rotation

53.200128.671

36

6.55

8.86

Rescaling

8.612

26.781

72

9.38

9.66

Relinearization

0.015

0.031

1

22.91

134.36

Addition

2.556

2.417

18

4.10

10.18

Multiplication

16.192

7.798

36

6.63

7.80

Rotation

127.979160.022

72

8.81

9.22

Rescaling

17.55750.6831

29.40

116.75

Relinearization

0.047

0.080

18

5.40

9.98

36

8.17

9.04

72

13.77

13.75

Table 9 Activation function latency in FIXEDMANUAL mode and
FLEXIBLEAUTO mode w/ OpenFHE
Activation function #threads

Squareswish-rg5-deg2

swish-rg7-deg4

10 dataset. As we mentioned in our experimental setup (Section

Latency[sec]FIXEDMANUAL FLEXIBLEAUTO

4.6), we used fewer NUMA nodes on the larger number of threads.
The reason why using more threads results in longer latency withFLEXIBLEAUTO in this case. The reason is that FLEXIBLEAUTO is

OpenFHE is that the latency reduced by the increment of threads

designed so that OpenFHE automatically applies rescaling just be-does not oﬀset the latency increased by the memory allocation andfore homomorphic multiplication, except for the ﬁrst homomorphic

deallocation due to the lack of memory management capability ofmultiplication. Speciﬁcally, we measured the multiplication latency

OpenFHE.

in FIXEDMANUAL and FLEXIBLEAUTO mode with OpenFHE

5. 1. 2 Mode comparison in OpenFHE

by executing a multiplication 5 times to have Table 8. As shown in

Table 9 shows the diﬀerence in the latency between FIXEDMANUAL

Table 8, FLEXIBLEAUTO consumes more time to apply automaticand FLEXIBLEAUTO in OpenFHE, where FLEXIBLEAUTO is eas-

rescaling than FIXEDMANUAL.

ier to use than FIXEDMANUAL because rescaling is performed au-

5. 2 Comparison of the Accuracy

tomatically by OpenFHE with FLEXIBLEAUTO. As shown in Ta-

As shown in Table 6, in the inferences of 1,000 images, OpenFHE

ble 9, FIXEDMANUAL performs better, i.e., smaller latency, thanshows the same accuracy as Microsoft SEAL w/ the MNIST dataset,Table 10 Accuracy w/ Microsoft SEAL, FIXEDMANUAL mode and
FLEXIBLEAUTO mode
Activation
function

Accuracy
Microsoft

OpenFHESEAL

OpenFHEFIXEDMANUAL FLEXIBLEAUTO

square

78%

77%

78%

swish-rg5-deg2

81%

78%

81%

swish-rg7-deg4

83%

83%

83%

and lower accuracy in the CIFAR-10 dataset except for the swishrg7-deg4. Although we used the same computational process in
SEAL and OpenFHE, the FIXEDMANUAL mode in OpenFHE will incur an additional precision loss of around 3-4 bits. Besides, Table 10compares the accuracy between Microsoft SEAL and FIXEDMANUAL
mode and FLEXIBLEAUTO mode in OpenFHE in 100 inferences. As
shown in Table 10, OpenFHE s FLEXIBLEAUTO mode achieves thesame precisions as SEAL. This fact implies the existence of anotherreason that our setting in FIXEDMANUAL mode is not the best solution in OpenFHE, even if the setting is the same as Microsoft SEAL.

6 Conclusion
This paper focused on the diﬀerences between two homomorphic
encryption libraries, Microsoft SEAL and OpenFHE, to reveal theessential concepts and implementations to improve the performanceof homomorphic encryption CNN inference. In our experimental
results, Microsoft SEAL performs better than OpenFHE in termsof latency because Microsoft SEAL has built-in memory management. OpenFHE shows the same accuracy as Microsoft SEAL in
the MNIST dataset and lower accuracy in the Cifar-10 dataset except for the swish-rg7-deg4.
OpenFHE is capable of automatically scaling numbers and supporting direct calculations with integers. Using SEAL is the best
choice for programs customized by FHE experts or compilers for
speciﬁc applications. For beginners of FHE or people who want
to use FHE without suﬃcient knowledge, OpenFHE is a better approach even though it incurs 10x longer latency than SEAL.
One of the prospects is to apply memory management in
OpenFHE and evaluate its performance of the homomorphic encrypted CNN inference. More importantly, we also need to evaluate
the performance of OpenFHE and Microsoft SEAL using diﬀerent
CNN models or other applications.

References[1] C Gentry. A fully homomorphic encryption scheme. phd thesis, stanford university, 2009. 2009.
[2] Microsoft SEAL (release 4.0). https://github.com/Microsoft/
SEAL, March 2022. Microsoft Research, Redmond, WA.
[3] Ahmad Al Badawi, Jack Bates, Flavio Bergamaschi, David Bruce
Cousins, Saroja Erabelli, Nicholas Genise, Shai Halevi, Hamish
Hunt, Andrey Kim, Yongwoo Lee, et al. Openfhe: Open-source
fully homomorphic encryption library. In Proceedings of the 10th
Workshop on Encrypted Computing & Applied Homomorphic Cryptography, pages pp.53–63, 2022.

[4] Junfeng Fan and Frederik Vercauteren. Somewhat practical fully homomorphic encryption. IACR Cryptol. ePrint Arch. 2012:144, 2012.
[5] Zvika Brakerski, Craig Gentry, and Vinod Vaikuntanathan. (leveled)fully homomorphic encryption without bootstrapping. ACM Transactions on Computation Theory (TOCT), 6(3):pp.1–36, 2014.
[6] Jung Hee Cheon, Andrey Kim, Miran Kim, and Yongsoo Song.
Homomorphic encryption for arithmetic of approximate numbers.
In Proceedings of the Advances in Cryptology – ASIACRYPT
2017,LNCS,vol.10624,pp.409–437. Springer, 2017.
[7] Jung Hee Cheon, Kyoohyung Han, Andrey Kim, Miran Kim, andYongsoo Song. Bootstrapping for approximate homomorphic encryption. In Proceedings of the Advances in Cryptology – EUROCRYPT 2018, LNCS, vol. 10820, pp. 360–384. Springer, 2018.
[8] Jung Hee Cheon, Kyoohyung Han, Andrey Kim, Miran Kim, andYongsoo Song. A full rns variant of approximate homomorphic encryption. In Proceedings of the Selected Areas in Cryptography –SAC 2018, LNCS, vol.11349, pp. 347–368. Springer, 2019.
[9] Nigel P Smart and Frederik Vercauteren. Fully homomorphic simd
operations. Designs, codes and cryptography, 71(1):pp.57–81, 2014.
[10] Roshan Dathathri, Blagovesta Kostova, Olli Saarikivi, Wei Dai, Kim
Laine, and Madan Musuvathi. Eva: An encrypted vector arithmetic
language and compiler for eﬃcient homomorphic computation. In
Proceedings of the 41st ACM SIGPLAN Conference on Programming
Language Design and Implementation, pages pp.546–561, 2020.
[11] Openfhe brings new encryption tools to developers.
https://
www.darkreading.com/dr-tech/openfhe-brings-new-encryption-toolsto-developers, 2022.
[12] Thi Van Thao Doan, Mohamed-Lamine Messai, Gérald Gavin, andJérôme Darmon. A survey on implementations of homomorphic encryption schemes, 2022. Preprint, doi: 10.21203/rs.3.rs-2018739/v2.
[13] Fabian Boemer, Yixing Lao, Rosario Cammarota, and Casimir
Wierzynski. Ngraph-he: a graph compiler for deep learning on homomorphically encrypted data. In Proceedings of the 16th ACM
International Conference on Computing Frontiers, pages pp.3–13,
2019.
[14] Fabian Boemer, Anamaria Costache, Rosario Cammarota, andCasimir Wierzynski. Ngraph-he2: A high-throughput framework forneural network inference on encrypted data. In Proceedings of the 7th
ACM Workshop on Encrypted Computing & Applied Homomorphic
Cryptography, pages pp.45–56, 2019.
[15] Roshan Dathathri, Olli Saarikivi, Hao Chen, Kim Laine, Kristin
Lauter, Saeed Maleki, Madanlal Musuvathi, and Todd Mytkowicz.
Chet: an optimizing compiler for fully-homomorphic neural-network
inferencing. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation, pages
pp.142–156, 2019.
[16] Takumi Ishiyama, Takuya Suzuki, and Hayato Yamana. Highly accurate cnn inference using approximate activation functions over homomorphic encryption. In Proceedings of the 2020 IEEE International
Conference on Big Data (Big Data), pages pp.3989–3995. IEEE,
2020.
[17] 石山琢己, 鈴木拓也, and 山名早人. 準同型暗号上での畳み込み
ニューラルネットワーク推論に対する channel pruning の適用.
In 第 14 回データ工学と情報マネジメントに関するフォーラム(DEIM2022), number J33-4, pages pp.1–8, 2022.
[18] Takumi Ishiyama, Takuya Suzuki, and Hayato Yamana. Latencyaware inference on convolutional neural network over homomorphic
encryption. In Proceedings of the 24th International Conference on
Information Integration and Web Intelligence (iiWAS 2022), LNCS,
vol.13635, pp.324–337.
[19] Yann Lecun, Leon. Bottou, Yoshua. Bengio, and Patrick. Haﬀner.
Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):pp.2278–2324, 1998.
[20] Alex Krizhevsky. Learning multiple layers of features from tiny images, 2009. https://www.cs.toronto.edu/˜kriz/learningfeatures-2009-TR.pdf.