Protecting Personal Information using
Homomorphic Encryption
for Person Re-identification
Kazunari Morita1 , Hiroki Yoshimura1,2 , Masashi Nishiyama1,2 and Yoshio Iwai1,212

Graduate School of Engineering, Tottori University, Japan
Cross-informatics Research Center, Tottori University, Japan
Email: nishiyama@tottori-u.ac.jp

Abstract—We investigate how to protect features corresponding to personal information using homomorphic encryption
when matching people in several camera views. Homomorphic
encryption can compute a distance between features without
decryption. Thus, our method is able to use a computing server ona public network while protecting personal information. To apply
homomorphic encryption, our method uses linear quantization torepresent each element of the feature as integers. Experimental
results show that there is no significant difference in the accuracy of person re-identification with or without homomorphic
encryption and linear quantization.
Index Terms—Person Re-identification, Personal Information,
Linear Quantization, Homomorphic Encryption

I. I NTRODUCTION
There is a high demand for an authentication system thatwidely covers many areas using several cameras. An authentication system operates using a person re-identification
technique that extracts and matches features representing identities from video sequences. In particular, an authentication
system needs to carefully protect the features because theycorrespond to personal information. Personal information isoften protected using a private network that cannot be accessedfrom the outside. However, a private network is an expensive
solution.
We discuss how to design an authentication system at alarge scale. We need to consider the scalability of person reidentification because the amount of computation for person
re-identification exponentially increases when many pedestrian
images are acquired. For instance, we frequently encounter acase in which the number of people increases when there is acrowd. Alternatively, the number of cameras increases whenmonitoring areas are expanded. To deal with such scalability
issues, a common solution is to use a computing server ona public network using public hardware. However, there isa danger that the features for person re-identification could
be leaked if a computing server is the victim of malicious
attacks. An authentication system must keep the information
on a computing server inaccessible from third persons byencrypting the features.
In this paper, we propose a novel method for protecting
features extracted from video sequences using homomorphic
encryption [1]. The aim is to use this method in the designof an authentication system on a publicly available computing

Client 𝑞

Client 𝑡…

CameraCameraQuery pedestrian image

Target pedestrian image

S1 Feature extraction

S1 Feature extraction

S2 Quantization

S2 Quantization

S3 Encryption

S3 Encryption

S4 Distance computation without decryption

ServerFig. 1. Overview of our method.

server. Using homomorphic encryption, we can compute thedistance between features for matching persons without decrypting the features on a computing server. Even though thescale of an authentication system has been expanded this way,
we can safely protect personal information. With respect topattern recognition techniques using homomorphic encryption,
deep learning [2] and principal component analysis [3] arereported. We focus on the distance calculation for matching features of person re-identification. Generally, features
are often represented using floating-point numbers. However,when applying homomorphic encryption, the features should
be represented as integers to increase calculation efficiency.
We thus investigate whether our method can quantize andencrypt features while maintaining the accuracy of person
re-identification. Experimental results show that our method
maintains this accuracy when using linear quantization andhomomorphic encryption.
II. O UR METHOD
A. Overview
Figure 1 shows an overview of our method. In S1, we
extract query and target features f q and f t , respectively,from video sequences Iq and It . We use the confidence of
co-occurrence attributes described in [4] for designing thed2=(f q − f t )T M(f q − f t )(1)=(f 0q − f 0t )T (f 0q − f 0t ),(2)

where f 0q = Lf q and f 0t = Lf t . We use f 0q and f 0t
because the computational cost of matrix operations usinghomomorphic encryption is very large.
B. Homomorphic encryption
Homomorphic encryption is able to perform addition andmultiplication operations on a finite field without decryption. Our method computes a square of distance d2 between
encrypted features for person re-identification using fullyhomomorphic encryption (FHE) library.1 To encrypt a N dimensional feature, we control the following parameters of
the library: prime p determines an integer ring Zp representing
a plain text space and parameter m determines the number ofslots n (n ≥ N ). The parameter n is defined as
n=

φ (m)
,order (m, p)(3)

where φ is the Euler function, order() is the order of p modulo
m. It is difficult to freely control n because p and m strongly
dominate n. Our method experimentally searches for p and m
until it obtains an n that is larger than or equal to N .
C. Quantization
From the viewpoint of computational cost, we need torepresent the features as integers because the FHE library uses
a ring of integers. Our method transforms each element of thefeature using linear quantization. When computing a distanced2 between encrypted features, we must be careful to avoid
overflow. Prime p needs to satisfy the following conditional
expression:
N × 22g < p(4)where g is the number of quantization bits. Our method
uses the conditional expression when searching p and m. We
empirically quantize each element of f 0q and f 0t in the range−1 ≤ fq0 ≤ 1, −1 ≤ ft0 ≤ 1.
III. E VALUATION
We evaluated the accuracy of person re-identification using
linear quantization and homomorphic encryption. We used theSARC3D dataset, which is included in the PETA dataset [6].
We randomly split the SARC3D dataset into two small sets
without overlapping individuals. We used one small set fortraining a metric matrix and the other small set for evaluating
the matching rate. We repeated this procedure 10 times. We
1 HElib https://github.com/shaih/HElib

100Accuracy (%)

features. Note that each element of f q and f t is within therange −1 ≤ fq ≤ 1, −1 ≤ ft ≤ 1. We explain S2 andS3 in Section II-B, Section II-C. In S4, we use a largemargin nearest-neighbor (LMNN) [5] technique to improve
the accuracy of person re-identification. We represent a metricmatrix as M = LT L (||M||f ≤ 1) and compute a square ofdistance d2 as

0

1st1

2

3

5th4
56Quantization bits

10th78Baseline

Fig. 2. Accuracy of person re-identification while changing the parametersof linear quantization and homomorphic encryption.

set the number of target pedestrian images to only one. We
used the remaining the PETA dataset without the SARC3D
dataset for learning the co-occurrence attribute classifiers. The
dimensionality N of the features was 95.
We evaluated the accuracy while changing the number ofquantization bits g. The baseline accuracy was also evaluated
without the use of linear quantization and homomorphic encryption. Figure 2 shows the average and standard deviation ofthe n-th matching rate (%). We see that the accuracy decreases
when the number of bits was less than four. For example, weset the parameters as g = 4, p = 65, 537, and m = 14, 089.
We confirmed that there is no significant decrease in accuracy
using our method.
IV. C ONCLUSIONSWe proposed a method for computing the distance between
encrypted features using linear quantization and homomorphicencryption. We confirmed that the accuracy of person reidentification is almost unchanged using four or more bits forquantization. As part of our future work, we intend to evaluate
the computational costs when the number of features increases.
Acknowledgment This work was partially supported by JSPS
KAKENHI under grant number JP17K00238 and MIC SCOPE
under grant number 172308003.
R EFERENCES[1] Z. Brakerski, C. Gentry, and V. Vaikuntanathan, “Fully homomorphic
encryption without bootstrapping,” in Proceedings of 3rd Innovations inTheoretical Computer Science Conference, 2012, pp. 309–325.
[2] N. Dowlin, R. Gilad-Bachrach, K. Laine, K. Lauter, M. Naehrig, andJ. Wernsing, “Cryptonets: applying neural networks to encrypted datawith high throughput and accuracy,” in Proceedings of 33rd International
Conference on Machine Learning, 2016, vol. 48, pp. 201–210.
[3] W. Lu, S. Kawasaki, and J. Sakuma, “Using fully homomorphic encryption for statistical analysis of categorical, ordinal and numerical data,”
in Proceedings of Network and Distributed System Security Symposium,
2016, pp. 201–210.
[4] M. Nishiyama, S. Nakano, T. Yotsumoto, H. Yoshimura, Y. Iwai, andK. Sugahara, “Person re-identification using co-occurrence attributes ofphysical and adhered human characteristics,” in Proceedings of 23rdInternational Conference on Pattern Recognition, 2016, pp. 2086–2091.
[5] K. Weinberger, J. Blitzer, and L. Saul, “Distance metric learning for large
margin nearest neighbor classification,” Journal of Machine Learning,
vol. 10, pp. 207–244, 2009.
[6] Y. Deng, P. Luo, L. Ping, C. C. Loy, and X. Tang, “Pedestrian attributerecognition at far distance,” in Proceedings of 22nd ACM International
Conference on Multimedia, 2014, pp. 789–792.