Reduction in Communication via Image Selection for Homomorphic
Encryption-based Privacy-Protected Person Re-identification
Shogo Fukuda1 , Masashi Nishiyama2 and Yoshio Iwai2
1 Graduate School of Sustainability Science, Tottori University, Tottori, Japan
2 Graduate School of Engineering, Tottori University, Tottori, Japannishiyama@tottori-u.ac.jp

Keywords:

Image selection, homomorphic encryption, features, person re-identification, privacy protection

Abstract:

We propose a method for reducing the amount of communication by selecting pedestrian images for privacyprotected person re-identification. Recently, it has become necessary to pay attention to how features corresponding to personal information can be protected. Owing to homomorphic encryption, which enables adistance between features to be computed without decryption, our method can use a cloud server on a public
network while protecting personal information. However, we must consider the problem of the large amountof communication that occurs between camera clients and the server when homomorphic encryption is used.
Our method aims to reduce the amount of this communication by selecting appropriate pedestrian images
using reference leg postures. In our experiment, we confirmed that the amount of communication dynamically reduces without significant degradation in the accuracy of privacy-protected person re-identification withhomomorphic encryption.

1

INTRODUCTION

Multiple-camera monitoring systems are needed
for realizing a safe and secure society. Such systems are useful when searching for a criminal or
lost person. In this paper, we consider a multiplecamera monitoring system set in indoor corridors
where pedestrians must pass when moving between
rooms.
The postures of the pedestrians change over time
while they are walking. To correctly match pedestrian images acquired from the same individual, it isnecessary to extract features that represent identities
from the images and compute the distances between
these features, as described in (Gong et al., 2014;
Wang et al., 2018; Almasawa et al., 2019; Ye et al.,2020; Leng et al., 2020). For example, some existing methods (Nakajima et al., 2003; Bird et al., 2005;
Farenzena et al., 2010) extract low-level image features such as color and gradient histograms, and otherexisting methods (Bourdev et al., 2011; Layne et al.,2012; Zhang et al., 2014; Lin et al., 2019; Zhao et al.,2019) extract human attribute features such as genderand age.
When designing a multi-camera monitoring system, we must pay attention to the fact that the features
used in person re-identification are personal informa-tion. This is because the features are personally identifiable1 . Thus, multiple-camera monitoring systems
need to handle features with encryption so that third
parties cannot access them. To protect the personal information of the features, we focus on homomorphic
encryption (Brakerski et al., 2012). There are someprevalent methods of pattern recognition using homomorphic encryption, such as principal component
analysis (Lu et al., 2016) and deep learning (Dowlin
et al., 2016). Homomorphic encryption also enables
the distance to be computed using addition and multiplication operations without decrypting the target features, as described in (Morita et al., 2018). In thispaper, we apply homomorphic encryption to features
for person re-identification and perform the distancecomputation while the features are still encrypted.
When homomorphic encryption is used, thiscauses a severe problem in the multi-camera monitoring system in that the amount of communication
between each camera client and the server becomesvery large. Since the data size of features after homomorphic encryption is much larger than that before
encryption, the amount of communication per feature
increases. Furthermore, the data size is proportional
to the number of pedestrian images, which is the stage
1 General Data Protection Regulation (May 25, 2018)

before the conversion to features, and thus the amountof communication further increases when the numberof images is increased.
To avoid the problem of the data size, it is necessary to devise ways to reduce the amount of communication. There are two possible ways: by reducing
the number of dimensions of the features or by selecting the features to be sent to the server. In thispaper, we focus on the feature selection approach. In
particular, we consider reducing the number of pedestrian images (before feature extraction), which thenreduces the amount of communication. Many pedestrian images are acquired as a person walks in the fieldof view of a camera client. Specifically, the number ofpedestrian images increases as a multiplication of thenumber of people in the field of view, the time length,
and the camera’s frame rate. Moreover, the number ofcameras causes a problem for the server side. If all theacquired pedestrian images are directly received at thecloud server, the amount of communication increase
drastically.
In this paper, to reduce the number of pedestrian images and the amount of communication, we
propose a method for selecting appropriate pedestrian images for feature extraction for person reidentification from among the large number of pedestrian images acquired from each camera. We incorporate an image selection approach into a privacyprotected person re-identification system using homomorphic encryption. To do this, we focus on thecyclic motion of leg postures during walking as a cue
for selecting pedestrian images. Using the leg posture as a cue, our method extracts the features fromthe pedestrian images containing the same appearances of the persons from the cameras. In our experiments, we confirmed that our method effectively
reduces the number of pedestrian images while maintaining the accuracy of privacy-protected person reidentification.

2

RELATED WORK

We consider an idea to reduce the amount ofcommunication between the camera client and theserver for person re-identification. One idea is to apply a tracking process (Shu et al., 2012; Kawanishi
et al., 2017; Li et al., 2013) to pedestrians in thevideo sequence and extract features from the timeseries images linked by the tracking. An existing
method (Khan and Brémond, 2017) extracts featuresof body parts from multiple images of a time series
generated from the tracking process. This method
uses Gaussian mixture models (GMMs) to represent

the features of each body part of a person. However, GMMs have the disadvantage of a large featuresize per person caused by maintaining the mean vector and the covariance matrix. Thus, it is not suitable
for reducing the amount of communication, which isthe purpose of this paper. An existing method (Hanand Bhanu, 2006) focuses on gait features to integrate
multiple images of a time series generated from thetracking process into a single averaged image. The
gait features containing cyclic limb movements are
represented in the single image. Because this method
assumes that one complete gait cycle has been acquired, it is not possible to extract the features when
the tracking process is interrupted. In this paper, instead of using tracking methods, we design a method
based on image selection.
We consider one other idea to reduce the amountof communication, which is to reduce the dimensionality of the features. For example, some existing
methods (Weinberger et al., 2009; Shi et al., 2009;
Chen et al., 2015; Ullah et al., 2020; Ma et al., 2019;
Fang et al., 2020) encode features using hash functions, and others (Gong et al., 2013; Ambai et al.,2015; Irie et al., 2017; Wu et al., 2020; Zang et al.,2019; Qayyum et al., 2019) use vector quantizationand sparse coding to reduce the dimensionality. The
large number of pedestrian images increases is still aproblem, even if hashing and sparse coding have been
applied. Thus, we develop a method to directly solve
the problem of the high number of pedestrian images.
Existing face recognition methods (Chang et al.,2008; Z. Yang et al., 2004; Wong et al., 2011) select frontal face images from the sets of images detected in a video sequence using head pose estimation. Although these existing methods aim to obtain
similar effect as that of our method, it is not easy toapply them directly to person re-identification, whichuses the whole body of a person. Indeed, the existing methods for facial image selection are not suitable when the person’s back is facing the camera.
Thus, our method selects appropriate pedestrian images from the sets of images acquired from each camera client.

3

PEDESTRIAN IMAGE
SELECTION FOR PERSON
RE-IDENTIFICATION

3.1

Overview

We use the leg posture to select a small number of appropriate pedestrian images from the many pedestrianRight hip

Left hip

Right knee

Left knee

Right ankle

Left ankle

Figure 1: Body joints for representing a spatial cue of the change of leg posture. The temporal cue refers to the cyclic
movements of the legs of the body joints that change over time.

T1

T2

T3

T4

Support: Right leg

Support: Both legs

Support: Left leg

Support: Both legs

40%

10%
40%
Ratio included in gait cycle

10%

Figure 2: Representation of the temporal cues of leg posture using the categories based on the gait cycle. We represent eachcategory in time direction from T1 to T4 . In general, the duration of one cycle of the gait cycle is about one second.

images acquired from each camera. A person walking
in a corridor repeats a cyclic motion of her/his legs.
Thus, we assume that there will be many opportunities to acquire pedestrian images containing reference
leg postures. If we acquire a small number of pedestrian images with the same reference leg posture, we
can expect to re-identify a person without decreasing
the accuracy.
We discuss the reason for the use of leg posture
below. The same arm posture cannot be repeatedly
acquired in the same way as leg posture because thearm movements during walking are more flexible, forinstance, if the pedestrian is carrying something or
folding his/her arms. The head and torso postures areless effective in reducing the number of images thanthe leg postures because the same posture of the head
and torso frequently appears during walking. Thus,
we adopt the leg posture to reduce the number ofpedestrian images while maintaining the person reidentification accuracy.
The leg posture of a walking person is represented
using spatial and temporal cues. A spatial cue refers
to the change of the locations of the body joints.
Specifically, we use the cue of the spatial locationsof the body joints shown in Figure 1. The temporalcue refers to the cyclic movements of the legs of the
body joints that change over time. In the next section,
we describe the motion cycle of the leg posture in detail to consider the reference leg postures for image
selection.

3.2

Gait cycle of the leg posture

The motion change in the leg posture during walking is called the gait cycle in (Perry and Burnfield,2010). It is stated that the gait cycle is repeated as acyclic leg movement while walking. The gait cycle isdivided into four categories Tc (c = 1, . . . , 4) according to the differences in the leg postures, as shown in
Figure 2. Although the start time of the gait cycle is
not clearly defined in (Perry and Burnfield, 2010), we
treat the moment when the left leg leaves the groundas the start time of the gait cycle. We represent eachcategory in time direction from T1 to T4 . In general,the duration of one cycle of the gait cycle is about
one second. Categories T1 and T3 are both 40% of theduration, and T2 and T4 are both 10% of the duration.

Neck
Torso

Waist
𝒃1 : Right-hip

𝒃2 : Left-hip

𝒃3 : Right-knee

𝒃4 : Left-knee

𝒃5 : Right-ankle

𝒃6 : Left-ankle

Figure 3: Representation of body joints b j after the length of the torso has been normalized. We assign two-dimensional
locations to the body joints b j ( j = 1, . . . , 6) of each person at each time in each category Tc .

3.3

Reference leg postures for image
selection

We describe how to generate reference leg postures
Pc (c = 1, . . . , 4), which are the cues for selecting appropriate pedestrian images. We represent the temporal cue of the leg posture using each Tc of the gait
cycle. We also represent the spatial cue as the average
locations of the body joints in each Tc . To generate
the reference leg postures, we use time-series signalsof the locations of the body joints in the gait cycle
observed from various persons. To do this, we needto consider the temporal variation of the locations foreach person and the spatial variation of the locations
among them. We describe the detail generation of thereference postures below.
Suppose that the three-dimensional locations of
the body joints are observed by a motion capture system during the gait cycle of a large number of individuals. We project the body joints onto the image
plane using a predefined perspective projection matrix. Here, our discussion assumes that the height andangle of each camera set in the corridor are the same.
We normalize the scale and rotation to suppress thedifferences in the body height of the persons. We thenmake the length of the torso, which connects the neckand the waist, constant in the image plane.
Figure 3 shows our representation of the body
joints after normalizing the length of the torso. According to the definition of the gait cycle describedin Section 3.2, we assign two-dimensional locations
to the body joints b j ( j = 1, . . . , 6) of each person ateach time in each category Tc . We generate the reference posture Pc for Tc by calculating the averagevalues of these two-dimensional locations. We experimentally determine which of the four reference postures Pc should be used in Section 4.

Pedestrian images
with body joints 𝒃𝑗

Image selection

・・・

Referenceleg posture・・・

Search images having 𝒃𝑗
close to the body joints of 𝑃𝑐

𝑃𝑐

Selected images
used forfeature extraction

Figure 4: Overview of appropriate pedestrian image selection for person re-identification. Our method searches forpedestrian images with leg postures close to the reference
posture Pc .

3.4

Pedestrian image selection

In this section, we describe how we select appropriate
pedestrian images from a set of images detected ineach camera client. Our method searches for pedestrian images with leg postures close to the reference
posture. An overview of this pedestrian image selection is shown in Figure 4. We use the reference
posture Pc described in Section 3.3. After image selection, our method extracts features for person reidentification from the selected images.
In the following, we describe the detail of the image selection. Given an image of a pedestrian, thelocations of the body joints b j are determined using
a two-dimensional posture prediction technique, e.g.,
OpenPose (Cao et al., 2017). To compensate for thedifferences in body length between persons, we nor-Camera client at L1

Camera client at L2

S1 : Image selection

S1 : Image selection

S2 : Feature extraction

S2 : Feature extraction

S3 : Quantization

S3 : Quantization

S4 : Encryption

S4 : Encryption

S5 : Distance computation without decryption
Public server
Figure 5: Overview of our method for privacy-protected person re-identification. In S1, our method selects the appropriate
pedestrian images at each camera client. In S2, our method extracts features from the selected pedestrian images using
co-occurrence attributes. In S3, our method applies linear quantization to the features as pre-processing for homomorphic
encryption. In S4, our method encrypts the quantized features. Then, our method sends them from each camera client to theserver. In S5, our method calculates the distance between the received features without decrypting them in the server.

malize the locations of b j using the geometrical transformation described in Section 3.3 so that the lengthand rotation of the torso of an input pedestrian image
is equal to that of the reference posture.
We compute the cost of the difference between thelocation of the body joints b j in the reference postureand that of the body joints in the normalized postureusing the L2 norm. If this cost is less than or equal toa threshold R, our method determines that the pedestrian image containing a similar b j is suitable for feature extraction for person re-identification. Note that
we experimentally determined R so that one or more
images of each person are selected for each camera
client while she/he exists in the field of view of thecamera.

3.5

Privacy-protected person
re-identification

Figure 5 shows an overview of our method for
privacy-protected person re-identification. In S1, ourmethod selects the appropriate pedestrian images at
each camera client (L1 , L2 ) using body joint b j withthe procedure described in Section 3.4. In S2, ourmethod extracts features f q and f t from the selectedpedestrian images using co-occurrence attributes proposed by (Nishiyama et al., 2016). Co-occurrence
attributes are represented by combinations of physical and adhered human characteristics (e.g., a man
wearing a suit, a 20-something woman, or a woman
with long hair who is wearing a skirt). In S3, ourmethod applies linear quantization to the features as
pre-processing for homomorphic encryption. In S4,
our method encrypts the quantized features using thefully homomorphic encryption (FHE) library2 . Then,
our method sends them from each camera client tothe server. In S5, our method calculates the distance
between the received features f q and f t without decrypting them in the server. We use the large margin nearest neighbor (LMNN) proposed by (Weinberger and Saul, 2009), which is a metric learningtechnique. The details of the distance computation
for privacy-protected person re-identification are described in (Morita et al., 2018). In our system, it is nolonger necessary to decrypt the features on the server.
Thus, we believe that it is possible to operate a securesystem for privacy-protected person re-identification.

4

EXPERIMENTS

4.1

Experimental conditions

We confirmed whether our method could reduce theamount of communication used for privacy-protected
person re-identification. In particular, we evaluatedthe effectiveness of pedestrian image selection using
the reference leg posture. We used the first match rate
to evaluate the accuracy of person re-identification.
We also used the reduction rate to evaluate the amount
2 HElib https://github.com/shaih/HElib

𝑃1

𝑃2

𝑃3

𝐿1

𝐿2

𝐿3

𝐿4

𝐿5

𝐿6

𝑃4

Figure 6: Reference leg postures Pc (c = 1, . . . , 4) generatedusing the projection matrix of the TUP dataset.

of communication. We used pedestrian images fromone camera client as the target and pedestrian images from another camera client as the query. We
selected pedestrian images using each reference posture (P1 , . . . , P4 ). Next, we computed the average ofthe first match rate for the target and query images
described above. The reduction ratio is the ratio of
the number of images not selected by the method tothe number of pedestrian images when no image selection is used. We averaged the reduction rates overvarious combinations of camera clients.
In this experiment, we used the TUP (Tottori University Pedestrian) dataset, which we originally collected, and a publicly available MARS (Motion Analysis and Re-identification Set) (Zheng et al., 2016)
dataset. The details of each dataset are described in
Sections 4.3 or 4.4, respectively. First, we describe
the parameters of person re-identification. We set thedimension of features f q or f t to 95. We used 200
pedestrian images from the SARC3D dataset (Baltieri
et al., 2010) to generate a metric matrix for LMNN inthe experiments on the TUP dataset. For the experiments on the MARS dataset, we used 1000 randomly
selected images for the LMNN metric matrix from thetraining images included in the MARS dataset.

4.2

Generation of the reference leg
posturesWe generated the reference postures using the AIST
Gait Database 2015 provided by (Kobayashi et al.,2015). The dataset consists of 214 healthy Japaneseparticipants. The three-dimensional locations of the
body joints were measured using a motion capturesystem from walking participants. We used 100 participants randomly selected from the dataset and averaged the locations of the body joints of each participant during one gait cycle using the procedure described in Section 3.3. By averaging the locations of
the participants, we generated a reference posture Pc
for each category Tc . Note that the motion capture
sensor attached to the surface of the body cannot directly measure the body joints b j ( j = 3, . . . , 6) in Fig-

Figure 7: Examples of images acquired from the camera
clients (L1 , . . . , L6 ) in the TUP dataset.

ure 3. We used average locations from several peripheral joints. Because the camera settings are different for the TUP and MARS datasets, we used a perspective projection matrix manually adjusted to eachdataset. Figure 6 shows the reference leg postures
Pc (c = 1, . . . , 4) generated using the projection matrixof the TUP dataset.
Next, we describe the parameters of the image selection for pedestrians. OpenPose (Cao et al., 2017)
was used to predict the two-dimensional locations of
the body joints shown in Figure 3 from the pedestrian
images. We adjusted the threshold R of image selection for each dataset. As described in Section 3.4,
we searched for the best R such that one or more images were selected for each participant. We obtained
R = 100 for the TUP dataset and R = 375 for the
MARS dataset.

4.3

Evaluations on the TUP dataset

4.3.1

Conditions of image acquisition

We collected pedestrian images when walking alongindoor corridors using six camera clients. Figure 7
shows examples of images acquired from the camera
clients (L1 , . . . , L6 ) in the TUP dataset. The cameras
were set at the same height of 2.4 m from the floor.
The angles were set the same so that the center ofthe camera image was located 3 m in front of it. We

𝐿1

𝐿2

𝐿3

𝐿4

𝐿5

𝐿6

Figure 8: Examples of pedestrian images of five different individuals contained in the TUP dataset. We acquired the pedestrian
images from the camera clients (L1 , . . . , L6 ).
90

Reduction rate (%)

Accuracy (%)

90
80
70

60
50

40

𝑃1

𝑃2

𝑃3

𝑃4

Baseline

80
70
60
50
40

𝑃1

𝑃2

𝑃3

𝑃4

Figure 9: Average accuracy of person re-identification forthe camera clients (L1 , . . . , L6 ) of the TUP dataset.

Figure 10: Average reduction rate of pedestrian images forthe camera clients (L1 , . . . , L6 ) of the TUP dataset.

used a USB camera (logicool C920, 1920 × 1080 pixels), and the frame rate was set to 30 fps. The number of participants was 31, but the number of people
walking at the same time was limited to one. We instructed the participants to pass through the center ofthe corridor while walking. Figure 8 shows examplesof pedestrian images in the TUP dataset. There were
5,117 pedestrian images collected from camera client
L1 , 5,060 collected from L2 , 2,895 collected from L3 ,2,418 collected from L4 , 4,841 collected from L5 , and
5,082 collected from L6 .

dataset. Our image selection method using reference leg postures P2 and P4 obtained better accuracy than the baseline method using all pedestrian images without image selection. Figure 10 shows theaverage reduction rate when using reference posturePc (c = 1, . . . , 4). If image selection is not applied (thebaseline method), the reduction rate was 0%, and allpedestrian images were used. The reduction rates ofP2 and P4 were higher than those of P1 and P3 . We
believe that reference postures P2 and P4 , which had
high reduction rates and person re-identification accuracy, are suitable for the TUP dataset.
Figure 11 shows examples of selected pedestrian
images. We used reference posture P2 at camera client
L4 . In the figure, we listed pairs of pedestrian images
and their leg postures picked up from one gait cycle.

4.3.2

Evaluation results

Figure 9 shows the average person re-identification
accuracy for camera clients (L1 , . . . , L6 ) of the TUP

Sequence of pedestrian images of a person

Not selected

Selected

Not selected(a)

Not selected

Selected

Not selected(b)

Not selected

Selected

Not selected(c)Figure 11: Examples of pedestrian images selected using the reference posture P2 from camera client L4 of the TUP dataset.
In the figure, we listed pairs of pedestrian images and their leg postures picked up from one gait cycle.

Our method selected 3 out of 16 images of person 1in (a), 2 out of 16 images of person 2 in (b), 4 out
of 17 images of person 3 in (c). We see that the leg
postures of the selected pedestrian images are similar
to the reference posture P2 in Figure 6.
4.3.3

Amount of communication from thecamera client to the server

We evaluated the amount of communication for thecase in which L1 and L2 camera clients send features
to the cloud server. The amount of communication
per feature increased from 760 B to 524.4 KB before and after encryption. The number of pedestrian
images detected by camera client L1 was 5,117 and

5,060 were detected by L2 . Moreover, the amount ofcommunication of the baseline method with the encrypted features extracted from all the pedestrian images was 2.68 GB for L1 and 2.65 GB for L2 . In contrast, when the pedestrian images were selected usingreference pose P2 , the number of pedestrian images
was reduced to 903 for L1 and 807 for L2 . The amountof communication of our method was 0.47 GB for
L1 and 0.42 GB for L2 . We see that the amount ofcommunication and number of images were both reduced by 82.4% in L1 and 84.1% in L2 . These results
confirm that our method can substantially reduce theamount of communication for privacy-protected person re-identification using homomorphic encryption.

𝑃1

𝑃2

𝑃3

𝑃4

𝑃3

𝑃4

(a) C1(a) C1𝑃1

𝑃2(b) C2

Figure 13: Reference leg postures generated using the projection matrix of the MARS dataset.

(b) C2
Figure 12: Examples of pedestrian images contained in the
MARS dataset.

4.4

Evaluations on the MARS dataset

4.4.1

Conditions of image acquisition

We also evaluated our method on the publicly available MARS dataset. Figure 12 shows examples ofpedestrian images contained in the MARS dataset.
This dataset includes people walking freely in an outdoor plaza, but it can be assumed that the pedestrian
traffic line is constant. We used the image sets ofcamera clients C1 and C2. These sets contain only
the pedestrian image sequences after detection andtracking. Because the tracking results of some image
sequences are often interrupted, we used only thosecontaining more than 20 pedestrian images in one sequence. We evaluated the accuracy of 4,540 pedestrian images for 41 subjects of C1 and 3,948 pedestrian images for 40 subjects of C2. We searched for
persons walking in the same direction in the field ofview of cameras C1 or C2 at different times.
4.4.2

Evaluation results

Figure 13 shows reference leg postures generated using the projection matrix of the MARS dataset. Because of the different heights and angles of cameras
C1 and C2, different respective perspective projection
matrices were applied. The reference posture shownin (a) was used in the evaluation of C1 and the reference leg posture shown in (b) was used in the evaluation of C2.
Figure 14 shows the accuracy of person re-identification on the MARS dataset. We see that theaccuracies of our method using image selection withreference leg postures P2 and P4 were better than thatof the baseline method without image selection. Figure 15 shows the reduction rate of pedestrian images
in the MARS dataset. If image selection is not applied(the baseline method), the reduction rate was 0%, andall pedestrian images were used. The reduction ratesof P2 and P4 were higher than those of P1 and P3 .
We see that reference postures P2 and P4 , which had
both good reduction rates and person re-identification
accuracy, were suitable for the camera settings ofthe MARS dataset. These results confirm that our
method can reduce the number of pedestrian images
in the MARS dataset while maintaining the accuracyof privacy-protected person re-identification.

5

CONCLUSIONSTo reduce the amount of communication needed
for privacy-protected person re-identification, we proposed a method for selecting appropriate pedestrian
images. In the proposed method, we focused on the
gait cycle of the leg posture as a cue for image selection. We generated the reference leg postures fromthe body joints distributed in each category of the gait
cycle. We incorporated the image selection into a
privacy-protected person re-identification system using homomorphic encryption and evaluated the effectiveness of our method on the TUP and MARS
datasets. We confirmed that the amount of communication could be substantially reduced while achieving
the same or better accuracy of person re-identification
using the reference leg postures for selecting appropriate pedestrian images. In future work, we intend

60Reduction rate (%)

Accuracy (%)

90
80
70
60
50

40

𝑃1

𝑃2

𝑃3

𝑃4

50
40
30
20
10
0

Baseline

𝑃1(a) C1𝑃4

𝑃3𝑃4(a) C1

Reduction rate (%)

Accuracy (%)

𝑃3

50

90
80
70
60
50
40

𝑃2

𝑃1

𝑃2

𝑃3

𝑃4

Baseline

40
30

20
10
0

𝑃1

𝑃2

(b) C2(b) C2

Figure 14: Accuracy of person re-identification on the
MARS dataset.

Figure 15: Reduction rate of pedestrian images in the
MARS dataset.

to develop a reference posture generation for the casewhere the pedestrian traffic line is not constant as wellas when the height and angle of the camera are notconstant. We will expand our evaluations to investigate the effectiveness of our method when the numberof cameras and number of people increase. We also
intend to combine our method with dimensional reduction, e.g., hashing and sparse coding.

re-identification. In Proceedings of Eurographics Italian Chapter Conference.
Bird, N. D., Masoud, O., Papanikolopoulos, N. P., andIsaacs, A. (2005). Detection of loitering individuals inpublic transportation areas. Journal of IEEE Transactions on Intelligent Transportation Systems, 6(2):167–
177.
Bourdev, L., Maji, S., and Malik, J. (2011). Describing
people: A poselet-based approach to attribute classification. In Proceedings of International Conference on
Computer Vision, pages 1543–1550.
Brakerski, Z., Gentry, C., and Vaikuntanathan, V. (2012).
Fully homomorphic encryption without bootstrapping. In Proceedings of Innovations in TheoreticalComputer Science Conference, pages 309–325.
Cao, Z., Simon, T., Wei, S., and Sheikh, Y. (2017). Realtime
multi-person 2d pose estimation using part affinity
fields. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pages 1302–
1310.
Chang, L., Rodés, I., Méndez, H., and del Toro, E. (2008).
Best-shot selection for video face recognition using
fpga. In Proceedings of Iberoamerican Congress on
Pattern Recognition, pages 543–550.
Chen, W., Wilson, J., Tyree, S., Weinberger, K., and Chen,
Y. (2015). Compressing neural networks with thehashing trick. In Proceedings of International Conference on Machine Learning, pages 2285–2294.
Dowlin, N., Gilad-Bachrach, R., Laine, K., Lauter, K.,
Naehrig, M., and Wernsing, J. (2016). Cryptonets:
applying neural networks to encrypted data with high

ACKNOWLEDGEMENT
This work was partially supported by JSPS
KAKENHI under grant number JP17K00238,
JP18H04114, and JP20K11865.

REFERENCES
Almasawa, M. O., Elrefaei, L. A., and Moria, K. (2019). A
survey on deep learning-based person re-identification
systems. Journal of IEEE Access, 7:175228–175247.
Ambai, M., Kimura, T., and Sakai, C. (2015). Fast and accurate object detection based on binary co-occurrence
features. Journal of IPSJ Transactions on Computer
Vision and Applications, 7:55–58.
Baltieri, D., Vezzani, R., and Cucchiara, R. (2010). 3d bodymodel construction and matching for real time peoplethroughput and accuracy. In Proceedings of International Conference on Machine Learning, volume 48,
pages 201–210.
Fang, S., Wang, J., Yang, C., and Tong, P. (2020). Fast
retrieval method of image data based on learning
to hash. Journal of Physics: Conference Series,
1631:012029.
Farenzena, M., Bazzani, L., Perina, A., Murino, V., andCristani, M. (2010). Person re-identification bysymmetry-driven accumulation of local features. In
Proceedings of IEEE Computer Society Conference
on Computer Vision and Pattern Recognition, pages
2360–2367.
Gong, S., Cristani, M., Yan, S., and Loy, C. C. (2014). Person Re-Identification. Springer-Verlag London.
Gong, Y., Lazebnik, S., Gordo, A., and Perronnin, F.
(2013). Iterative quantization: A procrustean approach to learning binary codes for large-scale image retrieval. Journal of IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(12):2916–
2929.
Han, J. and Bhanu, B. (2006). Individual recognition using gait energy image. Journal of IEEE Transactions on Pattern Analysis and Machine Intelligence,28(2):316–322.
Irie, G., Arai, H., and Tanigushi, Y. (2017). Multimodal
learning of geometry-preserving binary codes for semantic image retrieval. Journal of IEICE Transactions
on Information and Systems, E100.D(4):600–609.
Kawanishi, Y., Deguchi, D., Ide, I., and Murase, H.
(2017). Trajectory ensemble: Multiple persons consensus tracking across non-overlapping multiple cameras over randomly dropped camera networks. In Proceedings of IEEE Conference on Computer Vision and
Pattern Recognition Workshops, pages 1471–1477.
Khan, F. M. and Brémond, F. (2017). Multi-shot person reidentification using part appearance mixture. In Proceedings of IEEE Winter Conference on Applicationsof Computer Vision, pages 605–614.
Kobayashi, Y., Hobara, H., and Mochimaru, H. (2015). Aist
gait database 2015. https://www.airc.aist.go.jp/dhrt/
gait2015/index.html.
Layne, R., Hospedales, T. M., and Gong (2012). Person
re-identification by attributes. In Proceedings of the
British Machine Vision Conference, number 24, pages
1–11.
Leng, Q., Ye, M., and Tian, Q. (2020). A survey of openworld person re-identification. Journal of IEEE Transactions on Circuits and Systems for Video Technology,
30(4):1092–1108.
Li, X., Hu, W., Shen, C., Zhang, Z., Dick, A., and Hengel,
A. V. D. (2013). A survey of appearance models in visual object tracking. Journal of ACM Transactions onIntelligent Systems and Technology, 4(4):58:1–58:48.
Lin, Y., Zheng, L., Zheng, Z., Wu, Y., Hu, Z., Yan, C., andYang, Y. (2019). Improving person re-identification
by attribute and identity learning. Journal of Pattern
Recognition, 95:151 – 161.
Lu, W., Kawasaki, S., and Sakuma, J. (2016). Using
fully homomorphic encryption for statistical analysis

of categorical, ordinal and numerical data. In Proceedings of Network and Distributed System Security
Symposium, pages 201–210.
Ma, X.-Q., Yu, C.-C., Chen, X.-X., and Zhou, L. (2019).
Large-scale person re-identification based on deephash learning. Journal of Entropy, 21(5):449.
Morita, K., Yoshimura, H., Nishiyama, M., and Iwai, Y.
(2018). Protecting personal information using homomorphic encryption for person re-identification. In
Proceedings of IEEE 7th Global Conference on Consumer Electronics, pages 166–167.
Nakajima, C., Pontil, M., Heisele, B., and Poggio, T.
(2003). Full-body person recognition system. Journalof Elsevier Pattern Recognition, 36(9):1997–2006.
Nishiyama, M., Nakano, S., Yotsumoto, T., Yoshimura,
H., Iwai, Y., and Sugahara, K. (2016). Person reidentification using co-occurrence attributes of physical and adhered human characteristics. In Proceedingsof International Conference on Pattern Recognition,
pages 2085–2090.
Perry, J. and Burnfield, J. M. (2010). Gait Analysis: Normal
and Pathological Function. Slack Inc.
Qayyum, A., Malik, A., M Saad, N., and Mazher, M.
(2019). Designing deep cnn models based on sparse
coding for aerial imagery: a deep-features reduction
approach. Journal of Remote Sensing, 52(1):221–239.
Shi, Q., Petterson, J., Dror, G., Langford, J., Smola, A.,
Strehl, A., and Vishwanathan, S. V. N. (2009). Hashkernels. In Proceedings of International Conferenceon Artificial Intelligence and Statistics, volume 5,
pages 496–503.
Shu, G., Dehghan, A., Oreifej, O., Hand, E., and Shah, M.
(2012). Part-based multiple-person tracking with partial occlusion handling. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition,
pages 1815–1821.
Ullah, A., Muhammad, K., Hussain, T., Baik, S. W., andDe Albuquerque, V. H. C. (2020). Event-oriented 3d
convolutional features selection and hash codes generation using pca for video retrieval. Journal of IEEE
Access, 8:196529–196540.
Wang, K., Wang, H., Liu, M., Xing, X., and Han, T.
(2018). Survey on person re-identification based on
deep learning. Journal of CAAI Transactions on Intelligence Technology, 3(4):219–227.
Weinberger, K., Dasgupta, A., Attenberg, J., Langford, J.,
and Smola, A. (2009). Feature hashing for large scale
multitask learning. In Proceedings of Annual International Conference on Machine Learning, pages 1113–
1120.
Weinberger, K. Q. and Saul, L. K. (2009). Distance metriclearning for large margin nearest neighbor classification. Journal of Machine Learning Research, 10:207–
244.
Wong, Y., Chen, S., Mau, S., Sanderson, C., and Lovell,
B. C. (2011). Patch-based probabilistic image quality assessment for face selection and improved videobased face recognition. In Proceedings of IEEE Computer Vision and Pattern Recognition, pages 74–81.

Wu, P., Liu, J., Li, M., Sun, Y., and Shen, F. (2020).
Fast sparse coding networks for anomaly detection invideos. Journal of Pattern Recognition, 107:107515.
Ye, M., Shen, J., Lin, G., Xiang, T., Shao, L., andHoi, S. C. C. (2020). Deep learning for person
re-identification: A survey and outlook. CoRR,
abs/2001.04193.
Z. Yang, H. Ai, B. Wu, S. Lao, and L. Cai (2004). Face pose
estimation and its application in video shot selection.
In Proceedings of the 17th International Conference
on Pattern Recognition, volume 1, pages 322–325.
Zang, M., Wen, D., Liu, T., Zou, H., and Liu, C. (2019).
A fast sparse coding method for image classification.
Journal of Applied Sciences, 9(3):505.
Zhang, N., Paluri, M., Ranzato, M., Darrell, T., and Bourdev, L. (2014). Panda: Pose aligned networks fordeep attribute modeling. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition,
pages 1637–1644.
Zhao, Y., Shen, X., Jin, Z., Lu, H., and , Hua, X. (2019).
Attribute-driven feature disentangling and temporal
aggregation for video person re-identification. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4908–4917.
Zheng, L., Bie, Z., Sun, Y., Wang, J., Su, C., Wang, S.,
and Tian, Q. (2016). Mars: A video benchmark forlarge-scale person re-identification. In Proceedings ofComputer Vision and Pattern Recognition European
Conference on Computer Vision, pages 868–884.