# Face Recognition Performance Comparison Between Real Faces and Pose Variant Face Images from Image Display Device

Mi-Young Cho, Young-Sook Jeong

► To cite this version:

Mi-Young Cho, Young-Sook Jeong. Face Recognition Performance Comparison Between Real Faces and Pose Variant Face Images from Image Display Device. 3rd International Conference on Information and Communication Technology-EurAsia (ICT-EURASIA) and 9th International Conference on Research and Practical Issues of Enterprise Information Systems (CONFENIS), Oct 2015, Daejon, South Korea. pp.229-234, 10.1007/978-3-319-24315-3_23. hal-01466224

HAL Id: hal-01466224

https://inria.hal.science/hal-01466224

Submitted on 13 Feb 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Face Recognition Performance Comparison between
Real Faces and Pose Variant Face Images from Image
Display Device

Mi-Young Cho, Young-Sook Jeong

Electronics and Telecommunications Research Institute (ETRI), Daejeon, Korea
{mycho, ysjeong}@etri.re.kr

**Abstract.** Face recognition technology, unlike other biometric methods, is conveniently accessible with the use of only a camera. Consequently, it has created an enormous interest in a variety of applications, including face identification, access control, security, surveillance, smart cards, law enforcement, human computer interaction. However, face recognition system is still not robust enough, especially in unconstrained environments, and recognition accuracy is still not acceptable. In this paper, to measure performance reliability of face recognition systems, we expand performance comparison test between real faces and face images from the recognition perspective and verify the adequacy of performance test methods using an image display device.

**Keywords:** Face Recognition, Image Display Device, Performance Evaluation

# 1 Introduction

Face recognition is a widely used biometric technology because it is more direct, user friendly, and convenient to use than other biometric approaches. Face recognition technology is now significantly advanced, has great potential in the application systems. However, it is difficult to guarantee of performance due to insufficient test methods in real environment. The best method is direct evaluation from human subjects in real environment. Unfortunately, in this case, it would be considered impossible to consistently obtain the same way for a lengthy period of time a certain number of persons. That is, it's difficult to guarantee objectivity and reproducibility.

There are many approaches for performance evaluation of the face recognition in the system level including methods using an algorithm[1], a mannequin[2], and a high-definition photograph[3]. The first method simply evaluates the performance of an algorithm installed in a face recognition system. However, the performance of an algorithm cannot guarantee the performance of a face recognition system. The second method uses mannequin instead of real human face. This method has a number of problems because the material coating the mannequin is not the same as human skin. Last, the method using a high-definition photograph has overcome some of the existing problems. However, it still experiences minor difficulties with automatic control interoperation with a computer, and a lack of reproducibility in real situations.
In this paper, we expand performance comparison test between real faces and face images from the recognition perspective and verify the adequacy of performance test methods using an image display device. The paper is organized as follows: in Section 2, we explain limitation of precious works. Section 3 describes how to construct the facial DB. In section 4, we show and analyze the experimental results. Section 5 concludes this paper.

## 2 Previous works

In the previous works, we have introduced performance evaluation method of face recognition using face images from a high definition monitor and prove similarity between real faces and face images[5][11]. However, the previous work has a limitation to reflect performance in real environments as it is a test only using frontal pose images.

<figure><img src="image_2.png" /><figcaption>**Fig. 1.** Previous works.</figcaption></figure>

Recognizing faces reliably across changes in pose and illumination has proved to be a much more difficult problem[12]. So, we need verification about the proposed test method according to not only illumination but also pose. In this paper, we expand previous works and compare face recognition performance according to various poses.

## 3 Facial DB

The majority of facial images used to evaluate face recognition algorithms such as Feret[6], PF07[7], and CMU PIE[8] could be used for the proposed test method. However, most images are not adequate because of the low-resolution output of the image display device. To overcome this challenge, high-resolution facial DB was required.

To obtain subject images under various pose conditions, seven cameras were used. The locations of cameras are shown in Figure 2. We took ultra-high definition images using a Sony Nex 7 so that the face area took up at least two thirds of the whole area of the image. The height of the camera was fixed, and we controlled the height of the chair depending on the subject's height.
<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Environment for capturing real face images.</figcaption></figure>

We captured 4200 real face images from 60 subjects, which were captured under ten different lighting directions and seven pose for each subject. Figure 3 shows sample images for one subject.

<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Sample images for one subject.</figcaption></figure>

To the re-capture, we displayed the high definition images captured with a camera on a 27-inch image display device to provide an output similar to a real face. The image display device was calibrated and characterized according to the ISO 15076-1:2010 standard[9], which contains the criteria for color management and standard image reproduction. To ensure proper display output, we used 2.2 gamma tone reproduction curve, D65 whitepoint color temperature as stated in IEC 61966-2-1:1999[10], which contains the sRGB and HDTV color space standards. The procedure for the face image DB construction is presented in Fig. 4.
<figure><img src="image_5.png" /><figcaption>Fig. 4. Procedure for building face image DB.</figcaption></figure>

## 4 Experiment

This experiment verifies the similarity of real faces and face images from an image display device from the perspective of face recognition performance. In particular, we focused on changes in face recognition performance according to pose. The test engines registered ten frontal pose images under ten lighting conditions and obtained recognition results from test images that consists of six groups according to pose. Figure 5 illustrates sample face images for registration and test.

<figure><img src="image_6.png" /><figcaption>Fig. 5. Registration and test purpose sample images.</figcaption></figure>

The performance comparison results from four face recognition engines that are used for commercial purposes are shown Table 1. To analyze the similarity of real faces and the facial images captured from the image display device, recognition rate deviations were analyzed. As a result, the maximum deviation between the real facial and face images is 1.56.

<table><caption>Table 1. Overall results.</caption><thead><tr><th rowspan="2">Engine</th><th colspan="2">Face recognition rate(%)</th><th rowspan="2">deviation</th></tr><tr><th>Real faces</th><th>Face images</th></tr></thead><tbody><tr><td>A</td><td>97.09</td><td>95.90</td><td>1.19</td></tr><tr><td>B</td><td>98.96</td><td>99.01</td><td>0.05</td></tr><tr><td>C</td><td>97.78</td><td>98.23</td><td>0.45</td></tr><tr><td>D</td><td>87.62</td><td>86.06</td><td>1.56</td></tr></tbody></table>
<figure><img src="image_7.png" /><figcaption>**Fig. 6.** Performance changes according to the pose for (a) Engine A. (b) Engine B. (c) Engine C. (d) Engine D.</figcaption></figure>

Figure 6 shows performance changes according to the pose for each engine. Engine A and B get results from all test images, other engines get those from face images of only 4 poses(top/bottom/left/right 15°) because of coverage. The x-axis represents recognition rate and the y-axis represents pose. The number means recognition rate deviations between real faces and face images. Although each engine exhibited different recognition performance according to pose, the deviations between the real face and face images were all less than 3%. In other words, there is no significant difference in face recognition performance when using face images instead of real faces.

**5 Conclusion**

In this paper, we expand the previous works and verified the similarity of real face and face images from an image display device by comparing face recognition performance changes according to pose. Based on the comparison results using an image display device, the proposed method can be applied to the face recognition performance evaluation in system level.

**Acknowledgments.** This work is partly supported by the R&D program of the Korea Ministry of Trade, Industry and Energy (MOTIE) and the Korea Evaluation Institute of Industrial Technology (KEIT). (Project: Technology Development of service robot's performance and standardization for movement/manipulation/HRI/Networking, 10041834).
References

1. TTAK.KO-10.0418, Performance Evaluation Method of Face Extraction and Identification Algorithm for Intelligent Robots: Part 1 Performance Evaluation of Recognition Algorithm (2010)

2. TTAK.KO-10.0419, Performance Evaluation Method of Face Extraction and Identification Algorithm for Intelligent Robots: Part 2. System Level Performance Evaluation using Human Model (mannequin) of Human Face Recognition (2010)

3. TTAK.KO-10.0507, Performance Evaluation Method of Face Extraction and Identification Algorithm for Intelligent Robots: Part 3. Performance Evaluation of Face Recognition using Face Photos (2011)

4. Mi-young Cho, Young-Sook Jeong, Hyun-suh Kim, Howard Kim, "A New Multimodal Database for Performance Evaluation in System Level", IT CoNvergence PRACTICE (INPRA), volume: 2, number: 4, pp. 1-6(2015)

5. M.Y. Cho, Y.S. Jeong, and B.T. Chun, "A Study on Face Recognition Performance Comparison of Real Images with Images from LED Monitor", Journal of the Institute of Electronics Engineers of Korea, 50(5), pp.1164-1169 (2013)

6. P.J. Phillips, H. Moon, P.J. Rauss, and S. Rizvi. "The FERET evaluation methodology for face recognition algorithms", IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(10), (2000)

7. H. Lee, S. Park, B. Kang, J. Shin, J. Lee, H. Je, B. Jun, and D. Kim, "The POSTECH Face Database (PF07) and Performance Evaluation", Proc. IEEE Int. Conf. Automatic Face & Gesture Recognition, pp. 1-6 (2008)

8. T. Sim, S. Baker, and M. Bsat. "The CMU pose, illumination, and expression database", IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(12) (2003)

9. ISO 15076-1:2010, Image technology colour management -- Architecture, profile format and data structure -- Part 1: Based on ICC.1:2010(2010)

10. IEC 61966-2-1:1999, Multimedia systems and equipment – Colour measurement and management – Part 2-1: Colour management(1999)

11. Cho, Mi-Young, and Young-Sook Jeong. "Face Recognition Performance Comparison of Fake Faces with Real Faces in Relation to Lighting." Journal of Internet Services and Information Security (JISIS) 4.4, pp. 82-90 (2014)

12. Phillips, P. Jonathon, et al. "Face recognition vendor test 2002." Analysis and Modeling of Faces and Gestures, 2003. AMFG 2003. IEEE International Workshop on. IEEE, (2003)