–

Systematic Analysis of Facial RecoImprovements in Multiple FRGC C

Michael Brauckmann, Stefan GehleViisage Technology FRGC/ICE Workshop, March 22-23, 2006 

© Viisage Technology All rights reserved 
–

Outline 

 Scanner Characteristics & 

Illumination Correction 

 3D Face Recognition  High Resolution Data  Overview to all Challenges  Conclusions & Outlook  References 

© Viisage Technology All rights reserved 2
© Viisage Technology All rights reserved – 3 

Scanner CharacteristiIllumination Correctio
–

Image Preprocessing Color Conversation (Minolta Scanne

Observations  R, G, B, and 3D scan not recorded at 

same time resulting in motion artifacts (e.g. pose correction) 

 G, B rather dark Corrective Actions  Modification of standard RGB to grey 

conversion, i.e. only use of R channel 

 Application of illumination correction 

algorithms 

 Mixed dataset experiments (Exp. 5 & 6) 

must be handled separately 

4 

© Viisage Technology All rights reserved 
(2R+7G+1B)/10 R G 

Experiment 3 - 1.2% / 96.7% 2.7% / 92.4% 

Experiment 5 4.2% / 79.3% 4.1% / 76.5% -

Experiment 6 5.2% / 76.1% 4.4% / 77.9% -

–

Image Preprocessing - Results 

Results  Exp. 3t: Accuracy on single channels – R>G>B  Exp. 5: Some overexposure on red channel of 

controlled dataset, R only is worse than RGB to gre

conversion [(2R+7G+1B)/10] 

 Exp. 6: Underexposure in all channels of 

uncontrolled dataset, R better than RGB to grey conversion [(2R+7G+1B)/10] 

* EER / Verification@FAR 0.

© Viisage Technology All rights reserved 5 
–

Illumination Correction (I) 

Regression Over Grey Level Values in Facia Symmetry assumption, compensates for the sl

regression plane 

 Standardized image to fixed mean value and fi

Gradient Compensation 

Normalization Backgrou

Suppress

Constant 

Gray Valu

around F

Best Fit Plane Regression 

Fixed Mean Fixed Variance 

© Viisage Technology All rights reserved 6 
–

Illumination Correction (II) 

Neighborhood Dependent Approaches  Cooperation with Prof. Vijayan Asari, Old 

Dominion University 

 Two approaches evaluated and optimized 

HPSRR [Asari, Seow, 2004] INDANE [Tao, Asari, 2004] 

Experimental Setup  Subset of FRGC Experiment 4 

 Gallery: 466 controlled images  Probe: 958 uncontrolled images  Training: 400 controlled, 400 uncontrolled 

images 

© Viisage Technology All rights reserved 7 
 

–

Experiments & Results 

 Evaluation Methods 

Two different recognition engines VISG 1: FaceTOOLS 3.2 (2004) VISG 2: Viisage’s Lab engine (2004) Enhanced images are used as input 

to recognition engine 

Verification and identification scenario 

 Results 

Both illumination correction methods 

improves Viisage‘s FR accuracy 

INDANE better than HPSRR 

(HPSRR annihilates small features) 

Improvement higher with VISG1 

(VISG2 already contains regression based illumination correction) 

6 7 8 9 

10 11 12 13 14 

Equal Error Rate (%)

50 55 60 65 70 

Identification rate (%)

© Viisage Technology All rights reserved 8 
© Viisage Technology All rights reserved – 9 

3D Facial Recognition 
–

Score

computation

Fusion Strategies for 2D/3D Algorith

Fusion on algorithmic level  Landmark finding (2D, 3D)  Pose estimation (2D, 3D)  Pose correction (2D, 3D)  ... Fusion on score level  Shape and texture yield 

independent scores and quality (confidence) 

 Fusion of scores 

Pose estimation 

Rendering of texture 

(shape) in frontal view 

2D landmark finding 

on pose transf. image 

2D landmark finding 

on texture image 

Features from 

texture image 

Feature

shape 

Features from texture + 

Score fusion 

Face detection 

(video, still) 

© Viisage Technology All rights reserved 10 
–

Viisage’s 3D Approach (DICAR) 

Foundation – HGM  Graph is automatically 

located to landmarks 

 Optimized features are 

extracted at the landmark positions -> facial template 

 Correlation in feature space 

determines the similarity between faces 

Extension to 3D  Extension from texture to 

depth images 

 Additional feature extraction 

on surface data 

 Fusion of texture and shape 

results on score level 

© Viisage Technology All rights reserved 11 
© Viisage Technology All rights reserved – 12 

Results of Score Fusion 

 Texture strongly outperforms shape, e

spite of unbalanced illumination 

 Score level fusion yields the best resul

2D landmark finding 

on texture image 

Features from 

texture image 

Features from 

shape image 

Score level fusion 

RO
–

Fusion of Algorithms and Scores 

Pose correction based 

on texture and shape 

2D landmark finding 

on pose transf. image 

Features from 

texture image 

Features from 

shape image 

Score level fusion RO

© Viisage Technology All rights reserved 13 

 Pose correction yields further improvem

both modalities and after fusion 
–

3D Results on Actual Engine 

Results of 2005 are confirmed  2D HGM engine only performs at 

level of 97–98% correct verification rate @ FAR of 0.1% 

 Pose correction using 3D shape 

improves verification rate by approx. 18 % in comparison to 2D engine 

 Combined pose correction, HGM on 

shape and score fusion of shape and texture improves verification rate by 37-44% in comparison to 2D engine 

 Absolute differences are less than 1% 

in verification rate 

18,0% 

44,3% 

0% 5% 

10% 15% 20% 25% 30% 35% 40% 45% 50% 

ROC I 

98,1 

98,4 

98,9 

9

96 

96,5 

97 

97,5 

98 

98,5 

99 

99,5 

100 

ROC I 

© Viisage Technology All rights reserved 14 
© Viisage Technology All rights reserved – 15 

High Resolution Data 
 

–

High Resolution Data 

Motivation  Analysis of FRGC results (2005) Solution  Correlation based method to exploit 

additional information in high resolution images 

 Selection of appropriate areas, where 

facial micro features are invariant to pose, illumination and expression 

 Fully integrated into Viisage’s core FR 

engine 

 Landmark/region finding  Score fusion HGM/Facial Micro Features 

95 96 97 98 99 

100 

Verification Rate @ 

FAR=0.1%

ROCIII, non-normalized

© Viisage Technology All rights reserved 16 
 

FRG

FRG

ROC 

93,2 

92 93 94 95 96 97 98 99 

100 

FMF 

Verification Rate @ 

FAR=0.1%

ROCII, z-normalized 

–

Facial Micro Features (FMF) 

Results  Successfully tested on multiple 

data sets (e.g., FRGC, FERET) 

 Significant improvement of 

accuracy on high resolution images (FRGC experiment #1) 

 Unchanged accuracy on 

insufficient images 

 Low matching speed for large 

scale tests -> hierarchical matching implemented 

 Small additional template size 

(~3kB) 

© Viisage Technology All rights reserved 17 
© Viisage Technology All rights reserved – 18 

Overview to all Challeng
B

VVVV

 

–

Evolution of FR Performance (FRGC 

© Viisage Technology All rights reserved 19 

55 65 75 85 95 

Verification Rate @ FAR=0.1%

ROCIII, non-normalized 

Best FRGC result (Jan 05) 98,92 100 86,55 96,22 97,28 VISG FRGC results (Jan 05) 97,64 99,96 --- 96,22 97,28 6VISG FRGC results (Apr 05) 97,64 99,96 89,56 96,88 97,28 6VISG FRGC results (Oct 05) 98,78 100,00 --- 98,11 --- 7VISG FRGC results (Mar 06) 98,98 99,91 98,06 98,21 6

Exp1 Exp2 Exp 3s Exp3t Exp3 (s+t) E
 

–

3D vs. Micro Features on Exp. 3 

95 

95,5 

96 

96,5 

97 

97,5 

98 

98,5 

99 

99,5 

100 

Verification Rate @ FAR=0.1%

ROCII 

HGM Standard HGM & Pose Correction HGM & Pose Correction HGM & FMF HGM & Pose Correction HGM & Pose Corr., 3D S

1-1 (non normalized) 1-1 (z-no

© Viisage Technology All rights reserved 20 
© Viisage Technology All rights reserved – 21 

Conclusions & Outlook 
–

Conclusions and Outlook 

Summary  Illumination correction greatly improves recognition 

images 

 High resolution works well for cooperative scenarios  High resolution adds more than 3D  We used a general purpose FR system in contrast to Outlook  FRGC provided a cornerstone to improve FR systema

 FRGC focused on specific aspects (high resolution, 3

 There are scenarios like low resolution, images with 

processing that have been left untouched 

 FRGC II may focus on those  Thanks to Jonathon Philips and TSWG 

 Thanks to all people, that contributed to the co

datasets either with their faces or their heads. 

© Viisage Technology All rights reserved 22 
© Viisage Technology All rights reserved –

THANK YOU! 

Dr. Micha

Director Research & 

mbrauckman

ww