Universität Karlsruhe (TH) 

Analysis of Local Appeara

Face Recognition on FRGC 2

Analysis of Local Appeara

Face Recognition on FRGC 2

HAZIM KEMAL EKENEL

Interactive Systems Labs (ISL), Universität Karlsru

e- mail: ekenel@ira.uka.de

web: http://isl.ira.uka.de/~ekenel/

http://isl.ira.uka.de/face_rec

Face Recognition Grand Challenge Wo

March 22, 2006

Arlington, VA, USA

HAZIM KEMAL EKENEL

Interactive Systems Labs (ISL), Universität Karlsruh

e- mail: ekenel@ira.uka.de

web: http://isl.ira.uka.de/~ekenel/

http://isl.ira.uka.de/face_rec

Face Recognition Grand Challenge Wo

March 22, 2006

Arlington, VA, USA
Universität Karlsruhe (TH) 

Outline Outline

– Interactive Systems Laboratories (– Local Appearance-based Face Re– Feature Sets– Feature Normalization– Experiments– Conclusions

– Interactive Systems Laboratories (– Local Appearance-based Face Re– Feature Sets– Feature Normalization– Experiments– Conclusions
Universität Karlsruhe (TH) 

Interactive Systems Labs (IS

Interactive Systems Labs (IS

• Founded by Prof. Dr. Alexander Waib• Research on:

– translation, speech, language, visi

multimodal, and cross-modal percsmart rooms, 

• Located at Carnegie Mellon Universit

University of Karlsruhe (TH), German

• Founded by Prof. Dr. Alexander Waib• Research on:

– translation, speech, language, visi

multimodal, and cross-modal percesmart rooms, 

• Located at Carnegie Mellon Universit

University of Karlsruhe (TH), German
Universität Karlsruhe (TH) 

ISL Vision Group ISL Vision Group

• Directed by Assist. Prof. Rainer Stiefe• Focus on visual perception of people • Areas of interest

– Person Identification– Person Tracking– Head Pose / Focus of Attention Estim– Activity Analysis

• Directed by Assist. Prof. Rainer Stiefe• Focus on visual perception of people • Areas of interest

– Person Identification– Person Tracking– Head Pose / Focus of Attention Estim– Activity Analysis
Universität Karlsruhe (TH) 

ISL Face Recognition Group ISL Face Recognition Group

• Objective:

– Face recognition for smart environme– Developing and deploying fully autom

recognition systems with the research simple, fast & robust face recognition 

• Implementation areas:

– Recognizing individuals entering a ro

camera

– Recognizing individuals in a room usi– Human Robot Interaction –Humanoid 

• Objective:

– Face recognition for smart environme– Developing and deploying fully autom

recognition systems with the research simple, fast & robust face recognition 

• Implementation areas:

– Recognizing individuals entering a roo

camera

– Recognizing individuals in a room usi– Human Robot Interaction –Humanoid 
Universität Karlsruhe (TH) 

ISL Door Face Database ISL Door Face Database

• Ten thousands pictures of more than 100 

been collected during 86 recording days Dec. 2005)

• ~30000 images of 30 individuals will beco

• Ten thousands pictures of more than 100 

been collected during 86 recording days (Dec. 2005)

• ~30000 images of 30 individuals will beco
Universität Karlsruhe (TH) 

Face Recognition @ CLEAR Ev

Face Recognition @ CLEAR E

• CHIL Project (EU IP FP6)

http://chil.server.de/

• NIST• Goal: To recognize the 

lecturer/seminar participant by using video and multiview data acquired by four cameras mounted to the corners of the room. 

• Problem Conditions:

Low resolution faces with improper lighting, especially because of the projector's beam.

• CHIL Project (EU IP FP6)

http://chil.server.de/

• NIST• Goal: To recognize the 

lecturer/seminar participant 

by using video and multiview data acquired by four cameras mounted to the corners of the room. 

• Problem Conditions:

Low resolution faces with improper lighting, especially because of the projector's beam.
Universität Karlsruhe (TH) 

Face Recognition for Human

Face Recognition for Human
Universität Karlsruhe (TH) 

Local Appearance-based Face 

Local Appearance-based Face 

• Merits:

– Robust against local variations– Facilitates weighting/selection of the “

regions for face recognition

• Previous approaches:

– Salient region based

• Modular Eigenfaces (Pentland et al., • FR with SVMs: Global vs. Componen

(Heisele et al. 2003)

• FR using Component-based DCT/LDA 

MPEG7)

– Generic

• Modular PCA (Gottumukkal & Asari, 2

• Merits:

– Robust against local variations– Facilitates weighting/selection of the “

regions for face recognition

• Previous approaches:

– Salient region based

• Modular Eigenfaces (Pentland et al., 1• FR with SVMs: Global vs. Componen

(Heisele et al. 2003)

• FR using Component-based DCT/LDA 

MPEG7)

– Generic

• Modular PCA (Gottumukkal & Asari, 2
Universität Karlsruhe (TH) 

Face Representation Face Representation

• Local appearance modelling: 8x8 pixe

– Less sensitiveness to illumination and loc

• Data-independent basis (Discrete Co

– Fast– Compact representation

• Feature/Decision fusion to perform cl

• Local appearance modelling: 8x8 pixe

– Less sensitiveness to illumination and loc

• Data-independent basis (Discrete Co

– Fast– Compact representation

• Feature/Decision fusion to perform cl
Universität Karlsruhe (TH) 

DCT Basis DCT Basis

v

u
Universität Karlsruhe (TH) 

Sample DCT Output & Scan 

Sample DCT Output & Scan 

Zig-Zag Scan
Universität Karlsruhe (TH) 

Feature Extraction Feature Extraction

Image Block 1

Image Block 2

Image Block N

DCTDCTDCT
Universität Karlsruhe (TH) 

Feature Selection Feature Selection

• Selecting the first M DCT coefficie• Removing the first coefficient, and 

first M DCT coefficients from the r(DCT-0)

• Removing the first three coefficien

the first M DCT coefficients from tones (DCT-3)

• Selecting the first M DCT coefficie• Removing the first coefficient, and 

first M DCT coefficients from the r(DCT-0)

• Removing the first three coefficient

the first M DCT coefficients from tones (DCT-3)
Universität Karlsruhe (TH) 

Feature Normalization Feature Normalization

• The blocks with different brightness leve

coefficients with different value levels. 

normalize the local feature vector’

unit norm:

• The first DCT coefficients have higher m

later ones, thus having more influence on results. 

divide each coefficient to its stand

learned from the training set:

• The blocks with different brightness leve

coefficients with different value levels. 

normalize the local feature vector’

unit norm:

• The first DCT coefficients have higher m

later ones, thus having more influence on results. 

divide each coefficient to its standa

learned from the training set:

fffn = /

)(/ , iiin = σ fff
Universität Karlsruhe (TH) 

Face Recognition Experimen

Face Recognition Experimen

• Experimental Data derived from FRG

Experiments 1 & 4

• Individuals that have at least 10 imag

are selected

• 120 individuals (10 images for trainin• Controlled vs. Controlled (Fall 2003 r

training & Spring 2004 recordings for 

• Uncontrolled vs. Uncontrolled (Fall 20

training & Spring 2004 recordings for 

• Experimental Data derived from FRG

Experiments 1 & 4

• Individuals that have at least 10 imag

are selected

• 120 individuals (10 images for training • Controlled vs. Controlled (Fall 2003 re

training & Spring 2004 recordings for 

• Uncontrolled vs. Uncontrolled (Fall 20

training & Spring 2004 recordings for 
Universität Karlsruhe (TH) 

Distance Metrics Distance Metrics

∑

=

= −

M

m

mtestmtraining ffd

1

,,

2/1

1

2

,,   = ∑ − =Mm mtestmtraining ffd

training test

training test

ff

ff d

**

=

( ) ( )

training training test test

training training test test

mfmf

mfmf d

− −

− −

=

*

*

L1:L2:cor:cov:
Universität Karlsruhe (TH) 

Sample Images Sample Images

Controlled Samples:Uncontrolled Samples:
Universität Karlsruhe (TH) 

FRGC Controlled vs. Controlled PCA scores @ 320 –no normaliz

FRGC Controlled vs. Controlled PCA scores @ 320 –no normaliza

DCT DCT –

w/o DC

DCT –

w/o 3

L1 74.8% 94.3% 92.8% 8

L2 62.2% 86.7% 82.2% 8Cos 78.8% 87.4% 86.3% 8Cov 79.0% 87.8% 86.1% 8
Universität Karlsruhe (TH) 

FRGC Controlled vs. Controlled PCA scores @ 320 –unit norm 

FRGC Controlled vs. Controlled PCA scores @ 320 –unit norm 

DCT DCT –

w/o DC

DCT –

w/o 3

L1 90.6% 96.8% 96.8% 8

L2 79.9% 93.6% 94.3% 8Cos 79.9% 93.6% 94.3% 8Cov 80.0% 93.6% 94.4% 8
Universität Karlsruhe (TH) 

FRGC Controlled vs. Controlled PCA scores @ 320 –over dim. no

FRGC Controlled vs. Controlled PCA scores @ 320 –over dim. no

DCT DCT –

w/o DC

DCT –

w/o 3

L1 91.3% 96.3% 95.7% 8

L2 89.4% 93.1% 91.2% 8Cos 92.7% 93.8% 94.4% 9Cov 93.2% 93.9% 94.4% 9
Universität Karlsruhe (TH) 

FRGC Controlled vs. Controlled 

FRGC Controlled vs. Controlled 
Universität Karlsruhe (TH) 

FRGC Uncontrolled vs. Uncontro& PCA scores @ 320 –no norma

FRGC Uncontrolled vs. Uncontro& PCA scores @ 320 –no normal

DCT DCT –

w/o DC

DCT –

w/o 3

L1 43.6% 61.4% 60.6% 4

L2 36.9% 56.1% 55.8% 4Cos 39.4% 69.3% 65.8% 3Cov 39.3% 69.6% 66.1% 3
Universität Karlsruhe (TH) 

FRGC Uncontrolled vs. Uncontro& PCA scores @ 320 –unit norm

FRGC Uncontrolled vs. Uncontro& PCA scores @ 320 –unit norm

DCT DCT –

w/o DC

DCT –

w/o 3

L1 70.5% 80.5% 80.8% 4

L2 63.2% 75.3% 76.8% 3Cos 63.2% 75.3% 76.8% 3Cov 63.6% 75.3% 76.7% 3
Universität Karlsruhe (TH) 

FRGC Uncontrolled vs. Uncontro& PCA scores @ 320 –over dim. 

FRGC Uncontrolled vs. Uncontro& PCA scores @ 320 –over dim. 

DCT DCT –

w/o DC

DCT –

w/o 3

L1 53.3% 63.1% 58.9% 4

L2 49.8% 57.8% 56.5% 4Cos 51.2% 67.6% 71.4% 5Cov 50.9% 68.3% 71.6% 5
Universität Karlsruhe (TH) 

FRGC Uncontrolled vs. Uncontro

FRGC Uncontrolled vs. Uncontro
Universität Karlsruhe (TH) 

Conclusions Conclusions

• Using proper local features / normaliz

contributes face recognition performaresults have been also obtained of thPIE face databases)

• Unit norm DCT-0 / DCT-3 local featur

• Using proper local features / normaliz

contributes face recognition performaresults have been also obtained of thPIE face databases)

• Unit norm DCT-0 / DCT-3 local featur

Controlled vs. Controlled

@ feature dimension of 320

PCA, 89.2%

DCT ver. 1, 94.3%DCT ver. 2, 96.8%

Uncontroll

@ feature 

P

DCT DCT 
Universität Karlsruhe (TH) 

Questions? Questions?