Face Recognition in Video Using 

Gabor Wavelet Networks 

Rajkiran Gottumukkal and Vijayan Asari 

OldDominion University
Overview 

• Introduction • Face Processing Tasks • Face Localization • Facial Feature Localization • Face Tracking & Recognition • Video Quality • Conclusion 

Department of Electrical and Computer Engineering 
Introduction 

• Video based recognition is less intrusive • Video provides soft biometrics and 

identification at a distance 

• Still a new field, lot needs to be done • Face not frontal, poor illumination, blurriness, 

bad focus 

• Not mature enough for intelligent video 

surveillance 

Department of Electrical and Computer Engineering 

11/9 Hijackers as seen on surveillance cameras 
Face Processing Tasks 

Face Localization 

Facial Feature 

Localization 

Face Tracking 

Face Identification Facial Event Recognition 

Face Memorization 

Department of Electrical and Computer Engineering 
Face Localization 

• Use methods to detect faces quickly for 

real time processing 

• Optical flow to find changes in scene • Viola-Jones face detector to detect faces • Can obtain real time face localization 

Department of Electrical and Computer Engineering 
Facial Feature Localization 

• The GWN approach represents a face image through a 

linear combination of 2D Gabor functions. 

• Considering the 2D image case, each single odd Gabor 

wavelet can be expressed as follows: 

• GWN for an image consists of n such wavelets and a set 

of associated weights 

Department of Electrical and Computer Engineering 
Facial Feature Localization Cont. 

• GWN parameters are learned : 

•Randomly drop n wavelets within the target 

object 

•Perform gradient descent to minimize the 

difference between the GWN and the training image 

• Hierarchal wavelet networks are used to 

localize eight facial features. 

Department of Electrical and Computer Engineering 
Facial Feature Localization Cont. 

• Level One : Face Matching 

• Determine an affine transformation of the level-one 

GWN that registers the candidate with the target image 

• The residual score in wavelet subspace is minimized 

to find the best match face 

• Level Two : Feature Localization 

• For each feature a brute-force search is preformed 

between a level-two feature GWN and the target image. 

• Candidate feature GWNs may be drawn from any of 

the faces in the database. 

Department of Electrical and Computer Engineering 
Facial Feature Localization Cont. 

1-Level Matching 2-Level Matching 

Courtesy: R. S. Feris, J. Gemmell, K. Toyama, and V. Krueger, "Hierarchical Wavelet Networks for Facial Feature Localization” 

Department of Electrical and Computer Engineering 
Face Tracking & Recognition 

Normal distribution 

• The face and its identity are 

simultaneously tracked using CONDENSATION filter 

• Consider a normal distribution 

with the observation as the mean. 

• Randomly select 

N samples( 

n

t ) Observation to be tracked N random samples around the mean. 0 (ex. position of face) 

n

xt with initial 

w

= 1 . n 0 t0 • Assign a weight(

w ) to each t0 n selected sample (initially

w

= 1). t0

(n) (n) (n) (n) • Construct a new sample set 

{

s

t+1,wt+1,ct+1, n

= 1,

...,N

} where 

ct+1 is 

the cumulative weight value such that (n) (n

−1) (n) (0)

c

=

c

+

w and

c

= 0. t+1 t+1 t+1 t+1

• The new sample set is constructed in three stages: 

• Sample, Predict, Measure 

Department of Electrical and Computer Engineering 
• Predict: Move particles according 

to deterministic dynamics (drift), then perturb individually (diffuse). 

( )i ( )i s

=

F

(

s

)

+

d t+1 t+1

drift diffusion 

(i) (i) s

= (−0.1

+ (s

+ 0.1)

* 0.4)

+ (0.075

*

r) t +1 t +1

• Measure: Weigh the new samples 

in terms of the measured features (ex. location) from the image. (i) 2 2

1

−( (s )−

 (x )) /(2

 ) ( )i t+1 t w

=

e 0

t+1 

2

 measured 

feature



= 0.03

• Position can be estimated from 

the N samples: N

N (n) (n) (n) s

=

w s

w t+1 t+1 t+1  t+1

n

=1 n

=1

 ( )

1

i

ts

+

( )

1

i

ts

+

Face Tracking & Recognition Cont. 

Predict: 

(drift & dif

uf

se) 

Measure: 

(n) wt0

wt+1 ( ) n

Assign

ni

g

new 

we

gi

hts 

Department of Electrical and Computer Engineering 
Demo 

Department of Electrical and Computer Engineering 
Video Quality 

• Video quality is not good enough on current 

surveillance cameras 

• Video quality is affected due to: 

• record to video tape • quality of frame grabbers • video compression 

• Can we recognize these faces accurately? 

Department of Electrical and Computer Engineering 
Conclusion 

• We can obtain good results in controlled 

conditions with good video quality 

• Face recognition in video is still not 

practical on real world surveillance video 

• We need to improve the algorithms • Also need better quality video from 

surveillance cameras 

Department of Electrical and Computer Engineering