Understanding The Face 

Format Standards

Paul Griffin, Ph.D.Chief Technology OIdentixApril 2005
© 2005 Identix Inc. All rights reserved. 2

Topics

• The Face Image Standard

– The Record Format– Frontal Face Images– Face Images and Compression

• On 3D Face Recognition• Conformance Standards and Testing
The Face Imag
© 2005 Identix Inc. All rights reserved. 4

General Approach to the Face 

• Specify face images because there is no agreem

face recognition template

– Unlike finger minutia ...

• Specify how a photograph should appear rathe

the photograph (i.e. lighting and cameras)

– A new project to address this is under way in S

• Allow for the specification of additional visible i

discernable by an operator pertaining to the faand eye color

– To improve identification performance 

• Verify that specified format and compression a

recognition performance

– Tested standard using leading algorithms on p

• Best practice appendices developed to allow fo

recognition

– Note: some ISO best practices are ICAO requir
© 2005 Identix Inc. All rights reserved. 5

US and International Face Image 

ANSI – ANSI INCITS 385 - May 

2004, the US Standard for Digital Image Formats for use with the Facial Biometric

– Referenced by DoC for US PIV ID 

Card, DHS for face capture

• ISO – 19794-5 FDIS –

November 2004, the Final Draft International Standard of Biometric Data Interchange Formats - Part 5 – Face Image Data.

– Referenced by ICAO “Biometrics 

Deployment of Machine Readable Travel Documents”, TAG MRTD/NTWG May 2004.
© 2005 Identix Inc. All rights reserved. 6

Scope of the Standard

Requirements Specified in the Stand

Scene Photographic Digital 

Lighting Digital Camera

Analogue to Digital

Positioning 

Camera Attributes 

Image and Subject 

Image Scanning
The Record Fo
© 2005 Identix Inc. All rights reserved. 8

Record Format Highlights

Can specif

seen on 

• Gender• Eye colo• Hair colo• Expressi• Propertie• Pose ang

and roll)

• Feature 

(e.g. eye 

• CBEFF header• Multiple images per 

record allowed

• Can encode

– Image source type (video or still)– Image color space– Vendor-specific device– Quality (to be defined)

• JPEG or JPEG2000 

encoding and compression allowed
© 2005 Identix Inc. All rights reserved. 9

Record Format – Specificatio

Facial R ecord Facial R ecor

CBEFF Header Header D ata 

Nu m b e r o f Fa c ia l F a cia l F eatu re 

Form at Id e n tifie r V e r s io n N u m be r Le ngth of Re cord Im ag e s 

In fo rm a tio n P o in t(s ) 

4 4 4 2 20 8x 

F a cia l Facial Num be r o f Prope rty 

In fo rm a tio n Re cord Data Fe ature Gender Eye C o lo u r Hair C o lo u r Exp r e s s 

M ask 

Le ngth Po in ts

4 2 1 1 1 3 2 

F eatu re Ho riz o n tal V e rtical Fe ature Fe ature 

Po s itio n Po s itio n Re s e rve d

P o in t Po in t T yp e Po in t C o d e (x) (y) 

1 1 2 2 2

Im age Im ag e 

Face Im ag e Im ag e Data

In fo rm a tio n W id th He ig h t C o lo u r Source Type De vice T 

Type Type Space 

1 1 2 2 1 1 2 

Im age D ata JPEG o r JPEG2000 

V a r ia b le
© 2005 Identix Inc. All rights reserved. 10

Face Image Types

• Basic: The fundame

Type that specifies a including header an

– No mandatory sc

digital requiremethis image type.

• Frontal: A face tha

additional requiremefrontal face recognitexamination. Either 

• Full Frontal: This t

includes the full heamost cases, as well shoulders. 

– Used for ePass

• Token Frontal: A 

that specifies frontaspecific geometric spositioning based on height of the image

BasicFrontal

Full Token
© 2005 Identix Inc. All rights reserved. 11

Feature Points (Optional)

x y

z

11.5

11.4

11.2

10.2

10.4

10.10

10.8

10.6

2.14

7.1

11.6 4.6

4.4

4.2

5.2

5.4

2.10

2.12

2.1

11.1

Tongue

6.2 6.4 6.3

6.1

Mouth

8.1

8.9 8.10 8.5

8.3

8.7

8.2

8.8

8.4

8.6

2.2

2.3

2.6

2.9 2.8

2.5 2.7 2.4

2.1

2.12 2.11

2.14

2.10

2.13

10.6

10.8

10.4

10.2

10.10

5.4

5.2

5.3

5.1

10.1

10.9

10.3

10.5

10.7

4.1 4.3 4.6 4.5 4.4 4.2

11.2 11.1 11.3

11.411.5

x

y

z

Nose

9.6 9.7

9.14 9.13

9.12

9.2

9.4 9.15 9.5

9.3

9.1

Teeth

9.10 9.11

9.89.9

Feature points affected by FAPsOther feature points

Right eye Left eye

3.13

3.7

3.9

3.53.13.3

3.11

3.14

3.10

3.12 3.6

3.43.2

3.8

• The (optional) featu

specification of eye face registration infoface recognition algo

• Definitions based on 
Frontal Face Im
© 2005 Identix Inc. All rights reserved. 13

Frontal Images

• Pose

– Pose is known to strongly affect performanc

face recognition systems. 

 The full-face frontal pose shall be used. Rota

shall be less than +/- 5 degrees from frontal roll, pitch and yaw.

• Expression

– The ISO best practice requirement is:

 The expression should be neutral (non-smilin

open normally (i.e. not wide-open), and moueffort should be made to have supplied imagspecification. A smile with closed jaw is not r

• Background

– The ISO best practice requirement is:

 The background should be plain, and shall co

containing lines or curves that could cause coalgorithms to become confused. Therefore tshould be a uniform colour or a single colour changes from light to dark luminosity in a sin
© 2005 Identix Inc. All rights reserved. 14

Frontal Images

• Lighting

– No shadows or point light source (single flas

 Lighting shall be equally distributed on the fa

significant direction of the light from the poinphotographer. 

 Diffused lighting, multiple balanced sources o

methods shall be used.

 A single bare “point” light source is not accep

Instead, the illumination should be accomplismethods that meet requirements specified...

• No Camera Capture Artifacts

– No NTSC or PAL video frames

 Interlaced video frames are not allowed for t

Type. All interlacing must be absent (not simabsent).

– No stretched images

 Digital cameras and scanners used to capture 

produce images with a pixel aspect ratio of 1
© 2005 Identix Inc. All rights reserved. 15

Full Frontal or Token Frontal

Full Frontal

• Cropping requirements specified to allow for full capture of face and shoulders • Consistent with most current mug- shot and passport standards• Recommended for use with ePassports, both on the printed passport and stored in the chip

• Eye p

positiimage Storag

• A 90 

eye tused 
© 2005 Identix Inc. All rights reserved. 16

Full Frontal Images – Geometric Constraints

Definition RequirementsVertical Position of Face 0.5 B ≤ BB ≤ 0.7 B

Vertical Position of Face 

(Children under the age of 11)

0.4 B ≤ BB ≤ 0.7 B

Width of Head A ≥ 1.4 CC

Length of Head B ≥ 1.25 DD

• 90 pixels from eye to eye is 

required

• 120 pixels – best practice 

recommendation 
© 2005 Identix Inc. All rights reserved. 17

Token Frontal Images – GeometrConstraints

Feature or Parameter ValueImage Width WImage Height W/0.75Y coordinate of Eyes 0.6 * WX coordinate of First (right) Eye 0.375 * WX coordinate of Second (left) Eye = 

0.625 * W

(0.625 * W) - 1

Width from eye to eye (inclusive) 0.25 *W

Width = 240 corresponds to 60 pixels from eye to eyeWidth = 480 corresponds to 120 pixels from eye to e
© 2005 Identix Inc. All rights reserved. 18

Frontal Images – Scene Constrai

• No hair covering fron• Eyes open• No portrait style imag• Eyes on same horizo• Single color backgrou• Face centered• No single flash or fla• No red-eye• No shadows on back• No shadows on face
© 2005 Identix Inc. All rights reserved. 19

Frontal Images – Scene Constrai

• No sunglasses• No glare on glasses• Tinted glasses OK (if • Glasses avoid eyes (if • Remove hats • Remove caps (affects • No shadows on face f

headgear

• Lower veil to expose c

from roughly crown to ear. 

• No other face or parti• No toys or other objec
Face Images a

Compression
© 2005 Identix Inc. All rights reserved. 21

Image Formats and Compres

• One of two possible encodings is to be used for all 

– The JPEG Sequential baseline (ISO/IEC 10918

operation and encoded in the JFIF file format format)

– The JPEG-2000 Part-1 Code Stream Format (I

and encoded in the JP2 file format (the JPEG2

– Both JPEG and JPEG2000 work equally well. 

• Face Recognition performance is a strong function 

– Over-compressed images cannot be used for 

background checks, and reduce verification e

– Prevent re-compression

• Best compromise between size and quality is Regio

compression with face region compression ratio of 

• For JPEG, and space concerns, use the allowed YU

where twice as many bits are dedicated to luminan

two colour components. 
© 2005 Identix Inc. All rights reserved. 22

ROI JPEG Compression 

• To be discussed in more detail in a

presentation
Additional Com
3D Face Recog
© 2005 Identix Inc. All rights reserved. 25

3D Face Recognition

• Standards Activities

– INCITS M1 has a new work item to amend 

standard to accommodate depth data.

– A similar technical contribution has been ma

be presented in South Africa this summer.

• Every biometric should have standardized data fo

context, 3D face is no different than hand geome

– This does not imply that 3D face autom

put on passports or in next generation piggyback to 2D face. 

• Performance Testing

– Tested 3D face recognition on cooperative s

Recognition Grand Challenge (FRGC)

– Results:

 http://www.biometricscatalog.org/docum

GC%20-%20Feb%202005-5.pdf
© 2005 Identix Inc. All rights reserved. 26

Current 2D vs. 3D Performance R

Conclusions• Texture (2D Image) is more important than S

– Most of the signal from 3D matching is from the 2D – Addition of shape is less important than increasing r

data.

• Single image matching generally beats match

3D sensors

FRGC Workshop 3• Experiment #1

– Single Face Image

• Experiment #2

– 4 Face Images

• Experiment #3: 

– 3D (Texture + Shape)

• Experiment #3t

– 3D (Texture only)

• Experiment #3s

– 3D (Shape only)
Conformance 

Standards and 
© 2005 Identix Inc. All rights reserved. 28

Conformance – Standards an

• Standards Activities

– INCITS M1 is a new work item to study speci

automatically insure that face data complies wstandards. 

 This new work item is currently being develop

NIST.

– Conformance is for the format (make sure by

right places), photographic, digitization, and srequirements.

– Scene requirements in the standard are to be 

an automated image quality module. 

• Image Quality Module

– Measures scene information to determine con

standard and the probability of good biometr

 Focus, Exposure, Expression, etc.
Example: Identix Face IQM 

QAU Resolution 

Under ExpGood ConWell Focu

Face Image 

(File, 

Sequence)

Others (H

...) Assessment

Detectable Face PreseFace GeomWearing E

Glare)

High ResoProper Lig

Backgro

Head ViewOthers (Na

Livenes

© 2005 Identix Inc. All rights reserved. 29

Face Finder

Face Quality 

Assessment

Image Quality
© 2005 Identix Inc. All rights reserved. 30

Conformance - Summary

• Significant standards activity in de

conformance standards for capturformatting systems

– Need a certification program.– File format conformance specificati

straightforward to determine with a dataset for common formats

– But each biometric will likely have s

algorithms that measure sample da

• Conformance testing is likely to re

sequestered data.

– Testing of relationship between IQ

matching scores

– Testing of data interchange proper
Thank You. Q