The Specification and Specification Specification Specification and Measureme

of Face Image Quality Face Image Quality

Measureme

Presented to: International Biometrics Performance Confer

National Institute of Standards and Technology Gaithersburg, Maryland 

Date: 4 March 2010 Authors: Donald P. D’Amato, Nathaniel Hall, Delia McGar
U.S. Department of State (DoS) Moti

• Face recognition (FR) accuracy is highly depen

face image quality 

• In conjunction conjunction with fingerprint fingerprint matching, DoS is 

DoS is 

FR to screen non-immigrant and immigrant visa applicants and for fraud detection in its Diversit(lottery) program 

• Started using FR with applicants for U.S. passp• Many diverse diverse sources sources for face images exist --- 

resolution scanned photos & online digital subm

• App roachin

g 100M visa-app licant face ima

ges 

• Applicants are required to provide images confo

standards & DoS must verify 
Easy: Quality Assessment with Test Targ

IT8.7/2 Macbeth 

ColorChecker 

USAF 1951 USAF 1951 

ISO-16067-1 

Scanner Test Chart 

 OECF (Capture 

 a

accuracy) - ISO 145

 Spatial Resolution (

depth of field depth of field 

- ISO 

ISO 

ISO 16067-1 

 Signal to Noise Rati

Signal to Noise Rati

ISO 15739 - and spuniformity 

 Color -



E







L



2


250

200

Digital 150

Output

 Values

100

50

0

0% 10% 20% 30% 40% 50% 60% 70%

Target Swatch Reflectance

Measurement of OECF & SFR 

Applied Image, Inc. QA-62-SFR Scanner Test Chart 

1 2 3 4 5 6 

789

1920 7 

8 9 

19 20 

9

10

14 13 12 11 16 15

1718 9 

10 

14 13 12 11 16 15

17 18 16 

250 200 

Digital 150 

Output

 Values 

100 

50 

0 

0% 10% 

Black 

OECFs 

for 3 color

20% 30% 40% 50% 60% 70% 

Target Swatch Reflectance 

SFR – M

for 3 colofor 3 colo

4 
DoS Passport & Visa Photo Guidelin

Traditional Photography Digital Photography 

Many Many 

provide

See http://travel state gov/visa/gSee http://travel.state.gov/visa/g
As of February 2010, all 

nonimmigrant visa applicants at about 72 U.S. Embassy and 

C l t onsulates are requi d t re o 

apply using the online 

CEAC: Consular Electronic Applications 

As of February 2010, all nonimmigrant visa applicants at about 72 U.S. Embassy and C l t i d t Consulates are required to 

apply using the online 

DS-160 form. 160 form.
•

• and visa 

applicants 

•

or portrait mode photos (up to 

64Mpx)

•

and crop using min and max head 

outlines

•

photos locally

• Later upload the cropped photos 

via CEAC or a possible similar Website for passports

Photo Cropping Tool for Applicants 

• Adobe Flash Adobe Flash-based tool based tool• For use by passport For use by passport and visa 

applicants 

• Applicants choose their landscape Applicants choose their landscape 

or portrait mode photos (up to 

64Mpx) 

• Can interactively position, resize, Can interactively position, resize, 

and crop using min and max head 

outlines outlines 

• Can then save the cropped Can then save the cropped 

photos locally 

• Later upload the cropped photos 

via CEAC or a possible similar Website for passports 
Synopsis of ISO 19794-5 Requireme

The face, from the crown to the base of the chin, and from ear-to-ear, shall be clearly visible and free of shadows. ≤ +/- 5 degrees from frontal in every direction – roll, 

pitch and yaw and yaw 

No other face in the image Shoulders “square on” Lighting equally distributed on the face 

After conversion to greyscale, there should be 7 bits of intensity variation in the facial region of the image. 

Face must be in focus from nose to crown. Sufficient depth of focubetter than two millimeter resolu

The iris and pupil of the eyes shvisible. visible. Eyeglasses shall allow the eye pto be clearly visible. There shall be no li

ghtin

g artifac

reflections on glasses. 
 

Desired Metrics and Their Character

1. Dynamic range in face2. Eyes closed/obstructed3. Color balance4. Lighting uniformity on face5. Background consistency6. Head width to imagewidth

ratio

7. Centering of face8. Distance between eyes

9. Focus/Sharpness of face10. Rotation (yaw)11. Tilt (roll) Tilt (roll)

12. Confidence in face13. Brightness and contrast

 Reasonably shaped 

unacce

poor good 

 Good correlation with 

Good correlation with 

perception



 Correlation with FR m Well defined algorith
Assessing Face Metric Performance 

• In 2006, Noblis tested four products (SDKs) that 

to measure face image quality

– Tested using same set of passport visa and other pho

Tested using same set of passport, visa, and other pho

– Comparison proved difficult – lack of standardization in

terminology & insufficient description of algorithms

– Some products had poorly distributed histograms – thre

determination sometimes proved problematic

– Values exhibited little exhibited little correlation with match correlation with match scores– In some cases, values had no obvious correspondence 

perceived quality

• D Si i f h d f i D

oS is now using one o

f t

hese pro

ducts 

for imag

checks before FR and during online photo subm
Record 2 

Metric Value 

Dynamic 34 

range Eyes closed/ obstructed Color balance Lighting uniformity Background consistency 

Record 1 

Results from Sorting Metric Values - by Dynamic Range 

...... 

Record n-1 

...... 

Metric Value 

Dynamicrange 

29 

Eyes closed/ obstructed Color balance Lighting uniformity Background consistency 

Metric Value 

Dy

Dynamic 

range 

89 

Eyes closed/ obstructed Color 

balance Lighting uniformity Background consistency consistency 

Bacoco

Eyob

CobaLiunbara
DyraEyobCobaLiun

Record 2 

Metric Value 

Dynamic range Eyes closed/ obstructed Color balance Lighting uniformity 

2.2

Record n-1 

Metric Value 

Dynamic range 

...... 

Background consistency 

Baco

Results from Sorting Metric Values - by Lighting Uniformity 

...... 

Metric Value

Dynamic range 

Record 1 

Background consistency consistency Lighting 0 

uniformity 

Eyes closed/ obstructed Color balance 

Eyes closed/ obstructed BackgroundconsistencyLighting uniformity 

100 

Color balance 
Results from Sorting Metric Values - by 

Background Consistency 

Record n-1 

DynranEyeobsColLighunifBaccon

Record 2 

Metric Value 

Dynamic range Eyes closed/ obstructed Color balance Lighting uniformity Background consistency 

21 

Record 1 

...... 

Metric Value

Dynamic range Eyes closed/ obstructed Color balance Lighting uniformity Background consistency 

0 

Metric Value

Dynamic range Eyes closed/ obstructed Color balance Lighting uniformity Background consistency 

98 

...... 
ness st ing ess ure ikeness es ution 

Brightne Contras Cropping Darknes Exposur Face like Focus Glasses Resoluti Te

Brightness 1 Contrast -0.07 1 Cropping 0.01 0.03 1 Darkness -0.12 0.33 0.03 1Exposure 0.86 0.11 0.02 0.4 1 Face likeness -0.05 0.26 0.17 0.28 0.09 1 Focus 0 0.70. 090. 020. 040. 09-0 1

Glasses -0 -0.01 -0.34 -0.02 -0.01 -0.08 -0.02 1 Resolution -0.04 0.05 -0.3 0.04 -0.02 -0.14 0.05 0.21 1 Texture -0.04 -0.06 0.02 -0.03 -0.05 0.01 0.11 -0 0.05 Face Confidence -0.01 0.13 0.1 0.13 0.06 0.34 0.08 -0.06 0.02 0.0

Correlations Among Quality Factors 
Minimum 

etric Value Face QualityM

for Probe and its Mate

100

Minimum Probe/Mate Face Quality Value 

versus True Match Score

80604020

0

y = 0.1538x + 71.108

R2 = 0.0102

0 10 20 30 40 50 60

True Match Score

Minimum etric Value Face QualityMe 

for Probe and its Mate 

100 

Minimum Probe/Mate Face Quality Value 

versus True Match Score 

80 60 40 20 

0 

 y =0.1538x + 71.108

R2 = 0.0102 

0 10 20 30 40 50 60

True Match Score 

70

Correlation with Match Scores

Low cofound fmetric, have focorrelacorrela

same vscores 

multi -va

regresscombin

quality quality 

70
Training & Testing Databases 

• Pose, illumination, & expression databases 

– Yale Face Database [Georghiades et al., IEEE PAMI, 2

– CMU PIE D t b f H F CMU PIE 

D

a

t

a

base of 

Human 

Faces 

[Sim et al., CMU-RI-TR-01-02, 2001] 

• PIE, etc. database characteristics --- should have: 

– Only a single changing variable for each sequence – Good resolution, accurate color, plain background, dive

representative ethnicities 

• Alternative might be to categorize many pre-existing visa o

photos – but 

privacy issues mi

ght 

prove 

prohibitive for dist

the database 

• Can artificially generated or suitably altered faces be empl
 Vendors are very reluctant to divulge their algorithms -

need the code, but we do want very clear descriptions

 Some flesh tones, hair styles and head coverings have re

Some flesh tones, hair styles and head coverings have re

rejected images that are otherwise acceptable

 To accommodate the wide range of flesh tones, head cov

from consular posts all over the world we have had to lowfrom consular posts all over the world, we have had to low

thresholds for some metrics to the point that they’re no lovalue

 Providing clear feedback to applicants and face examiner

defective images has proven difficult 

 Combining p multi le metrics into a sing , y le readily understo

face image quality value has been particularly difficult

 Finding evidence of intentional digital alteration (i.e., 

Some Challenges 

 Vendors are very reluctant to divulge their algorithms w

- w

need the code, but we do want very clear descriptions 

 Some flesh tones, hair styles and head coverings have re

Some flesh tones, hair styles and head coverings have re

rejected images that are otherwise acceptable 

 To accommodate the wide range of flesh tones, head cov

from consular posts all over the world we have had to lowfrom consular posts all over the world, we have had to low

thresholds for some metrics to the point that they’re no lovalue 

 Providing clear feedback to applicants and face examiner

defective images has proven difficult 

 Combining multiple metrics into a single, readily understo

face image quality value has been particularly difficult 

 Finding evidence of intentional digital alteration (i.e., Phot

Phot
 Insist upon best practices for image capture and enforce t

periodic measurement with test patterns

 P t i i d t ti d t b f f i repare training and testing databases of face images wi

assessment of image quality

 Request clear descriptions of face quality metrics from the 

Request clear descriptions of face quality metrics from the 

providers

 Conduct objective testing of the accuracy of face quality m

Conduct objective testing of the accuracy of face quality m

similar to FRVT/FRGC

 Consider develop p ment and publication of a face imag q 

eu

standard, similar to NFIQ, as well as additions to the NIST Image Software (NBIS) suite for face image processing

Recommendations 

 Insist upon best practices for image capture and enforce t

periodic measurement with test patterns 

 Prepare training and testing databases of face images wi

assessment of image quality 

 Request clear descriptions of face quality metrics from the 

Request clear descriptions of face quality metrics from the 

providers 

 Conduct objective testing of the accuracy of face quality m

Conduct objective testing of the accuracy of face quality m

similar to FRVT/FRGC 

 Consider development and publication of a face image qu

standard, similar to NFIQ, as well as additions to the NIST Image Software (NBIS) suite for face image processing 
Thanks for your attention..... 

Contact Information: 

Donald P. D’Amato Noblis, Inc. 3150 Fairview Park Drive South 3150 Fairview Park Drive South Falls Church, VA 22042 USA ddamato@noblis.org 703-610-2463 www.noblis.org