Multiple Biometric Grand 

Kick-Off Worksho
MBGC Sponsors 

Director of National Intelligence Intelligence Advanced Research Projects Activity

Executing Agency

Sponsoring Agencies

– Science & Technology D

– Criminal Jus– Operational 
The MBGC Team

• NIST

– P. Jonathon Phillips, Test Director– Elaine Newton

• Colorado State University

– Ross Beveridge & Geoff Givens

• SAIC

– Todd Scruggs

• Schafer Corporation

– Cathy Schott

• University of Notre Dame

– Kevin Bowyer & Patrick Flynn

• University of Texas at Dallas

– Alice O’Toole
Overview• Background• MBGC Overview

– MBGC Goal– MBGC Structure– Challenge Problem Descriptions– How to get the MBGC Data
Challenge Problems• What are challenge problems?

– A series of experiments designed to 

technology’s state-of-the-art

• Experiments designed• Experiments and test data distributed t• Researchers complete experiments an• Scores are consolidated and reported

– Introduction of new technology

• Specific to MBGC
Challenge Problem• What is provided to participants

– Data (biometric samples)– Ground truth (subject ids)– Experiment description– Score code– Standard results format

• What is requested from participa

– Submission of results
New Technology Developme• Very high resolution Near Infrare

sequences

• Iris video• Fusion of multi-modal video• Unconstrained face recognition• Realistic mug shot and passport 
What to Expect from the MBG• MBGC will determine performanc

improvement on face and iris rec

• Results will measure:

– Improvement obtained from video – If more samples of existing mode im

performance

– Effect of resolution on still face– Improvement obtained through fusio

iris
Background

• FERET 1993-1997

– Army Research Lab challenge problem/evaluation

• Face Recognition Vendor Test (FRVT) 2000

– DARPA, NIST, DoD CTDP and NIJ Evaluation

• FRVT 2002

– DARPA, NIST and DoD CTDP Evaluation

• HumanID Gait Challenge 2002-2003

– DARPA Challenge problem

• Face Recognition Grand Challenge 2004-2006

– NIST directed challenge problem

• FRVT 2006

– NIST directed evaluation

• Iris Challenge Evaluation (ICE) 2005-2006

– NIST directed challenge problem and evaluation

• Multiple Biometrics Grand Challenge (MBGC) 

– NIST directed challenge problem
1996

2006

2008 008

Technology Progress

2

1993 1993
Improved FR Performance

Face Recognition Error Rate
Iris Recognition• ICE 2005 and 2006

– First challenge problem and indepe

technology evaluation on multiple a

– Provided performance baseline– Patent expiration allowed broader te

development
What does progress mean?• FERET, FRGC and FRVT 2006

– Controlled images:
What about iris recognition?• ICE 2006 images • MBGC 
What about these images?• MBGC plans to address these ty

images:
Multiple Biometric Grand 
MBGC Goal• The main goal of the Multiple Biometri

Challenge (MBGC) is to

– Address face and iris recognition 

that are more relevant to those fooperational data

• Low to medium resolution face• Still and video iris• Near Infrared (NIR) & High Definition 

portals

• Unconstrained face recognition from 
MBGC Goal• Programmatic Method

– Sequence of challenge problems 

• Modeled after the FRGC and ICE 200• Challenge problems and data distrib

researchers

– Workshops– Planned independent technology 
Timeline• Estimated Task Schedule for MBGC: 

Estimated Schedule Task

December 2007

Formal annoucement of MBGC project Begin data collection at University of NotDesign protocols, challenge problems a

April/May 2008 1st MBGC Workshop Release 1st challenge problem 

October 2008

2nd MBGC Workshop Self-reported results on Challenge ProbRelease 2nd Challenge Problem 

Spring 2009 3rd MBGC Workshop Self-reported results on Challenge Prob

Spring/Summer 2009 Planned Multi-Biometric Evaluation (MBE
MBGC Challenge Problems

Portal Video

St

Mugshot

Video Face 

Worked with sponsors over a 12-month pproblems
Portal Video

St

Mugshot

Video Face 

Portal Challenge Problem

Portal Video
Motivations• Rapid acquisition of biometrics a

entry

• Non-invasive capture
Portal Goals

• Algorithm development

– Biometrics: face and iris– Modes: iris video, NIR video, HD 
Meet the Sensors...
Meet the LG 2200

LG EOU2200

• Take 3 iris images• One above quality threshold• Save all three

Sufficient 

Quality

MBGC Iris Acquisition System
Meet the Portal
Meet the Portal
Meet the Portal

High 

Definition 

(HD) Video 

Camera
Meet the Portal

High 

Definition 

(HD) Video 

Camera

Near Infrared 

(NIR) Video 

Cameras
Meet the Portal

Near Infrared (NI

12

ac

2k

2k
Experiment Terminology

• Target

– Enrolled image

• Query

– Image to be recognized
Experiment: Still Iris versus NIR

LG 2200 Still Near Infrar

Target Query

Matching

Similarity Score
Experiment: Still Iris versus NIR

Near 

LG 2200 Still

Target Query

Matching

Similarity Score

• Algorithm 

• Proce

• Locat• Segm• Generother R

Note: Input wilGround truth p
Meet the LG 2200 again....

LG EOU2200

Iris Video Sequence

LG EOU22
Experiment: Video Iris versus NIR

Near 

Target Query

Matching

Similarity Score

LG 2200 Video
Experiment: Video Iris versus NIR

Near 

Target Query

Matching

Similarity Score

LG 2200 Video

Both inputs will be full video sequences

• Algorithms will need to:

• Process Video

• Locate Irises• Segment Irises• Generate Templates or other Representation
Meet the Portal again....

High Definition 
Experiment: Still Face versus HD Video Face

Target Query

Matching

Similarity Score

High Definitio

Face Still
Experiment: Multiple Biometrics

Target Query

Matching

Similarity Score

Ne

High 

LG 2200 Video

Face Still
Experiment: Multiple Biometrics

Target Query

Matching

Similarity Score

Ne

High 

• Algorithms will need to:

• Process all video seque

• Generate templates or o• Decide fusion strategy

Input is 

LG 2200 Video

Face Still

Input is iris video 

sequence and face still
Quick Summary

Target Query

LG 2200 Still Near Infrare

Still Iris versus NIR

LG 2200 Video Near Infrare

Video Iris versus NIR

High Defini

Face Still

Still Face versus 

HD Video Face

N

Hig

LG 2200 

Face Still Video

Multiple Biometrics
Considerations you may not thought of......
Experiment: Cross-Mode Face

Target Query

Matching

Similarity Score

Face Still

Near 

Near Infrar
Experiment: Multi-Mode Face

Target Query

Matching

Similarity Score

Face Still

Near I

High D
Experiment: Multi-Mode Face

Target Query

Matching

Similarity Score

Near Infrar

Face Still

LG 2200 Video Near 
Portal Summary• Introduce new technology• Seven experiments• Multiple biometrics• Assess impact of fusion
Portal Video

St

Mugshot

Video Face 

Still Face Challenge Problem

Still Face

Mugshot ICAO-
Frontal Face• FERET, FRGC and FRVT 2006

– Controlled images:
Frontal Still Face• Unconstrained Illumination:

– High resolution camera

• 4 and 6 megapixels

• Images for mug shots, passports 

and credential cards

– Face size 90-120 pixels between th

eyes compressed to 8k and 20k

• ISO/IEC Standard
Still Face• Two target sets – AY03-04 (FRG

– Controlled Illumination Frontal 

• 16,028 images

– Uncontrolled Illumination Frontal 

• 8,014 images

• One query set – AY04-05

– Uncontrolled Illumination Frontal

• 13,785 images

– Outdoors– Hallways and Atriums
Experiment: Still Face versus Still Face

Target Query

Face Still

Uncontrol

Face Sti

Uncontrol

Face Sti

Target Set QTarget Set Q

Uncontrolled 

Face Still

Uncontrolled 

Face Still
Effects of Lower Resolution Compression on FR• Determine effect of

– Lower resolution– Compression 

• Process original images to meet 

requirements

– Scale and crop

• 120 and 90 pixels between the eyes

– Compress images

• 20k and 8k
Still Face Processing

90 Pixels 

between the eyes

120 Pixels 

between the eyes

C

Scale / Crop

Original Image

I

Face Still
Experiment: Still Face Compression 1

Target Query Comp

Target Set QTarget Set Q

Compressed: 120 pixels between 

ressed

Face Still

Uncontrolled 

Face Still

Uncontrolled 

Face Still

Uncontrolled 

Face Still

Uncontrolled Uncontrolled Face Still

Face Still
Experiment: Still Face Compression 2

Target Query Compressed

Target Set QuTarget Set Qu

Compressed: 90 pixels between e

Uncontrolled 

Face Still

Uncontrolled 

Face Still

Face Still

Uncontrolled 

Face Still

Uncontrolled Uncontrolled Face Still

Face Still
Still Face Summary• Varying illumination• Measure effects of compression 

resolution

• Three experiments

– High resolution– Two scaled and compressed
Portal Video

St

Mugshot

Video Face 

Video Face Challenge Proble

Video Face 
Goal and Motivition

• Unconstrained face recognitio

– Still versus video– Video versus video
Experiment: Video versus Video

Target Query

Walking Co
Experiment: Still versus Video

Uncontrolled Face Still

Target Query

Fac
Video Summary• Video to video• Addressing unconstrained face re

Walking Conversation Uncontrolled Face Still 
Quality
Motivations• Ensure control over enrollment p

– Do you get images that give you go

• Confidence level in matching for 

images

– Reliability in producing a good matc– What consitutes a good quality mea
Compute Image Quality Still Images

Performer’s

Image Quality

Module

Input Iris Image

Performer’s

Image Quality

Module

Input Face Image
Image Quality• Optional submission• Quality score per biometric sam
Compute Image Quality Video

Performer

Image Qua

Module

Input NIR

Input Iris video

Input HD Video

Performer

Image Qua

Module

Performer

Image Qua

Module
Compute Image Quality Multi-Modal

Input

Performer’s

Image Quality

Module

Input

Face 

Still

LG 2200 Video

Performer’s

Image Quality

Module

Near Infrared (NIR)

High Definition (HD) Video
Quality Summary• Optional part of MBGC• Assessment of quality measures• Impact on fusion
MBGC Mechanics
Timeline• Estimated Task Schedule for MBGC: 

Estimated Schedule Task

December 2007

Formal annoucement of MBGC project Begin data collection at University of NotDesign protocols, challenge problems a

April/May 2008 1st MBGC Workshop Release 1st challenge problem 

October 2008

2nd MBGC Workshop Self-reported results on Challenge ProbRelease 2nd Challenge Problem 

Spring 2009 3rd MBGC Workshop Self-reported results on Challenge Prob

Spring/Summer 2009 Planned Multi-Biometric Evaluation (MBE
What is provided by us....• Challenge problem data• Subject IDs• XML sigset description• Common results format• Classes to read/write formats• Scoring code• Bash script for running experime

– Same arguments as FRVT 2006 & 
General Rules for Submitting• Submit

– Matching results (Similarity matrix)– Quality scores [optional]

• Can submit subset of experiment• NIST reserves right to attribute p
Challenge Problems and DatDistribution• Challenge problems and the first 

set will be released sequentially s1 May 2008

• To obtain data, you will need to s

licenses signed by proper legal a

• Instructions for requests will be p

MBGC website at http://face.nist.
Estimated Size of Challenge Pr• Portal

– 600 video sequences– 1400 stills– 100 Gigabytes

• Still face

– 150,000 stills– 75 Gigabytes

• Video face

– 1500 video sequences– 1500 stills– 150 Gigabytes
Challenge Problems and DatDistribution• What is the method of distribution

– Downloadable via rsync– Instructions for downloading data w

after 

• Data licenses have been properly signe

and your request has been processed
Summary• Multiple Biometric Grand Challen• Three challenge problems• Multiple biometrics for portals• Towards unconstrained face reco

Portal Video Still Face

ICAO-ISO 

Standard

Mugshot