BIOMETRICS LAB

Biometric Standards, Performance and Assurance Laboratory

Department of Technology, Leadership and Innovation

EVOLUTION OF THE HBSI 

MODEL 

IPBC 2012 CONFERENCE PRESENTATION

14:25-14:50
CONTRIBUTORS TO THE 

PRESENTATION 

• Michael Brockly

• Kevin O’Connor

• Carl Dunkelberger

• Tyler Veegh

• Thomas Cimino

• Amanda Simpson

• Rob Pingry

• Brent Shuler

• Jacob Hasslegren

• Rob Larsen

• Chris Clouser

• Dan Vander Wall

• Todd Walters

• Craig Hebda

• Weng Kwong

Chan

• Tony Fuji
PRESENTATION

• Evolution of the model

• HBSI v3.0

• Future roadmap
DEVELOPMENT OF THE MODEL

• The HBSI model is concerned with the 

data collection portion of the biometric 

model

– Consistent and repeatable presentation to 

the sensor
HBSI MODEL 

Conceptual model for HBSI 

Human

Biometric 

System Sensor

Ergonomics

Usability

Image Quality

Human-Biometric

Sensor Interaction (HBSI)
UNDERLYING MODEL
MODALITY TESTING AND HBSI

Year Hand Finger Iris Face DSV

2004 Age Mobile iris Illumination Different devices

2005 Co-Rec

2006 Height /Placement

2007 Habituation Force 

2008 Gender

2009 Initial HBSI Calc Force 

Training

2010 Fixed iris

2011 Gender Device (different 

sensors) 

2012 Hand alignment Force 

Finger interactions / 

Kinect

HBSI

Training / 

Kinect

Detractors Forgery

2012 InteractionAge InteractionAge InteractionAge InteractionAge InteractionAge
MODEL DEVELOPMENT - V1
MODEL DEVELOPMENT - V2
INCLUSION OF OTHER MODELS

• General Biometric Model

• Operation Times Model (Lazarick, 

Kukula, et.al)
Fuji, et. al (2011)
HBSI METRICS V2

Metrics created and validated for:

• Iris

• Fingerprint

• Signature Verification
HBSI METRICS V2

Metrics created and validated for:

• Iris

• Fingerprint

• Signature Verification

Video record the 

environment from 

different angles in 

order to watch the 

subject and to 

classify their 

presentation

-typically 3 video 

angles and operator 

screen + (audio 

sometimes)
HBSI METRICS V2

Metrics created and validated for:

• Iris

• Fingerprint (different sensors)

• Signature Verification

Record the 

environment (video 

and sometimes 

audio) from different 

angles in order to 

watch the subject 

and to classify their 

presentation

-typically 3 video 

angles and operator 

screen
BIOMETRICS LAB

Biometric Standards, Performance and Assurance Laboratory

Department of Technology, Leadership and Innovation

HBSI MODEL 3.0
UNDERLYING MODEL
UNDERLYING MODEL EXAMPLES

More actors:

• Subject (typically 

the biometric donor)

• Operator 

• Other people in the 

environment
UNDERLYING MODEL EXAMPLES

Additional variables: 

Subject conditions

• moisture, 

• elasticity, 

• oiliness

Collected conditions 

such as:

• temperature 

• humidity
EXAMPLES: HAND GEOMETRY 

ACCESSIBILITY

Biometric Biometric Donor

Sensor 

operator

Climate Device

Test Administrator

Training
EXAMPLES: HAND GEOMETRY 

ACCESSIBILITY

Biometric Biometric Donor

Sensor 

operator

Climate Device

Test Administrator

Training
EXAMPLE METRIC CALCULATIONS

Biometric Biometric Donor

Sensor 

operator

Climate Device

Test Administrator

Training
DETERMINATION OF ERRORS V1

• Process:

– Recorded in real time as the study is 

underway

– Interactions are coded

– Metrics of the evaluation model are 

completed

– Interaction errors are classified as HBSI 

terms
CURRENT WORK

• Generation of the errors is time consuming

• We notice other potential errors 

• Contribution of the operator to the error

• Contribution of the test administrator to the error

• HBSI workflow

• Semi-automatic coding of the model using Kinect

• New work 

• Accessibility study – hearing and sight impaired (started Jan 2012) 

• Contribution of cost to the model (started Jan 2011)

• Examining the role of the impostor (thinking .... As this model 

only has been rested in a “genuine” environment)

• Development of products that can help improve interactions
ASSIGNING A COST MODEL

• Identify interaction issues

• Classify where these errors are 

occurring – what is causing this

• Assign a cost to “retry” for example –

based on:

– Poor interaction

– Sensor feedback

– Operator not paying attention

• Assess the impact on fixing this error
BIOMETRICS LAB

Biometric Standards, Performance and Assurance Laboratory

Department of Technology, Leadership and Innovation

KINECT

CRAIG HEBDA | ROB PINGRY | WENG KWONG CHAN |

BRENT SHULER
MOTIVATION

• Video coding is time consuming

• Inter-rater reliability

– Requires good robust definitions
DEFINING MOVEMENT

Jake Hasselgren (2011)

Slouched: Subject is not standing up straight during fingerprint scan.

Head Movement: Subject’s head is not still during fingerprint scan.

Body Movement: Subject’s body is not still during fingerprint scan.

Upright: Subject is standing up straight during fingerprint scan.

Labored Walking: Subject has bag or other item on shoulder when approaching device

Pivoting Palm: Subject’s hand pivots on edge of device

Rocking Fingers: Subjects fingers rock from one finger to the next when hand is placed on 

device 

Slapping Hand: Subject slaps hand on to the device

Angled Fingers: Subjects fingers are at an angle other then 90 degrees from edge of the 

device 
MICROSOFT ® SDK INTERFACE
MICROSOFT ® KINECTTM SKELETAL 

TRACKING SYSTEM

Source: http://msdn.microsoft.com/enus/library/hh438998.aspx

Source: 

http://www.genbetadev.com/herramientas/disponible-elsdk-de-kinect-para-desarrollar-nuestras-propiasaplicaciones-usando-los-sensores
SLOUCHING

• Dictionary Definition

• Slouching-A gait or posture characterized by an 

ungainly stooping of the head and shoulders or 

excessive relaxation of body muscles.

• Points of interest:

• -Shoulders

• -Head

• -Spine

• -Hips

Source:http://www.merriam-webster.com/dictionary/slouch
SLOUCHING

• Tracking Points to be used:

• Shoulder_Right

• Shoulder_Left

• Shoulder_Center

• Head

• Spine

• Hip_Center

• Hip_Right

• Hip_Left
SLOUCHING

• Left Slouching: • Left shoulder will be 

lower then the right 

shoulder. 

• All points on left arm 

will be lower then 

base image.

• Head will be tilted to 

left.

• Left hip will be lower 

then right hip.

• Spine point will move 

slightly up and right.

Highlight what is 

slouching

Break it into left right 

 =Movement Up

 =Movement Down
SOLUTIONS TO THE CHALLENGES 

OF DEFINING 

• A multi point approach can help solve 

the majority of the problems when 

describing what is slouching.

• Use a combination of how much each 

point moves to determine if the subject is 

slouching or just moving one part of their 

body.
HEAD DISPLACEMENT

• Definition of Head Movement:

• Voluntary or involuntary motion of head 

that may be relative to or independent of 

body.

• http://www.medical-dictionary.cc/whatdoes/head-movement-mean
HEAD DISPLACEMENT

• Critical Tracking Points (TPs):

1. Head (H)

2. Shoulder_Center (SC)

• Associated Tracking Points 

(TPs):

1. Shoulder_Right (SR)

2. Shoulder_Left (SL)
DEFINITION OF HEAD MOVEMENT 

BASED ON TRACKING POINTS

Head Movements Tracking Points Definition Changes in 

Coordinates

Lowering head H approaches SC X, Y, maybe Z too

Nodding H moves back and forth from SC repeatedly X, Y, maybe Z too

Head turning Turn to the left: H moved to the left X, Y, Z

Turn to the right: H moved to the right 

Head tilted to one side Tilt to the left: H moved to the left X, Y

Tilt to the right: H moved to the right

Head bobbing H moves in random direction with minimal distance X, Y, Z

Head sliding forward H moves forward Z

Head wagging H moves in left and right rapidly X, Y

Source: http://www.thefreedictionary.com/Head+Movements
HOW WILL THIS WORK?

A QUICK ROADMAP

Observation of 

error

Automatic 

identification 

and 

classification of 

error

Feedback to 

the user, 

customized to 

their interaction 

error
ACTIVITIES

• Link behavior and interaction to the 

image

• Understand the basic performance 

characteristics

• Relay back whether the interaction (or 

change in interaction) affects 

performance
The benefit is to examine 

the information associated 

with the sample, but also 

the video interaction of the 

image. 

HBSI V3 has (will have):

• video and audio 

interaction

– Watch the interaction

– Understand who is 

contributing the error

– Replay the interaction in 

real time as it was 

collected

• Metadata collected and 

searchable
QUESTIONS?

• Other actors

• Contribution of the operator to the error

• Contribution of the test administrator to the error

• HBSI workflow

• Semi-automatic coding of the model using Kinect

• New work 

• Accessibility study – hearing and sight impaired (started Jan 2012) 

• Contribution of cost to the model (started Jan 2011)

• Examining the role of the impostor (thinking .... As this model only 

has been rested in a “genuine” environment)

• Development of products that can help improve interactions

Get Involved in shaping these projects – contact elliott@purdue.edu to participate in the 

development of the model

Teleconferences over the summer 2012 period
BIOMETRICS LAB

Biometric Standards, Performance and Assurance Laboratory

Department of Technology, Leadership and Innovation

ADDITIONAL SLIDES