Towards a Mobile Biometric 

Test Framework 

NIST IBPC | 8 March 2012 

Presented by: Eric Kukula, PhD & Frank Shaw

Noblis Team Members: 

Eric Kukula, Technical Lead & Project Manager 

Ann Breckenkamp, Emily Keener, George Kiebuzinski, 

Larry Nadel, PhD, Frank Shaw & Rachel Wallner 

DHS S&T Team Members: 

Patty Wolfhope, DHS S&T Biometrics Transition Program Manager 

Ryan Bednar, Rick Lazarick & Brad Wing 

This work is sponsored by DHS S&T HSARPA 

Human Factors Division
2 

Background 

Why test mobile 

biometric devices? 

How was the test framework 

developed? 

Who will use the test 

framework and 

repository? 
3 

Why Test? 

Image references included as hyperlinks. Images without hyperlinks were taken by Eric Kukula. In 

no case do such references imply recommendation or endorsement by Noblis or DHS S&T. 
4 

S 

D 

D 

d d 

Methodology & Roadmap 

T&E for Operational Pilots 

Documentation of state-ofthe-art & MBHD Taxonomy 

Development 

Methodology development for the 

Test Framework & Requirements 

Formalization of use cases 

Top-down & bottom-up development d of requirements Test Framework Development 

User Workshop 

Development of Prototype Repository 

Gap Analysis / Roadmap 

Development (and solicitation) of Test 

Methods & Pilot Testing 

Requirements & Test 

Framework Mapping 
5 

MBHD Taxonomy 
6 

MBHD Expanded Taxonomy 

System

Subsystem Form Factor Biometrics Data Input Platform Communication

Chassis Imager (size/characteristics) Keyboard Processor & Memory Wired Connectivity

Ingress Protections Camera Programmable CPU RS-232*

Battery Casings Sensor Trackpad Memory Ethernet*

Access Panels Other Mouse Power USB*

External Connectors Processor/Controller Touchscreen Battery Firewire*

Switches Imager Housing Stylus Charging Circuit Docking Station Interface*

Frame Microphone Charge Status Indicator Wiegand Interface*

Seals Readers Charger Interface Wireless Connectivity

Protective Coating Magnetic Stripe Docking Station Interface* PAN

Illuminator Bar Codes Output BlueTooth

Optical Smart Card Speaker Body Area Networks

Flash RFID Printer ZigBee Multi-Spectral MRZ / OCR Display Device LAN

I R Other Backlight IEEE 802.11 a/g/n

Storage IEEE 802.11af

Internal WAN

Fixed GSM/GPRS/EDGE/UMTS

External 1xEV-DO

Remove HSPA and HSPA+

Interfaces WiMAX (IEEE 802.16e and IEEE 802.16m)

SAM LTE and LTE-Advanced

SDIO Mobile Satellite Communication Systems Memory Expansion Global Navigation Satellite Systems (GNSS)

RS-232*

Ethernet*

USB*

Firewire*

Docking Station Interface* Wiegand Interface*

Feedback

LEDs

Symbols/Pictograms

Aural

Tactile (Haptic)

N/A Data Acquisition Acquisition Operating System Network Management Protocols

Signal Processing Encoding/Decoding Applications Secure Communications

Segmentation Metadata General Status Monitoring Mobile Virtual Private Network

Quality Dynamic Workflow Manager

Feature Extraction Output Formatting

Template Generator* Formatting/Template Matching Compression

On-Board (Biometric Module) Encryption

Host/API/Software Transmission Workstation Template Generator*

CMS Security

Data Management Physical Access Control

Storage Logical Access Control

Case Management Hard Drive Encryption

Template Generator* Cryptography 

Interface Control Template Generator*

Biometric Status Monitoring Protocol Management

Dynamic Workflow Manager

Spoofing/Evasion

Liveness

Biometric

*Exists in multiple subsystems

Hardware 

Components

Software 

Components
7 

COTS Devices Mapped to the Taxonomy 

 Analyzed over 30 COTS MBHD devices* 

*Trade names and company products have been listed in the text above. In no case does such 

identification imply recommendation or endorsement by Noblis or DHS S&T, nor does it imply that the 

products are necessarily the best available. 
8 

User Workshop :: 31 March 2011 

Tucson Border Patrol Sector HQ

 Report for the Mobile Biometric Technology Workshop for Developing a Test 

Framework and Supporting Requirements 

• Workshop context, rationale, and purpose 

• User Survey Results and Analysis 

• User/Participant Presentation Summaries 

• Use Cases 

• Scenarios 

• Mapping of Scenarios to Use Cases 

• Results 

• Conclusions 

• Recommendations 
9 

Consolidated Use Cases 
10

Requirement Profiles & Sub-Profiles 

Indoor in NIST 

SP 500-280

Law Enforcement in 

NIST SP 500-280

Military in NIST 

SP 500-280
11

Requirements Methodology 
12

Operational Requirements 

Strategic Goals Through Customer Requirements 
13

Linkage of the Taxonomy to the Test 

Framework 
14

Test Framework Overview 

 Currently only houses 

component-level tests for the 

biometric subsystem 

 Organization based on general 

biometric model subsystems 

• Data Input 

• Signal Processing 

• Data Storage 

• Matching 

• Decision 

 New subsystem 

• Data Formatting 

 Types of testing based on existing 

test programs and reports 
15

Structure of the Test Framework 

 Each subsystem has 3 components 

• Framework 

– Structure for user interaction 

– Relationship between products tested vs. tests passed 

• Description 

– Description of the purpose of each test 

– Breakdown of test structure within the repository 

• Methods 

– Breakdown of test methods for each test 

– Where applicable, metric(s) and threshold(s) are specified 
16

Test Framework Organization & Navigation 

Example :: Appendix F 
17

Test Framework Organization & Navigation 

Example :: MINEX
18

Prototype Repository 

 Place for storage of the test framework information and mobile biometric 

device requirements. 

 Provides methods of user interaction and navigation through information 

 Role-based access 

• Acquisition Personnel 

• Testing Laboratories 

• Vendors and Manufacturers 

 Built using LabKey Software open-source framework* 

*Trade names and company products have been listed in the text above. In no case does such 

identification imply recommendation or endorsement by Noblis or DHS S&T, nor does it imply that the 

products are necessarily the best available. 
19

Next Steps 

 Integrate all levels of test integration to the test framework 

• Subsystem 

• System 

• System-of-Systems 

 Map requirements to the test framework 

• Using metrics and thresholds 

• Maps to engineering space requirements (functional, performance, derived) 
20

S 

D 

D 

d 

d d 

Methodology & Roadmap 

T&E for Operational Pilots 

Documentation of state-ofthe-art & MBHD Taxonomy 

Development 

Methodology development for the 

Test Framework & Requirements 

Formalization of use cases 

Top-down & bottom-up development 

of requirements Test Framework Development 

User Workshop 

Development of Prototype Repository 

Gap Analysis / Roadmap 

Development (and solicitation) of Test 

Methods & Pilot Testing 

Requirements & Test 

Framework Mapping 
21

Requirements Traceability Matrix 

ID Performance Requirement Condition Source Biometric 

Subsystem

Backward 

Traceability 

(corresponding 

functional 

requirements)

Forward 

Traceability 

(corresponding 

derived 

requirements)

Corresponding 

Test(s)

P 6

The mobile device shall have a minimum FAP level of 

____ as specified in the most current version of 

ANSI/NIST-ITL

F1.1, F1.2, F1.3, 

F1.4

P 7

The mobile device shall capture a single flat fingerprint 

in less than 3 seconds

F1.1, F1.2, F1.3, 

F1.5

ID Functional Requirement Condition Requirement 

Source

Biometric 

Subsystem

Backward 

Traceability 

(corresponding 

customer 

requirements)

Forward Traceability 

(corresponding 

performance 

requirements)

Corresponding 

Test(s)

F1.1

The mobile device shall capture a single flat fingerprint image for 

use in identification C1 P6, P7

F1.2

The mobile device shall capture a single flat fingerprint image for 

identity verification C2, C3, C9 P6, P7

F1.3

The mobile device shall capture a single flat fingerprint image for 

enrollment in a fingerprint database C4, C5 P6, P7

F1.4

The mobile device shall capture a single flat fingerprint image for 

documentation C4, C5 P6, P7
22

S 

D 

D 

d 

d d 

Next Steps 

T&E for Operational Pilots 

Documentation of state-ofthe-art & MBHD Taxonomy 

Development 

Methodology development for the 

Test Framework & Requirements 

Formalization of use cases 

Top-down & bottom-up development 

of requirements Test Framework Development 

User Workshop 

Development of Prototype Repository 

Gap Analysis / Roadmap 

Development (and solicitation) of Test 

Methods & Pilot Testing 

Requirements & Test 

Framework Mapping 

Biometric Interagency 

Testing and Evaluation 

Schema (BITES)* 

*For more information on BITES, see http://biometrics.nist.gov/cs_links/ibpc2012/presentations/Day3/333_Lazarick-Wolfhope_IBPC.pdf 
23

Benefits 

 Improved testing efficiency and thoroughness 

• Traceability between devices, requirements and test 

methods 

 Uniformity of test methods to support sharing 

between agencies and programs 

 Addresses challenges laid out in the NSTC National 

Biometrics Challenge Document 

• Repository of test methods and results 

• Lowers costs by reusing test procedures and certifications 

• Development of testing and evaluation methodologies 

• Development of frameworks for test data and results 
24

Thank You For Your Attention 

Questions? 

Contact Information: 

 Patricia Wolfhope, DHS S&T patricia.wolfhope@dhs.gov 

 Eric Kukula, PhD, Noblis eric.kukula@noblis.org

 Frank Shaw, Noblis frank.shaw@noblis.org 

Noblis Team Members: 

Eric Kukula, Technical/Project Manager, 

Ann Breckenkamp, Emily Keener, George Kiebuzinski, 

Larry Nadel, PhD, Frank Shaw & Rachel Wallner 

DHS S&T Team Members: 

Patty Wolfhope, DHS S&T Biometrics Transition Program 

Manager 

Ryan Bednar, Rick Lazarick & Brad Wing 

Sponsor: