Biometric 

RecognitionChallenges and Opportunities

17 Decemb

Joseph Pato, Committee Chair, Whith

Computer Science and Telecomm

National 
Study Background 

Committee
Charge*

• Provide a comprehensive assessment of biom

examines current capabilities, future possibiliof government in their development. 

• Explore the technical and policy challenges a

development, evaluation, and use of biometrand systems that incorporate them. 

• Examine associated research challenges and 

and inter-disciplinary research agenda to beg

• Examine multiple stakeholders and points of 

technologies, applications, and implementatio

* Project sponsors: DHS, DARPA, CIA, NSF
Background

• Grounded in previous CSTB work

– IDs—Not That Easy: Questions about 

Nationwide Identity Systems

– Who Goes There? Authentication 

Through the Lens of Privacy

– Software for Dependable Systems: 

Sufficient Evidence? 

... 20+ years of CSTB work in cybersecuand information systems...
Whither Biometrics Com

• JOSEPH N. PATO, Hewlett-Packard Labs, Chair• BOB BLAKLEY, Gartner• JEANETTE BLOMBERG, IBM Almaden Research Center• JOSEPH P. CAMPBELL, Massachusetts Institute of Technolo

• GEORGE T. DUNCAN, Carnegie Mellon University• GEORGE R. FISHER, Prudential-Wachovia (retired)• STEVEN P. GOLDBERG, Georgetown University Law Center

• PETER T. HIGGINS, Higgins & Associates, International• PETER B. IMREY, Cleveland Clinic and Case Western Reser• ANIL K. JAIN, Michigan State University• GORDON LEVIN, Disney• LAWRENCE D. NADEL, Noblis• JAMES L. WAYMAN, San Jose State University• LYNETTE I. MILLETT, CSTB Study Director
Committee Approa

• Be neutral with respect to application and 

wherever possible

• Take a broad systems view• Consider issues at scale• Acknowledge privacy issues, but focus on 

and cultural considerations

• Learn from information security communi• Informed by previous NRC work• Informed by related forensic efforts

– But focused on automated biometric recognition, not fore
Principal Conclusio
Recognize Inherently Pro

Nature of System

• Biometric systems contend with unc

nearly every stage of system operat

• Some fraction of interactions will pro

incorrect or indeterminate results

• Error and exception processing criti

success

• Qualitatively different from traditiona

information system bugs or glitches
Rigorous Systems App

Necessary for Succe

• Biometric components are embedded in 

technical systems

– Environment, operations, adjudication me

choices, requirements, data managementmaintenance, . . . 

• Effectiveness depends as much on soc

does on technical & engineering factors

• Biometric systems should be designed 

relative to their specific intended purposrather than generically or in isolation.
Scientific Basis 

Needs Strengtheni

• The distinctiveness of biometric char

not well understood at global populat

– Develop a science of human individua

• The effects of human behavior and in

large-scale biometric system performvice versa) is not well explored.

• Numerous opportunities for research 

Chapter 5
A Well-Designed Biometri

• Takes into account that recognition is based on s

probabilistic not absolute matching; presumption

proof are correspondingly conservative

• Anticipates a lifecycle corresponding to changes 

distributions, stability of traits, and technology

• Assesses the reliability of information associated 

independently of the confidence in correct reco

• Provides exception handling as robust as the pr

process and handles errors gracefully, without vi

privacy or due process rights

• Publicly states explicit security, privacy and poli

• Recognizes that biometric traits are inherently no

minimize risks to privacy and of misrecognition ar
Fundamental Conc
Definition

• Biometrics is the automated rec

individuals based on their behabiological characteristics.*

*ISO/IEC JTC1/SC37 Standing Docu

Harmonized Biometric Vocab
Operational Overvie

Compare

References

Capture Sample

Subject 

presents 

biometric 

characteristic

Sensor

Capture Matcher 
Fundamental Dogm

• An individual is more similar to 

herself over time than to anyon
Grounded Fundamental 

• An individual is more likely sim

or herself over time than to anylikely to be encountered.

... to some degree of confid
Stability and Distinctive

• Some traits appear stable over 

others can change significantly short periods of time.

• Underlying distinctiveness and 

biometric traits are not well undlarge scales. 
Numerous 

Sources of Uncertai

• Physical change in person over time• Interface & environment change• Motivation & social factors• Processed features

– Noisy data, sensors, algorithms

• Data integrity

Biometric systems perform decisio

under uncertainty
Error Rates Can be Mis

• Error rates capture component and s

performance in a narrow sense

– FAR, FRR– FMR, FNMR 

• Confidence in system behavior depe

on error rates, but also relies on knowprobability of a false claim.

– FMR and FNMR do not predict the imp
Confidence in Recogn

Decisions

• College dorm access control sce

– 0.1% FMR– 0.1% FNMR

• Imposters attempt to gain access 

as a legitimate resident

• Confidence in rejections depends 

imposter base rate and declines wreduction in the number of impos
91% Confidence W

10 Imposters per 1

11 Nonmatches;91 percent correct

1,000 Candidates

50

Rows

990 Residents

1 False nonmatc

10 Nonresident impostors

10 True nonmatches

0.1% FMR 0.1% 
50% Confidence W

1 Imposter per 10

2 Nonmatch

50 percent Correct

1,000 Candidates

50

Rows

999 Residents

1 True nonmatch 1 False nonmatc

1 Nonresident impostor

0.1% FMR 0.1% F
0% Confidence W

0 Imposters per 10

0.1% FMR

1 False nonmatc

1,000 Resident

1 Nonmatch0 percent correct

1,000 Candidates

50

Rows

0 Nonresident impostors

0 True nonmatches

0.1% F
Similar Issues for Watc

• What confidence to have in a match wh

presenting to be checked against a wat

– Most will not match– Need prior probability of expected matches 

confidence to have in match

• Increasing size of watchlist cannot be e

improve all aspects of system performa

• As watch list size increases we should 

confident that a match is correct

– with implications for the push towards 

increased interoperability
Engineering Biometric S

in Context
System Life-Cycle Consid

• Issues for all systems

– Training– Commissioning – Component fault replacement– Decommissioning

• Issues for large scale, long-lived systems

– Technology refresh – hardware and software– Data quality, currency, & integrity– Changes in target population– Evolving threat models– Policy flexibility
Immediate Operational C

• Operational requirements and c

affect system design decisions effectiveness

– User Context– Application Context– Technology Context– Performance Context
User Context

Parameter <Less Challenging

Mo

Data Subject

Awareness Very 

Motivation Cooperative Indifferent 

Training Well trained 

Habituation Very 

Party Benefiting Both User / Consumer 

= Thumb access to PDA= Access to Elbonian Fit= Elbonian border watch 
Application Contex

Parameter <Less Challenging

Mo

Supervision Direct Remote Claim type Positive 

Recognition type Verification One to few 

= Thumb access to PDA= Access to Elbonian Fit= Elbonian border watch 
Technology Conte

Parameter <Less Challenging

Mo

Environment Controlled Engagement Active Passive with

cooperation

Sample Capture Overt 

Dataset Proprietary 

= Thumb access to PDA= Access to Elbonian Fit= Elbonian border watch 
Performance Conte

Parameter <Less Challenging

Mo

Throughput Low Medium Sensitivity to error rate

Low Medium 

= Thumb access to PDA= Access to Elbonian Fit= Elbonian border watch 
These Contexts Ma

• Stating that a system is a “biome

or uses “biometrics” says little

– ... about what the system is for or– ... how difficult it is to successfully i

– ... its likelihood of successful deploResearch needed: Taxonomy of syst

implications 
Biometric Systems a

Trustworthiness
Typical Security Go

in Biometric Syste

• Determine that an observed tra

to a living human who is presenacting intentionally

• Accurate comparison of the ob

to reference data maintained in (within desired confidence leve

• Backend data security, integrity
Trustworthiness

• Model threats to understand po

points of attack and estimate pattack

• Assume biometric traits are pub

information

• Manage the trustworthiness of 

recognition process
Multiple Potential Tar

• Resource being protected by biom

system

– will have particular vulnerabilities an

threats separate from mode of prote

• Biometric system itself

– will have particular vulnerabilities an

threats separate from what it’s prote

• Both drive analysis and decisions 

best to provide overall security
Social, Cultural, and L

Implications
Interaction Between 

Systems & Individua

• Motivating participation by individ

– Clear benefits for participants– Limits on system uses

• Facilitating individual participation

– Planning for diversity– Ease of use (beyond sensor interfa– Graceful exception handling and 

accommodations for failures 
Societal Impact

• Community acceptance is influenced 

proportionality related to perceived oeffects 

– Universality and the potential for disen– Potential for record linkage and the los– Covert surveillance and the potential f

power

– Constraints on individuality and identit
Legal Issues

• Remediation 

– Identity fraud by falsifying, altering, or conceali– Inappropriate denial of due process rights resu

recognition

– Responsibility of system operators to minimize 

samples

• Reliability

– Effects of depiction of technology in popular cu– Role of expert testimony and Frye & Daubert s

• Privacy

– Legal issues overlap (but do not encompass) c

matter
Information Sharin

• Information sharing is attractive

– Administrative efficiencies & business purpose– Research uses

• Information sharing requires caution

– Biometric data are personally identifiable infor– Biometric data can serve to correlate disparat

degree of confidence)

– Access to data may allow discovery of doppel

discovery of enrolled users
Data Policies

• Guidance from 2002 IDs – Not Th

remains relevant

– What is the purpose?– What is the scope of the population– What is the scope of the data?– Who would be the users?

– What types of use?– Is participation mandatory?– What legal structures protect integr

and due process?
Research Opportuniti

Public Policy Consider
Technical and Enginee

Research Opportunit

• Distinctiveness and stability of underlying phe

– both absolutely and under common conditions 

• Modality-related

– Sensors, segmentation, invariant representati

• Human factors and affordance• Testing and evaluation

– Test data– Usability Testing

• Information & system security• Scale – numbers, geography, time, and so on
Social Science Research Op

• Individuals

– Performance & effectiveness– Behaviors & affordance

• Society

– Social impacts, direct and indirect– Community acceptance

• Develop data that predict how well a sy

perform

– Experimental studies– Field studies, ethnography
Public Policy Consider

• Feasibility of large-scale deployment

– Does existing technology adequately satisfy the – Do deployment plans integrate adequate risk m– Does national origin of technology create undue – Is there an adequate biometrics workforce?

• Are social impact assessments helpful?• Do deployments represent serious potential fo• Can authoritarian regimes exploit human reco

could such a risk be mitigated?

• What considerations limit or facilitate research 

data?
Lessons from Othe

Large-Scale System
Lessons from Othe

Large-Scale System

• Many factors contributing to the succ

scale systems are not unique to biom

• Opportunity to cross-fertilize biometr

with expertise from other domains su

– information security – medical diagnostics – manufacturing– systems engineering
Biometrics Share Charac

with Other Large-Scale S

• Key success factors for large-scale system

– Good project management– Alignment of capabilities with underlying nee

environment

– Thorough threat and risk analysis

• Common contributors to failures:

– Inappropriate technology choices– Lack of sensitivity to user perceptions and n– Poor understanding of population issues– Lack of a viable business case
Software, Systems, 

Demonstrating Depend

• Biometric systems are software syste

other things), and thus

– general lessons in developing robust s

systems apply.

• How to demonstrate reliability of mis

systems? 

• How to evaluate whether a system is 

(meeting its stated goals)? 
What We Know

About Software Syst

• Extent of failures to date

– software has already resulted in critical system f– death, injury and major economic loss

• Roots of failure

– bugs in code account only for 3% of failures blam

software

– most failures blamed on interactions with operat

environment

– often poor understanding of requirements

• Development strategies

– building dependable software is difficult and cos– quality is highly variable– certification regimes and standards have mixed 
What We Don’t Kno

• Incomplete and unreliable data about

– extent and frequency of software failures– efficacy of development approaches– benefits of certification schemes

• Consequences

– mandating particular process does not guaran– avoid being too prescriptive about particular to– put in place mechanisms for collecting industr– make evidence focus of dependable system d
Three E’s for 

Dependable Software S

• Be explicit

– properties established– assumptions about domain and usage– level of dependability

• Develop and present evidence

– dependability case that properties hold– scientifically justifiable claims– open to audit by a third-party

• Exploit expertise

– approach is technology-independent– demand for evidence stretches today’s best practic– deviate from best practice only with good reason
Distinctive Nature of Bio

Systems and Problem 

• Similar to... medical diagnostics, digital authe

manufacturing production lines, mission-critical tigsoftware systems, information kiosks...

... but not strictly analogous to any of the ab

• And poised to address major public p

challenges

...

...while interacting with large portions of the ...about complex personal notions of identity

privilege.
Acknowledge and Em

• Acknowledge the importance of w

asked of this technology

... especially when used at scale for b

or public policy goals

• Embrace the challenges inherent

... in multidisciplinary research – deehard engineering, social aspects, and... in deploying reliable, broadly-truste
Questions?
For more information...

Joe Pato, Committee Chair, joe.patoLynette Millett, Study Director, lmillettwww.cstb.orgReports available at: www.nap.edu
Extra Slides
CSTB on Cybersecurity and Trustwo

• Critical Code: Software Producibility for Defense (2010) assesses the growing importance of software the U.S. Department of Defense can most effectively meet its future software needs.• Proceedings of a Workshop on Deterring Cyberattacks: Informing Strategies and Developing Optioeconomical, technical, legal, and psychological challenges involved in deterring cyber attacks.• Biometric Recognition: Challenges and Opportunities (2010) presents a broad and comprehensive a

systems -- articulating design and operational considerations as well as outlining a research agenda to bolster 

underpinnings of these systems.• Toward Better Usability, Security, and Privacy of Information Technology (2010) identifies research 

considerations in design and development related to security and privacy, and vice versa.• Technology, Policy, Law, and Ethics Regarding U.S. Acquisition and Use of Cyberattack Capabiliti

cyberattack capabilities are an important asset for the United States, the current policy and legal framework for 

highly uncertain and that U.S. policy should be informed by an open and public national debate on technologicapose.• Toward a Safer and More Secure Cyberspace (2007) explores the nature of online threats, considers so

improving cybersecurity has had less impact than anticipated, and offers a strategy for future research aimed a

• Software for Dependable Systems: Sufficient Evidence? (2007) discusses how the growing use and cdifferent approach to dependability and recommends an evidence-based approach to achieving greater depend• Who Goes There? Authentication Through the Lens of Privacy (2003) describes and examines issueauthentication from the perspective of how they implicate privacy—and how adverse impacts on privacy might • Critical Information Infrastructure Protection and the Law: An Overview of Key Issues (2003) discus

in protecting critical information infrastructure, given technical and economic conditions.• IDs -- Not That Easy: Questions About Nationwide Identity Systems (2002) outlines challenging polic

presented by nationwide identity systems.• Cybersecurity Today and Tomorrow: Pay Now or Pay Later (2002) Recaps highlights from past CSTB identification and practical guidance.• Trust in Cyberspace (1999) provides an assessment of the state of the art procedures for building trustwproposes directions for research in computer and network security, software technology, and system architectu

market trends in order to better inform public policy as to where progress is likely and where incentives could h

• Realizing the Potential of C4I: Fundamental Challenges (1999) addresses the intersecting arenas of sand processes as they relate to challenges in command, control, communications, computers, and intelligence• Cryptography's Role in Securing the Information Society (1996) describes the growing importance of 

to interests in the spread and control of encryption, and recommends policy changes.• Computers at Risk: Safe Computing in the Information Age (1991), an enduring primer for information 

terms, outlines the technology and procedures that give rise to and can alleviate security problems, relates sec

privacy and safety, and describes the private and public sector institutional contexts.
Medical Diagnostic

• Individual components in general u

rarely as sensitive and specific as u

• Confirming a test by repetition is les

than confirming it by a different test

• Limitations in individual component

the effectiveness of others.

• Effectiveness of a system is highly 

specific
Manufacturing

• System objectives must be clear and co

priorities resolved

• The full range of the operational environ

be anticipated

• Ongoing operational testing and blind c

operational systems 

• Biometric systems may be considered a 

line

– Individuals presenting for recognition are – High quality decisions are output 
Dependable Software S

Sufficient Evidence
Large-Scale Identity Sy

It’s Not That Easy

• What would be the purpose of the 

– Assumptions differ about ends

• Who would be issued an ID?

• What data would be collected?

• Who would use the system?

• What types of use would be allowe• Is participation voluntary or mand

• What legal structures would be ne
What does Identity Pro

• What is identity? • Individuals often have multiple identities• What identity information is relevant to th

purpose of the system?

• When are group identities (e.g., “older tha

appropriate? How is identity initially estab

• What is the meaning of the ID?• Where does the identity information resid• What kinds of access to and analysis of t

is allowed or disallowed?
Who Uses the Syste

For What?

• Under what circumstances may someone request 

data within the system?

• Would the private sector be allowed to use the sy• Who will manage, oversee, and maintain the syst• Will public-private partnerships be required? • What are the implications of broad vs. narrow sets • What questions can be asked of the system?• Will data mining be allowed? By whom? • Will real-time correlation of system transactions b• How will information be linked to other systems? • What are the privacy implications of allowing broa

linkages?
Data and Backend Sys

(Not just “IDs”)

• Would consolidation of other databases be neces• Centralization creates a single target for adversar• What are the risks from denial of service attacks• How are potential privacy invasions mitigated if co

capabilities allowed?

• What are the tracking, surveillance, and prediction • Is high availability necessary? What are the implic• What backups and redundancy would be necessa• Differing levels of access and query capabilities fo• Procedurally, how would maintenance and admin
Major Findings from Authe

study

• Context, scope, implementation ma• Local contexts/uses usually more 

sensitive to privacy considerations

• Secondary uses are particularly 

problematic

• Toolkit for thinking through design i• Checklist for evaluating/designing a

systems is presented
When Designing a Pri

Sensitive Authentication 

• Authenticate only for necessary, well-defined purp• Minimize the scope of data collected• Minimize the retention interval of data collected• Articulate what entities will have access to the col

data

• Articulate what kinds of access to and use of the d

will be allowed

• Minimize the intrusiveness of the process• Overtly involve the individual to be authenticated • Minimize the intimacy of the data collected • Ensure that the use of the system is audited and t

record is protected against modification and destr

• Provide for individuals to check on and correct inf

used for authentication