The ISO/IEC 19795 Biometric Performance 

Testing and Reporting standard

Other SC37 WG5 standards: 

ISO/IEC 29156, 29197, 29198

Raul Sanchez-Reillo, Belen Fernandez-Saavedra

Carlos III University of Madrid

Tlf.: +34 609 766 222 Fax: +34 91 624 94 30

{rsreillo, mbfernan}@ing.uc3m.es
ISO/IEC TR 29156 (4th WD)

Guidance for specifying performance 

requirements to meet security and 

usability needs in applications using 

biometrics

Raul Sanchez-Reillo – IDTestingLab (UC3M) 2 
Scope

● Guidance on the use of biometrics WITH and

WITHOUT the use of a second factor

 Guidance on comparing and quantifying security and

usability in authentication mechanisms

● Security and Usability, not only technical

performance, but also on technical, human and

procedural vulnerabilities

● Conditions in maintaining security and usability

● DOES NOT cover identification, only verification and

enrolment

● Target small to medium systems, although much of

the content is applicable to large-scale systems 

● Seeking a balance between security and usability

Raul Sanchez-Reillo – IDTestingLab (UC3M) 3 

29156

29197

29198
Security

● Security failures:

 Inherent limitations

 Failures of design, implementation and/or operation

● Security requirements to establish:

 Acceptable level of false acceptance

 Impact of failure to enrol (how secure is the intended secondary 

procedure? Is a secondary biometric needed?)

 Resistance to/detection of active imposter attacks at the biometric 

sensor point of attack (spoofing, liveness, artefacts,...)

 Detection of physical attack/tampering

 Detection of non-zero effort imposter attacks using liveness or 

artifact detection

 Methods to mitigate attacks

 Adjustable threshold setting (s) for changes in the threat state 

(increased alert level)

Raul Sanchez-Reillo – IDTestingLab (UC3M) 4 

29156

29197

29198
Usability and Use Cases

● Usability requirements to establish:

 Ergonomics

 Performance aspects (throughput, FTA, FRR, etc.)

 Methods to reduce FRR such as training, signage, subject 

feedback (at the sensor), ways to encourage habituation

 Dynamic template updating

 Re-enrolment of subjects (periodically or based on trend monitoring 

and detection of high FRR individuals)

● Use cases illustrated:

 Time and attendance

 Physical access control

 Computer sign-on

 E-authentication

 Point of Commerce

● Annex C contains information about Risk Assessment

Raul Sanchez-Reillo – IDTestingLab (UC3M) 5 

29156

29197

29198
ISO/IEC 29197 (5th WD)

Evaluation methodology for 

environmental influence in biometric 

system performance

Raul Sanchez-Reillo – IDTestingLab (UC3M) 6 
Raul Sanchez-Reillo – IDTestingLab (UC3M)

Introduction

7 

29156

29197

29198

Environmental evaluation

Performance metrics 

Target

Environment

Test subjects

Test procedures

Reference

Performance metrics 

Reference

Biometric system Environment

Biometric

Performance

Evaluation

Biometric

Performance

Evaluation

Environmental 

influence
Raul Sanchez-Reillo – IDTestingLab (UC3M)

Introduction

8 

● Environmental conditions 

Atmosphere parameters: temperature, humidity, etc

Physical and chemical phenomena: illumination,

noise, vibration, etc

● “End-to-end” biometric performance evaluation

Scenario evaluation

 Modelled environment considering a real-world target

application and population

 In accordance to ISO/IEC 19795-2: Testing methodologies

for technology and scenario evaluation

Operational evaluation

 Real environments using a target population

 In accordance to ISO/IEC 19795-6: Testing methodologies

for operational evaluation

29156

29197

29198
Raul Sanchez-Reillo – IDTestingLab (UC3M)

This standard addresses...

● Requirements for planning and execution of

environmental testing evaluations for biometric

systems based on scenario and operational

evaluations

● Specifications to define, establish and measure

specific conditions to assess 

● Requirements for establishing a baseline performance

in order to compare the influence of environmental

parameters

● A specification of the biometric evaluation including

requirements for test population, test protocols, data to

record and test results

● Procedures for carrying out the overall evaluations

9 

29156

29197

29198
Raul Sanchez-Reillo – IDTestingLab (UC3M)

This standard does not ...

● Determine which parameters shall be analyzed for a 

specific biometric modality

ISO/IEC TR 19795-3: Modality specific testing

● Specify requirements to perform a vulnerability

analysis modifying environmental factors

● Classify biometric systems upon performance against

different environmental conditions

● Specify requirements for determining the functional

effects of environmental conditions on hardware

components of biometric systems 

10

29156

29197

29198
Raul Sanchez-Reillo – IDTestingLab (UC3M)

Overview

11

Environmental 

parameters 

to analyse

Evaluation environment

Environmental 

conditions

Evaluation 

configuration

Environmental 

parameters 

to control

Environment 

generators Instruments

29156

29197

29198
Raul Sanchez-Reillo – IDTestingLab (UC3M)

Scenario evaluation

● Suitable for: 

 Measuring the influence of one or a combination

of environmental parameters

Analyse an specific controlled environment

● Requirements related to: 

Environment

Test population: size, selection, training, guidance 

Acclimatization

Level of effort and decision policies

Error protocols

Data to record and test results

Execution sequence

12

29156

29197

29198
Raul Sanchez-Reillo – IDTestingLab (UC3M)

Operational evaluation

● Suitable for: 

 Measuring biometric systems performance in their

operational environment

● Requirements related to: 

Environment

Test period

Data recording processes

Performance measurements

 Establishment of baseline performance

Impostor transactions

Reporting

13

29156

29197

29198
ISO/IEC TR 29198 (PDTR)

Characterization and measurement of 

difficulty for fingerprint databases for 

technology evaluation

Raul Sanchez-Reillo – IDTestingLab (UC3M) 14
Scope

● Scope: 

 characterizing level of difficulty attributable to differences between

samples acquired from the same finger,

 developing statistical methodologies for representing the level of

difficulty of a fingerprint dataset by aggregating influencing factors,

 comparing the level of difficulty of different fingerprint datasets,

 defining procedures for testing and reporting the level of difficulty of

fingerprint datasets collected for technology evaluation,

 describing the archived data selection methodology for building a 

dataset for evaluation

● Outside of the scope is: 

 Defining the quality of individual fingerprint images,

 Defining the methodologies or explicit measures for evaluating or

predicting the performance of fingerprint recognition algorithms

Raul Sanchez-Reillo – IDTestingLab (UC3M) 15

29156

29197

29198
Overview

● It provides the explanation about the differential

factors in fingerprint images

 Capabilities of the capture device

 Environmental conditions

 Demographics

 Intrinsic to the biological characteristics of the modality

 Usability

 Common area among samples

 Relative deformation among samples

 Relative sample quality among samples and within the ones used

to get the biometric reference

Raul Sanchez-Reillo – IDTestingLab (UC3M) 16

29156

29197

29198
Metrics

● It also provides the way to calculate the Level Of Difficulty of the

dataset: 

 Measuring the LOD of individual pairs

● And how to analyze mated pair data characteristics based on

comparison scores

● Finally it provides the way databases can be built for achieving

different levels of difficulty: 

 Easy-level

 Difficult-level

Raul Sanchez-Reillo – IDTestingLab (UC3M) 17

29156

29197

29198
THANK YOU 

FOR YOUR ATTENTION

Raul Sanchez-Reillo, Belen Fernandez-Saavedra

Carlos III University of Madrid

Tlf.: +34 609 766 222 Fax: +34 91 624 94 30

{rsreillo, mbfernan}@ing.uc3m.es