Environmental Testing Methodology in Biometrics 

Belen Fernandez-Saavedra, Raul Sanchez-Reillo, Raul Alonso-Moreno, Oscar Miguel-Hurtado 

University Carlos III of Madrid – Dpt. Electronics Technology 

University Group for Identification Technologies (GUTI) 

Avda. del Mar Mediterráneo, 22 

E-28918 – Leganes (Madrid) SPAIN 

{mbfernan, rsreillo, ramoreno, omiguel}@ing.uc3m.es 

Abstract— Recently, biometrics is used in many security 

systems and these systems can be located in different 

environments. As many experts claim and previous works have 

demonstrated, environmental conditions influence biometric 

performance. Nevertheless, there is not a specific methodology 

for testing this influence at the moment. Due to it is essential to 

carry out this kind of evaluations, a new ISO standard was 

proposed for regularizing them. Such standard was accepted 

and it has to be specified now. 

In this work, authors have analyzed the first proposal of this 

project and other standards about environmental testing. 

According to this new ISO proposal and current ISO standards 

and considering aspects of these related standards focused in 

biometric evaluations, a detailed methodology has been 

defined. This methodology consists of two parts: on the one 

hand, the definition of the environmental factors to analyse, 

including how to generate, control and measure them, and on 

the other hand, the specification about how to perform the 

biometric evaluation. 

Keywords: Biometric Systems, Environmental Evaluation, 

Scenario Evaluations, Biometric Performance 

I. INTRODUCTION 

Nowadays there are several biometric systems operating 

in different environments, e.g. airports, ATMs, companies or 

gyms, and it has been predicted that this kind of systems will 

be used increasingly in the next years. However, it is well 

known that the environment can affect biometric systems 

performance, as well as previous works [1] have proved that 

most of them do not work properly in all environments. 

Therefore, it is necessary to know what could be the 

influence of a specific environment on a biometric system 

before this system will be running in such environment. This 

fact guarantees that the security levels specified for the 

corresponding application will be met regardless ambient 

conditions. 

Currently, there are defined international standards (ISO 

standards) for conducting biometric evaluations. ISO/IEC 

19795-1 [2] defines three types of evaluations: technology, 

scenario and operational evaluations. Technology and 

scenario evaluations are addressed in ISO/IEC 19795-2 [3] 

and operational evaluations will be addressed in ISO/IEC 

19795-6 [4]. Nevertheless, these standards are general for 

analysing biometric systems performance but they do not 

specify exactly how to evaluate the influence of the 

environment. Specifically, an environmental evaluation is a 

set of scenario evaluations. In each scenario evaluation the 

influence of one environmental parameter is analysed. 

To complete the previous standards, ISO/IEC 19795-3 

[5] Technical Report (TR) describes the variations in 

methodologies according to the modality under test. This TR 

explains the environmental factors that could influence in 

biometric performance; however, it does not specify essential 

aspects to perform repeatable and reproducible evaluations 

and obtain intercomparable results as follows: 

- Detailed approaches and requirements for 

generating, controlling and measuring 

environmental parameters during performance 

evaluations. 

- Ranges of each environmental factor in which the 

biometric system have to be examined. 

- Requirements for test instruments. 

Taking into account this situation and the lack of 

procedures for carrying out these evaluations, a New Work 

Item (NWI) was proposed in order to develop a new ISO 

standard for covering them. This NWI was accepted for the 

ISO organization with the name “Evaluation methodology 

for environmental influence in biometric systems”. 

This NWI is based on previous works [1], [6] and [7], but 

there are several aspects that shall be thoroughly defined and 

others that shall be improved. During the first discussion of 

this project, it was decided to look for other standards related 

to environmental testing of equipment in order to have a 

reference for defining it. 

In this work, authors have analyzed such standards, and 

considering them, they have improved the specification of 

the environmental evaluation proposed at the first version of 

the NWI. Standard conditions, scenarios to analyze, 

procedures to generate and control such scenarios, test 

instruments requirements and how to plan and execute 

biometric proofs will be described. In addition, authors have 

defined several details that were not taken into account in 

such version and have justified their choices. 

Next section describes the NWI and explains its structure 

and the main characteristics of it. In the following section, 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

related standards will be summarized and relevant 

information about them will be described. Then, in Section 

IV, the proposed environmental evaluation will be specified. 

Finally, section V presents the main conclusions and future 

works. 

II. NEW WORK ITEM 

The NWI intends to define an evaluation methodology 

for testing biometric systems performance when they are 

working under different environments. In this case, the 

environment consists of atmosphere parameters (temperature, 

humidity and pressure) and other physic phenomena that 

surround the system (illumination and noise). There are other 

environmental parameters that could be studied, but at 

present the NWI only considers the above mentioned. 

This proposed methodology involves the generation and 

control of the scenarios corresponding to each parameter to 

analyse and performs a biometric system evaluation. 

Particular scenarios will be defined for assessing biometric 

systems in its operational range (specified by suppliers), and 

other, for checking it outside this range, including extreme 

conditions. 

To sum up, the standard will address: 

- How to plan biometric performance evaluation 

taking into account the environmental factors. 

- Scenarios to analyse and ranges for each 

environmental parameter. 

- Requirements for the test instruments. 

- The corresponding specifications for performing 

biometric trials. 

This is a first description of the NWI. However, there is 

one relevant matter that must be clarified: type of biometric 

evaluation to execute. The NWI talks about generating and 

controlling scenarios for environmental factors but it does 

not really specify these aspects. 

As it was mentioned in the introduction, there are 3 types 

of evaluations. The best thing to do is to specify a scenario 

evaluation, due to these evaluations analyse end-to-end 

system performance and it is essential in this case to control 

environmental conditions for producing repeatable results (it 

is feasible in scenario evaluations but not in operational 

evaluations). Nevertheless, it is important to emphasize two 

issues related to the other evaluations: 

- A database could be collected following the same 

environmental directives for performing technology 

evaluations lately. 

- Operational test will reduce its variability if 

environmental scenario evaluations have been 

previously executed and the effects caused on the 

biometric system performance are known. 

Therefore, the scope of this project must be redefined to 

include the type of biometric evaluations to perform and also 

it shall be explained that these evaluations are based on the 

current standard that address them (ISO/IEC 19795-2 [3]). 

III. RELATED STANDARDS 

Before defining the evaluation methodology, several 

standards related to environmental testing and other related 

works have been studied. These are the following: 

- MIL-STD-810G, "Test Method Standard for 

Environmental Engineering Considerations and 

Laboratory Tests" [8]. It is a military standard of the 

Department of Defense of The United States. This 

standard analyzes the influence that environmental 

stresses have on materiel throughout all phases of 

its service live. It contains tables with temperature 

and humidity values for different climatic areas and 

world maps in which these areas have been 

identified. 

- DEF STAN 00-35 Part 3 Issue 4, “Environmental 

Handbook for Defence Materiel (Part 3) 

Environmental Test Methods” [9]. It is also a 

military standard of The United Kingdom. This 

standard describes environmental data and tests for 

analyzing conditions of the life of Defence Materiel. 

It includes data of standard conditions, requirements 

for test apparatus and a detailed definition of the 

trials. 

- IEC 60068-1, "Environmental Testing" [10] and 

IEC 60068-2-X "Tests" with multiple parts (e.g. 

IEC 60068-2-1 " Tests – Test A: Cold " [11]). This 

is an international standard that addresses 

environmental tests and severities to measure how 

different products work under normal conditions of 

transportation, storage and operational use. Part 1 

defines values for standard conditions and the 

multiple documents of Part 2 specify different tests. 

- CEN EN 12464-1, " Light and lighting - Lighting of 

work places - Part 1: Indoor work places " [12]. 

This is a European standard which specify 

requirements for illumination systems in different 

indoor working areas. It has tables with the 

recommended illumination values for many places 

and for the activities performed in these places. 

- Occupational Safe & Safety, "Office Ergonomic 

Guidelines – General Office Environment" [13]. 

This document is a guide that suggests a proper 

office environment. It includes the ranges for many 

environmental factors. 

- ISO/IEC 10373-1, “Identifications Cards - Test 

methods- Part 1: General characteristics” [14]. This 

international standard addresses testing methods for 

characterizing identification cards. These methods 

do not assess the environment, but other issues. In 

spite of this fact, such methods have been described 

for one type of identification systems and can be 

useful to specify the current methodology. 

In general, these documents describe standard conditions, 

ranges for several environmental factors, tolerances for these 

factors, requirements for test instruments and methods and 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

sequences of tests that are very useful to define the NWI 

methodology. 

IV. ENVIRONMENTAL TESTING FOR BIOMETRICS 

SYSTEMS 

The whole evaluation for testing biometric systems 

working in different environments shall be defined taking 

into account two parts: environment and biometrics. It entails 

specifying, on the one hand, how to define, generate and 

control environmental scenarios, and on the other hand, the 

biometric system evaluation. Both of them will be explained 

next. 

A. Environmental scenarios 

1) Definition of environmental scenarios 

a) Standard scenario: most of the related standards 

define standard conditions for the tests (MIL-STD-810G, 

DEF STAN 00-35 Part 3 Issue 4 and IEC 60068-1) and 

other recommend values for an office environment (OHS 

Office Ergonomic Guidelines and CEN EN 12464-1). These 

conditions are shown in Tables 1 and 2. 

TABLE 1. STANDARD CONDITIONS IN RELATED STANDARDS: 

TEMPERATURE, RELATIVE HUMIDITY AND PRESSURE 

MIL-STD810G 

(Controlled 

ambient) 

IEC 

60068-1 

DEF STAN 

00-35 Part 3 

Issue 4 

OHS 

Office 

Ergonomic 

Guidelines 

Temp. 23o± 2o C 15o C to 35o C 

21o to 24o C 

(summer) 

19o to 22o C 

(winter) 

Relative 

Humidity 50% ± 5% 25% to 75% 40% to 60% 

Pressure 96.45 kPa 

+ 6.6 / -10.0 kPa 86 kPa to 106 kPa ----

TABLE 2. STANDARD CONDITIONS IN RELATED STANDARDS: 

ILLUMINATION AND NOISE 

OHS 

Office Ergonomic 

Guidelines 

CEN EN 12464-1 

Illumination 

Common task 

300 to 400 lux 

Visual tasks 

600 lux 

Common tasks 

500-1000 lux 

Visual tasks 

>1000 lux 

Operating room 

5000 lux 

Noise 55 to 65 dBA ----

The previous version of the NWI defines this scenario 

taking in consideration typical indoor conditions. After 

reviewing the above information, the conditions of the 

standard scenario shall be redefined as it is shown in Table 3. 

Authors suggest that it would be mandatory to control all 

of them during the evaluation in order to have certain 

guarantee that other ambient conditions are not affecting the 

biometric system. 

TABLE 3. STANDARD SCENARIO 

Temperature 23o± 3o C 

Relative 

Humidity 40% to 60% 

Illumination Visible: 300 – 1500 lux 

NIR: 0 – 1000 lux 

Noise < 60 dBA 

Pressure 86 to 103 kPa 

b) Types of scenarios. To quantify the influence of 

each environmental parameter, certain scenarios shall be 

specified. In each scenario only one parameter will be 

studied. The rest of them shall be kept inside the range 

specified in the standard scenario. For example, if the 

evaluated scenario is a illumination scenario and the 

biometric system under test is a vascular system, only the 

illumination parameter has a specific value, fixed according 

to the scenario, and the rest of mandatory parameters to 

control, such as temperature, relative humidity or noise, must 

be set to standard conditions. 

This requirement allows measuring the effect of each 

ambient factor in the biometric performance separately. The 

combination of multiple parameters is more appropriate to 

evaluate using an operational evaluation. 

As well as the related standards in which each test has 

specified the corresponding value for the parameter to assess, 

each environmental scenario must also set the environmental 

factor to examine. These shall be described using common 

units or units utilized by the most common instruments. 

Next, the types of scenarios and how to specify them will be 

explained. 

i) Temperature scenarios: These scenarios will be 

specified by means of temperature and relative 

humidity values. Both parameters have to be set 

together because they are very dependent. The 

relative humidity has certain percentage given a 

specified temperature. This is because if 

temperature increases, relative humidity 

decreases because cold air contains less humidity 

than warm air. In addition, these parameters are 

usually generated and controlled by the same 

instrument simultaneously, and therefore, 

evaluators will set these parameters at the same 

time. 

Temperature shall be described using oC or oF and 

relative humidity using %. 

ii) Humidity scenarios: As above mentioned, these 

scenarios will be also defined especifying 

temperature and relative humidity values using 

the same units. 

iii) Illumination scenarios: Such scenarios have to be 

determined considering two areas of the 

spectrum: visible and near infrared light. The 

illumination standard CEN EN 12464-1 does not 

do it like this, but there are many biometric 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

systems that work using near infrared light and it 

is necessary to consider it. 

Illumination shall be described using the lux unit. 

iv) Noise scenarios: These scenarios shall be specify 

by means of the noise level corrected by A scale. 

That is why noise shall be set using dBA. 

v) Pressure scenarios: Such scenarios must be 

described by the pressure level. 

The unit to define pressure has to be kPa or 

mmHg. 

c) Scenarios to analyze. These scenarios shall be 

specified for analyzing the influence of certain ambient 

conditions. There are two different approaches: consider the 

biometric system and its operational range, or the place in 

which the system will be located. The first option analyze 

directly the biometric system performance independently 

where it will be located and the second option check if the 

specific biometric system is affected by particular 

environmental factors as in military standards. Both options 

are acceptable for the purpose of the NWI, but authors think 

that the first one must be mandatory whereas the second must 

be optional. In addition, evaluation in the standard 

conditions scenario always has to be obligatory. 

Now, the scenarios corresponding to each option will be 

specified: 

i) Operational range: At least three scenarios must 

be defined to evaluate this range. One scenario 

for the minimum value, other for the maximum 

value and one more scenario for the middle value. 

This range shall be defined by suppliers at the 

beginning of the evaluation. If there is not an 

specified range, the range considered for basic 

climate regions shall be used. These regions will 

be explain next. 

ii) Extreme conditions: These scenarios have to 

simulate specific ambient conditions that are 

achieved in different regions of the world. MILSTD-810G provides tables and maps to select it. 

This tables are based on NATO standards and 

define four kind of climate regions: hot, basic, 

cold and severe cold. 

iii) Free conditions: The NWI should also be open 

for defining other ranges that suppliers or 

customers want to analyze. These scenarios will 

be set before the evaluation and will be reported. 

Any scenario will be specified following the directives 

provided in the previous section (section b). 

As well as considering the range of parameters to assess, 

the set of environmental scenarios have to take into account 

the type of scenarios to evaluate depending on the biometric 

modality under test. Table 4 shows the ambient factors that 

could affect each biometric modality. This table is based on 

ISO/IEC TR 19795- 3 [5]. 

TABLE 4. AMBIENT FACTORS THAT AFFECT EACH BIOMETRIC 

MODALITY 

AMBIENT FACTOR BIOMETRIC MODALITIES 

Temperature Face, Fingerprint, Vascular, Voice 

and Hand Geometry 

Humidity Face, Fingerprint, Vascular, Voice 

and Hand Geometry 

Illumination Face, Vascular, Iris and 

Fingerprint (optical sensors) 

Noise Voice and audio guides for all 

modalities 

Pressure Signature 

2) Generation of environmental scenarios 

In order to generate environmental scenarios some 

specifications for instruments have to be defined. Also, 

requirements to know if the features of the scenario have 

been achieved must be addressed. Both definitions are based 

on similar specification described in the related standards. 

a) Instruments: Most of the related standards do not 

specify exactly which equipment must be used during tests. 

These provide general specification as follows: 

- Instruments must be able to achieve the maximum 

and minimum value of the scenarios to assess. 

- The resolution of the instruments must be properly in 

order to adjust the value of the different scenarios. 

- Instruments have to be calibrated previously. 

b) Requirements to assure that the feautures of 

scenarios have been achieved: The particular requirements 

of one scenario will be achieved if the corresponding 

parameters do not change or are inside the defined range in a 

minimum period of half an hour. 

During the evaluation, test subjects are going to interact 

with the biometric system, so the environment could vary 

sensitively. When this happens, evaluators must wait until 

the features of the scenario will be achieved again plus an 

additional time of 2 or 3 minutes for the biometric system to 

get acclimatized. 

In addition, it is recommendable to introduce the 

biometric system in the scenario before generating the 

scenario because some factors as temperature could damage 

the biometric system if they are modified quickly. 

3) Control of environmental scenarios 

In order to produce repeatable and reproducible results, 

environmental scenarios have to be controlled during all the 

evaluation. As the generation of environmental scenarios, 

instruments and requirements to perform this task have to be 

specified. 

a) Instruments: in many situations these instruments 

will be the same instruments used for generating scenarios. 

Nevertheless, these equipment shall met the following: 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

- Instruments must be able to measure the maximum 

and minimum value of the scenarios to assess. 

- The resolution of the instruments shall be the 

appropriate for registering changes. 

- Instruments have to be calibrated previously. 

b) Requirements for controlling scenarios. The main 

parameter to control during the evaluation of one scenario 

will be the specific parameter of such scenario. It will be 

analysed continuously and it has to be measured near the 

biometric capture device. 

The rest of parameters will be measured before and after 

the execution of each session of biometric trials. These, are 

standard parameters that shall surround the biometric system, 

so it is not necessary to measure them very close to the 

biometric system (1 or 2 m). In case the evaluator detects 

anomalies, all these measurements must be repeated. 

B. Biometric system evaluation 

Biometric system evaluation consists of a scenario 

evaluation as it was explained above. This kind of evaluation 

is addressed by ISO/IEC 19795-2 [3]. Also ISO/IEC 19795-5 

[15] describes how to plan and execute a scenario evaluation 

for access control systems.

 Taking into account both standards, a general scenario 

was specified in [16] for performing usability evaluations. 

Specifically, this scenario is conformance with ISO/IEC 

19795-2 and it was tested for some usability factors. Results 

disclosed that it was appropriate to assess the influence of 

usability factors in biometric performance. 

Considering the previous work, authors have determined 

a similar scenario adapting it to analyse the influence of 

environmental factors. Next, the most relevant features of 

this scenario will be explained. More details about this 

scenario can be found in [16]. 

1) General 

Firstly, a description of the biometric system under test 

and the objectives of the evaluation have to be done. For this 

kind of evaluations, this description must include the 

definition of environmental scenarios to assess according to 

the way that has been previously explained. 

2) Environment 

a) Ambient conditions: These conditions depend on the 

environmental scenario to evaluate. For each scenario they 

shall be generated and controlled as it was described in the 

previous part. 

b) Biometric system placement: Sensor must be located 

in the corresponding environment. This location shall allow 

that users can interact with the biometric system 

comfortably. As far as possible, the biometric system has to 

be placed as suppliers recommend. If there is no guidance for 

it, sensor will be positionated straight, in front of test 

subjects, without inclinations. The height depends on the 

type of sensor: 

• Desk device: between 70 to 75cm (standard height 

table). 

• Wall device: sensor has to be placed according the 

average human height and the biometric 

characteristic to evaluate. 

3) Test crew 

The test crew for a scenario evaluation has to be general 

and representative of a real-world application. It also shall be 

enough to achieve significant results. 

a) Demographics: A balanced set of test subjects 

minimizes the effects of biologic differences between 

human beings. ISO/IEC 19795 – 5 [15] specifies a proper 

group. Authors suggest to use the same directives for this 

kind of evaluations: 

• Age: people from 18 to 70 years old. The number of 

test subjects shall be equally distributed for each 

range of age: 18 to 30, 31 to 50, and 51 to 70. It shall 

be from 25 to 40% of the test crew. 

• Gender: 40% to 60% of men and 60% to 40% of 

women. 

b) Size: The number of test subjects that are going to 

take part in the evaluation must be defined taking into 

account the rule of three or the rule of thirty following 

ISO/IEC 19795-2 specification. 

After applying one of these rules, evaluators will obtain 

the following data: 

- Number of test subjects to recruitment. Authors suggest 

increasing the number of test subjects around 10% because 

some subjects could leave the evaluation before it has 

finished. 

- Number of genuine and impostor transactions to 

execute per test subject to achieve a significant number of 

comparisons. To calculate this number, evaluators must 

consider that the number of attempts per transaction will be 

three. 

c) Selection: Test subjects could be selected randomly 

whenever the age and gender requirements will be met and 

they do not have to be involved during the biometric system 

implementation or tune. 

d) Training and guidance of test subjects: 

i) Test information: Test subjects must receive 

relevant information about the evaluation: 

objectives, number of visits and legal issues 

related to their participation in the evaluation. 

ii) Test instructions: An explanation of evaluation 

phases (enrolment and verification) and what test 

subjects must do in each of these phases have to 

be explained to them. 

If there are some instructions that test subjects 

must follow according to the environmental 

scenarios, they have also to be explained. 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

iii) Training: Test subjects shall perform practical 

trials. Test operators shall supervise these trials 

and correct mistakes. 

iv) Guidance: During the training and the enrolment 

process test subjects must be guided. During the 

verification process test operators will checked 

that test subjects interact correctly with the 

biometric system. 

v) Feedback: If the biometric system show 

information related to the capture of the biometric 

sample and the result of matching per each 

attempt, it shall be made known to test subjects 

for improving their interactions. 

e) Visits: The number of visits should be as much as 

possible. As minimum, test subjects will attend to test facility 

twice. At the first visit test, subjects shall perform practical 

trials, then, perform the enrolment process and finally the 

first session of verification transactions. At the subsequent 

visits, test subjects will only have to carry out the next 

sessions of verification transactions. Visits shall be separated 

as later as possible (one week at least). 

In addition, all transactions shall be done with 

disengagement from the device. 

4) Levels of effort and decision policies 

• Two enrolment transactions. The second transaction 

should be only executed if the first transaction fails. 

• Three verification transactions and three attempts per 

each transaction. 

• The maximum time for enrolment and verification 

depends on the biometric system under test. A 

reasonable time is 2.5 times the average time. The 

evaluators must calculate the average time executing 

some trials. 

5) Error protocols 

During the evaluation, different errors could happen. It is 

essential to specify some actions for the most typical errors. 

These are the following: 

a) General errors: These errors happen when the 

biometric sensor does not work correctly. In this case, the 

test operator has to stop the evaluation and solve the 

problem. Once the biometric system works well again, the 

evaluation can continue. Errors and its solution shall be 

reported. 

b) Environmental anomalies: If test operators detect 

changes at the environment, they must measure the 

environmental parameters and check if these are inside its 

specified range. If there are any parameters outside the 

range, they shall stop the evaluation and correct the potential 

problems. Once the scenario is stable and inside the 

corresponding range, the evaluation can be resumed. Test 

operators must report these anomalies. 

c) Enrolment and verification errors: When test 

operators detect that the test subject has introduced a wrong 

identifier or has presented a wrong biometric characteristic, 

he has to cancel the attempt, inform the test subject about 

the error and repeat the attempt. 

6) Data to record and test results 

To quantify biometric performance in each scenario, 

several metrics and statistics must be calculated and reported. 

It means that relevant information shall be collected during 

the evaluation. Regarding biometrics, ISO/IEC 19795 Part 1 

[2], Part 2 [3] and Part 5 [15] address which data shall be 

collected and reported. Regarding environment, the 

following information shall be reported: 

• Environmental scenarios to assess including the 

specific values of them. 

• Test instruments used to generate and control 

scenarios. 

• Metrics and/or graphics of the environmental 

parameters that shall be controlled during all 

evaluation. 

• Measurements of the standard conditions parameters 

that have to be measured, only before and after 

performing biometric trials. 

• Environmental anomalies. 

7) Tests procedures 

The procedure for quantifying biometric system 

performance should allow comparing the performance of the 

multiples scenarios. Authors recommend measuring the 

biometric system performance in the standard conditions 

scenario firstly, and then in the other environmental 

scenarios. 

Moreover, they consider that enrolment is performed in 

indoors conditions most of the times (e.g. the human 

resources department, police station, etc) and therefore, the 

enrolment process has to be carried out in the standard 

conditions scenario. 

8) Execution sequence 

For performing the biometric evaluation, a set of tasks 

has to be executed by test operators and test subjects. These 

tasks have to follow an appropriate sequence. Authors 

suggest the following one: 

a) Pre-test activities 

• Examine the biometric system under testing and 

implement the essential application for performing 

the evaluation. It has to be able to collect the 

essential information and has to be conformance 

with the levels of effort and decision policies defined 

above. 

• Plan the evaluation scenarios and the proper test 

instruments. 

• Develop the evaluation schedule and how to recruit 

test subjects. 

• Implement legal forms, data forms and guides for 

test subjects. 

• Instruct test operators about how the biometric 

system works, how to use the evaluation application, 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

how to handle test instruments and all necessary 

details to carry out the evaluation. 

• Calibrate test instruments and check how they work 

if it is necessary. 

b) Test activities 

i) Procedures before the first visit 

- Recruit test subjects. 

- Introduce the biometric system in the 

standard conditions scenario and establish 

this scenario for enrolment. 

- Conduct a system operability analysis for 

enrolment and verification processes. 

ii) First visit 

- Explain test instructions to test subjects. 

- Train test subject for enrolment 

- Measure environmental conditions before 

the enrolment 

- Carry out enrolment. Remind that the 

second transaction must only be executed 

when the first transaction fails. 

- Measure again the environmental 

conditions after enrolment. 

- Generate the first environmental scenario. 

Do no forget to introduce the biometric 

system in the simulated environment 

before starting to adjust the corresponding 

values. 

- Wait until this scenario is stable. During 

this time test operators can perform the 

training for test subjects in verification 

process. 

- Measure environmental parameters. 

- Verify test subject in this scenario: 

1. Perform the first session of genuine and 

impostor transactions in the common 

scenario. If the environmental 

conditions are modified due to the 

interaction of the test subject, between 

each transaction, it is important to wait 

for the scenario stabilization and for the 

biometric system to get acclimatized. 

2. Measure environmental parameters 

again. 

3. Generate the next scenario. 

The last two steps have to be done for the 

evaluation of all the specific scenarios. 

iii) Subsequent visits 

- Establish again the standard scenario and 

check that it works. 

- Remind the test subjects the test 

instructions for the verification process. 

- Measure the environmental conditions. 

- Perform a session of genuine and impostor 

transactions in the standard scenario. 

- Measure again the environmental 

parameters. 

- Establish the first environmental scenario 

to test. Remind to introduce previously the 

biometric system in the scenario. 

- Measure the environmental parameters 

before the verification transactions 

- Execute the set of verification transactions 

as it was indicated at the first visit. Take 

into account the above mentioned 

protocols between transactions if the 

ambient conditions are affected by test 

subject interactions. 

- Measure again the environmental 

parameters. 

- Generate the next scenario and repeat the 

four last steps for all the environmental 

scenarios to analyze. 

c) Post-test activities 

• Analyze all the information collected during the 

evaluation. 

• Calculate performance rates, graphics and metrics. 

• Generate reports including all mandatory data to 

report. 

V. CONCLUSIONS AND FUTURE WORKS 

In this work, a general and independent methodology has 

been implemented to assess the influence of the environment 

in biometric systems performance. Firstly, a review of the 

first ISO proposal for this methodology has been performed. 

Then, other standards for executing the environmental testing 

of different products have been studied. Finally, a detailed 

environmental methodology has been described improving 

the first version of the NWI and taking into account the 

information obtained from the other standards. 

Nevertheless, part of this methodology has been specified 

considering standards aimed to other products, so as a future 

work, the methodology should be exhaustively checked for 

biometric systems. 

ACKNOWLEDGMENT 

This work has been developed within the FPU fellowship 

program, funded by the Spanish Ministry of Science and 

Education. 

REFERENCES 

[1] Sanchez-Reillo, R., Fernandez-Saavedra, B., Liu-Jimenez, J., Kwon, 

Y.-B., “Changes to vascular biometric system security & 

performance,” Aerospace and Electronic Systems Magazine, IEEE 

Volume 24, Issue 6, pp. 4 – 14, June, 2009. 

[2] ISO/IEC 19795-1:2006, Information Technology - Biometric 

Performance Testing and Reporting - Part 1: Principles and 

framework. 

[3] ISO/IEC 19795-2:2007, Information Technology - Biometric 

Performance Testing and Reporting - Part 2: Testing methodologies 

for technology and scenario evaluation. 

[4] ISO/IEC CD 19795-6, Information Technology - Biometric 

Performance Testing and Reporting - Part 6: Testing methodologies 

for operational evaluation. 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

[5] ISO/IEC TR 19795-3, Information Technology - Biometric 

Performance Testing and Reporting - Part 3: Modality specific 

testing. 

[6] B. Fernandez-Saavedra, R. Sanchez-Reillo, R. Alonso-Moreno, “ 

Evaluation Methodology Based on CEM for Testing Environmental 

Influence in Biometric Devices”, 9th International Commom Criteria 

Conference, Jeju Island, South Korea, 23-25 September, 2008. 

[7] B. Fernandez-Saavedra, R. Sanchez-Reillo, R. Alonso-Moreno, “ 

Evaluation Methodology for Measuring the Influence of 

environmental Factors in Biometric Systems”, 10th International 

Conference on Control, Automation, Robotics and Vision, 

Proceedings, pp. 1342 - 1346, Hanoi, Vietnam, 17-20 December, 

2008. 

[8] MIL-STD-810G, "Test Method Standard for Environmental 

Engineering Considerations and Laboratory Tests", presently (2008) 

version G. 

[9] DEF STAN 00-35 Part 3 Issue 4, "Environmental Handbook for 

Defence Materiel (Part 3) Environmental Test Methods", Issue 4 

Publication Date 18 Sept 2006. 

[10] IEC Standard: IEC 60068-1, Environmental Testing. Part 1: General 

Guidance, June, 1988. 

[11] IEC Standard: IEC 60068-2-1, Environmental Testing. Part 2-1: Test 

– Test A: Cold, Mach, 2007. 

[12] CEN EN 12464-1 Light and lighting - Lighting of work places - Part 

1: Indoor work places, November, 2002. 

[13] http://www.usyd.edu.au/ohs/ohs_manual/ergonomics/ERGO7.shtml 

[14] ISO/IEC 10373-1:2006, Identification cards - Test methods - Part 1: 

General characteristics. 

[15] ISO/IEC CD 19795-5, Information Technology - Biometric 

Performance Testing and Reporting - Part 5: Grading scheme for 

access control scenario evaluation. 

[16] B. Fernandez-Saavedra, R. Alonso-Moreno, J. Uriarte-antonio R. 

Sanchez-Reillo, “Evaluation Methodology for Analyzing Usability 

Factors in Biometrics,” 43rd Annual IEEE International Carnahan 

Conference on Security Technology (ICCST 2009), Proceedings, pp. 

347–354, Zürich, Switzerland, 5-8 October, 2009.