IREX IV 

Evaluation of One-to-Many Iris Recognition 

Concept, Evaluation Plan, and API Specifcation 

Version 0.2 

George W. Quinn and Patrick Grother 

Image Group 

Information Access Division 

Information Technology Laboratory 

May 4th, 2012 
i 

1 Status of this Document 

2 This is the second public version of this document. Comments and questions should be submitted to irex@nist.gov. 

3 This document can be downloaded from http://iris.nist.gov/irex. 

4 Timeline 

Table 1: Milestones and deadlines 

June 28, 2012 Last day to submit and still receive an interim report back. 

May 16 - August 2, 2012 Anticipated submission window. 

May 16, 2012 Final API will be released. 

May 14, 2012 Comments due on version 0.2 of API 

May 4, 2012 NIST releases API version 0.2 

May 1, 2012 Comments due on Initial API 

April 16, 2012 NIST releases API version 0.1 

5 Release Notes 

Changes relating to the second release of this document are highlighted with a green background color. Most 

notably: 

1. The submission window has been extended to Aug 2, 2012. 

2. The process of interim reporting of results is explained in greater detail in Section 2.4.2. 6 

3. The dead links pointing to the encryption procedures and participation agreement have been corrected. 

4. The decision parameter of identify_template() is now a pointer type. 

5. gcc version 4.4.6 is now specifed as the system that will be used to link the libraries. 

NOTE: IREX IV is very similar to IREX III with respect to its API and implementation requirements. Notable 7 changes are highlighted throughout this document with a yellow background color. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
10

15

20

25

30

35

40

45

CONTENTS ii 

8 Contents 

9 1 IREX IV Concepts 1 

1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 

11 1.2 Market Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 

12 1.3 Application Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 

13 2 Evaluation Overview 2 

14 2.1 Performance Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 

2.2 Iris Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 

16 2.3 Test Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 

17 2.4 Reporting of Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 

18 3 Software Submission 8 

19 3.1 Participation Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 

3.2 Submission Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 

21 3.3 Requirements for Library Submissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 

22 3.4 Linking Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 

23 3.5 Single-thread Requirement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 

24 3.6 Installation Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 

3.7 Runtime Behavior Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 

26 4 API Specifcation 12 

27 4.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 

28 4.2 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 

29 4.2.1 Function Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 

4.2.1.1 get_pid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 

31 4.2.1.2 get_max_template_sizes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 

32 4.2.1.3 initialize_enrollment_session . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 

33 4.2.1.4 convert_multiiris_to_enrollment_template . . . . . . . . . . . . . . . . . . . . . 16 

34 4.2.1.5 fnalize_enrollment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 

4.2.1.6 initialize_feature_extraction_session . . . . . . . . . . . . . . . . . . . . . . . . 18 

36 4.2.1.7 convert_multiiris_to_identifcation_template . . . . . . . . . . . . . . . . . . . . 18 

37 4.2.1.8 initialize_identifcation_session . . . . . . . . . . . . . . . . . . . . . . . . . . 19 

38 4.2.1.9 identify_template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 

39 4.2.1.10 convert_raster_to_cropped_and_masked . . . . . . . . . . . . . . . . . . . . . 20 

5 Supporting Data Structures 21 

41 5.1 CANDIDATE Struct Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 

42 5.2 MULTIIRIS Struct Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 

43 5.3 MULTISEGMENTATION Struct Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 

44 5.4 ONEIRIS Struct Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 

5.5 ONESEGMENTATION Struct Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 

46 6 References 25 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
1 IREX IV Concepts 1 

47 Terms and Defnitions 

Table 2: The following terms and defnitions are used in this document 

ANSI American National Standards Institute 

ANSI/NIST Type 17 American National Standard for Information Systems - Data Format for the Interchange of Fingerprint, Facial, and Other Biometric Information - Part 1 

API Application Programming Interface 

EDB Enrollment Database 

FNIR False Negative Identifcation Rate 

FPIR False Positive Identifcation Rate 

FTS Failure to search 

FTX Failure to extract features from an enrollment image 

DET Detection Error Tradeoff 

ISO International Standards Organization 

ISO/IEC 19794-6 ISO/IEC standard titled "Information technology - Biometric data interchange 

formats - Part 6: Iris image data" 

ISO/IEC 29794-6 ISO/IEC standard titled "Biometric Sample Quality - Part 1: Framework" 

IREX Iris Exchange 

NIST National Institution of Standards and Technology 

UID India’s Unique Identity scheme 

48 1 IREX IV Concepts 

49 1.1 Overview 

50 This document establishes a concept of operations (CONOPS) and application programming interface (API) for the 

51 Iris Exchange (IREX) IV Evaluation. IREX IV will be a large-scale evaluation of iris recognition technology over 

52 operational data. Like IREX III [1], it will focus exclusively on one-to-many applications. 

53 The goals of this evaluation are 

54 • To investigate the use of cost parameters for application specifc optimization (see Section 2.1.2). 

55 • To establish a compression profle for the effcient and compact storage of iris images (see Section 2.1.3). 

56 • To measure the speed and accuracy of iris matchers over the OPS-II dataset of operational iris images. 

57 This marks the fourth installment in the IREX program (see Figure 1). See http://iris.nist.gov/irex for all IREX related 

58 documentation. 

59 1.2 Market Drivers 

60 This evaluation is intended to support a plural marketplace of iris recognition systems. While the largest applica61 tions, in terms of revenue, have been for border control and war zone identity management, India’s Unique Identity 

62 (UID) scheme is currently using iris (in conjunction with fngerprints) for de-duplication on a massive scale. 

63 The expanding marketplace for iris recognition has fueled the development of iris cameras designed to operate in 

64 a variety of applications. For example: 

65 • Some standoff-capture cameras can rapidly image and verify (in a one-to-many mode) high volumes of 

66 people. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
1.3 Application Scenarios 2 

Figure 1: Current extent of the IREX program as well as planned expansions. 

67 • Some mobile cameras can be preloaded with frm-ware based segmentation and identifcation capability for 

68 rapid one-to-many watchlist searches. 

69 These applications are differentiated by population size, hardware capabilities, quality of the iris samples, and other 

70 variables. 

71 1.3 Application Scenarios 

72 The evaluation will focus on practical applications of iris recognition with an emphasis on large-scale deployments 

73 (i.e. where the enrollment database contains up to several million subjects). The interest is in one-to-many open-set 

74 identifcation systems. Systems operating in a one-to-many mode (sometimes referred to as "identifcation mode") 

75 are tasked with identifying the individual without a prior claim to identity. Open-set means there is no guarantee 

76 that the searched individual is enrolled in the database. To explore the potential for application-specifc algorithm 

77 optimization, participants will submit two classes of implementations, each focusing greater attention on reducing 

78 a different type of error (see section 2.1.2). Table 3 details the parameters of this evaluation. 

79 Participants may also submit implementations that perform cropping and masking of the iris images to convert 

80 them into an ISO/IEC 19794-6 compact format. Representing iris images compactly is crucial for applications 

81 operating over limited-bandwidth networks. India’s Unique Identity (UID) scheme is seeking to reduce bandwidth 

82 requirements for the transmission of iris data. 

83 2 Evaluation Overview 

84 The evaluation will be conducted offine. Offine evaluations are attractive because they allow uniform, fair, re85 peatable, and convenient testing. However, they do not capture all aspects of an operational system. While this 

86 evaluation is designed to mimic operational reality as much as possible, it does not include a live image acquisition 

87 component or any interaction with real users. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
2.1 Performance Metrics 3 

Table 3: Application Parameters 

Parameter Class P 

(Positive Identifcation System) 

Class N 

(Negative Identifcation System) 

Application Type One-to-many open-set identifcation systems 

(e.g. watchlists, de-duplication operations). 

Class Description High cost associated with false 

positives 

High cost associated with false 

negatives 

Example Applications 

Biometric authentication for 

restricted access to high value 

information, resources, or facilities. 

Watchlists for high-profle 

individuals. Investigational-mode 

searches. 

Enrolled Database Size Anywhere from O(102) to O(107) subjects. 

Prior NIST References IREX III Final Report [1] 

IREX III Supplement I: Failure Analysis [2] 

Multiple Biometric Evaluation (MBE) 2010 [3] 

Performance Criteria Primarily accuracy and speed. 

Also, memory usage, scalability, template-size, etc. 

88 2.1 Performance Metrics 

89 2.1.1 Accuracy 

90 Accuracy will be measured for open-set applications, which means that no assumption can be made as to whether 

91 the searched individual is enrolled in the database. Most real-world applications of biometrics operate in this way 

92 (e.g. watchlists and de-duplication tasks). Closed-set applications, which assume that every searched individual 

93 is enrolled in the database (and thus only concern themselves with which of those enrollees the searched person 

94 matches best) are operationally uncommon and will not be tested. 

95 Open-set biometrics systems are tasked with searching an individual against an enrollment database and returning 

96 zero or more candidates. Two types of decision errors are usually considered for this type of system. The frst 

97 occurs when a candidate is returned for an individual that is not enrolled in the database. This is referred to as a 

98 false positive. The second occurs when the correct candidate is not returned for an individual that is enrolled in the 

99 database. This is referred to as a false negative. 

NIST will compute false positive statistics exclusively from non-mated searches and false negative statistics exclusively from mated searches. Although this refects operational reality better than computing false positive statistics 

100 from mated searches (by simply ignoring correct mates on the candidate lists), it does not cover all factors that 

could affect the accuracy of a system (e.g. the position of the correct mate on the candidate list, the number of 

incorrect candidates returned for a mated search). 

101 This evaluation will present core matching accuracy in the form of Detection Error Tradeoff (DET) [4] and Sensitivity102 Reliability [5] plots, both of which show the tradeoff between the two types of error. The Application Programming 

103 Interface (API) will require searches to return a fxed number of candidates but will only consider a candidate 

104 viable if its dissimilarity score is below some decision threshold. Table 4 defnes how the accuracy metrics will be 

105 computed. 

106 In some plots, line segments will be drawn between curves to connect points of equal threshold. These line 

107 segments are intended to show how error rates at specifc operating thresholds vary depending on factors such as 

108 the number of entries in the enrollment database or the quality of the iris samples. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
2.1 Performance Metrics 4 

Table 4: DET and SEL-REL accuracy metrics 

Performance Plot Metric Description 

Detection-error 

FPIR The fraction of non-mated searches for which at least one candidate has 

a distance score at or below threshold. 

Tradeoff Curve FNIR The fraction of mated searches for which the correct candidate is not on 

the list or has a distance score above threshold. 

Selectivity-Reliability 

Curve 

SEL The average number of candidates for a non-mated search having a 

distance score at or below threshold. 

REL One minus FNIR 

109 2.1.2 Cost Function Optimization 

This evaluation will investigate the use of cost parameters for application-specifc algorithm optimization. The 

110 goal is to determine if matching algorithms can be modifed to improve performance when the costs of errors are 

known in advance. The following cost model will be used as an evaluation metric for recognition performance: 

111 

E[ Cost(τ) ] = (1−PMated) FPIR(τ) CP +PMated FNIR(τ) CN (1) 

where PMated is the a priori probability that the user of the system is mated, CP is the cost of a false positive, 

CN is the cost of a false negative, FPIR(τ) is the false positive identifcation rate, FNIR(τ) is the false negative 

112 identifcation rate, and τ is the operating threshold. The model estimates the expected cost per user attempt, 

which could be a measure of time, workload, money, etc. The participant is tasked with minimizing the cost for a 

predetermined and fxed set of cost parameters (CP, CN, and PMated). 

Cost parameters are often chosen to correspond to a specifc application. Consider a biometric system that 

provides bank vault access to specifc individuals. One might reasonably set the cost of a false positive to be the 

113 monetary value of whatever is in the vault, and the cost of a false negative to a value that refects the amount of 

inconvenience incurred from having to open the vault by some other method. Setting PMated to 0.1 assumes that 

one out of every ten access attempts is by an allowed user. 

NIST requires each participant to submit two implementations, each corresponding to a different set of cost 

parameters. These parameters are defned in Table 5. Class P implementations penalize false positives heavily 

114 and false negatives lightly. Class N implementations assign comparatively greater penalty to false negatives. For 

this class of implementations, suppression of false positives is less important. Both classes will be tested over 

one-eye and dual-eye tests. Participants may wish to use a different fusion rule for the two class types. 

Table 5: Cost parameters for both submission types 

Implementation Class CN CP Pmated

Class P 1 10 0.01 

Class N 200 1 0.1 

Additionally, failures to extract (FTXs) and failures to search (FTSs) will be treated differently depending on the 

implementation class. For Class P implementations, both will be treated as failures in a positive recognition 

system (e.g. access control). This is the way NIST has handled FTXs and FTSs in prior evaluations. For Class 

115 N implementations, FTXs and FTSs be treated like failures in a negative recognition system (e.g. a watchlist). 

Failures in a negative recognition system increase the FPIR when they occur for non-mated searches, but do not 

increase the FNIR when they occur for mated searches. This differs from the way NIST has traditionally handled 

these types of failure. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
5 

116 

2.1 Performance Metrics 

The motivation for requiring participants to submit two implementations is to see if it is possible to change the 

shape of a DET to reduce cost for a specifc set of cost parameters. Figure 2 plots standard DET curves for 

two identifcation algorithms. The two curves cross one another, making it impossible to state which is more 

accurate in any absolute sense. Since Class N implementations are penalized heavily for false negatives, and 

only lightly for false positives, both algorithms are expected to achieve their lowest cost toward the right end of the 

fgure, where the blue curve performs better. Conversely, Class P implementations are penalized heavily for false 

positives but only lightly for false negatives. Thus, for this set of cost parameters, both algorithms are expected to 

achieve their lowest cost toward the left end of the fgure, where the red curve performs better. 

Figure 2: Notional DET plots demonstrating how the two classes place greater emphasis on different 

regions of the DET. 

117 2.1.3 JPEG 2000 Compression 

India’s UID scheme will use the iris biometric for recognition tasks, and a desire has been expressed to represent 

iris feature information more compactly to reduce bandwidth usage during network transfer. The ideal solution 

is to store the images according to one of the compact and interoperable formats specifed in ISO/IEC 19794-6. 

This evaluation seeks to further support the standard by establishing JPEG 2000 compression profles for the 

118 effcient and compact storage of iris images. Toward this end, NIST will subject the images to lossy JPEG 2000 

compression while tweaking various compression parameters. JPEG 2000 encoders that NIST may use include 

OpenJPEG [6] and Kakadu [7]. Participants are requested to submit implementations that can convert a raw iris 

image into an ISO/IEC 19794-6 Type 7 (cropped and masked) image (as shown in Figure 3). Support for this 

operation is optional but encouraged. 

119 2.1.4 Single-eye and Dual-eye Testing 

120 NIST will evaluate performance for scenarios where: 

121 • one iris sample is available per person. 

122 • two samples (of opposite eyes) are available per person. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
2.1 Performance Metrics 6 

Figure 3: An example of an ISO/IEC 19794-6 Type 7 (cropped and masked) image. 

Due to the high frequency of erroneous (left/right) eye labelings in the OPS-II dataset, NIST will no longer provide 

labeling information for iris samples. All samples will simply be labeled "U", indicating "Unknown". NIST suspects 123 the mislabelings are due to ambiguity with respect to whether "left" is intended to represent the subject’s left eye 

(correct) or the eye on the left from the perspective of the camera operator (incorrect). 

NIST will never provide more than two samples per person. Although eye labels will not be provided, it can be 124 assumed that if two samples are provided, they represent opposite eyes of the same person. 

125 When testing single-eye performance, NIST will enroll left and right eyes of one person under different identifers 

126 as though they came from different persons. This will allow NIST to test over larger enrollment databases. The test 

127 harness will never enroll two samples of the same iris under different identifers. 

128 2.1.5 Accuracy-speed Trade-off 

129 NIST will perform an analysis of the trade-off between speed and accuracy. However, participants are no longer re130 quested to submit implementations of varying speeds. Rather, participants should submit different implementations 

131 that are each optimized to a different set of cost function parameters (see Section 2.1.2). 

132 2.1.6 Timing Statistics 

133 NIST will report the computation time for all core functions of the implementations (e.g. feature extraction, search134 ing). As was done in previous IREX evaluations, search time will be plotted as a function of enrollment size with 

135 a focus on whether the trend is sub-linear for any of the implementations. Batch mode processing, where more 

136 than one search is conducted at a time, will not be tested. Timing estimates will be made on an unloaded machine 

137 running a single process at a time. The machine’s specifcations are described in Section 2.3.0.1. 

138 2.1.7 Template Sizes 

139 The size of the proprietary templates generated by the implementations is relevant because it impacts storage 

140 requirements and computational effciency. Therefore, NIST will report statistics on the size of enrollment and 

141 identifcation templates. 

142 2.1.8 Runtime Memory Usage 

143 NIST will monitor runtime memory usage during one-to-many searches and report the results. 

144 2.1.9 Automated Quality Assessment 

145 Automated quality assessment has a number of useful applications in iris recognition (e.g. determining in real-time 

146 whether a sample should be reacquired during a capture session). Automated quality assessment of iris samples 

147 was the primary focus of IREX II: IQCE [8]. In IREX IV, NIST will analyze the quality scores returned by the 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
2.2 Iris Datasets 7 

148 implementations during feature extraction. Error vs. reject curves, as described in [9], will be plotted. NIST may 

149 choose to perform additional analyses with an emphasis on how strongly quality scores correlate with matching 

150 accuracy. Support for automated quality assessment in the submitted implementations is optional. 

151 2.2 Iris Datasets 

152 2.2.1 The OPS-II Dataset 

The primary test dataset for this evaluation is identical to the OPS dataset used in IREX III with one notable 

153 exception: The images in the current dataset were never compressed, while the vast majority of those in the 

original OPS dataset had been previously compressed using JPEG at a quality setting of 75. 

The OPS-II consists of several million operational images collected from 18 distinct commercial iris cameras. 

Some subjects’ irides were captured by more than one camera model. Most of the iris images have a pixel resolution of 640x480, but some are 480x480. NIST intends to exclude the pathological 330x330 images discussed 

154 in IREX III from this evaluation. Some of the non-pathological images still have poor sample quality (e.g. high 

amounts of occlusion, specular refections, heavy pupillary constriction). Some were captured outside and contain heavily constricted pupils. See the IREX III Supplement I [2] for more information. Search and enrollment 

samples will be pulled from the same source and will therefore be of comparable quality. 

155 2.2.2 Ground Truth Integrity 

156 A hazard with collecting operational data is that ground truth identity labels can be incorrectly assigned due to 

157 clerical error. A Type I error occurs when a person’s iris image is present under two or more identities. To correct 

158 for this type of error during evaluation, NIST will estimate FPIR using search images that have been horizontally 

159 fipped1 . The effect of fipping is discussed in the IREX III report. Type II errors occur when two or more persons 

160 are assigned the same subject identifer, which can lead to apparent false negatives. NIST cannot correct for this 

161 type of error, but analyses in IREX III and its supplement indicate that Type II errors accounted for only a small 

162 fraction of the false negatives that occurred when the algorithms were tested over the OPS dataset. 

163 2.3 Test Environment 

164 2.3.0.1 Hardware Specifcations 

165 The test machines are high-end PC-class blades, each having 4 CPUs with 4 cores per CPU. The blades are 

166 labeled Dell M905, equipped with 4x Qual Core AMD Opteron 8376HE processors2 running at 2.3GHz. Each 

167 CPU has 512K of cache. The bus runs at 667 MHz. Main memory consists of 192GB as 24 8GB modules. 

168 Sixteen processes can run without time slicing. NIST may use some test machines that have slightly different 

169 hardware specifcations, but the operating system and compilation environment will remain homogenous across 

170 all blades. Furthermore, timing statistics will only be computed on machines having the aforementioned hardware 

171 specifcations. 

172 2.3.0.2 Operating System 

173 The test machines will have CentOS 6.2 installed, which runs Linux kernel 2.6.32-220.7.1 (http://www.centos.org/). 

174 An ISO image of the distribution can be downloaded from NIGOS (http://nigos.nist.gov:8080/evaluations/CentOS6.2-x86_64-bin-DVD1.iso). 

1Using the jpegtran application provided by the Independent JPEG Group, present on most LINUX platforms. 2cat /proc/cpuinfo returns fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat

npse36 clflush mmx fxsr sse sse2 ht syscall nx pni

IREX IV: Concept, Evaluation Plan, and API Specifcation 
2.4 Reporting of Results 8 

175 2.4 Reporting of Results 

176 2.4.1 Final Report 

177 Following completion of the testing, NIST will publish one or more Interagency Reports (IRs) on the results. NIST 

178 may also use the results to publish in other academic journals or present at conferences or workshops. 

179 2.4.2 Interim Reports 

NIST will provide participants with "score-card" performance results prior to the release of the fnal report. These 

interim reports will be sent as they become available, so participants who submit earlier are more likely to receive 

180 their results sooner. NIST expects the turn-around time to be about two weeks. A participant may submit one pair 

of Class P and Class N libraries followed by another pair after receipt of the interim report. To receive an interim 

report, the submission must be received by June 28, 2012. 

181 While the score cards can be used by the participants for arbitrary purposes, they are intended to promote develop182 ment and to provide the participants with a faster turnaround on how well their implementations performed. Score 

183 cards will be auto-generated for each implementation and will 1) include timing, accuracy, and other performance 

184 statistics, 2) include results from other participants without identifying them, 3) be expanded and modifed as addi185 tional analyses are performed, and 4) be released asynchronously with implementation submissions. NIST does 

186 not intend to release the score cards publicly, though it may show them to U.S. government test sponsors. While 

187 the score cards are not intended for wider distribution, NIST can only request that sponsoring agencies not release 

188 their content. 

189 3 Software Submission 

190 3.1 Participation Requirements 

191 Participation is open to any commercial organization or academic institution that has the ability to implement a 

192 large-scale one-to-many iris identifcation algorithm. There is no charge and participation is open worldwide. 

193 The following rules apply: 

Participants must complete and submit the Participation Agreement 194 • (http://biometrics.nist.gov/cs_links/iris/irexIV/IREX_IV_Application_v1.pdf). 

195 • Participants must submit at least one Class P, and one Class N, implementation. 

Participants are permitted to submit up to two Class N and two Class P implementations (so up to four 196 • submissions in total are permitted). 

197 • Participants must adhere to the cryptographic protection procedures when submitting their implementations 

198 (see Section 3.2). 

199 • All implementations must successfully validate to ensure their proper operation. 

200 NIST will not perform phased testing (i.e. the submission window will close before NIST provides participants with 

201 preliminary results). 

202 3.2 Submission Procedure 

203 All software, data, and confguration fles submitted to NIST must be signed and encrypted. Signing is performed to 

204 ensure authenticity of the submission (i.e. that it actually belongs to the participant). Encryption is performed to enIREX IV: Concept, Evaluation Plan, and API Specifcation 
3.3 Requirements for Library Submissions 9 

205 sure privacy. The full process is described at http://biometrics.nist.gov/cs_links/iris/irex/NIST_biometrics_crypto.pdf . 

206 Note: NIST will not accept any submissions that are not signed and encrypted. NIST accepts no responsibility for 

207 anything that occurs as a result of receiving fles that are not encrypted with the NIST public key. 

208 Implementations shall be submitted to NIST as encrypted gpg fles. If the encrypted implementation is below 20MB, 

209 it can be emailed directly to NIST at irex@nist.gov. If the encrypted implementation is above 20MB, it can either be 

210 provided to NIST as a download from a webserver3 , or mailed as a CD/DVD to the following address: 

211 IREX IV Test Liason (A214)

212 100 Bureau Drive

213 A214/Tech225/Stop 8940

214 NIST

215 Gaithersburg, MD 20899-8940

216 USA

217 Upon receipt, NIST will validate the implementation to ensure its correct operation. The validation process involves 

218 running the implementation over a small sample of test data. This test data will be provided to the participant, who 

219 must run the implementation in-house and provide NIST with the comparison results. NIST will then verify that 

220 the participant’s in-house results are consistent with the output produced on the NIST blades. The test data along 

221 with full instructions will be posted on the IREX IV homepage ( http://www.nist.gov/itl/iad/ig/irexiv.cfm ) as part of a 

222 validation suite. 

223 3.3 Requirements for Library Submissions 

224 Participants shall provide NIST will pre-compiled and linkable libraries. Dynamic libraries are permitted, but static 

225 ones are preferred. Participants shall not provide any source code. Header fles should not be necessary, but if 

226 provided, should not contain intellectual property of the company nor any material that is otherwise proprietary. 

227 At least one "core" library must be submitted that adheres to the API specifcation in section 4.2. This library shall 

228 adhere to the naming convention described in Table 6. Additional dynamic or shared library fles may be submitted 

229 that support this core library. 

Table 6: Naming convention for an implementation library. 

Form: libIREX_provider_class_sequence.suffx 

Part: libIREX provider classes sequence suffx 

Description: First part of 

the name, 

fxed for all 

submissions 

a single word name 

of the main provider. 

EXAMPLE: thebes 

Functional class 

described in 

Table 5 (N or P). 

A two-digit decimal 

identifer starting at 

00 and incrementing 

any time a new 

submission is sent to 

NIST 

Either 

.so or .a 

Example: libIREX_thebes_N_03.a 

230 Implementation libraries must be 64-bit. This will support large memory allocations that are necessary when an 

231 enrollment database contains millions of entries. To achieve faster running times, NIST expects implementations 

232 will load the enrollment templates into main memory before the enrollment database is searched. It is safe to 

233 assume that NIST will not build enrollment databases containing more than 10 million entries (generated from 10 

234 million iris samples). This means that template sizes should not exceed ~19K on average. 

235 NIST will ignore requests to alter parameters by hand (e.g. modify specifc lines in an XML confguration fle). Any 

236 such adjustments must be submitted as a new implementation. 

3NIST shall not be required to register or enroll in any kind of membership before downloading the implementation. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
3.4 Linking Requirements 10 

237 3.4 Linking Requirements 

238 NIST will link the submitted library fle(s) to our ISO 98/99 C/C++ language test drivers. Participants are required 

239 to provide their libraries in a format that is linkable using gcc version 4.4.6 . The standard libraries are: 

240 • /usr/lib64/libstdc++.so.6.0.13 (GLIBCXX 3.4.13)

241 • /lib/libc.so.6 -> libc-2.12.so (GLIBC 2.12)

242 • /lib/libm.so.6 -> libm-2.12.so

243 Participants may provide customized command-line linking parameters. A typical link line might be: 

244 gcc -I. -Wall -m64 -o irex_main irex_main.c -L. -lirex_thebes_N_01 -lpthread

245 Participants are strongly advised to verify library-level compatibility with gcc (on an equivalent platform) prior to 

246 submitting their software to NIST to avoid linkage problems (e.g. symbol name and calling convention mis247 matches, incorrect binary fle formats, etc.). Intel ICC is not available. Access to GPUs is not permitted. 

Intel Integrated Performance Primitives (IPP) libraries are permitted if they are delivered as part of the developer- 248 supplied library package. It is the provider’s responsibility to establish proper licensing of all libraries. 

249 On request, NIST will allow the use of g++ for linking, but the library must export its functions according to the C 

250 linkage specifed in the API. The Standard C++ library is available. 

251 Dependencies on external dynamic/shared libraries such as compiler-specifc development environment libraries 

252 are discouraged. If absolutely necessary, external libraries must be provided to NIST after receiving prior approval 

253 from the test liaison. Image processing libraries such as libpng and NetPbm should not be required since NIST will 

254 handle image reading and decompression. 

255 IMPORTANT: Windows machines will not be used for testing. Windows-compiled libraries are not permitted. All 

256 software must run under LINUX. 

257 3.5 Single-thread Requirement 

Implementations must run in single-threaded mode. Multithreading was supported in IREX III, but all participants 258 eventually chose to submit single-threaded implementations during the latter stages of phased testing. 

259 3.6 Installation Requirements 

260 3.6.1 Installation Must be Simple 

261 Installation shall require the simple copying of fles followed by a linking operation. There shall be no need for 

262 interaction with the participant provided everything goes smoothly. It shall not require an installation program. 

263 3.6.2 No License Requirements or Usage Restrictions 

264 The implementation shall allow itself to be executed on any number of machines without the need for machine265 specifc license control procedures or activation. The implementation shall neither implement nor enforce any usage 

266 controls or restrictions based on licenses, number of executions, presence of temporary fles, etc. No activation 

267 dongles or other hardware shall be required. The implementations shall remain operable until at least October 31st, 

268 2013. 

269 3.6.3 Suffcient Documentation Must be Provided 

270 Participants shall provide complete documentation of their implementations and detail any additional functionality 

271 or behavior beyond those specifed here. The documentation must defne all (non-zero) vendor-defned error or 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
3.7 Runtime Behavior Requirements 11 

272 warning return codes. 

273 3.6.4 Disk-Space Limitations 

274 The implementation may use confguration fles and supporting data fles. The total size of all libraries and confg275 uration and data fles shall be no more than a gigabyte. 

276 3.7 Runtime Behavior Requirements 

277 NOTE: If an implementation is buggy or does not comply with these requirements, NIST may not test or report 

278 results for the implementation in publications. 

279 3.7.1 No writing to Standard Error or Standard Output 

280 The implementation will be tested in a non-interactive "batch" mode without terminal support. Thus, the submitted 

281 library shall run quietly (i.e. it should not write messages to "standard error" or "standard output". An implementation 

282 may write debugging messages to a log fle. This log fle must be declared in the documentation. 

283 3.7.2 Exception Handling Should be Supported 

284 The implementation should support error/exception handling so that, in the case of an unexpected error, a return 

285 code is still provided to the calling application. The NIST test harness will gracefully terminate itself if it receives an 

286 unexpected return code, as it usually indicates improper operation of the implementation. 

287 3.7.3 No External Communication 

288 Implementations running on NIST hosts shall not side-effect the runtime environment in any manner except through 

289 the allocation and release of memory. Implementations shall not write any data to an external resource (e.g. a 

290 server, connection, or other process). Implementations shall not attempt to read any resource other than those 

291 explicitely allowed in this document. If detected, NIST reserves the right to cease evaluation of the software, notify 

292 the participant, and document the activity in published reports. 

293 3.7.4 Components Must be Stateless 

294 All implementation components shall be "stateless" except as noted elsewhere in this document. This applies to 

295 iris detection, feature extraction and matching. Thus, all functions should give identical output, for a given input, 

296 independent of the runtime history. NIST will institute appropriate tests to detect stateful behavior. If detected, NIST 

297 reserves the right to cease evaluation of the software, notify the participant, and document the activity in published 

298 reports. 

299 3.7.5 No Switches or Command-line Options 

300 Each implementation must be capable of running stand-alone (i.e. no two submissions shall depend on the same 

301 copies of libraries or confguration fles). Each implementation shall support only one "mode" of operation. NIST 

302 will not entertain the option to "fip a switch" or modify a confguration fle to produce a new implementation. Rather, 

303 the participant must submit each "mode" as a separate implementation. 

304 3.7.6 Handling Large Enrollment Templates 

305 Enrollment templates should not require more than 200K of persistent storage, on average, per enrolled image. 

306 Participants should inform NIST if their implementations require more than 100K of persistent storage. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
4 API Specifcation 12 

307 3.7.7 Minimum Speed Requirements 

308 The implementations shall perform operations within the time constraints specifed by Table 7. These time limits 

309 apply to the function call invocations defned in Section 7. Since NIST cannot regulate the maximum runtime per 

310 operation, limitations are specifed as 90th percentiles (i.e. 90% of all calls to the function shall complete in less 

311 time than the specifed duration). The limitations assume each template was generated from a single iris sample. 

Table 7: Time limitations for specifc operations. 

Operaton Timing Restriction 

Creation of an enrollment template from a single 640x480 pixel image 1,000 ms 

Creation of an identifcation template from a single 640x480 pixel image 1,000 ms 

Finalization of a 1 million template enrollment database 7,200,000 ms 

Search duration on a database of one million templates 20,000 ms 

312 3.7.8 Failed Template Generations 

313 When the implementation fails to produce an enrollment template, it shall still return a blank template (which can 

314 be zero bytes in length). The template will be included in the manifest like all other enrollment templates, but is not 

315 expected to contain any feature information. 

316 4 API Specifcation 

317 4.1 Overview 

318 Library submissions must export and properly implement all of the functions defned in this section. The testing 

319 process will proceed in two phases: (1) enrollment, followed by (2) identifcation. The order in which the test 

320 harness will call the functions is outlined in Table 8. 

321 The design refects the following testing objectives: 

322 • Support distributed enrollment on multiple machines, with multiple processes running in parallel.

323 • Support graceful failure recovery and the ability to log the frequency of errors.

324 • Respect the black-box nature of proprietary templates.

325 • Provide fexibility and freedom to the participant to use arbitrary algorithms.

326 • Support the ability to collect timing statistics for specifc operations.

327 • Support the ability to collect statistics on template sizes.

Table 8: Program Flow 

Stage 

IREX IV: Concept, Evaluation Plan, and API Specifcation 

Function Metrics of Interest 
4.1 Overview 13 

Enrollment initialize_enrollment_session() 

Allows the implementation to perform initialization procedures. 

Provides the implementation with: 

• advanced notice of the number of individuals and images 

that will be enrolled. 

• read-only access to the participant-supplied confguration 

data directory. 

• read-only access to the directory where the enrollment 

database will reside. 

convert_multiiris_to_enrollment_template() 

Generates an enrollment template from one or more images of an 

individual. The implementation is permitted read-only access to 

the enrollment directory at this stage. The implementation must 

be able to handle multiple calls to this function from multiple instances of the calling application. 

Statistics on template 

size and generation 

time. 

fnalize_enrollment() 

Constructs an enrollment database from the enrollment templates. Templates are provided to the function through a manifest 

fle. The contents of the enrollment directory should be populated 

with everything that is necessary to perform searches against 

it. This function allows post-enrollment book-keeping, normalization, and other statistical processing of the templates. 

Pre-search initialize_feature_extraction_session() 

Prepares the implementation for the generation of identifcation 

templates. The implementation is allowed read-only access to 

the enrollment directory during this stage. 

convert_multiiris_to_identifcation_template() 

Generates an identifcation template from one or more images of 

an individual. 

Statistics on template 

size and generation 

time. 

Search initialize_identifcation_session() 

Prepares the implementation for searches against the enrollment 

database. The function may read data (e.g. templates) from the 

enrollment directory and load them into memory. 

identify_template() 

Searches a template against the enrollment database and returns a list of candidates. 

Statistics on search 

time and accuracy. 

Compression 

(optional) 

convert_raster_to_cropped_and_masked() 

Converts a raw image to an ISO/IEC 19794-6 Type 7 (cropped 

and masked) image. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
4.2 Functions 14 

328 4.2 Functions 

329 Functions 

330 • int32_t get_pid (char ∗sdk_identifer, char ∗email_address) 

331 Retrieves a self-assigned identifer and contact email address for the software under test. 

332 • int32_t get_max_template_sizes (uint32_t ∗max_enrollment_template_size, uint32_t ∗max_recognition_-

333 template_size) 

334 Retrieves the maximum (per-image) enrollment and search template sizes. 

335 • int32_t initialize_enrollment_session (const char ∗confguration_location, const char ∗enrollment_directory, 

336 const uint32_t num_persons, const uint32_t num_images) 

337 Initialization function, called once prior to one or more calls to convert_multiiris_to_enrollment_template(). 

338 • int32_t convert_multiiris_to_enrollment_template (const MULTIIRIS ∗input_irides, MULTISEGMENTATION 

339 ∗output_properties, uint32_t ∗template_size, uint8_t ∗proprietary_template) 

340 Generates an enrollment template from a MULTIIRIS object. 

341 • int32_t fnalize_enrollment (const char ∗enrollment_directory, const char ∗edb_name, const char ∗edb_-

342 manifest_name) 

343 Finalization function, used to construct an enrollment database from an EDB and its manifest. 

344 • int32_t initialize_feature_extraction_session (const char ∗confguration_location, const char ∗enrollment_-

345 directory, uint64_t ∗expected_memsize) 

346 Initialization function, to be called once prior to one or more calls to convert_multiiris_to_identifcation_template(). 

347 • int32_t convert_multiiris_to_identifcation_template (const MULTIIRIS ∗input_irides, MULTISEGMENTATIO348 N ∗output_properties, uint32_t ∗template_size, uint8_t ∗identifcation_template) 

349 Generates an identifcation template from a MULTIIRIS object. 

350 • int32_t initialize_identifcation_session (const char ∗confguration_location, const char ∗enrollment_-

351 directory) 

352 Initialization function, to be called once prior to one or more calls to identify_template(). 

353 • int32_t identify_template (const uint8_t ∗identifcation_template, const uint32_t identifcation_template_size, 

354 const uint32_t candidate_list_length, CANDIDATE ∗const ∗candidate_list, uint8_t ∗decision ) 

355 Searches a template against the enrollment database and returns a list of candidates. 

356 • int32_t convert_raster_to_cropped_and_masked (const ONEIRIS ∗input_iris, ONEIRIS ∗output_iris) 

357 Convert a raw (640x480 or 480x480) image to an ISO/IEC 19794-6 Type 7 (cropped and masked) image. 

358 4.2.1 Function Documentation 

359 4.2.1.1 int32_t get_pid (char ∗ sdk_identifer, char ∗ email_address ) 

360 Retrieves a self-assigned identifer and contact email address for the software under test. 

Parameters

361 

362 

out sdk_identifer A hexidecimal integer stored as a null terminated ASCII string. The value can 

be whatever the participant chooses, but must be unique for each implementation. 5 bytes will be pre-allocated for this. 

out email_address The point of contact for the software under test, stored as a null terminated 

ASCII string. 64 bytes will be pre-allocated for this. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
4.2 Functions 15 

Returns

363 Zero indicates success. Other values indicate a vendor-defned failure. 

364 4.2.1.2 int32_t get_max_template_sizes (uint32_t ∗ max_enrollment_template_size, 

365 uint32_t ∗ max_recognition_template_size ) 

366 Retrieves the maximum (per-image) enrollment and search template sizes. 

367 These values will be used by the test harness to pre-allocate space for template data. For a MULTIIRIS containing K 

368 images, the test-harness will pre-allocate K times the provided value before calling convert_multiiris_to_enrollment369 _template() or convert_multiiris_to_identifcation_template(). 

Parameters

370 

371 

out max_enrollment-

_template_size 

The maximum (per-image) size of an enrollment template in bytes. 

out max_-

recognition_-

template_size 

The maximum (per-image) size of a search template in bytes. 

Returns

372 Zero indicates success. Other values indicate a vendor-defned failure. 

373 4.2.1.3 int32_t initialize_enrollment_session (const char ∗ confguration_location, 

374 const char ∗ enrollment_directory, const uint32_t num_persons, const uint32_t 

375 num_images ) 

376 Initialization function, called once prior to one or more calls to convert_multiiris_to_enrollment_template(). 

377 The implementation shall tolerate execution of multiple calls to this function from different processes running on the 

378 same machine. Each process may be reading and writing to the enrollment directory. 

Parameters

379 

380 

in confguration_-

location 

Path to a read-only directory containing vendor-supplied confguration parameters and/or runtime data fles. 

in enrollment_-

directory 

The directory will be initially empty, but may have been initialized and populated 

by separate invocations of the enrollment process. The software may populate 

this folder in any manner it sees ft. 

in num_persons The number of persons who will be enrolled in the database. 

in num_images The number of images, summed over all identities, that will be used to build the 

enrollment database. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
16 

398 

4.2 Functions 

Returns

381 

382 

Return Value Meaning 

0 Success 

2 The confguration data is missing, unreadable, or in an unexpected format. 

4 An operation on the enrollment directory failed (e.g. insuffcient permissions, insuffcient disk-space, etc). 

6 The software cannot support the number of persons or images requested 

Other Vendor-defned failure 

383 4.2.1.4 int32_t convert_multiiris_to_enrollment_template (const MULTIIRIS ∗

384 input_irides, MULTISEGMENTATION ∗ output_properties, uint32_t ∗

385 template_size, uint8_t ∗ proprietary_template ) 

386 Generates an enrollment template from a MULTIIRIS object. 

In addition to handling raw OPS-II images, this function must be able to process ISO/IEC 19794-6 Type 7 (cropped 387 and masked) images. 

388 If the function returns a zero exit status, the calling application will store the template in the EDB, which is later be 

389 passed to fnalize_enrollment(). If the function returns a value of 8, NIST will debug. Otherwise, a non-zero return 

390 value will indicate a failure to enroll. The template will still be added to the EDB and the manifest to ensure that an 

391 N person enrollment database contains N entries. If the function crashes, NIST will include a zero-length template 

392 in the EDB and the manifest. The fnalization process must be able to process zero-length templates. 

393 IMPORTANT: The implementation shall not attempt to write to the enrollment directory (nor to other resources) 

394 during this call. Data collected from the MULTIIRIS object should be stored in the template or created from the 

395 templates during the fnalization step. 

Parameters

396 

397 

in input_irides The iris samples from which to generate the template. 

out output_-

properties 

Segmentation and quality information for each iris sample. The NIST test harness will pre-allocate the memory for the ONESEGMENTATION objects (one 

per ONEIRIS object). The calling application shall NOT initialize this memory. 

out template_size The size, in bytes, of the output template. 

out proprietary_-

template 

Template generated from the MULTIIRIS object. The template’s format is proprietary and NIST will not access any part of it other than to store it in the EDB. 

The memory for the template will be pre-allocated by the NIST test harness. 

The implementation shall not allocate this memory. 

Returns

Return Value Meaning 

0 Success. 

2 Elective refusal to process the MULTIIRIS. 

4 Involuntary failure to extract features. 

6 Elective refusal to produce a template. 

8 Cannot parse the input data. 

Other Vendor-defned failure. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 

399 
17 

414 

4.2 Functions 

400 4.2.1.5 int32_t fnalize_enrollment (const char ∗ enrollment_directory, const char ∗

401 edb_name, const char ∗ edb_manifest_name ) 

402 Finalization function, used to construct an enrollment database from an EDB and its manifest. 

403 Finalization shall be performed after all enrollment processes are complete. It should populate the contents of 

404 the enrollment directory with everything that is necessary to perform searches against it. This function allows 

405 post-enrollment book-keeping, normalization, and other statistical processing of the generated templates. It should 

406 tolerate being called multiple times, altough subsequent calls should probably not do anything. 

407 The format of the two input fles is described in the table below. The enrollment database (EDB) fle stores a 

408 concatenation of the templates generated by calls to convert_multiiris_to_enrollment_template() in binary format. 

409 It does not contain a header or any delimiters between templates. This fle can potentially be several gigabytes 

410 in size. The EDB manifest is an ASCII fle that stores information about each template in the EDB fle. Each line 

411 contains three space-delimited felds specifying the id, length, and offset of the template in the EDB fle. If the EDB 

412 fle contains N templates, the manifest will contain N lines. 

413 For all intents and purposes, the template id can be regarded as a person id. 

Field Description Datatype Size 

Template ID Non-negative decimal integer, not necessarily zero-indexed or in any 

particular order. 

4 bytes 

Template Length Non-negative decimal integer. 4 bytes 

Offset of template 

in EDB fle 

Non-negative decimal integer. 8 bytes 

Example: 

901231 1024 0 

5834891 0 1024 

50403 1024 1024 

... 

Parameters

415 

416 

in enrollment_-

directory 

The top-level directory in which the enrollment database will reside. The implementation will have read and write access to this directory. 

in edb_name The path to a single read-only fle containing the concatenated templates. -

The implementation should extract content from this fle and place it in the 

enrollment directory. 

in edb_manifest_-

name 

The path to a single read-only fle containing the EDB manifest. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
4.2 Functions 18 

Returns

417 

418 

Value Meaning 

0 Success. 

2 Cannot locate the input data - the input fles or names seem incorrect. 

4 An operation on the enrollment directory failed. 

6 One or more template fles are in an incorrect format. 

Other Vendor-defned failure. 

419 4.2.1.6 int32_t initialize_feature_extraction_session ( const char ∗

420 confguration_location, const char ∗ enrollment_directory, uint64_t ∗

421 expected_memsize ) 

422 Initialization function, to be called once prior to one or more calls to convert_multiiris_to_identifcation_template(). 

423 The implementation shall tolerate execution of multiple calls to this function from different processes running on the 

424 same machine. 

Parameters

425 

426 

in confguration_-

location 

Path to a read-only directory containing vendor-supplied confguration parameters and/or runtime data fles. 

in enrollment_-

directory 

The top-level directory in which the enrollment data was placed when fnalize-

_enrollment() was called. 

in expected_-

memsize 

Given the enrollment data, the implementation shall specify the expected or 

peak memory size (in bytes) that will be used during searching. 

Returns

427 

428 

Return Value Meaning 

0 Success. 

2 The confguration data is missing, unreadable, or in an unexpected format. 

4 An operation on the enrollment directory failed. 

Other Vendor-defned failure. 

429 4.2.1.7 int32_t convert_multiiris_to_identifcation_template (const MULTIIRIS ∗

430 input_irides, MULTISEGMENTATION ∗ output_properties, uint32_t ∗

431 template_size, uint8_t ∗ identifcation_template ) 

432 Generates an identifcation template from a MULTIIRIS object. 

In addition to handling raw OPS-II images, this function must be able to process ISO/IEC 19794-6 Type 7 (cropped 433 and masked) images. 

434 If the function returns a zero exit status, the template will be used for matching. If the function returns a value of 8, 

435 NIST will debug. Otherwise, a non-zero return value will indicate a failure to acquire and the template will not be 

436 used in subsequent search operations. 

Parameters

437 

438 

in input_irides The iris samples from which to generate the template. 

out output_-

properties 

Segmentation and quality information for each iris sample. The NIST test harness will pre-allocate the memory for the ONESEGMENTATION objects (one 

per ONEIRIS object). The implementation shall NOT initialize this memory. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
4.2 Functions 19 

out output_-

properties 

Segmentation and quality information for each iris sample. The NIST test harness will pre-allocate the memory for the ONESEGMENTATION objects. 

out template_size The size, in bytes, of the output template 

out identifcation_-

template 

Template generated from the MULTIIRIS object. The template’s format is proprietary and NIST will not access any part of it other to pass it to identify_-

template() and possibly store it temporarily. The memory for the template will 

be pre-allocated by the NIST test harness. The implementation shall not allocate this memory. 

Returns

439 

440 

Return Value Meaning 

0 Success. 

2 Elective refusal to process the MULTIIRIS. 

4 Involuntary failure to extract features. 

6 Elective refusal to produce a template. 

8 Cannot parse the input data. 

Other Vendor-defned failure. 

441 If the MULTIIRIS contains multiple images, then a zero status should be returned as long as feature information 

442 could be extracted from at least one of the images. 

443 4.2.1.8 int32_t initialize_identifcation_session (const char ∗ confguration_location, 

444 const char ∗ enrollment_directory ) 

445 Initialization function, to be called once prior to one or more calls to identify_template(). 

446 The function may read data (e.g. templates) from the enrollment directory and load them into memory. 

Parameters

447 

448 

in confguration_-

location 

Path to a read-only directory containing vendor-supplied confguration parameters and/or runtime data fles. 

in enrollment_-

directory 

The top-level directory in which the enrollment data was placed when fnalize-

_enrollment() was called. 

Returns

449 

450 

Return Value Meaning 

0 Success. 

Other Vendor-defned failure. 

451 4.2.1.9 int32_t identify_template (const uint8_t ∗ identifcation_template, const 

452 uint32_t identifcation_template_size, const uint32_t candidate_list_length, 

453 CANDIDATE ∗const ∗ candidate_list, uint8_t ∗ decision ) 

454 Searches a template against the enrollment database and returns a list of candidates. 

455 NIST will typically set the candidate list length to operationally feasible values (e.g. 20), but may decide to extend it 

456 to values that approach the size of the enrollment database. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
4.2 Functions 20 

Parameters

457 

458 

in identifcation_-

template 

A template generated by a call to convert_multiiris_to_identifcation_template(). 

in identifcation_-

template_size 

The size, in bytes, of the template. 

in candidate_list_-

length 

The length of the candidate list array. 

out

out

candidate_list 

decision 

An array (of length candidate_list_length) of pointers to candidates. Each candidate shall be populated by the implementation and shall be sorted in ascending order of distance score (e.g. the most similar entry shall appear frst). The 

candidate list must be populated with sensible values. The memory for the 

candidates will be pre-allocated by the NIST test harness. 

A boolean decision on whether the implementation believes the top ranked 

candidate matches the identifcation template (1=yes, 0=no). This decision 

should attempt to minimize the cost function for the given class type (see 

Section 2.1.2). 

Returns

459 

460 

Return Value Meaning 

0 Success. 

2 The input template is defective. 

Other Vendor-defned failure. 

461 4.2.1.10 int32_t convert_raster_to_cropped_and_masked (const ONEIRIS ∗ input_iris, 

462 ONEIRIS ∗ output_iris ) 

463 Convert a raw (640x480 or 480x480) image to an ISO/IEC 19794-6 Type 7 (cropped and masked) image. 

This function shall perform the same operations that were required to generate a KIND 7 record in IREX I. This 

involves cropping the image and masking the sclera and eyelids with a solid color. As described in ISO/IEC 

19794-6, cropping shall provide a margin 0.6R wide on both the left and right sides of the iris. The margin above 464 and below the iris shall be 0.2R. The upper and lower eyelids shall be masked with a color of 128 while the sclera 

shall be masked with a color of 200. The boundary between the sclera and eyelids shall be smoothed. See 

ISO/IEC 19794-6 for further description. 

Implementation of this function is optional. Implementations that do not support cropping and masking shall 

return a value of 2. Otherwise, a zero exit status indicates success and the image will be used for matching. If the 465 function returns a value of 8, NIST will debug. Other return values shall indicate an error and the output image 

will not be used for matching. 

Parameters

466 

467 

in input_iris The input iris. 

out output_iris The result of the masking and cropping operations. Memory for the raster 

data will already have been allocated prior to the function call. The amount of 

memory allocated will be equal to that of the input iris. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
5 Supporting Data Structures 21 

Returns

468 

469 

Return Value Meaning 

0 Success. 

2 The implementation does not support this function. 

4 Involuntary failure to localize boundaries or perform masking. 

6 Elective refusal to produce the output on quality grounds. 

8 Cannot parse the input data. 

Other Vendor-defned failure. 

470 5 Supporting Data Structures 

471 This section describes the data structures used by the API. 

472 5.1 CANDIDATE Struct Reference 

473 Defnes a structure that holds a single candidate. 

474 Public Attributes 

475 • uint8_t failed 

476 Indicates whether the candidate is valid (0=valid, 1-255=invalid). 

477 • uint32_t template_id 

478 Template identifer from the enrollment database. 

479 • double distance_score 

480 Measure of distance between the searched template and the candidate. 

481 • double probability 

482 Estimate of the probability that the biometric data and candidate belong to different persons. 

483 5.1.1 Detailed Description 

484 Defnes a structure that holds a single candidate. 

485 5.1.2 Member Data Documentation 

486 5.1.2.1 uint8_t failed 

487 Indicates whether the candidate is valid (0=valid, 1-255=invalid). 

488 5.1.2.2 uint32_t template_id 

489 Template identifer from the enrollment database. 

490 5.1.2.3 double distance_score 

491 Measure of distance between the searched template and the candidate. 

492 Lower scores indicate greater similarity. The distance score must be non-negative, unless the search template is 

493 somehow broken, in which case it shall be set to -1. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
495

500

505

510

515

520

5.2 MULTIIRIS Struct Reference 22 

494 5.1.2.4 double probability 

Estimate of the probability that the biometric data and candidate belong to different persons. 

496 Stated differently, it shall be the probability that a comparison between two randomly chosen people would produce 

497 a distance score less than or equal to the distance score reported above. 

498 broken, this value shall be set to -1. 

499 5.2 MULTIIRIS Struct Reference 

Defnes a structure that holds an array of irides for a single person. 

501 Public Attributes 

502 • uint32_t num 

503 Number of irides. 

504 • ONEIRIS ∗∗ irides 

Zero-indexed array of pointers to the irides. 

506 5.2.1 Detailed Description 

507 Defnes a structure that holds an array of irides for a single person. 

508 5.2.2 Member Data Documentation 

509 5.2.2.1 uint32_t num 

Number of irides. 

511 5.2.2.2 ONEIRIS∗∗ irides 

512 Zero-indexed array of pointers to the irides. 

513 5.3 MULTISEGMENTATION Struct Reference 

514 Defnes a structure that holds an array of ONESEGMENTATION objects. 

Public Attributes 

516 • uint32_t num 

517 Number of ONESEGMENTATION objects. 

518 • ONESEGMENTATION ∗∗ segs 

519 Zero-indexed array of pointers to ONESEGMENTATION objects. 

5.3.1 Detailed Description 

521 Defnes a structure that holds an array of ONESEGMENTATION objects. 

If the search template is somehow 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
5.4 ONEIRIS Struct Reference 23 

522 5.3.2 Member Data Documentation 

523 5.3.2.1 uint32_t num 

524 Number of ONESEGMENTATION objects. 

525 5.3.2.2 ONESEGMENTATION∗∗ segs 

526 Zero-indexed array of pointers to ONESEGMENTATION objects. 

527 5.4 ONEIRIS Struct Reference 

528 Defnes a structure that holds a single iris with corresponding attributes. 

529 Public Attributes 

530 • uint8_t eye 

531 Eye label (subject’s left or right eye). 

532 • uint16_t image_width 

533 Image width in pixels. 

534 • uint16_t image_height 

535 Image height in pixels. 

536 • uint8_t image_type 

537 Image type integer code. 

538 • uint16_t camera 

539 The camera sensor ID. 

540 • uint8_t ∗ data 

541 Pointer to image raster data, 8 bits-per-pixel. 

542 5.4.1 Detailed Description 

543 Defnes a structure that holds a single iris with corresponding attributes. 

544 5.4.2 Member Data Documentation 

545 5.4.2.1 uint8_t eye 

546 Eye label (subject’s left or right eye). 

The eye label information for the OPS-II dataset has proven unreliable and will not be used for testing. This feld 547 will always be set to 0, indicating that it is unspecifed or unknown. 548 

549 5.4.2.2 uint16_t image_width 

550 Image width in pixels. 

551 5.4.2.3 uint16_t image_height 

552 Image height in pixels. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
5.5 ONESEGMENTATION Struct Reference 24 

553 5.4.2.4 uint8_t image_type 

554 

555 

Image type integer code. 

This feld has different meaning in IREX IV than it did IREX III. A value of 0 indicates that the image will be 

either 640x640 or 480x480 with no geometric constraints on the locations of the pupil or iris boundaries. A 

value of 7 indicates an ISO/IEC 19794-6 Type 7 (cropped and masked) image, the result of a call to convert_raster_to_cropped_and_masked(). 

556 5.4.2.5 uint16_t camera 

557 The camera sensor ID. 

558 This feld will always be set to 0x0000, meaning that it is either unknown or unspecifed. 

559 5.4.2.6 uint8_t∗ data 

560 Pointer to image raster data, 8 bits-per-pixel. 

561 5.5 ONESEGMENTATION Struct Reference 

562 Defnes a structure that holds segmentation and quality information for an iris sample. 

563 Public Attributes 

564 

565 

566 

567 

568 

569 

570 

571 

572 

573 

574 

575 

576 

577 

578 

579 

• double iris_radius 

Iris radius in pixels. 

• uint16_t iris_center_x 

x coordinate of iris center. 

• uint16_t iris_center_y 

y coordinate of iris center. 

• double pupil_radius 

Pupil radius in pixels. 

• uint16_t pupil_center_x 

x coordinate of pupil center. 

• uint16_t pupil_center_y 

y coordinate of iris center. 

• uint8_t quality 

Assessment of iris sample quality. 

• uint8_t failed 

Indicates whether segmentation of the iris failed (0=success, 1=failed). 

580 5.5.1 Detailed Description 

581 Defnes a structure that holds segmentation and quality information for an iris sample. 

582 5.5.2 Member Data Documentation 

583 5.5.2.1 double iris_radius 

584 Iris radius in pixels. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
6 References 25 

585 5.5.2.2 uint16_t iris_center_x 

586 x coordinate of iris center. 

587 5.5.2.3 uint16_t iris_center_y 

588 y coordinate of iris center. 

589 5.5.2.4 double pupil_radius 

590 Pupil radius in pixels. 

591 5.5.2.5 uint16_t pupil_center_x 

592 x coordinate of pupil center. 

593 5.5.2.6 uint16_t pupil_center_y 

594 y coordinate of iris center. 

595 5.5.2.7 uint8_t quality 

596 Assessment of iris sample quality. 

597 Quality is a prediction of how well the sample will perform when matched. 254 indicates quality assessment is 

598 unsupported. 255 indicates a failed attempt to assign quality. Otherwise, quality values shall range from 0 to 100, 

599 with higher values indicating better quality. 

600 5.5.2.8 uint8_t failed 

601 Indicates whether segmentation of the iris failed (0=success, 1=failed). 

602 6 References 

603 [1] P. Grother, G.W. Quinn, J.R. Matey, M. Ngan, W. Salamon, G. Fiumara, and C. Watson. IREX: Performance of 

604 Iris Identifcation Algorithms. Technical report, NIST, 2011. 1, 3 

605 [2] G. Quinn and P. Grother. IREX III supplement I: Failure analysis. Technical report, NIST, 2011. 3, 7 

606 [3] P. Grother, G.W. Quinn, and Jonathan Phillips. Report on the Evaluation of 2D Still-image Face Recognition 

607 Algorithms. Technical report, NIST, 2010. 3 

608 [4] A. Martin, G. Doddington, T. Kamm, M. Ordowski, and M. Przybocki. The DET curve in assessment of detection 

609 task performance. In Proc. Eurospeech, pages 1895–1898, 1997. 3 

610 [5] R. Bolle, J. Connell, S. Pankanti, N. Ratha, and A. Senior. Guide to Biometrics. Springer, 2004. 3 

611 [6] H. Drolon, F. Devaux, A. Descampe, Y. Verschueren, D. Janssens, and B. Macq. OpenJPEG. http://www.

612 openjpeg.org/. 5 

613 [7] David Taubman. Kakadu software. www.kakadusoftware.com. 5 

614 [8] E. Tabassi, P. Grother, and W. Salamon. IREX - IQCE performance of iris image quality assessment algorithms. 

615 Technical report, NIST, 2011. 6 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
6 References 26 

616 [9] Patrick Grother and Elham Tabassi. Performance of biometric quality measures. IEEE Trans. Pattern Anal. 

617 Mach. Intell, pages 531–543, 2007. 7 

IREX IV: Concept, Evaluation Plan, and API Specifcation