IREX IV 

Evaluation of One-to-Many Iris Recognition 

Concept, Evaluation Plan, and API Specifcation 

Version 1.0 

George W. Quinn and Patrick Grother 

Image Group 

Information Access Division 

Information Technology Laboratory 

May 16th, 2012 
i 

1 Status of this Document 

2 This is the third public version of this document. Comments and questions should be submitted to irex@nist.gov. 

3 This document can be downloaded from http://iris.nist.gov/irex. 

4 Timeline 

Table 1: Milestones and deadlines 

Sept 16, 2012 Tentative target deadline for frst public report. 

June 28, 2012 Last day to submit and still receive an interim report back. 

May 16 - August 2, 2012 Submission window. 

May 16 - August 2, 2012 If you have not already done so, organizations are kindly asked whether 

they will participate. Please send a non-binding best guess email to 

irex@nist.gov. 

May 16, 2012 Final API released. 

May 14, 2012 Comments due on version 0.2 of API 

May 4, 2012 NIST releases API version 0.2 

May 1, 2012 Comments due on Initial API 

April 16, 2012 NIST releases API version 0.1 

5 Release Notes 

Changes relating to the third release of this document are highlighted with a pink background color. The only 6 changes are to the timeline above. 

Changes relating to the second release of this document are highlighted with a green background color. Most 

notably: 

1. The submission window has been extended to Aug 2, 2012. 

2. The process of interim reporting of results is explained in greater detail in Section 2.4.2. 7 

3. The dead links pointing to the encryption procedures and participation agreement have been corrected. 

4. The decision parameter of identify_template() is now a pointer type. 

5. gcc version 4.4.6 is now specifed as the system that will be used to link the libraries. 

NOTE: IREX IV is very similar to IREX III with respect to its API and implementation requirements. Notable 8 changes are highlighted throughout this document with a yellow background color. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
10

15

20

25

30

35

40

45

CONTENTS ii 

9 Contents 

1 IREX IV Concepts 1 

11 1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 

12 1.2 Market Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 

13 1.3 Application Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 

14 2 Evaluation Overview 2 

2.1 Performance Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 

16 2.2 Iris Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 

17 2.3 Test Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 

18 2.4 Reporting of Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 

19 3 Software Submission 8 

3.1 Participation Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 

21 3.2 Submission Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 

22 3.3 Requirements for Library Submissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 

23 3.4 Linking Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 

24 3.5 Single-thread Requirement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 

3.6 Installation Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 

26 3.7 Runtime Behavior Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 

27 4 API Specifcation 12 

28 4.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 

29 4.2 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 

4.2.1 Function Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 

31 4.2.1.1 get_pid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 

32 4.2.1.2 get_max_template_sizes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 

33 4.2.1.3 initialize_enrollment_session . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 

34 4.2.1.4 convert_multiiris_to_enrollment_template . . . . . . . . . . . . . . . . . . . . . 16 

4.2.1.5 fnalize_enrollment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 

36 4.2.1.6 initialize_feature_extraction_session . . . . . . . . . . . . . . . . . . . . . . . . 18 

37 4.2.1.7 convert_multiiris_to_identifcation_template . . . . . . . . . . . . . . . . . . . . 18 

38 4.2.1.8 initialize_identifcation_session . . . . . . . . . . . . . . . . . . . . . . . . . . 19 

39 4.2.1.9 identify_template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 

4.2.1.10 convert_raster_to_cropped_and_masked . . . . . . . . . . . . . . . . . . . . . 20 

41 5 Supporting Data Structures 21 

42 5.1 CANDIDATE Struct Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 

43 5.2 MULTIIRIS Struct Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 

44 5.3 MULTISEGMENTATION Struct Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 

5.4 ONEIRIS Struct Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 

46 5.5 ONESEGMENTATION Struct Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 

47 6 References 25 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
1 IREX IV Concepts 1 

48 Terms and Defnitions 

Table 2: The following terms and defnitions are used in this document 

ANSI American National Standards Institute 

ANSI/NIST Type 17 American National Standard for Information Systems - Data Format for the Interchange of Fingerprint, Facial, and Other Biometric Information - Part 1 

API Application Programming Interface 

EDB Enrollment Database 

FNIR False Negative Identifcation Rate 

FPIR False Positive Identifcation Rate 

FTS Failure to search 

FTX Failure to extract features from an enrollment image 

DET Detection Error Tradeoff 

ISO International Standards Organization 

ISO/IEC 19794-6 ISO/IEC standard titled "Information technology - Biometric data interchange 

formats - Part 6: Iris image data" 

ISO/IEC 29794-6 ISO/IEC standard titled "Biometric Sample Quality - Part 1: Framework" 

IREX Iris Exchange 

NIST National Institution of Standards and Technology 

UID India’s Unique Identity scheme 

49 1 IREX IV Concepts 

50 1.1 Overview 

51 This document establishes a concept of operations (CONOPS) and application programming interface (API) for the 

52 Iris Exchange (IREX) IV Evaluation. IREX IV will be a large-scale evaluation of iris recognition technology over 

53 operational data. Like IREX III [1], it will focus exclusively on one-to-many applications. 

54 The goals of this evaluation are 

55 • To investigate the use of cost parameters for application specifc optimization (see Section 2.1.2). 

56 • To establish a compression profle for the effcient and compact storage of iris images (see Section 2.1.3). 

57 • To measure the speed and accuracy of iris matchers over the OPS-II dataset of operational iris images. 

58 This marks the fourth installment in the IREX program (see Figure 1). See http://iris.nist.gov/irex for all IREX related 

59 documentation. 

60 1.2 Market Drivers 

61 This evaluation is intended to support a plural marketplace of iris recognition systems. While the largest applica62 tions, in terms of revenue, have been for border control and war zone identity management, India’s Unique Identity 

63 (UID) scheme is currently using iris (in conjunction with fngerprints) for de-duplication on a massive scale. 

64 The expanding marketplace for iris recognition has fueled the development of iris cameras designed to operate in 

65 a variety of applications. For example: 

66 • Some standoff-capture cameras can rapidly image and verify (in a one-to-many mode) high volumes of 

67 people. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
1.3 Application Scenarios 2 

Figure 1: Current extent of the IREX program as well as planned expansions. 

68 • Some mobile cameras can be preloaded with frm-ware based segmentation and identifcation capability for 

69 rapid one-to-many watchlist searches. 

70 These applications are differentiated by population size, hardware capabilities, quality of the iris samples, and other 

71 variables. 

72 1.3 Application Scenarios 

73 The evaluation will focus on practical applications of iris recognition with an emphasis on large-scale deployments 

74 (i.e. where the enrollment database contains up to several million subjects). The interest is in one-to-many open-set 

75 identifcation systems. Systems operating in a one-to-many mode (sometimes referred to as "identifcation mode") 

76 are tasked with identifying the individual without a prior claim to identity. Open-set means there is no guarantee 

77 that the searched individual is enrolled in the database. To explore the potential for application-specifc algorithm 

78 optimization, participants will submit two classes of implementations, each focusing greater attention on reducing 

79 a different type of error (see section 2.1.2). Table 3 details the parameters of this evaluation. 

80 Participants may also submit implementations that perform cropping and masking of the iris images to convert 

81 them into an ISO/IEC 19794-6 compact format. Representing iris images compactly is crucial for applications 

82 operating over limited-bandwidth networks. India’s Unique Identity (UID) scheme is seeking to reduce bandwidth 

83 requirements for the transmission of iris data. 

84 2 Evaluation Overview 

85 The evaluation will be conducted offine. Offine evaluations are attractive because they allow uniform, fair, re86 peatable, and convenient testing. However, they do not capture all aspects of an operational system. While this 

87 evaluation is designed to mimic operational reality as much as possible, it does not include a live image acquisition 

88 component or any interaction with real users. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
2.1 Performance Metrics 3 

Table 3: Application Parameters 

Parameter Class P 

(Positive Identifcation System) 

Class N 

(Negative Identifcation System) 

Application Type One-to-many open-set identifcation systems 

(e.g. watchlists, de-duplication operations). 

Class Description High cost associated with false 

positives 

High cost associated with false 

negatives 

Example Applications 

Biometric authentication for 

restricted access to high value 

information, resources, or facilities. 

Watchlists for high-profle 

individuals. Investigational-mode 

searches. 

Enrolled Database Size Anywhere from O(102) to O(107) subjects. 

Prior NIST References IREX III Final Report [1] 

IREX III Supplement I: Failure Analysis [2] 

Multiple Biometric Evaluation (MBE) 2010 [3] 

Performance Criteria Primarily accuracy and speed. 

Also, memory usage, scalability, template-size, etc. 

89 2.1 Performance Metrics 

90 2.1.1 Accuracy 

91 Accuracy will be measured for open-set applications, which means that no assumption can be made as to whether 

92 the searched individual is enrolled in the database. Most real-world applications of biometrics operate in this way 

93 (e.g. watchlists and de-duplication tasks). Closed-set applications, which assume that every searched individual 

94 is enrolled in the database (and thus only concern themselves with which of those enrollees the searched person 

95 matches best) are operationally uncommon and will not be tested. 

96 Open-set biometrics systems are tasked with searching an individual against an enrollment database and returning 

97 zero or more candidates. Two types of decision errors are usually considered for this type of system. The frst 

98 occurs when a candidate is returned for an individual that is not enrolled in the database. This is referred to as a 

99 false positive. The second occurs when the correct candidate is not returned for an individual that is enrolled in the 

100 database. This is referred to as a false negative. 

NIST will compute false positive statistics exclusively from non-mated searches and false negative statistics exclusively from mated searches. Although this refects operational reality better than computing false positive statistics 

101 from mated searches (by simply ignoring correct mates on the candidate lists), it does not cover all factors that 

could affect the accuracy of a system (e.g. the position of the correct mate on the candidate list, the number of 

incorrect candidates returned for a mated search). 

102 This evaluation will present core matching accuracy in the form of Detection Error Tradeoff (DET) [4] and Sensitivity103 Reliability [5] plots, both of which show the tradeoff between the two types of error. The Application Programming 

104 Interface (API) will require searches to return a fxed number of candidates but will only consider a candidate 

105 viable if its dissimilarity score is below some decision threshold. Table 4 defnes how the accuracy metrics will be 

106 computed. 

107 In some plots, line segments will be drawn between curves to connect points of equal threshold. These line 

108 segments are intended to show how error rates at specifc operating thresholds vary depending on factors such as 

109 the number of entries in the enrollment database or the quality of the iris samples. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
2.1 Performance Metrics 4 

Table 4: DET and SEL-REL accuracy metrics 

Performance Plot Metric Description 

Detection-error 

FPIR The fraction of non-mated searches for which at least one candidate has 

a distance score at or below threshold. 

Tradeoff Curve FNIR The fraction of mated searches for which the correct candidate is not on 

the list or has a distance score above threshold. 

Selectivity-Reliability 

Curve 

SEL The average number of candidates for a non-mated search having a 

distance score at or below threshold. 

REL One minus FNIR 

110 2.1.2 Cost Function Optimization 

This evaluation will investigate the use of cost parameters for application-specifc algorithm optimization. The 

111 goal is to determine if matching algorithms can be modifed to improve performance when the costs of errors are 

known in advance. The following cost model will be used as an evaluation metric for recognition performance: 

112 

E[ Cost(τ) ] = (1−PMated) FPIR(τ) CP +PMated FNIR(τ) CN (1) 

where PMated is the a priori probability that the user of the system is mated, CP is the cost of a false positive, 

CN is the cost of a false negative, FPIR(τ) is the false positive identifcation rate, FNIR(τ) is the false negative 

113 identifcation rate, and τ is the operating threshold. The model estimates the expected cost per user attempt, 

which could be a measure of time, workload, money, etc. The participant is tasked with minimizing the cost for a 

predetermined and fxed set of cost parameters (CP, CN, and PMated). 

Cost parameters are often chosen to correspond to a specifc application. Consider a biometric system that 

provides bank vault access to specifc individuals. One might reasonably set the cost of a false positive to be the 

114 monetary value of whatever is in the vault, and the cost of a false negative to a value that refects the amount of 

inconvenience incurred from having to open the vault by some other method. Setting PMated to 0.1 assumes that 

one out of every ten access attempts is by an allowed user. 

NIST requires each participant to submit two implementations, each corresponding to a different set of cost 

parameters. These parameters are defned in Table 5. Class P implementations penalize false positives heavily 

115 and false negatives lightly. Class N implementations assign comparatively greater penalty to false negatives. For 

this class of implementations, suppression of false positives is less important. Both classes will be tested over 

one-eye and dual-eye tests. Participants may wish to use a different fusion rule for the two class types. 

Table 5: Cost parameters for both submission types 

Implementation Class CN CP Pmated

Class P 1 10 0.01 

Class N 200 1 0.1 

Additionally, failures to extract (FTXs) and failures to search (FTSs) will be treated differently depending on the 

implementation class. For Class P implementations, both will be treated as failures in a positive recognition 

system (e.g. access control). This is the way NIST has handled FTXs and FTSs in prior evaluations. For Class 

116 N implementations, FTXs and FTSs be treated like failures in a negative recognition system (e.g. a watchlist). 

Failures in a negative recognition system increase the FPIR when they occur for non-mated searches, but do not 

increase the FNIR when they occur for mated searches. This differs from the way NIST has traditionally handled 

these types of failure. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
5 

117 

2.1 Performance Metrics 

The motivation for requiring participants to submit two implementations is to see if it is possible to change the 

shape of a DET to reduce cost for a specifc set of cost parameters. Figure 2 plots standard DET curves for 

two identifcation algorithms. The two curves cross one another, making it impossible to state which is more 

accurate in any absolute sense. Since Class N implementations are penalized heavily for false negatives, and 

only lightly for false positives, both algorithms are expected to achieve their lowest cost toward the right end of the 

fgure, where the blue curve performs better. Conversely, Class P implementations are penalized heavily for false 

positives but only lightly for false negatives. Thus, for this set of cost parameters, both algorithms are expected to 

achieve their lowest cost toward the left end of the fgure, where the red curve performs better. 

Figure 2: Notional DET plots demonstrating how the two classes place greater emphasis on different 

regions of the DET. 

118 2.1.3 JPEG 2000 Compression 

India’s UID scheme will use the iris biometric for recognition tasks, and a desire has been expressed to represent 

iris feature information more compactly to reduce bandwidth usage during network transfer. The ideal solution 

is to store the images according to one of the compact and interoperable formats specifed in ISO/IEC 19794-6. 

This evaluation seeks to further support the standard by establishing JPEG 2000 compression profles for the 

119 effcient and compact storage of iris images. Toward this end, NIST will subject the images to lossy JPEG 2000 

compression while tweaking various compression parameters. JPEG 2000 encoders that NIST may use include 

OpenJPEG [6] and Kakadu [7]. Participants are requested to submit implementations that can convert a raw iris 

image into an ISO/IEC 19794-6 Type 7 (cropped and masked) image (as shown in Figure 3). Support for this 

operation is optional but encouraged. 

120 2.1.4 Single-eye and Dual-eye Testing 

121 NIST will evaluate performance for scenarios where: 

122 • one iris sample is available per person. 

123 • two samples (of opposite eyes) are available per person. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
2.1 Performance Metrics 6 

Figure 3: An example of an ISO/IEC 19794-6 Type 7 (cropped and masked) image. 

Due to the high frequency of erroneous (left/right) eye labelings in the OPS-II dataset, NIST will no longer provide 

labeling information for iris samples. All samples will simply be labeled "U", indicating "Unknown". NIST suspects 124 the mislabelings are due to ambiguity with respect to whether "left" is intended to represent the subject’s left eye 

(correct) or the eye on the left from the perspective of the camera operator (incorrect). 

NIST will never provide more than two samples per person. Although eye labels will not be provided, it can be 125 assumed that if two samples are provided, they represent opposite eyes of the same person. 

126 When testing single-eye performance, NIST will enroll left and right eyes of one person under different identifers 

127 as though they came from different persons. This will allow NIST to test over larger enrollment databases. The test 

128 harness will never enroll two samples of the same iris under different identifers. 

129 2.1.5 Accuracy-speed Trade-off 

130 NIST will perform an analysis of the trade-off between speed and accuracy. However, participants are no longer re131 quested to submit implementations of varying speeds. Rather, participants should submit different implementations 

132 that are each optimized to a different set of cost function parameters (see Section 2.1.2). 

133 2.1.6 Timing Statistics 

134 NIST will report the computation time for all core functions of the implementations (e.g. feature extraction, search135 ing). As was done in previous IREX evaluations, search time will be plotted as a function of enrollment size with 

136 a focus on whether the trend is sub-linear for any of the implementations. Batch mode processing, where more 

137 than one search is conducted at a time, will not be tested. Timing estimates will be made on an unloaded machine 

138 running a single process at a time. The machine’s specifcations are described in Section 2.3.0.1. 

139 2.1.7 Template Sizes 

140 The size of the proprietary templates generated by the implementations is relevant because it impacts storage 

141 requirements and computational effciency. Therefore, NIST will report statistics on the size of enrollment and 

142 identifcation templates. 

143 2.1.8 Runtime Memory Usage 

144 NIST will monitor runtime memory usage during one-to-many searches and report the results. 

145 2.1.9 Automated Quality Assessment 

146 Automated quality assessment has a number of useful applications in iris recognition (e.g. determining in real-time 

147 whether a sample should be reacquired during a capture session). Automated quality assessment of iris samples 

148 was the primary focus of IREX II: IQCE [8]. In IREX IV, NIST will analyze the quality scores returned by the 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
2.2 Iris Datasets 7 

149 implementations during feature extraction. Error vs. reject curves, as described in [9], will be plotted. NIST may 

150 choose to perform additional analyses with an emphasis on how strongly quality scores correlate with matching 

151 accuracy. Support for automated quality assessment in the submitted implementations is optional. 

152 2.2 Iris Datasets 

153 2.2.1 The OPS-II Dataset 

The primary test dataset for this evaluation is identical to the OPS dataset used in IREX III with one notable 

154 exception: The images in the current dataset were never compressed, while the vast majority of those in the 

original OPS dataset had been previously compressed using JPEG at a quality setting of 75. 

The OPS-II consists of several million operational images collected from 18 distinct commercial iris cameras. 

Some subjects’ irides were captured by more than one camera model. Most of the iris images have a pixel resolution of 640x480, but some are 480x480. NIST intends to exclude the pathological 330x330 images discussed 

155 in IREX III from this evaluation. Some of the non-pathological images still have poor sample quality (e.g. high 

amounts of occlusion, specular refections, heavy pupillary constriction). Some were captured outside and contain heavily constricted pupils. See the IREX III Supplement I [2] for more information. Search and enrollment 

samples will be pulled from the same source and will therefore be of comparable quality. 

156 2.2.2 Ground Truth Integrity 

157 A hazard with collecting operational data is that ground truth identity labels can be incorrectly assigned due to 

158 clerical error. A Type I error occurs when a person’s iris image is present under two or more identities. To correct 

159 for this type of error during evaluation, NIST will estimate FPIR using search images that have been horizontally 

160 fipped1 . The effect of fipping is discussed in the IREX III report. Type II errors occur when two or more persons 

161 are assigned the same subject identifer, which can lead to apparent false negatives. NIST cannot correct for this 

162 type of error, but analyses in IREX III and its supplement indicate that Type II errors accounted for only a small 

163 fraction of the false negatives that occurred when the algorithms were tested over the OPS dataset. 

164 2.3 Test Environment 

165 2.3.0.1 Hardware Specifcations 

166 The test machines are high-end PC-class blades, each having 4 CPUs with 4 cores per CPU. The blades are 

167 labeled Dell M905, equipped with 4x Qual Core AMD Opteron 8376HE processors2 running at 2.3GHz. Each 

168 CPU has 512K of cache. The bus runs at 667 MHz. Main memory consists of 192GB as 24 8GB modules. 

169 Sixteen processes can run without time slicing. NIST may use some test machines that have slightly different 

170 hardware specifcations, but the operating system and compilation environment will remain homogenous across 

171 all blades. Furthermore, timing statistics will only be computed on machines having the aforementioned hardware 

172 specifcations. 

173 2.3.0.2 Operating System 

174 The test machines will have CentOS 6.2 installed, which runs Linux kernel 2.6.32-220.7.1 (http://www.centos.org/). 

175 An ISO image of the distribution can be downloaded from NIGOS (http://nigos.nist.gov:8080/evaluations/CentOS6.2-x86_64-bin-DVD1.iso). 

1Using the jpegtran application provided by the Independent JPEG Group, present on most LINUX platforms. 2cat /proc/cpuinfo returns fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat

npse36 clflush mmx fxsr sse sse2 ht syscall nx pni

IREX IV: Concept, Evaluation Plan, and API Specifcation 
2.4 Reporting of Results 8 

176 2.4 Reporting of Results 

177 2.4.1 Final Report 

178 Following completion of the testing, NIST will publish one or more Interagency Reports (IRs) on the results. NIST 

179 may also use the results to publish in other academic journals or present at conferences or workshops. 

180 2.4.2 Interim Reports 

NIST will provide participants with "score-card" performance results prior to the release of the fnal report. These 

interim reports will be sent as they become available, so participants who submit earlier are more likely to receive 

181 their results sooner. NIST expects the turn-around time to be about two weeks. A participant may submit one pair 

of Class P and Class N libraries followed by another pair after receipt of the interim report. To receive an interim 

report, the submission must be received by June 28, 2012. 

182 While the score cards can be used by the participants for arbitrary purposes, they are intended to promote develop183 ment and to provide the participants with a faster turnaround on how well their implementations performed. Score 

184 cards will be auto-generated for each implementation and will 1) include timing, accuracy, and other performance 

185 statistics, 2) include results from other participants without identifying them, 3) be expanded and modifed as addi186 tional analyses are performed, and 4) be released asynchronously with implementation submissions. NIST does 

187 not intend to release the score cards publicly, though it may show them to U.S. government test sponsors. While 

188 the score cards are not intended for wider distribution, NIST can only request that sponsoring agencies not release 

189 their content. 

190 3 Software Submission 

191 3.1 Participation Requirements 

192 Participation is open to any commercial organization or academic institution that has the ability to implement a 

193 large-scale one-to-many iris identifcation algorithm. There is no charge and participation is open worldwide. 

194 The following rules apply: 

Participants must complete and submit the Participation Agreement 195 • (http://biometrics.nist.gov/cs_links/iris/irexIV/IREX_IV_Application_v1.pdf). 

196 • Participants must submit at least one Class P, and one Class N, implementation. 

Participants are permitted to submit up to two Class N and two Class P implementations (so up to four 197 • submissions in total are permitted). 

198 • Participants must adhere to the cryptographic protection procedures when submitting their implementations 

199 (see Section 3.2). 

200 • All implementations must successfully validate to ensure their proper operation. 

201 3.2 Submission Procedure 

202 All software, data, and confguration fles submitted to NIST must be signed and encrypted. Signing is performed to 

203 ensure authenticity of the submission (i.e. that it actually belongs to the participant). Encryption is performed to en204 sure privacy. The full process is described at http://biometrics.nist.gov/cs_links/iris/irex/NIST_biometrics_crypto.pdf . 

205 Note: NIST will not accept any submissions that are not signed and encrypted. NIST accepts no responsibility for 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
3.3 Requirements for Library Submissions 9 

206 anything that occurs as a result of receiving fles that are not encrypted with the NIST public key. 

207 Implementations shall be submitted to NIST as encrypted gpg fles. If the encrypted implementation is below 20MB, 

208 it can be emailed directly to NIST at irex@nist.gov. If the encrypted implementation is above 20MB, it can either be 

209 provided to NIST as a download from a webserver3, or mailed as a CD/DVD to the following address: 

210 IREX IV Test Liason (A214)

211 100 Bureau Drive

212 A214/Tech225/Stop 8940

213 NIST

214 Gaithersburg, MD 20899-8940

215 USA

216 Upon receipt, NIST will validate the implementation to ensure its correct operation. The validation process involves 

217 running the implementation over a small sample of test data. This test data will be provided to the participant, who 

218 must run the implementation in-house and provide NIST with the comparison results. NIST will then verify that 

219 the participant’s in-house results are consistent with the output produced on the NIST blades. The test data along 

220 with full instructions will be posted on the IREX IV homepage ( http://www.nist.gov/itl/iad/ig/irexiv.cfm ) as part of a 

221 validation suite. 

222 3.3 Requirements for Library Submissions 

223 Participants shall provide NIST will pre-compiled and linkable libraries. Dynamic libraries are permitted, but static 

224 ones are preferred. Participants shall not provide any source code. Header fles should not be necessary, but if 

225 provided, should not contain intellectual property of the company nor any material that is otherwise proprietary. 

226 At least one "core" library must be submitted that adheres to the API specifcation in section 4.2. This library shall 

227 adhere to the naming convention described in Table 6. Additional dynamic or shared library fles may be submitted 

228 that support this core library. 

Table 6: Naming convention for an implementation library. 

Form: libIREX_provider_class_sequence.suffx 

Part: libIREX provider classes sequence suffx 

Description: First part of 

the name, 

fxed for all 

submissions 

a single word name 

of the main provider. 

EXAMPLE: thebes 

Functional class 

described in 

Table 5 (N or P). 

A two-digit decimal 

identifer starting at 

00 and incrementing 

any time a new 

submission is sent to 

NIST 

Either 

.so or .a 

Example: libIREX_thebes_N_03.a 

229 Implementation libraries must be 64-bit. This will support large memory allocations that are necessary when an 

230 enrollment database contains millions of entries. To achieve faster running times, NIST expects implementations 

231 will load the enrollment templates into main memory before the enrollment database is searched. It is safe to 

232 assume that NIST will not build enrollment databases containing more than 10 million entries (generated from 10 

233 million iris samples). This means that template sizes should not exceed ~19K on average. 

234 NIST will ignore requests to alter parameters by hand (e.g. modify specifc lines in an XML confguration fle). Any 

235 such adjustments must be submitted as a new implementation. 

3NIST shall not be required to register or enroll in any kind of membership before downloading the implementation. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
3.4 Linking Requirements 10 

236 3.4 Linking Requirements 

237 NIST will link the submitted library fle(s) to our ISO 98/99 C/C++ language test drivers. Participants are required 

238 to provide their libraries in a format that is linkable using gcc version 4.4.6 . The standard libraries are: 

239 • /usr/lib64/libstdc++.so.6.0.13 (GLIBCXX 3.4.13)

240 • /lib/libc.so.6 -> libc-2.12.so (GLIBC 2.12)

241 • /lib/libm.so.6 -> libm-2.12.so

242 Participants may provide customized command-line linking parameters. A typical link line might be: 

243 gcc -I. -Wall -m64 -o irex_main irex_main.c -L. -lirex_thebes_N_01 -lpthread

244 Participants are strongly advised to verify library-level compatibility with gcc (on an equivalent platform) prior to 

245 submitting their software to NIST to avoid linkage problems (e.g. symbol name and calling convention mis246 matches, incorrect binary fle formats, etc.). Intel ICC is not available. Access to GPUs is not permitted. 

Intel Integrated Performance Primitives (IPP) libraries are permitted if they are delivered as part of the developer- 247 supplied library package. It is the provider’s responsibility to establish proper licensing of all libraries. 

248 On request, NIST will allow the use of g++ for linking, but the library must export its functions according to the C 

249 linkage specifed in the API. The Standard C++ library is available. 

250 Dependencies on external dynamic/shared libraries such as compiler-specifc development environment libraries 

251 are discouraged. If absolutely necessary, external libraries must be provided to NIST after receiving prior approval 

252 from the test liaison. Image processing libraries such as libpng and NetPbm should not be required since NIST will 

253 handle image reading and decompression. 

254 IMPORTANT: Windows machines will not be used for testing. Windows-compiled libraries are not permitted. All 

255 software must run under LINUX. 

256 3.5 Single-thread Requirement 

Implementations must run in single-threaded mode. Multithreading was supported in IREX III, but all participants 257 eventually chose to submit single-threaded implementations during the latter stages of phased testing. 

258 3.6 Installation Requirements 

259 3.6.1 Installation Must be Simple 

260 Installation shall require the simple copying of fles followed by a linking operation. There shall be no need for 

261 interaction with the participant provided everything goes smoothly. It shall not require an installation program. 

262 3.6.2 No License Requirements or Usage Restrictions 

263 The implementation shall allow itself to be executed on any number of machines without the need for machine264 specifc license control procedures or activation. The implementation shall neither implement nor enforce any usage 

265 controls or restrictions based on licenses, number of executions, presence of temporary fles, etc. No activation 

266 dongles or other hardware shall be required. The implementations shall remain operable until at least October 31st, 

267 2013. 

268 3.6.3 Suffcient Documentation Must be Provided 

269 Participants shall provide complete documentation of their implementations and detail any additional functionality 

270 or behavior beyond those specifed here. The documentation must defne all (non-zero) vendor-defned error or 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
275

280

285

290

295

300

305

3.7 Runtime Behavior Requirements 11 

271 warning return codes. 

272 3.6.4 Disk-Space Limitations 

273 The implementation may use confguration fles and supporting data fles. The total size of all libraries and confg274 uration and data fles shall be no more than a gigabyte. 

3.7 Runtime Behavior Requirements 

276 NOTE: If an implementation is buggy or does not comply with these requirements, NIST may not test or report 

277 results for the implementation in publications. 

278 3.7.1 No writing to Standard Error or Standard Output 

279 The implementation will be tested in a non-interactive "batch" mode without terminal support. Thus, the submitted 

library shall run quietly (i.e. it should not write messages to "standard error" or "standard output". An implementation 

281 may write debugging messages to a log fle. This log fle must be declared in the documentation. 

282 3.7.2 Exception Handling Should be Supported 

283 The implementation should support error/exception handling so that, in the case of an unexpected error, a return 

284 code is still provided to the calling application. The NIST test harness will gracefully terminate itself if it receives an 

unexpected return code, as it usually indicates improper operation of the implementation. 

286 3.7.3 No External Communication 

287 Implementations running on NIST hosts shall not side-effect the runtime environment in any manner except through 

288 the allocation and release of memory. Implementations shall not write any data to an external resource (e.g. a 

289 server, connection, or other process). Implementations shall not attempt to read any resource other than those 

explicitely allowed in this document. If detected, NIST reserves the right to cease evaluation of the software, notify 

291 the participant, and document the activity in published reports. 

292 3.7.4 Components Must be Stateless 

293 All implementation components shall be "stateless" except as noted elsewhere in this document. This applies to 

294 iris detection, feature extraction and matching. Thus, all functions should give identical output, for a given input, 

independent of the runtime history. NIST will institute appropriate tests to detect stateful behavior. If detected, NIST 

296 reserves the right to cease evaluation of the software, notify the participant, and document the activity in published 

297 reports. 

298 3.7.5 No Switches or Command-line Options 

299 Each implementation must be capable of running stand-alone (i.e. no two submissions shall depend on the same 

copies of libraries or confguration fles). Each implementation shall support only one "mode" of operation. NIST 

301 will not entertain the option to "fip a switch" or modify a confguration fle to produce a new implementation. Rather, 

302 the participant must submit each "mode" as a separate implementation. 

303 3.7.6 Handling Large Enrollment Templates 

304 Enrollment templates should not require more than 200K of persistent storage, on average, per enrolled image. 

Participants should inform NIST if their implementations require more than 100K of persistent storage. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
4 API Specifcation 12 

306 3.7.7 Minimum Speed Requirements 

307 The implementations shall perform operations within the time constraints specifed by Table 7. These time limits 

308 apply to the function call invocations defned in Section 7. Since NIST cannot regulate the maximum runtime per 

309 operation, limitations are specifed as 90th percentiles (i.e. 90% of all calls to the function shall complete in less 

310 time than the specifed duration). The limitations assume each template was generated from a single iris sample. 

Table 7: Time limitations for specifc operations. 

Operaton Timing Restriction 

Creation of an enrollment template from a single 640x480 pixel image 1,000 ms 

Creation of an identifcation template from a single 640x480 pixel image 1,000 ms 

Finalization of a 1 million template enrollment database 7,200,000 ms 

Search duration on a database of one million templates 20,000 ms 

311 3.7.8 Failed Template Generations 

312 When the implementation fails to produce an enrollment template, it shall still return a blank template (which can 

313 be zero bytes in length). The template will be included in the manifest like all other enrollment templates, but is not 

314 expected to contain any feature information. 

315 4 API Specifcation 

316 4.1 Overview 

317 Library submissions must export and properly implement all of the functions defned in this section. The testing 

318 process will proceed in two phases: (1) enrollment, followed by (2) identifcation. The order in which the test 

319 harness will call the functions is outlined in Table 8. 

320 The design refects the following testing objectives: 

321 • Support distributed enrollment on multiple machines, with multiple processes running in parallel.

322 • Support graceful failure recovery and the ability to log the frequency of errors.

323 • Respect the black-box nature of proprietary templates.

324 • Provide fexibility and freedom to the participant to use arbitrary algorithms.

325 • Support the ability to collect timing statistics for specifc operations.

326 • Support the ability to collect statistics on template sizes.

Table 8: Program Flow 

Stage 

IREX IV: Concept, Evaluation Plan, and API Specifcation 

Function Metrics of Interest 
4.1 Overview 13 

Enrollment initialize_enrollment_session() 

Allows the implementation to perform initialization procedures. 

Provides the implementation with: 

• advanced notice of the number of individuals and images 

that will be enrolled. 

• read-only access to the participant-supplied confguration 

data directory. 

• read-only access to the directory where the enrollment 

database will reside. 

convert_multiiris_to_enrollment_template() 

Generates an enrollment template from one or more images of an 

individual. The implementation is permitted read-only access to 

the enrollment directory at this stage. The implementation must 

be able to handle multiple calls to this function from multiple instances of the calling application. 

Statistics on template 

size and generation 

time. 

fnalize_enrollment() 

Constructs an enrollment database from the enrollment templates. Templates are provided to the function through a manifest 

fle. The contents of the enrollment directory should be populated 

with everything that is necessary to perform searches against 

it. This function allows post-enrollment book-keeping, normalization, and other statistical processing of the templates. 

Pre-search initialize_feature_extraction_session() 

Prepares the implementation for the generation of identifcation 

templates. The implementation is allowed read-only access to 

the enrollment directory during this stage. 

convert_multiiris_to_identifcation_template() 

Generates an identifcation template from one or more images of 

an individual. 

Statistics on template 

size and generation 

time. 

Search initialize_identifcation_session() 

Prepares the implementation for searches against the enrollment 

database. The function may read data (e.g. templates) from the 

enrollment directory and load them into memory. 

identify_template() 

Searches a template against the enrollment database and returns a list of candidates. 

Statistics on search 

time and accuracy. 

Compression 

(optional) 

convert_raster_to_cropped_and_masked() 

Converts a raw image to an ISO/IEC 19794-6 Type 7 (cropped 

and masked) image. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
4.2 Functions 14 

327 4.2 Functions 

328 Functions 

329 • int32_t get_pid (char ∗sdk_identifer, char ∗email_address) 

330 Retrieves a self-assigned identifer and contact email address for the software under test. 

331 • int32_t get_max_template_sizes (uint32_t ∗max_enrollment_template_size, uint32_t ∗max_recognition_-

332 template_size) 

333 Retrieves the maximum (per-image) enrollment and search template sizes. 

334 • int32_t initialize_enrollment_session (const char ∗confguration_location, const char ∗enrollment_directory, 

335 const uint32_t num_persons, const uint32_t num_images) 

336 Initialization function, called once prior to one or more calls to convert_multiiris_to_enrollment_template(). 

337 • int32_t convert_multiiris_to_enrollment_template (const MULTIIRIS ∗input_irides, MULTISEGMENTATION 

338 ∗output_properties, uint32_t ∗template_size, uint8_t ∗proprietary_template) 

339 Generates an enrollment template from a MULTIIRIS object. 

340 • int32_t fnalize_enrollment (const char ∗enrollment_directory, const char ∗edb_name, const char ∗edb_-

341 manifest_name) 

342 Finalization function, used to construct an enrollment database from an EDB and its manifest. 

343 • int32_t initialize_feature_extraction_session (const char ∗confguration_location, const char ∗enrollment_-

344 directory, uint64_t ∗expected_memsize) 

345 Initialization function, to be called once prior to one or more calls to convert_multiiris_to_identifcation_template(). 

346 • int32_t convert_multiiris_to_identifcation_template (const MULTIIRIS ∗input_irides, MULTISEGMENTATIO347 N ∗output_properties, uint32_t ∗template_size, uint8_t ∗identifcation_template) 

348 Generates an identifcation template from a MULTIIRIS object. 

349 • int32_t initialize_identifcation_session (const char ∗confguration_location, const char ∗enrollment_-

350 directory) 

351 Initialization function, to be called once prior to one or more calls to identify_template(). 

352 • int32_t identify_template (const uint8_t ∗identifcation_template, const uint32_t identifcation_template_size, 

353 const uint32_t candidate_list_length, CANDIDATE ∗const ∗candidate_list, uint8_t ∗decision ) 

354 Searches a template against the enrollment database and returns a list of candidates. 

355 • int32_t convert_raster_to_cropped_and_masked (const ONEIRIS ∗input_iris, ONEIRIS ∗output_iris) 

356 Convert a raw (640x480 or 480x480) image to an ISO/IEC 19794-6 Type 7 (cropped and masked) image. 

357 4.2.1 Function Documentation 

358 4.2.1.1 int32_t get_pid (char ∗ sdk_identifer, char ∗ email_address ) 

359 Retrieves a self-assigned identifer and contact email address for the software under test. 

Parameters

360 

361 

out sdk_identifer A hexidecimal integer stored as a null terminated ASCII string. The value can 

be whatever the participant chooses, but must be unique for each implementation. 5 bytes will be pre-allocated for this. 

out email_address The point of contact for the software under test, stored as a null terminated 

ASCII string. 64 bytes will be pre-allocated for this. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
4.2 Functions 15 

Returns

362 Zero indicates success. Other values indicate a vendor-defned failure. 

363 4.2.1.2 int32_t get_max_template_sizes (uint32_t ∗ max_enrollment_template_size, 

364 uint32_t ∗ max_recognition_template_size ) 

365 Retrieves the maximum (per-image) enrollment and search template sizes. 

366 These values will be used by the test harness to pre-allocate space for template data. For a MULTIIRIS containing K 

367 images, the test-harness will pre-allocate K times the provided value before calling convert_multiiris_to_enrollment368 _template() or convert_multiiris_to_identifcation_template(). 

Parameters

369 

370 

out max_enrollment-

_template_size 

The maximum (per-image) size of an enrollment template in bytes. 

out max_-

recognition_-

template_size 

The maximum (per-image) size of a search template in bytes. 

Returns

371 Zero indicates success. Other values indicate a vendor-defned failure. 

372 4.2.1.3 int32_t initialize_enrollment_session (const char ∗ confguration_location, 

373 const char ∗ enrollment_directory, const uint32_t num_persons, const uint32_t 

374 num_images ) 

375 Initialization function, called once prior to one or more calls to convert_multiiris_to_enrollment_template(). 

376 The implementation shall tolerate execution of multiple calls to this function from different processes running on the 

377 same machine. Each process may be reading and writing to the enrollment directory. 

Parameters

378 

379 

in confguration_-

location 

Path to a read-only directory containing vendor-supplied confguration parameters and/or runtime data fles. 

in enrollment_-

directory 

The directory will be initially empty, but may have been initialized and populated 

by separate invocations of the enrollment process. The software may populate 

this folder in any manner it sees ft. 

in num_persons The number of persons who will be enrolled in the database. 

in num_images The number of images, summed over all identities, that will be used to build the 

enrollment database. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
16 

397 

4.2 Functions 

Returns

380 

381 

Return Value Meaning 

0 Success 

2 The confguration data is missing, unreadable, or in an unexpected format. 

4 An operation on the enrollment directory failed (e.g. insuffcient permissions, insuffcient disk-space, etc). 

6 The software cannot support the number of persons or images requested 

Other Vendor-defned failure 

382 4.2.1.4 int32_t convert_multiiris_to_enrollment_template (const MULTIIRIS ∗

383 input_irides, MULTISEGMENTATION ∗ output_properties, uint32_t ∗

384 template_size, uint8_t ∗ proprietary_template ) 

385 Generates an enrollment template from a MULTIIRIS object. 

In addition to handling raw OPS-II images, this function must be able to process ISO/IEC 19794-6 Type 7 (cropped 386 and masked) images. 

387 If the function returns a zero exit status, the calling application will store the template in the EDB, which is later be 

388 passed to fnalize_enrollment(). If the function returns a value of 8, NIST will debug. Otherwise, a non-zero return 

389 value will indicate a failure to enroll. The template will still be added to the EDB and the manifest to ensure that an 

390 N person enrollment database contains N entries. If the function crashes, NIST will include a zero-length template 

391 in the EDB and the manifest. The fnalization process must be able to process zero-length templates. 

392 IMPORTANT: The implementation shall not attempt to write to the enrollment directory (nor to other resources) 

393 during this call. Data collected from the MULTIIRIS object should be stored in the template or created from the 

394 templates during the fnalization step. 

Parameters

395 

396 

in input_irides The iris samples from which to generate the template. 

out output_-

properties 

Segmentation and quality information for each iris sample. The NIST test harness will pre-allocate the memory for the ONESEGMENTATION objects (one 

per ONEIRIS object). The calling application shall NOT initialize this memory. 

out template_size The size, in bytes, of the output template. 

out proprietary_-

template 

Template generated from the MULTIIRIS object. The template’s format is proprietary and NIST will not access any part of it other than to store it in the EDB. 

The memory for the template will be pre-allocated by the NIST test harness. 

The implementation shall not allocate this memory. 

Returns

Return Value Meaning 

0 Success. 

2 Elective refusal to process the MULTIIRIS. 

4 Involuntary failure to extract features. 

6 Elective refusal to produce a template. 

8 Cannot parse the input data. 

Other Vendor-defned failure. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 

398 
17 

413 

4.2 Functions 

399 4.2.1.5 int32_t fnalize_enrollment (const char ∗ enrollment_directory, const char ∗

400 edb_name, const char ∗ edb_manifest_name ) 

401 Finalization function, used to construct an enrollment database from an EDB and its manifest. 

402 Finalization shall be performed after all enrollment processes are complete. It should populate the contents of 

403 the enrollment directory with everything that is necessary to perform searches against it. This function allows 

404 post-enrollment book-keeping, normalization, and other statistical processing of the generated templates. It should 

405 tolerate being called multiple times, altough subsequent calls should probably not do anything. 

406 The format of the two input fles is described in the table below. The enrollment database (EDB) fle stores a 

407 concatenation of the templates generated by calls to convert_multiiris_to_enrollment_template() in binary format. 

408 It does not contain a header or any delimiters between templates. This fle can potentially be several gigabytes 

409 in size. The EDB manifest is an ASCII fle that stores information about each template in the EDB fle. Each line 

410 contains three space-delimited felds specifying the id, length, and offset of the template in the EDB fle. If the EDB 

411 fle contains N templates, the manifest will contain N lines. 

412 For all intents and purposes, the template id can be regarded as a person id. 

Field Description Datatype Size 

Template ID Non-negative decimal integer, not necessarily zero-indexed or in any 

particular order. 

4 bytes 

Template Length Non-negative decimal integer. 4 bytes 

Offset of template 

in EDB fle 

Non-negative decimal integer. 8 bytes 

Example: 

901231 1024 0 

5834891 0 1024 

50403 1024 1024 

... 

Parameters

414 

415 

in enrollment_-

directory 

The top-level directory in which the enrollment database will reside. The implementation will have read and write access to this directory. 

in edb_name The path to a single read-only fle containing the concatenated templates. -

The implementation should extract content from this fle and place it in the 

enrollment directory. 

in edb_manifest_-

name 

The path to a single read-only fle containing the EDB manifest. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
4.2 Functions 18 

Returns

416 

417 

Value Meaning 

0 Success. 

2 Cannot locate the input data - the input fles or names seem incorrect. 

4 An operation on the enrollment directory failed. 

6 One or more template fles are in an incorrect format. 

Other Vendor-defned failure. 

418 4.2.1.6 int32_t initialize_feature_extraction_session ( const char ∗

419 confguration_location, const char ∗ enrollment_directory, uint64_t ∗

420 expected_memsize ) 

421 Initialization function, to be called once prior to one or more calls to convert_multiiris_to_identifcation_template(). 

422 The implementation shall tolerate execution of multiple calls to this function from different processes running on the 

423 same machine. 

Parameters

424 

425 

in confguration_-

location 

Path to a read-only directory containing vendor-supplied confguration parameters and/or runtime data fles. 

in enrollment_-

directory 

The top-level directory in which the enrollment data was placed when fnalize-

_enrollment() was called. 

in expected_-

memsize 

Given the enrollment data, the implementation shall specify the expected or 

peak memory size (in bytes) that will be used during searching. 

Returns

426 

427 

Return Value Meaning 

0 Success. 

2 The confguration data is missing, unreadable, or in an unexpected format. 

4 An operation on the enrollment directory failed. 

Other Vendor-defned failure. 

428 4.2.1.7 int32_t convert_multiiris_to_identifcation_template (const MULTIIRIS ∗

429 input_irides, MULTISEGMENTATION ∗ output_properties, uint32_t ∗

430 template_size, uint8_t ∗ identifcation_template ) 

431 Generates an identifcation template from a MULTIIRIS object. 

In addition to handling raw OPS-II images, this function must be able to process ISO/IEC 19794-6 Type 7 (cropped 432 and masked) images. 

433 If the function returns a zero exit status, the template will be used for matching. If the function returns a value of 8, 

434 NIST will debug. Otherwise, a non-zero return value will indicate a failure to acquire and the template will not be 

435 used in subsequent search operations. 

Parameters

436 

437 

in input_irides The iris samples from which to generate the template. 

out output_-

properties 

Segmentation and quality information for each iris sample. The NIST test harness will pre-allocate the memory for the ONESEGMENTATION objects (one 

per ONEIRIS object). The implementation shall NOT initialize this memory. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
4.2 Functions 19 

out output_-

properties 

Segmentation and quality information for each iris sample. The NIST test harness will pre-allocate the memory for the ONESEGMENTATION objects. 

out template_size The size, in bytes, of the output template 

out identifcation_-

template 

Template generated from the MULTIIRIS object. The template’s format is proprietary and NIST will not access any part of it other to pass it to identify_-

template() and possibly store it temporarily. The memory for the template will 

be pre-allocated by the NIST test harness. The implementation shall not allocate this memory. 

Returns

438 

439 

Return Value Meaning 

0 Success. 

2 Elective refusal to process the MULTIIRIS. 

4 Involuntary failure to extract features. 

6 Elective refusal to produce a template. 

8 Cannot parse the input data. 

Other Vendor-defned failure. 

440 If the MULTIIRIS contains multiple images, then a zero status should be returned as long as feature information 

441 could be extracted from at least one of the images. 

442 4.2.1.8 int32_t initialize_identifcation_session (const char ∗ confguration_location, 

443 const char ∗ enrollment_directory ) 

444 Initialization function, to be called once prior to one or more calls to identify_template(). 

445 The function may read data (e.g. templates) from the enrollment directory and load them into memory. 

Parameters

446 

447 

in confguration_-

location 

Path to a read-only directory containing vendor-supplied confguration parameters and/or runtime data fles. 

in enrollment_-

directory 

The top-level directory in which the enrollment data was placed when fnalize-

_enrollment() was called. 

Returns

448 

449 

Return Value Meaning 

0 Success. 

Other Vendor-defned failure. 

450 4.2.1.9 int32_t identify_template (const uint8_t ∗ identifcation_template, const 

451 uint32_t identifcation_template_size, const uint32_t candidate_list_length, 

452 CANDIDATE ∗const ∗ candidate_list, uint8_t ∗ decision ) 

453 Searches a template against the enrollment database and returns a list of candidates. 

454 NIST will typically set the candidate list length to operationally feasible values (e.g. 20), but may decide to extend it 

455 to values that approach the size of the enrollment database. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
4.2 Functions 20 

Parameters

456 

457 

in identifcation_-

template 

A template generated by a call to convert_multiiris_to_identifcation_template(). 

in identifcation_-

template_size 

The size, in bytes, of the template. 

in candidate_list_-

length 

The length of the candidate list array. 

out

out

candidate_list 

decision 

An array (of length candidate_list_length) of pointers to candidates. Each candidate shall be populated by the implementation and shall be sorted in ascending order of distance score (e.g. the most similar entry shall appear frst). The 

candidate list must be populated with sensible values. The memory for the 

candidates will be pre-allocated by the NIST test harness. 

A boolean decision on whether the implementation believes the top ranked 

candidate matches the identifcation template (1=yes, 0=no). This decision 

should attempt to minimize the cost function for the given class type (see 

Section 2.1.2). 

Returns

458 

459 

Return Value Meaning 

0 Success. 

2 The input template is defective. 

Other Vendor-defned failure. 

460 4.2.1.10 int32_t convert_raster_to_cropped_and_masked (const ONEIRIS ∗ input_iris, 

461 ONEIRIS ∗ output_iris ) 

462 Convert a raw (640x480 or 480x480) image to an ISO/IEC 19794-6 Type 7 (cropped and masked) image. 

This function shall perform the same operations that were required to generate a KIND 7 record in IREX I. This 

involves cropping the image and masking the sclera and eyelids with a solid color. As described in ISO/IEC 

19794-6, cropping shall provide a margin 0.6R wide on both the left and right sides of the iris. The margin above 463 and below the iris shall be 0.2R. The upper and lower eyelids shall be masked with a color of 128 while the sclera 

shall be masked with a color of 200. The boundary between the sclera and eyelids shall be smoothed. See 

ISO/IEC 19794-6 for further description. 

Implementation of this function is optional. Implementations that do not support cropping and masking shall 

return a value of 2. Otherwise, a zero exit status indicates success and the image will be used for matching. If the 464 function returns a value of 8, NIST will debug. Other return values shall indicate an error and the output image 

will not be used for matching. 

Parameters

465 

466 

in input_iris The input iris. 

out output_iris The result of the masking and cropping operations. Memory for the raster 

data will already have been allocated prior to the function call. The amount of 

memory allocated will be equal to that of the input iris. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
5 Supporting Data Structures 21 

Returns

467 

468 

Return Value Meaning 

0 Success. 

2 The implementation does not support this function. 

4 Involuntary failure to localize boundaries or perform masking. 

6 Elective refusal to produce the output on quality grounds. 

8 Cannot parse the input data. 

Other Vendor-defned failure. 

469 5 Supporting Data Structures 

470 This section describes the data structures used by the API. 

471 5.1 CANDIDATE Struct Reference 

472 Defnes a structure that holds a single candidate. 

473 Public Attributes 

474 • uint8_t failed 

475 Indicates whether the candidate is valid (0=valid, 1-255=invalid). 

476 • uint32_t template_id 

477 Template identifer from the enrollment database. 

478 • double distance_score 

479 Measure of distance between the searched template and the candidate. 

480 • double probability 

481 Estimate of the probability that the biometric data and candidate belong to different persons. 

482 5.1.1 Detailed Description 

483 Defnes a structure that holds a single candidate. 

484 5.1.2 Member Data Documentation 

485 5.1.2.1 uint8_t failed 

486 Indicates whether the candidate is valid (0=valid, 1-255=invalid). 

487 5.1.2.2 uint32_t template_id 

488 Template identifer from the enrollment database. 

489 5.1.2.3 double distance_score 

490 Measure of distance between the searched template and the candidate. 

491 Lower scores indicate greater similarity. The distance score must be non-negative, unless the search template is 

492 somehow broken, in which case it shall be set to -1. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
495

500

505

510

515

520

5.2 MULTIIRIS Struct Reference 22 

493 5.1.2.4 double probability 

494 Estimate of the probability that the biometric data and candidate belong to different persons. 

Stated differently, it shall be the probability that a comparison between two randomly chosen people would produce 

496 a distance score less than or equal to the distance score reported above. 

497 broken, this value shall be set to -1. 

498 5.2 MULTIIRIS Struct Reference 

499 Defnes a structure that holds an array of irides for a single person. 

Public Attributes 

501 • uint32_t num 

502 Number of irides. 

503 • ONEIRIS ∗∗ irides 

504 Zero-indexed array of pointers to the irides. 

5.2.1 Detailed Description 

506 Defnes a structure that holds an array of irides for a single person. 

507 5.2.2 Member Data Documentation 

508 5.2.2.1 uint32_t num 

509 Number of irides. 

5.2.2.2 ONEIRIS∗∗ irides 

511 Zero-indexed array of pointers to the irides. 

512 5.3 MULTISEGMENTATION Struct Reference 

513 Defnes a structure that holds an array of ONESEGMENTATION objects. 

514 Public Attributes 

• uint32_t num 

516 Number of ONESEGMENTATION objects. 

517 • ONESEGMENTATION ∗∗ segs 

518 Zero-indexed array of pointers to ONESEGMENTATION objects. 

519 5.3.1 Detailed Description 

Defnes a structure that holds an array of ONESEGMENTATION objects. 

If the search template is somehow 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
5.4 ONEIRIS Struct Reference 23 

521 5.3.2 Member Data Documentation 

522 5.3.2.1 uint32_t num 

523 Number of ONESEGMENTATION objects. 

524 5.3.2.2 ONESEGMENTATION∗∗ segs 

525 Zero-indexed array of pointers to ONESEGMENTATION objects. 

526 5.4 ONEIRIS Struct Reference 

527 Defnes a structure that holds a single iris with corresponding attributes. 

528 Public Attributes 

529 • uint8_t eye 

530 Eye label (subject’s left or right eye). 

531 • uint16_t image_width 

532 Image width in pixels. 

533 • uint16_t image_height 

534 Image height in pixels. 

535 • uint8_t image_type 

536 Image type integer code. 

537 • uint16_t camera 

538 The camera sensor ID. 

539 • uint8_t ∗ data 

540 Pointer to image raster data, 8 bits-per-pixel. 

541 5.4.1 Detailed Description 

542 Defnes a structure that holds a single iris with corresponding attributes. 

543 5.4.2 Member Data Documentation 

544 5.4.2.1 uint8_t eye 

545 Eye label (subject’s left or right eye). 

The eye label information for the OPS-II dataset has proven unreliable and will not be used for testing. This feld 546 will always be set to 0, indicating that it is unspecifed or unknown. 547 

548 5.4.2.2 uint16_t image_width 

549 Image width in pixels. 

550 5.4.2.3 uint16_t image_height 

551 Image height in pixels. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
5.5 ONESEGMENTATION Struct Reference 24 

552 5.4.2.4 uint8_t image_type 

553 

554 

Image type integer code. 

This feld has different meaning in IREX IV than it did IREX III. A value of 0 indicates that the image will be 

either 640x640 or 480x480 with no geometric constraints on the locations of the pupil or iris boundaries. A 

value of 7 indicates an ISO/IEC 19794-6 Type 7 (cropped and masked) image, the result of a call to convert_raster_to_cropped_and_masked(). 

555 5.4.2.5 uint16_t camera 

556 The camera sensor ID. 

557 This feld will always be set to 0x0000, meaning that it is either unknown or unspecifed. 

558 5.4.2.6 uint8_t∗ data 

559 Pointer to image raster data, 8 bits-per-pixel. 

560 5.5 ONESEGMENTATION Struct Reference 

561 Defnes a structure that holds segmentation and quality information for an iris sample. 

562 Public Attributes 

563 

564 

565 

566 

567 

568 

569 

570 

571 

572 

573 

574 

575 

576 

577 

578 

• double iris_radius 

Iris radius in pixels. 

• uint16_t iris_center_x 

x coordinate of iris center. 

• uint16_t iris_center_y 

y coordinate of iris center. 

• double pupil_radius 

Pupil radius in pixels. 

• uint16_t pupil_center_x 

x coordinate of pupil center. 

• uint16_t pupil_center_y 

y coordinate of iris center. 

• uint8_t quality 

Assessment of iris sample quality. 

• uint8_t failed 

Indicates whether segmentation of the iris failed (0=success, 1=failed). 

579 5.5.1 Detailed Description 

580 Defnes a structure that holds segmentation and quality information for an iris sample. 

581 5.5.2 Member Data Documentation 

582 5.5.2.1 double iris_radius 

583 Iris radius in pixels. 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
6 References 25 

584 5.5.2.2 uint16_t iris_center_x 

585 x coordinate of iris center. 

586 5.5.2.3 uint16_t iris_center_y 

587 y coordinate of iris center. 

588 5.5.2.4 double pupil_radius 

589 Pupil radius in pixels. 

590 5.5.2.5 uint16_t pupil_center_x 

591 x coordinate of pupil center. 

592 5.5.2.6 uint16_t pupil_center_y 

593 y coordinate of iris center. 

594 5.5.2.7 uint8_t quality 

595 Assessment of iris sample quality. 

596 Quality is a prediction of how well the sample will perform when matched. 254 indicates quality assessment is 

597 unsupported. 255 indicates a failed attempt to assign quality. Otherwise, quality values shall range from 0 to 100, 

598 with higher values indicating better quality. 

599 5.5.2.8 uint8_t failed 

600 Indicates whether segmentation of the iris failed (0=success, 1=failed). 

601 6 References 

602 [1] P. Grother, G.W. Quinn, J.R. Matey, M. Ngan, W. Salamon, G. Fiumara, and C. Watson. IREX: Performance of 

603 Iris Identifcation Algorithms. Technical report, NIST, 2011. 1, 3 

604 [2] G. Quinn and P. Grother. IREX III supplement I: Failure analysis. Technical report, NIST, 2011. 3, 7 

605 [3] P. Grother, G.W. Quinn, and Jonathan Phillips. Report on the Evaluation of 2D Still-image Face Recognition 

606 Algorithms. Technical report, NIST, 2010. 3 

607 [4] A. Martin, G. Doddington, T. Kamm, M. Ordowski, and M. Przybocki. The DET curve in assessment of detection 

608 task performance. In Proc. Eurospeech, pages 1895–1898, 1997. 3 

609 [5] R. Bolle, J. Connell, S. Pankanti, N. Ratha, and A. Senior. Guide to Biometrics. Springer, 2004. 3 

610 [6] H. Drolon, F. Devaux, A. Descampe, Y. Verschueren, D. Janssens, and B. Macq. OpenJPEG. http://www.

611 openjpeg.org/. 5 

612 [7] David Taubman. Kakadu software. www.kakadusoftware.com. 5 

613 [8] E. Tabassi, P. Grother, and W. Salamon. IREX - IQCE performance of iris image quality assessment algorithms. 

614 Technical report, NIST, 2011. 6 

IREX IV: Concept, Evaluation Plan, and API Specifcation 
6 References 26 

615 [9] Patrick Grother and Elham Tabassi. Performance of biometric quality measures. IEEE Trans. Pattern Anal. 

616 Mach. Intell, pages 531–543, 2007. 7 

IREX IV: Concept, Evaluation Plan, and API Specifcation