Title: Disposition of comments on IREX II IQCE test plan + API v3 

(Iritech + Cambridge University(CAM)) 

Date: 24 Dec 2009 Organization: NIST 

1 2 (3) 4 5 (6) (7)

 # Clause 

No./ 

Subclause 

No./ 

Annex 

(e.g.3, 3.1, 

Annex A, 

A.1) 

Paragraph/ 

Figure/Table/ 

Note 

(e.g. Table 1) 

Type 

of 

comment1 

Comment (justification for change) Proposed change Proposed NIST Disposition 

Iritech/

1

Page 4, line

1

ed Typographic error “January 4, 2009” should be

“January 4, 2010”.

Accept

CAM/1 2. scope P. 8.38‐39 I find the meaning of "standalone IQAAs"

somewhat elusive. The distinction intended

between IQAAs described in line 38 and line 39 is

also difficult to grasp.

Perhaps if the examples (e.g., "image in, quality

out") were also identified according to their class

(e.g., "image in, quality out (class X in Table 3)") the

intended meaning would be clearer. This would be

most helpful for "image in, proprietary template +

quality out," which could be either class Y or class

Z.

Accept

Added class identification.

Standalone IQAA ‐ "image in,

quality out" is a class X. Quality as

part of template generation ‐

"image in, quality and proprietary

template out" can be class Y or Z.

CAM/2 2. scope P. 8.38‐39 It might also be helpful to clarify why the IQAAs

described in line 39 are not "standalone" ‐‐ for

instance, because they are not currently intended

to be interoperable and are not intended

to be compared with the performance of other

submissions.

Is the distinction between "standalone" and

"proprietary" (or "proprietary only") the one

intended? See next question also.

Standalone meant "quality

computation only".

Quality scores computed as part of

proprietary template generations

are not completely "proprietary" ‐

they have to conform to format

outlined in the Table 4. Also, note

that quality vectors have a

standard part (positions 1‐32) and

Type of comment: ge = minor general Ge = major general te =minor technical Te =major technical ed = minor editorial Ed = major editorial. (Take care of Word changning te to Te etc. 

NOTE 1 Columns 1, 2, 4, 5 and 6 are compulsory. If column 4 is ge, use ge in column two also (and leave column 3 blank) if it applies to the whole document. 

NOTE 2 NBs should not change the width of columns, in order to make merging easier. 

Note 3 Please submit this document in .doc format and NOT .pdf 

page 1 of 16 

1 
 

 

 

 

 

 

 

 

 

 

 

 

     

   

   

             

           

     

           

                       

               

     

               

       

               

             

               

           

                 

                   

           

     

           

       

           

               

             

Title: Disposition of comments on IREX II IQCE test plan + API v3 

(Iritech + Cambridge University(CAM)) 

Date: 24 Dec 2009 Organization: NIST 

1 2 (3) 4 5 (6) (7)

 # Clause 

No./ 

Subclause 

No./ 

Annex 

(e.g.3, 3.1, 

Annex A, 

A.1) 

Paragraph/ 

Figure/Table/ 

Note 

(e.g. Table 1) 

Type 

of 

comment1 

Comment (justification for change) Proposed change Proposed NIST Disposition 

a proprietary or semi‐proprietary

part (positions 33‐64)

CAM/3 8.

Audienc

e and

options

for

participa

tion

P. 11.6‐7 In discussing Class Y submissions, the document

states that "Vendors can submit a stand‐alone

quality measurement algorithm or quality

computation could be part of their proprietary

template generation."

It took some time for me to conclude that this is a

statement that class Y submissions can use either

the convert_image_to_proprietary_template()

function or the

compute_quality_from_image_data() function

(which are detailed later in "14. PC‐based API

Specification") to report quality assessments. If

this is correct, explicit reference to the API

specification and the relevant functions at this

point would be helpful. Otherwise, because of the

use of the terms "stand‐alone" and "proprietary"

that are shared with section 2 (see above), the

statement could be taken to suggest that a class Y

submission might alternatively be submitted as

Accept

References to API added.

Class Y participants can choose to

"mate" quality algorithm and

comparison scores for analysis. It

is " Analyze this quality alg for this

matcher only."

Class Z submissions will be used for

all analysis.

Type of comment: ge = minor general Ge = major general te =minor technical Te =major technical ed = minor editorial Ed = major editorial. (Take care of Word changning te to Te etc. 

NOTE 1 Columns 1, 2, 4, 5 and 6 are compulsory. If column 4 is ge, use ge in column two also (and leave column 3 blank) if it applies to the whole document. 

NOTE 2 NBs should not change the width of columns, in order to make merging easier. 

Note 3 Please submit this document in .doc format and NOT .pdf 

page 2 of 16 

1 
 

 

 

 

 

 

 

 

 

 

 

 

           

                   

               

               

               

                   

               

           

               

         

   

   

       

       

           

  

     

     

           

     

   

       

         

       

           

     

     

     

Title: Disposition of comments on IREX II IQCE test plan + API v3 

(Iritech + Cambridge University(CAM)) 

Date: 24 Dec 2009 Organization: NIST 

1 2 (3) 4 5 (6) (7)

 # Clause 

No./ 

Subclause 

No./ 

Annex 

(e.g.3, 3.1, 

Annex A, 

A.1) 

Paragraph/ 

Figure/Table/ 

Note 

(e.g. Table 1) 

Type 

of 

comment1 

Comment (justification for change) Proposed change Proposed NIST Disposition 

separate class X and class Z submissions.

CAM/4 8 & 14 The points above (CAM/3) raise another set of

questions about which functions are to be included

in the SDKs for submissions in the different

classes. Are all five functions defined in section 14

to be included in the SDKs for each class, even

though, as an example, two of the functions will

never be used for class X submissions?

Should a return code indicating that the function

call is not supported be includedfor

convert_image_to_proprietary_template() and

match_proprietary_templates() as a

safeguard for class X submissions?

1 If submitting quality algorithm

only, you are a class X

participant.

― API to use:

compute_quality_from_i

mage_data()

2 If submitting matching

algorithm only, you are a class

Z participant.

― APIs to

use: convert_image_to_p

roprietary_template()

and

match_proprietary_temp

lates()

3 If submitting quality and

matching, you are either class

Y or class Z participant

3.1 Class Y: If you want your

quality algorithm be

evaluated against your

matching algorithm only,

Type of comment: ge = minor general Ge = major general te =minor technical Te =major technical ed = minor editorial Ed = major editorial. (Take care of Word changning te to Te etc. 

NOTE 1 Columns 1, 2, 4, 5 and 6 are compulsory. If column 4 is ge, use ge in column two also (and leave column 3 blank) if it applies to the whole document. 

NOTE 2 NBs should not change the width of columns, in order to make merging easier. 

Note 3 Please submit this document in .doc format and NOT .pdf 

page 3 of 16 

1 
 

 

 

 

 

 

 

 

 

 

 

 

           

           

             

           

   

     

       

         

       

             

           

           

         

   

       

   

       

Title: Disposition of comments on IREX II IQCE test plan + API v3 

(Iritech + Cambridge University(CAM)) 

Date: 24 Dec 2009 Organization: NIST 

1 2 (3) 4 5 (6) (7)

 # Clause 

No./ 

Subclause 

No./ 

Annex 

(e.g.3, 3.1, 

Annex A, 

A.1) 

Paragraph/ 

Figure/Table/ 

Note 

(e.g. Table 1) 

Type 

of 

comment1 

Comment (justification for change) Proposed change Proposed NIST Disposition 

you are a class Y participant.

Class Y matcher is not used

to evaluate class X or class Z

quality scores.

3.2 Class Z: If you want your

quality algorithm be

evaluated against all

possible (class Z) matchers,

and your matcher be used

for analysis and evaluation

of other class X or Z quality

algorithms you are a class Z

participant.

In either case (3.1‐class Y or 3.2‐

class Z), quality computation can

be part of:

― template generation

(image in, quality and

proprietary image out)

o API to use (for

quality

computation):

Type of comment: ge = minor general Ge = major general te =minor technical Te =major technical ed = minor editorial Ed = major editorial. (Take care of Word changning te to Te etc. 

NOTE 1 Columns 1, 2, 4, 5 and 6 are compulsory. If column 4 is ge, use ge in column two also (and leave column 3 blank) if it applies to the whole document. 

NOTE 2 NBs should not change the width of columns, in order to make merging easier. 

Note 3 Please submit this document in .doc format and NOT .pdf 

page 4 of 16 

1 
 

 

 

 

 

 

 

 

 

 

 

 

   

     

       

           

           

Title: Disposition of comments on IREX II IQCE test plan + API v3 

(Iritech + Cambridge University(CAM)) 

Date: 24 Dec 2009 Organization: NIST 

1 2 (3) 4 5 (6) (7)

 # Clause 

No./ 

Subclause 

No./ 

Annex 

(e.g.3, 3.1, 

Annex A, 

A.1) 

Paragraph/ 

Figure/Table/ 

Note 

(e.g. Table 1) 

Type 

of 

comment1 

Comment (justification for change) Proposed change Proposed NIST Disposition 

convert_image_to

_proprietary_tem

plate(), or

― standalone (image in,

quality out)

o API to use (for

quality

computation) co

mpute_quality_fr

om_image_data()

Therefore classes Y and Z use these

APIs:

if quality computation is part of

template generation:

convert_image_to_proprietary_te

mplate() and

match_proprietary_templates(),

otherwise

compute_quality_from_image_dat

a(), convert_image_to_proprietary

_template() and

match_proprietary_templates().

Type of comment: ge = minor general Ge = major general te =minor technical Te =major technical ed = minor editorial Ed = major editorial. (Take care of Word changning te to Te etc. 

NOTE 1 Columns 1, 2, 4, 5 and 6 are compulsory. If column 4 is ge, use ge in column two also (and leave column 3 blank) if it applies to the whole document. 

NOTE 2 NBs should not change the width of columns, in order to make merging easier. 

Note 3 Please submit this document in .doc format and NOT .pdf 

page 5 of 16 

1 
 

 

 

 

 

 

 

 

 

 

 

 

             

     

               

                 

                 

             

           

   

     

                 

                 

       

         

         

     

         

         

             

   

           

   

         

             

       

       

   

         

Title: Disposition of comments on IREX II IQCE test plan + API v3 

(Iritech + Cambridge University(CAM)) 

Date: 24 Dec 2009 Organization: NIST 

1 2 (3) 4 5 (6) (7)

 # Clause 

No./ 

Subclause 

No./ 

Annex 

(e.g.3, 3.1, 

Annex A, 

A.1) 

Paragraph/ 

Figure/Table/ 

Note 

(e.g. Table 1) 

Type 

of 

comment1 

Comment (justification for change) Proposed change Proposed NIST Disposition 

Hope this clarifies. Above text is

added to section 8.

CAM/5 8 More generally, it would be helpful to understand

the reason IQCE is allowing the options in each

class and the approach IQCE will take to testing

given those options. For example, why allow

quality to be reported either from

convert_image_to_proprietary_template() or

compute_quality_from_image_data() for class Y?

If both report quality, will both be assessed? Will

assessments of speed rely on one rather than the

other?

This is to support operationally

relevant cases where quality is

computed as part of template

generation.

Quality computation time for

standalone quality (class X) and

quality as part of template

generation (class Y or class Z) will

be reported separately.

CAM/6 8 Similarly, why make reporting quality optional for

class Z?

Reporting quality is optional for

class Z to be flexible and allow

wider participation. Some

organization may only have

matching algorithm and some

(specially academic institutions)

may only have quality algorithm

Type of comment: ge = minor general Ge = major general te =minor technical Te =major technical ed = minor editorial Ed = major editorial. (Take care of Word changning te to Te etc. 

NOTE 1 Columns 1, 2, 4, 5 and 6 are compulsory. If column 4 is ge, use ge in column two also (and leave column 3 blank) if it applies to the whole document. 

NOTE 2 NBs should not change the width of columns, in order to make merging easier. 

Note 3 Please submit this document in .doc format and NOT .pdf 

page 6 of 16 

1 
 

 

 

 

 

 

 

 

 

 

 

 

     

         

         

       

     

             

   

       

                 

                   

         

             

               

               

                 

             

                   

               

   

   

           

       

               

     

   

Title: Disposition of comments on IREX II IQCE test plan + API v3 

(Iritech + Cambridge University(CAM)) 

Date: 24 Dec 2009 Organization: NIST 

1 2 (3) 4 5 (6) (7)

 # Clause 

No./ 

Subclause 

No./ 

Annex 

(e.g.3, 3.1, 

Annex A, 

A.1) 

Paragraph/ 

Figure/Table/ 

Note 

(e.g. Table 1) 

Type 

of 

comment1 

Comment (justification for change) Proposed change Proposed NIST Disposition 

and no matching capabilities. By

allowing submitting only quality or

only matcher or both, we are

facilitating and encouraging a

wider participation.

Iritech/

2

8 4th column

of Table 3

in page 11

ed Typographic error “A Class Y matcher” should be “A

class Y matcher”.

“by ClassZ” should be “by class Z”.

Accept

CAM/7 9.2 IQAA

output

P. 12.3‐5 This section states that quality computation shall

be done on uncompressed iris images in one of two

forms: raw (KIND_VGA); or centered and cropped

(KIND_CROPPED). It also states that "To do the

center‐crop operation, the IQAA will need to find

the iris center and crop symmetrically around it."

This clearly suggests that the input image in the

KIND_CROPPED case will not already have been

cropped, but I find this surprising ‐‐ and at odds

with the specifications in Section 14. Should the

input parameter "kind" in

convert_image_to_proprietary_template() (Table

8, p. 16‐17) and

The plan was we/NIST do the

cropping.

Delete

"To do the center‐crop operation,

the IQAA will need to find the iris

center and crop symmetrically

around it."

Type of comment: ge = minor general Ge = major general te =minor technical Te =major technical ed = minor editorial Ed = major editorial. (Take care of Word changning te to Te etc. 

NOTE 1 Columns 1, 2, 4, 5 and 6 are compulsory. If column 4 is ge, use ge in column two also (and leave column 3 blank) if it applies to the whole document. 

NOTE 2 NBs should not change the width of columns, in order to make merging easier. 

Note 3 Please submit this document in .doc format and NOT .pdf 

page 7 of 16 

1 
 

 

 

 

 

 

 

 

 

 

 

 

       

                     

                 

               

       

                 

     

                 

               

           

                     

                 

                 

                 

           

               

                 

             

             

           

         

       

           

       

        

   

         

         

           

       

         

Title: Disposition of comments on IREX II IQCE test plan + API v3 

(Iritech + Cambridge University(CAM)) 

Date: 24 Dec 2009 Organization: NIST 

1 2 (3) 4 5 (6) (7)

 # Clause 

No./ 

Subclause 

No./ 

Annex 

(e.g.3, 3.1, 

Annex A, 

A.1) 

Paragraph/ 

Figure/Table/ 

Note 

(e.g. Table 1) 

Type 

of 

comment1 

Comment (justification for change) Proposed change Proposed NIST Disposition 

compute_quality_from_image_data() (Table 10, p.

19) be taken as an indication of the type of image

being input or as an instruction about what action

to take on the image before subsequent processing

(i.e., center and crop or not)?

Will these functions need to center and crop input

images? Please reconcile these apparently

contradictory messages.

CAM/8 9.2 P. 12.20‐22 The change dated Nov 25, 09 is unclear, possibly

incomplete, and does not appear to have been

implemented consistently in the rest of the

document.

I take this note as a response to Iritech/17 in the

comments on draft 2 of IQCE. Iritech/17 seems to

me to be making at least two separate points.

First, that not all of the quality measures are

monotonic (in their example, pupil‐iris‐ratio), so

that the requirement that 0 represent the worst

quality for the particular measure and 100 the best

is not always possible. So 12.20‐22 and subsequent

discussions should also underline the relaxation of

We relaxed the 0‐100 constrain for

quality scores except the scalar

quality (position 1 in quality

vector).

The intent is to report "raw"

measurement instead of the

normalized (to 0‐100) range.

Add this text 9.2.2:

The 0‐100 constrain on range of

quality scores has been relaxed,

for all quality metrics except the

scalar quality (position 1 in quality

vector). The intent is to report

Type of comment: ge = minor general Ge = major general te =minor technical Te =major technical ed = minor editorial Ed = major editorial. (Take care of Word changning te to Te etc. 

NOTE 1 Columns 1, 2, 4, 5 and 6 are compulsory. If column 4 is ge, use ge in column two also (and leave column 3 blank) if it applies to the whole document. 

NOTE 2 NBs should not change the width of columns, in order to make merging easier. 

Note 3 Please submit this document in .doc format and NOT .pdf 

page 8 of 16 

1 
 

 

 

 

 

 

 

 

 

 

 

 

             

               

           

             

             

               

             

                 

       

                 

               

                 

       

        

       

         

         

             

       

         

       

   

     

       

   

             

             

             

             

           

             

     

             

Title: Disposition of comments on IREX II IQCE test plan + API v3 

(Iritech + Cambridge University(CAM)) 

Date: 24 Dec 2009 Organization: NIST 

1 2 (3) 4 5 (6) (7)

 # Clause 

No./ 

Subclause 

No./ 

Annex 

(e.g.3, 3.1, 

Annex A, 

A.1) 

Paragraph/ 

Figure/Table/ 

Note 

(e.g. Table 1) 

Type 

of 

comment1 

Comment (justification for change) Proposed change Proposed NIST Disposition 

the requirement for the elements of the quality

vector to increase as a function of increasing

quality, *if* that was the intention.

Second, Iritech/17 pointed out that some measures

(such as iris diameter) lose information when

normalized to the range 0‐100. This is conveyed in

these lines, but not extended to all subsequent

relevant sections.

Is the intention to remove the constraint that all

quality component measures be a monotonic

function of quality for all elements of the quality

vector ‐‐ or only pupil‐iris‐ratio ‐‐ or none? If the

intention is to remove the constraint for all, won't

that complicate analysis

tremendously? Clarification, please.

"raw" measurement instead of the

normalized (to 0‐100) range.

However, the constraint that

quality component measures be a

monotonic function of quality (that

is the higher the better) for all

elements of the quality vector, 

except pupil‐iris‐ratio, remains.

For metrics listed in positions

1,3,5,6,7,10,11,13 of Table 4,

quality component measure is

clearly a monotonic function. To

ensure the monotonic behaviour

for other metrics:

Positions 8, 9: It is expected that

circular iris and pupil shape are the

easiest to process, so closer to a

circle should get a higher score.

Position 12: less motion blur is

desired so 254 – motion blur will

be a monotonic function.

Positions 15, 16: frontal is best, so

Type of comment: ge = minor general Ge = major general te =minor technical Te =major technical ed = minor editorial Ed = major editorial. (Take care of Word changning te to Te etc. 

NOTE 1 Columns 1, 2, 4, 5 and 6 are compulsory. If column 4 is ge, use ge in column two also (and leave column 3 blank) if it applies to the whole document. 

NOTE 2 NBs should not change the width of columns, in order to make merging easier. 

Note 3 Please submit this document in .doc format and NOT .pdf 

page 9 of 16 

1 
 

 

 

 

 

 

 

 

 

 

 

 

         

       

           

               

                   

                   

               

                 

                     

                   

                 

                   

   

                     

                 

                 

                 

   

             

   

       

         

           

          

         

         

       

         

Title: Disposition of comments on IREX II IQCE test plan + API v3 

(Iritech + Cambridge University(CAM)) 

Date: 24 Dec 2009 Organization: NIST 

1 2 (3) 4 5 (6) (7)

 # Clause Paragraph/ Type Comment (justification for change) Proposed change Proposed NIST Disposition 

No./ Figure/Table/ of 

Subclause 

No./ 

Note 

(e.g. Table 1) 

comment1 

Annex 

(e.g.3, 3.1, 

Annex A, 

A.1) 

closer to frontal gets higher score.

Positions 2,14, 17 depends on

what will be computed by the

SDKs.

Iritech/ 9.2 Page 12, te Does “un‐signed integer” mean 1 byte unsigned Change un‐signed to non‐negative.

3 line 21 char or 4 bytes unsigned integer? If this is more I should have written one‐byte

than 1 byte, the type of quality_vector in the input unsigned integer to avoid this

argument of Table 10 needs to change. Also, confusion.

doesn’t this conflict with the statement in page 14,

line 1? If this is unsigned char, in other words, in

the range of [0,255], some quality metric in Table 4

have some problems in the range of values they

can have. For example, what if iris size is greater

We meant un‐signed integer as a

non‐negative value. With 1 byte,

the range for non‐negative values,

will 0‐255.

than 255?

If some quality metric is a mere value rather than a

score, do you plan to test the performance of

Changes iris diameter to iris radius

in Table 4 (see CAM/10)

overall score only or all other metrics? If some

quality metric is a mere value, it is questionable to Revised Table 4 caption (see

test its performance. CAM/9)

Since the 0‐100 requirement is lifted, the caption

Type of comment: ge = minor general Ge = major general te =minor technical Te =major technical ed = minor editorial Ed = major editorial. (Take care of Word changning te to Te etc. 

NOTE 1 Columns 1, 2, 4, 5 and 6 are compulsory. If column 4 is ge, use ge in column two also (and leave column 3 blank) if it applies to the whole document. 

NOTE 2 NBs should not change the width of columns, in order to make merging easier. 

Note 3 Please submit this document in .doc format and NOT .pdf 

page 10 of 16 

1 
 

 

 

 

 

 

 

 

 

 

 

 

                 

         

               

             

                 

                   

         

             

                 

               

             

                 

           

                   

                 

                   

         

           

       

              

             

         

         

   

               

Title: Disposition of comments on IREX II IQCE test plan + API v3 

(Iritech + Cambridge University(CAM)) 

Date: 24 Dec 2009 Organization: NIST 

1 2 (3) 4 5 (6) (7)

 # Clause 

No./ 

Subclause 

No./ 

Annex 

(e.g.3, 3.1, 

Annex A, 

A.1) 

Paragraph/ 

Figure/Table/ 

Note 

(e.g. Table 1) 

Type 

of 

comment1 

Comment (justification for change) Proposed change Proposed NIST Disposition 

“The range of each metric shall be [0,100],...” in

Table 4 needs to be changed.

Iritech/

4

9.2 Page 12,

line 21

te What is the range of “iris size” and “pupil‐iris‐

ratio”? Since they are allowed to be un‐signed

integer, can they be in [0, 4294967295]? If they can

be greater than or equal to 255, what about the

convention of writing 255 for uncalculated quality

value?

See Iritech/3

1 byte & non‐negative => range is

0‐255.

CAM/9 Table 4 "The range of each metric shall be [0,100]" should

be "[0,254]."

Yes

CAM/1

0

Table 4 Note also that field 3, "Iris size (diameter in

pixel[s])" should probably be changed to "Iris

radius," because not all iris diameters will be less

than 255 pixels (some were 372 in IREX I).

Yes – thanks

changed accordingly

Iritech/

5

9.2 Position 10

in Table 4

te What exactly do you mean by “Margin”? Is this

mere a pixel distance between iris boundary and its

Margin is a measure of pixel

distance between iris boundary

closest edge of image or a sort of score value and its closest edge of image. The

measuring how good the margin is? intent is to quantify if there is

enough margin (larger than 0.6R

horizontally and 0.2R vertically per

ISO/IEC 19794‐6) for processing

the image. If iris is cut, this metric

Type of comment: ge = minor general Ge = major general te =minor technical Te =major technical ed = minor editorial Ed = major editorial. (Take care of Word changning te to Te etc. 

NOTE 1 Columns 1, 2, 4, 5 and 6 are compulsory. If column 4 is ge, use ge in column two also (and leave column 3 blank) if it applies to the whole document. 

NOTE 2 NBs should not change the width of columns, in order to make merging easier. 

Note 3 Please submit this document in .doc format and NOT .pdf 

page 11 of 16 

1 
 

 

 

 

 

 

 

 

 

 

 

 

     

                                     

       

               

               

   

           

             

             

               

                   

                 

           

             

       

                                                           

         

       

   

       

            

         

         

Title: Disposition of comments on IREX II IQCE test plan + API v3 

(Iritech + Cambridge University(CAM)) 

Date: 24 Dec 2009 Organization: NIST 

1 2 (3) 4 5 (6) (7)

 # Clause 

No./ 

Subclause 

No./ 

Annex 

(e.g.3, 3.1, 

Annex A, 

A.1) 

Paragraph/ 

Figure/Table/ 

Note 

(e.g. Table 1) 

Type 

of 

comment1 

Comment (justification for change) Proposed change Proposed NIST Disposition 

should be 0.

Iritech/

6

9.2 Position 14

in Table 4

te What is “magnification”? For the self‐completeness

of the document, it seems better to include the

definition of that term.

Magnification of the lens

It is the size (radius) of an iris

image relative to the size of the iris

creating the image.

This is an iris acquisition covariate.

It depends on the focal length and

the distance from the lens to the

object.

CAM/1

1

9.2.2

Vector

quality

P. 13.1‐2 "Each computed element of a vector quality shall

be in the range of 0‐100, where 0 means lowest

and 100 means the best quality." This needs to be

updated: 0‐254; and not necessarily an increasing

function of quality (if that was the intention).

changed  0‐100 to 0‐254. Add text

re: monotonic.

See CAM/8

Iritech/

7

14.2.3 Page 16,

line 24

te What do you mean by “SDKs can choose to alter

image”? Doesn’t this conflict with “Quality score

shall be computed ... without any image alteration

or manipulation” in line 23?

Quality metrics shall always be

computed on unaltered images.

However, template generation 

and/or matching algorithm may

choose to alter the image. If

image is altered, quality metrics

shall be re‐calculated for the

Type of comment: ge = minor general Ge = major general te =minor technical Te =major technical ed = minor editorial Ed = major editorial. (Take care of Word changning te to Te etc. 

NOTE 1 Columns 1, 2, 4, 5 and 6 are compulsory. If column 4 is ge, use ge in column two also (and leave column 3 blank) if it applies to the whole document. 

NOTE 2 NBs should not change the width of columns, in order to make merging easier. 

Note 3 Please submit this document in .doc format and NOT .pdf 

page 12 of 16 

1 
 

 

 

 

 

 

 

 

 

 

 

 

           

           

       

       

               

           

           

           

           

           

             

                 

               

                   

             

       

               

         

       

   

       

         

       

     

       

            

     

       

         

         

Title: Disposition of comments on IREX II IQCE test plan + API v3 

(Iritech + Cambridge University(CAM)) 

Date: 24 Dec 2009 Organization: NIST 

1 2 (3) 4 5 (6) (7)

 # Clause 

No./ 

Subclause 

No./ 

Annex 

(e.g.3, 3.1, 

Annex A, 

A.1) 

Paragraph/ 

Figure/Table/ 

Note 

(e.g. Table 1) 

Type 

of 

comment1 

Comment (justification for change) Proposed change Proposed NIST Disposition 

altered image and be reported in

positions 32+n where n<32 is the

position of quality metric

computed on unaltered image.

See CAM/12.

CAM/1 14.2.3 P. 16.23‐25 "Quality scores shall be computed on the input Agree that analysis of quality 2 image without any image alteration or

manipulation. SDKs can choose to alter

images. Their success in compensating for certain

image impairments will be reflected in the

matching accuracy. Participants are encouraged to

disclose if and what image enhancements they

perform."

It is good that image alterations will be allowed,

but reporting only the pre‐alteration quality of an

image may introduce a great deal of noise into any

analysis of the relationship between measures of

image quality and matching performance.

For example, one could correct for deviated gaze

computation after alteration is

useful.

Make these changes:

Add the following text:

Quality metrics shall always be

computed on unaltered images.

However, template generation

and/or matching algorithm may

choose to alter the image. If

image is altered,

1. SDKs should report if

image has been altered or

not. That could be an

Type of comment: ge = minor general Ge = major general te =minor technical Te =major technical ed = minor editorial Ed = major editorial. (Take care of Word changning te to Te etc. 

NOTE 1 Columns 1, 2, 4, 5 and 6 are compulsory. If column 4 is ge, use ge in column two also (and leave column 3 blank) if it applies to the whole document. 

NOTE 2 NBs should not change the width of columns, in order to make merging easier. 

Note 3 Please submit this document in .doc format and NOT .pdf 

page 13 of 16 

1 
 

 

 

 

 

 

 

 

 

 

 

 

               

                 

               

                 

               

       

           

         

           

               

               

                 

             

             

         

         

           

                 

             

                 

             

   

       

     

       

         

       

         

     

     

      

     

        

Title: Disposition of comments on IREX II IQCE test plan + API v3 

(Iritech + Cambridge University(CAM)) 

Date: 24 Dec 2009 Organization: NIST 

1 2 (3) 4 5 (6) (7)

 # Clause 

No./ 

Subclause 

No./ 

Annex 

(e.g.3, 3.1, 

Annex A, 

A.1) 

Paragraph/ 

Figure/Table/ 

Note 

(e.g. Table 1) 

Type 

of 

comment1 

Comment (justification for change) Proposed change Proposed NIST Disposition 

and derive a new image that corresponded to what

would be expected for on‐axis gaze. In this case,

for a badly deviated iris image, under current

guidelines, one would have to assign a "gaze angle"

score that is poor. However, given the successful

correction for deviated gaze, matching

performance with that particular image would be

good, and the relationship between the

"uncorrected" measure of "gaze angle" quality (and

also overall quality) and performance for this SDK

would be weakened. The incentive would then be

to qualify the measures of "gaze angle" and overall

quality, so that only images that cannot be

"corrected" are given poor scores. That would

have the result of improving the correlation

between "matched" measures of quality and

performance, but at the expense of interoperability

‐‐ the measures of quality would no longer predict

the performance of others as well as they would

have in the first situation. The overall utility of

such quality measures for screening and such,

would also suffer.

output parameter e.g.

image_enhanced.

2. If image is enhanced,

quality vector shall be re‐

calculated for the altered

image and be reported in

positions 32+n where n<32

is the position of (the

standard) quality metric

computed on unaltered

image. Any proprietary

quality metrics (position

32‐64) should be in

positions

Type of comment: ge = minor general Ge = major general te =minor technical Te =major technical ed = minor editorial Ed = major editorial. (Take care of Word changning te to Te etc. 

NOTE 1 Columns 1, 2, 4, 5 and 6 are compulsory. If column 4 is ge, use ge in column two also (and leave column 3 blank) if it applies to the whole document. 

NOTE 2 NBs should not change the width of columns, in order to make merging easier. 

Note 3 Please submit this document in .doc format and NOT .pdf 

page 14 of 16 

1 
 

 

 

 

 

 

 

 

 

 

 

 

               

             

           

         

   

                   

                 

               

               

               

                

             

         

       

             

                   

         

     

     

Title: Disposition of comments on IREX II IQCE test plan + API v3 

(Iritech + Cambridge University(CAM)) 

Date: 24 Dec 2009 Organization: NIST 

1 2 (3) 4 5 (6) (7)

 # Clause 

No./ 

Subclause 

No./ 

Annex 

(e.g.3, 3.1, 

Annex A, 

A.1) 

Paragraph/ 

Figure/Table/ 

Note 

(e.g. Table 1) 

Type 

of 

comment1 

Comment (justification for change) Proposed change Proposed NIST Disposition 

If there are many instances where corrections of

this or other sorts are performed by SDKs

submitted to IQCE, the relationship between

quality and performance could be substantially

clouded by noise.

At the least, I would have thought that if an

alteration is performed on an image, this should be

indicated (and indicated for each image). You may

need to exclude those instances from analysis to

get a "bedrock" picture of the relationship between

quality and performance. I don't think you should

rely on merely encouraging participants to disclose

what enhancements they perform. (A statement

from participants that "some images were

subjected to alteration" won't help you sharpen

your analysis.)

I would also like to see the reporting of an

additional post‐alteration quality (score and vector

components) allowed/encouraged for those

images that are altered.

Type of comment: ge = minor general Ge = major general te =minor technical Te =major technical ed = minor editorial Ed = major editorial. (Take care of Word changning te to Te etc. 

NOTE 1 Columns 1, 2, 4, 5 and 6 are compulsory. If column 4 is ge, use ge in column two also (and leave column 3 blank) if it applies to the whole document. 

NOTE 2 NBs should not change the width of columns, in order to make merging easier. 

Note 3 Please submit this document in .doc format and NOT .pdf 

page 15 of 16 

1 
 

 

 

 

 

 

 

 

 

 

 

 

                 

                                                                                       

         

       

Title: Disposition of comments on IREX II IQCE test plan + API v3 

(Iritech + Cambridge University(CAM)) 

Date: 24 Dec 2009 Organization: NIST 

1 2 (3) 4 5 (6) (7)

 # Clause 

No./ 

Subclause 

No./ 

Annex 

(e.g.3, 3.1, 

Annex A, 

A.1) 

Paragraph/ 

Figure/Table/ 

Note 

(e.g. Table 1) 

Type 

of 

comment1 

Comment (justification for change) Proposed change Proposed NIST Disposition 

Iritech/

8

14.2.8 Output

parameters

in Table 11

ed Typographic error “dilation, OR null string” should be

“dilation, or null string”. Accept

Iritech/

9

14.2.4 Table 8 te Is the pointer “image_data” in prototype a pointer

to uncompressed raster data or image record K1,

K3? In IREX I, image record K1 or K3 contains

headers with a raster data in it. It seems better to

make clear the meaning of image_data in the

prototype.

Image_data is a pointer to

uncompressed raster data, no

header.

Type of comment: ge = minor general Ge = major general te =minor technical Te =major technical ed = minor editorial Ed = major editorial. (Take care of Word changning te to Te etc. 

NOTE 1 Columns 1, 2, 4, 5 and 6 are compulsory. If column 4 is ge, use ge in column two also (and leave column 3 blank) if it applies to the whole document. 

NOTE 2 NBs should not change the width of columns, in order to make merging easier. 

Note 3 Please submit this document in .doc format and NOT .pdf 

page 16 of 16 

1