1

Type 10 Modifications – draft version 8

Date: 2005-11-29 

 

A NIST Type 10 Transaction Update Proposal 

The Type 10 Ad-Hoc Working Group 

Participant Institution 

Richard Carter AAMVA 

David Benini Aware 

Scott Hills Aware 

Rob Mungovan Aware 

Joan Vitt AFIX Technologies 

Rebecca Mills California DoJ 

Michael Cherry Cherry Biometrics 

Anne Wang Cogent Systems 

Alan Viars DoD 

Tom Hopper FBI 

Scott Swann FBI 

Richard Vorder Bruegge FBI 

Paul Griffin (editor) Identix 

Kirsten Nobel Identix 

Vincent Hsu Identix 

Gene Rosendahl IntelliDyne 

Tomas Smith LASD 

Mike McCabe (editor) NIST 

John Libert NIST 

Patrick Grother NIST 

Charlie Wilson NIST 

John Mayer-Splain Mitretek 

Don D'Amato Mitretek 

Martin Werner Viisage 

 

1

1 Introduction....................................................................................................................................................3

2 The Record Layout........................................................................................................................................4

3 Modified field 10.011: Compression algorithm (CGA)...............................................................................5

4 Modified field 10.012: Color space (CSP)....................................................................................................6

4.1 Introduction and backwards compatibility .................................................................................................6

4.2 The color space sRGB ..................................................................................................................................6

4.3 Field code entries ..........................................................................................................................................6

5 New field 10.013: Subject Acquisition Profile (SAP)..................................................................................8

5.1 Introduction....................................................................................................................................................8

5.2 Level 0 (Unknown profile).............................................................................................................................8

5.3 Level 1 (Latent facial image).........................................................................................................................8

5.4 Levels 10-15 (Other application profiles)....................................................................................................8

5.5 Level 20 (Legacy facial mugshot)................................................................................................................9

5.6 Level 30 (Best Practice Recommendation (BPR) mugshot) .....................................................................9

5.7 Level 40 (Enhanced Best Practice Recommendation (EBPR) facial image) ...........................................9

5.7.1 Face image capture requirements ...............................................................................................................9

5.7.2 Scene requirements.......................................................................................................................................9

5.7.3 Digital Requirements.....................................................................................................................................9

5.8 Requirements of Level 50 and Level 51 (Forensic facial images)..........................................................10

5.8.1 Face image capture requirements .............................................................................................................10

5.8.2 Scene requirements.....................................................................................................................................10

5.8.3 Digital requirements ....................................................................................................................................11

5.8.4 Format Requirements..................................................................................................................................11

6 Face image capture requirements for SAP levels 40, 50, and 51 ...........................................................12

6.1 Introduction..................................................................................................................................................12

6.2 Scene Requirements ...................................................................................................................................12

6.2.1 The “Head and Shoulders” photo composition........................................................................................12

6.2.2 The “Head Only” photo composition.........................................................................................................13

6.2.3 Head centering .............................................................................................................................................13

6.2.4 Visibility of Ears ...........................................................................................................................................14

6.2.5 Facial expression.........................................................................................................................................14

6.2.6 Eyeglasses....................................................................................................................................................14

6.2.7 Eye patches ..................................................................................................................................................14

6.2.8 Background ..................................................................................................................................................14

6.3 Photographic Requirements ......................................................................................................................14

6.3.1 Depth of field ................................................................................................................................................14

6.3.2 Subject lighting ............................................................................................................................................14

6.3.3 Background lighting....................................................................................................................................15

6.3.4 Exposure calibration ...................................................................................................................................15

6.3.5 No saturation................................................................................................................................................15

6.3.6 No unnatural color or “red-eye”.................................................................................................................15

6.3.7 No color or grayscale enhancement..........................................................................................................15

6.3.8 No unnatural radial distortion of the camera lens....................................................................................15

6.4 Digital Requirements...................................................................................................................................15

6.4.1 Pixel aspect ratio .........................................................................................................................................15

6.4.2 Image aspect ratio .......................................................................................................................................15

6.4.3 No interlacing ...............................................................................................................................................16

6.4.4 No digital zoom ............................................................................................................................................16

6.4.5 Image Compression.....................................................................................................................................16

6.4.6 Allowed colorspace .....................................................................................................................................16

6.5 Format Requirements .................................................................................................................................16

6.5.1 Subject Pose (POS) and Subject pose angles (SPA)...............................................................................16

6.5.2 Subject facial description (SXS).................................................................................................................16

6.5.3 Subject hair color (SHC)..............................................................................................................................16
2 

6.5.4 Subject eye color (SEC)...............................................................................................................................17

7 Modified field 10.020: Subject pose (POS)................................................................................................17

8 Modified field 10.022: Photo description (PXS)........................................................................................17

9 New field 10.023: Photo acquisition source (PAS)...................................................................................18

10 New field 10.024: Subject quality score (SQS) .........................................................................................18

11 New field 10.025: Subject pose angles (SPA)...........................................................................................19

11.1 The definition and range of pose angles...................................................................................................19

11.2 The order of rotation through pose angles...............................................................................................20

12 New field 10.026: Subject facial description (SXS) ..................................................................................22

13 New field 10.027: Subject eye color (SEC)................................................................................................23

14 New field 10.028: Subject hair color (SHC) ...............................................................................................24

15 New field 10.029: Subject feature points (SFP) ........................................................................................24

15.1 Subject feature point type...........................................................................................................................25

15.2 Subject feature point code..........................................................................................................................25

15.2.1 MPEG4 Feature Points.................................................................................................................................25

15.2.2 Eye and nostril centre Feature Points........................................................................................................26

16 Example Transaction...................................................................................................................................28

References.................................................................................................................................................................28
 

3 

1 Introduction 

This document contains proposed revisions and additions to the Type-10 facial & SMT image record 

described in ANSI/NIST-ITL-2000 [1]. The changes are designed to be enhancements and clarifications of 

the existing record. 

• Existing records can be converted to this new format with no loss of data. There are no dropped fields 

or lost information 

• The revisions maintain the existing record fields. Changes are made by adding new fields as opposed 

to deleting or significantly modifying existing fields, in order to minimize the changes required to 

existing software. 

• The new record format contains compliance levels, which serve as application profiles. Users can 

select which level of compliance they need for their applications. 

• The new format does not change the encoding scheme of the record, as defined by Section 17.1 of [1]. 

Fields are still encoded using ASCII text, and the record separators remain the same. 

• The new format allows for the interchange of data with existing face image standards, such as ISO 

19794-5. This data includes text data such as eye color. 

The table below denotes the major additions to the record type, and the Section in which this addition is 

discussed in detail. 

Item Section 

Addition of JPEG2000 as an allowed image format 3, 4 

An increase of the maximum length of the source agency (SRC) text field 2 

Inclusion of a set of facial mugshot capture requirements. These requirements 

are called out in new face application profiles 

5, 6 

Face capture application profiles, from non-mugshot photo capture to 

advanced mugshot application profiles. Includes in certain cases the 

requirement to adhere to mugshot best practices for capture and compression 

limits. 

5, 6 

Allow for the encoding of face image quality in the record 10 

The encoding of the photo acquisition type, e.g. digital photo or video. 9 

Allow for the encoding of 3D pose angles for each face photo 11 

Addition of the encoding of Face Image Attributes such as blinking and major 

expression changes. 

12, 13, 14 

Addition of feature points to mark points of interest on the face. 15 

The sections below, in this document, are intended to either replace existing sections or contain new text for 

new sections. That is, the proposed text below is intended to be placed directly into a new standard. Note 

however, that for brevity, this document does not contain the text in the existing type-10 record that is not to 

be changed when a new standard is issued. If existing fields included in the existing type-10 document are 

not discussed below, then the intent is to include these fields and their descriptions verbatim in the new 

standard. 
4 

2 The Record Layout 

Table 1 below is the proposed updated version of Table 9 in ANSI/NIST-ITL 1-2000 [1]. Modifications are 

highlighted with shaded background. Note that this table includes all proposed changes. 

The existing fields 10.004 and 10.999 have their maximum field sizes increased. Existing field 10.022 has 

revised text in its description. Existing fields 10.011, 10.012, and 10.020 have new field code additions. The 

entirely new fields, 10.013, and 10.023 – 10.029 are allocated from previously reserved field codes. 

Table 1 – Type-10 facial and SMT record layout. 

Field size per 

occurrence 

Ident Cond Occur count 

code 

Field 

Number 

Field Name IMG Char 

type 

min max min max 

Max byte 

count 

LEN M 10.001 LOGICAL RECORD LENGTH N 4 8 1 1 15 

IDC M 10.002 IMAGE DESIGNATION 

CHARACTER 

N 2 5 1 1 12 

IMT M 10.003 IMAGE TYPE A 5 7 1 1 14 

SRC M 10.004 SOURCE AGENCY /ORI AN 10 36 1 1 43 

PHD M 10.005 PHOTO DATE N 9 9 1 1 16 

HLL M 10.006 HORIZONTAL LINE LENGTH N 4 5 1 1 12 

VLL M 10.007 VERTICAL LINE LENGTH N 4 5 1 1 12 

SLC M 10.008 SCALE UNITS N 2 2 1 1 9 

HPS M 10.009 HORIZONTAL PIXEL SCALE N 2 5 1 1 12 

VPS M 10.010 VERTICAL PIXEL SCALE N 2 5 1 1 12 

CGA M 10.011 COMPRESSION ALGORITHM A 4 6 1 1 13 

CSP M 10.012 COLOR SPACE A 4 5 1 1 12 

SAP C11 10.013 SUBJECT ACQUISITION PROFILE FAC N 2 4 1 1 11 

RSV - 10.014 

10.019 

RESERVED FOR FUTURE 

INCLUSION 

-- -- -- -- -- -- 

POS O 10.020 SUBJECT POSE FAC A 2 2 0 1 9 

POA O 10.021 POSE OFFSET ANGLE FAC N 2 5 0 1 12 

PXS O 10.022 PHOTO DESCRIPTION FAC A 4 21 0 9 196 

PAS C22 10.023 PHOTO ACQUISITION SOURCE A 7 15 0 1 22 

SQS O 10.024 SUBJECT QUALITY SCORE FAC N 10 35 0 9 322 

SPA C2 10.025 SUBJECT POSE ANGLES FAC N 9 23 0 1 30 

SXS C 10.026 SUBJECT FACIAL DESCRIPTION FAC A 6 21 0 50 1057 

SEC C2 10.027 SUBJECT EYE COLOR FAC A 5 14 0 1 21 

SHC C2 10.028 SUBJECT HAIR COLOR FAC A 5 12 0 2 31 

SFP O 10.029 SUBJECT FEATURE POINTS FAC N 10 18 0 88 1584 

RSV - 10.030 

10.039 

RESERVED FOR FUTURE 

INCLUSION 

-- -- -- -- -- -- 

SMT M 10.040 NCIC DESIGNATION CODE SMT A 4 11 1 3 40 

SMS O 10.041 SCAR/MARK TATTO SIZE SMT N 4 6 0 1 13 

SMD O 10.042 SMT DESCRIPTORS SMT AN 16 51 0 9 466 

COL O 10.043 COLORS PRESENT SMT A 4 21 0 9 196 

RSV - 10.044 

10.199 

RESERVED FOR FUTURE 

INCLUSION 

-- -- -- -- -- -- 

UDF O 10.200 

10.998 

USER-DEFINED FIELDS -- -- -- -- -- -- 

DAT M 10.999 IMAGE DATA B 2 6*HLL*VL

L +13

1 1 6*HLL*VLL+8 

 

1 The Subject Acquisition Profile (SAP) is a mandatory ASCII text field when field 10.003 has code “SAC” (and therefore 

the transaction contains a facial image). 

2 The Condition Code “C2” refers to fields that are mandatory if the SAP (field 10.013) has a numeric value 40 or above. 

Otherwise, these fields are optional. 
 

5 

 

Key for character type: N = Numeric; A = Alphabetic; AN = Alphanumeric; B = Binary 

3 Modified field 10.011: Compression algorithm (CGA) 

This mandatory ASCII field shall specify the algorithm used to compress the color or grayscale image. An 

entry of "NONE" in this field indicates that the data contained in this record is uncompressed. For those 

images that are to be compressed, the method for the compression of facial and SMT images is specified by 

the baseline mode of the JPEG algorithm or JPEG2000. 

An entry of "JPEGB" indicates that the scanned or captured image was compressed using baseline JPEG. An 

entry of "JPEGL" indicates that the lossless mode of the JPEG algorithm was used to compress the image. If 

the image is captured in grayscale, then only the luminance component will be compressed and transmitted. 

For JPEG, the data shall be formatted in accordance with the JPEG File Interchange Format, Version 1.02 

(JFIF)4. 

An entry of "JP2" indicates that the scanned or captured image was compressed using lossy JPEG2000 

(Compliance with ISO 15444-1 is provided through part 4 of the standard, ISO 15444-4 “Conformance 

Testing”.) An entry of "JP2L" indicates that the lossless mode of the JPEG 2000 algorithm was used to 

compress the image. For JPEG 2000, the data shall be formatted in compliance with JP2 format as described 

in ISO 15444-1 [11]. 

Where JPEG2000 is used, the following specifications/options of the compression algorithm shall be 

supported: 

• Filters: The 9-7 irreversible filters [11] should be used for lossy mode; however for handheld devices 

(fixed point processors), the 5-3 reversible filters may be used instead. The 5-3 reversible filters shall 

be used for lossless mode. A compliant decoder shall be able to decode code streams created 

through both filters. 

• Number of resolution levels: The image shall be encoded using enough resolution levels to ensure 

that a thumbnail with max(width, height) <= 64 is available in the image. Example: a 640x480 image 

shall be encoded with 5 resolution levels, which enables sub-resolution decodes of 320x240, 

160x120, 80x60, and 40x30. 

• Resolution as the dominant progression: JPEG2000 allows five progression orders - LRCP, RLCP, 

RPCL, PCRL and CPRL. The RLCP progression order (resolution, layer, component, position) shall 

be used since it best facilitates decode and display of lower resolution derivative images by remote 

networked devices. Through the RLPC progression order, the code stream shall be formatted so that 

the resolution information of the image is the first data made available to a decoder in a streaming 

mode of operation. 

• Bits per Channel: The number of bits per channel for encoders and decoders shall be 8-16 bits. 

• Single tile images: Facial images shall be encoded using only single tile to avoid tiling artifacts. 

• JPEG200 quality layers: The image shall be encoded using at least 10 quality layers to enable quality 

progressive decoding or sub-quality image extraction. 

4 Developed by C-Cube Microsystems, 1778 McCarthy Blvd., Milpitas, CA 95035.
Region of Interest Encoding is allowed: This encoding method is a useful way to compress a facial image to a 

small size, while retaining sufficient image quality within the specified ROI to perform either human or 

automated identification. 

An example transaction for JPEG2000 is “10.011:JP2 GS”. 

4 Modified field 10.012: Color space (CSP) 

4.1 Introduction and backwards compatibility 

This mandatory ASCII field shall contain the color space used to exchange the image data. 

 

In previous versions of this standard [1], the term “color space” referred to device-dependent color information 

with a particular sequence and range for the three color channels. The choice was either RGB or an RGBderivative space known as YCC. Neither space provides an objective definition of a particular color or relates 

to the way in which humans perceive color. 

Moreover, in the previous version of this standard [1], it was stated that “the preferred color space for 

compressed images using baseline JPEG and JFIF is YCbCr to be coded as ‛YCC’,” while the color space for 

uncompressed color images was to be labeled RGB. Therefore, for backwards compatibility purposes, new 

systems must accommodate JPEG images that have been labeled in Field 10.012 as using the YCC color 

space. Specifically, systems compliant with this standard must accept an entry of YCC and interpret it as 

meaning a (device-dependent) RGB color space. 

4.2 The color space sRGB 

To ensure that color images exchanged between differing systems can be correctly displayed or printed, 

images should be converted to the device-independent color space, sRGB [12], before compression or 

transmission to another system. sRGB, as defined by IEC 61966-2-1 [12], is a non-linear display profile that 

accommodates the voltage-to-color response characteristics of most high quality CRT monitors. The colors of 

the red, green, and blue phosphors (primaries) and the white point setting of an sRGB-compliant monitor are 

specified in the IEC document. 

The relationship between sRGB and a linear RGB space having the IEC-defined primaries and white point is 

as follows: 

⎧12.92value ≤ 0

sRGB = ⎨ lin , for valuelin .0031308

value

⎩1.055value(1/ 2.4 )

lin − 0.055, for valuelin > 0.0031308

where valuelin is an R, G, or B value in linear RGB space 

(with a range of 0 to 1) and valuesRGB is the corresponding 

R, G, or B value in non - linear sRGB space (also with a

range of 0 to 1). To convert from/to the range of 0 to 255, 

divide/multiply by 255.

 

Typically, modern digital cameras, desktop scanners, LCD monitors, and printers, although they don't 

inherently operate in sRGB space, are designed with circuitry or software to produce sRGB output or to 

accommodate sRGB as an input space. If an image acquisition device's color space is unknown, sRGB is 

usually a reasonable choice. If an acquisition device and its software cannot provide sRGB output, various 

color management products are available commercially that use its color profile, often available from its 

manufacturer, to convert images in its native color space to sRGB. 

4.3 Field code entries 

If the color space for an RGB image cannot be determined, an entry of “RGB” shall be used for field 10.012. 

6 
 

7 

JPEG: For JPEG-compressed color image files (stored using the JFIF file format), the preferred (external) 

color space is sRGB and an entry of “SRGB” shall be used for Field 10.012. For all grayscale (monochrome) 

images, an entry of "GRAY" shall be used for Field 10.012. 

JPEG2000: For JPEG2000 images stored using the JP2 file format, the available enumerated color spaces 

are sRGB, sYCC, and grayscale, to be entered, respectively, as “SRGB”, “SYCC”, and “GRAY” in Field 

10.012. The preferred (external) color space for color images is sRGB. If a photo acquisition device uses 

another ICC color profile, the acquisition system must convert the image data to one of these enumerated 

color spaces before the JP2 file may be embedded in a Type 10 record. 

Uncompressed color images: For uncompressed color images containing non- interleaved red, green, and 

blue pixels in that order, the preferred color space is sRGB and an entry of “SRGB” shall be used for Field 

10.012. 

Note that the field codes do not determine if the image data is JPEG, JPEG2000, or uncompressed color 

images. The header in the image data itself will need to be examined to make that determination. 

An example transaction for an image in the sRGB color space is “10.012:SRGB GS”. 
8 

5 New field 10.013: Subject Acquisition Profile (SAP) 

5.1 Introduction 

The Subject Acquisition Profile (SAP) is a mandatory ASCII text field when field 10.003 has code “FACE” (and 

therefore the transaction contains a facial image). 

When included, this field shall contain an ASCII character code selected from Table 2 to indicate the numeric 

value of the acquisition profile. Typically, the higher the value, the stronger the acquisition requirements 

become. Therefore, in the text below, the SAP value will also be denoted as a “level”. 

Table 2: Subject Acquisition Profiles 

Subject acquisition profile attribute Attribute code 

Unknown profile 0 

Latent facial image 1 

Driver’s license image (AAMVA) 10 

ANSI Full Frotal facial image (ANSI 385) 11 

ANSI Token facial image (ANSI 385) 12 

ISO Full Frontal facial image (ISO/IEC 19794-5) 13 

ISO Token facial image (ISO/IEC 19794-5) 14 

PIV facial image (NIST SP 800-76) 15 

Legacy Mugshot 20 

BPR Mugshot 30 

EBPR Facial Image (Head and Shoulders) 40 

Forensic Facial Images (Head and Shoulders) 50 

Forensic Facial Images (Head only) 51 

An example transaction for SAP of value 40 is “10.013:40 GS”. 

5.2 Level 0 (Unknown profile) 

This level denotes any case when the Subject Acquisition Profile is unknown. This value can be used to alert 

systems that the profile of the face image needs to be determined manually or via advanced face image 

quality evaluation techniques. 

5.3 Level 1 (Latent facial image) 

This SAP denotes a latent facial image: a face image captured without specific regard to scene, photographic, 

or digital requirements. For example, an image of a face from commonly available surveillance video 

equipment is generally considered a latent facial image. Typically latent facial images are of relatively poor 

quality compared to mugshots, including significant pose angle from frontal, poor image resolution, poor image 

contrast, etc. 

5.4 Levels 10-15 (Other application profiles) 

Levels 10-15 shall denote transaction associated with capture under the guidance of other facial standards or 

application profiles as defined below. 

• Level 10 denotes a driver license facial portrait described in AAMVA DL/ID-2000 [5]. 

• Level 11 denotes an ANSI facial image which meets requirements of the Full Frontal Image type 

defined in ANSI/INCITS 385-2004 [10]. 

• Level 12 denotes an ANSI facial image which meets requirements of the Token Face Image type 

defined in ANSI/INCITS 385-2004 [10]. 
 

9 

• Level 13 denotes an ISO facial image that meets the requirements of the Full Frontal Image defined in 

International standard ISO/IEC 19794-5 [4]. 

• Level 14 denotes an ISO facial image that meets the requirements of the Token Face Image type 

defined in International standard ISO/IEC 19794-5 [4]. 

• Level 15 denotes a PIV facial image which meets requirements of Biometric Data Specification for 

Personal Identity Verification [3]. 

Note that the facial images of Levels 13 and 14 may come from travel documents as described in 

“Deployment of Machine Readable Travel Documents’’, ICAO Technical Report, version 2.0 [4a, 6]. 

5.5 Level 20 (Legacy facial mugshot) 

A transaction conforming to this application profile level shall be a mugshot captured according to ANSI/NISTITL 2000 [1], but not necessarily or known to be conforming to the best practice requirements given in profile 

30 below. The subject pose(s) can be Frontal, Profile, or Angled. 

5.6 Level 30 (Best Practice Recommendation (BPR) mugshot) 

A transaction conforming to a level 30 application profile shall include at least one mugshot record conforming 

to all best practice requirements (BPR) of reference [2]. These mugshots adhere to strict background, lighting, 

and resolution requirements as specified in [2]. In particular, the background is 18% gray, the lighting is threepoint, and the resolution is at least 480x600 with and aspect ratio of 1:1.25. 

5.7 Level 40 (Enhanced Best Practice Recommendation (EBPR) facial image) 

5.7.1 Face image capture requirements 

A transaction conforming to a level 40 application profile shall include frontal face images conforming to the 

“face image capture requirements” of Section 6. At least one frontal face image shall be included in the 

transaction. 

5.7.2 Scene requirements 

Level 40 facial images shall conform to the “head and shoulders” composition requirements of Section 6.2.1. 

5.7.3 Digital Requirements 

5.7.3.1 Minimum number of pixels 

The minimum number of pixels in the electronic digital image shall be 768 pixels in the horizontal direction by 

1024 pixels in the vertical direction. 

The composition shall conform to a “head and shoulders photo” (see Section 6.2.1), which can be captured 

with an off-the-shelf 1 mega pixel camera. 

It should be noted that the image quality of the captured facial images will be improved as the number of 

pixels in both directions are increased. However, as images are captured with an increased number of pixels, 

the 3:4 (Width:Height) aspect ratio shall be maintained. 
10 

5.8 Requirements of Level 50 and Level 51 (Forensic facial images) 

5.8.1 Face image capture requirements 

A transaction conforming to a level 50 and 51 application profiles shall include face images conforming to the 

“face image capture requirements” of Section 6. 

These profile levels are intended to allow for examination of up to forensic-level (0.1 millimeter) detail on a 

subject’s face. Approximately 1700 pixels wide by 2515 pixels high on the face are needed for the 99th 

percentile male in the U.S. population [13]. 

With the width of the head taking 50% of the image width, this corresponds to an image width of 3400 pixels 

and this composition corresponds to a “head and shoulders photo” which can be capture with of-the-shelf 15 

(or more) mega pixel digital cameras. Alternatively, with the width of the head taking 70% of the image width, 

this corresponds to an image width of roughly 2400 pixels and this composition corresponds to a “head photo” 

which can captured with of-the-shelf 8 mega pixel digital cameras. 

5.8.2 Scene requirements 

Level 50 shall conform to the “head and shoulders” composition requirements of Section 6.2.1. 

Level 51 shall conform to the “head only ” composition requirements of Section 6.2.2. 
 

11 

5.8.3 Digital requirements 

5.8.3.1 Minimum number of pixels 

• Level 50 (head and shoulders): The minimum number of pixels in the electronic digital image shall be 

3300 pixels in the horizontal direction by 4400 pixels in the vertical direction. 

• Level 51 (head only): The minimum number of pixels in the electronic digital image shall be 2400 pixels in 

the horizontal direction by 3200 pixels in the vertical direction. 

It should be noted that the image quality of the captured facial images will be improved as the number of 

pixels in both directions are increased. However, as images are captured with an increased number of pixels, 

the 3:4 (Width:Height) aspect ratio shall be maintained. 

a. Level 30 b. Level 40 c. Levels 50 and 51 

Figure 2 – Examples of resolution for levels 30, 40, 50, and 51.

5.8.3.2 Compression algorithm requirements 

Non-frontal facial images shall be compressed using JPEG 2000 (JPEG is not allowed) meeting the maximum 

compression limits specified in Section 6.4.5. 

Frontal facial images shall be compressed only using lossless JPEG 2000 compression. 

5.8.4 Format Requirements 

5.8.4.1 Number of photographs 

Level 50 and 51 records shall include at least five photographs of the subject: frontal, left and right profile, and 

left and right 3⁄4 profile. 

A 3⁄4 profile view consists of a face with a Yaw pose angle of ± 67.5 degrees, and with Pitch and Roll angles of 

zero (see Section 11, “New field 10.025: Subject pose angle (SPA)”). Note that the pose of the head is rotated 

to 3⁄4 profile, and care should be taken to prevent the subject from keeping the head fixed while changing only 

the gaze. In 3⁄4 profile photographs, both eyes must be visible in the image. 
6 Face image capture requirements for SAP levels 40, 50, and 51 

6.1 Introduction 

This set of “enhanced best practice recommendation” (EBPR) clauses is a set of constraints similar to those 

given in references [2, 4]. These constraints can be categorized into four types of requirements: scene, 

photographic, digital, and format. Scene requirements refer to the content, subject and background in the 

image. Photographic requirements refer to lighting, focus and other constraints required for photo capture. 

Digital requirements refer to the conversion of the captured image into a digital record. Finally, format 

requirements refer to additional or conditional required fields and in a Type-10 record or NIST transaction. 

Note that the set of requirements in the remainder of Section 6 below applies to all poses of a subject. 

6.2 Scene Requirements 

6.2.1 The “Head and Shoulders” photo composition 

The composition consists of a subject’s head, partial shoulders, and plain background. For a frontal-facing 

pose, the width of the subject's head shall occupy approximately 50% of the width of the captured image. This 

width shall be the horizontal distance between the mid-points of two imaginary vertical lines. Each imaginary 

line shall be drawn between the upper and lower lobes of each ear and shall be positioned where the external 

ear connects to the head. A template and an example is shown in figure 3. For other poses, the composition 

shall be rotated about an imaginary axis extending from the top of the head though the base of the neck. 

This composition is applied to SAP levels 30, 40, and 50. 

a. A template of the “head and 

shoulders” photo. The width of the 

head is 1⁄2 the width of the photo. 

b. An example “head and 

shoulders” photo. 

Figure 3 – A facial image template and an example image that meets “Head and Shoulders” (levels 40 

and 50) scene constraints. 

12 
 

13 

6.2.2 The “Head Only” photo composition 

The composition consists of a subject’s head, and a plain background. For a frontal-facing pose, the width of 

the subject's head shall occupy approximately 70% of the width of the captured image. This width shall be the 

horizontal distance between the mid-points of two imaginary vertical lines. Each imaginary line shall be drawn 

between the upper and lower lobes of each ear and shall be positioned where the external ear connects to the 

head. An example is shown in figure 3b. For other poses, the composition shall be rotated about an 

imaginary axis extending from the top of the head though the base of the neck. 

This composition is applied to SAP level 51. 

a. A template of the “head only” 

photo. The width of the head is 7/10th

the width of the photo. 

b. An example “head only” 

photo. 

Figure 4 – A facial image template and an example image that meets “Head and Only” (level 51) scene 

constraints. 

6.2.3 Head centering 

For the frontal pose, the face shall be positioned to satisfy all of the following conditions: 

• The approximate horizontal mid-points of the mouth and of the bridge of the nose shall lie on an 

imaginary vertical straight line positioned at the horizontal center of the image. 

• An imaginary horizontal line through the center of the subject's eyes shall be located at approximately 

the 55% point of the vertical distance up from the bottom edge of the captured image. 

For non-frontal pose, the subject shall satisfy these conditions when the head is rotated about an axis though 

the head and torso from the current pose back to center (zero angles) pose. 
14 

 

6.2.4 Visibility of Ears 

The ear(s) shall be visible in frontal, profile and angled views for both “Head and Shoulders” and “Head Only” 

scene compositions. The hair shall be pushed back or tied behind the ears when appropriate. 

Move all other subsections down one subsection. 

6.2.5 Facial expression 

The expression should be neutral (non-smiling) with both eyes open normally (i.e. not wide-open), and mouth 

closed. Every effort should be made to have supplied images comply with this specification. A smile with 

closed jaw is not recommended. 

6.2.6 Eyeglasses 

For subjects who normally wear eyeglasses, every effort should be made to capture the mugshots with the 

glasses on. If significant glare in the glasses is evident in the photograph, then a second frontal mugshot 

image should be captured of the subject without glasses. Specification of eyeglasses in the SXS field is 

required. 

6.2.7 Eye patches 

The wearing of eye patches is allowed only for medical reasons. In these cases, the specification of the patch, 

in the SXS field is required. 

6.2.8 Background 

The subject whose image is being captured shall be positioned in front of a background which is 18% gray 

with a plain smooth flat surface. A Kodak or other neutral gray card or densitometer shall be used to verify this 

18% gray reflectance requirement. 

The boundary between the head and the background should be clearly identifiable about the entire subject 

(very large volume hair excepted). There should be no shadows visible on the background behind the face 

image. 

6.3 Photographic Requirements 

6.3.1 Depth of field 

The subject's captured facial image shall always be in focus from the nose to the ears. Although this may 

result in the background behind the subject being out of focus, this is not a problem5. It is recommended that 

auto-focus on the central part of face be used with digital camera photography. 

6.3.2 Subject lighting 

Lighting shall be equally distributed on the face. There shall be no significant direction of the light from the 

point of view of the photographer. 

The region of the face, from the crown to the base of the chin, and from ear-to-ear, shall be clearly visible and 

free of shadows. In particular, there shall be no dark shadows in the eye-sockets due to the brow and the iris 

and pupil of the eyes shall be clearly visible. 

Subject illumination can be accomplished using three point balanced illumination sources. An example is 

discussed in [2]. A single bare “point” light source, such as a camera flash, is not acceptable for imaging. 

Appropriate diffusion techniques shall be employed to eliminate hot spots on the facial image. These hot 

spots usually appear on reflective areas such as cheeks and foreheads. 

5 For optimum quality of the captured mugshot, the f-stop of the lens should be set at two f-stops below the maximum 

aperture opening when possible. 
 

15 

6.3.3 Background lighting 

Proper lighting shall contribute to the uniformity of illumination of the background, and the background shall be 

free of shadows. 

6.3.4 Exposure calibration 

The exposure shall be keyed to the background. Several areas of the recorded 18% gray background shall be 

used to verify the proper exposure. The averages of the 8-bit Red, Green, and Blue (RGB) components within 

each area shall be calculated. Each of the RGB means shall fall between 105 and 125 with a standard 

deviation of plus or minus 10. Furthermore, for every area examined, the maximum difference between the 

means of any two of the RGB components shall not exceed 10. 

6.3.5 No saturation 

For each patch of skin on the person’s face, the gradations in textures shall be clearly visible. In this sense, 

there will be no saturation (over or under exposure) on the face. 

6.3.6 No unnatural color or “red-eye” 

Unnaturally colored lighting (e.g. yellow, red) is not allowed. Care shall be taken to correct the "white balance" 

of image capture devices. The lighting shall produce a face image with natural looking flesh tones when 

viewed in typical examination environments. “Red-eye” is not acceptable. 

6.3.7 No color or grayscale enhancement 

A process that overexposes or under-develops a color or grayscale image for purposes of beauty 

enhancement or artistic pleasure is not allowed. The full spectrum shall be represented on the face image 

where appropriate. Teeth and whites of eyes shall be clearly light or white (when appropriate) and dark hair or 

features (when appropriate) shall be clearly dark. 

6.3.8 No unnatural radial distortion of the camera lens 

Fish eye effect, a type of distortion where central objects of the image erroneously appear closer than those at 

the edge, typically resulting in what appear to be unusually large noses in the image, is not allowed. While 

some distortion is almost always present during portrait photography, that distortion should not be noticeable 

by human examination. 

For a typical photo capture system with a subject 1.5 to 2.5 meters from the camera, the focal length of the 

camera lens should be that of a medium telephoto lens. For 35 mm photography this means that the focal 

length should be between 90 mm and 130 mm. For other negative formats/sensors the recommended focal 

length is 2 to 3 times the diagonal of the negative/sensor 

6.4 Digital Requirements 

6.4.1 Pixel aspect ratio 

Digital cameras and scanners used to capture facial images shall use square pixels with a pixel aspect ratio of 

1:1 

6.4.2 Image aspect ratio 

The Width:Height (i.e., HLL:VLL) aspect ratio of the captured image shall be 3:4. This corresponds to 

commonly used format sizes such as 600 x 800, 768x1024, etc., allowing for a COTS digital camera to be 

used for capture. 
16 

6.4.3 No interlacing 

Interlaced video frames are not allowed and all interlacing must be absent (not simply removed, but absent). 

6.4.4 No digital zoom 

The use of digital zoom (interpolation) to achieve specified resolution associated with Subject Application 

Profiles that point to Section 6 as normative is not allowed. 

6.4.5 Image Compression 

The maximum compression ratio for both JPEG and JPEG2000 of a rectangular region containing any 

exposed skin of the face, from crown to chin and ear to ear, shall be at most 15:1. This requirement is derived 

from studies of face algorithm matching at high and low resolutions. The non-facial portion of the mugshot, as 

well as the SMT Type-10 record, can be compressed up to a ratio of 120:1. 

For JPEG, reference [14] provides source code to implement compression with both ROI and fixed 

compression ratios. For JPEG2000, these capabilities are built into the implementation. 

For both JPEG and JPEG2000, care must be taken to account for automatic compression by camera 

hardware. Multiple compression stages can damage the quality of photographic data. When possible, 

minimum compression (highest resolutions) should be applied at the camera level when external software 

performs the final (15:1 or less) compression stage. 

Note that SAP levels 50 and 51 include additional constraints on compression for the frontal pose facial 

image. 

6.4.6 Allowed colorspace 

A full color image shall be captured. To ensure that color images exchanged between differing systems can 

be correctly displayed or printed, images shall be converted to the device-independent color space, sRGB 

[12]. See Section 4 above (field 10.012) for further discussion of color spaces. 

6.5 Format Requirements 

6.5.1 Subject Pose (POS) and Subject pose angles (SPA) 

One of either the POS or SPA fields shall be used to denote pose angles. 

The POS field code values “F”, “R”, and “L” can be used for images in which the Pitch and Roll angles are 0 

and the Yaw angle is 0, 90, and -90 respectively. (The sign of the Yaw angle in the previous sentence 

corresponds to the field 10.020 convention where a right profile is when the subject turns to the left). 

The SPA field 10.025 can be used for the above poses and shall be used for all other angled poses. The 

POS field 10.020 shall then be of type code “D”, for determined 3D pose, instructing the user to use 10.025 as 

the reference for pose angles. 

In all cases, the uncertainty in the Yaw pose angle determination shall be less than 5 degrees of the frontal 

photograph, and 10 degrees in the non-frontal photographs. The uncertainty in the Pitch and Roll angles shall 

be less than 5 degrees. 

6.5.2 Subject facial description (SXS) 

The Subject facial description field shall be present in the transaction when one or more of the facial attributes 

given by the type codes of 10.026 is present in the image. 

6.5.3 Subject hair color (SHC) 

The Subject hair color field shall be present in the transaction. The code “UNKNOWN” for this field is not 

allowed. 
 

17 

6.5.4 Subject eye color (SEC) 

The Subject eye color field shall be present in the transaction. The code “UNKNOWN” for this field is not 

allowed. 

7 Modified field 10.020: Subject pose (POS) 

This optional field is to be used for the exchange of facial image data. When included, this field shall contain 

one ASCII character code selected from Table 4 to describe the pose of the subject. For the angled pose 

entry "A", Field 10.021 shall contain the offset angle from the full face orientation. For the determined 3D pose 

entry "D", Field 10.025 shall contain a set of determined 3D pose angles (i.e., Yaw, Pitch, and Roll angles) 

away from the full frontal face orientation. Note that the offset angle in Field 10.021 is opposite from the yaw 

angle in Field 10.025 by a minus sign. 

Table 4: Subject pose 

Pose description Pose code 

Full-Face, Frontal F 

Right Profile (90 degree) R 

Left Profile (90 degree) L 

Angled Pose A 

Determined 3D Pose D 

An example transaction for determined 3D pose is “10.020:D GS”. 

8 Modified field 10.022: Photo description (PXS) 

This optional ASCII field, retained for legacy systems, is used for the exchange of facial image data. When 

present, it shall consist of one or more subfields and shall describe special attributes of the captured facial 

image. Attributes associated with the facial image may be selected from Table 5 and entered in this field or 

subfield as one or two information items and a "US" separator character between the items. The attribute 

code listed in Table 5 shall be listed as the first information item. 

Table 5: Photo descriptors 

Facial image attribute Attribute code 

Subject Wearing Glasses GLASSES 

Subject Wearing Hat HAT 

Subject Wearing Scarf SCARF 

Physical Characteristics PHYSICAL 

Other Characteristics OTHER 

Physical characteristics, such as "FRECKLES" may be entered as a subfield consisting of two information 

items. The first is "PHYSICAL" followed by the “US” separator, followed by the characteristic as listed in Part 4 

Section 13 of the Eighth (or current) Edition of the NCIC Code Manual, July 14, 1999. The "OTHER" category 

is used to enter unlisted or miscellaneous attributes of the facial image. This information shall be entered as a 

two-information item subfield. The first is "OTHER" followed by the “US” separator, followed by the 

unformatted text used to describe the attribute. Multiple attributes and subfields may be listed but must be 

separated by the “RS” character. 
18 

Note: The Subject facial description (SXS), field 26, is intended as a replacement for this PXS field. Table 5 

entries are now duplicated in Table 7. 

An example transaction for subject wearing glasses is “10.022:GLASSES GS”. 

9 New field 10.023: Photo acquisition source (PAS) 

This optional field shall specify the classification of the source of the image contained in this record. When 

included, this field shall contain an ASCII character code selected from Table 6 to describe the source of 

captured image data. 

Table 6 – Acquisition source type codes 

Acquisition source type attribute Attribute code 

Unspecified UNSPECIFIED 

Static photograph from an unknown source UNKNOWN PHOTO 

Static photograph from a digital still-image camera DIGITAL CAMERA 

Static photograph from a scanner SCANNER 

Single video frame from an unknown source UNKNOWN VIDEO 

Single video frame from an analogue video camera ANALOGUE VIDEO 

Single video frame from a digital video camera DIGITAL VIDEO 

Unknown UNKNOWN 

Vendor Specific source VENDOR 

The "VENDOR" category is used to enter unlisted or miscellaneous source attributes of the facial image. This 

information shall be entered as a two-information item subfield. The first is "VENDOR" followed by the “US”

separator, followed by the unformatted text used to describe the attribute. 

An example transaction for static photograph from a digital still-image camera is “10.024:DIGITAL CAMERA GS”. 

10 New field 10.024: Subject quality score (SQS) 

This optional ASCII field shall specify quality score data for facial images stored in this record. Each subfield 

shall contain four information items. They identify a quality score and the algorithm used to create the quality 

score, and should be present if known. This information is useful to enable the recipient of the quality score to 

differentiate between quality scores generated by different algorithms and adjust for any differences in 

processing or analysis as necessary. 

1. The first information item shall be a quantitative expression of the predicted matching performance of the 

biometric sample. This item contains the ASCII representation of the integer image quality score between 1 

and 100 assigned to the image data by a quality algorithm. Higher values indicate better quality. An entry of 

“-1” shall indicate a failed attempt to calculate a quality score. The use of additional values to convey other 

information should be harmonized with ISO/IEC 19794 [4] standards. 

2. The second information item shall specify the alphanumeric ID value of the vendor of the quality algorithm 

used to calculate the quality score. NIST will maintain a Vendor Registry that will map the values in this field 

to registered quality algorithm vendors. The Vendor ID shall be composed of ASCII printable characters up to 

16 characters in length. 

3. The third information item shall specify a numeric product code assigned by the vendor of the quality 

algorithm. It indicates which of the vendor’s algorithms was used in the calculation of the quality score. This 

field contains the ASCII representation of the integer product code and should be within the range 1 to 65535. 
 

19 

4. The fourth item shall specify the version number of the quality algorithm used. The version number shall 

include a major number and minor number separated by a period. The major number and minor numbers 

should each be the ASCII representation of integer values in the range 0 to 255. 

An example transaction for a quality score of 100 generated using a quality algorithm with a product code of 

65530, version 255.255, by a vendor with a Vendor ID of “NIST” is “10.023:100 USNIST US65530 US255.255 GS”. 

11 New field 10.025: Subject pose angles (SPA) 

This optional ASCII field shall only be used when Field 10.020 (POS) is not present or contains an "D" to 

indicate a set of determined 3D pose angles of the same subject. If the entry in the POS Field is an "F", "L", 

or "R", the contents of this field are ignored. When present, this information shall be entered as three or six 

information items. 

The first is the Yaw angle followed by the “US” separator, followed by the Pitch angle, followed by the “US” 

separator, followed by the Roll angle. The fourth, fifth and sixth information items denote the uncertainty in the 

Yaw, Pitch, and Roll angles respectively. If the second triple of angles is not present, then the uncertainty in 

the angles is not determined, but the additional three “US” separators shall still be included. 

The first three items specify the pose of the subject estimated or measured at constrained possible 

orientations within a sphere (See Figure 5). Each angle value shall be to the nearest integer degree. 

If both field 10.021 and this field are present, the Yaw angle of this field shall supersede the offset angle 

contained in Field 10.021. Note that the Yaw angle of this field has the opposite sign of the offset angle 

contained in Field 10.021. 

11.1 The definition and range of pose angles 

The Yaw and Roll angles shall be measured from the full face pose position and have a range of values from -

180 degrees to +180 degrees. The Pitch angle shall have a range of values from -90 degrees to +90 degrees. 

The pose angle set is given by Tait-Bryan angles. 

• Yaw angle: rotation about the vertical (y) axis. A positive Yaw angle is used to express the angular 

offset as the subject rotates from a full-face pose to their left (approaching a right profile). A negative 

Yaw angle is used to express the angular offset as the subject rotates from a full-face pose to their 

right (approaching a left profile). 

• Pitch angle: rotation about the horizontal side-to-side (x) horizontal axis. 

• Roll angle: rotation about the horizontal back to front (z) axis. 

The angles are defined relative to the frontal view of the subject, which has angles (0, 0, 0) as shown in Figure 

5. Examples are shown in Figure 6. 
Figure 5 – The definition of pose angle set is with respect to the frontal view of the subject. 

a b c d e f g 

Figure 6 – Examples of pose angles and their encodings. The pose angles (Y, P, R) of Figures (a) – (g) 

are given by (0, 0, 0), (+45, 0, 0), (-45, 0, 0), (0, -45, 0), (0, +45, 0), (0, 0, -45), and (0, 0, +45), respectively. 

The uncertainty in the pose angles is given by the range 0 to 90 inclusive. It shall denote approximately a 

maximum value of possible deviation in the measurement of the pose. This shall correspond to a two 

standard deviation confidence interval. 

The encoding of angles is in ASCII format, with the minus sign “-“ used to denote a negative value and the 

plus “+” sign optionally used to denote a positive value. Pose angle uncertainty angles always are positive. 

For example, the optional ASCII field of the pose angle set (PAS) for the subject in Figure 5(b) is 

“10.025:+45US0US0GS”. 

11.2 The order of rotation through pose angles 

As order of the successive rotation around the different axes does matter, the encoded rotation angle shall 

correspond to an order of execution starting from the frontal view. This order shall be given by Roll (about the 

front axis), then Pitch (about the horizontal axis) and finally Yaw (about the vertical axis). The (first executed) 

Roll transformation will therefore always be in the image (x, y) plane. Examples are shown in Figure 7. 

From the point of view of executing a transformation from the observed view to a frontal view, the 

transformation order will therefore be Yaw, Pitch, and then Roll. Note however that the encoded angle is from 

the frontal view to the observed view.

20 
 

21 

 (Y, P, R)=(0, 0, +45) (Y, P, R)=(0,-30, +45) (Y, P, R)=(-45, -30, +45) 

a b c

(Y, P, R)=(0, 0, -30) (Y, P, R)=(0, +20, -30) (Y, P, R)=(-30, +20, -30) 

d e F 

Figure 7 – Examples of the order of rotation through pose angles with an origin of coordinate system 

at the nose tip. Figures (a)-(c) show three successive rotation steps to achieve the pose angles (Y, P, 

R) of (-45, -30, +45). Figures (d)-(f) show three successive rotation steps to achieve the pose angles (Y, 

P, R) of (-30, +20, -30). 
22 

12 New field 10.026: Subject facial description (SXS) 

This optional ASCII field shall be used for the exchange of facial image data. When present, it shall describe 

the facial expression of the subject and other attributes associated with the subject's captured facial image. 

This field may have one or more subfields each containing a single information item. Attributes associated 

with the facial image may be selected from Table 7 and entered in this field. For "Physical Characteristic", 

enter a characteristic as listed in Part 4 Section 13 of the Eighth (or current) Edition of the NCIC Code Manual, 

July 14, 1999. For the "Other Characteristic" enter unlisted or miscellaneous attributes as unformatted text 

used to describe the attribute. Multiple attributes may be listed but must be separated by the “RS” character. 

Table 7: Subject facial description codes 

Facial description attribute Attribute code 

Expression unspecified UNKNOWN 

Neutral (non-smiling) with both eyes 

open and mouth closed)

NEUTRAL 

Smiling where the inside of the mouth 

and/or teeth is not exposed (closed jaw).

SMILE 

Subject Having Mouth open MOUTH OPEN 

Having Teeth visible TEETH VISIBLE 

Raising eyebrows RAISED BROWS 

Frowning FROWNING 

Looking away from the camera EYES AWAY 

Squinting SQUINTING 

Subject Wearing Left Eye Patch LEFT EYE PATCH 

Subject Wearing Right Eye Patch RIGHT EYE PATCH 

Subject Wearing Clear Glasses CLEAR GLASSES 

Subject Wearing Dark or Visible Colored 

Glasses (medical) 

DARK GLASSES 

head covering/hat HAT 

Wearing Scarf SCARF 

Having Moustache MOUSTACHE 

Having Beard BEARD 

Ear(s) obscured by hair NO EAR 

Blinking (either or both eyes closed) BLINK 

Having Distorting Medical Condition 

impacting Feature Point detection 

DISTORTING CONDITION 

Physical Characteristics <From NCIC Code Manual>

Other Characteristics <Unformatted Text> 

Note: This field is intended to replace the photo description field (PXS) and to enhance the content with 

additional descriptive information. As such, photo descriptors found in Table 5 also appear in Table 7. 

An example transaction for subject having teeth visible is “10.026:TEETH VISIBLEGS”. 
 

23 

13 New field 10.027: Subject eye color (SEC) 

This optional ASCII field shall be used for the exchange of facial image data. When present, it shall describe 

the eye color of the subject as seen in the photograph. If unusual or unnatural such as may be the case when 

colored contact lenses are present and the “real” eye color cannot be ascertained, then the color should be 

labeled as “UNKNOWN”. 

Table 8 – Eye Color codes 

Eye color attribute Attribute code

Unspecified UNSPECIFIED 

Black BLACK 

Blue BLUE 

Brown BROWN 

Gray GRAY 

Green GREEN 

Multi-Colored MULTI-COLORED 

Pink PINK 

Unknown (e.g. can not be 

determined from image, 

monochrome image) 

UNKNOWN 

An example transaction for subject blue eyes is “10.027:BLUEGS”. 
24 

14 New field 10.028: Subject hair color (SHC) 

This optional ASCII field shall be used for the exchange of facial image data. When present, it shall describe 

the hair color of the subject as seen in the photograph. If unusual or unnatural color such as blue or orange is 

present and the “real” color cannot be ascertained, then the color should be labeled as “UNKNOWN” 

If the subject is completely bald, or has a completely shaved head, then the hair color shall be labeled as 

“BALD”. When the subject is predominantly bald, but hair color is discernable, then the appropriate hair color 

attribute code shall follow “BALD” (separated by the “RS” character). 

Table 9 – Hair Color codes 

Hair color attribute Attribute code 

Unspecified UNSPECIFIED 

Bald BALD 

Black BLACK 

Blonde BLONDE 

Brown BROWN 

Gray GRAY 

White WHITE 

Red RED 

Unknown UNKNOWN 

An example transaction for subject is bald and having gray hair is “10.028:BALDRSGRAYGS”. 

15 New field 10.029: Subject feature points (SFP) 

The optional ASCII field shall be used for the exchange of facial image data. When present, it shall describe 

special attributes of manually or automatically detected facial feature points of the captured facial image. This 

information shall be entered as four information items described in Table 10. The first is Feature Point Type, 

followed by the “US” separator character. The second is Feature Point Code, followed by the “US” separator 

character. The third is the X coordinate of a Feature Point, followed by the “US” separator character. The 

fourth and final item is the Y coordinate of a Feature Point in the facial image. Multiple facial points may be 

listed but must be separated by the “RS” separator character. The maximum number of feature points shall be 

88, with the use of 84 MPEG4 feature points and 4 additional center feature points. 

Feature points shall be included in the record format if they have been accurately determined, thereby 

providing the option that that these parameters do not have to be re-determined when the image is processed 

for face recognition tasks. Note that feature points of eye centers are coded with feature point codes 12.1 and 

12.2 as shown in in Figure 9. 

Typically a computer algorithm will either accurately determine the position of the feature point or completely 

fail and provide either clearly erroneous or no landmark information. Therefore, a method for accurate 

determination is the use of computer-automated feature point determination followed by human verification 

and potential override of the computer determined feature points. 

Table 10 – The Subject Feature Point Field 
 

25 

Item Size Value Notes 

Feature Point Type 1 character 1 Denotes a 2D Feature 

Point. 

 

All other values are 

reserved. 

Feature Point Code 5 characters A.B in ASCII text 

A and B are specified in 

Section 15.2. 

The maximum values of A 

and B are 15. 

X coordinate 1-4 characters Horizontal pixel count from 

upper left pixel. 

Count starts at 0. 

Y coordinate 1-4 characters Vertical pixel count from 

upper left pixel. 

Count starts at 0. 

The four information items are described as follows. 

15.1 Subject feature point type 

The Subject Feature Point Type item represents the type of the Feature Point stored in the Feature Point 

block. This field shall be set to “1” to denote the version 1 of feature point type that the position of the Feature 

Point is represented by the coordinate of the image. All other field values are reserved for future definition of 

Feature Point types. 

15.2 Subject feature point code 

15.2.1 MPEG4 Feature Points 

The Subject feature point code item shall specify the Feature Point that is stored in the Feature Point block. 

The codes of the Feature Points in Section 15.2.1, taken from the MPEG4 standard and defined as MPEG4 

Feature Points, or the additional eye and nostril Feature Points in Section 15.2.2 shall be stored in this field. 

Each Feature Point code is represented by a notation A.B using a major (A) and a minor (B) value. The 

encoding of the Feature Point code is given by the numeric ASCII representation of the value of A.B. 

Figure 8 denotes the Feature Point codes associated with Feature Points as given by Annex C of ISO/IEC 

14496-2. 
x y

z

11.5

11.4

11.2

10.2

10.4

10.10

10.8

10.6

2.14

7.1

11.6 4.6

4.4

4.2

5.2

5.4

2.10

2.12 2.1

11.1

Tongue

6.2 6.4 6.3

6.1 Mouth

8.1 8.9 8.10 8.5 8.3

8.7

8.2

8.8

8.4

8.6

2.2

2.3

2.6

2.9 2.8

2.5 2.7 2.4

2.1

2.12 2.11

2.14

2.10

2.13

10.6 10.8

10.4

10.2

10.10

5.4

5.2

5.3

5.1

10.1

10.9

10.3

10.5 10.7

4.1 4.3

4.6 4.5

4.4 4.2

11.2 11.1 11.3

11.4

11.5

x

y

z

Nose

9.6 9.7

9.14 9.13

9.12

9.2

9.4 9.15 9.5

9.3

9.1

Teeth

9.10 9.11

9.8

9.9

Feature points affected by FAPs

Other feature points

Right eye Left eye

3.13

3.7

3.9

3.5

3.1

3.3

3.11

3.14

3.10

3.12 3.6

3.4

3.2

3.8

Figure 8 – The Feature Point codes defined in ISO/IEC 14496-2. 

Each Feature Point code in Figure 8 is given by major value A and minor value B. For example, the code for 

the left corner of the left eye is given by major value 3 and minor value 7. 

15.2.2 Eye and nostril centre Feature Points 

The eye center Feature Points 12.1 (left) and 12.2 (right) are defined to be the horizontal and vertical 

midpoints of the eye corners (3.7, 3.11) and (3.8, 3.12) respectively. The left nostril center Feature Point 12.3 

is defined to be the midpoint of the nose Feature Points (9.1, 9.15) in the horizontal direction and (9.3, 9.15) in 

the vertical direction. Similarly, the right nostril center Feature Point 12.4 is defined to be the midpoint of the 

nose Feature Points (9.2, 9.15) in the horizontal direction and (9.3, 9.15) in the vertical direction. Both the eye 

center and nostril center Feature Points are shown in Figure 9 and values given in Table 11. 

26 
 

27 

Figure 9 – The eye and nostril center Feature Points are defined by midpoints of MPEG4 Feature 

Points. 

Table 11 – Eye and nostril center Feature Point codes 

Center Feature Point Midpoint of Feature Points Feature Point code 

Left Eye 3.7, 3.11 12.1 

Right Eye 3.8, 3.12 12.2 

Left Nostril Horizontal Vertical 

9.1, 9.15 9.3,9.15 

12.3 

Right Nostril Horizontal Vertical 

9.2, 9.15 9.3,9.15 

12.4 

An example transaction for representing two feature points of eye centers is “10.029:1US12.2US120US130 RS1US12.1US240US129GS”. 
28 

16 Example Transaction 

Type-10 Transaction Record (New or Modified fields)

COMPRESSION ALGORITHM (CGA) 10.011:JP2GS

COLOR SPACE (CSP) 10.012:YCCGS

SUBJECT ACQUISITION PROFILE (SAP) 10.013:40GS

SUBJECT POSE (POS) 10.020:DGS

PHOTO DESCRIPTION (PXS) 10.022:GLASSESGS

PHOTO ACQUISITION SOURCE (PAS) 10.023:DIGITAL CAMERAGS

SUBJECT QUALITY SCORE (SQS) 10.024:100USNISTUS65535US255.255GS

SUBJECT POSE ANGLE (SPA) 10.025:-45US0US0USUSUSGS

SUBJECT FACIAL DESCRIPTION (SXS) 10.026:OPEN MOUTHGS

SUBJECT EYE COLOR (SEC) 10.027:BLUEGS

SUBJECT HAIR COLOR (SHC) 10.028:BALDRSGRAYGS

FACIAL FEATURE POINT (FFP) 10.029:1US12.2US120US130RS1US12.1US240US129GS 

(Example for describing two feature points of eye centers) 

References 

[1] R.M. McCabe, "ANSI/NIST-ITL 1-2000 Data Format for the Interchange of Fingerprint, Facial, and Scar 

Mark & Tattoo (SMT) Information," < ftp://sequoyah.nist.gov/pub/nist_internal_reports/sp500-245-a16.pdf > 

[2] R.M. McCabe, "Best Practice Recommendation for the Capture of Mugshots,’’ Version 2.0, Sept. 1997. < 

http://www.itl.nist.gov/iad/894.03/face/bpr_mug3.html > 

[3] NIST DRAFT Special Publication 800-76, ``Biometric Data Specification for Personal Identity 

Verification,’’ January 24, 2005. < http://csrc.nist.gov/piv-program/fips201-support-docs/SP800-76-

Draft.pdf > 

[4] ISO/IEC 19794-5 (published version), Information technology — Biometric data interchange formats — 

Part 5: Face image data, June, 2005. 

[4a] ISO/IEC 19794-5 (Final Committee Draft, or FCD, version) SC37 N0506, March 2004. 

[5] ``AAMVA National Standard for the Driver License/Identification Card,’’ American Association of Motor 

Vehicle Administrators, June 30, 2000. 

<http://www.aamva.org/Documents/stdAAMVADLIDStandrd000630.pdf > 

[6] ``Biometrics Deployment of Machine Readable Travel Documents,’’ Technical Report, version 2.0, ICAO 

TAG MRTD/NTWG, April 2004. 

<http://www.icao.int/mrtd/download/documents/Biometrics%20deployment%20of%20Machine%20Read

able%20Travel%20Documents%202004.pdf > 
 

29 

[7] Biometrics Deployment of Machine Readable Travel Documents, Annex A: ``Guidelines for Taking 

Photographs to Maximize Facial Recognition Results.’’ < 

http://www.icao.int/mrtd/download/documents/Annex%20A%20-%20Photograph%20Guidelines.pdf > 

[8] Biometrics Deployment of Machine Readable Travel Documents, Annex D: ``Facial Image Format for 

Interoperable Data Interchange.’’ < 

http://isotc.iso.org/livelink/livelink/fetch/2000/2122/327993/2262372/2263034/2300191/JTC001-SC37-N506.pdf?nodeid=3924597&vernum=0 > 

[9] ANSI/INCITS 358-2002 Information Technology - ``BioAPI Specification,’’ InterNational Committee for 

Information Technology Standards, Feb. 2002. < 

http://webstore.ansi.org/ansidocstore/product.asp?sku=ANSI+INCITS+358-2002 > 

[10] ANSI/INCITS 385-2004 Information Technology - ``Face Recognition Format for Data Interchange,’’ 

InterNational Committee for Information Technology Standards, May 2004. < 

http://webstore.ansi.org/ansidocstore/product.asp?sku=ANSI+INCITS+385%2D2004 > 

[11] ISO/IEC 15444-1:2000. ``Information technology -- JPEG 2000 image coding system -- Part 1: Core 

coding system,’’ 2000. < 

http://webstore.ansi.org/ansidocstore/product.asp?sku=INCITS%2FISO%2FIEC+15444-1-2000 > 

[12] For information on sRGB, see http://www.w3.org/Graphics/Color/sRGB or many of the white papers 

available at http://www.color.org. 

[13] Approximately 1700 pixels wide by 2515 pixels high on the face are needed for the 99th percentile male 

in the U.S. population, Richard W. Vorder Bruegge, FBI-OTD, Forensic Audio, Video and Image 

Analysis Unit, Building 27958A, Quantico, VA. 

[14] JROI Project, a publicly available source of JPEG Region of Interest Compression for Face Recognition, 

at SourceForge.net http://sourceforge.net/projects/jroi.