1

2001/10/30 

Data Formats of the NSRL Reference Data Set (RDS) 

Distribution 

Change Record 

1. 2001/01/25 – Editorial changes, corrections to record layouts, modification of 

distribution naming and versioning. 

2. 2001/01/29 – Addition of Virus data element to File record, modification of data 

records to use ASCII 10 character instead of ASCII 13 to end a record, addition of 

information on physical distribution to indicate additional records or replacement 

database. 

3. 2001/03/08 – Added section on fifth file, SHA-1.txt. 

4. 2001/07/06 – Modified file structures, file names, distribution information. 

5. 2001/09/05 – Editorial changes and corrections. 

6. 2001/10/11 – Modified manufacturer file structure, added SpecialCode field to 

NSRLFILE.TXT structure, and added clarifications to physical file descriptions. 

7. 2001/10/23 – Modified NSRLFILE structure to remove PathHash. 

8. 2001/10/30 – Modified NSRLFILE structure to remove FileExt. 

Introduction 

This report describes the format of data included in the distribution of the National Software 

Reference Library (NSRL) Reference Data Set (RDS). The NSRL is a repository of commercialoff-the-shelf (COTS) software. The RDS consists of file profiles made up of file identifying 

information and hash values from the individual files provided in a software package’s 

installation media. An extract of the NSRL database is produced periodically and made available 

through the U.S. Department of Commerce’s (DoC) National Institute of Standards and 

Technology (NIST) Special Database program as NIST Special Database #28. (See “Physical 

Distribution Media.”) 

As software is received in the NSRL, it is processed on a production machine that runs programs 

to compute a file signature, or hash value, for each file within the installation media. The file 

name, hash value, and other descriptive information about each file are stored in the NSRL 

database. When sufficient changes to the database have been made, a new distribution package of 

the extracted information is produced and distributed. 

Typical uses for this data set include the forensic examination of computers and stored data 

seized by law enforcement agencies, copyright infringement investigations, and similar types of 

functions. The structure of the data in the RDS is described in the remainder of the report. The 

sections that follow cover the RDS terms and logical record structure. 

RDS Data Elements 

The following represent data elements used in the NSRL RDS distribution package. Char

represents data of type character using UTF-8 encoding of 8-bit bytes. Integer represents data of 

type integer including variations of the integer type (short, long, etc.) 
 2

All of the data is stored in the distribution files in human-readable form. No binary data or 

nonstandard characters are used. Char fields are represented by alphabetic, numeric, and 

punctuation character strings surrounded by double quotes (“). Integers are represented by 

unquoted strings of decimal digits. 

 

DATA 

ELEMENT TYPE MAXIMUM LENGTH (IN CHARACTERS) DESCRIPTION 

CRC32 Char 8

32-bit Cyclic Redundancy Checksum 

(file signature) of a specific file as 

defined in CCITT X.25 link-level 

protocol and FIPS PUB 71 

FileName Char 100 Name of a specific file within a software 

product 

FileSize Integer 15 Size in bytes of a specific file 

MD4 Char 32

128-bit Message Digest 4 (file signature) 

of a specific file as defined in IETF RFC 

1320 

MD5 Char 32

128-bit Message Digest 5 (file signature) 

of a specific file as defined in IETF RFC 

1321 

MfgCode Char 15 Character identifier of a specific vendor 

or manufacturer 

MfgName Char 150

Identifying name of the vendor or 

manufacturer of the software product, 

e.g., “Microsoft” 

OpSystemName Char 150

Identifying name of the operating 

system on which the software product 

executes, e.g., “Windows NT” 

OpSystemCode Char 15 Code identifier of a specific operating 

system version 

OpSystemVersion Char 15

Characters that identify individual 

versions of an operating system on 

which the software product executes, 

e.g., “4.0” 

ProductCode Char 10 Identifier of a specific software product, 

e.g., “103”; maps to the NSRL database 

ProductName Char 150 Identifying name of the software 

product, e.g., “Netscape Communicator” 

ProductVersion Char 15

Characters that identify individual 

versions of a software product, e.g., 

“3.0” 

RDSVersion Char 20 Character string that identifies the date 

and version of the RDS distribution. 

SHA-1 Char 40

160-bit Secure Hash Algorithm message 

digest (file signature) of a specific file as 

defined in FIPS PUB 180-2 

SpecialCode Char 1

A single character field that identifies 

special file signature entries, such as 

malicious code signatures or other types 

of special entries 
 3

Logical Record Structure 

A logical record forms one item or grouping of information from the data elements defined in the 

above table within the NSRL RDS. There are five such logical record types: (1) file record, (2) 

manufacturer record, (3) product record, (4) operating system record, and (5) version record. 

Each is described in the following tables. Examples of each type of record are also provided. 

Figure 1 illustrates how these files relate to each other. 

Table 1. FILE Record Type 

RECORD FORMAT EXAMPLE COMMENTS 

FILE RECORD 

SHA-1 “AC91EF00F33F12DD491CC91E

F00F33F12DD491CA” 

 FileName “WORD.EXE” 

 FileSize 1217654 In bytes 

ProductCode 103 

The Product record will 

contain more information 

about this product code. 

OpSystemCode “NT4WKS” 

The Operating System 

record will contain more 

information about this 

operating system code. 

MD4 “CC12130FF145DE78DCC12130F

F145DE79” 

MD5 “DC2311FFDC0015FCCC12130F

F145DE78” 

 CRC32 “14CCE9061FFDC001” 

SpecialCode “” 

Blank (no value) – normal 

file 

“M” – malicious file 

“S” – special file 

Table 2. MANUFACTURER Record Type

RECORD FORMAT EXAMPLE COMMENTS 

MANUFACTURER RECORD 

MfgCode “Microsoft” 

MfgCode is referenced in 

the Operating System and 

Product records. MfgCode 

is unique within this record 

set. 

 MfgName “Microsoft Corporation” 
 4

Table 3. OPERATING SYSTEM Record Type

RECORD FORMAT EXAMPLE COMMENTS 

OPERATING SYSTEM RECORD 

OpSystemCode “NT4WKS” 

OpSystemCode is 

referenced in the File 

record and is unique within 

the Operating System 

record set. 

 OpSystemName “Windows NT” 

 OpSystemVersion “4.0” 

MfgCode “Microsoft” 

MfgCode references an 

entry in the Manufacturer 

record. 

Table 4. PRODUCT Record Type

RECORD FORMAT EXAMPLE COMMENTS 

PRODUCT RECORD 

ProductCode 103 

ProductCode is referenced 

in the File record and is 

unique within the Product 

record set. 

 ProductName “Microsoft Word” 

 ProductVersion “2000” 

OpSystemCode “Win98” 

OpSystemCode is 

referenced in the Operating 

System record 

MfgCode “Microsoft” 

MfgCode references an 

entry in the Manufacturer 

record. 

Table 5. RDS VERSION Record Type

RECORD FORMAT EXAMPLE COMMENTS 

SHA-1 RECORD 

SHA-1 “AC91EF00F33F12DD491CC91E

F00F33F12DD491CA” 

This value of SHA-1 is 

computed from the SHA-1 

values of the four other 

files. 

RDSVersion “2001/03/08 0.2” Assigned to each quarterly 

release of the RDS. 
 5

Figure 1. NSRL RDS Logical Record Relationships 

OPERATING 

SYSTEM 

RECORD 

MANUFACTURER 

RECORD 

FILE 

RECORD 

PRODUCT 

RECORD 

MfgCode 

OpSystemCode ProductCode 

VERSION 

RECORD 

Physical Record Structure 

The RDS consists of five physical data files that correspond to the five logical record types, one 

file per logical record type. The character format is UTF-8 (8-bit ASCII), one logical record per 

physical line terminated with ASCII character 10, which corresponds to the newline character. 

Individual fields are separated by comma (,) within each line. Character field values are 

surrounded by double quotation marks (“”). The first record of each file contains the field names 

instead of data values. Examples of the contents of each file are presented in figures 2 through 6. 

The first record in each figure represents the first or header record found in each file. The second 

record in each figure represents all subsequent or detail records in each file. 

Figure 2. FILE Example Data 

”SHA-1”,“FileName”,”FileSize”,”ProductCode”,”OpSystemCode”, 

”MD4”,”MD5”,”CRC32”,”SpecialCode” <10> 

“AC91EF00F33F12DD491CC91EF00F33F12DD491CA”, “WORD.EXE”,1217654,103, ”T4WKS”,

“CC12130FF145DE78DCC12130FF145DE79”,“DC2311FFDC0015FCCC12130FF145DE78”, 

“14CCE9061FFDC001”,”” <10> 

Figure 3. MANUFACTURER Example Data 

“MfgCode”,”MfgName” <10> 

“Microsoft”,“Microsoft Corporation” <10>

Figure 4. OPERATING SYSTEM Example Data 

“OpSystemCode”,“OpSystemName”,“OpSystemVersion”,“MfgCode” <10>

“NT4WKS”,“Windows NT”,“4.0”,“Microsoft” <10>

Figure 5. PRODUCT Example Data 

“ProductCode”,“ProductName”,“ProductVersion”,“MfgCode”,”OpSystemCode” <10>
 6

“103”,“Microsoft Office”,“2000”,“Microsoft”,”Win98” <10>

Figure 6. RDS VERSION Example Data 

“SHA-1”,”RDSVersion” <10>

“DD161AEFCC271124533FFFA1445764BDE12515AE”,“2001/03/08 0.2” <10> 

Physical Distribution Media 

The distribution of the RDS will take the form of quarterly compact discs (CD) from NIST’s 

Standard Reference Data (SRD) Office (http://www.nist.gov/srdata) as Special Database #28. 

Each CD will include a full version of the RDS, i.e., each issue is cumulative and can replace 

previous versions. Each file will be variable in size with a full complement of files from one or 

more packages. Further, the files will be named NSRLFILE.TXT, NSRLOS.TXT, 

NSRLMFG.TXT, NSRLPROD.TXT, and VERSION.TXT. 

It will be up to an individual user to determine how these files are used and provided to 

applications running in an investigative environment. They can be used separately as is, 

combined into a single database of information, or in various other combinations depending on 

the requirements of the particular environment in which they are used. 

Additional software for demonstration purposes may be added to the distribution from time to 

time. 

If space permits, other formats or subsets of data may be added to the distribution as requested by 

users. Those sets of data will be documented within the directories associated with those data 

sets. 

A typical distribution of the RDS will contain several hundred thousand NSRLFILE records, 

several hundred NSRLMFG and NSRLPROD records, several dozen NSRLOS records, and only 

one VERSION record. The current RDS takes less than 100 megabytes of storage on the CD. The 

size will grow as new and updated applications are added to the database.