July –August 2014

Issue 130

Information Technology Laboratory Newsletter

INSIDE THIS 

ISSUE

ITL’s National Software 

Reference Library Receives 

Innovation Award

ITL’s National Cybersecurity

Center of Excellence (NCCoE) 

Focuses on Identity and Access 

Solutions

ITL Performs Multilingual 

Evaluation of Key Word Search 

Technology

Staff Recognition

Selected New Publications

Upcoming Technical 

Conferences

ITL’s National Software Reference Library Receives 

Innovation Award 

The National Software Reference Library (NSRL) is the recipient of the 2014 National Digital 

Stewardship Alliance (NDSA) Innovation Award. Selected from more than 30 nominations, the 

National Software Reference Library is being recognized for their substantial leadership in 

building a national collection of software, developing and sharing workflows and approaches for 

software preservation, and for modeling approaches to corpus analysis of born digital 

collections. 

Without efficient methods, investigation of crimes involving computer files requires a 

tremendous amount of effort. The NSRL provides an automated method for eliminating known 

files with verified metadata from an investigation, leaving unknown files and files with 

suspicious metadata to be reviewed. The NSRL consists of four components:

A large collection of software packages. Software is donated by software manufacturers and 

other organizations or purchased. These packages (including physical media and purely 

electronic) include both new and older versions of operating systems, database management 

systems, utilities, graphics images, component libraries, etc. The electronic library is on an 

isolated NIST network.

A database. The NSRL database contains detailed information about the files that make up the 

packages listed above. 

The Reference Data Set (RDS). This data set contains signatures and identifying metadata, 

but not the software files. The metadata includes manufacturer name, operating system 

information, product information, application type, cryptographic information, and file storage 

information. Since its release in 2001, the RDS has grown to include over 100 million file 

signatures.

A research environment. This environment facilitates the collaboration of NIST with 

researchers to access NSRL’s collection of software packages.

Computer forensics examiners, federal, state, and local law enforcement, government 

agencies, and industry organizations that perform investigations on computer technology use 

the NSRL to obtain digital evidence much more quickly and efficiently. Researchers also use 

the resource to quickly and automatically reduce the amount of data involved in experiments, 

where known files need to be identified. 
2

ITL’s National Cybersecurity Center of 

Excellence (NCCoE) Focuses on 

Identity and Access Solutions

A partnership of NIST, the State of Maryland, and 

Montgomery County, the NCCoE facilitates innovation 

through the rapid identification and adoption of practical 

standards-based cybersecurity solutions. The center is 

developing example solutions so that energy companies 

can better control physical and logical access to their 

resources, including buildings, equipment, information 

technology, and industrial control system networks and 

systems. 

NCCoE recently hosted 20 representatives from 13 

cybersecurity technology companies that responded to a 

March Federal Register notice calling for participation. 

Vendors of commercial technologies for identity and access 

management will provide the products that are modules in 

the end-to-end approach to this challenge. The companies 

presented their products’ capabilities and learned about the 

cooperative research and development agreements 

(CRADAs) that will guide their collaboration with NIST. 

Once the CRADAs are signed, collaboration will begin 

under the direction of the NCCoE project manager. 

The solution developed by these collaborators in the 

NCCoE labs will help to: 

 Reduce opportunities for attack or error, as well as the 

impact of such incidents on energy delivery, thereby 

lowering overall business risk;

 Increase the probability that investigations of attacks or 

anomalous system behavior will reach successful 

conclusions;

 Improve accountability and traceability, leading to 

valuable operational lessons learned; and

 Simplify regulatory compliance by automating 

generation and collection of access information.

ITL Performs Multilingual Evaluation of 

Key Word Search Technology

ITL recently performed two additional multilingual 

evaluations of KeyWord Search (KWS) technology. These 

evaluations assessed the capability of software to detect a 

specific term, defined textually in the language’s native 

orthography, within a conversational telephone speech 

recording. The first evaluation assessed performance of 

systems developed by performers in year 2 of the 

Intelligence Advanced Research Projects Activity (IARPA) 

Babel Program. The second evaluation, OpenKWS, 

leveraged tools and data resources to assess the 

technology across a wider research community. These 

evaluations were the culmination of a joint effort by the 

Babel Program’s testing and evaluation team, which 

included IARPA, ITL, MIT Lincoln Labs, University of 

Maryland’s Center for Advanced Study of Language, and 

MITRE to develop language resources for KWS systems in 

five languages.

The OpenKWS evaluation was open to the entire research 

community. The evaluations assessed systems across 

different factors expected to affect performance, including 

the amount of transcribed training data, the size of 

language resources brought to bear, and system retraining 

with knowledge of the keywords. Twelve teams participated 

in OpenKWS this year, three of which were new to the 

evaluation. Four teams were members of IARPA’s Babel 

Program (namely, the BABELON, LORELEI, RADICAL, 

and SWORDFISH teams) and the other eight teams 

represented various organizations from around the world, 

including China, Israel, Singapore, and the United States. 

Participating teams represented both industry and 

academia. The OpenKWS community of researchers will 

discuss the evaluation and the results in early July, after 

which results will be made public on the NIST OpenKWS14 

website. 

STAFF RECOGNITION

Ron Boisvert, Chief of ITL’s Applied and Computational 

Mathematics Division, received the 2014 Mathematics and 

Computer Sciences Award from the Washington Academy 

of Sciences. Boisvert was recognized for outstanding 

contributions to computational science, mathematical 

software, and applied mathematics.

Manny Knill, of ITL's Applied and Computational 

Mathematics Division (in Boulder) was named one of 12 

Arthur S. Flemming Award winners for 2013. Administered 

by the Trachtenberg School of Public Policy and Public 

Administration at George Washington University, these 

awards recognize outstanding federal workers with less 

than 15 years of service. Knill was recognized for his 

accomplishments in the field of quantum information 

theory. 

Adam Sedgewick, ITL’s Senior IT Policy Advisor, was 

selected by the Armed Forces Communication and 

Electronics Association (AFCEA) Bethesda Chapter as one 

of the winners of the 7th Annual Governmentwide Initiatives 

Excellence Awards. These awards recognize individuals or 

groups whose contributions in information technology have 

significance beyond their organization. Sedgewick was 

selected for the security category.

Kevin Mangold was recently selected to participate in the 

2014 International Electrotechnical Commission (IEC) 

Young Professionals Program. The three young 

professionals selected to represent the United States will 

attend the IEC 2014 General Meeting to be held in 

November 2014, in Tokyo, Japan, and participate in the 

workshop on IEC standardization strategies and conformity 

assessment.
3

Selected New Publications

Guide for Applying the Risk Management 

Framework to Federal Information Systems

Joint Task Force Transformation Initiative

NIST Special Publication 800-37 Revision 1

February 2010 (includes updates as of 6/5/14)

This publication provides guidelines for applying the Risk 

Management Framework (RMF) to federal information 

systems. The RMF promotes the concept of near real-time 

risk management and ongoing information system 

authorization through the implementation of robust 

continuous monitoring processes, providing senior leaders 

the necessary information to make cost-effective, riskbased decisions with regard to the organizational 

information systems supporting their core missions. 

Guidelines on Mobile Device Forensics

By Rick Ayers, Sam Brothers, and Wayne Jansen

NIST Special Publication 800-101, Revision 1

May 2014

Mobile device forensics is the science of recovering digital 

evidence from a mobile device under forensically sound 

conditions using accepted methods. This guide provides 

an in-depth look into mobile devices and explains 

technologies involved and their relationship to forensic 

procedures. The guide discusses procedures for the 

validation, preservation, acquisition, examination, analysis, 

and reporting of digital information.

The ghost in the machine: Don’t let it haunt your software 

performance measurements

By Vreda Pieterse and David Flater

NIST Technical Note 1830

April 2014

This paper describes pitfalls, issues, and methodology for 

measuring software performance. Ideally, measurement 

should be performed and reported in such a way that 

others will be able to reproduce the results in order to 

confirm their validity. Repeatability of experiments, 

comparability of reported results, and verifiability of claims 

that are based on such results can be achieved only when 

measurements and reporting procedures can be trusted. 

Human Engineering Design Criteria Standards Part 1: 

Project Introduction and Existing Standards

By Susanne Furman, Mary Theofanos, and Hannah Wald

NISTIR 7889

April 2014

The Department of Homeland Security (DHS) requires 

general human systems integration (HSI) criteria for the 

design and development of human-machine interfaces for 

their technology, systems, equipment, and facilities. The 

goal of the DHS Human Systems Engineering Project was 

to identify, develop, and apply a standard process to 

enhance technology and system design, system safety, 

and operational efficiency. The project manager partnered 

with ITL’s Visualization and Usability Group to advance 

this effort. The goal of this phase of the project was to 

identify and review the body of existing human factors and 

HSI standards, best practices, and guidelines in order to 

map these to potential DHS needs, technology, and 

processes.

LTE Physical Layer Performance Analysis

By Wen-Bin Yang and Michael Souryal

NISTIT 7986

May 2014

This report presents results of a physical layer 

performance study for all MCSs (from MCS0 to MCS28) in 

terms of block error rate (BLER) and spectral efficiency. 

The results are obtained by using the Steepest Ascent1 

LTE toolbox for MATLAB which provides functions for 

modeling the LTE physical layer based upon 3GPP 

specifications. The performance measures for each MCS 

are evaluated as a function of signal to noise ratio (SNR), 

MIMO configuration, and 3GPP multipath fading channel. 

The 3GPP channels include models for pedestrian, 

vehicular, and typical urban environments. The results of 

this study can be used in higher-level modeling tools such 

as cellular network planning and network modeling tools 

that take abstractions of physical layer performance as 

input.

Exploring the Methodology and Utility of Standardized 

Latent Fingerprint Matcher Scoring

By Vladimir N. Dvornychenko and George W. Quinn

NISTIR 7992

March 2014

Automated searches of fingerprints against a repository/

database are important tools of the forensic community. 

Systems performing these searches are referred to as 

Automated Fingerprint Identification Systems (AFISs). The 

output of an AFIS is a fairly small set of prospective 

candidates with attendant matching scores. These scores 

provide an indication of how likely a particular candidate is 

a true mate of the search fingerprint. One difficulty in 

interpreting matching scores in usage is that there is no 

accepted standard for its range and exact meaning. This 

report proposes a standardization of the scoring system. 
The Information Technology Laboratory (ITL) is a major research component of the National Institute of 

Standards and Technology (NIST). As a world-class measurement and testing laboratory encompassing a wide 

range of areas of computer science, mathematics, statistics, and systems engineering, our research program 

supports NIST’s mission to promote U.S. innovation and industrial competitiveness by advancing measurement 

science, standards, and technology in ways that enhance economic security and improve our quality of life. ITL 

cybersecurity experts collaborate to develop cybersecurity standards, guidelines, and associated methods and 

techniques for federal agencies and industry. Our mathematicians and statisticians collaborate with 

measurement scientists across NIST to help ensure that NIST maintains and delivers the world’s leading 

measurement capability. ITL computer scientists and other research staff provide technical expertise and 

development that underpins national priorities such as cloud computing, the Smart Grid, homeland security, 

information technology for improved healthcare, and electronic voting. We invite you to learn more about 

how ITL is enabling the future of the nation’s measurement and standards infrastructure for information 

technology by visiting our website at http://www.itl.nist.gov.

ITL Editor: Elizabeth B. Lennon

National Institute of Standards and Technology

100 Bureau Drive, Stop 8900

Gaithersburg, MD 20899-8900

Phone: (301) 975-2832

Fax: (301) 975-2378

Email: elizabeth.lennon@nist.gov

To subscribe to the 

electronic edition of 

the ITL Newsletter, go 

to 

ITL homepage

Selected New Publications (continued)

Applied and Computational Mathematics Division, Summary 

of Activities for Fiscal Year 2013

Ronald Boisvert, Editor

NISTIR 7994

March 2014

This report summarizes the technical work of the Applied 

and Computational Sciences Division of the Information 

Technology Laboratory (ITL). Part I (Overview) provides a 

high-level overview of the Division’s activities, including 

highlights of technical accomplishments during the previous 

year. Part II (Features) provides further details on five 

projects of particular note this year. This is followed in Part 

III (Project Summaries) by brief synopses of all technical 

projects active during the past year. Part IV (Activity Data) 

provides listings of publications, technical talks, and other 

professional activities in which Division staff members have 

participated. The reporting period covered by this document 

is October 2012 through December 2013.

Identifying Face Quality and Factor Measures for Video

By Yooyoung Lee, P. Jonathon Phillips, James J. Filliben, J. 

Ross Beveridge, and Hao Zhang

NISTIR 8004

May 2014

This paper identifies important factors for face recognition 

algorithm performance in video. The goal of this study is to 

understand key factors that affect algorithm performance 

and to characterize the algorithm performance. We evaluate 

four factor metrics for a single video as well as two 

comparative metrics for pairs of videos. The study 

investigated the effect of nine factors on three algorithms 

using the Point-and-Shoot Challenge (PaSC) video dataset.

Upcoming Technical 

Conferences

SHA-3 2014 Workshop

Date: August 22, 2014

Place: University of California Santa Barbara

Sponsor: NIST

The goal of the workshop is for the community to help NIST 

get a better understanding of the Secure Hash Standard 

(SHA)-3 and its possible applications, with particular focus 

on additional modes of operation for SHA-3 that might be 

worth standardizing in the future.

NIST contact: Shu-jen Chang

2014 Global Identity Summit

Dates: September 16-18, 2014

Place: Tampa Convention Center, Tampa, Florida

Sponsors: NIST and National Security Agency

Presented by the Biometric Consortium and the Armed 

Forces Communications and Electronics Association, this 

conference promotes information exchanges on scientific 

and technical advancements, operational requirements, and 

future strategic directions of biometrics and identity 

management. It focuses on solutions for the corporate, 

defense, and homeland security communities.

NIST contact: Fernando Podio

Disclaimer: Any mention of commercial products or reference to commercial organizations is for 

information only; it does not imply recommendation or endorsement by the National Institute of 

Standards and Technology nor does it imply that the products mentioned are necessarily the best 

available for the purpose.

The NIST campus in

Gaithersburg, Maryland.

Credit: Katherine Green

4