AN ASSESSMENT OF THE 

NATIONAL INSTITUTE OF 

STANDARDS AND TECHNOLOGY 

ENGINEERING LABORATORY

FISCAL YEAR 2014

Panel on Review of the Engineering Laboratory at the 

National Institute of Standards and Technology

Committee on NIST Technical Programs

Laboratory Assessments Board

Division on Engineering and Physical Sciences

THE NATIONAL ACADEMIES PRESS

Washington, D.C. 

www.nap.edu
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001

NOTICE: The project that is the subject of this report was approved by the Governing Board of the National 

Research Council, whose members are drawn from the councils of the National Academy of Sciences, the 

National Academy of Engineering, and the Institute of Medicine. The members of the authoring panel

responsible for the report were chosen for their special competences and with regard for appropriate balance.

This study was supported by Contract No. S81341-12-CQ-0036 between the National Academy of Sciences 

and the National Institute of Standards and Technology. Any opinions, findings, conclusions, or 

recommendations expressed in this publication are those of the authors and do not necessarily reflect the 

views of the agency that provided support for the project.

International Standard Book Number-13: 978-0-309-36735-6

International Standard Book Number-10: 0-309-36735-2

Copies of this report are available from

Laboratory Assessments Board

Division on Engineering and Physical Sciences

National Research Council

500 Fifth Street, NW 

Washington, DC 20001

Additional copies of this report are available from the National Academies Press, 500 Fifth Street, NW, Keck 

360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu.

Copyright 2015 by the National Academy of Sciences. All rights reserved.

Printed in the United States of America
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in 

scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general 

welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to 

advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of 

Sciences.

The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a 

parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing 

with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of 

Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and 

recognizes the superior achievements of engineers. Dr. C. D. Mote, Jr., is president of the National Academy of Engineering.

The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent 

members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts 

under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal 

government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Victor J. Dzau is 

president of the Institute of Medicine.

The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community 

of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. 

Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating 

agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the 

government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies

and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. C. D. Mote, Jr., are chair and vice chair, respectively, of the National 

Research Council.

www.national-academies.org

PANEL ON REVIEW OF THE ENGINEERING LABORATORY AT 

THE NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY

JEREMY ISENBERG, Stanford University, Chair

DONALD B. BIVENS, Consultant

JOSEPH P. COLACO, CBM Engineers

MEREDITH (MED) B. COLKET III, United Technologies Research Center

CARLOS FERNANDEZ-PELLO, University of California, Berkeley

GERALD G. FULLER, Stanford University

ERIC GUILLAUME, Laboratoire National de Metrologie et d’Essais 

TIMOTHY G. GUTOWSKI, Massachusetts Institute of Technology

MICHAEL R. JAHADI, Lockheed Martin Aeronautics Company

AGNES CHAU KLUCHA, UTC Aerospace Systems

RICHARD G. LITTLE, Rensselaer Polytechnic Institute

MICHAEL F. MCGRATH, McGrath Analytics, LLC

R. SHANKAR NAIR, Exp US Services, Inc.

NABIL NASR, Rochester Institute of Technology

SURENDRA P. SHAH, Northwestern University

E. SARAH SLAUGHTER, Built Environment Coalition

SUSAN SMYTH, General Motors Corporation

WILLIAM A. THORNTON, Cives Engineering Corporation

A. GALIP ULSOY, University of Michigan

BRUCE H. VARNER, Santa Rosa Fire Department 

THERESA WESTON, DuPont Building Innovations

JIANSHUN (JENSEN) ZHANG, Syracuse University

STEVEN J. ZINKLE, University of Tennessee, Knoxville

Staff

LIZA HAMILTON, Associate Program Officer

EVA LABRE, Administrative Coordinator

JAMES P. MCGEE, Director 

ANDREA SHELTON, Administrative Assistant

v
COMMITTEE ON NIST TECHNICAL PROGRAMS

ELSA REICHMANIS, Georgia Institute of Technology, Chair

WILLIAM C. GEAR, NEC Research Institute, Inc. (retired)

JENNIE S. HWANG, H-Technologies Group

KANTI JAIN, University of Illinois, Urbana-Champaign

KUMAR C. PATEL, Pranalytica, Inc.

ALICE WHITE, Boston University

Staff

LIZA HAMILTON, Associate Program Officer

EVA LABRE, Administrative Coordinator

JAMES P. MCGEE, Director 

ANDREA SHELTON, Administrative Assistant

vi
LABORATORY ASSESSMENTS BOARD

JOHN W. LYONS, National Defense University, Chair

ROSS B. COROTIS, University of Colorado, Boulder

PAUL A. FLEURY, Yale University

C. WILLIAM GEAR, NEC Research Institute (retired)

WESLEY L. HARRIS, Massachusetts Institute of Technology

JENNIE S. HWANG, H-Technologies Group

W. CARL LINEBERGER, University of Colorado, Boulder

C. KUMAR N. PATEL, Pranalytica, Inc.

ELSA REICHMANIS, Georgia Institute of Technology

LYLE H. SCHWARTZ, U.S. Air Force Office of Scientific Research (retired)

Staff

LIZA HAMILTON, Associate Program Officer

EVA LABRE, Administrative Coordinator

JAMES P. MCGEE, Director 

ARUL MOZHI, Senior Program Officer

ANDREA SHELTON, Administrative Assistant

vii
Acknowledgment of Reviewers

This report has been reviewed in draft form by individuals chosen for their diverse perspectives 

and technical expertise, in accordance with procedures approved by the National Research Council’s 

Report Review Committee. The purpose of this independent review is to provide candid and critical 

comments that will assist the institution in making its published report as sound as possible and to ensure 

that the report meets institutional standards for objectivity, evidence, and responsiveness to the study 

charge. The review comments and draft manuscript remain confidential to protect the integrity of the 

deliberative process. We wish to thank the following individuals for their review of this report:

Hadi Abu-Akeel, AMTENG Corporation,

Ross B. Corotis, University of Colorado, Boulder,

Gonzalo V. Castro, GEI Consultants, Inc. (retired),

William F. Baker, Skidmore, Owings & Merrill LLP,

Jennie S. Hwang, H-Technologies Group,

Kanti Jain, University of Illinois, Urbana-Champaign,

Jun Ni, University of Michigan, and

Robert Zalosh, Worcester Polytechnic Institute (emeritus).

Although the reviewers listed above have provided many constructive comments and suggestions, 

they were not asked to endorse the conclusions or recommendations, nor did they see the final draft of the 

report before its release. The review of this report was overseen by David E. Crow, University of 

Connecticut. Appointed by the National Research Council, he was responsible for making certain that an 

independent examination of this report was carried out in accordance with institutional procedures and 

that all review comments were carefully considered. Responsibility for the final content of this report 

rests entirely with the authoring panel and the institution.

viii
Contents

SUMMARY 1

1 THE CHARGE TO THE PANEL AND THE ASSESSMENT PROCESS 11

2 ENERGY AND ENVIRONMENT DIVISION 13

3 FIRE RESEARCH DIVISION 21

4 INTELLIGENT SYSTEMS DIVISION 37

5 MATERIALS AND STRUCTURAL SYSTEMS DIVISION 47

6 SYSTEMS INTEGRATION DIVISION 55

7 KEY FINDINGS AND RECOMMENDATIONS 59

ACRONYMS 65

ix

Summary

The mission of the Engineering Laboratory (EL) of the National Institute of Standards and 

Technology (NIST) is to promote U.S. innovation and industrial competitiveness through measurement 

science and standards for technology-intensive manufacturing, construction, and cyberphysical systems in 

ways that enhance economic prosperity and improve the quality of life. To support this mission, the EL 

has developed thrusts in smart manufacturing, construction, and cyberphysical systems; in sustainable and 

energy-efficient manufacturing materials and infrastructure; and in disaster-resilient buildings, 

infrastructure, and communities. The technical work of the EL is performed in five divisions: Intelligent 

Systems; Materials and Structural Systems; Energy and Environment; Systems Integration; and Fire 

Research; and two offices: Applied Economics Office and Smart Grid Program Office.

At the request of the Director of NIST, the National Research Council (NRC) formed the Panel 

on Review of the Engineering Laboratory at the National Institute of Standards and Technology and 

established the following statement of task for the panel:

The Panel on Review of the Engineering Laboratory at the National Institute of Standards and 

Technology will assess the scientific and technical work performed by the National Institute of 

Standards and Technology (NIST) Engineering Laboratory. The panel will attend an orientation 

session at the NIST facility, will review technical reports and technical program descriptions 

prepared by NIST staff, and will visit the facilities of the NIST laboratory. Visits will include 

technical presentations by NIST staff, demonstrations of NIST projects, tours of NIST facilities, 

and discussions with NIST staff. The panel will deliberate findings in a closed session panel 

meeting and will prepare a report summarizing its assessment findings.

The Director of NIST requested that the panel focus its assessment on the following factors:

1. Assess the organization’s technical programs.

• How does the quality of the research compare to similar world class research in the 

technical program areas?

• Is the quality of the technical programs adequate for the organization to reach its stated 

technical objectives? How could it be improved?

2. Assess the portfolio of scientific expertise within the organization.

• Does the organization have world class scientific expertise in the areas of the 

organization’s mission and program objectives? If not, what areas should be improved?

• How well does the organization’s scientific expertise support the organization’s technical 

programs and the organization’s ability to achieve its stated objectives?

3. Assess the adequacy of the organization’s facilities, equipment, and human resources.

• How well do the facilities, equipment, and human resources support the organization’s 

technical programs and its ability to achieve its stated objectives? How could they be 

improved?

4. Assess the effectiveness by which the organization disseminates its program outputs.

1
• How well are the organization’s research programs driven by stakeholder needs?

• How effective are the technology transfer mechanisms used by the organization? Are 

these mechanisms sufficiently comprehensive?

• How well is the organization monitoring stakeholder use and impact of program outputs? 

How could this be improved?

TECHNICAL MERIT

Overall, the technical programs of the EL and their supporting projects are well formulated to 

achieve their goals, addressing the appropriate aspects of their fields and advancing them at a reasonable 

rate. Measurement science remains one cornerstone of the research program, advancing at a high 

technical level the conviction that processes can be controlled to the degree they are measured. Computer 

simulation, effectively integrated with unique testing and physical measurements, is becoming a second 

cornerstone of the program, making possible the generalization of experimental results and facilitating the 

dissemination of technology to the user community. A third cornerstone is information gathering by EL 

managers to identify and prioritize the detailed physical experiments, simulations, measurements, and 

equipment needed to perform high-level research. These are the bases for the excellence of the EL 

research program and its relevance to national priorities. 

Execution of the technical programs is generally excellent, owing to the first-rate facilities and 

staff. Research and development conducted at the EL are important for national prosperity and safety. For 

example, integration of advances in material science with advanced manufacturing or with efficient 

building systems that resist extreme winds is essential to realize the value of research investments. It is 

important that the EL continue its search and support for such integration throughout the laboratory.

The impact of some technical programs is highly significant. Vivid examples include the Fire 

Research Division’s (FRD’s) contributions to science, codes, standards, assessment tools, equipment, and 

firefighter tactics, which are highly influential. The pace of research will increase as experimental results 

from the National Fire Research Laboratory (NFRL) come online. Another example is the Net-Zero 

Energy Residential Test Facility (NZERTF), which, while simulating the ordinary life of a family of four 

over the past year, has recently demonstrated a small net energy surplus at the end of the 1-year test 

period. The NZERTF is already a technical success and is expected to have great impact on the 

architecture and engineering of single-family dwellings. The impacts of other technical activities in 

progress are also highly promising. Among these are the material and structural systems activities to 

reduce the carbon footprint of concrete and to improve the sustainability of polymeric materials. The 

updating of wind speed maps and collection of data on tornado-induced damage will advance structural 

safety. The Intelligent Systems Division (ISD) industrial robot program has contributed to development 

of unified international standards for safe modes of human-robot collaboration. The Systems Integration 

Division (SID) had a significant impact on the International Organization for Standardization (ISO)

Standard for the Exchange of Product (STEP) model data that establishes protocol and standards for 

interoperability testing of manufacturing processes such as model-based engineering. A tangible 

accomplishment of the program is the NIST STEP file analyzer, which detects errors in manufacturing 

data exchange files.

SCIENTIFIC EXPERTISE

There are highly capable, motivated, and enthusiastic staff in all EL divisions. Overall, the 

personnel demonstrate their ability to apply technical knowledge and to remain connected to the broader 

research and standards community. Generally, EL staff efficiently provide a bridge between the scientific 

and user communities. As an example, the FRD collects information on the needs of the fire-fighting 

community and has developed in response scientifically based content within its programs and clear 

2
interpretations for training firefighters and improving specifications for their equipment. EL staff in all 

divisions have advanced their expertise in computer simulation and modeling, which now play a major 

role in creating or underpinning new and significant capabilities. This evolution requires parallel expertise 

in evaluating uncertainties associated with numerical methods and data sets. Support from the statistics

engineering division within the NIST Information Technology Laboratory is essential.

Relationships with outside researchers have many benefits, including cost-effective access to 

outside large-scale testing facilities. These valuable relationships often arise from the continuing 

education of EL staff members. As is customary, EL personnel are active in codes and standards 

committees. However, travel restrictions may hamper the ability of EL personnel to make or extend 

useful contacts with other government laboratories and private sector stakeholders.

In some instances first-rate research expertise resides in a small number of personnel or, in a few 

cases, in only one person. Some disciplines with fewer than one junior and two senior experts appear to 

be at risk of losing competency. Formal mapping of competencies can be extended to staff participation in 

key scientific and engineering communities. 

The ISD is able to hire permanent staff, but there are external and internal challenges to doing so. 

There are limited numbers of U.S. citizens graduating with advanced degrees in engineering disciplines, 

and there is a competitive hiring environment for qualified candidates in high-priority manufacturing 

areas such as robotics, industrial control system (ICS) security, and additive manufacturing. Internal 

challenges are imposed by such factors as the speed of the hiring process, travel restrictions for bringing 

in new hires, difficulties in arranging moving expenses, and pay considerations. Sources of permanent 

hires include the pool of guest researchers and the Summer Undergraduate Research Fellowship (SURF), 

Pathways, and NRC postdoctoral fellows programs.

Overreliance on guest researchers jeopardizes investments in research with long-term payoffs. 

The creation of exciting new programs such as the NRFL, the NZERTF, and the Sustainability Initiative 

bring with them the need for new expertise. It is important to support these programs with permanent 

staff. 

FACILITIES AND EQUIPMENT

Major new facilities reflect judgment that the national interest lies in advancing such technologies 

as net-zero energy residences, fire hazard mitigation at full building scale, resistance of buildings to 

disproportionate collapse due to terrorist action, and robotic response to disaster investigation.

The NZERTF is a unique testbed that applies simulation, measurement science, and energy

efficiency technologies aimed at developing and deploying advances in measurement science to move the

nation toward cost-effective net-zero energy buildings while maintaining a healthy indoor environment. 

This program and its showcase facility are supported by a complex of computer simulation and physical 

testbeds that ensure the efficiency and scientific integrity of the program. To realize the full potential of 

the NZERTF, it will be necessary to develop a metric system for evaluating indoor environmental quality

(IEQ) and a comprehensive framework for computer simulation of whole building performance. It is 

important that the project team consider these concepts when articulating the goals of the project. It will 

also be necessary to develop a system of databases to manage efficiently the large amount of data to be 

collected from the various advanced system testbeds. 

The NRFL is a physical fire test facility that overcomes previous limitations on the size of fires 

and building elements that could be tested under working loads.1 Testing of large-scale fires is important 

because the properties of fire and its impact do not scale linearly with fire size. The new NFRL provides a 

unique opportunity to study larger fires (up to 20 MW continuously and 30-40 MW peak) and their 

impact on large structures and surfaces under working load conditions. The combination of the new 

1 A description of the National Fire Research Laboratory is available at http://www.nist.gov/el/fire_research

/nfrl/, accessed November 20, 2014.

3

 
facility, which accommodates two-story structures with multiple rooms and their furnishings, and legacy 

facilities represents a new, versatile capability that exists nowhere else. 

The manufacturing robotics testbed and the intelligent sensing and perception testbed are 

laboratory spaces containing physical robotic systems supported by measurement science for robotic 

perception and safety of human-robot systems. Applications addressed through joint research with other 

federal agencies, such as robotic assistance in emergency response, are in progress, demonstrating the 

versatility of the facility. Long-standing programs in machine tool performance and machining metrology 

are supported by excellent facilities that, although aging, are well maintained and suited to their purpose. 

In additive manufacturing, the EL machine for powder bed fusion is old and may not match the 

capabilities of modern equipment to detect and measure defects in additive manufactured parts. 

Facilities in the Materials and Structural Systems Division (MSSD) for characterization of 

polymers are among the best in the world. The division’s work on polymer sustainability has focused on 

photo-degradation using the unique 2-meter integrating sphere to accelerate ultraviolet (UV) effects on 

polymers. Facilities for examining sustainable substitutes, fly ash and limestone, and for studying the 

rheology of cement pastes are adequate for their purpose. Facilities for small-scale testing of structural 

elements are adequate. Because gravity loading is an essential factor in investigating disproportional 

collapse, it is important that full-scale testing be performed. Rather than incur the expense of building its 

own facility for infrequent tests, EL staff have been creative in using outside testing facilities for full-size 

frames and even complete buildings. 

It is important that the SID develop an interoperability testbed to demonstrate integration of life

cycle engineering, information modeling and testing, systems engineering, and process engineering for 

specific manufacturing processes. The testbed might be virtual, entirely based on computational 

modeling, or it might be a hybrid of physical and virtual modeling. Such an activity would offer crosscutting possibilities with programs in the ISD and the Energy and Environment Division (EED). It is also 

important that the SID consider developing equipment energy measurement protocols— for example,

under conditions of different production volumes and parameter settings, leading to a testbed for 

sustainable manufacturing. This activity, too, offers cross-cutting possibilities with the EED.

DISSEMINATION

The dissemination practices of the EL divisions share a common traditional approach with a 

strong focus on publishing and conference attendance and the development and publication of standards. 

Research work is published in good journals and conference proceedings, and some work has achieved a 

good citation record. Examples of NIST accomplishments include:

• The NIST Guide to Industrial Control Systems (ICS) Security has been downloaded from its 

Internet site more than 2.5 million times since its initial draft release in 2006.

• An article in the American Society for Testing and Materials (ASTM) International

Standardization News2 highlighted how EL’s work led to the success of a response robot used at 

the Fukushima nuclear power plant.

• EL staff have participated in more than 200 technical and standards committees of the ASTM; the 

American Society of Heating, Refrigerating and Air-conditioning Engineers (ASHRAE); the 

National Fire Protection Association (NFPA); the American National Standards Institute (ANSI); 

ISO; the American Concrete Institute (ACI); the American Society of Civil Engineers (ASCE);

and other organizations. By their personal participation, NIST staff exert influence directly on the 

formulation and expression of industry standards.

2 K. Nelson and C. Enright, “Robots to the Rescue,” Standardization News, May/June 2013.

4

 
• The EL published Technical Investigation of the May 22, 2011, Tornado in Joplin, Missouri.

3

• EL staff supported development of ASHRAE standards for residential air-tightness and 

ventilation.

• The EL has disseminated through its Internet site widely used software for sustainable building 

materials (24,000 users from 80 countries have accessed the software at the rate of 5,000 

hits/month); for contaminant transport in buildings (there have been 4,000 downloads of this 

software); and for natural ventilation design (there have been 1,400 downloads of this software).

• EL staff have supported the development of the ISO 10303 STEP model data.

There are opportunities for wider use of social media to publicize the EL’s work, including 

focused dissemination to target audiences. Targets include industry leaders in heating, ventilation, air 

conditioning, and refrigeration (HVAC&R); the architectural and design community; home energy raters;

and energy auditors. 

EL divisions sponsor workshops to bring together stakeholders from private and government 

sectors and academia. While the workshop approach is useful, workshops occasionally overlook sister 

government laboratories and occasionally do not optimally engage private sector participants. 

The divisions of the EL support a variety of stakeholders, whose inputs are solicited when 

developing research programs. Stakeholders are identified by several means: leadership in professional 

organizations, participation in other national laboratories’ planning, participation in standards 

development activities, participation in professional and technical meetings, initiating and/or participating 

in roadmapping exercises, sponsorship of workshops, and direct interactions with national laboratories, 

universities, and industry.

KEY FINDINGS AND RECOMMENDATIONS

Key findings and recommendations are provided here by reference to divisions within the EL. 

Chapters 2 through 6 contain these and additional findings and recommendations.

Materials and Structural Systems Division

Research performed in the Inorganic Materials Group of the MSSD supports life prolongation 

and rehabilitation of concrete materials and improves sustainability by decreasing cement content. It has 

applications in the hydrocarbon exploration industry, which uses concrete to encase well bores. Proper 

formulation of concretes that can properly set up and seal the well bore is essential for both increased 

production and decreased leakage. 

It is important that the group establish better linkages between chemical and physical 

degradation and the mechanical properties of construction materials and that it consider developing a 

method for noninvasive in situ testing of the strength of fresh concrete. There is a need for the division to 

conduct further research into bio-based materials and fiber-reinforced structural systems, as well as into 

ubiquitous existing materials (steel, masonry, and wood) for new construction and in situ rehabilitation, 

for durability, functional performance, and resilience. 

Recommendation 1. The Inorganic Materials Group should leverage work for the Nuclear 

Regulatory Commission on alkali-silica reaction to connect the nanoscale investigations of 

concrete deterioration to structural performance.

3 E.D. Kuligowski, F.T. Lombardo, L.T. Phan, M.L. Levitan, and D.P. Jorgensen, Final Report, National 

Institute of Standards and Technology (NIST) Technical Investigation of the May 22, 2011, Tornado in Joplin, 

Missouri, March 26, 2014, doi:http://dx.doi.org/10.6028/NIST.NCSTAR.3.

5

 
Recommendation 2. The Inorganic Materials Group should engage the oil and gas industry 

to develop safe encasement of well bores.

The work by the Polymeric Materials Group on the photophysics of polymeric degradation is 

mature. The group needs to engage the solar panel industry where polymeric materials are used to 

encapsulate thin films between panes of glass. Standard materials used by the industry are ethyl vinyl 

alcohol and polybutadiene, which are problematic and require guidance for formulating sealants and 

adhesives. It would be beneficial if the polymer and concrete teams were to increase their collaboration—

for example, polymer fibers to strengthen concrete and resist cracking and polymer additives to enhance 

the flow properties of concrete.

Recommendation 3. The Polymeric Materials Group should expand research, including 

molecular modeling, on polymeric degradation to non-ultraviolet degradation effects and 

should include the data in models for predicting service life.

Recommendation 4. The Polymeric Materials Group should engage the solar panel industry 

where polymeric materials are used to encapsulate thin films between panes of glass.

Work of the Structures Group could be improved by collaborating with the research and 

practitioner community to identify and evaluate alternative approaches for quantitative metrics to describe 

phenomena such as disproportionate collapse. A metric for robustness would be useful. Additional 

collaboration with the FRD on the durability and performance of materials and structures under fire 

conditions would also be beneficial. Fully integrating the group’s capabilities in analysis, computational 

modeling, and physical testing would help to obtain the maximal amount of information from the work in 

disproportionate collapse. 

Much of the work of the Structures Group is disseminated through the code development process. 

However, significant results, such as those on plasticity found through the work on disproportionate 

collapse, could be disseminated more broadly through technical and academic journals. 

The concept of a virtual wind tunnel based on historic data, new wind speed maps, and 

computational modeling would be valuable for the group studying wind research. It would also be 

beneficial to consider in future work the potential of microbursts to cause structural damage.

Recommendation 5. The group studying wind research should develop the concept of a 

virtual wind tunnel, based on observations and computational modeling.

The group studying community disaster resilience needs to monitor its staffing requirements to 

assure that human resources are adequate and in the appropriate disciplines for the magnitude and scope 

of the tasks assigned. The group also needs to develop and publicize an explicit roadmap to define where 

and how the work of the group will mesh with existing and future codes and standards and identify who 

will benefit from these efforts. A useful roadmap would also make clear the overarching governmental 

interest in the program’s contributions to improving economic impacts and enhancing life safety during 

and following extreme events. 

The group’s efforts in disaster resilience could beneficially include partnerships across NIST and 

externally (e.g., those involved with research in the wildlife-urban interface, earthquakes, and post-flood 

indoor air quality) for science-based resilience performance and standards. The group could leverage 

existing U.S. and international research, methodologies, and tools to accelerate advancements in theory 

and practice, including standards and measurements. Operations and maintenance considerations are good 

candidates for incorporation into the high-level objectives for the Community Disaster Resilience 

Program.

6
The excellent capabilities in the group and the findings of the National Construction Safety 

Team’s field-based failure studies can be integrated to inform the research program and identify critical 

research topics. Incorporating into the resilience program factors relating to the environment and natural 

systems may require new partnerships—with, for example, the Environmental Protection Agency’s

Green Infrastructure Collaborative.

Intelligent Systems Division

The ISD measurement science research programs aim to advance the versatility of intelligent 

automation technologies for smart manufacturing and cyberphysical systems applications; enable 

performance optimization of smart manufacturing systems; and enable rapid, cost-effective production of 

products through advanced manufacturing processes and equipment.

The division’s work on wireless factory networks addresses an important need, as does the 

project on cybersecurity of industrial control systems. The objective of the Smart Manufacturing 

Construction Systems (SMCS) program is to develop and deploy advances in measurement science for 

sensing, modeling, and optimizing manufacturing activities in a smart factory. In its factory cybersecurity 

project, the division has demonstrated leadership in addressing cybersecurity in smart factory systems and 

other industrial cyberphysical systems. Indicative of this leadership is the recent publication of a new 

draft of NIST SP 800-82, Guide to Industrial Control Systems (ICS) Security.

4 The ISD technical staff 

also led the development of IEEE Standard 1588, “Precision Clock Synchronization Protocol for 

Networked Measurement and Control Systems.” Current industrial robots cannot operate safely with 

humans, and so they need to be isolated. This limits their applicability and increases the costs of 

automation. The Next-Generation Robotics and Automation (NGRA) program is beginning to develop 

performance evaluation methods and standards for safe robot-human interactions. The division has started 

the standardization effort on the safety of automated guided vehicles (AGVs). The laboratory is extending 

safety evaluation to other systems, attempting to cover various aspects of evaluating robotic hand 

capabilities, such as position, torque, grasp types, graspable object size and ranges, touch location, force, 

and pressure via tactile sensing.

The decision in fiscal year 2014 to terminate the smart machining activities and focus on 

measurement science for additive manufacturing may risk the loss of critical capability and recognized 

expertise in smart machining. Advanced manufacturing needs both additive and subtractive processes. 

Subtractive manufacturing processes typically refer to conventional manufacturing processes by which 

material is removed—for example, from a billet, to make a part and achieve final desired geometries and 

tolerances. Additive manufacturing processes refer to a suite of methods to add material, typically in 

powder or wire form, to make a part, add features to a part, or to repair a part. A hybrid approach 

consisting of both additive manufacturing and smart machining is needed to support the development of 

measurement science for additive manufacturing processes.

New additive manufacturing machines with enhanced features and capabilities are introduced to 

the market each year. There are several laser powder bed fusion of metal machines, many with higher 

performance lasers and optics, controlled environmental chambers, automated powder delivery, inprocess monitoring, powder compaction, and other features and tools to further advance measurement 

science knowledge. These new machines are being purchased by industry, academia, research institutions, 

and government entities globally. It may be difficult for industry and academia to see the ISD at the 

forefront of measurement science when its equipment is not adequate to the task. The current machines at 

the ISD are adequate for the initial work and developing expertise. Ownership of and/or access to 

machines with capabilities and equipment that can detect and measure defects of additive manufactured 

4 NIST, Guide to Industrial Control Systems (ICS) Security, SP 800-82, Revision 2 Initial Public Draft, 

Gaithersburg, Md., May 2014.

7

 
parts in situ or postprocess would support the ISD’s goal of enabling widespread adoption by the U.S. 

manufacturing industry of additive manufacturing processes for metal. 

The ISD is appropriately planning and executing programs in new technology areas such as 

SMCS, additive manufacturing, industrial robotics, and cyberphysical systems. In addition, the division 

has seen an explosion of government and global industry interest in its work on ICS cybersecurity, with 

more than 2.5 million downloads from the NIST Internet site of this year’s revision of the Guide to 

Industrial Control System (ICS) Security (NIST SP 800-82). The division currently has one staff member 

who is widely recognized for deep expertise in this area and who is spread very thin in responding to 

current program demands. The division is trying to hire at a time when ICS cybersecurity experts can 

demand high salaries and other benefits from industry. Because the need to staff ongoing and new 

programs is urgent, the division cannot wait and hope for qualified, affordable candidates to become 

available. A strategy is needed to hire now, where possible (by emphasizing factors beyond salary alone 

that make NIST employment attractive), to invest in developing the needed multidisciplinary skills in 

current staff and entry-level hires, and to provide interim support through contracts and cooperative 

research arrangements with universities and others. 

The ISD would benefit from an extensive data set supporting its additive manufacturing efforts. 

The pilot round-robin tests that the division and its partners have been engaged in address build-to-build 

variation and machine-to-machine variation of the same machine model. Building on the pilot roundrobin testing with equipment manufacturers, manufacturing companies, academia, and government 

institutions that are working with different laser powder bed fusion machines or materials would help 

generate a larger data set and address machine-model-to-machine-model variation. The ISD’s approach 

can be standardized and disseminated widely.

Recommendation 6. The Intelligent Systems Division should build on the pilot round-robin 

testing with certified sources to generate a large data set for their additive manufacturing 

efforts.

Energy and Environment Division

The EED has been focusing on the measurement of energy performance and indoor 

air/environment quality. The EED has two major programs that support its goal of sustainable and energyefficient manufacturing, materials, and infrastructure. The Net-Zero Energy, High-Performance Buildings 

Program focuses on energy-efficient building systems and advancing the measurement science for their 

characterization and performance assessment; the Embedded Intelligence in Buildings Program focuses 

on the development and application of intelligent information and control technologies to improve 

building operation.

Metrics are essential for evaluating IEQ and energy performance, and sustainability of buildings 

remains a major challenge for the field. A single index does not exist for quantifying IEQ. Many factors 

need to be considered, including thermal environment, pollution and noise levels, lighting quality, and 

occupant satisfaction. It is important to develop an accepted means of quantifying IEQ benefit that can be 

integrated with the more readily measured energy benefit. Such integration includes developing a

comprehensive framework for modeling and simulating whole building performance, based on the 

fundamental understanding of the combined heat, air, moisture, and pollutant transport processes in 

building systems. The transport processes are affected by material characteristics, properties of pollutant 

species, and environmental conditions. With its capabilities in laboratory measurements and modeling,

the EED is in position to lead a national effort in such a development. Overall, the EED exhibits excellent 

capabilities in the areas of current demands such as energy management and metrics, but the scope of its 

expertise may need to be expanded to include critical IEQ areas such as management of and metrics for 

moisture/mold and nanoparticles.

8
Improvements in methods and procedures are needed to obtain reliable field-scale data for 

validating the models and measurement methods developed from laboratory studies. Many existing field 

studies do not have enough rigor to generate reliable data. A standard, scientifically based test protocol 

for field use is needed to catch up with EED laboratory research to capitalize fully on the laboratory 

program.

Recommendation 7. The Energy and Environment Division should continue to develop 

metrics for evaluating indoor air quality and energy performance concurrently.

Recommendation 8. The Energy and Environment Division should perform field

investigations to collect reliable data for validating the models and measurement methods

developed from the laboratory studies.

Systems Integration Division

The SID contributes to measurement science and standards needed to integrate engineering 

information systems used in manufacturing, construction, and cyberphysical systems. Areas of work 

include manufacturing enterprise integration; green manufacturing and construction; engineering and 

manufacturing products, processes, equipment, technical data, and standards; collaborative manufacturing 

research under pilot grants; research performed through manufacturing fellowships; systems integration 

and engineering; life-cycle assessment; cyberphysical systems; productivity measurement; sustainability;

and energy efficiency.

5

SID staff have developed strong expertise in several key areas, including standards development 

for systems engineering, model-based engineering, development and support of a STEP standards (ISO 

10303-242 [STEP AP242]), composite manufacturing, additive manufacturing, and assisting vendors in 

implementation of the new functionality.

SID staff have developed process-level energy models for injection molding and welding. These 

models appear to be mechanism-based rather than equipment component-based models. However, such 

models contain numerous adjustable parameters, such as mechanism efficiencies, that practitioners need 

to measure. Rather than trying to simulate all manufacturing processes (an enormous task), the SID team 

could consider developing equipment energy measurement protocols. Such an activity could serve as a 

testbed for the division. A standard scheme for equipment measurement might be a very useful 

contribution to sustainable manufacturing. 

Recommendation 9. The Systems Integration Division should establish a testbed to support 

development, validation, demonstration, and technology transfer.

Fire Research Division

The FRD has made progress toward improving the safety and effectiveness of firefighters through 

measurement science to advance suppression tactics, examining nontraditional means of fire suppression, 

and transferring the results to the fire service. Field-scale burn tests have been accomplished through 

outside funding opportunities, collaboration, and partnerships. 

The efforts to communicate research findings on fire dynamics have generated much interest and 

discussion within the professional fire service community. Presentations, articles in fire service print and 

5 NIST, “The Engineering Laboratory—Summaries of Our Activities, Accomplishments and Recognitions,” 

Gaithersburg, Md., July 2014.

9

 
online media providing opportunities for fire service organizational involvement, and, most recently, the 

use of social media have all contributed to this success. 

The wildland-urban interface (WUI) problem is growing very fast, and its growth may accelerate 

due to climate change and diminishing safety margins between wildland and the built environment. 

Because the WUI problem is relatively recent, there is little information about WUI fire characteristics 

and how to reduce its consequences. Therefore, it is of high priority to collect data from WUI fires to 

underpin standards and codes for fire-hardening structures located in WUI-prone areas. Field studies, 

some of which have been conducted by the NIST National Construction Safety Team, provide essential 

data on WUI fires, including the finding that ignition of structures by embers accounts for up to 50 

percent of the structure fires in a WUI. 

The Fire Dynamics Simulator (FDS) modeling team has developed new goals for the future. One 

of these is the coupling of three types of complex codes (gas-phase fluid dynamics and reaction, thermal 

transport to and through solid structures, and failure of solid structures under load and thermal stress). 

This is a significant effort in computation, validation, and interpretation of results. 

The NFRL is an exciting new facility for the FRD and the EL. It will provide a unique capability 

to consider the fire heating of structures under structural load and will enable the FRD team to break new 

ground. The ability to test structures exposed to fires while they are under realistic gravity loads is an 

exciting new capability. The community expects that findings from these tests will form the basis for new 

building requirements. The FRD conducted a planning workshop to establish long-term priorities for 

testing so that when commissioning is complete testing can begin promptly.

Recommendation 10. The Fire Research Division should complete the extremely important 

new National Fire Research Laboratory and prioritize the test series for the first 2 years 

following the laboratory’s commissioning. 

Recommendation 11. The Fire Research Division should strengthen its research on 

wildland-urban interface fires, including data collection from real events and new modeling

approaches.

10
1

The Charge to the Panel and the Assessment Process

At the request of the National Institute of Standards and Technology (NIST), the National 

Research Council (NRC) has, since 1959, annually assembled panels of experts from academia, industry, 

medicine, and other scientific and engineering environments to assess the quality and effectiveness of the 

NIST measurements and standards laboratories, of which there are now seven,1 as well as the adequacy of 

the laboratories’ resources. 

At the request of the Director of NIST, in 2013 the NRC formed the Panel on Review of the 

Engineering Laboratory at the National Institute of Standards and Technology and established the 

following statement of task for the panel:

The Panel on Review of the Engineering Laboratory at the National Institute of Standards and 

Technology will assess the scientific and technical work performed by the National Institute of 

Standards and Technology (NIST) Engineering Laboratory. The panel will attend an orientation 

session at the NIST facility, will review technical reports and technical program descriptions 

prepared by NIST staff, and will visit the facilities of the NIST laboratory. Visits will include 

technical presentations by NIST staff, demonstrations of NIST projects, tours of NIST facilities, 

and discussions with NIST staff. The panel will deliberate findings in a closed session panel 

meeting and will prepare a report summarizing its assessment findings.

The Director of NIST requested that the panel focus its assessment on the following factors:

1. Assess the organization’s technical programs.

• How does the quality of the research compare to similar world class research in the 

technical program areas?

• Is the quality of the technical programs adequate for the organization to reach its stated 

technical objectives? How could it be improved?

2. Assess the portfolio of scientific expertise within the organization.

• Does the organization have world class scientific expertise in the areas of the 

organization’s mission and program objectives? If not, what areas should be improved?

• How well does the organization’s scientific expertise support the organization’s technical 

programs and the organization’s ability to achieve its stated objectives?

3. Assess the adequacy of the organization’s facilities, equipment, and human resources.

1 The seven NIST laboratories are the Engineering Laboratory, the Physical Measurement Laboratory, the 

Information Technology Laboratory, the Material Measurement Laboratory, the Communication Technology 

Laboratory, the Center for Nanoscale Science and Technology, and the NIST Center for Neutron Research.

11

 
• How well do the facilities, equipment, and human resources support the organization’s 

technical programs and its ability to achieve its state objectives? How could they be

improved?

4. Assess the effectiveness by which the organization disseminates its program outputs.

• How well are the organization’s research programs driven by stakeholder needs?

• How effective are the technology transfer mechanisms used by the organization? Are 

these mechanisms sufficiently comprehensive?

• How well is the organization monitoring stakeholder use and impact of program outputs? 

How could this be improved?

The context of this technical assessment is the mission of NIST, which is to promote U.S. 

innovation and industrial competitiveness by advancing measurement science, standards, and technology 

in ways that enhance economic security and improve the quality of life. The NIST laboratories conduct 

research to anticipate future metrology and standards needs, to enable new scientific and technological 

advances, and to improve and refine existing measurement methods and services.

In order to accomplish the assessment, the NRC assembled a panel of 23 volunteers, whose 

expertise matches that of the work performed by the Engineering Laboratory (EL) staff.2

In April 2014, most of the panel members attended one of two orientation sessions provided at 

the NIST facility in Gaithersburg, Maryland, during which they attended interactive presentations by the 

NIST Director, the Director of the NIST Program Office, and EL management and staff.

On July 22-24, 2014, the panel assembled for two and a half days at the NIST facility during 

which it received welcoming remarks from the NIST Director’s representative, heard an overview 

presentation by EL management, and attended an interactive session with EL management. Each panel 

member was assigned to one of the five division review teams, whose expertise matched that of the work 

performed in the five divisions of the EL: (1) Energy and Environment, (2) Fire Research, (3) Intelligent 

Systems, (4) Materials and Structural Systems, and (5) Systems Integration. The division review teams 

separately attended division-level presentations and visited division laboratories. The panel also met in a 

closed session to deliberate on its findings and to define the contents of this assessment report.

The panel’s approach to the assessment relied on the experience, technical knowledge, and 

expertise of its members. The panel reviewed selected examples of the technical research performed at the 

EL; because of time constraints, it was not possible to review the EL programs and projects exhaustively. 

The examples reviewed by the panel were selected by the EL. The panel’s goal was to identify and report 

salient examples of accomplishments, challenges, and opportunities for improvement with respect to the 

factors suggested above by the Director of NIST. These examples are intended collectively to portray an 

overall impression of the laboratory, while preserving useful suggestions specific to the projects and 

programs that the panel examined. The panel applied a largely qualitative rather than a quantitative 

approach to the assessment, although it is possible that future assessments will be informed by further 

consideration of various analytical methods that can be applied.

Given the necessarily nonexhaustive nature of the review, the omission of any particular EL 

program or project should not be interpreted as a negative reflection on the omitted program or project.

2 See http://www.nist.gov/el/ for information on Engineering Laboratory organization and programs, accessed 

August 6, 2014.

12

 
2

Energy and Environment Division

The Energy and Environment Division (EED) of the EL develops measurement science,

predictive models, and performance metrics to improve the energy efficiency of building components and

systems, reduce building-related CO2 emissions, enhance the quality of the indoor environment, and

improve the building design and construction process through the integration of information,

communications, sensing, and automation technologies.1

TECHNICAL PROGRAMS

The EED has two major programs that support its goal of sustainable and energy-efficient

manufacturing, materials, and infrastructure. The Net-Zero Energy, High-Performance Buildings Program

focuses on understanding building systems and advancing the measurement science for their

characterization and performance assessment, and the Embedded Intelligence in Buildings Program

focuses on the development and application of intelligent information and control technologies to

improve building operation.

Net-Zero Energy, High-Performance Buildings Program

The program includes 12 projects distributed among four thrusts—whole building metrics,

envelope load reduction, equipment efficiency, and on-site energy generation—all of critical importance

for achieving the program objective of developing and deploying advances in measurement science to

move the nation toward cost-effective, net-zero energy buildings while maintaining a healthy indoor

environment.

Accomplishments

The program has focused on the measurement science necessary for the development of net-zero

energy buildings with healthy indoor environments. A unifying piece of the work is the development of

an approach to designing and operating net-zero energy single-family houses, as demonstrated in the

NZERTF, which has been collecting data on its energy use for a year. The NZERTF will enable detailed

simulation of the effects of occupant activities and building dynamics on energy and IEQ. Through a

year-long monitoring of performance, the NZERTF has generated previously unavailable data, including

data on the effects of outdoor temperature variation on several parameters, including the pollutant

emissions from building envelope materials, the impact of these emissions on indoor air quality, the

variation of ventilation rate over time, and energy and water consumptions. The facility is also versatile

1 National Institute of Standards and Technology, “The Engineering Laboratory—Summaries of Our Activities, 

Accomplishments and Recognitions,” Gaithersburg, Md., July 2014.

13

 
for evaluating different heating, ventilating, and air-conditioning systems. For example, three different

types of geothermal heat pump systems can be readily tested concurrently at the facility.

Another highlight of the program is the development of new metrics and tools to assess the

sustainability of buildings, including a novel metric for assessing the different aspects of IEQ. Using the

new metrics, the Building Industry Reporting and Design for Sustainability (BIRDS) software has been

developed for the assessment of building systems’ energy, economic, and environmental performance.

A third area of accomplishment is the development of reference materials and reference databases

in an effort to reduce uncertainties in the testing and evaluation of indoor air contaminant emission

sources, insulation materials, HVAC&R equipment with various refrigerants, and energy monitoring

systems. Examples include a new reference material for calibrating environmental chambers for emission

testing; thermal insulation standard reference materials (SRMs) and the reference database for building

insulation; and two standard reference databases for HVAC&R equipment using various refrigerants,

Cycle D (SRD49) and REFLEAK (SRD73).

A fourth area of accomplishment is the development of modeling and simulation tools for modelbased testing and evaluation that would enable extrapolation of laboratory testing results to field

conditions, useful for system optimization. The tools include a model and software tool (EVAP-COND)

to optimize the effectiveness of heat exchangers used in a vapor compression system; and an integration

of multizone airflow, IAQ, and energy modeling tools via coupling of the CONTAM airflow modeling

tool with the TRANSYS (transient system simulation) building system modeling tool and with the

Department of Energy’s EnergyPlus software modeling suite.

Research findings from the program have also made significant contributions to the development

of industrial, national, and international standards. They include several ASTM standards in the area of

photovoltaics, thermal insulation, material emissions, building economics, and green concrete and

corresponding ISO standards.

Opportunities and Challenges

There remain several significant challenges and research opportunities for the program. Metrics

are essential for evaluating IEQ and energy performance, and sustainability of buildings remains a major

challenge in the field. Many design factors such as form and massing, building envelope, and ventilation

system interact to affect both IEQ and energy performance of buildings. A single index does not exist for

quantifying IEQ. Many aspects need to be considered, including thermal environment, pollution and noise

levels, lighting quality, and occupant satisfaction. It is important to develop an accepted means of

quantifying the IEQ benefit to allow comparison with the more measurable energy benefit.

A second challenge lies in the development of an approach to organizing the large amount of data

to be collected from the various advanced system testbeds, ranging from material to assembly/component

to system-level data, each with associated uncertainties. Such vast and valuable data need to be organized

effectively and shared efficiently among research and industrial communities.

A third challenge is to develop a more comprehensive framework for modeling and simulating

whole building performance, based on the fundamental understanding of the combined heat, air, moisture,

and pollutant transport processes in building systems. Building systems are complex, involving multiscales both spatially and temporally. The transport processes are affected by material characteristics,

properties of pollutant species, and environmental conditions. With its capabilities in laboratory

measurements and modeling, the EED is in the right position to lead a national effort to develop such a

framework.

A fourth challenge is to develop methods and procedures for obtaining reliable data in field

investigation that would validate the models and measurement methods that are developed based on

laboratory studies. Many existing field studies do not have enough rigor to generate reliable data, and

such data sets are difficult, if not impossible, to compare with one another. A standard test protocol based

on sound measurement science is essential, which presents an opportunity for EED research.

14
Overall Assessment

The program has focused on the essential technical areas and built unique research facilities and

modeling capabilities needed to achieve its objective of advancing measurement science for developing

healthy net-zero energy buildings. A combination of experimental and modeling approaches has been

applied, enabling the development of test and evaluation methods from materials, assemblies/components,

and systems levels. The quality of the research conducted in the program is nationally and internationally

recognized, and it is at a very high level. The results of the research have contributed significantly to the

advancement of both code reference standards such as ASHRAE/IES 90.1 and ASHRAE 62.1 and 62.2

and the standard for high-performance buildings such as ASHRAE/IES/USGBC 189.1. These standards

are being adopted across the country, universally referenced, and used by the building industry, and they

have significantly influenced building standards in other countries.

Embedded Intelligence in Buildings Program

The program focuses on improving building operation through the use of advanced information

and control technologies. It has six projects in four thrusts: system commissioning, automated fault

detection and diagnosis (FDD), intelligent agent-based optimization, and integration with smart grids, all

of which are critically important in achieving the program’s objective of developing and deploying

advances in measurement science that will improve building operations to achieve energy efficiency,

occupant comfort, and safety through the use of intelligent building systems.

Accomplishments

The program has focused on the measurement science necessary for the successful application of

information and control technologies to improve the operation of buildings for human health, safety, and

performance. A unique smart building automation and control testbed has been developed that is capable

of emulating a variety of buildings and climate conditions in support of commissioning and FDD

technology development and of providing technical inputs to improve key industrial standards, including

the widely adopted building automation and control network (BACnet) and BACnet conformance testing

standards. The testbed enables accurate and reliable assessments of a variety of control algorithms,

industrial controllers, and communication devices that are compatible with BACnet. More than 780

companies manufacturing building controllers and mechanical equipment have adopted the BACnet

protocol for controller communication, and the number of BACnet vendors continues to increase at a fast

pace. Market estimates indicate that more than 50 percent of worldwide sales of new building controllers

are BACnet.

The division has developed several important and valuable software tools, including an advanced

commissioning software tool, HVAC-Cx, that enables the use of data from building energy management

systems (BEMS) for FDD as well as for improving the commissioning process; a beta version of the FDD

software tool FDD EA (FDD Expert Assistant); an FDD tester/evaluator software for air conditioners and

heat pumps; and a fault-free and fault database for wide application by industrial FDD developers.

Comprehensive simulations and analyses have also been conducted to show the energy-saving

potential of using the intelligent agent optimization approach. A unique full-scale facility for testing and

evaluating intelligent agent algorithms has been designed and is under construction. It will enable

collection of reliable reference data sets for the development and evaluation of intelligent controllers (e.g.,

model-based predictive controllers), FDD algorithms and devices, and intelligent agent-based

optimization methods.

15
Opportunities and Challenges

There remain several significant challenges and research opportunities to fully realizing the

program’s objective. It is essential to investigate the applicability of the FDD and intelligent agent

optimization algorithms to a wider range of building types. Prototype buildings defined by the EED can

be compared with those defined by the DOE’s Pacific Northwest National Laboratory to suggest a set of

standard baselines for buildings of different types. Initial testing and evaluation of the algorithms can be

conducted for these reference buildings, and then the sensitivity of the FDD and optimization results to

the variation of physical parameters of buildings under different climate conditions can be examined.

Such an effort would result in standard test and evaluation methods for the FDD and intelligent-agentbased optimization algorithm with quantifiable uncertainty, and this could help in the development of

more robust industrial FDD and intelligent controllers for building operation.

The testbeds for communication, FDD, and intelligent controllers provide opportunities to

generate a large number of reference data sets that can be used by industry to accelerate the development

of intelligent controllers and building systems. A systematic approach needs to be developed to manage

such a large set of databases and disseminate them to industries.

Opportunities also exist for developing simplified local controllers that are coordinated by central

intelligence residing in server(s). This is possible with advances in the “Internet of things” and cloud

computing. This approach could enable “plug and play” controllers and significantly reduce the

complexity in control system maintenance at the local level.

The current focus of the program has been at the building level/scale; at the same time, a

communication method has been developed for integration with Smart Grid. With more resources,

substantially more energy saving and environmental benefit can be obtained by extending the intelligentagent optimization approach to the neighborhood, city, and regional scales.

Overall Assessment

The Embedded Intelligence in Buildings Program has focused on the essential technical areas. It 

has built unique research facilities and modeling capabilities needed to achieve its objective of improving 

building operation through the use of intelligent information technologies for FDD and system 

optimization. Unique testbeds combining model simulation and hardware emulation of controllers have 

been established, enabling the development of test and evaluation methods for both conventional and 

intelligent FDD and controllers. The quality of the research conducted in the program is nationally and 

internationally recognized, and it is at a very high level. The results of the research have contributed 

significantly to the advancement of critical national and international standards such as ASHRAE 135 and 

EN/SIO 16484-5 for BACnet and ASHRAE 135.1 and EN/ISO 16484-6 for BACnet testing, which are 

widely used by the building control industry.

Overall Assessment

EED is among the world’s best governmental laboratories in its field, conducting research in

carefully selected areas of high-performance buildings, complemented by collaboration with universities

and other agencies.

The division designed and had constructed several state-of-the art research facilities to enable

benchmark data collection, model validation, and development of standard methods of tests. Examples are

the NZERTF, the smart building automation and control testbed, the mini breadboard heat pump, and the

intelligent agent control laboratory, whose initial construction is under way. EED used the NZERTF to

develop and demonstrate an approach to achieve net-zero energy performance in single-family residential

houses under Washington, D.C., climate conditions. Year-long data were collected on energy and IAQ

16
performance; these data will enable validation of simulation models for prediction and extrapolation to

other climates. The facility is unique in its capability to simulate the effects of occupants’ behavior and its

detailed monitoring of the environmental conditions, indoor and out.

Integrated multizone airflow and IAQ modeling and building energy simulation tools were

developed by coupling CONTAM a (multizone indoor air quality and ventilation analysis computer

program) with TRANSYS (an energy simulation software package) and by coupling CONTAM with

EnergyPlus (a whole building energy simulation program).

EED personnel were the first to propose and demonstrate the capability to optimize refrigerant

circuitry in evaporators and condensers using an artificial intelligence-based optimization module named

ISHED (Intelligent System for Heat Exchanger Design), which is embedded in EVAP-COND 3.0

software package. The EED also developed comprehensive standard reference refrigerant databases that

provide the information needed for refrigerant screening by manufacturers of refrigeration and air

conditioning systems; the first reference materials for calibration of environmental chambers for

emissions testing; and an Internet interface, BIRDS, which allows users to compare a whole building’s

energy, environment, and economic performance across a variety of factors.

Opportunities and challenges for the division include the need for collection of long-term energy

and IAQ performance data at the NZERTF; field validation of models and measurement methods that are

based on laboratory studies, such as the FDD evaluation and intelligent building agents; and development

of metrics for concurrently evaluating IAQ and energy performance.

PORTFOLIO OF SCIENTIFIC EXPERTISE

Accomplishments

The EED’s yearly research portfolio selection and review process is clearly defined and followed; 

it includes quarterly reviews of programs and continuous milestone tracking. The division has excellent 

capabilities in its current focus areas. The projects appear to be focused to take advantage of these 

excellent capabilities. Recruiting is targeted at specific areas of expertise that build on existing expertise 

or that expand to areas adjacent to existing areas of expertise. New or expanded skills include fault 

detection and diagnostics, particle imaging, velocimetry measurements, volatile organic compound 

(VOC) measurements, and photovoltaic spectral response measurement.

There is need for expertise that is not contained within the division, but it is found elsewhere in 

the EL or through contract and grant programs. A review of funding allocations for the EED showed that

significant support was received from other divisions of the EL and other laboratories at NIST. This 

approach to pulling in expertise on an as-needed basis helps to maintain consistency within division 

technical expertise. Successful recruitment of competent early-career researchers and good retention of

capable staff has led to healthy demographics, with 28 percent of scientific and engineering staff having 

joined within the past 5 years. The successful recruitment of qualified early-career researchers and good 

retention of highly capable staff has led to healthy demographics, complemented by the extension of 

capabilities through appropriate use of grants and contracts. The division exhibits a good mix of modeling 

and experimental expertise.

Opportunities and Challenges

The division needs to expand its areas of expertise, especially in the areas of combined heat, air,

moisture, and pollutant transport. This expansion will be needed because of the increasing need to control

moisture and mold in net-zero energy buildings.

17
Overall Assessment

Overall, the EED exhibits excellent capabilities in the areas of existing expertise, but the scope of

its expertise may be too narrow to include critical IAQ areas such as moisture/mold and nanoparticles.

FACILITIES, EQUIPMENT, AND HUMAN RESOURCES

Accomplishments

Consistent with its objective of developing and deploying advances in measurement science

within its program areas, the EED has built on team expertise to expand its capabilities to develop data

sets and unique facilities capable of best-in-class measurements. In some cases, EED staff were able to

leverage one-time funding opportunities—for example using American Recovery and Reinvestment Act

funds to aid in the building of the NZERTF.

The new facilities and equipment include the NZERTF, which was designed to improve and

develop test methods and performance metrics for energy-efficient and renewable energy technologies

and to examine system interactions; the mini breadboard heat pump, designed to measure the cycle

performance of new refrigerants; the Solar Cell Spectral Response Measurement Facility, designed to

measure the absolute spectral response of solar cells; the 500 mm guarded hot plate, designed to measure

the thermal conductivity of insulation materials over a temperature range of 90 K to 900 K ; the

Intelligent Agent Control Laboratory, designed to examine advanced intelligent control techniques; the

VOC emission microchambers, designed to measure VOC emissions from building materials; and the

particle image velocimetry equipment, designed to measure flow distributions of gaseous fluids.

Overall Assessment

The division has accomplished the upgrading and building of new strategically important

facilities. The EED has developed several unique and best-in-class equipment and facilities.

Communication within the division benefited from constant informal exchanges and an open-door policy

along with formal team and group meetings. A challenge for the division is to obtain sufficient manpower

and resources to fully utilize its facilities and to process and disseminate data.

DISSEMINATION OF OUTPUTS

Accomplishments

Stakeholders, customers, and collaborators are identified by the division by several means: its

leadership in professional organizations; participation in Department of Energy (DOE) and national

laboratories planning; participation in standards development activities; participation in professional and

technical meetings; initiating and/or participating in roadmapping exercises; and direct interactions with

DOE, national laboratories, universities, and industry. Stakeholders include standards organizations—for

example, the ASHRAE, the ASTM, the American Society of Mechanical Engineers (ASME), the

International Electrotechnical Commission (IEC), and the ISO—trade associations such as the Air

Conditioning Contractors of America (ACCA); the Association of Home Appliance Manufacturers

(AHAM); the Air-Conditioning, Heating, and Refrigeration Institute (AHRI); and the National Electrical

Manufacturers Association (NEMA); and building owners and operators, contractors, and home buyers.

The EED primarily disseminates research findings through publications, support of standards

development, and development and distribution of software. Significant accomplishments are evidenced

18
by the incorporation of the division’s research into and their support of the development of major industry

standards.

One example of this is the leadership shown in improved requirements in green building codes,

standards, and programs, including performance credit for improved airtightness for ASHRAE/IES 90.1,

the performance-based ventilation option included in Standard 62.2; a proposal for the ASHRAE

Residential IAQ Design Guide, the air barrier commissioning addendum included in Standard 189.1; the

proposal for ASHRAE Standard 189.2P; and Standard 189.1-2014, published for inclusion in the 2015

International Green Construction Code.

Another example is the development of intelligent systems and technology standards. EED staff

hold leadership positions in several key national and international standards committees, including

ASHRAE/IES/USGBC 189.1, ASHRAE GPC 10, ASHRAE SSPC 135 (BACnet), ASHRAE SPC 201P

Facility Smart Grid Information Model (FSGIM), ASHRAE GPC 0.2, 1.2 (Existing Building Cx) SPC

207P (Method of Test), OASIS (EMIX and Energy Interoperation), ISO/TC 205 Building Environment

Design, and IEC PC 118 Smart Grid User Interface.

The division has an excellent publication record, including 63 peer-reviewed journal articles, 46

conference papers, and 45 NIST publications since 2010.

The division has developed widely used specialty software, including CONTAM contaminant

transport in buildings (accessed through the NIST Internet site more than 4,000 times); the Climate

Suitability Tool for natural ventilation; LoopDA, for natural ventilation design (accessed through the

NIST Internet site more than 1,400 times); EVAP-COND, for the design of heat exchangers (accessed

through the NIST Internet site more than 1,200 times per year); the APAR air handling unit performance

assessment tool; the APAR-DD dual duct air handling unit performance assessment tool; the VPACC

variable air volume performance assessment control chart; the TPACC terminal unit performance

assessment control chart; the FDD-EA fault diagnostics and detection expert assistant; and the HVAC-CX

heating, ventilating, air conditioning commissioning software tool. The Applied Economics Office (AEO)

worked with the EED to develop the Building for Environmental and Economic Sustainability (BEES)

assessment of sustainable building materials, which has more than 24,000 users in over 80 countries and

is accessed through the NIST Internet site more than 5,000 times per month, and the BIRDS tool for

assessment of sustainable buildings.

Opportunities and Challenges

The current output dissemination methods are well suited to the HVAC&R and commercial

building sector, but they may not be as effective when the research area broadens to include the

residential sector. New methods need to be developed, including more systematic dissemination to

targeted audiences such as HVAC&R industry technical leaders (more frequent updates relating to

refrigerant and vapor compression system advances are needed); the architectural and design community;

home energy raters and energy auditors, who have an ascending influence in the construction industry;

and university students, by creating, for example, data sets for virtual laboratories.

The division has significant impact through standards, publications, and software distribution, but

informal dissemination methods and use of social media still need to be improved to affect a wider

audience.

RECOMMENDATIONS

Recommendation: The Energy and Environment Division should continue to develop

metrics for concurrently evaluating indoor air quality (IAQ) and energy performance.

19
Recommendation: The Energy and Environment Division should expand its areas of

expertise, especially in combined heat, air, moisture, and pollutant transport, to support

concurrent indoor air quality and energy analysis.

Recommendation: The Energy and Environment Division should perform field

investigations to collect reliable data for the validation of models and measurement methods

developed from laboratory studies.

Recommendation: The Energy and Environment Division should ensure sufficient

manpower and resources to fully utilize its state-of-the-art research and demonstration

facilities and should broadly disseminate the data collected at those facilities.

Recommendation: The Energy and Environment Division should evaluate the adequacy of

its dissemination methods when residential sector research becomes more prevalent. The

Energy and Environment Division should more systematically disseminate its research

findings and databases to targeted audiences, including heating, ventilation, air

conditioning, and refrigeration (HVAC&R) industry technical leaders, the architectural

and design community, home energy raters, energy auditors, and university students.

20
3

Fire Research Division

The Fire Research Division (FRD) develops, verifies, and utilizes measurements and predictive 

methods to quantify the behavior of fire and advances techniques to reduce the impact of fire on people, 

property, and the environment. The FRD strives to provide leadership for advancing the theory and 

practice of fire safety engineering, fire-fighting, fire investigation, fire testing, fire data management, and 

intentional burning.

1

The FRD consists of five staff groups—Fire Fighting Technology, Engineered Fire Safety, 

Flammability Reduction, Wildland-Urban Interface Fire, and the National Fire Research Laboratory 

(NFRL). The groups work in two main program areas: fire risk reduction in communities (with thrusts in 

fire service and wildland-urban interface) and fire risk reduction in buildings (with thrusts in residential 

safety and performance-based design).

TECHNICAL PROGRAMS

Fire Risk Reduction in Communities

Fire Service Thrust

The fire service thrust focuses on the development and communication of information to educate 

firefighters on safe and effective means to fight fires. Efforts include developing safer personal protective 

equipment, providing data for new codes and standards, supporting creation of new training materials and 

fire-fighting tactics, and communicating related information. 

Accomplishments

The FRD has disseminated information on fire dynamics that can help protect firefighters. 

Through interaction with firefighter groups, the division has provided information on the fundamental 

physics of fire phenomena that has been incorporated into both basic and higher training levels of 

firefighter training. 

Opportunities and Challenges

Significant work is still needed to change fire-fighting tactics to reduce deaths and injuries to 

firefighters during structural fire-fighting operations. This will require incorporating new understanding 

into science-based training documents and fire departments’ standard operating procedures. The NFRL 

and the FDS can help provide needed information. The FDS is a computational fluid dynamics (CFD) 

modeling tool for simulating details of fire growth and heat release rates. By coordinating use of these 

1 National Institute of Standards and Technology, “The Engineering Laboratory—Summaries of Our Activities, 

Accomplishments and Recognitions,” Gaithersburg, Md., July 2014.

21

 
two capabilities, there will be opportunities to conduct full-scale experiments while examining the 

limitations of ventilation openings and their impact on heat release rates. 

An ongoing challenge for the FRD is to maintain and expand its communication of research 

findings to firefighters and fire-fighting organizations in a manner that effectively overcomes knowledge 

limitations pertinent to the dynamics of fire and relevant to the training of firefighters. The imposed travel 

limitations continue to make every experimental field deployment and effective interactions with 

stakeholders a significant challenge. 

The general lack of facilities to conduct large-scale fire research has hampered research efforts, 

and even with the new NFRL, there are concerns about the long waiting list of projects, staffing, and 

funding.

Overall Assessment

The fire service thrust of the FRD has made progress toward the objective of improving the safety 

and effectiveness of firefighters through measurement science to advance suppression tactics, examine 

nontraditional means of fire suppression, and transfer the results to the fire service. The work has been 

accomplished despite the fact that the NFRL has been under renovation and expansion since 2010, 

limiting the use of the facility in 2010 and 2011 and severely curtailing most research from 2012 to 2014. 

Field experiments have been accomplished through outside funding opportunities, collaboration, and 

partnerships. 

The efforts to communicate research findings on fire dynamics have generated a great deal of 

interest and discussion within the professional fire service community. Presentations, articles in the fire 

service print, and online media provide opportunities for fire service organizational involvement, and the 

use of social media has recently been explored. 

Wildland-Urban Interface Fires Thrust

The WUI problem is growing at a very fast rate and may grow even faster as a result of climatic 

changes and an increase in the proximity between wildland and residential homes. Because the increasing 

frequency of this problem is relatively recent, there is not much information about its characteristics or 

the means to mitigate its consequences. Therefore, it is of high priority to collect data from WUI fires to 

underpin standards and codes for fire-hardening structures located in WUI-prone-areas. 

Accomplishments

The FRD has been addressing the WUI fire problem since 2007. Field studies, which included 

several large WUI fires (the 2007 Witch fire in California, the 2011 Amarillo fire in Texas, and the 2012 

Waldo fire in Colorado) resulted in reports describing the impact of the wildfires on communities, with an 

emphasis on structures. The reports include information on the development of WUI fires, including the 

role of fire spotting by embers and the resulting propagation of the fire. Case studies to date suggest that 

ignition by transported embers may account for 50 percent of the structural ignitions in a WUI fire. 

Recognizing this from field studies, FRD laboratory studies have concentrated primarily on the ignition of 

structures by showers of embers. This issue, the ignition of surfaces by embers, has not been well studied, 

and a FRD project is introducing new approaches and technologies to study this issue. For example, the

FRD has developed a tool (Dragon) that produces embers in a controlled fashion; it has facilitated the 

study of ember ignition of structures. Different versions of the Dragon have been developed at NIST and 

at other laboratories in the United States and abroad. 

Significant results of the project include those from studies on the ignition of roofs by embers. 

These studies involved observing the importance of penetration of embers into attics through air vents, 

and the ignition of external surfaces by accumulation of embers in corners and crevices. Identification of 

22
these primary causes of fire propagation has led to specific recommendations for changes in vent design. 

The information obtained in the project is being disseminated effectively and has already resulted in the 

implementation of improvements to the construction of air vent design of structures. 

Opportunities and Challenges

The FRD has acquired extensive experience and knowledge by studying indoor fires in structures, 

from prevention to suppression. This experience provides a unique opportunity to develop approaches for 

reducing the building damage from WUI fires. The NFRL could address issues related to the WUI 

problem, particularly with respect to the hardening of external surfaces of structures to prevent ignition by 

radiant and convective heating from flames, and ignition by embers, or combined action from these two 

sources. Wind simulation capabilities may be adapted by NFRL for that purpose. 

The lack of a wind tunnel capable of real-scale fire experiments at NIST limits the study of ember 

ignition to idealized situations. This problem is being addressed by conducting the experiments in a wind 

tunnel in Japan, although this reduces the productivity of the study. Modeling the transportation of embers 

and their ignition capability would create predictive models that include risk mitigation. Future research 

could consider the increased propensity for ignition when a surface is exposed to heat flux or when

embers are deposited on preheated surfaces. 

A reasonable, though modest, effort has been conducted in computer modeling of WUI fires 

primarily by applying the FDS software. However, since the WUI problem has characteristics that are 

different from those of indoor fires, modifications may be needed in the FDS for its application to WUI 

fires. Given its expertise, the FRD is well-qualified to extend FDS to WUI fires. Collaboration with other 

services, such as the U.S. Forest Service, will be helpful in calibrating source terms of heat and embers 

produced by the vegetation in various combustion conditions.

Unfortunately, the number of studies of recent wildland fires has been reduced due to restrictions 

in travel funding and limited personnel resources; and so only a few such cases have been examined 

carefully. This is a missed opportunity to develop and share needed information that would help in further 

understanding of the WUI fire problem, protecting citizens, and optimizing the action of firefighters at the 

interface. This information would be very useful for the development of WUI models and the 

standardization of structures resistant to external fires.

Overall Assessment

The FRD has begun to have an impact on the WUI fire problem, but there remains much more 

work, including the creation of standard methods for studying the problem. To make a stronger impact in 

this rapidly growing disaster arena, the FRD will need additional mobile data acquisition equipment and 

flexible staff to collect data on real fire scenarios, as well as new wind facilities at the NFRL, to validate 

the findings of existing, yet limited, small-scale laboratory studies.

Fire Risk Reduction in Buildings

Residential Safety Thrust

The governing objective of this thrust is to reduce deaths, injuries, and property losses in 

residences through application of measurement science, with recent focus on cigarettes, mattresses, 

smoke alarms, and upholstered furniture. The main topics are evaluation of innovative processes, 

technologies, and materials to improve the fire performance of materials and other products; smoke 

detection through innovative systems and improvement of smoke alarms in dwellings; and guidance on 

upholstered furniture.

23
Accomplishments

In general, the teams have very high levels of competence, and the individuals are internationally

recognized scientists in their areas of expertise. Participation of the teams in the development of 

standardization and codes at the national level is adequate. 

The FRD has conducted valuable and timely research on the effectiveness and possible toxicity of 

flame retardants for upholstered furniture.

The smoke toxicity project has been completed, but its implications for performance-based 

designs have not yet been determined. Research on smoke detection is adequate, and the FRD has made 

good contributions to standardized testing procedures.

The work on development of standards for reducing the propensity of cigarettes to ignite other 

materials has had worldwide impact and has been incorporated in codes and regulations for all U.S. states, 

in Europe, and in many other countries. The FRD recently proposed a modification to the ASTM standard 

for measuring the ignition propensity of cigarettes.

The FRD team’s knowledge of uncertainties calculation and traceability through metrology is 

very useful, and it is important that this knowledge be disseminated to the fire science community.

Opportunities and Challenges

The selection of projects is well performed, but the number of projects jointly performed with 

other divisions could be increased; it is likely that this would increase their impact. For example, studies 

on chemical health and safety could be performed collaboratively with studies on fire behavior of 

upholstered furniture.

The team has the facilities that could be used to develop needed scaling procedures for 

experiments and modeling of fires. The group would benefit from and have greater impact through 

increased coordination with the international community. 

The staff is highly competent, but some (e.g., guest researchers and postdoctoral researchers) are 

not permanent, and others may be approaching retirement. The retention and/or expansion of key 

competences require management attention.

The FRD’s work on upholstered furniture in U.S. dwellings could benefit from comparing it with 

European research and standards in railway transportation furniture.

Overall Assessment

The work on fire risk reduction is performed at a very good overall scientific level, and the topics 

are well developed. Some improvements are needed in dissemination and resource management, and in 

travel funding.

Performance-Based Design Thrust 

The broad goals for the performance-based design thrust include developing and validating

modeling tools to predict the evolution of fire and ensuing hazards and the development of performancebased methods to predict and evaluate fire behavior of steel and concrete structures. The latter includes 

delivery of validated and improved tools, guidance, and draft standards for the fire resistance design and 

assessment of structures.

The FRD has led the field in the creation of widely used tools for modeling the growth and 

evolution of fires. Two codes are available, the FDS and the Consolidated Model of Fire and Smoke 

Transport (CFAST). The CFAST is a two-zone fire model that is used to estimate the combustion 

products in a building. CFAST was originally developed in the 1980s and is still used today by fire 

protection engineers for buildings with relatively simple geometries because of its rapid computing times. 

24
In particular, CFAST has been verified and validated by the U.S Nuclear Regulatory Commission and the 

DOE for use by the nuclear utilities and DOE safety professionals. 

Accomplishments

The NIST team continues to update the capabilities of the FDS and so continues to enjoy its 

position as the primary reference for fire simulation methods. Interactions with FMGlobal in their 

development of the FireFOAM modeling software will continue to expand opportunities for advancement 

of the FDS tool. Citations of FDS exceeded 1,000 in 2013. NIST has released a new version, FDS 6, 

which is more stable and has better parallel processing capabilities and better physics-based submodels 

describing fire and radiation phenomena. 

In the structures area, EL staff have published best practices guidelines for structural fire 

resistance design; created a model for mechanical properties of steel at elevated temperatures (including 

material strengths, methods for analysis, and information on load combinations and required strengths) 

that has been accepted for balloting by an applicable American Institute of Steel Construction (AISC) 

committee; created an integrated tool that transfers fire modeling results to thermal analysis tools and then 

enables structural analyses; created a temperature-dependent material modeling approach for finite 

element models that include element erosion and empirical data on plastic-strain-based failures; 

developed an understanding of the behavior and failure modes of steel shear; and written an appendix of 

recommended procedures for performance-based design of structures for fire effects, with 10 of 12 

sections successfully balloted (as of July 2014) for inclusion into ASCE standards.

Opportunities and Challenges

The FDS modeling team has developed new stretch goals. The major goal is to couple three types 

of complex codes—gas-phase fluid dynamics and reaction, thermal transport to and through solid 

structures, and failure of solid structures under load and thermal stress. There are few examples of such 

undertakings in the literature. This is a significant undertaking that will require efforts in computation, 

extensive validation, and interpretation of results. Increased access to computational facilities and 

experimental data for validation will also be required. 

This new capability would enable fire-structure interactions, treatment of immersed boundaries, 

adaptive mesh refinement, and better physics-based, fully coupled pyrolysis models to compute online 

heat release rates from a variety of burning materials.

There are no science-based measurement procedures for evaluating the performance of an entire 

building structure under load or for realistic fire scenarios. This lack of information represents an 

opportunity for the FRD to make important contributions to this field. 

Overall Assessment

The availability of open source access to FDS with documentation that is reasonably easy to use

has been a valuable contribution to the community. Development of new FDS capabilities needs to be 

continued, with expected requirements for increased computer resources and staff expertise. Updates to 

CFAST are needed; it is a useful tool but is becoming outdated.

The FRD is beginning to have an impact on areas relating to steel properties under stress and 

failure mechanisms. However, much remains to be explored. This area will be evolving over the next 

several years as new data and understanding are acquired from the NFRL, with expectations for providing 

the underlying data for new codes, standards, and industry-accepted material property models. 

25
National Fire Research Laboratory

The near-term objectives of this effort are to complete the construction and commissioning of the 

NFRL and to develop plans for the early test series for the new laboratory.

Accomplishments

The NFRL is anticipated to be an exciting new facility for the FRD and the EL. Much progress 

has been made in its preparation for testing. It will provide a unique capability to assess the impact of fire 

heating of structures under structural load. By designing and constructing this facility, the FRD team has 

broken new ground. With foresight and assistance from external experts, capabilities, metering systems, 

and roadmaps for the program have been developed. The multi-staged commissioning plan ensures that 

the system well be calibrated and ready for use in the near future.

Opportunities and Challenges

The ability to test stress on structures under realistic gravity loads while exposed to fires is an 

exciting new capability. The community expects that findings from these tests will define new (fire-safe) 

building requirements.

Just as fire phenomena and related damage do not scale with fire size, neither will the planning 

nor the resource requirements. The team has relatively little experience with such long-term planning. A 

commissioning plan is established, but the team needs to refine specific plans for the series of tests for the 

year or two following commissioning. The team has been concentrating on finishing construction and

taking delivery of the completed facility. Construction and commissioning obstacles that continue to 

delay the date on which fire tests can be initiated need to be overcome. 

Full utilization and interpretation of data will partly depend on the availability of the fully 

coupled FDS code. Efforts on the evolution of the NFRL and the new FDS capabilities need to be closely 

coordinated.

Overall Assessment

The NFRL team has done an excellent job in designing and preparing to create a new fire test 

facility that will be unmatched worldwide. This is a very significant accomplishment. The FRD needs to 

continue its efforts and to work with the contractors to complete delivery so that the commissioning 

process can start. Because of the long-term planning requirements for this large facility, it may be prudent 

to hold a planning session to review its capabilities and specific goals for testing. Such an activity would 

help to prioritize tests to be completed early and to identify resource needs and possible partners in 

executing or funding the work. It is important to consider international participation.

Overall Assessment

Over the past several years, the FRD has had a significant impact on science, codes and 

standards, tools, and firefighter training and tactics. A focus of its efforts is on measurements of firerelated phenomena and identification of controlling processes. This information in turn has been used by 

industry and practical engineers to create new and revised codes that are based upon FRD-developed 

science and solid engineering principles. The FRD team participates in or contributes to approximately 30 

different committees that develop fire codes and standards for a broad range of applications. 

26
The FRD has been using its expertise to teach fire phenomena and their implications to 

firefighters. Staff have translated their scientific understandings into simple engineering principles and 

clear, practical language. These understandings have been instrumental in developing new fire-fighting 

training materials (with specific reference to NIST for guidance), which are now helping to provide safer 

and more effective strategies and tactics for engaging a fire. 

The FRD team has completed and released a new version of the FDS modeling tool, which has 

been widely adopted by the engineering community. They provide full documentation and responsive 

user support. The modeling tool is now in use regularly for performance-based design, for design and 

development of more effective fire protection and fire-fighting products, and for use in litigation and 

incident investigation. Papers documenting prior versions of this tool have been cited more than 1,000 

times.

The division has evaluated failure mechanisms for the facepiece lens of a self-contained breathing 

apparatus (SCBA) used by firefighters to enable close encounters with fire under extreme conditions such 

as high temperatures, insufficient oxygen, and noxious gases. Multiple failures previously resulted in 

firefighter deaths. Through a careful evaluation of the thermal environment, infrared energy absorption, 

and materials, the FRD has assisted in the development of new technologies to replace these lenses, which 

are now being implemented throughout the country.

Frequency of WUI fires has been rising nearly exponentially over the past 20 years, caused by 

increased building of homes and communities in wildland areas. The fire phenomena and threats posed by 

such events are drastically different from those posed by interior building fires, due to fast moving flames 

of WUI fires, the lack of exterior fire protection, and the jumping of flames from the leading fire line due 

to ember transport. Approximately 50 percent of the home fires occurring during WUI fires are caused by 

firebrands. New standards and procedures for protection of homes are required, including regulation of 

materials and better methods for fire protection and fire-fighting from external sources.

Firebrands or embers, produced in wildland fires near urban areas, jump the fire line and can be 

transported thousands of feet to start fires in homes well removed from the main fire. As a step toward 

understanding and then minimizing the impact of such transported embers, the FRD has developed a 

standardized process called Dragon to create such firebrands so that experiments can be conducted in a 

controlled, repeatable manner. Dragon is now being utilized in the United States and in several places 

around the world.

Burning upholstered furniture accounts for a large fraction of the initial heat release in home fires. 

The FRD has helped to develop new technologies and standards in this area, but continued research is 

needed. 

The impact of new data from the NFRL will be enhanced significantly when they are coupled 

with the FDS fluid/fire modeling capability for structures under load. Much data is expected to be 

collected, and interpretation of the data and controlling processes will be facilitated by developing and 

utilizing advanced modeling techniques. Creation of such tools is under way at the FRD and with 

partners, but significant work will still be needed to validate and to learn from such tools.

The technical program developed and executed by the FRD has had great impact, in that its

measurements have supported the creation of new standards and codes based on fundamental science and 

engineering principles, provided the foundation for creation of new fire-fighting training materials and 

tactics, developed technologies for life safety equipment, created and distributed widely used fire 

modeling tools, and invented a standard procedure for generating firebrands. 

PORTFOLIO OF SCIENTIFIC EXPERTISE

Accomplishments

The FRD covers a wide range of expertise in fire science, from combustion dynamics to 

firefighter tactics that include competence in experimentation, modeling, and computational methods. The 

27
scientific and engineering knowledge residing within this division covers materials and products for 

building interiors (upholstered furniture, smoke alarms) and construction. The staff are well aware of the 

need to simplify and extend science and engineering principles to practical applications.

Serving as a bridge between fire science and fire-fighting communities, FRD staff have translated 

scientifically based content into training materials for firefighters and specifications for their equipment.

FRD staff continue to play a major role in the creation of new and significant capabilities for the 

modeling of fires. Their efforts include the two-zone fire model (CFAST) and the FDS, both of which are 

widely used and are considered primary references for fire modeling. The development team includes 

mathematicians and fire scientists and benefits from the experience of a wide community of users around 

the world. The management by FDS developers with international feedback contributes to the success of 

the FDS. 

The expertise in model evaluation and uncertainty calculation is excellent. The team excels in the 

assessment of uncertainties, which is critical in the use of numerical schemes and for the evaluation of 

validation data sets. The continued support of the Statistical Engineering Division of the NIST 

Information Technology Laboratory is essential for enabling such capabilities.

The division’s scientific expertise in development of test methods, standards and codes, and 

reference materials is impressive and covers all aspects of fire. The level of scientific expertise associated 

with work performed on fire toxicity and on evacuation is very high. This strong expertise is needed to 

allow a proper introduction of these elements to physics-based design.

The understanding of WUI fire phenomena has been increasing over the past few years. Analyses 

of wildland fires have been completed, a risk scale has been created, and a new standardized test method 

for generating firebrands has been developed. However, further efforts are needed on WUI modeling. 

The scientific expertise in structural assessment in fires is developing but needs further work if 

the FRD is to be fully established as a world leader in this area. The completion and utilization of the 

NFRL and new improvements to FDS will enhance the experimental and computational aspects, 

respectively.

Growth of scientific expertise at the FRD, through interactions with faculty and students, is 

enhanced by university collaboration. A critical part of the FRD strategy for managing expertise is to 

bring in fresh ideas from the outside. Guest researchers and postdoctoral researchers are a source of new 

ideas and competences, and they are an excellent potential source of future staff. 

Scientific management includes the organization of regular seminars and presentations from 

visiting scientists. This approach helps to keep staff informed on advances to the state of the art and 

changing needs. 

There continues to be significant collaboration between the FRD and other NIST divisions (e.g., 

the Material and Structural Systems Division within the EL and the NIST Material Measurement 

Laboratory, Physical Measurement Laboratory, and Information Technology Laboratory) as well as with 

academia, codes and standards organizations, government agencies, the construction industry, 

manufacturers, the structural and engineering design community, testing laboratories, and the fire service. 

Some examples of organizations involved in these research partnerships include the Consumer Product 

Safety Commission (CPSC), the Federal Aviation Administration (FAA), the Government Services 

Administration (GSA), the National Institute of Occupational Safety and Health (NIOSH), the 

Department of Health and Human Services (DHHS), the National Aeronautics and Space Administration 

(NASA), the NFPA, the Federal Emergency Management Agency (FEMA), the U.S. Fire Administration

(USFA), Underwriters Laboratories (UL), the U.S. Forest Service (USFS), the Society of Fire Protection 

Engineers (SFPE), Boeing, U.S. Gypsum, DuPont, and the fire departments of New York City, Los 

Angeles County, Chicago, San Diego, Colorado Springs, and many others. The Fire Grants and 

Cooperative Agreements program continues to support research with seven projects ongoing in 2014, 

although support has been significantly diminished relative to historic levels.

28
Opportunities and Challenges

The FRD competences cover all aspects of fire science but focus on fires in buildings. This focus 

is understandable. However, the division would benefit from expanding the field of application to areas 

such as industry, transportation, or wildlands. This would provide a more global overview and offer 

increased opportunities for collaboration (including with other federal agencies) and information sharing;

it might also contribute to the development of solutions to FRD’s building-focused concerns.

The NIST competence in structural fire behavior has been developing recently, mainly as a 

consequence of the reconstruction efforts in identifying root causes for the collapse of the World Trade 

Center. This expertise will increase in future years in parallel with the development and utilization of the 

NFRL. One step toward enlisting international peers in this learning process was the NIST-CIB 

Roadmapping Workshop in May 2014. The workshop results were used by the EL to inform a set of 

proposals for new activities at the NFRL. Through these developments, the competences of the team for 

assessing non-linear structural properties of construction materials and assemblies will rapidly grow. The 

team decided to start with steel, which is the best known material in terms of thermal and mechanical 

properties and the methods to reduce the uncertainties in testing and modeling connections between steel 

beams. Concrete and timber are more challenging materials for future study, because of spalling and 

pyrolysis, respectively. The scientific expertise may be adapted according to these future developments 

and the increased use of the NFRL. To support this development, competence in advanced thermomechanical modeling may need to be increased in parallel with the associated experimental competence.

The mapping and management of key competences could also be improved. There is only a 

partially formalized identification of existing competences, and there is an efficient management of

resources only in the short term. Better definition of medium- and long-term competences is needed. 

Maintenance of competences is vulnerable to the retirement or departures for other reasons of experts. In 

some cases, critical technologies have only one technical staff member at the senior expert level. A 

system with at least two senior experts per key competence, plus at least one junior expert, would limit 

the risk. This mapping of competences is an essential tool for mitigating risk relative to the adequacy of 

scientific competences. Mapping of participation to key scientific and engineering communities (e.g., 

membership in organizations such as the International Association for Fire Safety Science [IAFSS] or the 

SFPE) is also not formalized.

Increased collaboration with the USFS would enhance understanding of the variables affecting 

WUI fire impact on communities, such as vegetation, topography, and weather-related parameters (e.g., 

temperature, humidity, and wind).

Using guest and postdoctoral researchers is an approach well adapted to filling gaps and 

providing fresh perspectives and also to modulating fluctuations in funding. Nevertheless, the FRD needs 

to be able to assimilate the knowledge of such researchers before their departure, because unless 

permanently hired they are not sustainable resources.

Overall Assessment

The level of scientific competence is excellent, with risks associated with limited resources (e.g., 

number of experts per key competence). A review of existing competences, specifically for medium- and 

long term-needs, could be improved and would support full and effective utilization of the new NFRL.

29
FACILITIES, EQUIPMENT, AND HUMAN RESOURCES

Accomplishments

The FRD has a long history of fire research in well-established facilities. The facilities are 

composed of laboratories and large-scale testing facilities. Although some of the laboratories are showing 

signs of age and there are variations among the different laboratories, overall the experimental apparatus 

and equipment in these laboratories appear adequate. 

The computer facilities of the FRD appear adequate for modeling development. However,

validation of FDS for large and WUI fires as well as structural load evaluations will likely require access 

to larger computing systems.

The new robot testing center for evaluating and setting standards for the robots used in searchand-recovery missions is a good first step in this area of research. Testing the robots in actual fire-fighting 

testing facilities would be valuable.

The large-scale testing facilities have been one of the strengths of the FRD because they have 

permitted the testing of fires up to 10 MW. Testing of large-scale fires is important because properties of 

fire and its impact do not scale linearly with fire size. In this regard the previous facilities were limited in 

their applicability because they were not capable of handling relatively large fires. The expanded NFRL 

provides unique measurement capabilities to study larger fires (up to 20 MW continuously, 30 to 40 MW 

peak) and their impact on the performance of relatively large structures and surfaces under structural load. 

Together, the new facility and the legacy facility represent versatile capability not replicated anywhere. It 

will allow testing of different size fires and consequently help address the issue of fire scaling and the 

impacts of fire on structures under load. 

Opportunities and Challenges

Long-term planning for program and project goals and resources for NFRL activities will be 

important to ensure that all portions of the program are balanced and consistent with division goals.

For WUI, there is a need for additional human and equipment resources. Postfire data collection 

and analysis in actual fires is an important component in the study of WUI, because conditions cannot be 

duplicated in a laboratory. The FRD has been very successful studying a few large WUI fires, but limited 

resources have curtailed the study of recent fires. There is a need to allocate resources and to expand the 

mobile measurement capability for field studies. Laboratory testing is primarily limited to the study of 

ignition of building components by embers. Although this work is unique and has great impact, the lack 

of wind facilities and the need to conduct testing in Japan limits the productivity of the project.

Overall Assessment

NIST is developing some exciting new facilities in the NFRL. There are large expectations for 

this new facility with its unmatched capabilities. Other laboratories and equipment elsewhere within the 

facility appear adequate. Construction of a fire test facility with cross-wind capability needs to be 

considered. 

30
DISSEMINATION OF OUTPUTS

Accomplishments

The FRD has published more than 240 papers since 2010, including scientific peer-reviewed 

research, conference presentations, and NIST technical notes and special publications. The publications 

address topics such as fire behavior in structures, simulation and modeling, and evaluation criteria. There 

have been thousands of citations and use of these papers during the same time period. 

The FDS, now in version 6, has more than 1,500 documented users for this open source program. 

It is reported that there have been significant reductions of fire protection costs through the use of the

FDS and CFAST models. 

In the area of fire and smoke detection, the FRD has provided information that has helped 

determine the technical basis for smoke and fire detector metrics and end use, resulting in changes to 

codes and standards to specify smoke detector location in homes and to require nuisance metrics and a 

methodology for measuring nuisance resistance. 

Work in flame-resistant technologies has enabled a regulation that pertains to smoldering ignition 

of furniture and a fire-resistant coating for foam that reduces the fire hazard of furniture beyond any other 

commercial solution. The division is providing technical guidance to the CPSC on testing methodologies 

and performance metrics for barrier fabrics.

The FRD continues to participate in standards committees to incorporate recommendations from 

investigations and research. The NFPA, ASTM, ISO, UL, and the International Code Council (ICC) are 

examples of this involvement across a broad spectrum of national and international code committees. 

The FRD-developed Dragon for firebrand generation has been disseminated and adopted 

worldwide for use in WUI research. 

The research in partnership with NIOSH, NFPA, and fire service organizations resulted in the 

redesign of the standard on Open-Circuit Self-Contained Breathing Apparatus (SCBA) facepiece and new 

testing requirements defined in the NFPA SCBA for Emergency Services.

Fire dynamics information has been included in the International Fire Service Training 

Association (IFSTA) Essentials of Fire Fighting, and a fire dynamics addendum is being published for 

Fundamentals of Fire Fighter Skills. These books are the primary documents used in training entry-level 

firefighters in the United States. 

Changes resulting from FRD’s fire dynamics research have been incorporated into the NFPA 

Standard on Live Fire Training Evolutions, Standard on Fire Service Respiratory Protection Training, 

Standard on Thermal Imaging Training, and Standard on Training for Initial Emergency Scene 

Operations. 

Based on research conducted by the FRD and the UL, revisions to practical fire training materials 

have been developed. For example, the Fire Department of New York City (FDNY) issued new 

procedures on ventilation, private dwellings, brownstones, and wind-impacted fires, and the Los Angeles 

County Fire Department has developed a series of training videos. 

The FRD has also established a social media Twitter account (@NIST_Fire) to disseminate 

information to firefighters. 

Presentations on fire dynamics to fire service organizations and conferences have ranged in class 

size from 100 to 2,500; 36 presentations were conducted in 2013 and 2014.

The following have been disseminated electronically in collaboration with stakeholders and 

partners: a “Science in the Big Room” Webinar during a Fire Department Instructors Conference; a DVD 

on fire dynamics; a Webinar entitled “The Changing Severity of Home Fires: Research to Empower a Fire 

Service Response” for the International Association of Fire Chiefs (IAFC); an Internet portal containing 

information on firefighter safety through advanced research (FSTAR); a video entitled “Time to Survival”

for the International Association of Fire Fighters (IAFF); an Internet fire ground survival course; an 

Internet course on single family dwelling fires with the FDNY and the International Society of Fire 

Service Instructors; and an Internet course on scientific research for the development of more effective 

31
tactics, for the UL Firefighter Safety Research Institute; and an Illinois Fire Service Institute (IFSI) video 

on the principles of modern fire attack—Size Up, Locate the Fire, Isolate the Flow Path, Cool from a Safe 

Distance, Extinguish, and then Rescue and Salvage (SLICE-RS). 

Opportunities and Challenges

The NIST website and use of social media provide an opportunity for outreach and dissemination 

of research, with a particular emphasis on reaching firefighters on the topics of fire dynamics and tactics. 

There are opportunities for other groups within the FRD to utilize social media as well. This has been 

challenging due to Department of Commerce constraints on the use of social media and the required 

shutdown of direct access to the www.fire.gov website, which has been incorporated into the NIST 

website, www.nist.gov/fire. The absence of a forwarding link from the fire.gov website has significantly

reduced in the number of times the site has been accessed. This situation needs to be corrected.

NIST needs to continue to expand the use of social media as a form of outreach. The Twitter 

account @NIST_Fire is an effective tool, but use of social media could be expanded to include sites on 

Facebook and YouTube. All of the NIST videos could be readily available online and through social 

media sites. 

Overall Assessment

The FRD has engaged effectively with standards and codes committees at the national level, but 

there has been limited involvement at the international level due to funding limitations. There are a good 

number of external peer-reviewed publications and NIST reports. Metrics on the quality of the 

publications were not available, and publications were not characterized by technology areas. The 

division has provided excellent communication to fire-fighting communities through presentations, 

conferences, and partnerships with national organizations and fire departments, with resulting impacts on 

training and tactics. NIST is constrained by Department of Commerce rules on the dissemination of 

information utilizing social media. There has been excellent dissemination of and support for the opensource FDS, resulting in its widespread use nationally and internationally. Travel restrictions have 

constrained the division’s ability to disseminate its work.

FINDINGS AND RECOMMENDATIONS

The efforts of the FRD staff to identify critically important research topics guided by the use of 

the division’s roadmap and project planning process to ensure maximal impact of its resources are 

commendable. The division has prioritized research goals to meet the measurement science, standards, 

and technology needs of the U.S. building and fire safety communities. 

The impact of the technical program executed by the FRD has been significant. The division has

made measurements to support the creation of new standards and codes, provided the foundation for new 

fire-fighting training materials and tactics, developed technologies for life safety equipment (e.g., the 

SCBA lens), developed new standards for flammability or smoldering of materials and for fire and smoke 

detection, created and distributed fire modeling tools, created new models for material properties at 

elevated temperatures and recommendations for designing new buildings, and invented a standard 

procedure (Dragon) for generating firebrands. 

The scientific expertise in development of test methods, toxicity, evacuation procedures, 

standards and codes, and reference materials is impressive and covers all aspects of fire. The scientific 

and stakeholder knowledge residing within this division covers materials and products for the interior 

(upholstered furniture and smoke alarms), building construction, and fire-fighting needs.

32
There continues to be strong participation in various standards committees to support

incorporation of recommendations from investigations and research. Organizations such as the NFPA, 

ASTM, ISO, UL, and the ICC are examples of this involvement across a broad spectrum of national and 

international code committees, although interactions with international partners appear to have weakened 

in recent years, mainly owing to travel restrictions. It is important that the FRD continue executing a wellplanned, diverse program to support development of and delivery to the United States of new standards 

for the protection of life and property.

Recommendation: The Fire Research Division should review its long-term objectives and 

strategies to identify competences that will be needed in the future. The Fire Research 

Division should give special attention to advanced thermomechanical modeling and 

associated experiments, real-time data acquisition in the field at wildland-urban interface

fire events, communication through social media, and computational modeling for new Fire 

Dynamic Simulator goals.

Laboratories and equipment within the legacy portion of the FRD appear adequate. However, in 

some cases, the equipment is old and nearly outdated. Despite this constraint, the FRD staff have done a 

good job of producing usable results with the equipment they have. During the next year or two, the FRD 

needs to review its equipment needs and update hardware where appropriate. The division has done an 

excellent job of designing and creating a new fire test facility, the NFRL. There are large expectations for 

this new facility with its capabilities that are not found elsewhere. By allowing the testing of different size 

fires, the NFRL will help address fire scaling and the impacts of fire on structures under load. The FRD 

has assembled external expertise to help identify needed capabilities and metering systems and create 

roadmaps for the NFRL. In addition, the division has developed a multistaged commissioning plan to 

ensure that the NFRL system is well calibrated and ready for use.

While the commissioning plan has been developed, the team needs to start soon on the creation of 

specific plans for the series of tests for the first year or two following commissioning. Such a planning 

activity is likely to result not only in prioritizing tests to be completed early but also in identifying

resource needs and possible partners in executing or funding the work. It would be beneficial to consider 

international participation. Long-term planning for program and project goals and resources for NFRL 

activities will be important to ensure that other portions of the program are not jeopardized. The NFRL 

can provide a focus for coordination of many of the activities of the FRD. 

Recommendation: During the next year or two, the Fire Research Division should review 

their equipment needs in the legacy portion of their laboratories and update hardware 

where appropriate.

Recommendation: The Fire Research Division should complete and commission the new 

National Fire Research Laboratory (NFRL) facility as planned and should work to avoid 

further slippage of the completion schedule. The Fire Research Division should initiate a 

planning session to prioritize the test series in the NFRL for the first 2 years following its 

commissioning, to include resource needs, identification of key test objectives, and possible 

national and international partners.

The FRD staff are well aware of the need to simplify and extend science and engineering 

principles to practical applications and execute such tasks competently and professionally. For example, 

new fire-fighting training materials refer specifically to NIST for guidance and are enabling more 

effective strategies and tactics for engaging a fire. The interest generated by the FRD for new information 

and guidelines needs to be maintained. Continued outreach through all forms of communication and 

education will be critical. The FRD would benefit by expanding its use of social media as a form of 

outreach. 

33
Recommendation: The Fire Research Division should maintain the momentum generated 

by the Fire Fighting Technology Group, with increased interactions with firefighter groups

nationwide and should increase its use of social media for broad communication.

The FRD has started recently to address the WUI problem with field and laboratory studies and 

computer modeling. Although the studies conducted are excellent, limited resources have restricted their

potential impact on the WUI community. Significant results have been produced by studies on the 

ignition of roofs by embers, the penetration of embers into the attics through the air vents, and the ignition 

of external surfaces by accumulation of embers in corners and crevices. 

The FRD needs to strengthen research in the WUI fire arena. New standards and procedures for 

protection of homes are required, including regulation of materials and better methods for fire protection 

and fighting fires from external sources. Because test conditions cannot be readily replicated in the 

laboratory, there is a need to put additional human and equipment resources at NIST’s disposal so it can 

collect data under actual WUI events. Additional trained staff that can be mobilized quickly are needed to 

effectively engage the limited field equipment to which NIST has access. It is important that the FRD 

continue work to establish guidelines for protection of homes in WUI areas.

Modifications to the FDS modeling approaches may be needed for application in WUI fires. 

Collaboration with the USFS could be increased to create a better understanding of WUI fire impact on 

communities, considering such factors as vegetation, topography, and weather-related parameters 

(temperature, humidity, and wind).

Recommendation: The Fire Research Division should strengthen research on phenomena 

related to wildland-urban interface fires, including data collection during real events and 

new modeling approaches. 

Recommendation: The Fire Research Division should increase collaborations with the U.S. 

Forest Service. 

The design and construction of a fire test facility with cross-wind capability would be a valuable 

addition to the FRD facility suite. 

Recommendation: The Fire Research Division should consider developing a fire test facility 

with cross-wind capability.

The FRD could increase the impact of its work through increased coordination with the 

international community and international standardization. As an example, work on upholstered furniture 

in U.S. dwellings could benefit by examining European research, recommendations, and standards in the 

area of railway transportation. For research and products for upholstered furniture, the role of the FRD is 

essential in determining the effectiveness and possible toxicity of flame retardants. The work on 

development of standards on reducing ignition propensity of cigarettes has had worldwide impact. The 

FRD has monitored and contributed to ISO standards for the tests. 

Recommendation: The Fire Research Division should review European experience on 

upholstered furniture use in railway transportation for comparison with U.S. standards.

The FRD team has completed and released a new open-source version of the FDS modeling tool, 

widely adopted by the engineering community. They provide full documentation and responsive user 

support. The FRD has created a new set of goals for the next version of FDS, to include fire–structure 

interactions, treatment of immersed boundaries, adaptive mesh refinement, and better physics-based and 

34
fully coupled pyrolysis models to compute online heat release rates from a variety of burning materials. 

These goals are appropriate.

Development of new FDS capabilities needs to be continued and may require increases in 

computer resources and staff. The FRD’s proposed updates to CFAST will be valuable for this useful 

tool, which is becoming outdated.

Recommendation: The Fire Research Division should continue development of the next 

versions of the Fire Dynamic Simulator and the Consolidated Model of Fire and Smoke 

Transport, as planned.

FRD has made significant progress over the past several years in the relatively new structures 

area, providing new guidelines for structural fire-resistant design, new models of high-temperature 

properties, an understanding of failure mechanisms, and new recommendations for performance-based 

design of structures that include fire effects.

Continued work on material properties under stress and on failure mechanisms is needed to 

provide the underlying data for new codes, standards, and industry-accepted material property models. In 

parallel, FRD’s competences in structural fire behavior need continued development, specifically in 

advanced thermomechanical modeling and associated experimental methods. These areas are outside the 

normal expertise required for fire research.

Recommendation: The Fire Research Division should extend to other materials its modeling

work on properties and failure mechanisms of steel at elevated temperatures and should 

continue to establish new codes and recommendations for building construction.

The imposed travel limitations make experimental field deployment a significant challenge, 

hinder the ability to interact with colleagues at both the national and international levels, and negatively 

affect the division’s ability to disseminate its work. 

Recommendation: The Fire Research Division should work with the Engineering 

Laboratory and NIST management and administrative offices to resolve issues with respect 

to travel restrictions.

The expertise in model evaluation and uncertainty calculations across the division’s activities is 

appropriate. The support of the Statistical Engineering Division of the NIST Information Technology 

Laboratory is essential to the enablement of such capabilities.

Since 2000, FRD staff have published more than 240 papers, including in peer-reviewed 

scientific publications, conference presentations, NIST technical notes, and special publications. The 

program allowing guest researchers and postdoctoral researchers to work at the FRD is utilized well as a 

source of new ideas and competences and is an excellent source for assessing potential staff. The fire 

grants program and cooperative agreements continue to support research, although external contract 

support to universities by NIST has diminished, according to FRD staff. 

Recommendation: The Fire Research Division should continue with and strengthen its 

program involving guest and post-doctoral researchers.

35

4

Intelligent Systems Division

The Intelligent Systems Division (ISD) develops measurement science and standards to support 

the evolution of intelligent technologies to advance U.S. manufacturing. The division’s measurement 

science research programs aim to improve the versatility of intelligent automation technologies for smart 

manufacturing and cyberphysical systems applications; to optimize the performance of smart 

manufacturing systems; and to enable rapid, cost-effective production of products through advanced 

manufacturing processes and equipment.1

The ISD consists of five organizational groups that address systems in the areas of manipulation 

and mobility, networked control, sensing and perception, cognition and collaboration, and production. 

The work of these groups addresses the key topics of smart manufacturing control systems (SMCS), nextgeneration robotics and automation, and smart manufacturing processes and equipment and responds to 

the needs of other agencies. The ISD has approximately 40 staff, and its budget for 2014 was

approximately $20 million, reflecting an upward trend over the past 5 years.

TECHNICAL PROGRAMS

Smart Manufacturing Control Systems

Approximately 65 percent of the investment in a modern factory goes toward factory automation, 

including monitoring, control, and communications. Consequently, it is essential to achieve modularity 

and interoperability of such complex systems for ease of design, maintenance, and reconfiguration. The 

ISD has supported progress toward such goals. The division’s current emphasis on performance 

evaluation of wireless factory networks addresses an important need, as does the project on cybersecurity 

of industrial control systems (ICS). The objective of the SMCS program is to develop and deploy 

advances in measurement science for sensing, modeling, and optimizing manufacturing activities in a 

smart factory. The main thrusts of the program are in measurement and sensing, modeling and simulation, 

and control and optimization. The strategy for SMCS standards is well suited to the vision of smart 

factories that are able to exploit access to real-time production information at all points across the facility. 

The six projects in this program area are well aligned with the program objectives. 

Accomplishments 

In the factory cybersecurity project the division has demonstrated leadership in addressing 

cybersecurity in smart factory systems and other industrial cyberphysical systems. Indicative of this 

leadership is the recent publication of a new draft of NIST SP 800-82, Guide to Industrial Control 

Systems (ICS) Security, which has been downloaded from the NIST Internet site more than 2.5 million 

1 National Institute of Standards and Technology, “The Engineering Laboratory—Summaries of Our Activities, 

Accomplishments and Recognitions,” Gaithersburg, Md., July 2014.

37

 
times since its initial draft release in 2006. Concepts from this guide are embodied in the ISD’s 

contributions to the ISA 99 Technical Committee and to the ISA/IEC suite of international standards for 

ICS security.

The ISD technical staff also led the development of IEEE Standard 1588, “Precision Clock 

Synchronization Protocol for Networked Measurement and Control Systems.” This standard will enable 

synchronization with an order of magnitude improvement in precision, now measured in nanoseconds. 

The standard is having broad impact in industrial automation and semiconductor manufacturing and in 

telecommunication and other applications beyond the smart factory. As networked sensors and actuators 

grow exponentially within the factory, this standard and the ongoing work in the SMCS project will have 

even broader impact. 

Opportunities and Challenges 

The ISD needs more scientific and technical staff in the area of factory cybersecurity. The 

cybersecurity needs of ICS, robotics, and other smart factory systems differ from the information 

technology cybersecurity needs addressed by other NIST laboratories. The leader of this project is well 

recognized for his expertise in both ICS functionality and network cybersecurity technologies and 

practices—a combination that is hard to find and much in demand. It will be important to grow 

multidisciplinary staff capabilities for this project and to develop a broad network of connections for 

dissemination of results, especially to small and mid-sized enterprises. It will be equally important to 

transfer knowledge from this project area to other programs in the ISD so that cybersecurity can be built 

into standards and measurement science for applications in smart factories, control systems, 

manufacturing processes and equipment, and next-generation robotics and automation. This will require 

staff development initiatives within the division to equip project teams to implement the ISD’s 

cybersecurity principles from the ground up in new projects. The complexity of networked systems makes 

it impractical to add cybersecurity after they have been developed, so there is a need for urgency in 

incorporating expertise into the teams during the current project windows of opportunity. 

Next-Generation Robotics and Automation 

Current industrial robots often cannot operate safely with humans, and so they need to be 

isolated. This limits their applicability and increases the costs of automation. The NGRA program is 

developing performance evaluation methods and standards for safe robot–human interactions and has 

established credibility in methods for measuring and evaluating safe robot–human interactions. Part of the 

ISD’s mission is to evaluate system performance and develop standard test methods for robotic systems. 

Accomplishments

Testbeds have been built to focus on evaluating performance in the following areas: perception, 

dexterity and manipulation, mobility, safety, collaboration, and agility. These testbeds include newgeneration robot arms, three-fingered hands, and automated guided vehicle (AGV) equipment. In the 

robotics performance measurement area, the division has started a standardization effort on the 

performance of AGVs. The division is also extending performance evaluation to other systems, preparing 

to cover various aspects in evaluating robotic hand capabilities, such as position, torque, grasp types, 

graspable object size and ranges, touch location, force, and pressure via tactile sensing. The ISD develops 

methods to objectively evaluate the performance of current systems in perception, dexterity, 

manipulation, mobility, safety, collaboration, and agility. ISD staff are performing tests that will generate 

assessment criteria to enable manufacturers to evaluate commercial systems with respect to critical factors 

38
such as perception and grasped object position. As an example, in the perception area, the ISD led the 

effort to develop the ASTM E2919-13 standard test methods for evaluating the performance of systems 

that measure static, six degree of freedom pose. Many companies benefit from this standard by asking 

suppliers to provide the test results on their systems.

Opportunities and Challenges 

Development of safe industrial robots is a critical activity that supports the use of automation in 

manufacturing facilities. Such automation enables high levels of production, repeatable quality, and a new 

era of human–machine interaction that can enable flexible manufacturing systems. The division’s effort in 

evaluating dexterity and manipulation will be valuable in the long term, when flexible hands or grippers 

are commonly utilized. In pursuit of the long-term goal, the short-term priority might best be focused on 

the repeatability and stability evaluation of the held objects (i.e., the end results) instead of the evaluation 

of performance of the individual components of the robot. 

Extending safety evaluation to other systems such as collaborative robots, hands, and endeffectors, which will operate in a more interactive environment with humans, will be valuable for industry 

as these capabilities mature to the use of fenceless automation, at which point humans will be able to 

work safely within the reach of robots. In the area of safety evaluation, the size of the group is small and 

may not have the critical mass to accomplish the scope of work.

The ISD recognizes that the scope of work needs to grow in order to address the need for 

evaluation criteria for many aspects of robot performance, such as the dexterity and manipulation 

characteristics of the new generation of industrial robots.

Smart Manufacturing Processes and Equipment

The objective of the Smart Manufacturing Processes and Equipment program is to develop and 

deploy advances in measurement science that will enable rapid and cost-effective production of 

innovative, complex products utilizing measurements, physics-based modeling, and simulation of 

machines and processes at the workstation level. The main thrusts of the program have been in smart 

machining, micro- and nanomanufacturing, and metal additive manufacturing.

Accomplishments

The program has been successful in establishing machine tool performance testing standards, as 

evidenced by several ASME and ISO standards and the dissemination of results through ISO publications. 

The ISD actively participates in several committees and consortia in each of the program thrusts and for 

years has been a leader in smart machining metrology. The ISD has been actively engaged in ASTM 

committees to establish additive manufacturing standards. The division has made progress in additive 

manufacturing standards by leading an ASTM/ISO joint working group to develop a prototype standard 

test artifact for additive manufacturing. The ISD actively participates in several committees and consortia

in each of the program thrusts. 

Opportunities and Challenges

The decision in fiscal year 2014 to terminate the smart machining activities and focus on 

measurement science for additive manufacturing may risk loss of critical capability and recognized 

expertise in smart machining. Advanced manufacturing needs both additive and subtractive processes. 

39
Subtractive manufacturing processes typically refer to conventional manufacturing processes by which

material is removed, e.g., from a billet, to make a part and achieve the final desired geometries and 

tolerances. Additive manufacturing processes refer to a suite of methods to add material, typically in 

powder or wire form, to make a part, add features to a part, or repair a part. These emerging additive 

manufacturing processes render near-net shape parts and present new challenges in machining. A hybrid 

approach consisting of both additive manufacturing and smart machining is needed for the development 

of measurement science for additive manufacturing processes. 

The suite of processes for the additive manufacturing of metal requires postthermomechanical 

processing (heat treatment, machining, and surface finishing) to achieve desired material and structural 

properties and geometries that meet tight tolerance requirements. In many respects, additive 

manufacturing and subtractive manufacturing are complementary. Subsequent machining or surface 

finishing is needed for near-net shape parts produced by additive manufacturing processes. The expertise 

developed at the ISD in science-based smart machining can be applied to the suite of processes for 

additive manufacturing of metal to accelerate learning and to provide results to help industry understand 

and adopt these technologies. The machining of a casting is different from the machining of a forging, so 

the machining of different additive manufacturing processes would also vary. Research is needed to 

elucidate the relationship between the amount of material to add in the additive processes and the amount 

of material to remove in subsequent machining or surface finishing in order to achieve the desired 

tolerances, efficiencies, and economics.

A hybrid approach will support achievement of the division’s desired impact on advances in 

measurement science to enable widespread adoption by the U.S. manufacturing industry of additive 

manufacturing processes for metal by helping to overcome technology barriers and enable robust, 

deterministic, and rapid production of innovative, customized, complex products. 

Projects for Other Agencies 

Projects sponsored by other government agencies have been an important part of the ISD 

portfolio for many years, providing both supplemental financial support for the division’s technical staff 

and new professional connections to researchers and practitioners in other fields. As the ISD’s budget has 

grown (having roughly doubled over the past 5 years), the need for supplemental financial support has 

decreased. Nonetheless, the selective acceptance of work for other agencies has benefitted staff by 

exposing them to new problem areas, and by providing opportunities for application of the ISD’s 

technical expertise and recognition of the division’s unique role in measurement science and independent 

evaluation.

The recent completion of a series of projects sponsored by the Defense Advanced Research 

Projects Agency (DARPA) is a case in point. The ISD served as the independent evaluator in the DARPA 

TRANSTAC automated language translation program and its follow-on transformative applications 

program, both of which delivered essential capabilities to warfighters in Iraq and Afghanistan. The scale 

of the DARPA projects exceeded the normal ISD project size, providing funding and the opportunity to 

gain deep insights into the work of DARPA’s best competitively selected technical performers. By being 

part of the DARPA team from the outset of these projects, the ISD was able to contribute significantly to 

the planning, play a highly influential role in project execution, develop general purpose evaluation 

protocols and tools for measuring performance of complex intelligent systems, participate in DARPA’s 

high-profile dissemination of results, and gain visibility and recognition for ISD’s contributions. 

Recognition included a 2013 NIST Gold Medal for the ISD team’s technical excellence and its role in 

delivering technologies that saved lives on the battlefield. 

Other agency (OA) projects are likely to continue to be a significant part of the ISD portfolio. In 

fiscal year 2014 OA projects made up about 15 percent of the ISD portfolio, down from as much as 40 

percent in the past. Given the benefits to the ISD and to stakeholders, it would be appropriate for future 

planning to include carefully selected OA projects. The ISD’s internally sponsored projects would 

40
logically have first claim on available staff, but a deliberate planning approach to make staff available for 

OA projects would serve the division well in the long run. 

Overall Assessment

The ISD’s research programs are focused on measurement science and standards for 

manufacturing. Two exceptions are the DARPA-funded projects on soldier smartphone applications and 

language translation systems and the projects aimed at developing performance test methods for response 

robots. These projects, too, directly build on the division’s expertise in performance evaluation. The ISD 

research portfolio is well aligned with areas of national priority, such as advanced manufacturing and 

robotics. The ISD personnel are aware of strategic directions of importance to both the government and 

industry, and they have focused their efforts on those critical areas. 

ISD projects evince a spectrum of technical quality, with some being strong in the planning 

phase, others in the execution phase, and others in the dissemination phase.

In the planning phase it is essential to engage the appropriate stakeholders from academia and 

industry. The ISD is proactive in its use of workshops to bring together its stakeholders, but there may be 

room for some improvement with respect to engagement of government research establishments. For 

example, the Robonaut team at NASA’s Johnson Space Center is among the leaders in research on 

dexterous manipulation and could have been included in the ISD’s robotics workshops. Although the 

correct mix of engagement with companies in the private sector was frequently achieved, the engagement 

was not always at the right level. 

With respect to the execution phase, the teams were highly engaged and enthusiastic and were 

comprised of individuals with the appropriate academic skill sets, enhanced by business domain 

knowledge (this was particularly evident in the machining department). However, there is a need to hire 

staff to sustain the skill base within the laboratories, because there is a significant retirement-eligible

population. The critical mass of personnel in several groups is a risk factor, with some areas—for 

example, cybersecurity of ICS—having a bench strength of only one.

Manufacturing research and development (R&D) is important for national prosperity and requires 

fundamental supporting science. Manufacturing R&D is critically important for realizing value from 

investments in product and material innovation. Despite growing national awareness of this importance, 

research in enabling processes and technologies for advanced manufacturing often fails to keep pace with 

material or product development research. It is crucial that the ISD leadership continue to support the 

division’s activities in manufacturing science. In order to exploit the extensive portfolio of fundamental 

research programs in materials, particularly at DOE and the National Science Foundation (NSF), the 

United States needs an equal level of innovation focused on the fundamental research and standards 

required to create manufacturing processes and technologies that enable commercialization of new 

products.

It is important that the many groups working effectively within the ISD continually consider 

innovative and cross-disciplinary activities, essential to maintain their vitality, including nontraditional 

business mechanisms aimed at driving game-changing innovations across the groups. A continuing 

challenge is to generate ideas and mature them, connect them across the groups, and predict which ones 

will deliver effective approaches to achieving mission objectives. The ISD could consider, for example, 

relevant business practices that combine evaluative competitions for entrepreneurial ideas with a gaming 

mechanism, such as prediction markets, which have been used by many companies to predict effective 

project timing and choose innovative projects.

41
PORTFOLIO OF SCIENTIFIC EXPERTISE

Accomplishments

The ISD has very capable, motivated, and enthusiastic staff. They have a clear command of the 

technical issues they are focused on and are actively involved in the broader research and standards 

community for those topics, both in the United States and globally. The personnel at the ISD have 

demonstrated the ability to apply their knowledge of mathematics, science, and engineering. They are 

very capable of designing and conducting experiments and of analyzing and interpreting data. Their

recognition of the need for, and ability to engage in, continuing education in multiple disciplines enable 

the personnel to function on multidisciplinary teams and look at problems from a broader systems 

perspective. The staff are professional and effective in communicating the work that they perform.

There are research areas in the ISD that are recognized as being among the leading groups in the 

world. For example, the division has a long-established reputation in machining metrology and machine 

tool error compensation. It is well recognized for its pioneering work on security of ICS. Its work on 

testing and performance assessment of small mobile robots used in disaster response is also widely well

regarded.

Opportunities and Challenges 

The popularity of additive manufacturing has revived in recent years. The initial enthusiasm for 

the technology when it emerged more than two decades ago became muted by the challenges of 

incorporating the systems into the manufacturing process beyond prototyping. The invention of new 

additive manufacturing systems that provide better material choices has created the need for objective 

assessments and evaluations. The ISD team recently established a small activity in this area, which is 

timely, given the significant growth of original equipment manufacturers and systems. 

Additive manufacturing is an area of growth for the ISD at NIST. The research performed is 

much needed to establish material standards and test standards, because there is no publicly available 

database of these standards for different additive manufacturing processes and materials, especially for 

additive manufacturing of structural parts.

As the ISD expands its portfolio, there is opportunity for growth in scientific expertise, which 

needs to be combined with increased domain expertise across different industries. This essential 

combination will drive the development of relevant standards to enable the manufacturing community to 

match equipment with appropriate applications. 

The ISD has chosen to focus on laser powder bed fusion of metals. This is an additive 

manufacturing process that has applicability across several industries. Staying abreast of the other 

additive manufacturing processes is necessary, even if they are not the primary focus of measurement 

science at the ISD. Division personnel are staying abreast of developments through participation in 

several committees and consortia. Further engagement with equipment manufacturers, research 

institutions, academia, other government entities, and the international community engaged in different 

additive manufacturing processes, would enable the ISD to influence and engage others in further 

advancing measurement science for its suite of additive manufacturing processes. An early success is the 

development of a prototype standard test artifact for additive manufacturing. Active engagement with the 

ASTM/ISO Joint Working Group to make this a test standard is underway.

The ISD has scientific expertise in the several disciplines that are needed to pursue material and 

process characterization methods and standards and an expanded scope that includes characterizing 

additive manufacturing materials; real-time control of additive manufacturing processes; qualification of 

additive manufacturing materials, processes, and parts; and systems integration for additive 

manufacturing. The ISD staff possess expertise in smart machining, metrology, sensors, controls, and 

nondestructive evaluation. With the existing expertise and with new expertise being developed in additive 

42
manufacturing failure modes, effects, and criticality analysis studies, there would be a natural progression 

to identify and optimize critical parameters for trade-offs between quality (surface finish enhancements 

and better properties), speed (fast build rates), and accuracy (tolerances). This would support the objective 

to enable widespread adoption by the U.S. manufacturing industry of additive manufacturing processes 

for metal. 

The pilot round-robin tests that the division and its partners have been engaged in addresses 

build-to-build variation and machine-to-machine variation of the same machine model. Building on the 

pilot round-robin testing with equipment manufacturers, manufacturing companies, academia, and 

government institutions that are working with different laser powder bed fusion machines or materials 

would help to generate a larger data set and address machine-model-to-machine-model variation. Similar 

work is being performed at other laboratories with different materials and different machines. The 

difference is that the ISD’s approach can be standardized and disseminated widely. This would support 

the objective of enabling widespread adoption by the U.S. manufacturing industry of additive 

manufacturing processes for metals, by addressing the differences in machine models through leveraging 

of external expertise. 

Attracting and retaining the best researchers is a perennial challenge for government laboratories, 

especially in technical fields with competing commercial demands. The ISD has done very well over the 

years in building a top-notch technical staff, and the long tenure of many current staff members evinces 

the ability of the division to retain expertise in critical areas. However, there is potential for increased 

turnover owing to the number of retirement-eligible staff, and there is a lack of bench strength and 

difficulties in hiring in the new technical areas the division is entering.

About 50 percent of the 39 current ISD staff members are eligible for retirement within 5 years. 

Action is needed now to identify key positions where succession planning is indicated, to identify pools 

of internal candidates from across NIST and invest in their development, and to build relationships with 

external sources of qualified expertise to encourage applicants when positions open up. The division has 

dealt with turnover before, and the actions needed are well understood. Execution will require support 

from the EL director.

The division is appropriately planning and executing programs in new technology areas such as 

SMCS, additive manufacturing, industrial robotics, and cyberphysical systems. In addition, the division 

has seen an explosion of government and global industry interest in its work on ICS cybersecurity, with 

more than 2.5 million downloads from the NIST Internet site of the Guide to ICS Security (NIST SP 800-

82) since its initial draft release in 2006. The division currently has one staff member who is widely 

recognized for deep expertise in this area and who is spread very thin in responding to current program 

demands. The division is trying to hire at a time when ICS cybersecurity experts can demand and receive 

high salaries and other benefits from industry. Government salaries are not competitive in this niche area. 

This example is a bellwether of expected competition for a limited pool of qualified talent in all of these 

new program areas that have high demand for unusual combinations of cross-disciplinary knowledge and 

skills. 

Because the need to staff ongoing and new programs is urgent, the division cannot wait and hope 

for qualified, affordable candidates to become available. A strategy is needed to hire now where possible 

(by emphasizing factors other than salary alone that make NIST employment attractive), to invest in 

developing the needed multidisciplinary skills in current staff and entry- level hires, and to provide 

interim support through contracts and cooperative research arrangements with universities and others.

FACILITIES, EQUIPMENT, AND HUMAN RESOURCES 

Overall, the ISD facilities and equipment are well suited to meet the division’s needs. The 

division has experienced varying levels of investment in buildings and facilities. There are state- of-theart facilities in the new, rapid-response robot initiatives, which contrast with less-funded areas such as the 

43
new additive manufacturing laboratories, although there appeared to be plans under discussion for 

addressing that area. 

Accomplishments

A new showcase facility has been created to support the work of the ISD in standard methods for

performance evaluation of mobile robots used in disaster response. This is a unique facility that provides 

excellent research space as well as a venue for outreach and dissemination of research results. The longstanding ISD programs related to machine tool performance and machining metrology are supported by 

excellent facilities. While these facilities are not new, they are well maintained and suitable for the 

purposes of the research programs. 

Opportunities and Challenges

New additive manufacturing machines with enhanced features and capabilities are introduced to 

the market each year. There are several laser powder bed fusion of metal machines, many with higherperformance lasers and optics, controlled environmental chambers, automated powder delivery, inprocess monitoring, powder compaction, and other features and tools to advance measurement science 

knowledge. These new machines are being purchased by industry, academia, research institutions, and 

government entities globally. It may be difficult for industry and academia to see the ISD at the forefront 

of measurement science if its equipment does not represent the best available. 

The current machine at the ISD is adequate for the initial work and for developing expertise. 

Ownership of and/or access to machines with capabilities and equipment that can detect and measure 

defects in additive manufactured parts in situ or postprocess would better support the ISD’s objective, 

which is to enable widespread adoption by the U.S. manufacturing industry of additive manufacturing 

processes for metal. Partnerships with equipment manufacturers, other government entities, academia, 

and others to gain access to equipment that the ISD does not own or exploring equipment loans are 

options to consider. 

Procuring the appropriate equipment for measurement science projects is essential to achieving 

program objectives. In the fast-moving field of additive manufacturing, it is particularly important that the 

ISD secure rapid access to necessary equipment and that it be able to measure the performance of 

particular characteristics in response to the needs of industry stakeholders. According to ISD staff, current 

ISD equipment procurement practices are not in line with this laboratory need. The Federal Acquisition 

Regulations (FAR) generally favor competitive procurement using performance specifications that are 

broadly enough defined to allow for competition among multiple brands of equipment and selection on a 

best-value basis. According to ISD staff, NIST and federal procurement regulations have set the bar high 

for justification of sole source procurements and have discouraged specifications that are so narrow that 

only one brand of equipment can meet them. This is entirely appropriate if the division needs to procure 

an air conditioner or another commodity item. However, when the objective of the research is 

measurement science based on the combination of features that only one machine offers, this practice can 

consume months of procurement cycle time and end up by selecting unsuitable equipment, or it may be

cancelled or restarted with different specifications.

The laboratory technical staff need an opportunity to justify their narrow specifications for 

required equipment, with appropriate technical management review. It would be useful for the ISD to 

benchmark its laboratory equipment procurement practices against those of other federal laboratories, 

which may be taking better advantage of sole source procurements and other flexibility built into the 

FAR. The ISD’s laboratories may be adequate to support the work performed, but the infrastructure 

would benefit from work-space freshness and modernization. The quality of the facilities influences 

recruitment and retention of employees and would appropriately be a part of the division’s strategic plan.

44
DISSEMINATION OF OUTPUTS

Accomplishments

The research work in the ISD is being published in good journals and conferences appropriate to 

the research topics. Some of the work has also achieved a good citation record, which speaks to its 

technical merit and broader impact. Given the scope of ISD activities (which include the planning, 

execution, and dissemination phases of research), their publication record is good. An example of 

accomplishment in dissemination is NIST Special Publication 800-82, Guide to Industrial Control 

Systems (ICS) Security, downloaded from the NIST Internet site more than 2.5 million times since its 

initial draft release in 2006. An ASTM International Standardization News article on emergency response 

robots successfully highlights the division’s scientific expertise and leadership of the ASTM International 

Subcommittee E54.08 on operational equipment to produce robot performance standards. It also 

highlights how the ISD’s work led to the success of Quince, a response robot used at the Fukushima 

nuclear power plant. Articles in ASTM International Standardization News have also featured the 

division’s leadership roles in smart manufacturing and AGVs.

Opportunities and Challenges

With respect to dissemination of information, there has been a tendency to take a traditional and 

somewhat limited approach with a strong focus on publishing and conferences, which, although 

important, could be enhanced with the adoption of additional mechanisms such as social media. A 

targeted marketing approach could be used to drive not only the dissemination mechanism but also the 

scope and format of the information to promote a deeper penetration of the ISD deliverables into the 

many different stakeholders in the manufacturing sectors. In conjunction with developing new 

mechanisms for the dissemination of information (or any other new business practices), it will be 

important to modify performance metrics to reward new staff behaviors. There may be opportunities for 

wider use of social media to publicize NIST’s work and accomplishments, including opportunities for 

more focused dissemination to a target audience via social media and potentially via engaging 

professional marketing and communication assistance.

Participation in scientific meetings and conferences is essential to the planning, execution, and 

dissemination of outputs of ISD programs. In 2012, the federal Office of Management and Budget (OMB) 

issued guidance to all federal agencies that travel budgets needed to be cut by 30 percent and that 

attendance at conferences be subject to tighter restrictions and senior level reviews. The impact on the EL 

was a freezing of the travel budget at 2010 levels. According to ISD staff, the effect on the division has 

been to constrain not only the ability of technical staff to engage in mission-essential travel but also the 

ability to cover the travel and relocation expenses that are essential to recruiting new technical staff. ISD 

staff expressed concern that these constraints are compromising the division’s ability to carry out its 

measurement science mission. It is understood that policy decisions in this area are beyond the division’s 

control, but the ISD needs to work with NIST management and administrative offices to develop budget 

categories and strategies that distinguish travel for programs, travel for staff development, and travel for 

personnel recruitment and relocation, so that policy makers can fine-tune future guidance.

Travel restrictions impede ISD personnel from making important contacts, developing 

professional relationships, and gaining a world perspective at events such as the World 

Radiocommunications Conference, which takes place every 3 or 4 years, at which in 2015 new radio 

frequency spectrum standards will be discussed and established; Euromold, the world’s largest conference 

on additive manufacturing; and AUTOMATICA, the world’s largest conference on robotics, assembly 

lines, and machine-vision systems. The European Commission announced the world’s largest civil 

robotics program, SPARC, at the AUTOMATICA 2014 Conference, with an investment of €2.8 billion 

and more than 240,000 jobs anticipated. 

45
FINDINGS AND RECOMMENDATIONS

The ISD needs to develop and implement plans for recruitment and retention of quality staff. One 

of the common challenges to R&D institutions is that of meeting required capital and human resource 

needs. The investment required to enable organizations to meet their mission goals has increased along

with the exponentially increasing generation of new technology. The ISD is not unique in experiencing 

this resource challenge and may also need to look at alternative business processes and mechanisms by 

which to generate meaningful data. The ISD may wish to explore the use of more affordable mechanisms 

to achieve some of its goals. 

Recommendation: The Intelligent Systems Division should identify and immediately 

address critical skill gaps and develop formal hiring and staff development strategies. 

Recommendation: The Intelligent Systems Division should investigate mechanisms for 

competitively evaluating innovative ideas and cross-division game-changing activities. 

Recommendation: The Intelligent Systems Division should benchmark its laboratory 

equipment procurement practices against those of other federal laboratories subject to the 

Federal Acquisition Regulations (FAR). Where practices are found that take advantage of 

flexibilities in the FAR to obtain the right equipment more quickly and efficiently, the 

Intelligent Systems Division should adopt them.

Recommendation: To address the challenge of limited resources the Intelligent Systems 

Division should explore such options as equipment loans and rotating scientist 

arrangements.

Recommendation: The Intelligent Systems Division should build on the pilot round-robin 

testing with certified sources to generate a large data set for their additive manufacturing 

efforts.

Recommendation: The Intelligent Systems Division should apply their recognized expertise 

in science-based smart machining to additive manufacturing by pursuing a hybrid 

approach to achieving the division’s desired impact on advances in measurement science to 

enable widespread adoption by the U.S. manufacturing industry of additive manufacturing 

processes for metals.

46
5

Materials and Structural Systems Division

The Materials and Structural Systems Division (MSSD) serves as a global resource for 

developing and promoting the use of science-based tools—measurements, data, models, protocols, and 

reference standards—to support innovations in building materials and construction technology and to 

ensure the safety, security, and sustainability of the nation’s buildings and physical infrastructure.

1

Areas of focus include sustainability, service life prediction and life-cycle assessment of materials 

performance and safety, and resilient performance of structures. The MSSD is organized into five formal 

groups: Structures, Inorganic Materials, Polymeric Materials, National Earthquake Hazards Reduction 

Program, and National Windstorm Impact Reduction Program. During the course of this review, five 

areas of work were examined: sustainable engineered materials–inorganic materials, sustainable 

engineered materials–polymeric materials, structures, wind research, and community disaster resilience. 

The first four of these areas are relatively mature within the Engineering Laboratory (EL); the community 

disaster resilience work is in the early stages of development.

TECHNICAL PROGRAMS

Sustainable Engineered Materials–Inorganic Materials

Within the MSSD, materials research is performed separately by inorganic and polymeric 

materials groups. Although the groups have similar objectives, they operate independently with separate 

staff and resources. 

The Inorganic Materials Group’s focus is on characterizing the properties of concrete from early 

age to hardened materials state. This includes the rheology2 of cement pastes and concrete slurries and the 

character of cured concrete. Increasing the understanding of these short-term and long-term properties 

will help to achieve one of the MSSD’s goals: to double the current service life of concrete, which will 

greatly enhance the sustainability of the material. 

The group is also working on reducing the cement content of concrete by developing 

measurement protocols and standards related to the substitution of fly ash and limestone powder to 

decrease the carbon footprint of concrete. This work on fly ash characterization, limestone powder 

replacement, and internal curing is excellent and well connected with the goal of sustainability. The 

group’s work on hydration modeling is pioneering and highly regarded nationally and internationally, and 

its model of hydration has been widely adopted by others. Building on the hydration modeling work, the 

group has established the Virtual Cement and Concrete Testing Laboratory, an industrial consortium, to 

carry out work in computational modeling and experimental validation.

1 National Institute of Standards and Technology, “The Engineering Laboratory—Summaries of Our Activities, 

Accomplishments and Recognitions,” Gaithersburg, Md., July 2014. 2 Rheology is the study of the flow of matter, primarily in the liquid state, but also as “soft solids” or solids 

under conditions in which they respond with plastic flow rather than deforming elastically in response to an applied 

force.

47

 
As concrete is increasingly pumped during construction, the number of failures due to flow 

problems has also increased. The group’s work on cement and concrete rheology combines rheometric 

measurements with simulations of particles being sheared within non-Newtonian dense suspensions. This 

work is very challenging. On the measurement side, these materials are highly prone to wall slip, and care 

needs to be given to texturing the shearing fixtures of rheometers to avoid this phenomenon. The 

simulations are difficult due to the high volume fractions of the solid fractions and the irregular shapes of 

the solid particles. Furthermore, in the case of concrete slurries, the interstitial fluid is a non-Newtonian 

cement paste, and proper characterization of the rheology of the cement is an important ingredient for a 

successful simulation. The group is seeking to develop standard reference materials (SRMs) based on 

simulation modeling that mimic the actual rheological properties of paste, mortar, and concrete.

Sustainable Engineered Materials–Polymeric Materials

Photodegradation impacts a very large number of materials used in buildings and infrastructure

applications. These range from coatings for metals, window glazing, and solar panels to a wide variety of 

building and joint sealants. The group’s work on polymer sustainability has focused on photodegradation 

of this class of materials and has taken advantage of the work on the ultraviolet (UV) photophysics of 

polymeric materials that began 12 years ago with the development of the 2-m integrating sphere. The 

existence of this unique test apparatus to accelerate the degradation process has allowed the group to 

produce important results on a number of polymeric materials. The group has developed a smaller, 

commercial version of the large integrating sphere that looks very promising. It is limited, however, in the 

size and quantity of samples that can be accommodated within the measurement chamber. A goal of this 

work is to develop a predictive model for the lifetime performance of polymeric materials. This would be 

best achieved by combining their existing capabilities in experimental measurements with computational 

molecular modeling. 

The group’s photophysical work is very well developed, and once the commercial unit is 

launched, thought could be given to redirecting the work on polymers to consider other products used in 

buildings and infrastructure. Many polymers are used in applications that are not exposed to light but still 

age and degrade by different mechanisms. Common examples are PVC and CPVC pipe and DuPont’s 

Tyvek building wrap. Another important class of products hidden from light are elastomers used in 

plumbing applications. Many cities have switched to chloramines to purify water; this can cause 

degradation in O-rings that were previously unaffected by the chlorination process. 

Structures

The MSSD is performing work on structural robustness and the mitigation of disproportionate 

collapse. The staff are well-qualified and possess excellent capabilities in structural analysis, 

computational modeling, and physical validation testing. The tests and modeling of collapse conducted by 

the team (off-site for the tests) are very good and have provided potentially useful data on detailing of 

structural steel. However, this expertise could be applied to a broader range of scenarios for 

disproportionate collapse. In addition, the planned path toward recommendations for achieving robustness 

and resistance to disproportionate collapse (without having to test each proposed structure or element) 

was not made clear. 

Wind Research

The staff performing wind research are well qualified and are focused on achievable goals. The 

project to develop improved wind-speed maps for incorporation in the ASCE 7 standard and subsequently 

48
in statutory building codes promises to be a significant service to the nation. The team spoke of 

examining the database on wind tunnel tests from various sources in an attempt to develop pressure 

coefficients for building design that are more realistic than those now in the ASCE 7 standard; this would 

also be a valuable service to the nation, although the team did not present a clear roadmap for the way 

forward. The development of a virtual wind tunnel is a worthwhile long-term goal, and the EL is arguably 

the best body to pursue it; more resources in this area would be a good investment. The study and report 

on the Joplin, Missouri, tornado in 2011 are outstanding and could serve as a model for future disaster 

studies. 

Community Disaster Resilience

The Community Disaster Resilience Program in the MSSD is in its infancy. However, the 

division is well positioned to apply its expertise in construction materials and structural performance to 

address the challenge of improving the nation’s disaster resilience by reducing the vulnerability of 

buildings and infrastructure to a broad array of physical hazards and to prolong the service life of the built 

environment in general. The group’s effort to develop metrics to rank resilience will help to identify the 

most vulnerable communities so that priorities for adaptation and mitigation efforts can be undertaken. 

This is a critical and appropriate first step in the process. The community disaster resilience research and 

development contracts and the center of excellence are excellent ways to quickly access the global state of 

the art for methods and approaches in this area. In particular, the request for proposal for a Community 

Resilience Center of Excellence contains an appropriate set of objectives for study and action in this area. 

PORTFOLIO OF SCIENTIFIC EXPERTISE 

The MSSD is staffed by high-quality personnel at various levels of professional development. 

Internationally recognized researchers work alongside postdoctoral fellows in a collaborative and 

synergistic research environment. With certain exceptions, skill sets are appropriate to the mission 

objectives, and within the programs reviewed there appears to be good diversity of age, gender, and 

ethnicity. Professional development is encouraged through publication and participation on code and 

standards writing groups, but government-wide travel restrictions are limiting opportunities for broader 

collaboration within the research community and for attendance at conferences and workshops. 

Two areas requiring staffing attention are molecular modeling and community disaster resilience. 

Within the Polymeric Materials Group, hiring someone to help link the existing capabilities in 

experimental measurements with computational molecular modeling has been under consideration, but 

this has not yet occurred. Although well qualified, the staff assigned to the Community Disaster 

Resilience Program do not possess expertise sufficiently broad to address the range of issues contained 

within the resilience mission. This is being addressed in the short-term through temporary fellows, 

contract support, and the proposed center of excellence. Staffing needs of the Community Disaster 

Resilience Program will need to be closely monitored to assure that human resources are adequate and in 

the appropriate disciplines for the magnitude and scope of the tasks assigned. 

FACILITIES, EQUIPMENT, AND HUMAN RESOURCES

Available research infrastructure is adequate for ongoing and anticipated efforts, and in some 

cases, such as for polymer and materials characterization, MSSD facilities are among the best in the 

world. The MSSD is also able to access on an as-needed basis advanced research capabilities within NIST 

such as those possessed by the Materials Measurement Laboratory and the Information Technology 

Laboratory. Onsite structural test facilities are adequate for smaller-scale testing congruent with the 

49
Structures Group’s mission. For large-scale structural testing, where the onsite facilities are sometimes 

insufficient, division staff have made use of facilities at universities and other government laboratories 

such as the Engineering Research and Development Center (ERDC) operated by the U.S. Army Corps of 

Engineers in Vicksburg, Mississippi. This is a cost-effective solution that has resulted in savings in both 

cost and time. Overall, research infrastructure is not an impediment to accomplishment of the MSSD 

mission. 

DISSEMINATION OF OUTPUTS

Dissemination of research results follows fairly traditional lines of peer-reviewed journals, 

conference proceedings, internal NIST reports, and input to technical codes and standards. Division staff 

are prolific in this regard and have produced many award-winning papers. However, government-wide 

travel restrictions are seen as an impediment to fuller dissemination of the division’s work. To a greater or 

lesser degree, all groups within the MSSD make use of workshops, symposia, and site visits to scope 

projects and identify issues. This, along with well-planned collaborations, is an effective means of 

securing stakeholder input, provided that attendance from all affected user groups is assured. Outreach to 

the practitioner community could be improved through more articles in professional and trade journals 

and more aggressive use of the Internet to quickly disseminate important findings. 

OVERALL ASSESSMENT

The MSSD is carrying out important work within its mission responsibilities in a highly 

competent and professional manner. The staff are well-qualified and motivated to perform the theoretical 

and experimental investigations within their areas of responsibility and fully engaged with research at the 

national and international levels. 

The current staff mix does not appear unduly skewed by age, gender, or ethnicity. Available 

research infrastructure is adequate for ongoing and anticipated efforts, and in some cases, such as for 

polymer and materials characterization, MSSD facilities are among the best in the world. For large-scale 

structural testing, where onsite facilities are sometimes insufficient, division staff have made use of 

facilities at universities and other government laboratories. This solution results in savings both in cost 

and time. Overall, research infrastructure is not an impediment to accomplishment of the division’s 

mission. Dissemination of research results follows fairly traditional lines of peer-reviewed journals, 

conference proceedings, internal NIST reports, and input to technical standards. Division staff are prolific 

in this regard, although government-wide travel restrictions are seen as an impediment to fuller 

dissemination of the division’s work. Outreach to the practitioner community could be improved through 

more articles in professional and trade journals and more aggressive use of the Internet to quickly 

disseminate important findings. 

FINDINGS AND RECOMMENDATIONS

Contained within the mission of the MSSD are two of the foremost issues confronting the 

nation’s built environment: how to cost-effectively address the large stock of physical infrastructure that 

is nearing or has exceeded its normal service life, and how to make communities more resilient in the face 

of multiple natural and anthropogenic hazards that are uncertain in both likelihood and severity. 

Both the challenges and the stakes are huge. Delaying necessary remediation of infrastructure 

systems increases the probability of failure and attendant higher life-cycle costs, community disruption, 

and the very real possibility of injury and death of those impacted. However, replacing or refurbishing 

50
systems prematurely spends money unnecessarily and reduces the limited funds available to address other 

system needs. 

As for resilience, depending on physical location, the vulnerabilities of communities to various 

hazards differ widely as do the consequences of a specific event. Even though the principles of resilience 

apply equally across the nation, those communities least resilient and the relevant threats or hazards need 

to be identified so that appropriate investments in adaptation, mitigation, and emergency response and 

recovery can be made in a timely and cost-effective manner. 

In this regard, the technical work of the division needs to be supported by sound economic and 

financial analysis. The Office of Applied Economics within the EL is a world-recognized leader in 

economics of building and has performed state-of-the-art analyses on such topics as the benefit-cost 

calculation of building protection strategies to address terrorist attacks. Such innovative analytical 

techniques are called for in evaluating options for infrastructure renewal and community disaster 

resilience owing to the long time horizons and the great degree of uncertainty involved. 

Although the MSSD programs reviewed are producing high-quality work, the projects and their 

results tend to be isolated from one another. That is, they are focused on answering a specific question or 

solving a specific problem, and opportunities for cross-cutting findings are missed. For example, there 

appears to be untapped potential to integrate the capabilities of the materials work in physical and 

chemical degradation with the capabilities of the Structures Group in mechanical performance. The 

Inorganic Materials Group possesses appropriate expertise in the chemical degradation of concrete, while 

the Structures Group has a vital interest in how such degradation affects strength and performance in 

applications. A quantitative method to determine the strength of partially degraded materials would be of 

enormous value in practice and could inform the decision of whether the time for necessary replacement 

had been reached. At present, inspection processes for infrastructure emphasize visual appearance as an 

important aspect of condition assessment. The MSSD could provide the means to greatly improve the 

precision and value of such assessments. 

Findings from individual projects can also apply to multiple situations. For example, the Nuclear 

Regulatory Commission is facing the arduous task of recertifying many of the nation’s aging nuclear 

power stations and has commissioned the Inorganic Materials Group in collaboration with the Structures 

Group to assess the effects of the alkali-silica reaction, which occurs in all concrete, on the residual 

strength of the material. Their findings will apply far beyond the immediate question, because there is an 

enormous amount of aged concrete in the nation’s bridges, dams, and other structures that will ultimately 

need to be replaced, but the quantitative science to underpin such decisions has yet to be developed. This 

project will provide a unique opportunity to incorporate the microscale research of the Inorganic 

Materials Group with the macroscale work on physical performance being carried out by the Structures 

Group. 

Despite the importance of inorganic and polymeric materials in buildings and infrastructure and 

the need to improve our understanding of their degradation and performance, steel remains an important 

material for infrastructure applications whose characterization is also incomplete. At the same time, 

masonry and timber structures are still in use and, like steel, could also benefit from additional study and 

possible code updates. None of these materials are currently under study by the MSSD, although the 

Structures Group has tested some steel assemblies as part of its work on disproportionate collapse. 

The recommendations that follow should not be viewed as a list of shortcomings; rather, they are 

intended to help the MSSD in its efforts to make an excellent program even better. 

It is important that the MSSD continue to recruit for diversity in gender, age, and ethnicity to 

enhance long-term capabilities across the materials, structures, and resilience groups, with attention to 

strategic opportunities for collaboration among the groups.

Recommendation: The Materials and Structural Systems Division should involve 

researchers and practitioners in the identification and evaluation of potential new projects 

to be undertaken by the division. The Materials and Structural Systems Division should 

51
continue interactive stakeholder involvement throughout the life of the project, including 

during the dissemination phase.

Recommendation: The Materials and Structural Systems Division should target the 

dissemination of its results to specific portions of industry, including design and 

engineering, construction, and facilities management, through appropriate means and 

modes.

Recommendation: The Materials and Structural Systems Division should work closely with 

the Engineering Laboratory’s Applied Economics Office to analyze the economic and 

financial implications of proposed actions.

Recommendation: The Inorganic Materials Group should leverage work for the Nuclear 

Regulatory Commission on alkali-silica reaction to connect the nanoscale investigations of 

concrete deterioration to structural performance. The group should apply the results of this 

work to support studies of life prolongation, rehabilitation, and new construction.

Recommendation: The Inorganic Materials Group should establish better linkages between 

chemical and physical degradation and the mechanical properties of construction materials.

Recommendation: The Inorganic Materials Group should encourage further research into 

new materials and systems, including bio-based materials and fiber-reinforced structural 

systems, as well as ubiquitous existing materials (steel, masonry, and wood) for new 

construction and in situ rehabilitation, for durability, functional performance, and 

resilience. This research should include the seismic performance of steel bracing 

connections.

Recommendation: The Inorganic Materials Group should engage the oil and gas industry 

for safe encasement of well bores. The group should examine the Deepwater Horizon oil 

spill disaster in the Gulf of Mexico as an excellent example of the importance of creating 

casings with proper mechanical strength in a short period of time. 

Recommendation: The Inorganic Materials Group should consider developing a method for 

noninvasive, in situ testing of the strength of fresh concrete. 

Recommendation: The Polymeric Materials Group should build on the mature work in the 

photophysics of polymeric degradation and expand it to non-ultraviolet degradation effects 

and should include the data in service life prediction models. 

Recommendation: The Polymeric Materials Group should engage the solar panel industry, 

where polymeric materials are used to encapsulate the active, thin films between panes of 

glass, to examine the industry standard (ethyl vinyl alcohol) and polybutadiene, which are 

problematic, and to guide the formulation of sealants and adhesives for this application.

Recommendation: Polymer and concrete teams should increase their collaboration, 

studying, for example, polymer fibers to strengthen concrete and resist cracking and 

polymer additives to enhance the flow properties of concrete. 

Recommendation: The Structures Group should work with the research and practitioner 

communities to identify and evaluate alternative approaches for quantitative metrics to 

52
better describe phenomena such as disproportionate collapse and select the most promising 

for further development. They should include in this work a metric for robustness. 

Recommendation: The Structures Group should fully integrate its capabilities in analysis, 

computational modeling, and physical testing to obtain the most information from the work 

in disproportionate collapse.

Recommendation: The Structures Group should collaborate with the Fire Research 

Division on the durability and performance of materials and structures under fire 

conditions.

Recommendation: The Structures Group should disseminate more broadly through 

technical and academic journals significant results such as those on plasticity, found 

through the work on disproportionate collapse.

Recommendation: The Structures Group should explore opportunities to better incorporate 

the input of the architectural, engineering, and construction communities throughout the 

project and code development process.

Recommendation: The group performing wind research should explore the concept of a 

virtual wind tunnel based on historic data, new wind speed maps, and computational 

modeling.

Recommendation: The group performing wind research should study microbursts and their 

potential for external structural damage.

Recommendation: The group studying community disaster resilience should examine its 

staffing needs to assure that the human resources are adequate and in the appropriate 

disciplines for the magnitude and scope of the tasks assigned.

Recommendation: The group studying community disaster resilience should develop and 

publicize an explicit roadmap to demonstrate where and how the work of the group will 

mesh with existing (or to-be-developed) codes and standards and who specifically will 

benefit from these efforts. This roadmap should also describe the overarching governmental 

interest in the program with respect to reducing the economic impacts of extreme events or 

enhancing life safety during and following such events. 

Recommendation: The group studying community disaster resilience should, for efforts in 

disaster resilience, build on the expertise of the Materials and Structural Systems Division, 

the Engineering Laboratory, NIST, and external partners for science-based resilience 

performance and standards. 

Recommendation: The group studying community disaster resilience should leverage 

existing U.S. and international research, methodologies, and tools to accelerate 

advancements in theory and practice, including standards and measurements. 

Recommendation: The group studying community disaster resilience should include facility 

owners (public, private, nonprofit), contractors, and skilled trades organizations in 

stakeholder engagement.

53
Recommendation: The group studying community disaster resilience should incorporate 

operations and maintenance into high-level objectives for the Community Disaster 

Resilience Program. 

Recommendation: The group studying community disaster resilience should, for the 

resilience effort, build on the capabilities in the Materials and Structural Systems Division 

and on the findings of National Construction Safety Team field-based failure studies to 

inform the research program and identify critical research topics. The group should use the 

failure studies to provide a source of data for empirical analysis and refinement and 

validation/verification of models across and among the groups.

Recommendation: The group studying community disaster resilience should incorporate 

the environment and natural systems into the resilience program and initiatives, which may 

require new partnerships such as the Environmental Protection Agency’s Green 

Infrastructure Collaborative.

54
6

Systems Integration Division 

The Systems Integration Division (SID) performs work that contributes to measurement science 

and standards needed for integrating engineering information systems used in manufacturing, 

construction, and cyberphysical systems. Areas of work include manufacturing enterprise integration; 

green manufacturing and construction; engineering and manufacturing products, processes, equipment, 

technical data, and standards; collaborative manufacturing research pilot grants; manufacturing 

fellowships; systems integration and engineering; life-cycle assessment; cyberphysical systems; 

productivity measurement; sustainability; and energy efficiency.

1

The SID is organized into four groups: Life Cycle Engineering, Information Modeling and 

Testing, Systems Engineering, and Process Engineering. The division’s work is divided into two areas:

systems integration for manufacturing and construction applications (referred to below as systems 

integration, or SI) and sustainable manufacturing. Systems integration is an area of significant 

development and strength for the division, while the sustainable manufacturing area is new and evolving. 

Both areas are of significant importance and relevance to the SID’s focus and mission. The SID has 25 

technical staff, 4 administrative staff, and 20 associates.

TECHNICAL PROGRAMS 

Systems Integration for Manufacturing and Construction Applications

Accomplishments

The mission of the systems integration (SI) program is to promote U.S. innovation and industrial 

competitiveness in areas of critical national priority by anticipating and meeting the measurement science 

and standards needs for integrating engineering information systems. The research projects are focused on 

much-needed technologies of relevance to the challenges currently faced by industry. The general areas 

are manufacturing interoperability, system integration for manufacturing and construction applications,

and sustainable manufacturing. The overall quality of research in SI is very high.

Many of the SI staff are well regarded by their peers. They have developed robust expertise in 

several key areas, including support of standards development for systems engineering, model-based 

engineering, development and support of the Standard for Exchange of Product (STEP) model data 

standards (ISO 10303-242 [STEP AP242]), composite manufacturing, additive manufacturing, and 

assisting vendors in implementation of the new functionality. The high-quality technical and laboratory 

capabilities of the SI serve collectively as a national and international testbed for interoperability. In 

researching and influencing interoperability standards, SI staff are among the best internationally. This 

1 National Institute of Standards and Technology, “The Engineering Laboratory—Summaries of Our Activities, 

Accomplishments and Recognitions,” Gaithersburg, Md., July 2014.

55

 
has been recognized through awards such as the PDES, Inc.2 award to a division staff member for 

excellence in the technical management and outstanding contributions to the STEP file analyzer program 

and for playing a leading role in accelerating the development and implementation of advanced 

capabilities in support of model-based manufacturing.

The SID has been a leader in the STEP standard’s development and its dissemination to industry 

and has created a system for measuring conformance of the system integration model-based enterprise 

(MBE) product manufacturing information (PMI) validation program. 

SI staff have been actively supporting the system integration and upgrading of standards related 

to enterprise integration; supporting pilot projects that include small and medium-sized businesses for 

new standards and enterprise integration; supporting the development, testing, promulgation and adoption 

of standards; developing a toolkit and training materials to permit small and medium-sized businesses to 

participate in an integrated enterprise; working with companies and associations to raise awareness of 

system integration and standards; and disseminating the information by participating in the workshops, 

meetings, and conferences. 

Opportunities and Challenges

The SI program has no laboratory testbeds, which are needed for the division to demonstrate its 

work. Travel restrictions present a serious challenge for the staff, curtailing their ability to connect with 

peers and the industry to understand changes in the technological landscape. There is a need to develop 

division- and project-level performance metrics and tracking methods.

Overall Assessment

The SI program has developed strong expertise in several key areas, including system integration 

and STEP standards. The SI program has had significant impact on relevant standards and has been very 

proactive in standards bodies—for example, STEP, the International Council on Systems Engineering

(INCOSE), Object Management Group (OMG), World Wide Web Consortium (W3C), Automotive 

Industry Action Group (AIAG), and ASME—by providing expertise and best practices and by holding 

workshops. The research projects are focused on appropriate technologies relevant to industry challenges. 

The quality of the programs is high. The SI program has no testbeds, which are needed to demonstrate its 

work. Travel restrictions present a serious challenge for the staff.

Sustainable Manufacturing

Sustainable manufacturing is a complex area involving many perspectives. The sustainability 

issue needs to be evaluated in a global framework.

The work presented focused primarily on energy use by manufacturing processes. This is a 

reasonable area of inquiry if it is clearly referenced to the larger framework. For example, for a gate-togate energy assessment, important parts of the analysis would include materials use, factory energy use, 

supply chain issues, and electric power generation. Under differing circumstances, any of the parts of the 

gate-to-gate analysis could surpass the process energy use. Even the gate-to-gate approach needs a larger 

framework. Nevertheless, manufacturing processes are legitimate targets for close scrutiny with respect to 

methods to assess the use of energy and the emission of carbon. 

2 PDES, Inc., is an international industry/government/university consortium committed to accelerating the 

development and implementation of standards-enabling enterprise integration and product life -cycle management

interoperability for its member companies.

56

 
The SID team developed a system for classifying energy use for manufacturing processes using a 

faceted system approach. The classification system categories seem reasonable, but further demonstration 

of their utility and possible adoption by industry are needed before a clear understanding of the scheme 

can be achieved. 

Two process-level energy models were addressed: injection molding and welding. These models 

appear to be mechanism-based models rather than equipment component-based models. The advantage of 

mechanism models is that they are simpler and intuitive. However, these models contain numerous 

adjustable parameters, such as mechanism efficiencies, that practitioners would need to measure. Rather 

than trying to model all manufacturing processes, an enormous task, the SID team could consider 

developing protocols for measuring equipment energy. For example, they could consider equipment 

measurement under different production volumes and parameter settings. Such an activity could serve as 

a testbed for this division. A standard scheme for equipment measurement might provide a very useful 

contribution to sustainable manufacturing. Pursuit of this activity could require supplementing current 

skills in the division with expertise in hardware and measurement. Such a scheme needs to be developed 

in a manner that allows it to be conducted in-house during production activities. Sustainable 

manufacturing, even if constrained to gate-to-gate analysis, involves far more than equipment 

characterization for energy use. Material use is arguably more important than equipment use and needs to 

be considered along with the factors previously mentioned. The task can seem overwhelming and begs for 

a framework that can partition the problem into identifiable and tractable portions. 

The SID’s efforts to collaborate with the ASTM and the ISO on the development of standards for 

sustainable manufacturing are commendable. Current sustainability reports by manufacturing companies 

are extremely varied and often nearly impossible to interpret. A common confusion is the mixing up of 

sustainability and corporate social responsibility. Uniform reporting on sustainable manufacturing is a 

very important need in this area, and this is a problem that the division could address.

PORTFOLIO OF SCIENTIFIC EXPERTISE

Accomplishments 

The diversity and expertise of the SID’s workforce is very good. Many of the experts in the 

division are well known in their field. The division has demonstrated excellent connectivity with industry 

and participation on standards committees. This contributes to enhancing the knowledge of the staff as 

well as the connectivity of the division to external activities, and it supports the division’s technology 

transfer and dissemination activities.

Opportunities and Challenges 

SID staffing seems to be sufficient to support its current projects. However, with a potential 

increase in projects, such as in the sustainable manufacturing area, key expertise will be needed to 

enhance and expand the division’s resources. 

FACILITIES, EQUIPMENT, AND HUMAN RESOURCES

Opportunities and Challenges

There are no testbeds associated with the SID. Most of the division’s activities involve software 

development, modeling, or analysis. However, many of the division’s activities include complex systems 

that require validation and demonstration tools. Having a testbed devoted to these activities could be of 

57
great value to the division’s activities in the development, validation, demonstration, and transfer of 

technology. 

DISSEMINATION OF OUTPUTS 

Accomplishments

The division’s dissemination and outreach activities are focused on publications and participation 

in conferences and professional events and forums. The publications list for the division is extensive and 

shows significant activities and contributions. The publications are of high caliber, and the subjects 

addressed are appropriate.

Opportunities and Challenges 

Other forms of dissemination can add value. For example, providing community colleges with 

assistance in instructing students in model-based engineering and specifically STEP would be of high 

value in helping to train young professionals on STEP and model-based engineering.

FINDINGS AND RECOMMENDATIONS

The division needs to expand its sustainable manufacturing activities and develop division- and 

project-level performance metrics for this area; periodically develop and evaluate its research portfolio, 

with the help of external experts; more effectively articulate its accomplishments; ensure, to the extent 

possible, that travel policies do not restrict business activities; assess recruitment and retention practices 

to ensure they are able to recruit and retain talented researchers; and establish a testbed to help in 

development, validation, demonstration, and technology transfer.

Recommendation: The Systems Integration Division should establish a testbed to support 

development, validation, demonstration, and technology transfer.

58
7

Key Findings and Recommendations

Key findings and recommendations for the Engineering Laboratory’s divisions are provided here. 

MATERIALS AND STRUCTURAL SYSTEMS DIVISION

Research performed in the Inorganic Materials Group of the Materials and Structural Systems 

Division (MSSD) supports life prolongation and rehabilitation of concrete materials and improves 

sustainability by decreasing cement content. It has applications in the hydrocarbon exploration industry, 

which uses concrete to encase well bores. Proper formulation of concretes that can properly set up and 

seal the well bore is essential for both increased production and decreased leakage. 

It is important that the group establish better linkages between chemical and physical 

degradation and the mechanical properties of construction materials and that it consider developing a 

method for noninvasive in situ testing of the strength of fresh concrete. There is a need for the division to 

conduct further research into bio-based materials and fiber-reinforced structural systems, as well as into 

ubiquitous existing materials (steel, masonry, and wood) for new construction and in situ rehabilitation, 

for durability, functional performance, and resilience. 

Recommendation 1. The Inorganic Materials Group should leverage work for the Nuclear 

Regulatory Commission on alkali-silica reaction to connect the nanoscale investigations of 

concrete deterioration to structural performance.

Recommendation 2. The Inorganic Materials Group should engage the oil and gas industry 

to develop safe encasement of well bores.

The work by the Polymeric Materials Group on the photophysics of polymeric degradation is 

mature. The group needs to engage the solar panel industry where polymeric materials are used to 

encapsulate thin films between panes of glass. Standard materials used by the industry are ethyl vinyl 

alcohol and polybutadiene, which are problematic and require guidance for formulating sealants and 

adhesives. It would be beneficial if the polymer and concrete teams were to increase their collaboration—

for example, polymer fibers to strengthen concrete and resist cracking and polymer additives to enhance 

the flow properties of concrete.

Recommendation 3. The Polymeric Materials Group should expand research, including 

molecular modeling, on polymeric degradation to non-ultraviolet degradation effects and 

should include the data in models for predicting service life.

Recommendation 4. The Polymeric Materials Group should engage the solar panel industry 

where polymeric materials are used to encapsulate thin films between panes of glass.

59
Work of the Structures Group could be improved by collaborating with the research and 

practitioner community to identify and evaluate alternative approaches for quantitative metrics to describe 

phenomena such as disproportionate collapse. A metric for robustness would be useful. Additional 

collaboration with the Fire Research Division (FRD) on the durability and performance of materials and 

structures under fire conditions would also be beneficial. Fully integrating the group’s capabilities in 

analysis, computational modeling, and physical testing would help to obtain the maximal amount of 

information from the work in disproportionate collapse. 

Much of the work of the Structures Group is disseminated through the code development process. 

However, significant results, such as those on plasticity found through the work on disproportionate 

collapse, could be disseminated more broadly through technical and academic journals. 

The concept of a virtual wind tunnel based on historic data, new wind speed maps, and 

computational modeling would be valuable for the group studying wind research. It would also be 

beneficial to consider in future work the potential of microbursts to cause structural damage.

Recommendation 5. The group studying wind research should develop the concept of a 

virtual wind tunnel, based on observations and computational modeling.

The group studying community disaster resilience needs to monitor its staffing requirements to 

assure that human resources are adequate and in the appropriate disciplines for the magnitude and scope 

of the tasks assigned. The group also needs to develop and publicize an explicit roadmap to define where 

and how the work of the group will mesh with existing and future codes and standards and identify who 

will benefit from these efforts. A useful roadmap would also make clear the overarching governmental 

interest in the program’s contributions to improving economic impacts and enhancing life safety during 

and following extreme events. 

The group’s efforts in disaster resilience could beneficially include partnerships across NIST and 

externally (e.g., those involved with research in the wildlife-urban interface, earthquakes, and post-flood 

indoor air quality) for science-based resilience performance and standards. The group could leverage 

existing U.S. and international research, methodologies, and tools to accelerate advancements in theory 

and practice, including standards and measurements. Operations and maintenance considerations are good 

candidates for incorporation into the high-level objectives for the Community Disaster Resilience 

Program.

The excellent capabilities in the group and the findings of the National Construction Safety 

Team’s field-based failure studies can be integrated to inform the research program and identify critical 

research topics. Incorporating into the resilience program factors relating to the environment and natural 

systems may require new partnerships—with, for example, the Environmental Protection Agency’s 

Green Infrastructure Collaborative.

INTELLIGENT SYSTEMS DIVISION

The Intelligent Systems Division (ISD) measurement science research programs aim to advance 

the versatility of intelligent automation technologies for smart manufacturing and cyberphysical systems 

applications; enable performance optimization of smart manufacturing systems; and enable rapid, costeffective production of products through advanced manufacturing processes and equipment. 

The division’s work on wireless factory networks addresses an important need, as does the 

project on cybersecurity of industrial control systems. The objective of the Smart Manufacturing 

Construction Systems (SMCS) program is to develop and deploy advances in measurement science for 

sensing, modeling, and optimizing manufacturing activities in a smart factory. In its factory cybersecurity 

project, the division has demonstrated leadership in addressing cybersecurity in smart factory systems and 

other industrial cyberphysical systems. Indicative of this leadership is the recent publication of a new 

draft of NIST SP 800-82, Guide to Industrial Control Systems (ICS) Security. The ISD technical staff also 

60
led the development of IEEE Standard 1588, “Precision Clock Synchronization Protocol for Networked 

Measurement and Control Systems.” Current industrial robots cannot operate safely with humans, and so 

they need to be isolated. This limits their applicability and increases the costs of automation. The NextGeneration Robotics and Automation (NGRA) program is beginning to develop performance evaluation 

methods and standards for safe robot-human interactions. The division has started the standardization 

effort on the safety of automated guided vehicles (AGVs). The laboratory is extending safety evaluation 

to other systems, attempting to cover various aspects of evaluating robotic hand capabilities, such as 

position, torque, grasp types, graspable object size and ranges, touch location, force, and pressure via 

tactile sensing.

The decision in fiscal year 2014 to terminate the smart machining activities and focus on 

measurement science for additive manufacturing may risk the loss of critical capability and recognized 

expertise in smart machining. Advanced manufacturing needs both additive and subtractive processes. 

Subtractive manufacturing processes typically refer to conventional manufacturing processes by which 

material is removed—for example, from a billet, to make a part and achieve final desired geometries and 

tolerances. Additive manufacturing processes refer to a suite of methods to add material, typically in 

powder or wire form, to make a part, add features to a part, or to repair a part. A hybrid approach 

consisting of both additive manufacturing and smart machining is needed to support the development of 

measurement science for additive manufacturing processes.

New additive manufacturing machines with enhanced features and capabilities are introduced to 

the market each year. There are several laser powder bed fusion of metal machines, many with higher 

performance lasers and optics, controlled environmental chambers, automated powder delivery, inprocess monitoring, powder compaction, and other features and tools to further advance measurement 

science knowledge. These new machines are being purchased by industry, academia, research institutions, 

and government entities globally. It may be difficult for industry and academia to see the ISD at the 

forefront of measurement science when its equipment is not adequate to the task. The current machines at 

the ISD are adequate for the initial work and developing expertise. Ownership of and/or access to 

machines with capabilities and equipment that can detect and measure defects of additive manufactured 

parts in situ or postprocess would support the ISD’s goal of enabling widespread adoption by the U.S. 

manufacturing industry of additive manufacturing processes for metal. 

The ISD is appropriately planning and executing programs in new technology areas such as 

SMCS, additive manufacturing, industrial robotics, and cyberphysical systems. In addition, the division 

has seen an explosion of government and global industry interest in its work on ICS cybersecurity, with 

more than 2.5 million downloads from the NIST Internet site of this year’s revision of the Guide to 

Industrial Control System (ICS) Security (NIST SP 800-82). The division currently has one staff member 

who is widely recognized for deep expertise in this area and who is spread very thin in responding to 

current program demands. The division is trying to hire at a time when ICS cybersecurity experts can 

demand high salaries and other benefits from industry. Because the need to staff ongoing and new 

programs is urgent, the division cannot wait and hope for qualified, affordable candidates to become 

available. A strategy is needed to hire now, where possible (by emphasizing factors beyond salary alone 

that make NIST employment attractive), to invest in developing the needed multidisciplinary skills in 

current staff and entry-level hires, and to provide interim support through contracts and cooperative 

research arrangements with universities and others. 

The ISD would benefit from an extensive data set supporting its additive manufacturing efforts. 

The pilot round-robin tests that the division and its partners have been engaged in address build-to-build 

variation and machine-to-machine variation of the same machine model. Building on the pilot roundrobin testing with equipment manufacturers, manufacturing companies, academia, and government 

institutions that are working with different laser powder bed fusion machines or materials would help 

generate a larger data set and address machine-model-to-machine-model variation. The ISD’s approach 

can be standardized and disseminated widely.

61
Recommendation 6. The Intelligent Systems Division should build on the pilot round-robin 

testing with certified sources to generate a large data set for their additive manufacturing 

efforts.

ENERGY AND ENVIRONMENT DIVISION

The Energy and Environment Division (EED) has been focusing on the measurement of energy 

performance and indoor air/environment quality. The EED has two major programs that support its goal 

of sustainable and energy-efficient manufacturing, materials, and infrastructure. The Net-Zero Energy, 

High-Performance Buildings Program focuses on energy-efficient building systems and advancing the 

measurement science for their characterization and performance assessment; the Embedded Intelligence 

in Buildings Program focuses on the development and application of intelligent information and control 

technologies to improve building operation.

Metrics are essential for evaluating indoor environment quality (IEQ) and energy performance, 

and sustainability of buildings remains a major challenge for the field. A single index does not exist for 

quantifying IEQ. Many factors need to be considered, including thermal environment, pollution and noise 

levels, lighting quality, and occupant satisfaction. It is important to develop an accepted means of 

quantifying IEQ benefit that can be integrated with the more readily measured energy benefit. Such 

integration includes developing a comprehensive framework for modeling and simulating whole building 

performance, based on the fundamental understanding of the combined heat, air, moisture, and pollutant 

transport processes in building systems. The transport processes are affected by material characteristics, 

properties of pollutant species, and environmental conditions. With its capabilities in laboratory 

measurements and modeling, the EED is in position to lead a national effort in such a development. 

Overall, the EED exhibits excellent capabilities in the areas of current demands such as energy 

management and metrics, but the scope of its expertise may need to be expanded to include critical IEQ

areas such as management of and metrics for moisture/mold and nanoparticles.

Improvements in methods and procedures are needed to obtain reliable field-scale data for 

validating the models and measurement methods developed from laboratory studies. Many existing field 

studies do not have enough rigor to generate reliable data. A standard, scientifically based test protocol 

for field use is needed to catch up with EED laboratory research to capitalize fully on the laboratory 

program.

Recommendation 7. The Energy and Environment Division should continue to develop 

metrics for evaluating indoor air quality and energy performance concurrently.

Recommendation 8. The Energy and Environment Division should perform field

investigations to collect reliable data for validating the models and measurement methods

developed from the laboratory studies.

SYSTEMS INTEGRATION DIVISION

The Systems Integration Division (SID) contributes to measurement science and standards 

needed to integrate engineering information systems used in manufacturing, construction, and 

cyberphysical systems. Areas of work include manufacturing enterprise integration; green manufacturing 

and construction; engineering and manufacturing products, processes, equipment, technical data, and 

standards; collaborative manufacturing research under pilot grants; research performed through 

62
manufacturing fellowships; systems integration and engineering; life-cycle assessment; cyberphysical 

systems; productivity measurement; sustainability; and energy efficiency.

1

SID staff have developed strong expertise in several key areas, including standards development 

for systems engineering, model-based engineering, development and support of a Standard for Exchange 

of Product (STEP) standards (ISO 10303-242 [STEP AP242]), composite manufacturing, additive 

manufacturing, and assisting vendors in implementation of the new functionality.

SID staff have developed process-level energy models for injection molding and welding. These 

models appear to be mechanism-based rather than equipment component-based models. However, such 

models contain numerous adjustable parameters, such as mechanism efficiencies, that practitioners need 

to measure. Rather than trying to simulate all manufacturing processes (an enormous task), the SID team 

could consider developing equipment energy measurement protocols. Such an activity could serve as a 

testbed for the division. A standard scheme for equipment measurement might be a very useful 

contribution to sustainable manufacturing. 

Recommendation 9. The Systems Integration Division should establish a testbed to support 

development, validation, demonstration, and technology transfer.

FIRE RESEARCH DIVISION

The FRD has made progress toward improving the safety and effectiveness of firefighters through 

measurement science to advance suppression tactics, examining nontraditional means of fire suppression, 

and transferring the results to the fire service. Field-scale burn tests have been accomplished through 

outside funding opportunities, collaboration, and partnerships. 

The efforts to communicate research findings on fire dynamics have generated much interest and 

discussion within the professional fire service community. Presentations, articles in fire service print and 

online media providing opportunities for fire service organizational involvement, and, most recently, the 

use of social media have all contributed to this success. 

The wildland-urban interface (WUI) problem is growing very fast, and its growth may accelerate 

due to climate change and diminishing safety margins between wildland and the built environment. 

Because the WUI problem is relatively recent, there is little information about WUI fire characteristics 

and how to reduce its consequences. Therefore, it is of high priority to collect data from WUI fires to 

underpin standards and codes for fire-hardening structures located in WUI-prone areas. Field studies, 

some of which have been conducted by the NIST National Construction Safety Team, provide essential 

data on WUI fires, including the finding that ignition of structures by embers accounts for up to 50 

percent of the structure fires in a WUI. 

The Fire Dynamics Simulator (FDS) modeling team has developed new goals for the future. One 

of these is the coupling of three types of complex codes (gas-phase fluid dynamics and reaction, thermal 

transport to and through solid structures, and failure of solid structures under load and thermal stress). 

This is a significant effort in computation, validation, and interpretation of results. 

The National Fire Research Laboratory (NFRL) is an exciting new facility for the FRD and the 

EL. It will provide a unique capability to consider the fire heating of structures under structural load and 

will enable the FRD team to break new ground. The ability to test structures exposed to fires while they 

are under realistic gravity loads is an exciting new capability. The community expects that findings from 

these tests will form the basis for new building requirements. The FRD conducted a planning workshop to 

establish long-term priorities for testing so that when commissioning is complete testing can begin 

promptly.

1 National Institute of Standards and Technology, “The Engineering Laboratory—Summaries of Our Activities, 

Accomplishments and Recognitions,” Gaithersburg, Md., July 2014.

63

 
Recommendation 10. The Fire Research Division should complete the extremely important 

new National Fire Research Laboratory and prioritize the test series for the first 2 years 

following the laboratory’s commissioning. 

Recommendation 11. The Fire Research Division should strengthen its research on 

wildland-urban interface fires, including data collection from real events and new modeling 

approaches.

64
Acronyms

ACCA Air Conditioning Contractors of America 

ACI American Concrete Institute

AEO Applied Economics Office

AGV automated guided vehicle 

AHAM Association of Home Appliance Manufacturers 

AHRI Air-Conditioning, Heating, and Refrigeration Institute 

AISC American Institute of Steel Construction 

ANSI American National Standards Institute

APAR-DD air handling unit performance assessment rule—dual duct 

ASCE American Society of Civil Engineers 

ASHRAE American Society of Heating, Refrigerating, and Air-Conditioning Engineers 

ASME American Society of Mechanical Engineers 

ASTM American Society for Testing and Materials

BACnet building automation and control network

BEES building for environmental and economic sustainability 

BEMS building energy management systems

BIRDS building industry reporting and design for sustainability

CFAST consolidated model of fire and smoke transport 

CFD computational fluid dynamics 

CIB International Council for Building 

CPSC Consumer Product Safety Commission

DARPA Defense Advanced Research Projects Agency

DHHS Department of Health and Human Services

DOE Department of Energy

EED Energy and Environment Division 

EL Engineering Laboratory

ERDC Engineering Research and Development Center

FAA Federal Aviation Administration

FAR Federal Acquisition Regulations 

FDD fault detection and diagnostics

FDNY Fire Department of New York City

FDS Fire Dynamics Simulator 

FEMA Federal Emergency Management Agency

FRD Fire Research Division 

FSGIM facility smart grid information model

GSA Government Services Administration

HVAC&R heating, ventilation, air conditioning, and refrigeration

IAFC International Association of Fire Chiefs

IAFF International Association of Fire Fighters

65
IAQ indoor-air quality 

ICC International Code Council 

ICS industrial control systems

IEC International Electrotechnical Commission 

IEQ indoor environment quality 

IFSI Illinois Fire Service Institute

IFSTA International Fire Service Training Association 

INCOSE International Council on Systems Engineering

ISD Intelligent Systems Division 

ISO International Organization for Standardization

MBE model-based enterprise

MSSD Materials and Structural Systems Division

NASA National Aeronautics and Space Administration

NEMA National Electrical Manufacturers Association

NFPA National Fire Protection Association 

NFRL National Fire Research Laboratory 

NGRA next- generation robotics and automation 

NIOSH National Institute of Occupational Safety and Health

NIST National Institute of Standards and Technology 

NRC National Research Council

NZERTF Net-Zero Energy Residential Test Facility

OA other agency

OMB Office of Management and Budget

OMG Object Management Group

PMI product manufacturing information

R&D research and development

SCBA self-contained breathing apparatus

SFPE Society of Fire Protection Engineers

SI systems integration 

SID Systems Integration Division 

SLICE-ERS size up, locate the fire, isolate the flow path, cool from a safe distance, extinguish, and 

then rescue and salvage

SMCS smart manufacturing construction systems 

SRMs standard reference materials 

STEP Standard for Exchange of Product 

SURF Summer Undergraduate Research Fellowship

TRANSYS transient system simulation 

UL Underwriters Laboratories 

USFA U.S. Fire Administration

USFS U.S. Forest Service

UV ultraviolet 

VOC volatile organic compound

WUI wildland-urban interface 

W3C World Wide Web Consortium 

66