AN ASSESSMENT OF THE 

NATIONAL INSTITUTE OF STANDARDS 

AND TECHNOLOGY 

MANUFACTURING ENGINEERING 

LABORATORY 

FISCAL YEAR 2010 

Panel on Manufacturing Engineering 

Laboratory Assessments Board 

Division on Engineering and Physical Sciences 

THE NATIONAL ACADEMIES PRESS 

Washington, D.C. 

www.nap.edu
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, N.W. Washington, DC 20001 

NOTICE: The project that is the subject of this report was approved by the Governing Board 

of the National Research Council, whose members are drawn from the councils of the 

National Academy of Sciences, the National Academy of Engineering, and the Institute of 

Medicine. The members of the panel responsible for the report were chosen for their special 

competences and with regard for appropriate balance. 

This study was supported by Contract No. SB134106Z011, TO#8, between the National 

Academy of Sciences and the National Institute of Standards and Technology, an agency of 

the U.S. Department of Commerce. Any opinions, findings, conclusions, or 

recommendations expressed in this publication are those of the authors and do not 

necessarily reflect the views of the agency that provided support for the project. 

International Standard Book Number-13: 978-0-309-16161-9 

International Standard Book Number-10: 0-309-16161-4 

Copies of this report are available from 

Laboratory Assessments Board 

Division on Engineering and Physical Sciences 

National Research Council 

500 Fifth Street, N.W. 

Washington, DC 20001 

Additional copies of this report are available from the National Academies Press, 500 Fifth 

Street, N.W., Lockbox 285, Washington, DC 20055; (800) 624-6242 or (202) 334-3313 (in 

the Washington metropolitan area); Internet, http://www.nap.edu. 

Copyright 2010 by the National Academy of Sciences. All rights reserved. 

Printed in the United States of America
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished 

scholars engaged in scientific and engineering research, dedicated to the furtherance of science and 

technology and to their use for the general welfare. Upon the authority of the charter granted to it by the 

Congress in 1863, the Academy has a mandate that requires it to advise the federal government on 

scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences. 

The National Academy of Engineering was established in 1964, under the charter of the National 

Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its 

administration and in the selection of its members, sharing with the National Academy of Sciences the 

responsibility for advising the federal government. The National Academy of Engineering also sponsors 

engineering programs aimed at meeting national needs, encourages education and research, and recognizes 

the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of 

Engineering. 

The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the 

services of eminent members of appropriate professions in the examination of policy matters pertaining to 

the health of the public. The Institute acts under the responsibility given to the National Academy of 

Sciences by its congressional charter to be an adviser to the federal government and, upon its own 

initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president 

of the Institute of Medicine. 

The National Research Council was organized by the National Academy of Sciences in 1916 to associate 

the broad community of science and technology with the Academy’s purposes of furthering knowledge and 

advising the federal government. Functioning in accordance with general policies determined by the 

Academy, the Council has become the principal operating agency of both the National Academy of 

Sciences and the National Academy of Engineering in providing services to the government, the public, 

and the scientific and engineering communities. The Council is administered jointly by both Academies 

and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, 

respectively, of the National Research Council. 

www.national-academies.org 
iv 

PANEL ON MANUFACTURING ENGINEERING 

NEIL A. DUFFIE, University of Wisconsin, Madison, Chair

JAY RAMANATHAN, Ohio State University, Vice Chair

VIOLA L. ACOFF, University of Alabama 

CAROL L. JONES ADKINS, Sandia National Laboratories 

DAVID E. ASPNES, North Carolina State University 

BENITO R. FERNANDEZ, University of Texas at Austin 

MICHAEL R. JAHADI, Lockheed Martin Aeronautics Company 

RICHARD L. KEGG, Milacron, Inc. (retired) 

RANGA KOMANDURI, Oklahoma State University 

THOMAS R. KURFESS, Clemson University 

STELIOS KYRIAKIDES, University of Texas at Austin 

JAY LEE, University of Cincinnati 

EUGENE S. MEIERAN, Intel Corporation (retired) 

A. GALIP ULSOY, University of Michigan 

PAUL WRIGHT, University of California, Berkeley 

Staff 

JAMES P. McGEE, Director 

CY BUTNER, Senior Program Officer 

LIZA HAMILTON, Administrative Coordinator 

EVA LABRE, Program Associate 
v 

Acknowledgment of Reviewers 

This report has been reviewed in draft form by individuals chosen for their 

diverse perspectives and technical expertise, in accordance with procedures approved by 

the National Research Council’s Report Review Committee. The purpose of this 

independent review is to provide candid and critical comments that will assist the 

institution in making its published report as sound as possible and to ensure that the 

report meets institutional standards for objectivity, evidence, and responsiveness to the 

study charge. The review comments and draft manuscript remain confidential to protect 

the integrity of the deliberative process. We wish to thank the following individuals for 

their review of this report: 

Montgomery Alger, Air Products and Chemicals, Inc., 

Dianne Chong, The Boeing Company, 

Harry Cook, University of Illinois (emeritus), and 

David Markle, Consultant, Saratoga, California. 

Although the reviewers listed above have provided many constructive comments 

and suggestions, they were not asked to endorse the conclusions or recommendations, nor 

did they see the final draft of the report before its release. The review of this report was 

overseen by Alton D. Slay, Warrenton, Virginia. Appointed by the National Research 

Council, he was responsible for making certain that an independent examination of this 

report was carried out in accordance with institutional procedures and that all review 

comments were carefully considered. Responsibility for the final content of this report 

rests entirely with the authoring panel and the institution.

vii 

Contents 

SUMMARY 1 

1 THE CHARGE TO THE PANEL AND THE ASSESSMENT PROCESS 9 

2 INTELLIGENT SYSTEMS DIVISION 13 

3 MANUFACTURING METROLOGY DIVISION 17 

4 MANUFACTURING SYSTEMS INTEGRATION DIVISION 21 

5 PRECISION ENGINEERING DIVISION 25 

6 OVERALL CONCLUSIONS 30 

 1 

Summary 

 The mission of the Manufacturing Engineering Laboratory (MEL) of the National 

Institute of Standards and Technology (NIST) is to promote innovation and the 

competitiveness of U.S. manufacturing through measurement science, measurement 

services, and critical technical contributions to standards. This mission is consistent with 

the NIST mission. The MEL is organized into five divisions: Intelligent Systems, 

Manufacturing Metrology, Manufacturing Systems Integration, Precision Engineering, 

and Fabrication Technology (not assessed because it provides instrument and fabrication 

support for NIST researchers). 

 The MEL total staffing for fiscal year (FY) 2009 was 183 (171 full-time 

permanent staff and 12 other staff), and its estimated annual budget for FY 2010 is $48.7 

million ($35.5 million from NIST appropriations, $7.1 million from other agencies and 

external research and development, and $4.8 million from calibration service fees and 

reimbursables; the total excludes shops). The MEL also has 99 guest researchers. For FY 

2006 through FY 2009, MEL total staffing has been 201, 198, 188, and 183, respectively, 

and for FY 2007 through FY 2010 (estimated), MEL total funding has been $51.0 

million, $50.1 million, $49.7 million, and $48.7 million, respectively. 

 The Intelligent Systems Division has 31 NIST (full-time permanent) staff and 14 

guest researchers (full-time-equivalent). Its FY 2010 estimated funding is about $8.9 

million, with about 38 percent coming from extramural sources, and it has 3 programs 

containing 12 projects. The Manufacturing Metrology Division has 33 NIST staff, 7 guest 

researchers, and 2 postdoctoral researchers. Its FY 2010 estimated funding is about $9.9 

million, with about 17 percent coming from extramural sources (including reimbursable 

services). It has 3 programs containing 20 projects. The Manufacturing Systems 

Integration Division has 26 NIST staff, 1 part-time permanent worker, 34 NIST 

associates, and 1 postdoctoral researcher. Its FY 2010 estimated funding is about $9.5 

million, with about 7 percent coming from extramural sources, and it has 2 programs 

containing 10 projects. The Precision Engineering Division has 35 NIST staff, 22 guest 

researchers, and 1 postdoctoral researcher. Its FY 2010 estimated funding is about $12.7 

million, with about 23 percent coming from extramural sources, and it has 3 programs 

containing 19 projects. 

 A panel of experts appointed by the National Research Council (NRC) has 

assessed the four divisions. Panel members visited these divisions and reviewed their 

activities. The scope of the assessment included the following three criteria: (1) the 

technical merit of the current MEL programs relative to current state-of-the-art programs 

worldwide; (2) the adequacy of the MEL budget, facilities, equipment, and human 

resources, as they affect the quality of MEL’s technical programs; and (3) the degree to 

which MEL programs in measurement science, standards, and services achieve their 

stated objectives and desired impact. 

 In this report, the summary assessment of the MEL is given, followed by a 

summary assessment for each division. Chapters 1 through 6 present a description of the 

assessment process, detailed assessments of the divisions, and overall report conclusions. 
2 

SUMMARY ASSESSMENT OF THE LABORATORY 

 The Manufacturing Engineering Laboratory continues to excel in measurement 

science, measurement services, and technical contributions to standards. The MEL is 

achieving its mission, making crucial contributions to innovation and to the 

competitiveness of U.S. manufacturing. From a national perspective, MEL activities such 

as standards development and calibration services are providing baselines for many 

sectors of the economy. On an international level, the MEL is enabling industries to relate 

standards used in the United States to those used in other countries; this capability is 

necessary and critical for the participation of U.S. industries in international commerce. 

Research by MEL staff is on the forefront of measurement sciences, enabling industry to 

develop and deliver products of ever-higher quality and complexity to world markets and 

enabling future innovative manufacturing industries and processes. It is essential that the 

MEL maintain the integrated level of effort required for success in this broad spectrum of 

activities. 

 Across the MEL, project management and portfolio management strategies have 

continuously improved. The portfolio of projects is increasingly customer-focused 

(taking into account present and future customers) and well distributed technically in 

terms of high and low risk and short- and long-term payoff. Although this achievement is 

noteworthy, the MEL will need to address looming challenges in measurement science, 

measurement services, and standards in areas such as transformable, high-precision 

factories that operate beyond the current state of the art in accuracy and quality1

 and new 

manufacturing processes and products with decreasing dimensional scale and integrated 

biotechnology.2

 To address these challenges will require more coupling of critical 

project selection with technical staff and equipment development than the MEL currently 

has done. 

Measurement science, measurement services, and standards that are well beyond 

the current state of the art are required to ensure the future competitiveness of U.S. 

industry in sustainable, high-value-added manufacturing with transformable factories that 

produce with high precision using manufacturing processes and products with everdecreasing dimensional scale. This is also true for information technology (IT) for 

manufacturing, including knowledge bases and “smart” products and factories, and for 

biotechnology for manufacturing, including biological product components and 

biologically derived process components. Closer coupling into the MEL of other 

technology areas of NIST (IT and materials, for example) and closer coupling into the 

MEL of external expertise (in the application of biological processes and materials in 

manufacturing, for example) are necessary for addressing these areas and preparing for 

the future. The technical expertise of the MEL staff, its equipment, and its relationships 

with other laboratories must evolve quickly to meet the future needs of industry in areas 

such as these. 

 The MEL is having a significant impact on today’s manufacturing industries, 

ranging from traditional production industries to optics industries to semiconductor 

 1

 Francesco Jovane, Engelbert Westkämper, and David Williams, The ManuFuture Road: Towards 

Competitive and Sustainable High-Adding-Value Manufacturing. Berlin: Springer-Verlag 2009. 2

 National Research Council, Visionary Manufacturing Challenges for 2020. Washington, D.C.: 

National Academy Press, 1998. 
3 

industries, and it is making significant enabling contributions to future U.S. 

manufacturing industries. Examples are numerous, including the following: 

 An industrial Ethernet network performance test tool that allows vendors and 

users to validate network performance before deployment; 

 NIST SP 800-53, Recommended Security Controls for Federal Information 

Systems, and NIST SP 800-82, Guide to Industrial Control Systems Security;

 Standard for the Exchange of Product model data (STEP); 

 Online testing services that enable software vendors to validate the 

conformance of their products to standards; 

 Standards for wireless sensors; 

 Techniques for using deoxyribonucleic acid (DNA) molecules as an intrinsic 

small-force standard; 

 Improvement of mass metrology and calibration by tying the current air-based 

kilogram definition to the vacuum-based alternative definition; 

 Development of an atomic force microscope (AFM) for line roughness 

measurements crucial for 22 nm lithography; and 

 Atomically sharp tip nano-writing to be used in developing standards for 

dimensions on the order of a few atoms that are traceable to NIST. 

The MEL is recognized as the world leader in many of these areas. 

 The MEL has excellent research facilities and equipment. In general, its 

equipment is state of the art or beyond, well maintained, and operated by knowledgeable 

experts. However, its equipment will age quickly, constraining the ability of the 

laboratory to respond to changing technology. Significant upgrades in metrology 

equipment will be required in the near future to update existing systems and put in place 

next-generation systems that are more reliable and that cost less. However, the Advanced 

Measurement Laboratory (AML)3 has provided best-in-the-world facilities for equipment 

that in many cases is the most advanced available. The MEL also will benefit from the 

construction of a new robot test facility, which will be an important national resource for 

this rapidly developing industry. However, other countries also are making major 

investments in such facilities. The MEL successfully leverages many of its programs 

through partnerships, primarily with federal agencies but also with key industries. The 

budget available for capital acquisitions seems to be inadequate for such a capitalintensive laboratory. 

 MEL staff are doing high-quality, innovative work. While progress has been made 

in hiring permanent scientific and engineering staff in key areas, the MEL remains 

significantly underfunded for personnel, with stagnant budgets exacerbated by increasing 

costs. Current staffing is highly senior, and a lack of new positions makes bringing in 

junior people difficult. Such issues continue to result in stop-gap reliance on guest 

researchers. Postdoctoral salaries are inadequate for engineering fields, making it difficult 

to attract the best candidates in critical technical areas—candidates who are potential 

 3

 The AML is a NIST facility that is used by multiple NIST operating units. Within the MEL, the 

Precision Engineering Division and the Manufacturing Metrology Division have laboratories in the AML. 
4 

future permanent staff. Several MEL staff have been in acting positions for a long time, a 

situation likely to affect career paths and threaten overall staff retention and development 

and hence the technical capability and competence of the laboratory. In response to such 

issues, NIST should critically assess its commitment to and support of manufacturing, in 

line with President Obama’s placement of manufacturing as a top national priority and its 

importance in job creation and increased competitiveness in U.S. industry. 

SUMMARY ASSESSMENTS OF FOUR DIVISIONS 

Intelligent Systems Division 

 The Intelligent Systems Division (ISD) develops the measurements and standards 

infrastructure needed for the application of intelligent systems. It has well-established 

core competencies in standards, performance evaluation, measurement, interoperability, 

safety, and security. The ISD is serving as a catalyst in promoting collaboration between 

industries, with a focus on standards and testbeds, and it has established new standards 

for the security of industrial control systems. The ISD’s technical capabilities are among 

the best in the world in STEP-NC (Standard for the Exchange of Product model data: 

Numeric Control; standard for programming machine tools) and OMAC (Open Modular 

Architecture Controller) for real-time data models, machine compensation, machining 

tool path optimization, Ethernet/Internet Protocol (IP) performance testing, and other 

applications. The division has established industrial controls and networks standards for 

federal government and industrial users. NIST SP 800-53 was established for security 

controls for federal information systems, and NIST SP 800-82 was established as a guide 

to industrial control systems security. These allow the federal agencies and the private 

sector to determine the proper security controls for their IT systems as well as to secure 

their industrial control systems effectively while addressing their unique requirements. 

The ISD is recognized as a world leader in the performance evaluation of complex, 

intelligent systems. 

 The work of ISD staff is sound and state of the art; yet, the work is small in a field 

of rapidly evolving technology and would benefit from the closer integration into the 

MEL of other NIST IT resources. The ISD has brought its strengths in measurement, 

performance evaluation, standards, and interoperability to bear on important related areas 

such as the security and safety of industrial control systems. Strengthened global 

benchmarking and self-assessment activities by the division should be performed to 

identify gaps and drive future project selection. The ISD could have broader impact 

through consortium development, further leveraging industry involvement and 

investment to accelerate the dissemination of developed standards. The ISD has excellent 

research facilities and equipment; however, many other places around the world (e.g., 

Germany, Japan, Korea, Singapore, Europe) are making major investments in the 

intelligent systems area, and ISD facilities and activities are no longer unique and 

unrivaled.
5 

Manufacturing Metrology Division 

 The Manufacturing Metrology Division (MMD) programs cover mechanical 

metrology, metrology for advanced optics, and measurements and standards for sciencebased manufacturing needed to promote innovation and international trade by U.S. 

industry. The MMD is supporting standards from cradle to grave and rebirth, including 

the development of new standards, the calibration of artifacts to the current standards, 

and new concepts that may become the fundamental foundation on which next-generation 

standards are built. The MMD is making measurements that could not be done before by 

improving existing technologies and developing new approaches. It also is enhancing 

current capabilities through automation to improve accuracy and to reduce uncertainty, 

costs, and turnaround time. The programs compare very favorably with peer activities at 

the national standards institutions of other countries such as England and Germany. 

MMD staff members are key participants in standards committees for such areas as mass 

metrology and wireless sensors, and they actively disseminate the results to a wide 

audience. 

 The research portfolio of the MMD is well distributed, from mature to emerging 

technologies and from low-risk projects such as automating calibration capabilities, to 

high-risk projects such as utilizing DNA molecules to serve as an intrinsic small-force 

standard. The MMD staff is engaged in high-quality, innovative work. The division is 

increasingly customer-driven and has improved its process for selecting new projects and 

focus areas. Much of its metrology infrastructure is aging and will need to be updated in 

the near future.

Manufacturing Systems Integration Division 

 The Manufacturing Systems Integration Division (MSID) is responsible for 

working with industry to develop and apply software interoperability and standards for 

both product innovation and life-cycle management. The MSID is also charged with 

developing the metrics and standards for the manufacturing information and knowledge 

essential to meet today’s highly integrated, distributed, and complex supply-chain 

environment. The division is building on its strengths and taking a longer-term approach 

in which the knowledge representation of manufacturing processes and products is 

expected to be an enabler for new manufacturing capabilities. MSID’s efforts follow the 

successful deployment of STEP standards for the distribution of engineering knowledge 

for machine parts. According to industry customers, the results of MSID’s research and 

development (R&D) are highly useful in paving the way for efficient, low-cost, highvolume manufacturing for the particular segment of manufacturing being looked at, and 

they are likely to be equally useful across many domains of manufacturing and even the 

full business enterprise. However, the R&D has been more focused on traditional 

industries such as the automotive industry than on the semiconductor and 

pharmaceuticals manufacturing industries. 

 Unfortunately, largely owing to human resource limitations, only a few selected 

areas can be worked on at the current time; nonetheless, the MSID has made impressive 

strides in those areas on which it is focused. In order to prove the viability of the 

knowledge representation, models of different aspects of manufacturing are created and 
6 

tested (e.g., the supply chain, shipping, processing, etc.), and these are discussed with 

potential customers (manufacturers, suppliers of equipment and materials, transportation 

companies, etc.) at well-attended workshops. Simulation of various aspects of 

manufacturing are carried out and compared to real manufacturing output to determine 

whether the assumptions and rules are justifiable and whether the model output is 

accurate, timely, and worthwhile. 

Precision Engineering Division 

 The Precision Engineering Division (PED) conducts R&D in precisionengineered length-metrology-intensive systems in both measuring and production 

machines, aiming for both high-accuracy measurement results and first-principles 

analysis of measurement systems. The division maintains many activities and services, 

many or possibly most of which are among the best in the field. With a relatively small 

level of funding, the PED successfully performs a critical role for the nation, carrying on 

in the ever-changing and challenging legacy of NIST. 

 The PED provides the foundation for dimensional measurements ranging over 12 

orders of magnitude (from kilometers to nanometers). Thus the work of this division is 

crucial to the current and future competitiveness of U.S. industry and in the development 

of standards traceable to NIST. The PED staff is enthusiastic about its work, dedicated 

and knowledgeable. 

 The PED divides its work into four groups, basically by length scale: nanometer 

(nm) to micrometer (m), micrometer to millimeter (mm), millimeter to meter (m), and 

greater than 1 m. These groups have primary responsibilities within these scales but 

interact strongly and provide expertise throughout the PED. 

 The PED provides NIST a remarkable and cost-effective interface with the 

outside world. Its impact is clearly evident. This is seen, for example, from the following: 

(1) continuous requests for its services from industry and government that far exceed the 

capacity of the PED to meet, (2) interest in technical work presented at meetings, 

(3) demonstrated collaborations with a variety of industrial concerns, (4) citations 

garnered by publications, and (5) the acquisition of equipment either donated or 

purchased at bargain prices. 

 Since the previous assessment by an NRC panel in 2008, the PED has made major 

strides in several areas. It is maximizing the advantages of the new Advanced 

Measurement Laboratory. By reducing artifacts such as vibration and temperature 

variations, the AML environment has enabled the limiting capabilities of machines to be 

assessed directly and routinely, avoiding the need to compensate for the environment. In 

helium-ion microscopy, the PED is operating as a beta site for advanced equipment. It is 

also moving into highly relevant new areas, such as metrology that is essential for fuel 

cells, the development of atomic-scale metrologies traceable to the meter, and new 

methods for detecting defects for next-generation integrated-circuits technology. The 

application of the PED’s coordinate measuring machines (CMMs) such as the Moore 

M48 provides unique capabilities for performing accurate measurements on difficult 

objects—for example, assisting the U.S. Army by making measurements of local damage 

on body armor impacted by projectiles to an accuracy of 0.1 mm. Overlay metrology, as 

done in the PED, is recognized as state of the art. 
7 

 The PED laboratories are spacious, with good infrastructure. They are well lit and 

have stable air temperature and low ambient vibration resulting from isolated floor pads. 

In general, the division’s equipment is state of the art or beyond, well maintained, and 

operated by knowledgeable experts. The minimal acquisition of capital equipment 

threatens future PED infrastructure and technical capabilities. Nevertheless, the PED is 

clearly the place to go for the best in manufacturing metrology. 

RECOMMENDATIONS 

 The recommendations of the panel based on its assessment of the Manufacturing 

Engineering Laboratory and its divisions are as follows: 

 Across the MEL, project management and portfolio management strategies 

have continuously improved; however, more coupling of critical project 

selection with technical staff and equipment development should be pursued. 

 Closer MEL coupling with other technology areas of NIST (information 

technology and materials, for example) and with external expertise (in the 

application of biological processes and materials in manufacturing, for 

example) should be pursued. 

 The MEL’s research facilities and equipment will age quickly, and significant 

upgrades in metrology equipment will be required in the near future to update 

existing systems and put in place next-generation systems. 

 The MEL should review the budget available for capital acquisitions, which 

seems to be inadequate for such a capital-intensive laboratory. 

 The MEL is adversely affected by issues such as the significant underfunding 

for personnel, the difficulty of bringing in junior staff, the inadequacy of 

postdoctoral salaries, and the fact that several staff members have remained in 

acting positions for a long time. To address such issues, NIST should critically 

assess its commitment to and support of manufacturing, in line with the 

President’s placement of manufacturing as a top national priority and its 

importance in job creation and increased competitiveness in U.S. industry. 

 For the Intelligent Systems Division, there is a need for strengthened global 

benchmarking and self-assessment activities. 

 The ISD is leveraging its program development through partnerships with 

other agencies. NIST and the MEL should provide incentives and not 

disincentives for such partnership activities through their budgeting processes, 

overhead rates, merit policies, and so on. The partnerships with industry are 

more limited, and they should grow. The ISD could make broader impacts 

through consortium development. 

 The ISD is limited by staffing constraints, flat budgets, and attrition, which 

may be impeding the development of and recognition of leadership by ISD 

staff in the world technical community, and this should be addressed. 

 The Manufacturing Metrology Division should continue to develop the 

process for selecting new projects and focus areas and ensure its transparency. 
8 

 The MMD workload has increased as new capabilities have been added. More 

funds for personnel should be provided. 

 The MMD should continue to enhance its current capabilities through 

automation to improve accuracy, reduce costs, broaden operational ranges, 

reduce uncertainty, and reduce turnaround time. 

 Much of the MMD metrology infrastructure is aging and will need to be 

updated or repaired in the near future. Funds will be necessary to update these 

systems, develop models of next-generation metrology systems that are more 

reliable and lower in cost, and then develop these next-generation systems. 

 The Manufacturing Systems Integration Division should better articulate its 

successes to a wider audience of researchers and practitioners. 

 Staffing in the MSID is highly senior. It is difficult to bring in junior staff, and 

attracting postdoctoral researchers remains a problem because of salary, 

permanence, and other issues. Other means of obtaining resources should be 

considered: for example, interns and people from industry working on-site. 

 The Precision Engineering Division should continue participation in regional 

and international round robins as a key benchmarking exercise. 

 The budget available for PED capital acquisitions is inadequate for such a 

capital-intensive division and should be increased. 

 The PED should pursue stronger interfaces with other areas of NIST: for 

example, the Center for Nanoscale Science and Technology and the Physics 

Laboratory. 

 The PED management needs to develop and implement a critical-skills 

staffing plan independent of future budget developments so that its critical 

expertise within NIST is maintained. 
9 

1 

The Charge to the Panel and the Assessment Process 

 At the request of the National Institute of Standards and Technology, the National 

Research Council has, since 1959, annually assembled panels of experts from academia, 

industry, medicine, and other scientific and engineering environments to assess the 

quality and effectiveness of the NIST measurements and standards laboratories, of which 

there are now nine,4

 as well as the adequacy of the laboratories’ resources. In 2010, NIST 

requested that five of its laboratories be assessed: the Building and Fire Research 

Laboratory, the Manufacturing Engineering Laboratory, the Materials Science and 

Engineering Laboratory, the NIST Center for Neutron Research, and the Physics 

Laboratory. A separate panel of experts assessed each of these; the findings of the 

respective panels are summarized in separate reports. This report summarizes the findings 

of the Panel on Manufacturing Engineering. 

 For the fiscal year (FY) 2010 assessment, NIST requested that the panel consider 

the following criteria as part of its assessment: 

1. The technical merit of the current laboratory programs relative to current 

state-of-the-art programs worldwide; 

2. The adequacy of the laboratory budget, facilities, equipment, and human 

resources, as they affect the quality of the laboratory’s technical programs; 

and 

3. The degree to which laboratory programs in measurement science, standards, 

and services achieve their stated objectives and desired impact. 

 The panel adopted the following additional assessment criteria to make the broad 

factors more explicit. These criteria were only suggested to guide the assessment process 

and did not need to be individually addressed. 

1. Technical Quality and Merit Assessment Criteria 

 Relative Technical Caliber: 

—How does the technical quality of the laboratory programs compare to 

current state-of-the-art programs worldwide? 

—Does the laboratory work product consistently demonstrate evidence of 

high technical quality (including but not limited to papers in highimpact technical publications, invited talks to major scientific and 

industry conferences and workshops, and external awards and 

recognition)? 

 4

 The nine NIST laboratories are the Building and Fire Research Laboratory, the Center for 

Nanoscale Science and Technology, the Chemical Science and Technology Laboratory, the Electronics and 

Electrical Engineering Laboratory, the Information Technology Laboratory, the Manufacturing Engineering 

Laboratory, the Materials Science and Engineering Laboratory, the NIST Center for Neutron Research, and 

the Physics Laboratory. 
10 

 Distinctness: 

—Do the projects reflect mission focus? 

—Is the value proposition distinctive and in keeping with NIST’s role and 

strategy? 

 Relevance: 

—Is there a clear tie between projects and NIST and MEL strategic 

priority areas? 

—Do the projects reflect a broad understanding of comparable work being 

done elsewhere (in other government laboratories, universities, and 

industry)? 

—Are there demonstrable links between NIST researchers and the 

external community? 

 Balance: 

—Does the laboratory adequately balance anticipatory, longer-term 

research and activities that respond to immediate customer needs? 

2. Adequacy of Budget, Facilities, Equipment, and Human Resources Assessment 

Criteria 

 Available Tools:

—Is the state of the equipment and facilities adequate to meet project 

objectives and customer needs? 

 Critical Mass:

—Are the available scientific and technical competencies adequate to 

achieve success? 

—Is available funding adequate to achieve success? 

 Agility: 

—Is the laboratory sustaining the technical competencies and capacity to 

respond quickly to critical issues as they arise? 

3. Achievement of Stated Objectives Assessment Criteria 

 Technical Planning: 

—Are appropriate milestones identified? Do they appear feasible? 

—Are obstacles and challenges defined (technical, resources)? 

—Are the roles of the investigators clear, and are the individual project 

tasks and objectives clearly linked to those of other projects within a 

given strategic priority area? 

 Dissemination: 

—Is the laboratory regularly implementing sound and effective techniques 

and practices for delivering products and services? 

—Are the results of the laboratory projects readily available to 

stakeholders? 

 Impact: 

—How well do the laboratory programs in measurement science, 

standards, and services achieve their stated objectives and desired 

impact? 

—Will the laboratory products have a consequential, long-term impact? 
11 

 Level of effort: 

—Are research projects scaled appropriately to meet the technical 

problems being addressed? 

 Responsiveness: 

—Are the research projects moving at a pace and in a direction that is 

well matched to current and emerging stakeholder needs? 

4. Other Considerations 

 Collaboration and crosscutting within a given laboratory 

 Project-level technical planning 

 The context of this technical assessment is the mission of NIST, which is to 

promote U.S. innovation and industrial competitiveness by advancing measurement 

science, standards, and technology in ways that enhance economic security and improve 

the quality of life. The NIST laboratories conduct research to anticipate future metrology 

and standards needs, to enable new scientific and technological advances, and to improve 

and refine existing measurement methods and services. 

 In order to accomplish the assessment, the NRC assembled a panel of 15 

volunteers, whose expertise matches that of the work performed by the MEL staff.5 The 

panel members were also assigned to four subgroups (division review teams), whose 

expertise matched that of the work performed in the four divisions reviewed in the MEL: 

(1) Intelligent Systems, (2) Manufacturing Metrology, (3) Manufacturing Systems 

Integration, and (4) Precision Engineering. 

 The panel met at the NIST facilities in Gaithersburg, Maryland, on March 8-10, 

2010. After the full panel had met for a session of welcoming remarks from the NIST 

Director’s representative and an overview presentation on the MEL by MEL 

management, the panel divided into its four review teams, and each team (led by a team 

leader chosen from within the panel) then visited with the staff of its respective MEL 

division for about a day. During these visits, the review team members attended 

presentations, tours, demonstrations, and interactive sessions with the MEL staff. 

Subsequently, the entire panel assembled for about a day-and-a-half meeting, during 

which it interacted with MEL and NIST management and also met in a closed session to 

deliberate on its findings and to define the contents of this assessment report. 

 The approach of the panel to the assessment relied on the experience, technical 

knowledge, and expertise of its members, whose backgrounds were carefully matched to 

the technical areas of MEL activities. The panel reviewed selected examples of the 

technological research covered by the MEL; because of time constraints, it was not 

possible to review the MEL programs and projects exhaustively. The MEL selected the 

examples reviewed by the panel. The panel’s goal was to identify and report salient 

examples of accomplishments and opportunities for further improvement with respect to 

the following: the technical merit of the MEL work, its perceived relevance to NIST’s 

own definition of its mission in support of national priorities, and specific elements of the 

MEL’s resource infrastructure that are intended to support the technical work. These 

examples are intended collectively to portray an overall impression of the laboratory, 

 5

 See http://www.nist.gov/mel/ for more information on MEL programs. Accessed May 1, 2010. 
12 

while preserving useful suggestions specific to projects and programs that the panel 

examined. The assessment is currently scheduled to be repeated biennially, which will 

allow, over time, exposure to the broad spectrum of MEL activity. While the panel 

applied a largely qualitative rather than a quantitative approach to the assessment, it is 

possible that future assessments will be informed by further consideration of various 

analytical methods that can be applied. 

 The comments in this report are not intended to address each program within the 

MEL exhaustively. Instead, this report identifies key issues. Given the necessarily 

nonexhaustive nature of the review process, the omission of any particular MEL program 

or project should not be interpreted as a negative reflection on the omitted program or 

project. 
13 

2 

Intelligent Systems Division 

 The Intelligent Systems Division’s stated goal is to develop the measurements and 

standards infrastructure needed for the application of intelligent systems. The ISD has 

well-established core competencies in the areas of standards, performance evaluation, 

measurement, interoperability, safety, and security. It plays an important role in 

promoting the science, technologies, and tools related to manufacturing engineering. 

Standards and services are very important to U.S. innovation and industrial 

competitiveness, especially in the current global economic environment in which many 

countries are making serious investments in intelligent manufacturing technologies. 

 The ISD is serving as a catalyst in promoting collaboration between industries, 

with a focus on establishing standards and testbeds in the areas of computerized 

numerical control (CNC), dimensional inspection equipment interoperability, 

manufacturing robotics and automation, and autonomous-vehicle and materials-handling 

systems. In the areas of safety and security, the ISD has established new standards for the 

security of industrial control systems. The ISD’s technical capabilities are among the best 

in STEP-NC and OMAC for real-time data models, machine compensation, machining 

tool path optimization, Ethernet/IP performance testing, and others. 

 The ISD currently has 31 NIST staff and 14 guest researchers. Its FY 2010 

estimated funding is about $8.9 million, with about 38 percent coming from extramural 

sources. 

TECHNICAL MERIT RELATIVE TO STATE OF THE ART 

 The ISD promotes the development of measurement science and interoperability 

standards to enhance manufacturing robotics and automation equipment and the 

underlying industrial control systems. The division’s recent efforts have focused on areas 

such as requirements and performance standards (defining, specifying, measuring, and 

evaluating robot and automation system performance and capabilities; proposing and 

facilitating repeatable, objective, and quantitative test methods and their associated 

environments, artifacts, and data); interoperability standards (requirements, interface 

definitions, data modeling, conformance assessment tools, facilitating the bringing 

together of industry stakeholders); and infrastructural technology to help build nextgeneration unmanned systems. The ISD operational environment includes strong interest 

and investment by federal agencies in robotics/intelligent systems, especially in the areas 

of defense and homeland security. 

 The ISD serves U.S. manufacturing by developing solutions for measurements, 

interoperability, and safety and security for intelligent robots, automation, and control 

systems. There is a renewed U.S. interest in manufacturing, robotic and automation 

systems at different levels. The ISD’s short- and long-range plans are well in line with 

those of NIST. For example, NIST’s proposed Innovations for 21st Century U.S. 

Manufacturing initiative points out the need to— 
14 

Provide the measurement tools and capabilities necessary to develop and apply 

robotic technologies that are “smarter,” more flexible, and better able to operate 

safely and effectively in less structured environments, to facilitate mass 

customization in manufacturing processes.6

 The ISD is recognized as a world leader in the performance evaluation of 

complex, intelligent systems. The division is also recognized for its work in the 

development of interoperability standards for manufacturing automation systems, and its 

Intelligent Manufacturing Interoperability Standards Program has been in operation for 

some time. The motto for this program is “Smarter Data for Smarter Manufacturing.” The 

newest activity in this framework is the Quality Information Interoperability project, in 

which the effort is focused on the seamless integration of quality requirements from 

design through planning, production, analysis, and reporting. 

 The ISD has been successfully leveraging the development of its manufacturing 

program through partnerships, primarily with Department of Defense (DOD) agencies, 

the Department of Homeland Security (DHS), and the Department of Transportation 

(DOT). Nearly 40 percent of ISD resources come from such partnerships. This is a strong 

indicator of the external recognition of ISD activities and expertise. 

ADEQUACY OF INFRASTRUCTURE 

 The ISD has excellent research facilities and equipment, some of which is loaned 

by partners. It also will benefit from the new NIST robot test facility whose construction 

is expected to begin in the fall of 2010. The facility will be an important national resource 

for this rapidly developing industry. However, many other places around the world (e.g., 

Japan, Korea, Singapore, Europe) are making major investments in this area, and ISD 

facilities and activities are no longer unique and unrivaled. 

 The staffing resources are limited, constraining the ability of the ISD to respond 

with agility to changing technology needs. Strong world leadership is likely to be difficult 

to maintain when many management positions within the division are “acting” positions. 

ACHIEVEMENT OF OBJECTIVES AND IMPACT 

 The ISD continues to develop its strategic technologies and testbeds in the areas 

of interoperability standards for quality information, next-generation manufacturing 

robotics for safety and security, and performance measurement of autonomous-vehicle 

systems, among other areas. It has had significant impact on the establishment of 

standards in these areas. One of its major impacts has been to establish industrial controls 

and networks standards for federal government and industrial users. NIST SP 800-53 was 

established for security controls for federal information systems, and NIST SP 800-82 

was established as a guide to industrial control systems security. These allow federal 

agencies, as well as the private sector if it so desires, to determine the proper security 

controls for their IT systems as well as to secure their industrial control systems 

 6

 Additional information on this proposed NIST program initiative can be found at 

http://www.nist.gov/public_affairs/factsheet/comp_manuf2011.cfm. Accessed August 16, 2010. 
15 

effectively while addressing their unique requirements. SP 800-53 has been downloaded 

more than 1.25 million times since its initial release in 2009. In the area of robot safety, 

the ISD has developed a technical specification as input for the next revision of the 

International Organization for Standardization (ISO) robot safety standard (ISO 10218: 

Robots for Industrial Environments—Safety Requirements) to allow human-robot 

collaboration. The ISD is also contributing technical expertise to a new standard (ISO 

13482: Robots and Robotic Devices—Safety Requirements—Non-medical Personal Care 

Robot) for nonmedical and personal-care robots and robotic devices. In the area of 

autonomous systems, the ISD has developed sensor performance metrics, standards, and 

infrastructure technology to support the use of semiautonomous and autonomous 

manipulators and vehicles, and control architecture. 

 In the area of quality information interoperability, the ISD and the International 

Association of Coordinate Measuring Machine Manufacturers demonstrated I++DME 

interoperability (I++DME is a nonproprietary language for communicating with a CMM) 

for different CMM manufacturers and enabled I++DME-compliant products in 

production facilities worldwide. In addition, the ISD has developed a new version of the 

Dimensional Measuring Interface Standard (DMIS) to support manufacturing industries’ 

needs in integrating measurement systems with design and quality-assurance systems. 

 In the area of intelligent manufacturing, industrial control systems, and network 

standards, the ISD has developed an Industrial Ethernet Network Performance (IENetP) 

Test Tool that is an open-source test tool for industry: it allows vendors to determine 

device network performance during development and allows users to validate device 

network performance before deployment. Version V1.0.1 was released in 2009, and there 

had been more than 375 downloads as of February 1, 2010. The ISD has developed 

Mobility Open Architecture Simulation and Tools (MOAST), which is an 

implementation of the 4D/RCS reference model architecture for unmanned vehicle 

systems. It continues to develop STEP-NC, which defines a feature-based view of 

operations and includes solid-model descriptions, material, tooling, and tolerances. 

STEP-NC is part of the ISO 10303 suite of standards for the exchange of product and 

process data: specifically, Application Protocol 238 (AP-238). AP-238 is intended to 

replace the 50-year-old “G-code” format for numerically controlled machine tools that 

specified primitive tool motions. 

CONCLUSIONS 

 Following are the conclusions of the panel based on its assessment of the 

Intelligent Systems Division: 

 The caliber of the work in the ISD is sound and state of the art. Many projects 

have had high impact, including the industrial Ethernet test tool, 

security/safety standards of industrial control systems, the DMIS for CMMs, 

and performance testing of robots. 

 There is a need in the division for strengthened global benchmarking and selfassessment activities. Such benchmarking can identify gaps and drive future 

project selection. 
16 

 It is commendable that nearly half of the ISD activities are not strictly in 

discrete-parts manufacturing. This is evidence that the ISD can bring its 

strengths in measurement, performance evaluation, standards, and 

interoperability to bear on important related areas such as the security and 

safety of industrial control systems, autonomous systems, and others. 

 The ISD is leveraging its program development through partnerships, 

primarily with DOD agencies and the DOT. It is important that NIST and the 

MEL provide incentives, and not disincentives, for such partnership activities 

through their budgeting processes, overhead rates, merit policies, and other 

means. The division’s partnerships with industry are more limited compared 

to those with government agencies, and they should grow. The ISD could 

make broader impacts through consortium development, further leveraging 

industry involvement and investment to accelerate the dissemination of the 

developed standards. 

 The ISD is limited by staffing constraints, flat budgets, and attrition. This may 

be impeding the development and recognition of leadership by ISD staff in the 

world technical community. 
17 

3 

Manufacturing Metrology Division 

 The mission of the Manufacturing Metrology Division is to fulfill the 

measurements and standards needs of U.S. discrete-parts manufacturers in mechanical 

metrology and advanced manufacturing technology. This mission is accomplished under 

the rubric of four focus areas: (1) research and development in realizing and 

disseminating the International System of Units (SI) mechanical units; (2) the 

development of methods, models, sensors, and data to improve metrology, machines, and 

processes; (3) the provision of services in mechanical metrology, optics metrology, 

machine metrology, process metrology, and sensor integration; and (4) leading in the 

development of national and international standards. 

 The research portfolio of the MMD is well distributed from low- to high-risk 

projects. Low-risk projects such as automating calibration capabilities in sound or mass 

have resulted in improved processes (e.g., lower uncertainty, larger dynamic range, 

reduced costs) while reducing the reliance on manual labor to execute these calibrations, 

enabling more efficient use of personnel and the ability to address customer needs better. 

High-risk projects—such as utilizing DNA molecules to serve as an intrinsic small-force 

standard, or the newly purchased laser lithography tool for fast diffractive-optics 

manufacture—are positioning NIST to continue its global leadership in nanotechnology 

and nanomanufacturing. The result of this research portfolio is a team and facilities that 

are well suited to target the four focus areas of the division. The MMD executes its 

mechanical metrology mission through two major efforts: (1) either by improving 

currently known processes or by developing completely new approaches, it makes 

measurements that could not be done before; and (2) it enhances current capabilities by 

means of automation to improve accuracy, reduce costs, broaden operational ranges, 

reduce uncertainty, and reduce turnaround time. 

 The MMD currently has 33 NIST staff, 7 guest researchers, and 2 postdoctoral 

researchers. Its FY 2010 estimated funding is about $9.9 million, with about 17 percent 

coming from extramural sources (including reimbursable services). 

TECHNICAL MERIT RELATIVE TO STATE OF THE ART 

 NIST is well known for the high quality of its work in manufacturing metrology, 

and it continues to build this reputation from a variety of perspectives. The measurements 

executed in the MMD are of high quality and are often the de facto standard to which the 

National Measurement Institutes (NMIs) of other nations are held. Furthermore, the 

MMD team continues to develop new means of making measurements to address the 

ever-increasing demand for improved tests. For example, the work in the redefinition of 

the kilogram places NIST in the leadership position for next-generation mass definitions. 

The kilogram redefinition will be enabled through the establishment of the first “mise-enpratique” (practical method for realization) and the dissemination of a new kilogram 

definition and a dissemination system that directly ties the current air-based kilogram 
18 

definition to the vacuum-based alternative definition using stable artifacts with wearresistant and chemical inertness characteristics that are maintained in controlled 

environments. Such work is critical in linking definitions employed in the United States 

to those used in other countries. Such links are necessary to ensure that U.S. industries 

can effectively compete in the global market. 

 Of particular note is the unique and highly innovative approach toward 

establishing SI traceability for small-force measurement. Atomic and single-molecule 

forces will be evaluated as potential intrinsic standards and references. New techniques 

using DNA molecules to serve as an intrinsic small-force standard and atomic force 

microscope microcantilevers and plastic deformation (nano-indentation) tests are being 

investigated to support nanomechanical investigators who are using atomic force 

microscopy to probe the physical characteristics of matter at the nanoscale. If successful, 

the results of this effort will place NIST ahead of other NMIs on measuring and 

calibrating extremely low forces. 

 The work by the MMD in science-based manufacturing is important in obtaining 

fundamental knowledge of manufacturing and metrology processes that, from a 

mechanistic perspective, will enable advances in processes and their control that are 

critical to moving current and next-generation manufacturing capabilities in the United 

States forward. However, this program cannot, and should not, address all fundamental 

issues in process modeling. It needs to be focused on measurement science and the needs 

of measurement services and standards. 

 In emerging areas such as wireless sensors, NIST researchers are acknowledged 

leaders in the development of standards. They developed a testbed for using and 

demonstrating wireless sensors (temperature and vibration). A sensor network on the 

shop floor was implemented as a validation platform for sensor and networking standards 

in factory environments. Given the likely widespread impact of this technology, it is 

critical that this leadership role be maintained with appropriate investment. 

ADEQUACY OF INFRASTRUCTURE 

 The facilities and equipment of the MMD are very good, with state-of-the-art 

equipment found in each of the programs. For example, the large-mass calibration 

equipment is globally unparalleled. Other examples include optics (flat and spherical 

surface metrology equipment) and nanostructured-optics fabrication facilities. New 

equipment such as the laser lithography tool (with expected delivery in late 2010), a 

metal-based additive manufacturing system (still in the procurement process), and the 

newly automated calibration (mass and microphone) facilities will help the division to 

meet its objectives. The staff is extremely well suited to meet the objectives of the 

division and has demonstrated this capability with a very productive and successful 

recent history. There is no doubt that the staff will be able to continue its mission in a 

successful manner. Furthermore, it is hoped that the newly automated systems will enable 

the staff to spend more time conducting research. 
19 

ACHIEVEMENT OF OBJECTIVES AND IMPACT 

 The MMD continues to do a very good job of developing standards, providing 

calibration services, and establishing traceability. It remains at the state of the art in force 

measurement, optics metrology, machine-tool metrology, and acoustics metrology. The 

division has significant impact on the nation and the world through its broad spectrum of 

standards work, from defining new standards, to supporting current standards, to 

providing new fundamental definitions for next-generation standards. For example, 

MMD personnel are actively involved in the development of new standards (e.g., fiveaxis coordinated motion standards and Institute for Electrical and Electronics Engineers 

[IEEE] and ISO standards for wireless sensors). With respect to supporting current 

standards, services for the calibration of geometry (e.g., flatness, sphericity), mass, force, 

acceleration, and acoustical properties are state of the art and second to none. The MMD 

is developing new techniques to address next-generation standards such as a low-power 

precision magnetic levitation system. This system provides a link between present mass 

calibrations that are done at atmospheric pressure and the next-generation alternative 

calibrations that will be executed in vacuum. Systems such as this will enable the United 

States to lead in mass metrology and calibration well into the 21st century. 

CONCLUSIONS 

 Following are the conclusions of the panel based on its assessment of the 

Manufacturing Metrology Division: 

 The staff of the MMD are doing high-quality, innovative work and are to be 

commended for their outstanding efforts. 

 The research portfolio of the MMD is well distributed, from mature to 

emerging technologies, and from low-risk to high-risk projects. 

 The MMD is supporting standards from cradle to grave and rebirth, including 

the development of new standards, the calibration of artifacts to the current 

standards, and new concepts on which next-generation standards are 

developed. 

 The MMD’s measurement services are invaluable to the U.S. economy and 

for national security. From a national perspective, the division’s calibration 

services provide a baseline for many sectors of the economy. From an 

international perspective, global commerce relies on the ability to relate 

standards used in the United States to those used in other countries. 

 The MMD is customer-driven and clearly understands the importance of its 

work with respect to manufacturing, engineering, and science. 

 The MMD has improved the process for selecting new projects and focus 

areas. It should continue to develop this process and ensure its transparency. 

 The workload has increased as new capabilities are added to the MMD, but 

funding for personnel has remained relatively flat. More funds for personnel 

are necessary to enable the MMD to continue to expand its critical functions. 
20 

 The MMD should continue to enhance its current capabilities by means of 

automation in order to improve accuracy, reduce costs, broaden operational 

ranges, reduce uncertainty, and reduce turnaround time. This change would 

also have the effect of freeing personnel from routine calibration tasks, 

enabling them to spend more time on advanced research efforts. 

 Much of the MMD metrology infrastructure is aging and will need to be 

updated or repaired in the near future. Funds will be necessary to update these 

systems, develop models of next-generation metrology systems that are more 

reliable and lower in cost, and then develop these next-generation systems. 
21 

4 

Manufacturing Systems Integration Division 

 Two main focus areas within the Manufacturing Systems Integration Division are 

that of sustainable and life-cycle information-based manufacturing and that of supplychain integration; both are appropriately aligned with the needs of U.S. industries. All the 

projects within these two programs are aligned with the MEL mission. 

 The Sustainable and Lifecycle Information-based Manufacturing (SLIM) Program 

has three primary objectives: 

1. Providing key standards requirements and best practices for sustainable 

manufacturing. This objective includes analyzing and defining the relevant 

standards for areas ranging from carbon-footprint determination, to energyresource management, to hazardous-material management. The program is 

also developing a scheme for computing the carbon footprint of a 

manufactured product—a critical mathematical “score” that can help resolve 

concerns such as the issue of reusable versus disposable products. NIST is 

looking to ensure that new standards mesh with current practices, so 

researchers are working to harmonize their “green” information standards 

with existing software standards for manufacturing product data, such as ISO 

10303, popularly known as STEP. 

2. Providing a framework for environmental manufacturing models by 

determining key attributes and developing information models necessary for 

sustainable manufacturing.

3. Developing protocols for testing and simulating the application of “green” 

standards.

 In the Supply Chain Integration Program, the MSID has selected to focus on and 

address a set of key industrial “pain points” for maximum impact: 

1. Improving the information flow across the supply chain; 

2. Gathering and disseminating best practices in sustainable manufacturing; and 

3. Identifying meaningful sustainability metrics that lead to improvements.

The supply/value chain sustainability question spans a very broad spectrum, from the 

mining of raw resources, through product manufacture and use, finally to product 

disposal. The focus is on supplier discovery and long-range logistics. Both of these issues 

involve significant integration problems. The Supply Chain Integration Program’s 

technical focus is on the development and testing of standards needed to address those 

problems, collaborating with organizations such as the Open Applications Group, Inc., 

which develops domain standards, and with organizations such as the Object 

Management Group (OMG) and the World Wide Web Consortium (W3C) that develop 

knowledge representation standards. 
22 

 The division currently has 26 full-time permanent NIST staff, 1 part-time 

permanent worker, 34 NIST associates, and 1 postdoctoral researcher. Its FY 2010 

estimated funding is about $9.5 million, with about 7 percent coming from extramural 

sources. 

TECHNICAL MERIT RELATIVE TO STATE OF THE ART 

 The MSID continues to undertake and execute programs of high technical merit 

and strong relevance and effectiveness with respect to U.S. competitive manufacturing. 

The division’s efforts in the SLIM Program, OMG, W3C, and STEP standards already 

worked out for engineering and manufacturing have been exemplary. In the SLIM 

Program, high-quality work is being performed related to heavy metal industries (e.g., 

aerospace, automotive). 

 MSID staff have elevated the status of product life-cycle management (PLM) 

from being merely an engineering concern to an enterprise-wide business concern, with 

emphasis on information integration and collaboration with partners. Achieving, 

ensuring, and maintaining global interoperability constitute too large a task for all but the 

biggest corporations. In addition, NIST is tasked to help address the interoperability 

problem at the global level. Working with industry, the MSID is helping to devise 

sustainable solutions for this multifaceted and global challenge in supply-chain 

requirements and the business aspects of PLM. 

 The MSID continues to bridge ISO 10303 application protocols to these 

modeling-language standards. A notable standardization achievement is the Simulation 

Interoperability Standard Organization’s Core Manufacturing Simulation Data 

specification standardization. The MSID has also continued to enhance online testing 

services that enable design-engineering, supply-chain, and e-business software vendors to 

validate the conformance of their products to emerging or existing standards. It has also 

provided tools to the standardization community to ensure rigorous quality for the 

standards themselves. 

 Work with the DHS shows the universality of the nature of informatics; the 

concepts being studied will be a bridge to the future in manufacturing. The systematic 

approach to informatics has applicability well beyond the domain of manufacturing, and 

this expertise as applied to the DHS is a special asset to NIST. 

 The development of a framework for an integrated tool set based on the widely 

used TOGAF (The Open Group Architecture Forum) is a solid beginning for defining the 

requirements of Enterprise interoperability and ensures completeness of the tool set. 

ADEQUACY OF INFRASTRUCTURE 

 Given the MSID’s information-centric paradigm, the advanced manufacturing 

systems and networking testbed and the simulation laboratories’ equipment and facilities 

are adequate to meet projects’ objectives. Given the limited resources of the MSID, the 

scale of specific projects and the work planning are appropriate. 

 The actual-dollar annual budget has remained relatively constant over the past 5 

years, at roughly $10 million in a time of generally increasing cost in all areas. This has a 

number of negative impacts in the context of MSID’s achieving its goals; due to 
23 

insufficient expertise in other areas, the efforts in manufacturing remain limited in scope. 

For example, the budget and number of staff members are lower in 2010 than they were 

in 2008. 

ACHIEVEMENT OF OBJECTIVES AND IMPACT 

 With its limited resources, the MSID has had to scope its focus carefully, and it 

has addressed the requirements and gaps in the standards. This work has been exemplary 

and should be extended deeper into the enterprise and to other areas such as the 

semiconductor and pharmaceuticals manufacturing industries through the resourcing of 

sufficient staffing. As a result of MSID’s highly qualified staff, it is the acknowledged 

leader in the development of STEP standards and internationally recognized as a leader in 

influencing global standards bodies and other multinational organizations. MSID staff are 

active in standards committees for ISO STEP standards and are active in disseminating 

information by participating in meetings and conferences. They are motivated and 

capable and are providing high-quality work and expertise. The work being done by the 

MSID on the standards such as STEP is but one of the areas that are of high value and 

much interest to the U.S. industries. In addition, the MSID has had a very strong impact 

on interoperability and standards modularization and harmonization. The standards that 

have been developed by the MSID are already in use by U.S. manufacturing industries as 

well as by the Honda Motor Company, Ltd., and have been adopted by vendors. 

CONCLUSIONS 

 Following are the conclusions of the panel based on its assessment of the 

Manufacturing Systems Integration Division: 

 Through workshops and other external activities, the MSID has significant 

impact in areas in which it is active: customer requirements, standards, 

regulations, and others. As a result of its highly qualified staff, the MSID is 

internationally recognized as a leader in influencing standards bodies and 

other multinational organizations (including OMG, W3C, the Automotive 

Industry Action Group, and STEP). 

 The MSID informatics effort is an essential and integrated effort across many 

domains and many issues, ranging from interoperability to “green” issues to 

supply-chain issues, from semiconductors to pharmaceuticals to aircraft. If 

this effort is disrupted, the chances of success in meeting goals become 

minimal. 

 MSID technical leadership in treating modeling as an inherent and inseparable 

part of the manufacturing and even enterprise processes is commendable. And 

given the systematic approach to modeling informatics developed over the 

years and the expertise within the division, the MSID has a unique 

opportunity to influence not only other segments of the manufacturing 

domain, but it can also impact health care, security, and other complex 

activities. 
24 

 The MSID has clearly met its goals and objectives as evidenced by a reading 

of the division’s progress reports. Nonetheless, it should articulate these 

successes better to a wider audience of researchers and practitioners. 

 Current MSID staffing is highly senior, and the consequent lack of new 

positions makes bringing in junior people difficult. Attracting postdoctoral 

researchers remains a problem because of salary, permanence, and other 

issues. All of these considerations will impact the influx of new ideas, as will 

the reduced budget compared to that of previous years. Perhaps other means 

of obtaining resources can be considered—for example, having interns, 

having people from industry working on-site, and other possibilities. 
25 

5 

Precision Engineering Division 

 The Precision Engineering Division delivers to industry important length-related 

measurements, standards, and technology services that directly support the products and 

processes of U.S. manufacturing. These standards are measured by a whole array of 

instrumentation ranging from atomic force microscopes to conventional and frameless 

coordinate measuring machines using contact, particle beam, and optical measuring 

probes, machines, and systems. Features of interest range in size from 1 kilometer 

(calibrated telecommunication cables) to meters (laser trackers) to nanometers (criticaldimension [CD] metrology instruments and nanoparticle dimensional metrology). 

 Modeling capabilities and methods are being developed together with 

experimental measurement technologies, and these are used jointly to address nextgeneration problems. For example, the code for modeling AFM data is now widely used 

in the industry. A new version, Java Monte-Carlo Simulator for Secondary Electrons 

(JMONSEL), of the widely used NIST code MONSEL is being developed for threedimensional scanning electron microscopy (SEM). Consistent with NIST policy, codes 

are made freely available to the technical community. 

 The PED currently has 35 NIST staff, 22 guest researchers (full-time-equivalent), 

and 1 postdoctoral researcher. Its FY 2010 estimated funding is about $12.7 million, with 

about 23 percent coming from extramural sources. 

TECHNICAL MERIT RELATIVE TO STATE OF THE ART 

 Essentially all of the PED’s programs reviewed for this assessment push the state 

of the art in their areas. These programs are well connected to NIST objectives and to the 

needs of the industrial and scientific communities. The PED’s leaders are internationally 

recognized within their fields, and they provide excellent technical leadership within their 

areas of expertise. The PED recognizes that one of its primary roles is to maintain 

metrology standards. Toward this end, the division has developed a nanowriting 

capability using an atomically sharp tip (whose shape can be determined and modified 

accordingly), which will generate badly needed standards for dimensions on the order of 

a few atoms that are traceable to NIST. This capability will be critically important at the 

22 nm node and beyond in integrated-circuit (IC) technology. Regarding metrology, the 

PED is developing a calibrated atomic force microscope (C-AFM) that is already 

returning line roughness measurements that will also be crucial for 22 nm lithography 

and beyond. Also important for metrology, the PED is developing step-height standards 

for calibrating AFMs. Its helium-ion microscopy is state of the art and gives the PED new 

capabilities in dimensional analysis. 

 The 65 m laser-tracker test-range distance metrology project represents an 

important project that currently only NIST can do. The objective is to allow 

measurements made in air to be referenced to the standard meter to within parts in 107

 by 

accurately measuring, then compensating for, pressure and temperature variations. This is 
26 

done by a creative use of frequency combs to transfer the standard meter to the laser 

wavelength through Global Positioning System (GPS) timing, then to use the laser 

wavelength as the secondary standard for the actual measurement. The project has 

already achieved a capability of 5 parts in 107. 

ADEQUACY OF INFRASTRUCTURE 

 The move to the Advanced Measurement Laboratory has provided best-in-theworld facilities for equipment that in many cases is the most advanced available 

anywhere. The quality of the AML facilities has translated into instrument performance 

that routinely provides fundamental limits independent of environmental effects. 

 Equipment availability in the PED is good, but the acquisition of equipment 

appears to be limited and overly reliant on fortuitous connections between PED personnel 

and outside organizations. The PED has been very creative in benefiting from these 

connections in that collaborators and customers have made equipment available at good 

prices. One example is the Moore M48 CMM that was brought in from the Sandia 

National Laboratories. The PED is an equipment-intensive division, yet it does not appear 

to have a regular program for the acquisition of critical equipment needed to maintain its 

technical capabilities. 

ACHIEVEMENT OF OBJECTIVES AND IMPACT 

 Dimensional metrology is advancing well, with existing capabilities being driven 

to new levels and new capabilities coming online. The JMONSEL modeling code for 

SEM imaging is a case in point, anticipating the needs for future IC technology, in which 

devices will be using the third dimension. A further example of anticipating future needs 

in IC technology is the C-AFM project, with the objective of providing accurate 

measurements of linewidths, pitch, and sidewall roughness, all of which will become 

critical at the 22 nm node and beyond. 

 Division members continue to receive awards and recognition—for example, the 

Nano-50 Award for Scatterfield Microscopy, the Department of Commerce Gold Medal 

for Leadership in International Standards, and two Department of Commerce Silver 

Awards: one for the development of optical methods in overlay and the other for the 

NIST microfeature CMM probe. Two members were elected fellows of SPIE. One paper 

(on removing tip-shape from scanning probe microscope images) by PED staff has been 

cited 187 times at last count, which is an unusually large number in a fast-moving field 

where publications in conference proceedings are more common, and many of these have 

a short lifetime. 

 In addition to the Department of Commerce, the PED provides services to the 

National Aeronautics and Space Administration; the Bureau of Alcohol, Tobacco, and 

Firearms; the Federal Bureau of Investigation; the Department of Energy; the Bethesda 

Naval Medical Center; and other government entities such as the U.S. Army. 

 One concern is the reliance of the PED on contractors rather than on permanent 

staff. The PED is to be commended for its creativity in making budgets stretch, but that is 

not likely to be a long-term solution. One of the major advantages of NIST relative to 

other organizations is institutional memory—that is, the continuity provided by experts 
27 

who have become very good at operating the specialized equipment that allows NIST to 

fulfill its role. Excessive reliance on contractors negates this advantage. 

 Over the past decade, PED staff have played an active role in the development of 

American Society of Mechanical Engineers (ASME) standards as well ISO standards. 

ISO standards are widely used, although U.S. standards are technically superior. The 

MEL leads an international task force to improve the rigor of the ISO CMM standard and 

also chairs the effort to harmonize the U.S. and ISO CMM standards. Unifying U.S. and 

ISO CMM standards reduces barriers to trade by enabling internationally based supply 

chains, clarifying contractual requirements, and eliminating redundant procedures and 

training. PED contributions have included the following: 

 ASME B89.7.3.1-2001: Guidelines for Decision Rules: Considering 

Measurement Uncertainty in Determining Conformance to Specifications

 ASME B89.7.3.3-2002: Guidelines for Assessing the Reliability of 

Dimensional Measurement Uncertainty Statements

 ASME B89.4.22-2004: Methods for Performance Evaluation of Articulated 

Arm Coordinate Measuring Machines

 ASME B89.7.4.1-2005 (technical report): Measurement Uncertainty and 

Conformance Testing: Risk Analysis

 ASME B89.7.5-2006: Metrological Traceability of Dimensional 

Measurements to the SI Unit of Length

 ASME B89.4.19-2006: Performance Evaluation of Laser-Based Spherical 

Coordinate Measurement Systems

 ASME B89.7.3.2-2007: (technical report): Guidelines for the Evaluation of 

Dimensional Measurement Uncertainty

 ASME B89.4.10360.2-2008: Acceptance Test and Reverification Test for 

Coordinate Measuring Machines (CMMs), Part 2: CMMs Used for 

Measuring Linear Dimensions

An MEL report7 (published in 2007) was cited heavily in the National Research Council 

report on ballistic imaging.8

 

CONCLUSIONS 

 Following are the conclusions of the panel based on its assessment of the 

Precision Engineering Division: 

 The PED research staff is knowledgeable, enthusiastic about what they are 

doing, and committed to excellence. They strive to maintain relevance by 

 7

 T.V. Vorburger, J.H. Yen, B. Bachrach, T.B. Renegar, J.J. Filliben, L. Ma, H.-G. Rhee, A. 

Zheng, J.-F. Song, M. Riley, C.D. Foreman, and S.M. Ballou, Surface Topography Analysis for a 

Feasibility Assessment of a National Ballistics Imaging Database, NISTIR 7362, Gaithersburg, Maryland: 

National Institute of Standards and Technology, May 2007. 8

 National Research Council, Ballistic Imaging, edited by Daniel L. Clark, John E. Ralph, 

Eugene S. Meieran, and Carol V. Petrie, Washington, D.C.: The National Academies Press, 2008. 
28 

remaining in contact with key industrial constituencies, such as SEMATECH, 

and focus on topical areas and projects that are likely to become important in 

the future. 

 The PED staff do well working with what they have. Laboratory facilities are 

very good, with available equipment generally being state of the art, often 

unique. Even though they may be decades old, facilities are maintained in top 

condition and are operated by experts. Consistent with the general NIST 

philosophy, this institutional expertise is a national resource. 

 The PED interface with the outside world is outstanding. Its staff participates 

in relevant meetings, which not only aids their goals of disseminating 

information to industry but also keeps PED personnel updated and attuned to 

customer needs. 

 PED impact is clearly significant. This is evidenced in many ways—for 

example, by the large number of ongoing collaborations with companies. The 

number of requests for services far exceeds the capacity of the division to 

meet them. Other measures include the number of citations for PED staff 

publications, and the division’s ability to acquire equipment, either donated or 

purchased at bargain prices. The PED is to be commended for handling the 

many demands on its time by appropriate triaging—for example, routing 

routine requests for gage-block metrology to commercial vendors. 

 The PED’s coordinate-measurement and uncertainty expertise is unique and 

not likely to be duplicated anywhere else. This expertise was critical in its 

recent work contributing to the U.S. Army’s resolution of an issue involving 

large, body-armor contracts. 

 The PED appreciates that one of its roles is to maintain metrology. Its state-ofthe-art atomically sharp tip nanowriting capability will, among other 

applications, be used to generate badly needed standards for dimensions on 

the order of a few atoms that are traceable to NIST. 

 Some semiconductor CD metrology work is not state of the art owing to the 

approaches being used. However, this can probably be justified as 

investigating ways of taking metrology in new directions. The acquisition of 

relevant equipment in 2010 will eliminate this concern. 

 The PED works toward positioning itself to the future. In particular, it 

concentrates on projects that are expected to be important in the near-term 

future. These include nanometrology, improving optical resolution, better 

sensors, and establishing metrology standards for fuel cell catalysts. At the 

same time it has dropped projects best handled elsewhere, such as sieve 

standards and civil engineering metrology now done more conveniently by 

GPS. 

 The planned equipment allotment from the American Recovery and 

Reinvestment Act of 2009 (Public Law 111-5) is a welcome development. 

 The PED should continue its participation in regional and international round 

robins as a key benchmarking exercise. 

 The budget available for capital acquisitions is inadequate for such a capitalintensive division. 
29 

 The PED could benefit from stronger interfaces with other areas of NIST. The 

panel heard little concerning the natural interfaces that should occur—for 

example, between the PED and NIST’s Center for Nanoscale Science and 

Technology and Physics Laboratory. 

 PED management needs to develop and implement a critical-skills staffing 

plan independent of future budget developments so that its critical expertise 

within NIST is maintained. At present there is too much dependence on guest 

workers, which appears to be as a direct consequence of flat long-term 

funding. 
30 

6 

Overall Conclusions 

 The Manufacturing Engineering Laboratory has excellent staff and exceptional 

facilities. Its work is essential in supporting the NIST mission of promoting U.S. 

innovation and industrial competitiveness by advancing measurement science, standards, 

and technology. The MEL is having a significant impact on today’s manufacturing 

industries, ranging from traditional production industries to optics industries to 

semiconductor industries. The MEL is making significant enabling contributions to future 

U.S. manufacturing industries and jobs. A few examples include the following: 

 Redefinition of the kilogram, across-the-board potential impact, from 

megascale to nanoscale industries; 

 Piconewton measurement using biological molecules (DNA) as intrinsic force 

standards, bridging “bio” and “nano,” opening new frontiers in materials and 

micro-/nano-devices; 

 Measurement technology development for larger, next-generation silicon 

wafers; 

 Safety and security of industrial control systems (1.25 million downloads of 

the standard); 

 Portable ability to measure to 100 micrometers over 65 meters, NISTtraceable to the standard meter, enabling quality and precision in large 

products, structures, and so on; and 

 Work in model-based enterprises in manufacturing that has been shown to be 

equally applicable to security operations. 

A significant number of projects at NIST are necessarily related to the integration and 

interoperability needed for effective commerce. Therefore, there is a requirement to 

maintain deep expertise in the applications of technologies and to maintain permeable 

interactions among disciplines. This deep expertise exists only at NIST, is not easily 

replicable, and is difficult to develop elsewhere; yet, it is crucial to the future of 

manufacturing. 

 Across the MEL, portfolio and portfolio management strategies have been 

continuously and significantly improved, resulting in a portfolio of projects that is 

customer-focused (taking into account present and future customers) and well distributed 

in terms of high and low risk and short- and long-term payoff. This needs to continue, 

with even better articulation of project focus, to achieve desired external impacts. 

Institute-level management needs to ensure that morale and knowledge-management 

issues receive due consideration (retirements, the use of “acting” managers, budget 

reductions, growth inequities, improvement of scientific methods, the role of external 

funding, etc.). Acting positions and added responsibilities without commensurate 

compensation may be adversely affecting the career paths of line managers, resulting in 

losses of key personnel and stagnation in promotions and opportunities for promotions. 
31 

Postdoctoral researcher salaries are inadequate for engineering fields, making it difficult 

to attract the best candidates, many of whom are potential future permanent staff. The 

budget available for capital acquisitions seems to be inadequate for such a capitalintensive laboratory. 

The future competitiveness of U.S. industry in sustainable, high-value-added 

manufacturing with high precision and using manufacturing processes and products with 

ever-decreasing dimensional scale will be in doubt without measurement science, 

measurement services, and standards that are well beyond the current state of the art. This 

is also true for biotechnology in manufacturing, including biological product components 

and biologically derived process components, as well as for information technology in 

manufacturing, including knowledge bases and “smart” products and factories. The 

equipment in the MEL and the technical expertise of its staff will need to evolve quickly 

to meet future needs in these areas. Furthermore, the breadth of the challenges to be faced 

will require closer coupling into the MEL of other technology areas of NIST, such as 

information technology and materials, and closer coupling into the MEL of external 

laboratories’ technological expertise, such as biological processes and materials in 

manufacturing.