AN ASSESSMENT OF THE 

NATIONAL INSTITUTE OF STANDARDS 

AND TECHNOLOGY 

MANUFACTURING ENGINEERING 

LABORATORY 

FISCAL YEAR 2008 

AN ASSESSMENT OF THE 

NATIONAL INSTITUTE OF STANDARDS 

AND TECHNOLOGY 

MANUFACTURING ENGINEERING 

LABORATORY 

FISCAL YEAR 2008 

Panel on Manufacturing Engineering 

Laboratory Assessments Board 

Division on Engineering and Physical Sciences 

THE NATIONAL ACADEMIES PRESS 

Washington, D.C. 

www.nap.edu 
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, N.W. Washington, DC 20001 

NOTICE: The project that is the subject of this report was approved by the Governing Board of 

the National Research Council, whose members are drawn from the councils of the National 

Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The 

members of the panel responsible for the report were chosen for their special competences and 

with regard for appropriate balance. 

This study was supported by Contract No. SB134106Z0011 between the National Academy of 

Sciences and the National Institute of Standards and Technology, an agency of the U.S. 

Department of Commerce. Any opinions, findings, conclusions, or recommendations expressed 

in this publication are those of the authors and do not necessarily reflect the views of the agency 

that provided support for the project. 

International Standard Book Number-13: 978-0-309-12731-8 

International Standard Book Number-10: 0-309-12731-9 

Copies of this report are available from 

Laboratory Assessments Board 

Division on Engineering and Physical Sciences 

National Research Council 

500 Fifth Street, N.W. 

Washington, DC 20001 

Additional copies of this report are available from the National Academies Press, 500 Fifth 

Street, N.W., Lockbox 285, Washington, DC 20055; (800) 624-6242 or (202) 334-3313 (in the 

Washington metropolitan area); Internet, http://www.nap.edu. 

Copyright 2008 by the National Academy of Sciences. All rights reserved. 

Printed in the United States of America
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged 

in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the 

general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate 

that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president 

of the National Academy of Sciences. 

The National Academy of Engineering was established in 1964, under the charter of the National Academy of 

Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection 

of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. 

The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, 

encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is 

president of the National Academy of Engineering. 

The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of 

eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. 

The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be 

an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and 

education. Dr. Harvey V. Fineberg is president of the Institute of Medicine. 

The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad 

community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal 

government. Functioning in accordance with general policies determined by the Academy, the Council has become the 

principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in 

providing services to the government, the public, and the scientific and engineering communities. The Council is 

administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest 

are chair and vice chair, respectively, of the National Research Council. 

www.national-academies.org 
PANEL ON MANUFACTURING ENGINEERING 

NEIL A. DUFFIE, University of Wisconsin, Madison, Chair

JAY RAMANATHAN, The Ohio State University, Vice Chair

VIOLA L. ACOFF, University of Alabama 

CAROL L. JONES ADKINS, Sandia National Laboratories 

DELL K. ALLEN, Utah State University 

MARYLYN HOY BENNETT, Texas Instruments Inc. (retired) 

SRINIVASAN CHANDRASEKAR, Purdue University 

BENITO FERNANDEZ-RODRIGUEZ, University of Texas, Austin 

MARION B. GRANT, Caterpillar, Inc. 

STEVEN W. HOLLAND, General Motors Corporation 

MICHAEL R. JAHADI, Lockheed Martin Aeronautics Company 

RICHARD L. KEGG, Milacron Inc. (retired) 

KARL G. KEMPF, Intel Corporation 

THOMAS R. KURFESS, Clemson University 

JAY LEE, University of Cincinnati 

MARK C. MALBURG, Digital Metrology Solutions, Inc. 

CHIA-HSIANG MENQ, The Ohio State University 

CARMEN PANCERELLA, Sandia National Laboratories 

LIVIA RACZ, Charles Stark Draper Laboratory, Inc. 

H. KUMAR WICKRAMASINGHE, University of California, Irvine 

JAMES C. WYANT, University of Arizona 

Staff 

JAMES P. McGEE, Director 

ARUL MOZHI, Senior Program Officer 

CY BUTNER, Senior Program Officer 

LIZA HAMILTON, Administrative Coordinator 

JONATHAN ELLSAESSER, Senior Project Assistant 

iv 
Acknowledgment of Reviewers 

This report has been reviewed in draft form by individuals chosen for their 

diverse perspectives and technical expertise, in accordance with procedures approved by 

the National Research Council’s Report Review Committee. The purpose of this 

independent review is to provide candid and critical comments that will assist the 

institution in making its published report as sound as possible and to ensure that the 

report meets institutional standards for objectivity, evidence, and responsiveness to the 

study charge. The review comments and draft manuscript remain confidential to protect 

the integrity of the deliberative process. We wish to thank the following individuals for 

their review of this report: 

Harry E. Cook, University of Illinois, 

Max G. Lagally, University of Wisconsin, 

Elsa Reichmanis, Georgia Institute of Technology, 

Larry Thompson, Henly, Texas, and 

Masayoshi Tomizuka, University of California, Berkeley. 

Although the reviewers listed above have provided many constructive comments 

and suggestions, they were not asked to endorse the conclusions or recommendations, nor 

did they see the final draft of the report before its release. The review of this report was 

overseen by Alton Slay, Warrenton, Virginia. Appointed by the National Research 

Council, he was responsible for making certain that an independent examination of this 

report was carried out in accordance with institutional procedures and that all review 

comments were carefully considered. Responsibility for the final content of this report 

rests entirely with the authoring panel and the institution.

v 

Contents 

SUMMARY 1 

THE CHARGE TO THE PANEL AND THE ASSESSMENT PROCESS 5 

INTELLIGENT SYSTEMS DIVISION 7 

MANUFACTURING METROLOGY DIVISION 11 

MANUFACTURING SYSTEMS INTEGRATION DIVISION 14 

PRECISION ENGINEERING DIVISION 18 

PROGRAMS FUNDED UNDER THE AMERICA COMPETES ACT 22 

OVERALL CONCLUSIONS 23 

vii 

Summary 

The mission of the Manufacturing Engineering Laboratory (MEL) of the National 

Institute of Standards and Technology (NIST) is to promote innovation and the 

competitiveness of U.S. manufacturing through measurement science, measurement services, 

and critical technical contributions to standards. This mission is aligned with the mission of 

NIST, which is to promote U.S. innovation and industrial competitiveness by advancing 

measurement science, standards, and technology in ways that enhance economic security and 

improve our quality of life. 

The MEL is organized in five divisions: Intelligent Systems, Manufacturing 

Metrology, Manufacturing Systems Integration, Precision Engineering, and Fabrication 

Technology. A panel of experts appointed by the National Research Council (NRC) assessed 

the first four divisions. Panel members visited these divisions and reviewed their activities.1 

As requested by the Director of NIST, the scope of the assessment included the 

following criteria: (1) the technical merit of the current laboratory programs relative to the 

current state of the art worldwide; (2) the adequacy of the laboratory facilities, equipment, 

and human resources, as they affect the quality of the laboratory technical programs; and (3) 

the degree to which the laboratory programs in measurement science and standards achieve 

their stated objectives and desired impact. In addition to these three criteria, the panel was 

asked by the Director of NIST to assess the projects within the laboratory conducted under 

the America COMPETES Act of 2007, which supports the President’s American 

Competitiveness Initiative (ACI).2 

In the sections below, the summary assessment of the MEL is given, followed by a 

summary assessment of each division. The chapters after the Summary then present the 

charge to the panel and a description of the assessment process, detailed assessments of the 

individual divisions, the progress of the programs funded under the America COMPETES 

Act of 2007, and overall report conclusions. 

SUMMARY ASSESSMENT OF THE LABORATORY 

 The panel’s summary assessment of the Manufacturing Engineering Laboratory is as 

follows: 

• The MEL is achieving its mission and helping NIST achieve its core mission 

through the development of needed technologies, tools, and standards in areas that 

are key to successful U.S. innovation and industrial competitiveness. 

• The MEL staff is highly capable, nationally and internationally respected, and well 

motivated; the staff possesses a strong positive outlook and is passionate about the 

work of the laboratory. 

• All four of the MEL divisions assessed have one or more projects whose technical 

work is among the best in the field. Examples include but are not limited to the 

Helium-Ion Microscopy project in the Precision Engineering Division (PED), the 

 1

 The fifth division of the MEL—the Fabrication Technology Division (FTD)—was not assessed 

because it is a support organization providing fabrication and technical support services to all NIST staff. FTD 

assists NIST staff in the design and development of instruments and measurement devices needed to maintain 

the national and international standards of measurement and measurement services. 2

 See Domestic Policy Council, Office of Science and Technology Policy, 2006, American 

Competitiveness Initiative, Washington, D.C. “America COMPETES Act” is the short title for the America 

Creating Opportunities to Meaningfully Promote Excellence in Technology, Education, and Science Act of 

2007 (Public Law 110-69). 

1
Mass and Force Metrology projects in the Manufacturing Metrology Division 

(MMD), the Automation Interoperability project in the Intelligent Systems Division 

(ISD), and the Supply Chain Integration project in the Manufacturing Systems 

Integration Division (MSID). 

• The MEL has excellent facilities and testbeds in many areas. It has maintained its 

equipment capabilities through innovative partnerships with equipment suppliers 

and other outside entities. Maintaining this level may require additional investment. 

• There is a need to reexamine program priorities through strategic planning at both 

the laboratory and division levels in order to achieve improved balance between 

attention to top-down program directives of national significance that cut across 

multiple disciplines and attention to innovative bottom-up ideas from individual 

researchers. 

• The continued achievement of technical objectives and the maintenance of core 

competencies will require taking action with respect to personnel succession 

planning, more aggressive hiring of young permanent staff, and addressing 

shrinking sources of personnel in key areas such as core manufacturing 

technologies, metrology, and optics. A lack of resources threatens the development 

of needed technologies and tools and the ability to hire more postdoctoral 

associates. Moreover, making use of guest workers who do not have the possibility 

of permanent employment does not address the succession planning issue. 

• It is too early to assess the progress of programs in the MEL that are funded 

through the President’s American Competitiveness Initiative and the America 

COMPETES Act of 2007. The support that these initiatives promise is providing 

for important Supply Chain Integration and Bio-Imaging programs. Future 

expanded MEL involvement in such initiatives is encouraged, and should take into 

account the fact that manufacturing includes broader areas of supply chain, life 

cycle, and service. To have greater impact on the core mission of NIST, the very 

definition of “manufacturing” should be expanded to cover the broader areas of 

service, enterprise productivity, and products for areas such as health care, farming 

automation, and intelligent transportation systems. 

Overall, the four individual divisions are performing to the best of their ability, given 

available resources. In many areas in all four divisions, the capabilities and the work being 

performed are among the best in the field. However, reduced funding and other factors such 

as difficulty in hiring permanent staff are limiting (and are likely to increasingly limit) the 

degree to which MEL programs can achieve their objectives and are threatening the future 

impact of these programs. 

SUMMARY ASSESSMENTS OF FOUR DIVISIONS 

Intelligent Systems Division 

The Intelligent Systems Division develops intelligent systems technologies, tools, and 

standards that have major impacts on the successful progress of U.S. innovation and 

industrial competitiveness. It has established strong leadership in the areas of intelligent 

systems measurement, interoperability, safety, and security. 

Measurement has been one of the ISD’s major strengths, and the division has 

maintained its excellent reputation and technology leadership in this area. In the area of 

interoperability, the focus has been on establishing standards and testbeds in the areas of 

computer numerical control (CNC), dimensional inspection equipment interoperability, and 

2
autonomous-vehicle and materials-handling systems, with the ISD serving as a catalyst in 

promoting collaboration among industries. In the area of safety and security, the ISD has 

made important contributions in establishing NIST SP 800-53, Recommended Security 

Controls for Federal Information Systems, and NIST SP 800-82, Guide to Industrial Control 

Systems Security. The ISD has technical capabilities in its core programs that are among the 

best. These programs include micro- and nanomanufacturing measurement and positioning 

systems, Standard for the Exchange of Product Model Data: Numerical Control (STEP NC)3

and Open Modular Architecture Controller (OMAC) for real-time data models, machine 

compensation, machining tool path optimization, and Ethernet/Internet Protocol (IP) 

performance test tools. 

While ISD facilities are generally very good, the human resources available to 

achieve important goals are inadequate. The prognosis for hiring successor personnel appears 

to be problematic in key areas, potentially threatening future core competency in the ISD’s 

critical skills. Finally, the ISD has opportunities to expand its capabilities and impact in 

diversified, value-added industries, covering broader areas of manufacturing relevance 

including the areas of service, enterprise productivity, health care, farming automation, and 

intelligent transportation systems. 

Manufacturing Metrology Division 

The Manufacturing Metrology Division programs compare very favorably with peer 

activities at the national standards institutions of other countries. For example, the Mass and 

Force Metrology projects are at the forefront in their domain. Even where investments in 

programs at other laboratories, such as the Physikalisch-Technische Bundesanstalt (PTB) in 

Germany, are higher, the MMD staff has the ability to maintain competitive programs. This is 

reflected in the results of a worldwide round-robin study conducted in the mass area. Also, in 

emerging areas such as wireless sensors, NIST researchers are acknowledged leaders in the 

development of standards. Given the likely widespread impact of wireless sensors 

technology, it is critical that this leadership role be maintained with appropriate investment. 

MMD staff members are active in standards committees for such areas as mass 

metrology and wireless sensors. They are active in the dissemination of results to a wide 

audience through the organization of and participation in meetings and conferences. An 

example is the upcoming International Academy for Production Engineering (CIRP) 

symposium in the machining area that is to be held at NIST. 

The MMD staff reviewed by the panel are extremely capable and well motivated, and 

they have a strong positive outlook. These facts, coupled with excellent facilities and 

equipment, have yielded high-quality work. However, a number of concerns may impede the 

continued excellent performance of the MMD. For example, more staff members are needed 

in critical areas such as optics. The hiring of trained replacements for soon-to-retire senior 

staff members and the recruiting and retaining of young engineers and scientists are needed. 

The use of postdoctoral associates is an excellent vehicle for bringing in and training new 

young talent; however, making use of the availability of guest workers who do not have the 

possibility of permanent employment is unlikely to address longer-term strategic personnel 

issues. 

 3

 STEP NC is a standard developed by the International Organization for Standardization (ISO) to 

bring computer-aided design/computer-aided manufacturing (CAD/CAM) data into CNC. 

3
Manufacturing Systems Integration Division 

The Manufacturing Systems Integration Division has an excellent international 

reputation and excellent technical expertise in electronic information exchange, rich semantic 

structures, and systems integration. The MSID has appropriately focused its scope on three 

major programs: Supply Chain Integration, Sustainable and Lifecycle Information-based 

Manufacturing, and Simulation-based Interoperability Standards and Testing. For each of 

these programs, the outputs from the MSID include rigorously defined standards and 

protocols, realistic pilot programs, and software and interoperability testing services. 

The scope of the MSID’s programs is in line with its budget, and it has produced 

high-quality research with concrete results. In general, the MSID is doing excellent technical 

work that is at the state of the art. The groups in the division that are researching and 

influencing interoperability standards are performing technical work that ranks among the 

best. This has been demonstrated, for example, with interoperability standards, the 

development and testing of standards, simulation, and the assisting of vendors in the 

implementation of the new functionality. Improved dissemination of the excellent work done 

in the MSID group on semantic interoperability could have significant impact far beyond the 

Supply Chain Integration project. Core personnel are working very well as a team, providing 

overall strong leadership. 

However, the effective MSID budget has been severely reduced over recent years, 

resulting in a shift from permanent personnel to guest researchers; furthermore, there appears 

to be a lack of successor personnel in key positions. These changes are reducing the 

division’s agility, effectiveness, ability to achieve stated objectives, and national impact. 

Precision Engineering Division 

The Precision Engineering Division is providing the foundation of dimensional 

measurement that is crucial to the U.S. industrial and scientific communities. The move to the 

division’s current facilities, which are the best of their kind, with their stringent environmental control, has resulted in a dramatic improvement in metrology capabilities. The PED 

has maintained equipment capabilities that are among the best through innovative 

partnerships with equipment suppliers and other outside entities. The PED’s efforts in 

helium-ion microscopy, atom-based metrology, nanoparticles for biosystems, and whole 

wafer/photomask capability are forward-looking. The PED has had major impact both 

nationally and internationally by delivering new standard reference materials, calibration 

services, and documentation. 

The PED research staff reviewed by the panel are knowledgeable, dedicated, and 

enthusiastic about their work. The PED has made significant improvements over the past 3 

years in meeting the needs of the semiconductor industry. Future opportunities exist for the 

division to seek feedback from U.S. industry, to increase the visibility of the PED within the 

national technical community, and to promote traceability and standards development. 

Benchmarking is performed currently for other national standards institutions, but this 

activity could be expanded to industry in order to capture significant cutting-edge capabilities 

not available at other national standards institutions. 

The PED needs more favorable cost structures to support the delivery of calibration 

and measurement services; these include, for example, the use of machine charges, 

consumables charges, service center charges, and activity-based cost accounting. Strategic 

planning is likely to be crucial in the following areas: capital equipment; human resources, 

including the succession of personnel; technical focus areas (roadmaps); and investment and 

disinvestment in calibration and measurement services. 

4
The Charge to the Panel and the Assessment Process 

At the request of the National Institute of Standards and Technology, the National 

Academies, through its National Research Council, has since 1959 annually assembled panels 

of experts from academia, industry, medicine, and other scientific and engineering 

environments to assess the quality and effectiveness of the NIST measurements and standards 

laboratories, of which there are now nine,4 as well as the adequacy of the laboratories’ 

resources. In 2008, NIST requested that five of its laboratories be assessed: the Building and 

Fire Research Laboratory, the Manufacturing Engineering Laboratory, the Materials Science 

and Engineering Laboratory, the NIST Center for Neutron Research, and the Physics 

Laboratory. Each of these was assessed by a separate panel of experts; the findings of the 

respective panels are summarized in separate reports. This report summarizes the findings of 

the Panel on Manufacturing Engineering. 

For the fiscal year (FY) 2008 assessment, NIST requested that the panel consider the 

following criteria as part of its assessment: 

1. The technical merit of the current laboratory programs relative to the current state 

of the art worldwide; 

2. The adequacy of the laboratory facilities, equipment, and human resources, as 

they affect the quality of the laboratory technical programs; and 

3. The degree to which the laboratory programs in measurement science and 

standards achieve their stated objectives and desired impact. 

In addition, because NIST has begun to receive increases in funding through the 

President’s ACI and the America COMPETES Act of 2007, the Director of NIST also 

requested that the assessment panels specifically examine and review the progress of all of 

the FY 2007-funded initiatives relevant to their respective laboratories and comment on these 

program growth areas explicitly in their reports. 

The context of this technical assessment is the mission of NIST, which is to promote 

U.S. innovation and industrial competitiveness by advancing measurement science, 

standards, and technology in ways that enhance economic security and improve the quality of 

life. The NIST laboratories conduct research to anticipate future metrology and standards 

needs, to enable new scientific and technological advances, and to improve and refine 

existing measurement methods and services. 

To accomplish the assessment, the NRC appointed a panel of 21 volunteers whose 

expertise matched that of the work performed by the Manufacturing Engineering Laboratory 

staff. Each panel member was also assigned to one of four review teams whose members’ 

expertise matched that of the work performed by the staff in the four MEL divisions being 

assessed: Intelligent Systems, Manufacturing Metrology, Manufacturing Systems Integration, 

and Precision Engineering.5

 

The panel met at the NIST facilities in Gaithersburg, Maryland, on March 26-28, 

2008. After the full panel met for a session of overview presentations by the MEL 

management and staff on March 26, the panel divided into its four review teams, each of 

which then visited its respective division for a little over a full day (on March 26-27). During 

 4

The nine NIST laboratories are the Building and Fire Research Laboratory, the Center for Nanoscale 

Science and Technology, the Chemical Science and Technology Laboratory, the Electronics and Electrical 

Engineering Laboratory, the Information Technology Laboratory, the Manufacturing Engineering Laboratory, 

the Materials Science and Engineering Laboratory, the NIST Center for Neutron Research, and the Physics 

Laboratory. 5

 See http://www.mel.nist.gov/ for more information on MEL programs. Accessed August 12, 2008. 

5
these visits, the review team members attended presentations, tours, demonstrations, and 

interactive sessions with the MEL staff. Immediately following the review team meetings, the 

full panel assembled for a meeting on March 27-28 during which it attended interactive 

sessions with the laboratory managers and also met in a closed session to deliberate its 

findings and to define the contents of this assessment report. 

The approach of the panel to the assessment relied on the experience, technical 

knowledge, and expertise of its members, whose backgrounds were carefully matched to the 

technical areas within which the MEL activities are conducted. The panel reviewed selected 

examples of the standards and measurements activities and the technological research 

presented by the MEL. It was not possible to review the MEL programs and projects 

exhaustively. The examples reviewed by the panel were selected by the MEL. The panel’s 

goal was to identify and report salient examples of accomplishments and opportunities for 

further improvement with respect to the following: the technical merit of the MEL work, its 

impact with respect to achieving its own definition of its objectives, and specific elements of 

the MEL’s resource infrastructure that are intended to support the technical work. These 

highlighted examples for each MEL division are intended collectively to portray an overall 

impression of the laboratory, while preserving useful suggestions specific to projects and 

programs that the panel considered to be of special note within the set of those examined. The 

assessment is currently scheduled to be repeated biennially. While the panel applied a largely 

qualitative rather than a quantitative approach to the assessment, it is possible that future 

assessments will be informed by further consideration of various analytical methods that can 

be applied. 

The comments in this report are not intended to address each program within the 

MEL exhaustively. Instead, this report identifies key issues and focuses on representative 

programs and projects relevant to those issues. Given the necessarily nonexhaustive nature 

of the review process, the omission of any particular MEL program or project should not be 

interpreted as a negative reflection on the omitted program or project. 

The report’s Summary first highlighted issues that apply broadly to several or all of 

the divisions or to the laboratory as a whole. Then, after this chapter on the charge to and 

approach taken by the panel, individual chapters present observations specific to the 

respective laboratory divisions. Comments on the progress of the programs funded under the 

America COMPETES Act are followed by overall conclusions. 

6
Intelligent Systems Division 

The Intelligent Systems Division has established strong leadership in the areas of 

measurement, interoperability, safety, and security. It plays a vital role in promoting 

manufacturing-engineering-related science, technologies, and tools, and its standards and 

services are crucial to U.S. innovation and industrial competitiveness. The ISD has 

maintained its excellent reputation and technology leadership in measurement. It is serving as 

a catalyst in promoting collaboration between industries, with a focus on establishing 

standards and testbeds in the areas of CNC, dimensional inspection equipment 

interoperability, and autonomous-vehicle and materials-handling systems. In the areas of 

safety and security, the ISD has established new standards for the security of industrial 

control systems. The ISD’s technical capabilities are among the best in micro- and 

nanomanufacturing positioning systems, STEP NC and OMAC for real-time data models, 

machine compensation, machining tool path optimization, and Ethernet/IP performance 

testing. 

TECHNICAL MERIT RELATIVE TO STATE OF THE ART 

The ISD programs are well aligned with NIST’s core roles and strategy, focusing on 

measurements, standards, and technology development for the benefit of industry and 

commerce. The division has done an excellent job of identifying the major needs and 

opportunities of U.S. industry. However, given restrictions in budget and personnel, it has 

been unable to pursue these opportunities fully. 

The caliber of the work of the ISD programs assessed is at the state of the art. The 

division’s collaboration with top universities, industry, and military agencies demonstrates 

the reputation and competence of the ISD products and services. The dimensional inspection 

equipment interoperability effort is the leader in cooperation with multinational companies. 

The division has made significant efforts at reaching out to industrial partners, educating 

them with respect to the benefits of standardization as a means to achieve cost reduction and 

interoperability. There is a healthy level of technical publications, invited talks, external 

awards, and participation in national and international committees by ISD staff. 

Measurement has been one of the ISD’s major strengths, which has been maintaining 

its technical leadership reputation. The following observations pertain to the area of 

measurement science for manufacturing robotics and automation: 

• Six-dimensional dynamic sensing represents the state-of-the-art technology for 

sensor calibration. 

• In the area of micro-, meso-, and nanomanufacturing and positioning (measurement 

and manipulation), the ISD has demonstrated significant improvement in the 

accuracy of measurements. The division could conduct a benchmarking workshop. 

Such a workshop could invite experts from the international community to evaluate 

how the competitive science and technologies from other research laboratories, 

institutions, and industry in the related areas can further enhance the standards 

efforts at NIST. 

• The ISD has significantly facilitated the establishment of the following industrial 

robot safety standards: ANSI [American National Standards Institute] B56.5: Safety 

Standard for Guided Industrial Vehicles and Automated Functions of Manned 

Industrial Vehicles and ISO [International Organization for Standardization] 

10218-1:2006: Robots for industrial environments—Safety requirements—Part 1: 

Robot, which have been deployed by industry users. 

7
• The ISD’s efforts in establishing consortia and competitions in robotics and sensor 

technologies for localization, detection, navigation, and materials handling 

constitute a commendable approach for leveraging diversified, global resources. 

Interoperability work in the ISD has focused on establishing standards and testbeds in 

the areas of CNC and dimensional inspection equipment. The ISD has been serving as a 

catalyst in promoting industrial collaboration. In the area of automation interoperability 

standards: 

• The ISD has done an excellent job in bringing together competitors and partners 

and aligning them to a common standard. Demonstrations have been effectively 

used to facilitate compliance. 

• STEP NC and OMAC for real-time data models, machine compensation, and 

machining tool path optimization have been ongoing efforts at the ISD, which has 

established ISO 10103 AP238: STEP NC—Standard for the Exchange of Product 

Model Data—Numerical Control and I++DME interoperability testing protocols. 

The ISD should consider working with industry associations and societies (such as 

the Association for Manufacturing Technology and the Society of Manufacturing 

Engineers and others) to establish user groups or working groups for further 

dissemination. Further, the ISD can move on to next-level challenges in enterpriselevel data modeling and standards efforts. 

• Safety and security work has focused on industrial controls and networks standards 

for federal government and industrial users. NIST SP 800-53 was established as 

Recommended Security Controls for Federal Information Systems. NIST SP 800-82 

was established as Guide to Industrial Control Systems Security in the areas of 

intelligent manufacturing industrial control systems and network standards. 

• Ethernet/IP performance test tools in partnership with the Open DeviceNet Vendor 

Association are a good way to promote standards in industry. This is a good area 

for the ISD to lead and in which to facilitate a national and international effort in 

establishing standards and guidelines. 

• The ISD might explore other opportunities in industrial wireless networks and 

diagnostics-related standards. 

ADEQUACY OF INFRASTRUCTURE 

The ISD has been successfully leveraging its program development through 

partnerships with industry and government. It has excellent research facilities and equipment, 

some of which are loaned by industry. The resources are limited (especially in terms of 

personnel), constraining the ability to respond with agility to changing technology needs. ISD 

researchers appear to be engaged in a number of collaborative projects for promoting crossdisciplinary activities. However, these diversified activities impede the impact of the ISD 

owing to the high workload of its staff as well as the lack of committed resources. The ISD 

has staff members approaching retirement; there have been several recent retirements, with 

only one new permanent technical staff member hired in the past 3 years. There should be a 

robust and stable succession plan to guarantee core competency in the ISD’s recognized 

critical expertise. 

8
ACHIEVEMENT OF OBJECTIVES AND IMPACT 

Regarding the degree to which ISD programs have achieved their stated objectives 

and desired impact: 

• The ISD has made major achievements in terms of its program objectives. It has 

had visible and important impacts on the establishment of standards for government 

and industry user groups in measurement science for robotic safety, machine tools 

interoperability, autonomous guided vehicles, and materials-handling systems. 

• The RoboCup Rescue competition6 has made visible impacts on the global robotics 

research community. Other programs could use this leveraging model for future 

collaboration and partnerships with the international research community. The ISD 

could also further develop partnerships with the National Science Foundation, the 

Department of Defense, the National Aeronautics and Space Administration 

(NASA), and others to leverage their Research Experience for Undergraduates, 

Research Experience for Teachers funding to encourage more K-12 teachers and 

undergraduate students to participate in its future activities. 

• Participation in consortia and external committees (e.g., automatic guided vehicle 

manufacturers’ consortium, OMAC, and others) shows an overarching effort to 

benefit industry. 

CONCLUSIONS 

The conclusions of the panel based on its assessment of the Intelligent Systems 

Division are as follows: 

• The division programs are well aligned with NIST’s core roles and strategy, and 

the division has done an excellent job of identifying the major needs of and 

opportunities with respect to U.S. industry. The division has shown increased 

emphasis on the concerns of U.S. industry about risk, cost, and compatibility of 

manufacturing technology. 

• The caliber of the work in the division is at the state of the art. The ISD has 

excellent research facilities and equipment. Measurement is a major area of 

strength in which it maintains a position of world leadership. Its dimensional 

inspection equipment interoperability effort, for example, is the leader in 

cooperation with multinational companies. 

• The division has a healthy level of technical publications, invited talks, external 

awards, and participation in national and international committees. It reaches out to 

industrial partners regarding the benefits of standardization as a means for cost 

reduction and product improvement. 

• Limited resources are constraining the agility with which the division can respond 

to changing technology needs. The division has staff members approaching 

retirement and needs a robust and stable succession plan to guarantee its future core 

competency. 

• Current efforts within the division are well aligned with opportunities in embedded 

machine and device intelligence, machine health prognostics and management 

 6

 See http://www.isd.mel.nist.gov/PerMIS_2007/proceedings/Papers/PerMIS07.Final_Balakirsky.pdf. 

Accessed August 15, 2008. 

9
standards, and the detection and prevention of product counterfeits, as well as 

virtual engineering simulation standards and tools. 

10
Manufacturing Metrology Division 

The mission of the Manufacturing Metrology Division is to fulfill the measurements 

and standards needs of U.S. discrete-parts manufacturers in mechanical metrology and 

advanced manufacturing technology. The division aims to conduct metrology research, 

provide calibration services, develop standards, establish traceability, and produce highquality scientific and technical output. It conducts research and development in realizing and 

disseminating the Systeme International (SI) mechanical units; develops methods, models, 

sensors, and data to improve metrology, machines, and processes; provides services in 

mechanical metrology, optics metrology, machine metrology, process metrology, and sensor 

integration; and leads in the development of national and international standards. The 

division currently has 35 full-time staff, 1 NRC postdoctoral researcher, and 14 guest 

researchers. Its yearly funding for 2007 was about $10 million, with about 20 percent of this 

total coming from extramural sources. 

The staff of the MMD reviewed by the panel is extremely capable and well motivated 

and has a strong, positive outlook. These facts, coupled with excellent facilities and 

equipment, have yielded high-quality work. However, a number of concerns may impede the 

continued excellent performance of the MMD. For example, more permanent staff members 

are needed in critical areas such as optics. Trained replacements are needed for soon-to-retire 

senior staff members. Succession planning should be executed so that gaps can be identified 

and young engineers and scientists recruited. The use of postdoctoral associates and the 

funding of graduate students are excellent vehicles for bringing new young talent into the 

division and properly training them. However, making use of the availability of guest 

workers who do not have the possibility of permanent employment does not address the 

succession planning issue. 

TECHNICAL MERIT RELATIVE TO STATE OF THE ART 

The MEL is well known for the high quality of its work in manufacturing metrology, 

and industry seems pleased with NIST’s calibrations and leadership. The MMD staff is 

extremely positive and motivated; many staff members have worked at NIST for many years 

and seem proud of their work. In some areas, especially optics metrology, the group is 

smaller than its reputation would suggest; the number of important projects that it has 

warrants a larger staff. The wireless sensors work is an exciting emerging area with high 

potential for growth. The MEL should position itself well to be a leader in setting global 

standards in this important area. The work in improving mass metrology and small-force 

measurement technology is extremely advanced and should prove to be very productive and 

useful to industry. There is considerable scope for outstanding publications in high-impact 

journals in this area. The new projects in optics involving the testing of the phase-transfer 

function of an interferometer and the use of nanostructured optics for the testing of mandrels 

for x-ray telescope mirrors and the measurement of the radius of curvature of large, precision, 

spherical surfaces in the radius-of-curvature range of 10 m < R < 1,000 m are important and 

should be valuable for the optics industry. The work on measurement methods for machine 

dynamics and machining process modeling is well done and leverages NIST’s historic 

expertise in measurements and machining. This is long-term research; significant findings 

will require the provision of long-term support. Given the high-risk nature of this work, it is 

doubtful that significant fundamental results will be generated in a year or two. If this project 

is pursued, it should be supported for a longer duration regardless of initial results. 

The MMD programs compare very favorably with peer activities at the other National 

Metrology Institutes (NMIs). For example, the Mass and Force Metrology projects are at the 

11
forefront in their domain. Even where investments in programs at other laboratories—for 

example, PTB in Germany—are higher, the MMD staff is confident of its ability to maintain 

competitive programs. This is reflected in the results of a worldwide round-robin 

measurement comparison conducted in the mass and force areas. Also, in emerging areas 

such as wireless sensors, MMD researchers are acknowledged leaders in development of 

standards. Given the likely widespread impact of this technology, it is critical that this 

leadership role be maintained with appropriate investment. 

ADEQUACY OF INFRASTRUCTURE 

The facilities and equipment of the MMD are very good, with state-of-the-art 

equipment found in each of the programs assessed. For example, the large-mass calibration 

equipment is unparalleled. Other examples include optics (flat and spherical-surface 

metrology equipment) and the NIST nano-optics fabrication facilities. The staff is highly 

motivated and appears to approach its assigned tasks with enthusiasm. The facilities and 

equipment appropriately support the MMD core mission and the carrying out of its various 

projects to completion. 

Some weaknesses exist with respect to personnel matters and succession planning. 

For example, a mechanism to hire replacements in anticipation of retirements appears to be 

lacking, a source of concern to staff. Furthermore, there are too few staff members in some 

key areas such as optics, given the facilities and importance of this area. The reliance on 

guest researchers to carry out core activities is a cause of concern. Among other things, there 

is a loss of knowledge and experience when there is insufficient overlap with succeeding 

guest researchers. More postdoctoral associates are needed. 

ACHIEVEMENT OF OBJECTIVES AND IMPACT 

The MMD continues to do a very good job of developing standards, providing 

calibration services, and establishing traceability. It remains at the top of the field in force 

measurement, optics metrology, machine tool metrology, and acoustics metrology. The 

MMD staff are active in standards committees in such areas as mass metrology. Their work 

on wireless sensors standards committees has resulted in establishing standards for wireless 

data communication among sensors. They are active in the dissemination of results to a wide 

audience through the organization of and participation in meetings and conferences. An 

example is the upcoming CIRP symposium in the machining area that is to be held at NIST. 

Several drivers for U.S. manufacturing have influenced the content of the MMD 

programs. Through frequent interactions and collaborations, the division ensures that MMD 

measurements and standards activities address the needs and priorities of its customers. The 

division also produces high-quality work on optics metrology and small-force measurement 

technology. It is, however, not clear how research projects are chosen. It is suggested that 

studies such as those by the NRC on grand challenges for manufacturing be considered in the 

interest of fostering more high-risk, high-payoff work. 

The division evinced multiple instances of good interlaboratory collaboration. Most 

notable among these is one between the Mass Metrology Project and the Argonne National 

Laboratory. The project identified a unique, diamond-like carbon coating developed at 

Argonne as a highly effective protective layer for new mass artifacts being developed for 

future mass dissemination purposes and has exploited this well. The coating originally was 

developed as a low-friction coating at Argonne. The impact of this collaboration is likely to 

be high for both participants. 

12
While the publication output is high, the staff should be encouraged to publish more 

in peer-reviewed journals with high impact factors. This would also be a plus in attracting 

high-quality postdoctoral associates. A study and mapping of customer needs with MMD 

capabilities would be very useful for highlighting the accomplishments of the program in the 

service of industry, federal, and state organizations. 

CONCLUSIONS 

The conclusions of the panel based on its assessment of the Manufacturing Metrology 

Division are as follows: 

• The MMD is at the top of the field in force measurement, optics metrology, 

machine tool metrology, and acoustics metrology. It has very good facilities and 

equipment that have yielded high-quality work. Its programs compare very 

favorably with peer activities at the other NMIs. 

• The MMD ensures that its measurements and standards activities address the needs 

and priorities of its customers. Examples include the division’s work in wireless 

sensors standards, in improving mass metrology and small-force measurement 

technology, and in the measurement of the radius of curvature of large, precision, 

spherical surfaces. 

• The MMD continues to do a very good job of developing standards, providing 

calibration services, and establishing traceability. The MMD staff is active in 

standards committees and in the dissemination of results to a wide audience 

through its organization of and participation in meetings and conferences. 

• Permanent staff members are needed in critical areas such as optics. Succession 

planning should be used to identify gaps and to recruit young engineers and 

scientists. The reliance on guest workers who do not have the possibility of future 

employment should be reduced in favor of postdoctoral associates and graduate 

students. 

• While the publication output is high, the staff should be encouraged to publish 

more in peer-reviewed journals with high impact factors. 

• A study and mapping of customer needs with MMD capabilities would be very 

useful for highlighting the accomplishments of the program in the service of 

industry, federal, and state organizations. 

13
Manufacturing Systems Integration Division 

The Manufacturing Systems Integration Division develops and applies interoperability standards and measurements for software used in manufacturing systems 

integration. The MSID has expertise in electronic information exchange, semantic structures, 

and systems integration that is comparable to that of the top 10 institutions in these areas. The 

MSID has identified supply chain integration, sustainable and lifecycle information-based 

manufacturing, and simulation-based manufacturing interoperability standards and testing as 

focus areas in addressing the ever-increasing complexity of manufacturing and supply chain 

processes. All division projects are aligned with one of these programs. The results from each 

project in the MSID include rigorously defined standards and protocols, realistic pilot 

programs, and software and interoperability testing services. 

The goal of the Supply Chain Integration Program is to demonstrate an infrastructure 

for the testing and integration of automated systems that exchange data and information 

(including semantics) throughout the supply chain. This program has demonstrated 

significant maturity during the past 3 years. One important impact of the program is the 

deployment of a number of syntax- and quality-based testing and validation tools. Another is 

significant advances in techniques for the sharing of semantic information between supply 

chain partners. Many of these tools and techniques have broad applicability within 

manufacturing and in other disciplines. 

The Sustainable and Lifecycle Information-based Manufacturing Program works on 

extending product and process data standards to the broader spectrum of information needed 

to sustain products over their life cycle. This program has produced a number of key results 

in the past few years, including transitioning STEP to integrate with Object Management 

Group (OMG) and World Wide Web Consortium (W3C) standards and migrating STEP for 

wider implementation. Continued work in this area is vital to the efficient allocation of both 

manufacturing capacity and valuable materials as well as to the protection of the 

environment. 

The Simulation-based Manufacturing Interoperability Standards and Testing Program 

is particularly topical, as standards in simulation are in their infancy. It is a good strategy for 

the MSID to invest in this area, as it is critical to industry. As the complexity of 

manufacturing systems continues to increase, it becomes exponentially more complex to 

design and operate these systems without multiple integrated software tools, including in 

many cases both simulations and manufacturing automation systems. Collaboration with the 

Department of Homeland Security (DHS) should pave the way to expanding this work into 

the management of manufacturing and supply chain disaster recovery. 

The MSID has increasingly focused its programs, which are in line with the MEL 

mission. Because funding and human resources have remained static or declined, it has been 

necessary for this division to remain lean and yet agile. Its technical work is generally 

excellent. Most of its projects are outstanding and have delivered significant standards, 

pilots, and testing programs that have benefited the targeted user communities. 

TECHNICAL MERIT RELATIVE TO STATE OF THE ART 

The three focus areas (supply chain integration, sustainable and lifecycle informationbased manufacturing, and simulation-based manufacturing interoperability standards and 

testing) are appropriately aligned with the needs of American industry. In addition, the 

projects within each focus area are in line with the MEL mission and reflect a broad 

understanding of comparable work being done elsewhere (in other government laboratories, 

14
at universities, and in industry). Links have been established with key members of the 

international community to achieve the division’s goals. 

In the supply chain integration area, high-quality work is being performed on logic 

and semantics in the basic Automated Methods for Integrating Systems (AMIS) concept and 

applied AMIS-related projects. This is a difficult problem of utmost importance to improving 

supply chain operational efficiency. The MSID has developed a very innovative approach in 

the modeling of logic-based semantics. Furthermore, the Naming and Design Rules and 

Quality of Design services are very important to facilitating and accelerating supply chain 

integration. These services in fact address problems encountered in areas beyond 

manufacturing and could form the basis of broadly applicable leading-edge capabilities. 

In the sustainable and lifecycle information-based manufacturing area, the MSID has 

had a very strong impact in STEP development. It has continued the development and testing 

of STEP modular development environment and modular application protocols; in particular, 

work relating to AP203 E2: Configuration Controlled 3D Design of Mechanical Parts and 

Assemblies is to be commended. This new (E2) version of AP203 was modularized and 

updated to include several module sets for new applications. Recent MSID efforts include 

transitioning STEP to integrate with OMG and W3C standards and migrating STEP for wider 

implementation. 

In the simulation-based manufacturing interoperability standards and testing area, 

work is proceeding with the collection and analysis of test cases, with the goal of structuring 

appropriate standards. While this work is going well and is important to building a 

technically sound foundation for future work, this program should be strategic in its selection 

of pilot programs and should not be seduced by the opportunity simply to build 

demonstrations. The DHS funds several projects within the simulation program. The MSID 

should not entirely rely on this soft money to grow this program. Since this integration 

infrastructure that the DHS funding partially supports has dual uses and can be applied to 

manufacturing, the division has opportunities to seek other agency funding to sustain its core 

mission and expertise. 

Within the MSID, the groups researching and influencing interoperability standards7

are performing technical work that ranks with the best in the field. This has been 

demonstrated with the development and testing of standards and the assisting of vendors in 

the implementation of the new functionality. Such activities have also been recognized 

through awards to staff members—for example, the Fellow of the American Society of 

Mechanical Engineers (ASME) award, and the Product Data Exchange using STEP (PDES, 

Inc.) Technical Excellence award for the technical efforts and leadership in developing the 

STEP modular architecture. Furthermore, standards bodies are engaged and practical 

problems are formulated and piloted in collaboration with key vendors and end customers. 

For example, recent MSID collaborative efforts include its work with the Automotive 

Industry Action Group (AIAG), aerospace companies, and the National Archives and 

Records Administration in the area of standards-based data archiving, and its work with the 

DHS in the area of simulation. 

ADEQUACY OF INFRASTRUCTURE 

Given the MSID’s information-centric paradigm, its equipment and facilities are 

adequate to meet project objectives. The scale of specific projects and work planning are 

appropriate, given limited resources. 

 7

 See http://www.isd.mel.nist.gov/projects/metrology_interoperability/. Accessed September 2, 2008. 

15
As of January 2008, staffing for the Manufacturing Systems Integration Division 

included 57 full-time-equivalent personnel, which included approximately 30 full-time 

permanent positions, as well as 41 visiting researcher or part-time workers who were 

equivalent to 27 full-time employees. The trend over the past 5 years has been a steady 

decrease in full-time permanent positions and an increase in part-time and visiting personnel. 

The actual-dollar annual budget has remained relatively constant over this same period, at 

roughly $10 million in a time of generally increasing cost in all areas. This has a number of 

negative impacts in the context of this division’s achieving its goals: 

• The MSID is approaching a critical mass threshold, and it is not provided with an 

appropriate budget for developing and executing a succession plan. This will soon 

affect the division’s effectiveness, agility, and competitiveness. 

• Increasing reliance on guest researchers is necessary to accomplish tasks but not 

sufficient to sustain the MSID’s mission. An infusion of temporary new help is 

very useful, but it is not a substitute for a continuous flow of permanent new hires. 

• Given current funding and the need to be opportunistic, the MSID is applying more 

tactical technical planning strategies than it prefers. 

ACHIEVEMENT OF OBJECTIVES AND IMPACT 

With the limited resources that the MSID has, it has carefully scoped its focus and has 

aggressively sought out the most effective partnerships to identify and address the 

requirements and the gaps in the standards. For example, as a result of collaborations with 

automotive and aerospace companies, the MSID has had a very strong impact in 

interoperability. The MSID disseminates its work through government and industry, NISTwide, and at internal forums. Work has also been disseminated through a significantly high 

number of workshops, technical seminars, reports, publications, newsletters, board 

memberships, and consortia. Some examples are AIAG and the involvement with STEP 

development. 

The MSID should be innovative with its marketing strategy and standards 

development in order to further increase its impact on U.S. competitiveness in the 

manufacturing arena. 

CONCLUSIONS 

The conclusions of the panel based on its assessment of the Manufacturing Systems 

Integration Division are as follows: 

• The MSID has expertise in electronic information exchange, rich semantic 

structures, and systems integration that ranks it among the best in these fields. It 

has appropriately focused its scope on three major programs: Supply Chain 

Integration, Sustainable and Lifecycle Information-based Manufacturing, and 

Simulation-based Interoperability Standards and Testing. For each of these 

programs, the outputs from MSID include rigorously defined standards and 

protocols, realistic pilot programs, and software and interoperability testing 

services. The division’s scope is in line with its budget, and it has produced highquality research with concrete results. 

• The overall quality of research in the MSID is high. In general, the division is 

doing excellent technical work. In many areas, the work at MSID is state of the art. 

In the MSID, the groups researching and influencing interoperability standards are 

16
performing work that ranks among the best in the field. This has been demonstrated, for example, with interoperability standards—with the development and 

testing of standards, simulation, and the assisting of vendors in the implementation 

of the new functionality. Core personnel work very well as a team, providing 

overall strong technical leadership. 

• The effective budget has been severely reduced over the past 5 years, responded to 

by a shift from permanent personnel to guest researchers. Current funding may be 

adequate to achieve tactical results, but more strategic projects and programs are 

being sacrificed due to lack of funding. 

• The MSID is approaching a critical-mass threshold, and the division is not provided 

with an appropriate budget for having employee succession planning in place in 

order to reduce the number of guest researchers over the years and to hire new 

permanent employees. If this situation is not addressed in a timely manner, it will 

very negatively impact the MSID’s ability to remain effective, agile, and 

competitive. 

• An improved dissemination of information and results would increase the national 

impact of the excellent work done in the MSID. For example, semantic 

interoperability has had significant impact potential far beyond the Supply Chain 

Integration project. 

• The MSID staff has demonstrated commendable success in getting outside funding 

(such as that received from DHS). 

• The MSID is collaborating well with the Intelligent Systems Division, the 

Manufacturing Metrology Division, and the Precision Engineering Division on 

interoperability standards projects. Because of its lean budgets, the MSID should 

continue to leverage its expertise with other organizations in order to solve 

problems of national interest. 

17
Precision Engineering Division 

The Precision Engineering Division continues to maintain activities and services, 

many of which are among the best in the field, in many areas of research despite significant, 

cumulative cuts in staffing and budgets. For a relatively small amount of funds, the PED is 

performing a critical role for the nation, carrying on the ever-changing and challenging 

legacy of the National Bureau of Standards and now NIST. The work of this division is 

crucial to the current and future competitiveness of U.S. industry and in standards 

development and traceability to the SI. The PED staff is knowledgeable, dedicated, and 

highly motivated in its work. 

Since the previous assessment, the PED has made major strides in the following 

areas: 

• The realization of the improved laboratory environment that it now has in the 

Advanced Measurements Laboratory (AML); 

• New equipment that the division has acquired, augmented, put to use, and 

developed for improved capability in establishing new and forward-looking 

calibrations and measurement service capabilities; 

• The generation of a significant number of new capabilities, artifacts, procedures, 

publications, and standards—all to make significant improvement in order to meet 

its mission and the needs of its customer base. 

TECHNICAL MERIT RELATIVE TO STATE OF THE ART 

All of the PED programs reviewed by the panel push the state of the art in their areas. 

They appear to be well connected to the PED’s stated objectives and to industrial and 

scientific community needs. The leaders of the different thrusts of the PED are all 

internationally recognized within their fields and are providing excellent technical leadership 

for the PED programs. There are very clear ties in their programs to the NIST, MEL, and 

PED vision and mission statements. They are appropriately aware of the work done by other 

NMIs, and they are maintaining strong collaboration with them in the appropriate areas. 

The PED has made significant strides in the past few years to satisfy the traceability 

and low uncertainty requirements of the semiconductor industry. Some examples are as 

follows: 

• Modeling capability and methods are being routinely developed together with 

experimental measurement technology and are used jointly to address nextgeneration problems; 

• New technologies such as helium-ion microscopy, worked on in collaboration with 

Zeiss, are being advanced in collaboration with the original equipment 

manufacturer (OEM) and are being explored for their impact on measurement 

capability and correlation to the SI; 

• The ability to perform three-dimensional coordinate measurement using microfiber 

probes is being developed to measure complex, submillimeter geometries. This 

work is performed in a coordinate measurement machine with demonstrated 

correlation to the SI to within ±20 nm; 

• Expertise is growing in standards associated with broad new fields of application, 

such as bionanoparticles. 

18
A minority of PED programs involve continuations of technically mature 

methodologies for which similar capabilities may exist in non-NIST laboratories and 

industry. Overall, the technical efficacy of the effort is very high. The PED should perform 

more of its own benchmarking with industrial (user) laboratories in addition to other NMIs. 

ADEQUACY OF INFRASTRUCTURE 

The move to the new AML has provided state-of-the-art facilities for equipment that 

is in many cases the most advanced available. This has translated into instrument 

performance beyond OEM expectations. The new AML building is clearly a significant 

positive factor differentiating NIST from the other NMIs. Within the next 1 to 2 years, a 

strategy will be needed for tool placement and space prioritization in the AML. 

Equipment availability in the division has improved significantly over the past 3 

years. The PED has demonstrated resourcefulness in securing state-of-the-art equipment from 

its collaborators with only a modest capital equipment budget. However, this approach has its 

limitations. Due to the lack of an adequate capital budget, there is a significant risk that 

infrastructure and equipment needs will go unfilled in the future. Current infrastructure 

strengths are as follows: 

• Line scale interferometry capability, 

• Calibrated atomic force microscopy, 

• New full-wafer scanning electron microscope capability, 

• A second Moore coordinate measurement machine, 

• Roundness measurement capability to under ±5 nm, 

• New Taylor-Hobson form and finish measurement instrumentation, and 

• Laser-interferometer-based calibration methods for spherical-coordinate large-scale 

three-dimensional measurement devices (laser trackers) traceable to the SI. 

The PED has a small number of highly qualified staff who lead the work of individual 

programs. Yet, there is generally only one such qualified staff member per program, in many 

cases a quite senior contributor, with little provision having been made for future program 

longevity. Budget issues appear to be driving the attrition of the staff, with only ad hoc 

opportunities for replacement. The PED has maintained its quality and capacity by using 

contractor personnel, guest researchers, and retiree volunteers. This is an effective tactic for 

the short term, but a strategic identification of core competencies and succession planning are 

required in order to maintain the economies of experience from which the division currently 

benefits. 

ACHIEVEMENT OF OBJECTIVES AND IMPACT 

There is excellent work in the dimensional metrology area, where measurement 

uncertainty is being improved to new levels and closer traceability to the SI is being 

established. Examples include the following: 

• Coordinate measurement with the Moore M48 coordinate measuring machine, 

• Laser interferometry in spherical-coordinate instrument calibration, and 

• Artifact and calibration methodologies for ballistics measurement. 

19
There are many notable metrology capabilities supporting current and future needs in 

micro- and nanotechnology, including the following: 

• The first commercial helium-ion microscope, 

• Improved uncertainty in the critical-dimension atomic force microscope, 

• Atom-based dimensional metrology, 

• Optical tweezers, 

• Improved optical overlay, and 

• Continued improvement in critical-dimension metrology. 

The PED has achieved its objectives by developing important new capabilities in 

measurement services needed in industry. Examples include the following: 

• Measurement of fiber-optics standard lengths, 

• Noncooperative target measurement in the 60 meter laboratory, 

• The ability to measure submillimeter features on the M48 with very low 

uncertainty and high traceability to the SI, and 

• New national calibration artifacts and calibration methodologies for sphericalcoordinate metrology devices (e.g., laser trackers) and for ballistics measurements. 

Maturation of this metrology research and development will lead to new metrology 

methods. The PED showed a clear path toward achieving this goal. 

The division provides a valuable service to industry with its measurement and 

calibration services. Examples include photomask measurements, forensics measurements, 

and survey tape calibration. 

The PED was very active in delivering 11 physical standards (standard reference 

materials and reference materials) and participating in the formation of 15 procedural 

standards (ANSI/ASME, ISO, ASTM [ASTM International, originally known as the 

American Society for Testing and Materials], SEMI [Semiconductor Equipment and 

Materials International]). 

The division exhibits an extensive list of publications, presentations, educational 

workshops, and advisory board memberships. More than 10 major awards were achieved in 

the past 3 years. The division has many strong ties to industry, including SEMATECH 

(Semiconductor Manufacturing Technology), key equipment suppliers, and other NMIs. The 

PED should take a more active role in bridging the metrology gap between NIST and 

industry capabilities and customers. 

The PED provides services to NASA; the Bureau of Alcohol, Tobacco, and Firearms; 

the Federal Bureau of Investigation; the Department of Energy; Bethesda Naval Medical 

Center; and other government entities. 

CONCLUSIONS 

The conclusions of the panel based on its assessment of the Precision Engineering 

Division are as follows: 

• The PED research staff is knowledgeable, dedicated, and highly enthusiastic about 

its work. 

• The PED has made significant improvements over the past 3 years in meeting the 

needs of the semiconductor industry. 

20
• The move to the Advanced Measurements Laboratory resulted in dramatic 

improvement in metrology capabilities, in several cases elevating the division’s 

capability to its being among the best in its field. 

• The PED has maintained its state-of-the-art equipment capabilities through 

innovative partnerships with equipment suppliers and other outside entities. 

• The PED has been engaged in forward-looking efforts in helium-ion microscopy, 

atom-based metrology, nanoparticle work in biosystems, and whole 

wafer/photomask capability. 

• The PED has had major impact both nationally and internationally by delivering 

new standard reference materials, calibration services, and documentation. 

• Opportunities exist for the PED to promote its specific mission regarding 

traceability and standards development. For example, in nanomanufacturing, the 

PED’s research differs substantially from research that others are doing in this area, 

both within and outside NIST, in that the PED’s focus is on traceability to the SI. 

• There is a need to define the core competencies of the PED and to define a 

succession plan for every key position. There are currently approximately 32 

permanent staff members, 2 NRC postdoctoral researchers, and 34 guest 

researchers. Of the 34 guest researchers, 10 are retirees. 

• The PED would be well served by strategic planning in the following areas: capital 

equipment, human resources, technical focus area roadmapping, and investment 

and disinvestment in calibration and measurement services. 

• Opportunities exist for the dimensional metrology groups to establish 

methodologies for receiving feedback from U.S. industry—for example, through 

workshops held both at NIST and at professional society meetings and through 

surveys by trade organizations. 

• Opportunities exist for the dimensional metrology groups to be more proactive in 

improving their visibility within the national technical community. 

• Benchmarking is performed currently for other NMIs, but this activity should be 

expanded to include industry in order to capture significant cutting-edge 

capabilities. 

• Opportunities exist for exploring alternative cost structures for the calibration and 

measurement services—for example, machine charges, consumables charges, 

service center charges, and activity-based cost accounting. 

21
Programs Funded Under the America COMPETES Act 

The Supply Chain Integration and Bio-Imaging projects in the Manufacturing 

Systems Integration Division have gained incremental funding under the America 

COMPETES Act of 2007. The funding has been used to bolster the more strategic segments 

of the semantic mediation work in supply chain. This will speed its deployment for the 

benefits of the nation. Specific accomplishments include the following: 

• Development and demonstration (in the IV&I/ATHENA program8) of a tool to 

automatically translate between Resource Description Framework and Extensible 

Markup Language schemas, both of which are heavily used standards in the 

operation of global supply chains; 

• Development and demonstration (in the IV&I/ATHENA program) of a universal 

framework for testing conformance to the Open Applications Group supply chain 

standards; and 

• Development and demonstration (in the Material Off-Shore Sourcing project) of 

expanded information mapping tests that advance the ability to manage data across 

supply chains that use ocean freight. 

 The funding has been used to further the collaboration with the NIST’s Information 

Technology Laboratory (ITL). Specific accomplishments include the following: 

• Developments of standard representations schemes and archival techniques that 

support very rapid context-based search, semantic annotation, and image 

recognition, including endoscopic video data; 

• A Workshop on Ontologies (as a collaboration between the MSID, the ITL, and 

Stanford University) to identify the barriers to effective utilization of ontologies. 

This will be very useful in the context of directing future work on standards and the 

testing of standards. 

The Precision Engineering Division received $87,000 to support bionanoparticle 

research, in which researchers are imaging, locating, and tracking nanoparticles in cancer 

cells for drug delivery applications. It is too early in the funding cycle to assess the 

progress of this work, but the effort demonstrates initiative for applying sensing, 

measurements, and fabrication technology to this very important biomedical need. 

 8

 IV&I/ATHENA is a program funded by the European Commission Information Societies Technology 

(EC/IST). See http://www.athena.ic.ac.uk/. Accessed March 27, 2008. 

22
Overall Conclusions 

Overall, the four divisions in the Manufacturing Engineering Laboratory reviewed by 

the panel are performing to the best of their capabilities, and much of the work being 

performed is among the best in the field. Their programs in measurement science and 

standards support NIST’s mission to contribute to the economic well-being of the nation. 

However, reduced funding and other factors are limiting (and are likely to increasingly limit) 

the degree to which MEL programs can achieve their objectives and are threatening the 

future impact of these programs. These factors include a significant percentage of division 

budgets relying on opportunistic funding, an overhead structure that has put increasing 

reliance on guest researchers and makes the postdoctoral rates noncompetitive, a lack of 

successor personnel in key technical positions, and a dwindling supply of workers in certain 

areas such as metrology. High-impact progress in the future will likely require disinvestment 

in some areas and strategic refocusing. NIST and the MEL have the technical capability to do 

this, driven by the rapidly changing scope and nature (and by the very definition) of manufacturing in the nation. 

23