Article Ref: J-020 

Weights and Measures Connection, Vol. 3, Issue 5 Page 1

August 28, 2012

Training Excellence: What Does Success Look Like?

Byline: Georgia L. Harris and Carol T. Hockert

The National Institute of Standards and Technology (NIST) Office of Weights and Measures (OWM) has 

been working the last few years toward compliance with several educational standards in support of our 

training program for the legal metrology and laboratory metrology community. We believe that 

complying with some rigorous models will help us provide excellent training opportunities for our 

stakeholders. These standards include the American National Standards Institute/International 

Association for Continuing Education and Training (ANSI/IACET) standard for Continuing Education and 

Training, the Baldrige Quality Award-Educational Criteria, and portions of ISO/IEC 17024 on personnel 

certification. In fact, the Baldrige Quality Award criteria are presented in its publications as a framework 

for “performance excellence” and references “benchmarking” in several instances. So, even without 

becoming an IACET Authorized Provider or receiving a Baldrige quality award, we can benchmark our 

operations against these documents to assess training excellence and to identify opportunities for 

improvement.

As we got started on the strategic aspects of applying these standards to our OWM training efforts, we 

began by asking the question, what does success look like? We asked our staff, key customer groups, 

and training participants, and we got a long list back. We matched up the list of success factors with the 

criteria in each of the documentary standards, and it fit together remarkably well. We didn’t get as 

complete a list from staff and stakeholders as were stated in the standards, so we identified a few gaps.

The educational standards included a few additional items that, upon review, we agreed were important 

for success, even though they were not explicitly stated in the feedback we gathered.

The rest of this article covers six ideas of what training excellence looks like in terms of the standards 

and our efforts at NIST. These concepts are important, whether you are a student, managing staff 

development, or are a trainer yourself. Some key sections of the documents are highlighted as notes in 

case you’d like to explore these documents further.

A key question for you, as you consider the highlighted areas, is how do you measure success of 

training? ISO/IEC 17025 for calibration and testing laboratories requires labs to plan for training needs 

and evaluate training effectiveness. What are your measures of success in your needs assessment and 

effectiveness evaluation?

Success means we have an infrastructure to manage training efforts.

We need a way to identify the organizational structure and which staff is involved in what aspects of our 

training efforts. In the Office of Weights and Measures, we have to coordinate with about eight other 

NIST groups to schedule, offer, and conduct one of our courses. Those of us involved in that 

coordination need that process documented for consistency and effectiveness. We have online 

calendars for our training and application processes for students to register for courses. We also have 

online access to student transcripts to enable participants to track their prior training from our office.

Note: ANSI/IACET Section 1, Organization, 2, Responsibility and Control, 3, Support Systems, and 9, 

Maintaining Learner Records. Baldrige Section 1, Leadership, Category 6, Process Management.
Article Ref: J-020 

Weights and Measures Connection, Vol. 3, Issue 5 Page 2

Success means instructors are educational professionals and subject matter experts.

In the NIST OWM, we have been providing regular professional development opportunities for our staff 

to become better trainers and to gain additional experience in our subject matter. We have sponsored 

several Train the Trainer workshops at NIST in the past three years and have opened up the training to 

others here at NIST. We have also been sharing best practices on effective activities and assessment 

methods for our seminars – including the classroom, the laboratory, and via webinar.

In partnership with the NCWM, additional Train the Trainer courses are being planned for regional 

weights and measures trainers. Individual jurisdictions and other organizations can also assure that their 

trainers are educational professionals by identifying local developmental opportunities

for those employees that conduct routine training.

Note: ANSI/IACET Section 6, Planning and Instructional Personnel. Baldrige Section 5, Workforce.

Success means we evaluate training and training programs.

Course evaluations have long been a part of the OWM training program. However, they have 

historically been little more than satisfaction-level evaluations to ask, “Did you like the training?”

Developing a more rigorous evaluation process for each course and the entire program has been our

effort in the last three years. We have been considering how to better evaluate training efforts through 

the use of the Kirkpatrick/Phillips evaluation models. Note that all of the seminar evaluations at NIST 

consider more than participant satisfaction levels. A key aspect of the evaluation is whether or not the 

data gathered is actually used to measure effectiveness of training and to identify areas for future 

improvement. So, at NIST, we also often conduct an After Action Review of the courses and consider

input from both the students and instructors in identifying opportunities for change. 

We have been conducting a follow up evaluation of some students to assess application and impact, and 

intend to expand this effort in the future.

Note: ANSI/IACET Section 10, Program Evaluation. Baldrige Category 4,Measurement, Analysis, and 

Knowledge Management.

Success means we design for and measure application and impact.

When attending a NIST training course, students are sometimes told, “It doesn’t matter how much you 

know if you don’t apply it to your work!” But, getting participants to apply new learning for 

effectiveness and impact takes more than just saying a few words about doing it. Designing courses so 

that students get beyond the “knowledge” level on Bloom’s taxonomy to the “application” (or higher) 

levels takes effort and time. We have completely redesigned our OWM laboratory metrology courses 

with “application” levels in mind. We have selected course learning objectives that drive application.

We have selected activities that allow practice at the application level. And, we assess student 

participation on whether they have successfully completed course objectives at the application level. As 

we begin to assess application, we are moving away from just automatically granting everyone a 

certificate of attendance (versus a certificate of successful completion). 

For the weights and measures community as a whole, it is up to each individual instructor to apply these 

concepts. Further, each instructor needs to be willing to assess students at the stated levels. Instructors 

may not want to police this aspect of learning, but it’s a responsibility all trainers should take on.

Note: ANSI/IACET Sections 4, Learning Event Planning, and 7, Content and Instructional Methods. 

Baldrige Section 7, Results.
Article Ref: J-020 

Weights and Measures Connection, Vol. 3, Issue 5 Page 3

Success means we gather data on training needs.

We need to have an internal planning process to identify future courses. This area is one where we 

have great opportunities for improvement. In the NIST OWM, we have many channels for input into 

future training offerings. These include individual requests, identified needs based on commonly asked 

questions, new technologies that will require a new training effort, and new marketplace challenges. 

Collecting all of this data in a standardized way to prioritize our efforts relies on the technical expertise

of our staff to filter all of the data and make decisions. However, this is far from a standardized, 

documented process.

While we may be doing a good job identifying training needs, we could do a better job by formalizing 

how we collect and analyze data on training needs and use it to prioritize our training efforts.

Note: ANSI/IACET Section 4, Learning Event Planning. Baldrige Section 6, Customer Focus.

Success means we have a partnership among the trainer, student, and student’s 

managers.

No matter how good a trainer is in a class and how excited a student is about gathering and applying 

new information into their work, no improvements will be made without the support of the manager 

back on the job. This is another area where we all have an opportunity to improve. Engaging the 

manager is essential for all aspects of our training cycle: from needs assessment to establishing learning 

objectives and from conducting the training to applying it on the job. We sometimes have requests 

from managers after the event on how well their staff performed in the class. But, rarely do we get 

questions from the managers ahead of time about what they could expect from the participant after the 

training. This is an effort we are pursuing within OWM.

Before you schedule training for your organization, think about this partnership as you begin planning 

the event. Provide your expectations to the instructor and students ahead of time, including desired 

learning objectives. And, if you are a trainer, interact with the manager prior to conducting training to 

make sure the training will meet management’s expectations. But don’t forget the student’s role in all 

of this. Both managers and trainers should communicate the same expectations to the students for 

maximum effectiveness of training.

Note: ANSI/IACET Sections 4, Learning Event Planning and 8, Assessment of Learning Outcomes. Baldrige 

Category 7, Results.

Your Turn!

Training excellence means many things to many people, just as successful training means different 

things to everyone. However, by using the benchmark standards we can focus on some uniform factors, 

assess for compliance, select areas to highlight for our effectiveness, and select action items for 

continual improvement. You’ve read what we’re doing. Now, it’s your turn. What does training 

excellence mean to you? How will you assess training effectiveness for your staff? How will you assess 

the next training event in which you participate? Let us know at owm@nist.gov!