January 9, 2007 

MEMORANDUM TO: Luis A. Reyes 

Executive Director for Operations 

FROM: Stephen D. Dingbaum /RA/ 

Assistant Inspector General for Audits 

SUBJECT: AUDIT OF NRC’S TECHNICAL TRAINING CENTER 

 (OIG-07-A-05) 

 

This report presents the results of the subject audit. The formal comments 

provided by your office on November 22, 2006, are presented in their entirety as 

Appendices C and D to this report. Appendix D also contains OIG’s response. 

Please provide information on actions taken or planned on each of the 

recommendations within 30 days of the date of this memorandum. Actions taken 

or planned are subject to OIG follow-up, as stated in Management Directive 6.1. 

If you have any questions, please call me at 415-5915 or Steven Zane at 

415-5912. 

Attachment: As stated 

cc: M. Johnson, OEDO 

 M. Malloy, OEDO 

 P. Tressler, OEDO 
ELECTRONIC DISTRIBUTION

 

John T. Larkins, Executive Director, Advisory Committee on Reactor 

Safeguards/Advisory Committee on Nuclear Waste 

E. Roy Hawkens, Chief Administrative Judge, Atomic Safety and 

 Licensing Board Panel 

Karen D. Cyr, General Counsel 

John F. Cordes, Jr., Director, Office of Commission Appellate Adjudication 

Jesse L. Funches, Chief Financial Officer 

Janice Dunn Lee, Director, Office of International Programs 

Rebecca L. Schmidt, Director, Office of Congressional Affairs 

Eliot B. Brenner, Director, Office of Public Affairs 

Annette Vietti-Cook, Secretary of the Commission 

William F. Kane, Deputy Executive Director for Reactor 

 and Preparedness Programs, OEDO 

Martin J. Virgilio, Deputy Executive Director for Materials, Research, 

 State and Compliance Programs, OEDO 

Jacqueline E. Silber, Deputy Executive Director for Information Services 

 and Administration and Chief Information Officer, OEDO 

Michael R. Johnson, Assistant for Operations, OEDO 

Timothy F. Hagan, Director, Office of Administration 

Cynthia A. Carpenter, Director, Office of Enforcement 

Charles L. Miller, Director, Office of Federal and State Materials 

 and Environmental Management Programs 

Guy P. Caputo, Director, Office of Investigations 

Edward T. Baker, Director, Office of Information Services 

James F. McDermott, Director, Office of Human Resources 

R. William Borchardt, Director, Office of New Reactors 

Jack R. Strosnider, Director, Office of Nuclear Material Safety and Safeguards 

James E. Dyer, Director, Office of Nuclear Reactor Regulation 

Brian W. Sheron, Director, Office of Nuclear Regulatory Research 

Corenthis B. Kelley, Director, Office of Small Business and Civil Rights 

Roy P. Zimmerman, Director, Office of Nuclear Security and Incident Response 

Samuel J. Collins, Regional Administrator, Region I 

William D. Travers, Regional Administrator, Region II 

James L. Caldwell, Regional Administrator, Region III 

Bruce S. Mallett, Regional Administrator, Region IV 
AUDIT REPORT 

Audit of NRC’s Technical Training Center 

OIG-07-A-05 January 9, 2007 

All publicly available OIG reports (including this report) are accessible through 

NRC’s Web site at: 

http:/www.nrc.gov/reading-rm/doc-collections/insp-gen/ 
Audit of NRC’s Technical Training Center 

i

EXECUTIVE SUMMARY 

BACKGROUND

Following its establishment as an independent agency in 1974, 

NRC provided technical training at its Headquarters in Bethesda, 

Maryland. After the 1979 accident at the Three Mile Island nuclear 

power plant, various investigative bodies recommended that NRC 

improve technical training for its staff. Consequently, in 1980, NRC 

moved its technical training to Chattanooga, Tennessee, a location 

near the Tennessee Valley Authority (TVA). 

Organizationally, the Technical Training Center (TTC) reports to the 

Associate Director for Training and Development in the Office of 

Human Resources (HR). TTC is divided into three groups: The 

Reactor Technology Training group, the Specialized Technical 

Training group, and the Technical Support Team. 

PURPOSE

The objective of this audit was to identify opportunities to improve 

the economy, efficiency, and effectiveness of the Technical 

Training Center’s operations. 

RESULTS IN BRIEF

Collectively, TTC instructors are knowledgeable, experienced and 

enthusiastic. While Office Directors and former TTC students were 

generally pleased with the training provided by TTC, OIG identified 

opportunities for improvement in the following areas: 

• Inventory practices, 

• Policies and procedures, 

• Maintenance of training materials, 

• Scheduling course revisions, 

• Instructor training and rotation, and 

• Course evaluations. 

RECOMMENDATIONS

This report makes eleven recommendations to the Executive 

Director for Operations (EDO) to improve the efficiency and 

effectiveness of TTC’s training program and better prepare NRC for 

the significant challenge that lies ahead. 
Audit of NRC’s Technical Training Center 

ii

OIG ANALYSIS OF AGENCY COMMENTS

At a September 28, 2006, exit conference with agency senior 

executives, NRC officials generally agreed with most of the report’s 

findings and recommendations. Subsequent to that meeting, NRC 

provided informal comments on the draft report. The EDO provided 

a formal response to this report on November 22, 2006. 

Appendix C contains the EDO’s transmittal letter. 

The agency’s formal comments along with OIG’s analysis and 

response to those comments are included as Appendix D. This 

final report incorporates revisions made, where appropriate, in 

response to the agency’s comments. 
Audit of NRC’s Technical Training Center 

iii

ABBREVIATIONS AND ACRONYMS 

NRC U.S. Nuclear Regulatory Commission 

OIG Office of the Inspector General 

 FTE Full Time Equivalent 

GAO Government Accountability Office1 

HR Office of Human Resources 

MD Management Directive 

TVA Tennessee Valley Authority 

TTC Technical Training Center 

 

1

 Effective July 7, 2004, GAO legally changed their name from the General Accounting Office to 

the Government Accountability Office, while keeping the same acronym. 
Audit of NRC’s Technical Training Center 

iv

[Page intentionally left blank.] 
Audit of NRC’s Technical Training Center 

v

TABLE OF CONTENTS 

EXECUTIVE SUMMARY...........................................................................i 

ABBREVIATIONS AND ACRONYMS ..................................................... iii 

I. BACKGROUND ...................................................................................1 

II. PURPOSE...........................................................................................4 

III. FINDING ............................................................................................5 

A. Inventory Practices .............................................................5 

B. Policies and Procedures .....................................................7 

C. Maintenance of Training Materials......................................9 

D. Scheduling Course Revisions ...........................................11 

E. Instructor Training and Rotation........................................13 

F. Course Evaluations...........................................................16 

IV. CONSOLIDATED LIST OF RECOMMENDATIONS........................18 

V. SURVEY OF FORMER STUDENTS ................................................19 

VI. BEST PRACTICES..........................................................................21 

APPENDICES 

 A. SCOPE AND METHODOLOGY ..........................................22 

 

 B. ANTICIPATED TRAINING...................................................23 

 

 C. MEMORANDUM TRANSMITTING AGENCY 

 RESPONSE ........................................................................25 

 

 D. FORMAL AGENCY COMMENTS AND 

 DETAILED OIG ANALYSIS OF AGENCY 

 COMMENTS .......................................................................27 
Audit of NRC’s Technical Training Center 

vi

[Page intentionally left blank.] 
Audit of NRC’s Technical Training Center 

1

TTC - Chattanooga, TN 

I. BACKGROUND 

Formation and Location 

Following its establishment as an independent agency in 1974, NRC 

provided technical training at its Headquarters in Bethesda, Maryland. After 

the 1979 accident at the Three Mile Island nuclear power plant, various 

investigative bodies recommended that NRC improve technical training for 

its staff. Consequently, in 1980, NRC 

moved its technical training to 

Chattanooga, Tennessee, a location near 

the Tennessee Valley Authority (TVA). 

Initially, NRC contracted with TVA for the 

use of its control room simulators. 

Subsequently, NRC purchased four 

simulators of its own for use in providing 

training in each of the four nuclear reactor 

vendor designs. 

Organizational Structure 

Organizationally, the Technical Training Center (TTC) reports to the 

Associate Director for Training and Development in the Office of Human 

Resources (HR). TTC is divided into three groups: The Reactor 

Technology Training group, the Specialized Technical Training group, and 

the Technical Support Team. 

Reactor Technology Training Group

The Reactor Technology Training group provides a spectrum of classroom 

and full scope simulator courses to meet the cumulative reactor technology 

and regulatory skills training needs of the NRC staff. The Reactor 

Technology Training group designs, develops, maintains, and implements a 

reactor technology curriculum in each of the General Electric, 

Westinghouse, Combustion Engineering, and Babcock and Wilcox reactor 

vendor designs. The Reactor Technology Training group also provides 

training to develop and maintain agency skills related to: 

• Performance-based regulatory safety assessment of control room 

configuration, 

• Integrated plant operation, 

• Application of emergency operating procedures, and 

• Severe accident guidelines. 
Audit of NRC’s Technical Training Center 

2

TTC instructors2

 teach these courses. 

Specialized Technical Training Group

The Specialized Technical Training group provides specialized non-reactor 

technical training. This group also supports Agreement State staff training. 

The training includes: 

• Probabilistic risk assessment, 

• Engineering support, 

• Radiation protection, 

• Fuel cycle technology, 

• Security and safeguards, and 

• Regulatory skills. 

TTC instructors teach some courses in these areas, but the Specialized 

Technical Training group establishes and manages interagency agreements 

and commercial contracts to satisfy the specialized technical training needs 

for most courses. 

Technical Support Team

The Technical Support Team manages the reactor simulation program and 

provides administrative support for the training programs. This team also 

manages administrative aspects of the TTC facilities including: 

• Building security, 

• Facility management, 

• Administrative funds control, 

• Property management and control, and 

• Administration of the local area network and all associated 

information technology infrastructure. 

In addition, the Technical Support Team is charged with developing and 

implementing the agency learning management system3

 and all related web 

pages. 

 

2

 TTC instructors are NRC employees who teach full-time at TTC. 

3

 The learning management system, currently under development, is a web-based application that will 

provide database interfacing, records storage, student registration, student enrollment, course and session 

cataloging, tracking of employee training records and qualification programs, and management of employee 

competencies. 
Audit of NRC’s Technical Training Center 

3

FY 2005 TTC Statistics 

In FY 2005, TTC’s budget was approximately $7.6 million. This included 

$3.9 million for interagency agreements, commercial contracts, and various 

administrative expenses and $3.7 million in salaries and benefits for 29 fulltime equivalents. During that year, TTC offered 70 courses and held 160 

course sessions with 2,220 students attending. The following chart provides 

details of the courses and student attendance. Approximately 15 percent of 

the students were non-NRC from Agreement States and other government 

and international agencies. 

Course Statistics 

Course Category 

Number of 

Sessions 

Number of 

Students 

Reactor Technology 

Reactor Concepts and Technology 18 392 

Advanced Technology 6 100 

Technology Review 13 94 

Cross-Training 2 16 

Simulator 16 86 

Simulator Refresher 13 67 

 

Total Reactor Technology Training 68 755 

Specialized Technical Training 

Engineering Support Technology 4 50 

Fuel Cycle Technology 1 30 

Regulatory Skills 39 583 

Radiation Protection Technology 28 499 

Risk Assessment Technology 20 303 

 

Total Specialized Training 92 1465 

Total TTC Training 160 2220 

 Note: OIG created the categories in this table 

Training Challenge 

In addition to its ongoing license renewal activities, recent developments 

have resulted in a marked increase in the workload associated with NRC’s 

traditional regulatory responsibilities. As NRC enters this new era of reactor 

regulation, the agency faces many challenges. The Energy Policy Act of 

2005 (Public Law 109-58) provides loan guarantees, production tax credits, 

and federal risk insurance for builders of new nuclear power plants. NRC 

estimates that it will receive 18 or more applications for new nuclear power 

plants in the coming years, with initial construction activities coming soon 

thereafter. Some applications may involve new reactor design technologies. 
Audit of NRC’s Technical Training Center 

4

Coinciding with the increase in work related to new-plant licensing 

applications is the expected retirement of many senior staff. As a result, 

NRC increased its recruitment and hiring efforts for FY 2006 and the next 

several years. Appendix B shows the breakdown of the expected number of 

new hires and anticipated TTC students from FY 2006 through 2009. The 

anticipated training to be provided by TTC is reflected in the graph below. 

NRC’s ability to effectively review and license the new generation of 

commercial nuclear reactors will depend significantly on how well 

employees new to the process are trained and developed into effective 

reviewers and regulators at the staff and senior management level. 

TTC Student Projection

2000

2500

3000

3500

4000

2006 2007 2008 2009

Fiscal Year

Number of Students

Number of

Students

NOTE: The data shown in the graph above and in Appendix B were provided by a TTC 

official. 

TTC’s challenge is to effectively and efficiently train the new and existing 

employees. NRC must have highly qualified employees to perform the new 

reactor licensing work while ensuring maintenance of the safe performance 

of the existing nuclear power plants. Current plans call for training on new 

reactor technology to be contracted-out until NRC develops in-house 

expertise. 

To meet this challenge successfully, HR/TTC management should continue 

to develop and implement proactive strategies to ensure that TTC operates 

at optimal performance. 

II. PURPOSE 

The objective of this audit was to identify opportunities to improve the 

economy, efficiency, and effectiveness of the Technical Training Center’s 

operations. 
Audit of NRC’s Technical Training Center 

5

III. FINDINGS 

Collectively, TTC instructors are knowledgeable, experienced and 

enthusiastic. While Office Directors and former TTC students were 

generally pleased with the training provided by TTC, OIG identified 

opportunities for improvement in the following areas: 

• Inventory practices, 

• Policies and procedures, 

• Maintenance of training materials, 

• Scheduling course revisions, 

• Instructor training and rotation, and 

• Course evaluations. 

Addressing these areas will improve the efficiency and effectiveness of 

TTC’s training program and better prepare NRC for the significant challenge 

that lies ahead. 

A. Inventory Practices 

Management Directive (MD) 13.1, “Property Management,” requires that 

physical inventories of all NRC equipment be conducted every two years. 

Specifically regional offices, including TTC, are required to conduct selfinventories of NRC equipment every two years and report the results to the 

Office of Administration. 

Control Activities, the third standard discussed in the Government 

Accountability Office’s Standards for Internal Control in the Federal 

Government4 (the GAO Standards) provides that key duties and 

responsibilities should be divided or segregated among different people. 

The methodology used by TTC in conducting an inventory did not include 

independent physical verification of the property by someone other than the 

property holder. A TTC representative explained that MD 13.1 does not 

require that regional inventories be conducted using the segregation of 

duties concept. Although compliant with MD 13.1, TTC’s training mission 

could be negatively impacted if property cannot be located when required. 

Noncompliance with GAO Standards for Internal Control

TTC staff did not perform an independent physical inventory of its property, 

but rather relied on individual property holders to provide electronic 

confirmation of the property's existence and location. To conduct the 

inventory, the TTC property custodian sent property holders an e-mail listing 

the inventory items assigned to them. The property holders would selfidentify the item’s existence and location via e-mail to the TTC property 

 

4 Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1, dated November 1999. 
Audit of NRC’s Technical Training Center 

6

custodian who subsequently prepared the inventory report. Finally, the 

Director of TTC Operations would send a memo reporting those results to 

the Office of Administration. 

TTC’s method of conducting the inventory pursuant to MD 13.1 did not meet 

the intent of the GAO Standards, which provide that key duties and 

responsibilities should be divided or segregated among different people. As 

applied to the TTC inventory practice, this means that the property holder 

should not be the same person verifying the existence and location of the 

property. Further, GAO’s executive guide, Best Practices in Achieving 

Consistent, Accurate Physical Counts of Inventory and Related Property,

5

provides that adequate segregation of duties for the inventory process 

includes using personnel who do not have overlapping responsibilities in 

custody or access to the inventory items. 

MD 13.1 provides that regional offices should conduct self-inventories, but 

does not require independent verification of property by someone other than 

the property holder. Effective implementation of segregation of duties 

reduces the risk of error or fraud so that no single person can adversely 

impact the accuracy and integrity of the physical inventory count. 

Recommendations

OIG recommends that the Executive Director for Operations: 

1. Revise Management Directive 13.1 to require that property inventories 

should include independent verification of the property by someone 

other than the property holder. 

2. Issue interim guidance to accomplish the intent of Recommendation 1, 

pending revision of Management Directive 13.1. 

 

5

 GAO-02-447G, issued March 2002. 
Audit of NRC’s Technical Training Center 

7

B. Policies and Procedures 

All of TTC’s policies and procedures are outdated and incomplete. They do 

not comply with Office of Management and Budget Circular A-123, 

Management’s Responsibility for Internal Control6

 (OMB Circular A-123) or 

the GAO Standards. This condition exists because HR management has 

not made finalizing policies and procedures a priority. Without established 

policies and procedures to clearly communicate management’s 

expectations regarding authorities and responsibilities, staff may use 

inconsistent and less than optimal methods in carrying out their duties. 

TTC Policies and Procedures Need to be Revised and Updated

The policies and procedures in use at TTC address a variety of areas 

including, but not limited to: 

• The general duties and responsibilities associated with TTC 

positions; 

• How TTC meets the technical training needs of the NRC; 

• Course scheduling; 

• Course administration; 

• The TTC staff qualification program; 

• The training materials control program; and 

• Simulator training. 

These policies and procedures are outdated and incomplete. Seven of nine 

policies are in draft form; six have been in draft form for more than 10 years. 

Of the two policies that are in final form, one is signed, but undated, though 

an attachment is dated November 1995; the other policy is dated August 

1993, but is unsigned. 

According to both OMB Circular A-123 and the GAO Standards, policies, 

procedures, and mechanisms should be in place to help ensure that agency 

objectives are met. Further, information that is relevant, reliable, and timely 

should be communicated to appropriate personnel at all levels within an 

organization. 

HR management, however, has not made finalizing TTC’s policies and 

procedures a priority. When contacted by OIG, they indicated that they plan 

to issue consolidated training policies which would also include the 

Professional Development and Knowledge Management Branch at 

Headquarters. 

 

6

 The Office of Management and Budget issued the revised OMB Circular A-123 on December 21, 2004. 
Audit of NRC’s Technical Training Center 

8

Without current policies and procedures new TTC staff could experience 

difficulty understanding their responsibilities and authorities and take longer 

to do what is expected of them. Additionally, existing TTC staff may be 

unsure of management’s expectations for carrying out the day-to-day 

operations of TTC. 

Recommendation

OIG recommends that the Executive Director for Operations: 

3. Update and finalize the training policies and procedures. 
Audit of NRC’s Technical Training Center 

9

C. Maintenance of Training Materials

In OIG’s opinion, prudent business practices call for a systematic and wellcommunicated method of maintaining training materials to ensure currency, 

accuracy and completeness. At TTC, each course has a cognizant 

instructor7 assigned to maintain training materials. OIG found that TTC 

cognizant instructors employ various methods of maintaining training 

materials and do not systematically communicate their methods to other 

instructors or management. Further, the procedures for maintaining training 

materials are ad hoc and inconsistent. TTC management does not review 

or approve the various maintenance methods. Accordingly, turnover of 

materials from one instructor to another can be difficult to execute. There is 

a risk that out-of-date or erroneous training materials will be used in 

courses. 

Inconsistent Methods of Maintaining Training Materials 

Cognizant instructors are assigned to maintain the course materials for each 

TTC course. Cognizant instructors are responsible for (1) identifying 

needed changes to course materials (2) revising course materials, (3) 

tracking revisions to course materials, and (4) maintaining the controlled 

copy of course materials current. Cognizant instructors have chosen a 

variety of methods for collecting, recording, and tracking changes needed to 

their assigned course materials. The degree of rigor, formality and available 

documentation varies from cognizant instructor to cognizant instructor. 

There also is no systematic process for cognizant instructors to 

communicate to other instructors the methods used to maintain their course 

materials. 

OIG found that written policies do not specifically address methods for 

updating and making available training materials for all courses. In addition, 

TTC management has not reviewed or approved the various means used to 

maintain training materials. As a result, there is a risk that an instructor 

substituting for a cognizant instructor may use outdated training materials. 

The outdated training materials may not include recent operating 

experience. 

 

7

 A cognizant instructor is the instructor who has responsibility for maintaining and updating course material 

for a particular course. 
Audit of NRC’s Technical Training Center 

10

Further, if a cognizant instructor takes a new position or leaves the agency, 

that individual’s responsibilities would be given to another agency 

employee. Since the maintenance processes are not systematic, the 

turnover of course materials can be difficult to execute and can result in an 

inefficient use of agency resources during the turnover process. 

Recommendation

OIG recommends that the Executive Director for Operations: 

4. Develop and implement a written policy that provides specific 

expectations for: 

• Revising course materials, 

• Tracking revisions to course materials, 

• Transferring cognizant instructor responsibilities, and 

• Organizing and storing course materials. 
Audit of NRC’s Technical Training Center 

11

D. Scheduling Course Revisions 

OIG found that TTC management does not obtain information needed to 

establish respective priorities and resource requirements relating to pending 

revisions of training materials. There is no specific written policy for: 

• Prioritizing the aggregate of needed training material revisions, 

• Identifying needed resources, and 

• Scheduling revision work to meet course-scheduling needs. 

Consequently, resources may not be available to ensure that needed 

course revisions are made in a timely manner. Additionally, there is a risk 

that training materials may not be completely updated for courses. Finally, 

excessive costs such as instructor overtime or contractor costs may be 

incurred to meet course deadlines. 

Improvements Needed in Scheduling Course Revisions 

In addition to classroom related responsibilities, a cognizant instructor is 

responsible for revising course materials for the assigned course. TTC 

offers a number of courses, each of which may have several changes 

pending at a given time. 

Cognizant instructors manage the workload for revising courses in isolation 

from the broader needs of the TTC organization. OIG identified several 

deficiencies related to this area. Specifically, there are no processes for: 

• Identifying and reporting pending changes for all courses to TTC 

management; 

• Prioritizing, allocating resources, and scheduling changes for the 

aggregate of all pending course changes; and 

• Tracking past changes to determine whether certain changes 

have previously been addressed or found to be unnecessary. 

These deficiencies have occurred because HR and TTC management have 

not established written policies to specifically address prioritizing the 

aggregate of needed training material revisions, identifying needed 

resources and scheduling revision work to meet course-scheduling needs. 

Available resources are not being assigned to match the priorities of all 

needed course revisions. As a result, resources may not be available to 

ensure that needed course revisions are made in a timely manner. 
Audit of NRC’s Technical Training Center 

12

Recommendation

OIG recommends that the Executive Director for Operations: 

5. Develop and implement a written policy that provides specific 

expectations for the aggregate of pending changes for all course 

materials on an ongoing basis. The policy should address: 

• Prioritizing individual changes, 

• Identifying available resources, and 

• Scheduling the changes. 
Audit of NRC’s Technical Training Center 

13

E. Instructor Training and Rotation 

Opportunities exist to enhance training effectiveness at TTC. To optimize 

instructional methodology and keep courses technically fresh and up-todate, instructor training and instructor rotation need improvement. In OIG’s 

opinion, sound business practices and the GAO Standards call for such 

improvements. As a result of the identified improvements noted above, 

students may not be getting the most current or best quality training 

available. 

Instructor Training

Instructors need to continuously sharpen their skills and receive intellectual 

stimulation. The GAO Standards state that, “Only when the right personnel 

for the job are on board and are provided the right training, tools, structure, 

incentives, and responsibilities is operational success possible.” Instructor 

training includes technical training and instructional methodology training. 

 Technical Training 

Most TTC instructors stated that, to stay current in their technologies, they 

rely on the NRC Daily Reports and discussions with their students. In OIG’s 

opinion, sound business practice dictates that instructors should be in a 

continuous learning mode – if they are not teaching or preparing/updating 

teaching materials, they could be learning by such means as: 

• Participating as observers or members of an inspection team at a 

nuclear power plant, 

• Obtaining qualification as an inspector, and/or 

• Completing relevant technical training courses. 

While some instructors are in the process of obtaining current field 

experience, many are not. 

 Instructional Methodology Training

TTC instructors have not had instructional methodology training for at least 

10 years. One instructor has not had this training for over 25 years. Many 

new instructors employed by TTC for fewer than 5 years received instructor 

training provided by their previous employers. One new instructor stated 

that typical industry practice requires instructor training every two years. 

The GAO Standards address management’s commitment to competence 

stating in part, “All personnel need to possess and maintain a level of 

competence that allows them to accomplish their assigned duties...” 

(Underlining added for emphasis.) For optimum effectiveness, instructors 

need periodic instruction in new and innovative methods of presenting 

course materials. 
Audit of NRC’s Technical Training Center 

14

HR management has not established a policy implementing a formal 

mechanism to ensure that all TTC instructors keep technically current and 

stay up-to-date with innovative teaching techniques. Consequently, TTC 

students may not be getting the most current or best quality training 

available. 

Instructor Rotation

Instructor rotation encompasses rotation of instructors in and out of TTC as 

well as rotation of cognizant instructor responsibilities as discussed below. 

 Rotational Instructor Positions

The concept of creating rotational instructor positions in TTC would create 

an environment in which new ideas, fresh perspectives, and recent handson experience would be of great value and benefit to TTC, its students, and 

the NRC. Currently, TTC instructors do not rotate in and out of TTC and 

typically remain at TTC until retirement. When contacted by OIG, the 

Director of the Office of Human Resources seemed receptive to the concept 

of creating rotational instructor positions. He mentioned the possibility of 

creating such positions on a pilot basis. One federal agency contacted by 

OIG has a policy of rotating instructors on a five-year cycle. 

 Rotation of Cognizant Instructor Responsibilities

Cognizant instructors are responsible for maintaining and updating course 

curriculum. A TTC instructor assigned as a cognizant instructor for a 

particular course generally retains the responsibility for that course for a 

long time. In OIG’s opinion, sound business practices call for keeping 

course materials fresh by obtaining new ideas and new views. Rotation of 

cognizant instructor responsibilities would provide an additional way to 

accomplish this objective, along with providing them intellectual stimulation 

and a heightened level of interest and engagement in the courses that they 

teach. A TTC official stated that rotation of cognizant instructor 

responsibilities is not a routine practice, but it is an idea worth considering. 

Recommendations

OIG recommends that the Executive Director for Operations: 

6. Develop and implement a policy requiring TTC instructors to maintain 

technical currency. 

7. Periodically provide instructional methodology training for all TTC 

instructors. 
Audit of NRC’s Technical Training Center 

15

8. Develop and implement a pilot program for rotational instructor 

positions at TTC. 

9. Periodically rotate cognizant instructor responsibilities. 
Audit of NRC’s Technical Training Center 

16

F. Course Evaluations 

When used effectively, course evaluations are a valuable tool to assure 

management that the training program fulfills the needs of students, as well 

as to identify opportunities for improvement. While TTC instructors obtain 

immediate feedback from students at the end of each training session, 

course evaluations are not compiled in an official database. Such action 

would allow TTC management to analyze results and identify areas needing 

improvement. Furthermore, the evaluations used to evaluate courses 

taught by TTC instructors do not include specific questions related to 

instructor performance. In OIG’s opinion, both of these conditions are at 

variance with prudent business practices and other relevant criteria. As a 

result, negative trends could go unnoticed and uncorrected. Uncorrected 

problems in course content and/or instructor performance could adversely 

affect the quality of training received. 

Official Database 

While some managers and instructors at TTC maintain individual databases 

containing information related to course evaluations, TTC management has 

not established a central, official database to compile and analyze course 

evaluation results. The GAO Standards provide that “Internal control should 

generally be designed to assure that ongoing monitoring occurs in the 

course of normal operations.” Accordingly, in OIG’s opinion, prudent 

business practice requires that TTC management should compile course 

evaluation results in an official database. In addition, an appropriate 

methodology should be established to facilitate data analysis. Such action 

would provide TTC management and instructors the necessary tool to 

identify and correct problem areas. In light of the fact that TTC managers 

do not routinely observe instructors in the classroom, course evaluation 

analysis is critical to successful operations. 

Evaluating Instructor Performance

Course evaluations used for contracted courses include questions 

specifically related to instructor performance; however, evaluations for 

courses taught by TTC instructors do not. The Performance Elements and 

Standards for Reactor Technology Training instructors provide that the 

instructor “Seeks feedback for both course and self-improvement from 

students, peers, management, and clients and acts on recommendations as 

appropriate.” By not including questions related to instructor performance 

on all course evaluations, TTC management has not fully implemented the 

elements and standards referred to above. As previously stated, TTC 

managers do not routinely observe instructors in the classroom. Therefore, 

it is important that managers, as well as instructors, obtain feedback from 

students on TTC instructor performance. 
Audit of NRC’s Technical Training Center 

17

In summary, without an official database to compile and analyze course 

evaluations, negative trends could go unnoticed and uncorrected. 

Additionally, failure to obtain student feedback on TTC instructor 

performance could negatively affect training quality. 

Recommendations

OIG recommends that the Executive Director for Operations: 

10. Establish a more formal method to track and trend TTC course 

evaluations and periodically analyze trends for appropriate action. 

11. Include questions specific to instructor performance on all course 

evaluations. 
Audit of NRC’s Technical Training Center 

18

IV. CONSOLIDATED LIST OF RECOMMENDATIONS 

OIG recommends that the Executive Director for Operations: 

1. Revise Management Directive 13.1 to require that property inventories 

should include independent verification of the property by someone 

other than the property holder. 

2. Issue interim guidance to accomplish the intent of Recommendation 1, 

pending revision of Management Directive 13.1. 

3. Update and finalize the training policies and procedures. 

4. Develop and implement a written policy that provides specific 

expectations for: 

• Revising course materials, 

• Tracking revisions to course materials, 

• Transferring cognizant instructor responsibilities, and 

• Organizing and storing course materials. 

5. Develop and implement a written policy that provides specific 

expectations for the aggregate of pending changes for all course 

materials on an ongoing basis. The policy should address: 

• Prioritizing individual changes, 

• Identifying available resources, and 

• Scheduling the changes. 

6. Develop and implement a policy requiring TTC instructors to maintain 

technical currency. 

7. Periodically provide instructional methodology training for all TTC 

instructors. 

8. Develop and implement a pilot program for rotational instructor 

positions at TTC. 

9. Periodically rotate cognizant instructor responsibilities. 

10. Establish a more formal method to track and trend TTC course 

evaluations and periodically analyze trends for appropriate action. 

11. Include questions specific to instructor performance on all course 

evaluations. 
Audit of NRC’s Technical Training Center 

19

V. SURVEY OF FORMER STUDENTS 

OIG surveyed 183 former TTC students8 via email to determine the degree 

of satisfaction with their learning experience. Using a scale of 1 to 10, with 

1 being the lowest and 10 the highest, sixty-eight former students 

responded, yielding a 37 percent response rate. Based on the response 

rate, OIG did not make any projections. Despite the level of response, the 

feedback received was generally favorable, as shown in the table below: 

 Sample 

Mgmt 

Measure 

Sample 

Mgmt 

Measure 

No. Question 

Description 

Mean of

Rating 

Median of

Rating 

Mode of 

Ratings

Favorable 

Rating 

> or = 7 * 

Percentage 

ratings 

< 7 * 

1 Training Applicability 7.8 8 8 79% 21% 

2 Training Used in Current Job 6.5 7 8 62% 38% 

3 Appropriate Technical Level 7.7 8 8 77% 23% 

4 Course Materials Current 8.0 8.5 9 87% 13% 

5 Instructors Knowledgeable 

and Current 

8.8 9 9 94% 6% 

6 Instructor Clarity 8.5 9 9 94% 6% 

7 Overall Course Rating 7.8 9 9 81% 19% 

Mean: Arithmetic Average 

Median: Half Values Above and Half of the Values Below 

Mode: Most Common Result 

* In the above table, a rating of “7” was used as a favorable rating for 

demonstration purposes only 

Using 7 as the minimum favorable rating, OIG depicted the results of the 

survey further as follows: 

The graph on the following page is for illustrative purposes only. The 

purpose of the graph is to show that valuable information can be learned by 

analyzing and presenting data from class evaluations. 

 

8

 OIG identified the 183 students for survey by choosing all students who attended Reactor Technology 

Training and/or Specialized Technical Training during a selected month. 
Audit of NRC’s Technical Training Center 

20

Sample Management Measure

50

55

60

65

70

75

80

85

90

95

100

1234567

Favorable Rating (Percent)

Training

Applicability

Used in

Current

Job

Appropriate

Technical

Level

Course

Materials

Instructor

Knowledge

Overall

Course

Rating

Instructor

Clarity

In this illustration, the green section in the graph represents areas that met 

student expectations. The yellow section represents areas that could 

possibly be in need of review by TTC management, and the red section 

typically represents areas where management action is necessary. 

Question 2 in red above relates to the degree that the students’ training is 

used in their current job. In this instance, management action may not be 

necessary because several respondents may not yet have had the 

opportunity to put what they learned into practice. 

As stated previously, the purpose of this analysis is to show that valuable 

information can be learned by analyzing evaluation data. The division 

between green and yellow at 90 percent and yellow and red at 70 percent is 

subjective and was chosen here for demonstrative purposes. This graph 

demonstrates the value of this type of illustration in identifying areas for 

further management review or action. 
Audit of NRC’s Technical Training Center 

21

TTC management could readily perform this type of analysis on course 

evaluation data to identify opportunities for improvement. 

VI. BEST PRACTICES 

OIG auditors visited three federal government-training facilities to gain 

insights on practices other agencies use in the training arena. Identified 

best practices that could improve TTC’s training program relate to the use of 

pre-requisite on-line courses, maintenance of course curricula in one central 

location and the use of post-course evaluations. 

Pre-requisite On-line Courses 

Pre-requisite on-line courses provide students the same basic knowledge of 

the subject area being taught. This could be especially helpful to TTC 

instructors who have noted significant differences in the knowledge and 

experience levels of students. While some students are recent college 

graduates, others have many years of experience in the nuclear industry. 

The disparate technical knowledge and experience of such students poses 

a challenge to TTC instructors who strive to keep all students engaged in 

the learning process. 

Maintenance of Course Curricula 

The maintenance of course curricula in one central location represents 

another best practice. For example, hard copies of all current course 

manuals could be maintained in a central file room, or electronic versions of 

the manuals could be stored on a common computer drive. This ensures 

that someone other than the cognizant instructor could easily locate the 

curriculum. This is especially helpful when unforeseen circumstances 

prevent the cognizant instructor from communicating to others where or how 

the curriculum is located. Finding C of this report addresses this key area. 

Post-Course Evaluations 

Post-course evaluations sent to students and their supervisors 

six-months to one-year after course completion can be used to determine 

how well the training prepared students for their work. This practice could 

be beneficial in determining the need for any curriculum revisions. The 

post-course evaluation observed was short and concise, with one or two 

questions and space for comments. 
Audit of NRC’s Technical Training Center 

22

Appendix A 

SCOPE AND METHODOLOGY 

To accomplish the audit objectives, OIG reviewed and analyzed pertinent 

regulations, authoritative guidance, policies and procedures, and prior 

relevant OIG, GAO, and State Department reports. In addition, OIG visited 

three other federal agencies and called a fourth agency to obtain training 

best practices information; surveyed former TTC students; reviewed TTC’s 

metrics; and tested TTC’s property management system, purchase card 

transactions, travel expenses, and contracts. 

OIG conducted interviews with selected Headquarters, TTC, and Regional 

officials to: 

1. Gain an understanding of TTC’s operations, 

2. Determine current issues, problems, known deficiencies, and 

3. Assess management controls. 

OIG conducted this audit in accordance with Generally Accepted 

Government Auditing Standards from September 2005 through July 2006. 

The major contributors to this report were Steven Zane, Team Leader; 

Kathleen Stetson, Audit Manager; Terri Cooper, Senior Auditor; and 

Michael Cash, Senior Technical Advisor. 
Audit of NRC’s Technical Training Center 

23

Appendix B 

ANTICIPATED TRAINING NEEDS (1)

 Anticipated 

 Expected Number of New Hires Total Based 

TTC 

Students New Total 

Fiscal Agency Requiring on Repeat Other Than Reactor TTC 

Year Attrition (2) Growth Total TTC Training (3) Attendance (4) New Hires (5) Training (6) Students

2006 169 231 400 2588(7)

2007 177 323 500 400 2000 392 803 3195 2008 178 222 400 320 1600 416 875 2891 2009 199 201 400 320 1600 437 812 2849 

Notes: (1) A TTC representative supplied the numbers shown in the preceding table. 

(2) From 2001 through 2005, the attrition average rate was 5.38 percent. (3) Technical hires average around 80 percent of total hires. (4) On average, a technical new hire will attend four classes the first year of employment. This does not take into account the 

number of training weeks associated with each class (one to three weeks per class). 

(5) Other than new hires include: refresher training, Agreement State personnel, other government and state agencies, ad hoc 

NRC training, specialized courses, and international regulatory agencies. Annual increase based on percentage increase of FTE ceiling. 

(6) New Reactor Training consists of new courses being developed for the Office of Nuclear Reactor Regulation and 

construction training needed for Region II. This number could increase based on the numbers and types of new plant applications requiring the development of additional courses. 

7) Number based on actual attendance figures 
Audit of NRC’s Technical Training Center 

24

[Page intentionally left blank.] 


Audit of NRC’s 

Agency Response and OIG Analysis and Response

The page number in column 1 refers to the page number in the final report.

27

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

3 In FY 2005, TTC’s budget

was approximately $7.6million. This included $3.9million for interagencyagreements, commercialcontracts, and variousadministrative expensesand $3.7 million in salariesand benefits for 29 full-timeequivalents. 

We were unable to confirmthe budgetary resourcescontained on page 3 underFY 2005 TTC Statistics. It

appears they may have beendrawn from the NRC's FY2005 Enacted budget. Werecommend that the reportshould be modified to reflectFY 2005 Current Budgetfigures to more accuratelydisplay this information. Seeattached for the datacontained in OCFO’s officialrecords. 

TTC Budget FY 2005Enacted Current Per IG Report$3.5M CST $3.1M CST $3.9 M IAG’s/CST29 FTE 30 FTE 29 FTE$3.7M S&B $3.8 S&B $3.7 S&B Total $7.2M Total $6.9M Total $7.5M

NRC“See contreco

attaOIG officcontbudunadata comrem

For the most part, currentemployees will receivetraining in the technology ofthe new reactors; newemployees will learn thetechnology of the existingreactors. 

The sentence is not correct, itis an oversimplification.

Suggest deleting the sentence Sen
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

28

5 III. FINDINGS 

Collectively, TTCinstructors areknowledgeable,experienced andenthusiastic. While OfficeDirectors and former TTCstudents were generally pleased with thetraining provided by TTC,OIG identified opportunitiesfor improvement in thefollowing areas: • Inventory practices,• Policies and procedures, • Maintenance of trainingmaterials, • Scheduling courserevisions, • Training effectiveness,and • Course evaluations. 

The report does not mentionany concerns with theeffectiveness of the trainingthat TTC provides.

Change “Training effectiveness” to “Continuingtraining for instructors”

Cha“InsRota
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

29

5 As a result, TTC’s training

mission could be negativelyimpacted if property cannotbe located when required.

The report does not give anyexamples of TTC propertythat was lost or misplaced tosubstantiate this opinion. Infact, in all cases, all TTCproperty was located for eachannual property inventory andfor the property inventory thatwas performed for OIG duringtheir audit.

Although there have been no cases where TTCproperty could not be located, TTC’s trainingmission could be negatively impacted if propertycannot be located when required.

The negraiseageloca
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

30

5 Noncompliance with GAO

Standards for InternalControlTTC staff did not performan independent physicalinventory of its property, butrather relied on individualproperty holders to provideelectronic confirmation ofthe property's existenceand location. To conductinventory, the TTC propertycustodian sent propertyholders an e-mail listing theinventory items assigned tothem. The property holderswould reportedly inventorythe property assigned tothem and confirm the item’sexistence and location viae-mail to the TTC propertycustodian whosubsequently prepared theinventory report. Finally,the Director of TTCOperations would send amemo reporting thoseresults to the Office of Administration.

TTC was in compliance withthese documents. Thedescription of the TTCinventory process was notcompletely and accuratelyreflected in the report. Additionally, the sentence “The property holders wouldreportedly inventory theproperty assigned to them...”(emphasis added) seems toquestion the integrity of theTTC property holders. Thereis no evidence or discussionin the report to substantiatethis opinion.

TTC Practice for Conducting Property Inventories

TTC staff did not perform an independentverification of its property, but rather relied onindividual property holders to provide electronicconfirmation of the property's existence andlocation. To conduct this inventory, the TTCproperty custodian sent property holders an e-mail with an attachment listing the inventory itemsassigned to them. The property holders wouldphysically inventory the property assigned to themand confirm the item’s existence and location viae-mail to the TTC property custodian. The propertycustodian then resolved any discrepancies andprepared the inventory report. Finally, the Directorof TTC Operations would review the results of theinventory and send a memo reporting the inventoryresults to the Office of Administration atHeadquarters. 

No c
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

31

6 MD 13.1 provides that

regional offices shouldconduct self-inventories,but does not requireindependent verification ofproperty by someone otherthan the property holder. Effective implementation ofsegregation of dutiesreduces the risk of error orfraud so that no singleperson can adverselyimpact the accuracy andintegrity of the physicalinventory count. Recommendation

OIG recommends that theExecutive Director forOperations:1. Revise ManagementDirective 13.1 to clarify thatself-inventories performedby the regions shouldinclude independentverification of the propertyby someone other than theproperty holder. 

Regional offices may not bethe only offices that performproperty inventories usingmethodology similar to TTC. If independent verification isrequired, MD 13.1 shouldreflect this requirement for alloffices.

MD 13.1 requires conducting physical inventoriesof all NRC equipment every 2 years, but does notrequire independent verification of property bysomeone other than the property holder. Effectiveimplementation of segregation of duties reducesthe risk of error or fraud so that no single personcan adversely impact the accuracy and integrity ofthe physical inventory count. Recommendation

OIG recommends that the Executive Director forOperations:1. Revise Management Directive 13.1 to requirethat property inventories should includeindependent verification of the property bysomeone other than the property holder. 

MD “At hwill jointpropprovverifsompropCharecolangSee 
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

32

7 B. Policies and Procedures 

All of TTC’s policies andprocedures are outdatedand incomplete. They donot comply with Office ofManagement and BudgetCircular A-123, Management’sResponsibility for InternalControl (OMB CircularA-123) or the GAOStandards. This conditionexists because HRmanagement has not madefinalizing policies andprocedures a priority. Without established policiesand procedures to clearlycommunicatemanagement’sexpectations regardingauthorities andresponsibilities, staff mayuse inconsistent and lessthan optimal methods incarrying out their duties. 

This overstates the conditionof TTC policies andprocedures. Updating TTCpolicies is an activity in HR’s2006 and 2007 OperatingPlans. Updating thesepolicies has been prioritizedlower than conductingtraining, maintaining coursemanuals and materials, andstaff hiring, training anddevelopment. Asrecommended by thereferenced OMB Circular andGAO Standards, HRmanagement assessed therisk of continuing to use thepolicies in their currentcondition and determined thatthere was little to no impacton the operations of TTC. TTC has established policiesand procedures. They werecompletely revised andapproved in 1995. Since thenthe policies have beenupdated individually toincorporate significantimprovements or essentialchanges. An example is the

B. Policies and Procedures

All of TTC’s policies and procedures have not beenupdated and draft revisions since 1995 have notbeen formally approved by management. This doesnot comply with Office of Management and BudgetCircular A-123, Management’s Responsibility forInternal Control (OMB Circular A-123) or the GAOStandards. This condition exists because HRmanagement has determined that using the draftpolicies poses minimal risk to TTC operations.Without updated and approved policies andprocedures to clearly communicate management’sexpectations regarding authorities andresponsibilities, staff may use inconsistent and lessthan optimal methods in carrying out their duties.

At saudof oHR OIGgoopolicform The ManexppagNo c
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

33

revision of Policy Document(PD) 7, TTC StaffQualification”, to parallel theimprovements made to thequalification program forinspectors (IMC 1245). PD 7was also reviewed andupdated in 2003 inanticipation of many instructorretirements and new hires. The updated PDs have beenapproved for use, but havenot been formally signed bythe ADTD pending a plannedrevision to update andincorporate PDKM.Examples of the types ofoutdated information in thePDs include organizationaltitles and the use of hardcopy course catalogs andcourse manuals vs theavailability of thesedocuments on the trainingwebsite. To ensure that staffare aware of currentmanagement expectations,the staff training andqualification process
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

34

(documented in TTC PolicyDocument 7) requires staffreview of the PDs anddiscussion with thesupervisor. The Supervisor’ssignature on the qualificationcard signifies that they havediscussed the PDs with thequalifying individual. Instructors are involved withand review and comment onrevisions to PDs before theyare implemented. Management’s continuinginvolvement and oversightensures that expectations regardingauthorities andresponsibilities are clearlycommunicated to ensureconsistency andeffectiveness of the staff incarrying out their duties. 

7

TTC Policies and

Procedures Are Outdated

This title overstates thecondition of TTC policies and

TTC Policies and Procedures Need to be Revised

and Updated

Incluhea
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

35

and IncompleteThe policies andprocedures in use at TTCaddress a variety of areasincluding, but not limited to: • The general duties andresponsibilities associatedwith TTC positions; • How TTC meets thetechnical training needs ofthe NRC; • Course scheduling; • Course administration; • The TTC staff qualificationprogram; • The training materialscontrol program; and • Simulator training. These policies andprocedures are outdatedand incomplete. Seven ofnine policies are in draftform; six have been in draftform for more than 10years. Of the two policiesthat are in final form, one issigned, but undated,

procedures and implies thatthey are unusable. Themeasure of the acceptabilityof a policy should not bebased on the date it was lastrevised, but rather on whetherthe contents of the policy arestill applicable to howbusiness is currentlyaccomplished. Thediscussion does notcompletely and accuratelyreflect the condition of thePDs or internal controls inplace at TTC. It appears thatthis is partly due to OIG beingprovided only copies of theworking drafts, and not thesigned 1995 final documents.The policies were written at areasonable level such thatwhen combined with theexperience level of the TTCstaff, the qualificationprocess, and on-the-jobexperiences, the instructor isknowledgeable about thepractices and expectations ofthe position. HR expected

The policies and procedures in use at TTC addressa variety of areas including, but not limited to: • The general duties and responsibilities associatedwith TTC positions; • How TTC meets the technical training needs ofthe NRC; • Course scheduling; • Course administration; • The TTC staff qualification program; • The training materials control program; and • Simulator training. The policies were revised and approved in 1995.Two of the policies have been updated but arelabeled draft and have not been formally approved(i.e., signed by management). The other policydocuments are labeled draft and are in variousstages of revision. TTC relies on ongoingmanagement oversight and communication toensure staff are current and consistent in theirapplication of the policies. Opportunities toimprove the documentation of TTC practices areidentified in subsequent sections of this report. 

Proc

and OIG provissudrafResto p199DuraudpolicbeinSubthat hold proctrainthe The mancomstaff in thpolicOIG TTC 
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

36

though an attachment isdated November 1995; theother policy is dated August1993, but is unsigned. 

that some areas would beidentified for expansionduring the planned reviewand update. TTC has a longtrack record of conducting anexceptional training programwith the policies in place inthis condition (working drafts.) Some opportunities toimprove the documentation ofTTC practices are identified insubsequent sections of thisreport. 

to reprocThe factrepoOIG the 

7 According to both OMB

Circular A-123 and theGAO Standards, policies,procedures, andmechanisms should be inplace to help ensure thatagency objectives are met. Further, information that isrelevant, reliable, andtimely should becommunicated toappropriate personnel at alllevels within anorganization. 

Policies, procedures, andmechanisms are in place atTTC to help ensure agencyobjectives are met. Information that is relevant,reliable, and timely iscommunicated to appropriatepersonnel at all levels withinTTC. In addition to the policydocuments and discussionsduring the qualificationprocess noted above, TTCmanagers meet via VTC orphone with HR management

No change suggested. These are statements offact.

Agre
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

37

at least twice per week. Current information that isrelevant for TTC staff ispassed along to them by theirsupervisor. TTC relies onongoing managementoversight and communicationto ensure staff are currentand consistent in theirapplication of the policies. This has proven effective inmaintaining a high qualitytraining operation for at leastthe past 10 years. 

7 HR management, however,

has not made finalizingTTC’s policies andprocedures a priority. When contacted by OIG,they indicated that theyplan to issue consolidatedtraining policies whichwould also include theProfessional Developmentand KnowledgeManagement Branch atHeadquarters. 

HR management has madefinalizing policies andprocedures a priority. Updating TTC policies is anactivity in HR’s 2006Operating Plan. Updatingthese policies has beenprioritized lower thanconducting training,maintaining course manualsand materials, and staffhiring, training anddevelopment. As stated bythe referenced OMB Circularand GAO Standards, HR

When contacted by OIG, HR managementindicated that they plan to issue updated andconsolidated training policies which would alsoinclude the Professional Development andKnowledge Management Branch at Headquarters. 

The dilutfindlang
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

38

management assessed therisk of continuing to use thepolicies in their currentcondition and determined thatthere was little to no impacton the operations of TTC. Since the staff of NRR issignificantly increasing toprepare for licensing newreactors, the need fortechnical training in HQ isincreasing. To moreeffectively and efficientlymeet this need, HR is in theprocess of hiring technicalinstructors in HQ in PDKM. This will allow significantreduction in the travel costsfor TTC instructors to come toHQ to teach or costs for HQstudents to travel to TTC. Itprovides for face-to-faceworking relationship withNRR managers and subjectmatter experts. It will alsoallow HQ and TTC instructorsto back each other upproviding more flexibility forcourse scheduling and
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

39

conduct. Therefore, the PDsneed to be revised toincorporate this change. Thiscannot occur until a numberof HQ instructors are onboard, complete theirqualification, and are familiarwith NRC and HR programsand processes. This will alsofree up TTC instructors timeas the HQ instructors begin toteach courses that currentlyTTC instructors travel to HQto teach. Then we will havethe resources and theknowledge base to effectivelyrevise the PDs. Forefficiency, at this same time,we plan to do a completereview and update of thePDs. 

8 Without current policies and

procedures new TTC staffcould experience difficultyunderstanding theirresponsibilities andauthorities and take longerto do what is expected ofthem. Additionally, existing

This implies that TTC policiesand procedures are unusablein their current form and thatTTC staff do not know how toperform their duties. Thereport does not include anyexamples that support thesestatements. The

Suggest this paragraph be removed. Kee

parapoteoutdprocconsOIG conc
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

40

TTC staff may be unsure ofmanagement’sexpectations for carryingout the day-to-day operations of TTC. 

understanding of individualduties and management’sexpectations are addressedthoroughly during thequalification process (Policy7) and in Policy 4, “CourseAdministration.” Section F ofPolicy 7 describes the rolesof the course instructors, theCourse Director and theCourse Cognizant Instructorin adequate detail. Inaddition, qualifying instructorsperform each of these rolesunder the tutelage ofexperienced instructors anddiscuss each role withsupervision prior to receivinga signature for those items intheir qualification cards, perPolicy 7. The likelihood that "new TTCstaff could experiencedifficulty understanding theirresponsibilities andauthorities and take longer todo what is expected of themor existing TTC staff beingunsure of management’s

manas d
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

41

expectations for carrying outthe day-to-day operations ofTTC" is considered to be lowconsidering the focus of onsite supervision to the needsof new employees and theexperience and grade levelsof TTC employees in general. Further the potential effectand impact of the risks on theagency are low and easilyidentified and mitigated. 

8 Recommendation

OIG recommends that theExecutive Director forOperations: 3. Formalize and implementpolicies and proceduresrelated to TTC operations,regardless of whether theyare combined with theProfessional Developmentand KnowledgeManagement Branch. 

Policy documents are formaland implemented at TTC. There is no bases included inthe report that supportsupdating and revising thetraining policies andprocedures for TTCoperations separately fromPDKM. PDKM and TTC staffteach at the PDC and theTTC so the processes andprocedures for operations ofthese facilities need to beconsistent, and both staffsmust be involved in therevisions to ensure they areaccurate and adequate for all

Recommendation

OIG recommends that the Executive Director forOperations: 3. Update and finalize the training policies andprocedures.

Chain thfinaprocHR. 
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

42

staff and both locations. Effectiveness and efficiencyconsiderations require thatthe PDs be updated at thesame time they are revised toinclude PDKM. 

9 C. Maintenance of Training

MaterialsIn OIG’s opinion, prudentbusiness practices call for asystematic and wellcommunicated method ofmaintaining trainingmaterials to ensurecurrency, accuracy andcompleteness. At TTC,each course has acognizant instructorassigned to maintaintraining materials. OIGfound that TTC cognizantinstructors employ variousmethods of maintainingtraining materials and donot systematicallycommunicate their methodsto other instructors ormanagement. Further, the

This section does notcompletely and accuratelyreflect TTC method ofmaintaining training materials. It implies that coursematerials are not maintainedproperly. There is noevidence or examples tosupport this implication in thereport. We agree that there shouldbe a systematic and wellcommunicated method ofmaintaining training materials. However, in a audit report, wewould expect a standard citedas opposed to “prudentbusiness practices”. Business practices that areprudent to one organizationmay not be prudent foranother organization.

C. Maintenance of Training Materials

Prudent business practices call for a systematicand well communicated method of maintainingtraining materials to ensure currency, accuracy and completeness. At TTC, each course has acognizant instructor assigned to maintain trainingmaterials. TTC cognizant instructors employvarious methods of collecting, recording andtracking changes needed to their training materials. OIG found that TTC cognizant instructors do notsystematically communicate the status of theirrevisions to other instructors or management. Accordingly, turnover of materials from oneinstructor to another can be difficult to execute. Further, written policies do not specifically addressTTC practices for maintaining course materials orcognizant instructor duties and responsibilities withregard to maintaining course materials. 

OIG the 
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

43

procedures for maintainingtraining materials are adhoc and inconsistent. TTCmanagement does notreview or approve thevarious maintenancemethods. Accordingly,turnover of materials fromone instructor to anothercan be difficult to execute.There is a risk thatout-of-date or erroneoustraining materials will beused in courses.

The TTC instructors aretraining experts. As such,they are empowered toimplement whatever methodof collecting, recording, andtracking changes needed totheir assigned trainingmaterials works best forthem. A systematic method formaintaining course materialsis currently implemented atTTC and there is noindication that this method isdeficient. Each course has acognizant instructor assignedto maintain training materials. TTC cognizant instructorsemploy various methods ofcollecting, recording andtracking changes needed totheir training materials. Controlled copies of trainingmaterials are maintainedelectronically in a designatedfolder on the TTC networkdrive. The documents areread only access, except forthe assigned cognizant
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

44

instructor and the managerfor the subject area. Theseare the files from whichcourse manuals andmaterials are obtained foreach scheduled course. Identification of neededchanges is accomplished byreview of student courseevaluation forms, instructorfeedback, management inputand post course reviewmeetings of assignedinstructors. Routine changessuch as fixing grammaticalproblems and minor technicalrevisions are accomplishedby the assigned cognizantinstructor. Majormodifications in response toindustry or regulatorychanges are analyzed anddiscussed with theappropriate manager todetermine resourcerequirements and priorities. This is an area we plan toexpand in our update of thepolicy documents.
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

45

The risk that an instructormay use outdated trainingmaterials is very unlikelysince all instructors use thesame controlled copy fromthe network drive. 

9 Inconsistent Maintenance

of Training Materials Cognizant instructors havechosen a variety ofmethods for (1) revisingcourse materials, (2)tracking revisions to coursematerials, and (3) organizing and storingcourse materials. Thedegree of rigor, formalityand availabledocumentation varies fromcognizant instructor tocognizant instructor. At times, instructors teachcourses for which they arenot the cognizant instructorand, therefore, are notdirectly responsible for

We agree that thedocumentation of TTCpractices for maintainingcourse materials needsimprovement. However, thereport does not give anyexamples that substantiateproblems with the practices atTTC used to maintain coursematerials. There appear tobe 2 issues here: (1)turnover of needed courserevisions from one cognizantinstructor to another, and (2)maintaining controlled copiesof course materials. The first sentence isincorrect. All instructorsteach in all courses for whichthey are qualified, not just forthose for which they are

Lack of Documented Policy for Maintaining

Training MaterialsCognizant instructors are assigned to maintainthe course materials for each TTC course. Cognizant instructors are responsible for (1)identifying needed changes to course materials(2) revising course materials, (3) trackingrevisions to course materials, and (4) maintainingthe controlled copy of course materials current.  Cognizant instructors have chosen a variety ofmethods for collecting, recording, and trackingchanges needed to their assigned coursematerials. The degree of rigor, formality andavailable documentation varies from cognizantinstructor to cognizant instructor. There is nosystematic process for cognizant instructors to communicate to other instructors thestatus of revisions to their course materials.

The heathe Cha“IncMaiOIG langbold kept whicis no cogcominstrmaimat
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

46

maintaining the training materials. Additionally, they may notknow how to locate themost current trainingmaterials. There also is nosystematic process forcognizant instructors to communicateto other instructors themethods used to maintaintheir course materials. 

cognizant instructor. All staffare expected to evaluate theneed for changes to coursematerials during preparationand conduct of courses. They are expected tocommunicate neededchanges to the cognizantinstructor (and the cognizantsupervisor if a major revisionis needed.) The location of the controlledcopies of training materials iscommunicated to instructorsas part of the instructorqualification process. Giventhat the course materials areused frequently, it is unlikelythat an instructor will forgetwhere the controlled copiesare located. However thiscould easily be remedied byasking a peer or thesupervisor.The TTC instructors aretraining experts. As such,they are empowered toimplement whatever methodof gathering, recording, and
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

47

tracking changes needed totheir assigned trainingmaterials works best forthem. There is no need forcognizant instructors to dothis the same way. TTC hashad two unusual situationswhere the cognizantinstructor was not available totransfer their duties toanother instructor whenrequired. That prompted TTCmanagement to begin toassign a primary and backupcognizant instructor for eachcourse to better facilitatetransfer of cognizantinstructor duties from oneinstructor to another.

99

OIG found that writtenpolicies do not specificallyaddress methods forupdating and makingavailable training materialsfor all courses. In addition,TTC management has notreviewed or approved thevarious means used tomaintain training materials. 

A systematic method formaintaining course materialsis currently implemented atTTC. We agree that thisshould be documented in aPD.TTC management hasreviewed and approved themethods for maintaining

OIG found that written policies do not specificallyaddress TTC practices for maintaining coursematerials or cognizant instructor duties andresponsibilities with regard to maintaining coursematerials. TTC management relies on thesupervisor to verbally communicate managementexpectations and practices for maintaining coursematerials during the instructor qualification processand when policies change. A documented policyregarding cognizant instructor duties and

The dilutfindlang
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

48

As a result, there is a riskthat an instructorsubstituting for a cognizantinstructor may use outdatedtraining materials. Theoutdated training materialsmay not include recentoperating experience. 

training materials. These arecommunicated by thesupervisor to the staff duringtheir qualification process. TTC management is awarethat cognizant instructors usevarious means to collect,record, and track changesneeded to their assignedtraining materials. This isappropriate given their gradelevel and expertise. The riskthat an instructor may useoutdated training materials isvery unlikely since allinstructors use the samecontrolled copy from thenetwork drive. 

responsibilities for maintaining course materialswould provide a higher level of confidence thatcourse materials are being maintainedappropriately and consistently. 

10 Further, if a cognizant

instructor takes a newposition or leaves theagency, that individual’sresponsibilities would begiven to another agency employee. Since themaintenance processes arenot systematic, the turnoverof course materials can be

Current practice is to have aprimary and a backupcognizant instructor. This willaddress the turnover issueand will be added to theappropriate procedure. 

Further, if a cognizant instructor takes a newposition or leaves the agency, that individual’sresponsibilities would be given to another agencyemployee. If there is no opportunity for turnoverbetween the two cognizant instructors, the turnoverof course materials can be difficult to execute andcan result in an inefficient use of agency resourcesduring the turnover process. 

OIG the 
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

49

difficult to execute and canresult in an inefficient useof agency resources duringthe turnover process. 

10 Recommendation

OIG recommends that theExecutive Director forOperations: 4. Develop and implementa written policy thatprovides specificexpectations for: • Revising course materials, • Tracking revisions tocourse materials, and • Organizing and storingcourse materials.

The OIG survey results donot support any processdeficiencies in maintainingcourse materials. However,documenting our processesin a PD would provide ahigher confidence level thatcourse materials are beingmaintained appropriately andconsistently. 

Recommendation

OIG recommends that the Executive Director forOperations: 4. Document in a written policy the TTC practicesfor: • Revising course materials, • Tracking revisions to course materials, • Transferring cognizant instructor responsibilities,

and 

• Organizing and storing controlled coursematerials. 

Inclusugg“Trainstrreco

11 D. Scheduling Course

RevisionsOIG found that TTCmanagement does notobtain information neededto establish respectivepriorities and resourcerequirements relating to

This section should beremoved. TTC managementdoes obtain informationneeded to establishrespective priorities andresource requirementsrelating to pending majorrevisions of training materialsthat are beyond the capability

Delete this section. Kee

The any mancontaggidenand 
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

50

pending revisions oftraining materials. There is no specific writtenpolicy for: • Prioritizing the aggregateof needed training materialrevisions, • Identifying neededresources, and • Scheduling revision workto meet course-schedulingneeds. Consequently, resourcesmay not be available toensure that needed courserevisions are made in atimely manner. Additionally, there is a risk that training materials maynot be completely updatedfor courses. Finally,excessive costs such asinstructor overtime orcontractor costs may beincurred to meet coursedeadlines. 

of the cognizant instructor toaccomplish alone in the timescheduled for coursemaintenance. Routinechanges such as fixinggrammatical problems andminor technical revisions areaccomplished by theassigned cognizant instructorin their assigned coursedevelopment time. Majormodifications in response toindustry or regulatorychanges are analyzed anddiscussed with theappropriate manager todetermine resourcerequirements and priorities. After the major revisions ofcourse materials toincorporate the Shorehamand Trojan simulators, PD 9was developed for any futuremajor revisions to coursematerials, however therehave been none requiredover the past 10 years. There is no evidence in thereport that training materials

GivecontconsOIG concmanas d
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

51

have not been updated forcourses or excessive costssuch as instructor overtime orcontractor costs have beenincurred to meet coursedeadlines. 

1111

Improvements Needed in

Scheduling CourseRevisionsIn addition to classroomrelated responsibilities, acognizant instructor isresponsible for revisingcourse materials for theassigned course. TTCoffers a number of courses,each of which may haveseveral changes pending ata given time. Cognizant instructorsmanage the workload forrevising courses in isolationfrom the broader needs ofthe TTC organization. OIGidentified severaldeficiencies related to thisarea. Specifically, there

The majority of cognizantinstructors are responsible foronly one course. They areexpected to prioritize andcomplete pending changes,other than major revisions,within their assigned coursedevelopment time. There area small number of instructorsqualified to make revisions tocourses in each curriculumarea. Therefore managingthe workload for revisingcourses does not need toconsider the broader needs ofthe TTC organization. Completing the majority ofcourse revisions is not acomplicated problemrequiring a documentedprocess. When a cognizant instructor

Delete this section. Kee

The any mancontaggidenand GivecontconsOIG concmanas d
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

52

are no processes for: • Identifying and reportingpending changes for allcourses to TTCmanagement;• Prioritizing, allocatingresources, and schedulingchanges for the aggregateof all pending coursechanges; and • Tracking past changes todetermine whether certainchanges have previouslybeen addressed or found tobe unnecessary. These deficiencies haveoccurred because HR andTTC management have notestablished written policiesto specifically addressprioritizing the aggregate of neededtraining material revisions,identifying neededresources and schedulingrevision work to meetcourse-scheduling needs. Available resources are not

informs the supervisor that arevision is necessary for thesubsequent course and isbeyond his capability tocomplete within thescheduled coursedevelopment time, thesupervisor determinespriorities and assigns thenecessary resources tocomplete necessary courserevisions prior to the courses. This is standard practiceeverywhere in the agency. Ifa staff person cannotaccomplish their assignedwork, for whatever reason,they are expected to raise theissue to management. Thereare no examples cited in thereport to substantiate anydeficiencies in this area. Course revisions necessaryto maintain the quality of TTCcourses have beenaccomplished prior to conductof the course in all cases. TTC has not conducted anycourse with an unacceptablequality (i.e. that would
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

53

being assigned to matchthe priorities of all neededcourse revisions. As aresult, resources may notbe available to ensure that needed courserevisions are made in atimely manner. 

negatively impact studentlearning) of course materials. There are infrequentlyextenuating circumstances(i.e. course inserted inschedule to meet agencyneed, unavailability ofcognizant instructor) wherecourse revisions have had tobe made under a very tightschedule. TTC managementproperly assigned resourcesto address these situations. TTC is now assigning aprimary and backupcognizant instructor for eachcourse to reduce the impactof these types of situations. This will be incorporated inthe actions for Section C.

12 Recommendation

OIG recommends that theExecutive Director forOperations: 5. Develop and implementa written policy thatprovides specific

This recommendation shouldbe deleted. Prioritizing andassigning work is a routinemanagement function.Expectations andresponsibilities of cognizantinstructors for maintainingcourse materials will be

Delete recommendation. Rec
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

54

expectations for theaggregate of pendingchanges for all coursematerials on an ongoingbasis. The policy shouldaddress: • Prioritizing individualchanges, • Identifying availableresources, and • Scheduling the changes. 

included in an appropriate PDin response to FindingC/Recommendation 4.

13 E. Training Effectiveness

Opportunities exist toenhance trainingeffectiveness at TTC. Tooptimize instructionalmethodology and keepcourses technically freshand up-to-date, instructortraining and instructorrotation need improvement. In OIG’s opinion, soundbusiness practices and theGAO Standards call forsuch improvements. As aresult of HR management’slack of attention to theseareas, students may not be

This section is titled “TrainingEffectiveness”, however thesection deals with instructoreffectiveness. TrainingEffectiveness is broader thaninstructor attributes andgenerally includes anythingthat results in an acceptablelevel of learning by thestudents. There is noevidence or examples givenin the report to substantiateineffective training orinstructors, or that studentsare not receiving current orhigh quality training. In fact,the OIG’s survey resultsrefute these assertions.

E. Continuing Training for Instructors

Opportunities exist to enhance the planning anddocumentation of continuing training for instructorsat TTC. To ensure TTC courses continue to betechnically fresh and up-to-date, the planning anddocumentation of instructor continuing trainingneed improvement. In OIG’s opinion, soundbusiness practices and other relevant criteria callfor such improvements.

Cha“InsRotaOtheto th
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

55

getting the most current orbest quality trainingavailable. 

13 Instructor Training 

Instructors need tocontinuously sharpen theirskills and receiveintellectual stimulation. TheGAO Standards state that,“Only when the rightpersonnel for the job are onboard and are provided theright training, tools,structure, incentives, andresponsibilities isoperational successpossible.” Instructor trainingincludes technical trainingand instructionalmethodology training. 

No changes suggested. These are statements offact.

13 Technical Training

Most TTC instructors statedthat, to stay current in theirtechnologies, they rely onthe NRC Daily Reports anddiscussions with their

This section does notaccurately reflect the trainingprovided for instructors atTTC. The last sentencestates “While some

Technical Training

Most TTC instructors stated that, to stay current intheir technologies, they rely on the NRC DailyReports and on discussions with their studentsand/or industry contacts. Instructors should be in acontinuous learning mode – if they are not teaching

At thprovinstrrefreinfotable clas
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

56

students. In OIG’s opinion,sound business practicedictates that instructorsshould be in a continuouslearning mode - if they arenot teaching orpreparing/updatingteaching materials, theycould be learning by suchmeans as:• Participating as observersor members of aninspection team at anuclear power plant, • Obtaining qualification asan inspector, and/or • Completing relevanttechnical training courses. While some instructors arein the process of obtainingcurrent field experience,many are not. 

instructors are in the processof obtaining current fieldexperience, many are not. Asshown in the attached table,all instructors have obtainedcurrent field experience withinthe last two years. Assuggested as a soundbusiness practice, TTCinstructors, when they are notteaching or preparing/updating teachingmaterials, are learning by:• Participating as observersor members of an inspectionteam at a nuclear powerplant, • Obtaining qualification as aninspector, and/or • Completing relevanttechnical training coursesas well as other activities.

or preparing/updating teaching materials, theycould be learning by such means as: • Participating as observers or members of aninspection team at a nuclear power plant, • Obtaining qualification as an inspector, and/or • Completing relevant technical training courses.As a matter of practice, TTC instructors routinelyparticipate in these activities. 

expeyea

13 Instructional Methodology

TrainingTTC instructors have nothad instructional. 

This section does notaccurately reflect the trainingprovided for instructors at

Instructional Methodology Training

TTC instructors have had minimal instructionaltraining methodology training. TTC instructors areprovided formal training, as the need is identified
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

57

methodology training for atleast 10 years. Oneinstructor has not had thistraining for over 25 years. Many new instructorsemployed by TTC for fewerthan 5 years receivedinstructor training providedby their previousemployers. One newinstructor stated that typicalindustry practice requiresinstructor training every twoyears. The GAO Standardsaddress management’scommitment to competencestating in part, “Allpersonnel need to possessand maintain a level of

competence that allowsthem to accomplish theirassigned duties...”(Underlining added foremphasis.) For optimumeffectiveness, instructorsneed periodic instruction innew and innovativemethods of presentingcourse materials. 

TTC. TTC instructors areprovided formal training, asthe need is identified eitherindividually or for TTCinstructors as a group, onnew instructional methodsbeing evaluated orimplemented at TTC, or forrefresher training, forexample training on Qwizdom(an instructional tool foranonymously quizzingstudents during the class toascertain understanding ofcourse material), Toolbook(e-learning authoring tool),NEWS (tool for presentinggraphical simulations of plantsystems and parameters inthe classroom), andPowerPoint (replacingoverheads). Since the TTCinstructors are technical andtraining experts, TTC alsorelies on informal on-the-jobtraining, coaching, andknowledge transfer bysupervisors and moreexperienced instructors.

either individually or for TTC instructors as a group,on new instructional methods being evaluated orimplemented at TTC. Many new instructorsemployed by TTC for fewer than 5 years receivedinstructor training provided by their previousemployers. One new instructor stated that typicalindustry practice requires instructor training every 2years. The GAO Standards address management’scommitment to competence stating in part, “Allpersonnel need to possess and maintain a level ofcompetence that allows them to accomplish theirassigned duties...” (Underlining added foremphasis.) For optimum effectiveness, instructorsneed periodic instruction in new and innovativemethods of presenting course materials. The TTC policy for continuing training forinstructors does not contain any specificexpectations for technical or instructionalmethodology training and relies on the supervisorand instructor to determine needed training on acase-by-case basis. Further, with the exception ofsome formal classroom training, continuing trainingactivities are not documented in the official trainingrecords. 

The in th(QwPowSepafteaud
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

58

14 HR management has not

established a policyimplementing a formalmechanism to ensure thatall TTC instructors keeptechnically current and stay up-to-date withinnovative teachingtechniques. Consequently,TTC students may not begetting the most current orbest quality trainingavailable. 

HR management hasestablished a policy forcontinuing instructor trainingin Policy Document 7: 

“G. CONTINUING

TRAINING

Continuing training forqualified individuals shallbe determined by theemployee and supervisorand may be included in theIndividual DevelopmentPlan (IDP).”

To maintain technical andinstructional currency, TTCpractice is for instructors toparticipate in inspectionteams, obtain qualification asan inspector, completerelevant technical orinstructional training courses,attend conferences, makeplant visits, etc. asdetermined by the instructorand supervisor as mostappropriate for the individual

DurTTC instrencoinspquaSuccha
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

59

instructor consideringavailable opportunities andinstructors’ teaching andcourse developmentschedules. All instructorshave participated in one ormore of these activities tomaintain their technical andinstructional expertise. Consequently, innovativeteaching techniques such asQwizdom and NEWs havebeen implemented at TTC.The report further states“Consequently, TTC studentsmay not be getting the mostcurrent or best quality trainingavailable.” There is no basisfor this statement. In fact, theOIG’s own survey results andHR’s training metrics appearto contradict this statement.

14 Instructor Rotation

Instructor rotationencompasses rotation ofinstructors in and out ofTTC as well as rotation of

The report states “Theconcept of creating rotationinstructor positions in TTCwould create an environmentin which new ideas, freshperspectives, and recent

We suggest this section and the accompanyingrecommendation be moved to the best practicessection of the report. 

“Besare incluThe was 
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

60

14

cognizant instructorresponsibilities asdiscussed below. Rotational Instructor

PositionsThe concept of creatingrotational instructorpositions in TTC wouldcreate an environment inwhich new ideas, freshperspectives, and recenthands-on experience wouldbe of great value andbenefit to TTC, its students,and the NRC. Currently,TTC instructors do notrotate in and out of TTCand typically remain at TTCuntil retirement. Whencontacted by OIG, theDirector of the Office ofHuman Resources seemedreceptive to the concept ofcreating rotational instructorpositions. He mentionedthe possibility of creatingsuch positions on a pilotbasis. One federal agency

hands-on experience wouldbe of great value and benefitto TTC, its students, and theNRC.”This statement implies that“an environment in which newideas, fresh perspectives, andrecent hands-on experience”doesn’t exist currently at TTC,and that it cannot existwithout rotational instructorpositions. While HR agreesthat creating rotationalinstructor positions is worthyof reconsideration (TTC hasconsidered this in the pastand decided against it for anumber of reasons) andmany instructors do remain atTTC until retirement (largelybecause they areprofessional instructors), thealmost complete turnover ofreactor technology instructors(12 of 17) over the past 5years, in combination with thecontinuing training activitiesdiscussed above and listed inthe attached table, provides
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

61

contacted by OIG has apolicy of rotating instructorson a five-year cycle. 

for an environment in whichnew ideas, freshperspectives, and recenthands-on experience havebenefitted the trainingprograms. Other practiceswhich facilitate new ideas,fresh perspectives, andrecent hands-on experienceinclude participation ofsubject matter experts fromthe field as instructors andcourse material developersfor technical training courses,close cooperation among theoffices/regions and TTC ontraining and non-trainingactivities, and the futureexchange of HQ and TTCtechnical instructors intechnical training courses. Instructors maintain contactwith other trainingprofessionals and obtaininformation on current andemerging trainingmethodologies via the web. TTC managers attend trainingconferences to keep currenton training methodologies. 
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

62

14 Rotation of Cognizant

Instructor ResponsibilitiesCognizant instructors areresponsible for maintainingand updating coursecurriculum. A TTCinstructor assigned as acognizant instructor for aparticular course generallyretains the responsibility forthat course for a long time. In OIG’s opinion, soundbusiness practices call forkeeping course materialsfresh by obtaining newideas and new views. Rotation of cognizantinstructor responsibilitieswould accomplish thisobjective, along withproviding them intellectualstimulation and aheightened level of interestand engagement in thecourses that they teach. ATTC official stated thatrotation of cognizantinstructor responsibilities isnot a routine practice, but it

The stated objective is“keeping course materialsfresh by obtaining new ideasand new views”. Since allinstructors teach in allcourses for which they arequalified, not just those forwhich they are cognizantinstructors, this objective isalready addressed. Allinstructors are expected toprovide feedback regardingthe courses in which theyparticipate. Rotation ofcognizant instructor duties isconsidered as part of thenormal assignment of work.

Rotation of Cognizant Instructor Responsibilities

Cognizant instructors are responsible formaintaining and updating course curricula. A TTCinstructor assigned as a cognizant instructor for aparticular course generally retains the responsibilityfor that course for a long time. In OIG’s opinion,sound business practices call for keeping coursematerials fresh by obtaining new ideas and newviews. Rotation of cognizant instructorresponsibilities would provide an additional way toaccomplish this objective, along with providingthem intellectual stimulation and a heightened levelof interest and engagement in the courses that theyteach. A TTC official stated that rotation ofcognizant instructor responsibilities is not a routinepractice, but it is an idea worth considering. 

IG inrepoan athis 
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

63

is an idea worthconsidering. 

141415

Recommendations

OIG recommends that theExecutive Director forOperations: 6. Develop and implementa policy requiring TTCinstructors to maintaintechnical currency. 7. Periodically provideinstructional methodologytraining for all TTCinstructors. 8. Develop and implementa pilot program forrotational instructorpositions at TTC. 9. Require periodic rotationof cognizant instructorresponsibilities. 

6. TTC has a policy forcontinuing training forinstructors and in practice,TTC instructors havecompleted many activities toget current field experience tomaintain technical currency. The continuing trainingsection in PD 7 could berevised to add more specificdetail about the purpose andexpectations for technicaltraining for instructors.7. Instructors have beenprovided instructionalmethodology training asneeded. For example,training in the use ofPowerPoint was conducted inOctober 2006. The continuing

Recommendations

OIG recommends that the Executive Director forOperations: 5. Revise the policy for continuing training for TTCinstructors to add specific detail about the purpose,expectations, and documentation for technicaltraining for instructors.6. Revise the policy for continuing training for TTCinstructors to add specific detail about the purpose,expectations, and documentation for instructionalmethodology training for instructors. 7. Evaluate whether requiring periodic rotation ofcognizant instructor responsibilities would assist inkeeping course materials fresh by obtaining newideas and new views. 

Chareco“Perinstrrepo
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

64

training section in PD 7 couldbe revised to add morespecific detail about thepurpose and expectations forinstructional methodologytraining for instructors. 8. There are many otherways in place at TTC to bringfresh new ideas, etc. It hasbeen more effective andefficient to send instructorsout rather than to bringinspectors in. 9. This needs to beconsidered on a case-bycase basis for eachcurriculum area. For someareas, there are only two orthree instructors qualified inthat area, so rotatingcognizant instructorassignments among fewpeople who all teach thecourses may not be efficientor effective.
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

65

16 F. Course Evaluations

When used effectively,course evaluations are avaluable tool to assuremanagement that thetraining program fulfills theneeds of students, as wellas to identify opportunitiesfor improvement. WhileTTC instructors obtainimmediate feedback fromstudents at the end of eachtraining session, courseevaluations are notcompiled in an officialdatabase. Such actionwould allow TTCmanagement to analyzeresults and identify areasneeding improvement. Furthermore, theevaluations used toevaluate courses taught byTTC instructors do notinclude specific questionsrelated to instructorperformance. In OIG’sopinion, both of theseconditions are at variance

There are no examples ofnegative consequencesattributed to the current TTCpractices regarding courseevaluation. There are othermore appropriate and costeffective ways to accomplishthe trending of the data. Theprojected impacts of nothaving an official database orquestions related to instructorperformance are significantlyoverstated and are refuted byhistory, the results of the OIGsurvey, and HR trainingmetrics.

F. Course Evaluations

When used effectively, course evaluations are avaluable tool to assure management that thetraining program fulfills the needs of students, aswell as to identify opportunities for improvement.While TTC obtains immediate feedback fromstudents at the end of each training session andhas a process in place to review the evaluationsand take actions in response to the feedback, themethods to track and trend evaluation results needimprovement. Such action would allow TTCmanagement to consider longer-term trends instudent comments. Furthermore, the evaluationsused to evaluate courses taught by TTC instructorsdo not include specific questions related toinstructor performance. TTC management relieson infrequent supervisory observation and openended questions on the course evaluations toidentify instructor performance problems. A morerigorous method for trending course evaluationsand obtaining feedback on instructor performancewould increase the likelihood that negative trendswould be noticed and corrected.

The dilutfindpresare effetrenNo c
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

66

with prudent businesspractices and other relevantcriteria. As a result,negative trends could gounnoticed and uncorrected. Uncorrected problems incourse content and/orinstructor performancecould adversely affect thequality of training received. 

16 Official Database

While some managers andinstructors at TTC maintainindividual databasescontaining informationrelated to courseevaluations, TTCmanagement has notestablished a central,official database to compileand analyze courseevaluation results. TheGAO Standards providethat “Internal control shouldgenerally be designed toassure that ongoingmonitoring occurs in thecourse of normal

This does not fully andaccurately represent theconditions at the TTC. HRcalculates and monitors twoperformance measuresrelated to trainingeffectiveness. These arepercentage of students thatrate training satisfactory orbetter, and percentage ofstudents that attain a passingexam score. The managerfor each program maintains aspreadsheet of evaluationand exam data to calculatethe quarterly training metrics. This methodology facilitates

Official Database

HR calculates and monitors two performancemeasures related to training effectiveness. Theseare percentage of students that rate trainingsatisfactory or better, and percentage of studentsthat attain a passing exam score. The manager foreach program maintains a spreadsheet ofevaluation and exam data to calculate the quarterlytraining metrics. HR and TTC management feelthat this methodology facilitates data analysis andhas been effective in monitoring the quality oftraining. In OIG’s opinion, TTC managementshould implement a methodology, such as acentral, official database, to more rigorously trackand trend course evaluation data. Such actionwould provide TTC management and instructors anadditional tool to identify and correct problemareas.

The meathe do naudNo c
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

67

operations.” Accordingly, inOIG’s opinion, prudentbusiness practice requiresthat TTC managementshould compile courseevaluation results in anofficial database. Inaddition, an appropriatemethodology should beestablished to facilitate dataanalysis. Such actionwould provide TTCmanagement andinstructors the necessarytool to identify and correctproblem areas. In light ofthe fact that TTC managersdo not routinely observeinstructors in theclassroom, course evaluation analysis iscritical to successfuloperations. 

data analysis and has beeneffective in monitoring thequality of training. Therefore,the cited GAO Standard ismet. Course evaluationresponsibilities are specifiedin PD 4. Regarding feedbackon course content, material,relevancy, and presentation,ongoing monitoring of TTCcourse evaluations occurs inthe course of normaloperations for each coursesession that is conducted.Comments from studentstaught by TTC staff arereviewed by the courseinstructors, cognizantinstructor, and the cognizantTTC manager. Likewise,comments for contractedcourses are reviewed by thecourse’s Project Officer andthe cognizant TTC manager.Actions are then taken toaddress issues, asnecessary, identified by thestudents, and to considerlonger-term trends in studentcomments. Instructors and
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

68

managers have access toprevious course evaluations,allowing them to look forrepeat issues. Because ofthe experience level of TTCstaff, management has foundthat routine observation ofinstructors in the classroom isnot necessary. At leastannual observation isperformed to support theperformance appraisalprocess. Tools other than databases,could be implemented toaccomplish more rigoroustrending. Use of a databaseis not necessary and may notbe supportable from a costbenefit basis.

16 Instructor Performance

Course evaluations usedfor contracted courses

With respect to feedbackinvolving instructorperformance, two separatecases exist. The fact that

Move to Best Practices section or:Monitoring Instructor Performance Course evaluations used for contracted courses

Cha“EvaPerf
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

69

16

include questionsspecifically related toinstructor performance;however, evaluations forcourses taught by TTCinstructors do not. ThePerformance Elements andStandards for ReactorTechnology Traininginstructors provide that theinstructor “Seeks feedbackfor both course andself-improvement fromstudents, peers,management, and clientsand acts onrecommendations asappropriate.” By notincluding questions relatedto instructor performanceon all course evaluations,TTC management has notfully implemented theelements and standardsreferred to above. Aspreviously stated, TTCmanagers do not routinelyobserve instructors in theclassroom. Therefore, it isimportant that managers,

TTC staff cannot monitorevery contracted course (dueto geographical or timeconstraints) has led to thepractice of asking students toprovide feedback oncontracted course instructors. This has allowed for bothfeedback to contractedinstructors on their individualperformance and responsiblemanagement of contractedservices. In contrast to thecase of contractedinstructional services, TTCmanagers have routineaccess to presentations byTTC instructors. Accordingly,TTC managers evaluate eachinstructor at least once peryear through attendance ofselected training sessions;sometimes more oftendepending on the experienceof the instructor. In bothcases of assessing instructorperformance, TTCmanagement has theadditional benefit of courseevaluations completed by

include questions specifically related to instructorperformance; however, evaluations for courses taught by TTC instructors do not. ThePerformance Elements and Standards for ReactorTechnology Training instructors provide that theinstructor “Seeks feedback for both course andself-improvement from students, peers, management, and clients and acts onrecommendations as appropriate.” TTC management and instructors haveimplemented this expectation through review ofstudent comments on instructor performanceincluded in the open-ended questions on theevaluation form, direct feedback from students,managers, and client offices, and the annualperformance appraisal process. As previously stated, TTC managers do notroutinely observe instructors in the classroom. More frequent and specific feedback on instructorperformance would increase the likelihood thatperformance problems would be identified andcorrected. 

“Besare incluIncoreposubschaStudperfinforoutanawar
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

70

as well as instructors,obtain feedback fromstudents on TTC instructorperformance.Without an official databaseto compile and analyzecourse evaluations,negative trends could gounnoticed and uncorrected. Additionally, failure to obtain student feedback onTTC instructor performancecould negatively affecttraining quality. 

students to provide bothdirect (through qualitativecomments solicited onfeedback forms) and indirect(through overall assessmentsstudents make of the coursesthey attend) feedback on thecourse and, by extension,instructor effectiveness. From a performance-basedperspective, it is reasonableto infer instructoreffectiveness from overallcourse effectiveness. Additionally in the future, RTTfirst line supervisors willspend 50 percent of their timeon instruction activities,significantly increasing thedirect observation ofinstructors in the classroom.Thus, TTC considers itsability to assess NRCinstructor performancesufficient to the goal ofproviding high-qualityinstruction. The OIG surveyresults substantiate this. TTC management and
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

71

instructors have implementedthe stated element andstandard. This isaccomplished by reviewingstudent evaluations (studentsfrequently comment oninstructor performance andthe comments areoverwhelminglycomplimentary), directfeedback from students,managers, and customers,and the annual performanceappraisals. There is no basisor evidence that thesepractices are ineffective orneed improvement. Wesuggest that the discussionabout questions regardinginstructor performance bemoved to the best practicessection or we offer thesuggested revisions. 

17 Recommendations

OIG recommends that theExecutive Director forOperations: 

Recommendations

OIG recommends that the Executive Director forOperations: 
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

72

10. Establish a singleofficial database to compileall TTC course evaluationsand periodically analyzeresults for appropriateaction. 11. Include questionsspecific to instructorperformance on all courseevaluations. 

9. Establish a more formal method to track andtrend TTC course evaluations and periodicallyanalyze trends for appropriate action. 10. Establish a method to obtain more frequentand specific feedback on TTC instructorperformance. 

Rev10 apg. The dilutreco

19 V. SURVEY OF FORMER

STUDENTS OIG surveyed 183 formerTTC students via email, todetermine the degree ofsatisfaction with theirlearning experience. Sixty-eight former studentsresponded, yielding a 37percent response rate. Thefeedback was generallyfavorable as shown in thetable below: 

One of the objectives of thisaudit was to identifyopportunities to improveeffectiveness of TTCoperations; it is important tonote the based on OIG’ssurvey information, most ofthe students found the TTCprogram to be very favorable(page 25). The favorableresults of the OIG survey arediscussed after the findingsand recommendations. Thefavorable results of thesurvey need to be integratedthroughout the report andreferred to as appropriate. Although one does not

BasrespOIG projthe grapillusinfoanafrom In thor cprogWe for i
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

73

dispute that we can do better,there is no empirical evidencethat we have missed thetarget.

opinope

2021

The purpose of thisanalysis is to show thatvaluable information can belearned by analyzingevaluation data. Thedivision between green and yellow at 90 percent andyellow and red at 70percent is subjective andwas chosen here fordemonstrative purposes. This graph is not meant asa measure of theeffectiveness or quality ofsurvey information providedto OIG. This chartdemonstrates the value ofthis type of graphic foridentifying areas for furthermanagement review oraction. TTC managementcould readily perform thistype of analysis on courseevaluation data to identifyopportunities for

Compiling data may not bethe most cost efficient methodfor determining improvementsneeded. Data needs to betracked and trended bycourse and by instructor to bemeaningful. Recommendation #10specifies that a single officialdatabase to compile all TTCcourse evaluations bedeveloped. We recommendyou refer to YellowAnnouncement datedSeptember 7, 2006, forinformation concerning theagency's policies andprocedures for developinginternal use software. (OCFOcomment)

This demanaidenneeused techReccha
Audit of NRC’s 

Pg. #Note 

OIG (Draft) Report Comment Suggested Report Language

 to Resolve Comment

The page number in column 1 refers to the page number in the final report.

74

improvement.