POLICY ISSUE

NEGATIVE CONSENT

March 14, 2002 SECY-02-0044

FOR: The Commissioners 

FROM: William D. Travers

Executive Director for Operations

SUBJECT: ANNUAL STATUS REPORT FOR FY 2001 ON THE ADMINISTRATION OF

THE NRC'S REQUALIFICATION PROGRAM AND THE RESULTS OF

INITIAL OPERATOR LICENSING EXAMINATIONS (WITS 198800098)

PURPOSE:

To inform the Commission of the status of the NRC's licensed operator requalification program

and the results of the agency's initial licensing examinations for reactor operator (RO) and

senior reactor operator (SRO) applicants for fiscal year (FY) 2001, and to obtain (by negative

consent) Commission approval to discontinue the requirement to submit an annual report on

these subjects, given the stability of the licensed operator requalification program and the initial

licensed operator program.

BACKGROUND:

On June 9, 1988, the Commission requested that the staff provide "... quarterly status reports

on the new NRC requalification program until such time as it is running smoothly." On 

August 28,1989, the staff issued the first of these periodic reports (SECY-89-264). On

March 5, 1991, in a staff requirements memorandum responding to SECY-91-034 (dated

February 8, 1991), the Commission relaxed its requirement, requesting that the staff provide "...

semi-annual reports until the staff is satisfied with the stability of the program." In another staff

requirements memorandum dated January 8, 1992, responding to SECY-91-386 (dated

December 2, 1991), the Commission again relaxed its requirement, directing the staff to

henceforth submit an annual report, which also communicated the results of the NRC's initial

RO and SRO licensing examinations and licensed operator requalification examinations. The

most recent of these reports was SECY-01-0038 (dated March 9, 2001). These reports were

instituted at a time when weaknesses had been identified in licensed operator requalification

training at several facilities and the NRCís requalification examination program had substantially 

CONTACT: David C. Trimble, NRR/DIPM/IEHB/IOHS

(301) 415-2942
-2-

changed. Those weaknesses were the subject of Information Notice 90-54 dated August 28,

1990, and included such issues as weaknesses in crew communications and command and

control. Since that time, facility licenseesí operator requalification programs have improved,

and NRC inspections have identified few problems. Over the past 8 years, the NRCís operator

requalification program requirements have stabilized. 

DISCUSSION:

NRC Requalification Program and Inspection Summary for Fiscal Year 2001

During FY 2001, the staff continued to monitor and inspect facility licenseesí licensed operator

requalification training and examination programs. The objectives of each inspection were to

(1) confirm that the requalification programs contributed to safe power plant operation by

evaluating how well individual operators and crews mastered training program objectives;

(2) assess licensee effectiveness in evaluating and revising requalification programs for

licensed operators on the basis of operational performance (including requalification

examinations); (3) determine licensee effectiveness in ensuring that licensed individuals satisfy

the conditions of their licenses as specified in 10 CFR 55.53, "Conditions of Licenses"; and (4)

provide regional management with the information it needs to assess the performance of

licenseesí operator requalification programs and to determine the need for additional

inspections or NRC-conducted examinations. The staff evaluated the programs using the

process described in NRC Inspection Procedure (IP) 71111.11, ìLicensed Operator

Requalification Program.î In December 2000, the staff enhanced the requalification inspection

program by implementing the operator requalification significance determination process 

(OR SDP) under the reactor oversight process (ROP), which uses a logic flowchart and matrix

to determine the risk-importance of any issues identified during a requalification inspection.

The regional staff conducts a requalification program inspection at each facility at least every

24 months, consistent with each licenseeís requalification examination cycle. In addition, the

resident inspector staff reviews licensed operator requalification testing and/or training

activities at least once each quarter. The staff may also conduct ìfor causeî requalification

examinations, as needed, when it loses confidence in a licenseeís ability to conduct its own

examinations or when it believes that the inspection process will not produce the necessary

insights into the quality of the licenseeís program. During FY 2001, the staff did not conduct

any "for cause" requalification examinations. 

The staff conducted biennial licensed operator requalification inspections at 36 power reactor

facilities during FY 2001 and identified no findings of more than very low significance, i.e.

greater than green. Therefore, the staff believes that the power reactor facilities inspected are

sustaining the performance level of their licensed operator requalification training programs. 

The following table depicts the licensees' improved performance in conducting their

requalification programs, as evidenced by no major program failures since FY 1994. 

Under the ROP, two findings of a safety-significant nature were identified outside the focus

period, i.e., FY 2001, of this paper and are currently under staff review. In October 2001, 

Indian Point Unit 2 experienced a high crew failure rate during facility-administered, annual

requalification operating examinations. Four of seven crews failed the scenario portion of their
-3-

annual exam. This finding was assessed using the OR SDP and has preliminarily been

determined to be a yellow finding. Corrective actions are underway, and affected operators

were not returned to shift duties until they satisfactorily completed training and subsequent

NRC Requalification Program Evaluation Results for Fiscal Years 1994 through 2001

Element 1994 1995 1996 1997 1998 1999 2000 2001

Number of Requalification

Programs Evaluated 

43 58 41 41 32 40 41 36

Number of

Satisfactory*/Unsatisfactory

Requalification Programs

43/0 58/0 41/0 41/0 32/0 40/0 41/0 36/0

Percent Satisfactory 100 100 100 100 100 100 100 100

*A program rating of satisfactory indicates that the licenseeís requalification program complied with the

requirements of 10 CFR 55.53 and 55.59 for the areas inspected, and the staff did not elect to conduct NRCadministered "for cause" requalification examinations as a result of any findings.

evaluations. In the fall of calender year 2001, a review of a July 2000 biennial written

requalification examination at Cooper Nuclear Station determined that the integrity of that

examination had potentially been compromised to the extent that the pass/fail determinations

of a limited number of operators may have been affected. The potential compromise resulted

from an improper examination validation process and has been preliminarily determined to be

a white finding. Requalification inspections at other facilities have not indicated that these

issues represent problems that might be generic to the industry. 

Summary of Initial Examination Results

The staff is continuing to administer initial licensing examinations to applicants for RO and

SRO

licenses at power and non-power reactor facilities. During FY 2001, the staff administered

approximately 58 site-specific initial licensing examinations to 439 applicants (195 ROs and

244 SROs) at power reactor facilities. This number includes 46 site-specific licensing

examinations that were prepared, in whole or in part, by facility licensees in accordance with the

NRCís examination guidance in NUREG-1021, "Operator Licensing Examination Standards for

Power Reactors." The NRC staff reviews and approves all facility-prepared examinations, and

NRC certified examiners administer all of the operating tests. In addition to the facility-prepared

examinations that were administered to 68 percent of the applicants (compared to 71percent in

FY 2000), the staff administered NRC-prepared examinations to 32 percent of the applicants

(compared to 29 percent in FY 2000). 

The following table summarizes the power reactor initial operator licensing examinations results

from FY 1998 through FY 2001; results for NRC-prepared and facility-prepared examinations

are listed separately.

These results indicate that initial operator training programs at power reactors continue to
-4-

produce a large number of applicants who pass the operator licensing examinations, regardless 

of whether the examinations were prepared by the NRC or by the facility licensees. Although

Power Reactor Initial Examination Results

Examination

Percentage of Applicants Who Passed During the Fiscal Year

1998 1999 2000 2001

Exam Prepared By Exam Prepared By Exam Prepared By Exam Prepared By

NRC Facility NRC Facility NRC Facility NRC Facility

RO

Written N/A 89 100 89 98 95 96 86

Operating N/A 99 100 93 100 98 99 100

SRO

Written 100 96 100 94 100 95 99 95

Operating 94 96 100 98 96 99 100 97

the FY 2001 RO written examination pass rate on NRC-prepared examinations is higher than

that on facility-prepared examinations, the average grades on those examinations differed by

only 1.8 percent (84.7 percent versus 86.5 percent). 

The quality of examinations prepared by facility licensees is generally adequate, but

consistency in examination quality across licensees and examination developers is still a

problem, in part because of the turnover of experienced examination developers. Several

facility licensees elected not to prepare their own examinations and, thus, have little or no

experience. The staff continues to seek ways to improve the initial licensing examination

process, while reducing unnecessary burden and maintaining the integrity of the examinations. 

Example program improvements are discussed later in this paper. 

The following table indicates the total number of applicants who requested NRC review of their

examination results, including the outcomes from FY 1995 through FY 2001. During FY 2001,

the staff noted a significant decrease in the percentage of proposed applicant denials being

overturned by the review process. (The one overturned denial was attributed to new

information provided by the applicant. The new information was not identified by the NRC or

facility licensee during the pre-examination review and validation process or the postexamination review and comment process.) The staff believes that regional and national

public workshops over the past 3 years with examiners and the industry regarding the

NUREG-1021 examination development process, coupled with routinely occurring public

meetings between the staff and the industry's focus group on initial operator licensing issues,

have brought attention to concerns regarding the examination review and validation process

and thereby increased the effectiveness of the facility licensees' quality review checks for

examination test items. The staff also believes that these efforts, together with NRC examiner

reviews of facility-prepared examinations, have contributed to the decrease in the number of

overturned denials. In FY 2002 to date, the staff has received no applicant requests for NRC

review of proposed license denials.
-5-

Power Reactor Initial Examination Denial Results

Description 1995 1996 1997 1998 1999 2000 2001

Number of Proposed

Applicant Denials

30 47 44 41 60 19 35

Number of Applicant

Requested Reviews

4 12 13 20 16 9 4

Number of Final Denials 30 44 35 28 47 16 34

Number of Licenses

Issued Upon Review

0 3 9 13 13 3 1

The following table summarizes the results of the non-power reactor initial operator licensing

examinations from FY 1997 through FY 2001. During FY 2001, the staff administered

approximately 26 site-specific initial licensing examinations to RO and SRO applicants at nonpower reactor facilities (compared to 20 in FY 2000) in accordance with the current examination

guidance in NUREG-1478, "Non-Power Reactor Operator Licensing Examiner Standards."

Non-Power Reactor Initial Examination Results

Examination

Percentage of Applicants Who Passed During the Fiscal

Year

1997 1998 1999 2000 2001

RO

Written 70 87 63 78 94

Operating 93 100 96 89 98

SRO

Written 100 94 100 82 89

Operating 95 100 100 100 100

The non-power initial operator licensing program has not been modified and remains consistent

and stable. The results continue to indicate that training programs for non-power reactor facility

operators generally produce applicants who pass the NRC's licensing examinations. 

Operator Licensing Program Initiatives

During FYs 2001 and 2002 to date, the NRC continued its efforts to improve and support the 
-6-

oversight of the operator licensing program and respond to stakeholder concerns. The staffís

initiatives include the following examples: 

(1) Augmented NRC Inspection Manual Chapter 0609 by issuing the OR SDP. This SDP

is used to determine the risk-importance of issues that are identified during licensed

operator requalification program inspections, such as exam quality, security and

grading, and the performance of licensed operators on the biennial written

examinations and annual operating tests. 

(2) Completed rulemaking (66 FR 52657) to amend 10 CFR Part 55, effective 

November 16, 2001, and issued Revision 3 to Regulatory Guide 1.149, "Nuclear Power

Plant Simulation Facilities for Use in Operator Training and License Examinations." 

This rule change takes advantage of improvements in simulator technology and

permits operator license applicants to fulfill a portion of the required experience

prerequisites by manipulating the controls of a plant-referenced simulator as an

alternative to manipulation of the controls of the actual nuclear power plant. It also

eliminates the requirements for facility licensee certification of their simulation facilities

and routine submittal of reports to the NRC. Continued assurance of simulator fidelity

is provided because a facility licensee must: (1) conduct performance testing and retain

results for four years; (2) correct modeling and hardware discrepancies and

discrepancies identified from scenario validation and from performance testing; (3)

make the results of any uncorrected performance test failures available onsite; and (4)

maintain the provisions for license application, examination, and test integrity

consistent with Section 55.49. Lastly, the final rule facilitates voluntary licensee

transition to the improved simulator testing described in ANSI/ANS-3.5-1998, "Nuclear

Power Plant Simulators for Use in Operator Training and Examination." 

(3) Issued a supplement to Revision 8 of NUREG-1021, "Operator Licensing Examination

Standards for Power Reactors,î dated April 2001. That supplement included three

significant changes. First, it implemented a systematic and random selection of topics

and questions for the written examination thereby making it possible to relax the limits

on question usage. Second, it updated the guidelines regarding training and

qualification of operator license applicants to conform with Revision 3 of Regulatory

Guide 1.8, "Qualification and Training of Personnel for Nuclear Power Plants." Finally,

it clarified the documentation of NRC staff concerns related to draft examination quality. 

(4) Developed a Generic Fundamentals Examination (GFE) website, which enables users to

access the entire GFE test bank consisting of validated items and past examinations. 

Users of the website are also able to view and download publicly available GFE test files

for training and testing purposes. 
-7-

CONCLUSION:

The NRC's licensed operator requalification inspection program continues to effectively ensure

that those individuals who are licensed to operate or supervise the reactor controls are

maintaining the required level of competence to safely perform their licensed duties. The

NRC's initial operator licensing examination program continues to provide reasonable

assurance that only those applicants who have mastered the knowledge, skills, and abilities

required to safely operate and supervise the reactor controls are being licensed to do so. 

RECOMMENDATION:

The staff believes, based upon improvements made in operator requalification training across

the industry, stabilization of operator requalification program requirements, and the results of

initial examinations, that the Commission's objective in instituting this periodic report has been

met, i.e., the staff is satisfied with the stability of these programs. Moreover, much of the

information that the staff has historically provided in this annual report, including initial

examination performance trends and requalification inspection findings, is available on the

NRCís operator licensing and reactor oversight process websites. Use of these websites in

lieu of development of this annual report will increase NRC staff effectiveness and efficiency. 

The resource savings of no longer providing this annual report is one-tenth FTE annually. 

The staff will continue to inspect and monitor the quality of facility licensed operator

requalification training and examination programs under the ROP. Additionally, the staff will

continue to administer initial licensing examinations to RO and SRO applicants, review and

approve facility-prepared examinations, monitor the initial operator licensing program to assure

continued stability, and will continue its efforts to improve the process and respond to

stakeholder concerns. Therefore, absent further direction from the Commission, the staff will

no longer provide this annual status report on the administration of NRC's requalification

program and the results of initial operator licensing examinations. 

Staff requests action within 10 days. Action will not be taken until the SRM is received. We

consider this action to be within the delegated authority of the EDO.

/RA/

William D. Travers

Executive Director 

 for Operations
CONCLUSION:

The NRC's licensed operator requalification inspection program continues to effectively ensure

that those individuals who are licensed to operate or supervise the reactor controls are

maintaining the required level of competence to safely perform their licensed duties. The

NRC's initial operator licensing examination program continues to provide reasonable

assurance that only those applicants who have mastered the knowledge, skills, and abilities

required to safely operate and supervise the reactor controls are being licensed to do so. 

RECOMMENDATION:

The staff believes, based upon improvements made in operator requalification training across

the industry, stabilization of operator requalification program requirements, and the results of

initial examinations, that the Commission's objective in instituting this periodic report has been

met, i.e., the staff is satisfied with the stability of these programs. Moreover, much of the

information that the staff has historically provided in this annual report, including initial

examination performance trends and requalification inspection findings, is available on the

NRCís operator licensing and reactor oversight process websites. Use of these websites in

lieu of development of this annual report will increase NRC staff effectiveness and efficiency. 

The resource savings of no longer providing this annual report is one-tenth FTE annually. 

The staff will continue to inspect and monitor the quality of facility licensed operator

requalification training and examination programs under the ROP. Additionally, the staff will

continue to administer initial licensing examinations to RO and SRO applicants, review and

approve facility-prepared examinations, monitor the initial operator licensing program to assure

continued stability, and will continue its efforts to improve the process and respond to

stakeholder concerns. Therefore, absent further direction from the Commission, the staff will

no longer provide this annual status report on the administration of NRC's requalification

program and the results of initial operator licensing examinations. 

Staff requests action within 10 days. Action will not be taken until the SRM is received. We

consider this action to be within the delegated authority of the EDO.

/RA/

William D. Travers

Executive Director 

 for Operations

ACCESSION #ML 020510182 (SECY Paper)

*See previous concurrence

To receive a copy of this document, indicate in the box: "C" = Copy without enclosures "E" = Copy with enclosures "N" = No copy

OFFICE IOHS/DIPM IOHS/IEHB IEHB/DIPM TECH ED DIR/DIPM

NAME LVick:lv/rc DTrimble TQuay PAGarrity BBoger

DATE 02/08 /2002 * 02/12/2002 * 02/13/2002 * 02/ 11 /2002 * 02/20/2002 *

OFFICE ADIP/NRR NRR EDO

NAME RWBorchardt SCollins WTravers

DATE 02/28/2002 * 02/28/2002 * 03/ 14/2002

OFFICIAL RECORD COPY