POLICY ISSUE

NEGATIVE CONSENT

March 12, 2003 SECY-03-0038

FOR: The Commissioners 

FROM: William D. Travers

Executive Director for Operations

SUBJECT: ANNUAL STATUS REPORT FOR FY 2002 ON THE ADMINISTRATION

OF THE NRC’S REQUALIFICATION PROGRAM AND THE RESULTS

OF INITIAL OPERATOR LICENSING EXAMINATIONS

(WITS 198800098)

PURPOSE:

To inform the Commission of the status of the Nuclear Regulatory Commission’s (NRC’s) 

licensed operator requalification program and the results of the agency's initial licensing

examinations for reactor operator (RO) and senior reactor operator (SRO) applicants for fiscal

year (FY) 2002, and to obtain (by negative consent) Commission approval to discontinue the

requirement to submit an annual report on these subjects, given the stability of the licensed

operator requalification program and the initial licensed operator program.

BACKGROUND:

Since August 28, 1989, the staff has provided periodic reports on the NRC's licensed operator

requalification program. In FY 1992, the staff started including status information in the report

on the initial licensed operator program. In the last annual report (SECY-02-0044) submitted on

March 14, 2002, the staff recommended discontinuing this report. On April 3, 2002, in a staff

requirements memorandum (SRM) responding to SECY-02-0044, the Commission disapproved

the staff’s recommendation but stated that, if the stabilization of the program continued, the

staff should recommend discontinuing the reporting requirement in the next (FY 2002) annual

report. In reference to facility-prepared initial licensing examinations, the SRM directed the staff

to evaluate the continuing issues associated with ensuring consistency in examination quality

among licensees and examination developers. The staff was also directed to provide additional

information regarding implementation of the operator requalification significance determination

process. 

CONTACT: 

David C. Trimble, NRR/DIPM

(301) 415-2942
-2-

DISCUSSION:

NRC Requalification Program and Inspection Summary for Fiscal Year 2002 

During FY 2002, the staff continued to monitor and inspect facility licensees’ licensed operator

requalification training and examination programs. The objectives of the inspections were to

(1) confirm that the requalification programs contributed to safe power plant operation by

evaluating how well individual operators and crews mastered training program objectives;

(2) assess licensee effectiveness in evaluating and revising requalification programs for

licensed operators on the basis of operational performance (including operator performance on

requalification examinations); (3) assess licensee effectiveness in ensuring that licensed

operators satisfy the conditions of their licenses as specified in 10 CFR 55.53, "Conditions of

Licenses;" and (4) provide regional management with the information it needs to assess the

effectiveness of licensees’ operator requalification programs and to determine the need for

additional inspections or NRC-conducted examinations. The staff evaluated the programs

using the process described in NRC Inspection Procedure (IP) 71111.11, “Licensed Operator

Requalification Program.”

In August 2002, the staff enhanced the requalification inspection program and revised

IP 71111.11 to implement the final rule to amend 10 CFR Part 55 (66 FR 52657), which

became effective on November 16, 2001. The final rule, known as the simulator rule,

amended 10 CFR Part 55 to permit applicants for operator and senior operator licenses to

fulfill some required experience prerequisites by manipulating a plant-reference simulator as

an alternative to manipulation of the actual plant controls. The IP now incorporates

assessments of (a) the adequacy of the facility licensee's simulation facility for use in operator

licensing examinations and for satisfying experience requirements as prescribed in 10 CFR

55.46, and (b) the effectiveness of the facility licensee's process for assuring simulator fidelity

by identifying, reporting, correcting, and resolving simulator discrepancies. 

The regional staff conducts a requalification program inspection at each facility at least every

24 months, consistent with the licensee’s requalification examination cycle. In addition, the

resident inspector staff reviews licensed operator requalification testing and/or training

activities at least once each quarter. The staff may also conduct “for cause” requalification

examinations if it loses confidence in a licensee’s ability to conduct examinations or if it

believes that the inspection process will not produce the necessary insights into the quality of

the licensee’s program. During FY 2002, the staff did not conduct any "for cause"

requalification examinations. 

The staff conducted biennial licensed operator requalification inspections at 47 power reactor

facilities during FY 2002 and identified several findings (discussed in the following section). As

shown in the table below, the staff continues to believe that the power reactor facilities

inspected are satisfactorily maintaining their licensed operator requalification training programs.
-3-

NRC Requalification Program Evaluation Results for Fiscal Years 1994 through 2002

1994 1995 1996 1997 1998 1999 2000 2001 2002

No. of programs evaluated 

 

43 58 41 41 32 40 41 36 47

No. of satisfactory /

unsatisfactory programs*

43/0 58/0 41/0 41/0 32/0 40/0 41/0 36/0 47/0

Percent satisfactory 100 100 100 100 100 100 100 100 100

*A program rating of satisfactory indicates that the licensee’s requalification program, with limited exceptions,

complied with the requirements of 10 CFR 55.53 and 55.59 for the areas inspected, and the staff did not

administer requalification examinations as a result of any findings.

Operator Requalification Findings

During this reporting period, the staff made one yellow, one white, and three green findings on

operator requalification.

The yellow finding was at Indian Point, Unit 2, where four of seven crews failed the dynamic

simulator examination in October 2001. Crews received remedial training and were reexamined

before resuming licensed duties. The staff conducted a supplemental inspection per IP 95001,

"Inspection for One or Two White Inputs in a Strategic Performance Area," and another IP

71111.11 requalification inspection in October 2002. The staff has noted substantial

improvement but the yellow finding remains open until the staff determines that the licensee’s

corrective actions have effectively addressed the underlying causes of the yellow finding. 

The white finding involved an examination compromise at the Cooper plant resulting from an

improper validation process for the July 2000 biennial written requalification examinations. The

staff identified the issue in the fall of calendar year 2001. After the examinations were

regraded, two operators were found to have failed the examination. In May 2002, the staff

conducted a supplemental inspection per IP 95001 and concluded that the licensee had

satisfactorily addressed all of the requalification issues associated with the finding and

implemented all required corrective actions. 

The green findings were as follows:

1. A Non-Cited violation of 10 CFR 55.53(f)(2) was identified at the Calvert Cliffs plant

regarding the licensee's methods and standards used to reactivate operator licenses

to support refueling outages.

2. Two of nine crews failed the requalification operating test at the LaSalle plant (the

crews were remediated and reexamined). 

3. Two of ten crews failed the dynamic simulator portion of the licensed operator

requalification examination at Millstone, Unit 3 (the crews were remediated and

reexamined).
-4-

Three other potential green findings have not yet been resolved, one on simulator fidelity and

two others on inadequate record keeping. At the Kewaunee plant, two potential simulator

issues have been identified concerning the failure to comply with 10 CFR 55.46, "Simulation

Facilities." The first issue involves the licensee’s use of the simulator to meet experience

requirements for applicants for initial operator and senior operator licenses in accordance with

10 CFR 55.46 (c)(2)(i). The second issue concerns the licensee’s program for correcting

simulator modeling and hardware discrepancies and discrepancies identified from performance

testing in accordance with 10 CFR 55.46(d)(2).

In addition, one significant requalification finding was identified at the Dresden plant. On

July 1, 2002, the licensee reported to the NRC that due to an oversight, Dresden had not

conducted a comprehensive biennial requalification written examination during the last

requalification program cycle, contrary to 10 CFR 55.59(c). The licensee administered the

required exam, albeit approximately 6 months late. The NRC issued the licensee an exemption

to allow the Dresden licensed operator requalification program to resume a proper examination

schedule. The NRC also issued 47 notices of enforcement discretion to individual operators

who were not administered a comprehensive written examination within the required time

period. The determination of the proper enforcement action for Dresden is still pending. The

NRC has discussed the issues related to scheduling of requalification examinations at two

public meetings with the Nuclear Energy Institute’s (NEI’s) operator licensing focus group in

August and October 2002. The NRC and facility licensees are reexamining licensed operator

requalification training programs for compliance with the scheduling requirements in

10 CFR Part 55. As of the end of fiscal year 2002, no additional requalification schedule

problems had been identified. The NRC plans to inform facility licensees of this issue by a

regulatory information summary in 2003.

The requalification findings identified in FY 2002 were restricted to a few facilities, which

indicates that these issues are not generic industry problems.

Operator Requalification Human Performance Significant Determination Process (ORSDP)

In December 2000, the staff started using the ORSDP (which consists of a logic flowchart and

matrix) to determine the risk importance of issues identified during requalification inspections.

Experience to date with application of the ORSDP indicates that it is easily understood and

objective, with clear criteria for categorization of the findings. The staff believes that the colors

assigned by the ORSDP to the Operator Requalification findings identified in FY2002

appropriately reflect the safety significance of the issues involved. Thus the ORSDP has

proven to be an effective tool for ensuring consistent, appropriate regulatory treatment of

identified operator requalification issues. Experience with the ORSDP has resulted in some

minor modifications (implemented on March 27, 2002) to improve clarity and ensure

consistency with applicable regulations, including the recent simulator rule. 

Summary of Initial Examination Results

The staff continues to administer initial licensing examinations to applicants for RO and SRO

licenses at power and non-power reactor facilities. During FY 2002, the staff administered

approximately 50 site-specific initial licensing examinations to 370 applicants (163 ROs and 

207 SROs) at power reactor facilities. This number includes site-specific licensing
-5-

examinations for 291 candidates that were prepared, in whole or in part, by facility licensees in

accordance with the NRC’s examination guidance in NUREG-1021, "Operator Licensing

Examination Standards for Power Reactors." The NRC staff reviews and approves all facilityprepared examinations prior to administration, and NRC certified examiners administer all of the

operating tests. In addition to the facility-prepared examinations that were administered to 79

percent of the applicants (compared to 68 percent in FY 2001), the staff administered

NRC-prepared examinations to 21 percent of the applicants (compared to 32 percent in

FY 2001). This reflects an improving trend, and the ratio of NRC-prepared and licenseeprepared examinations nationwide aligns well with budgetary planning assumptions.

The following table summarizes the power reactor initial operator licensing examination results

from FY 1998 through FY 2002; the results for NRC-prepared and facility-prepared

examinations are listed separately. These results indicate that initial operator training programs

at power reactors continue to produce a large number of applicants who pass the operator

licensing examinations, regardless of whether the examinations were prepared by the NRC or

by the facility licensees. 

Power Reactor Initial Examination Results

Examination

Percentage of Applicants Who Passed During the Fiscal Year

1998 1999 2000 2001 2002

Exam Prepared Exam Prepared Exam Prepared Exam Prepared Exam Prepared

NRC Facility NRC Facility NRC Facility NRC Facility NRC Facility

RO

Written N/A 89 100 89 98 95 96 86 97 98

Operating N/A 99 100 93 100 98 99 100 100 98

SRO

Written 100 96 100 94 100 95 99 95 98 97

Operating 94 96 100 98 96 99 100 97 100 96

The overall quality and consistency of facility-prepared examinations is good, and shows an

improving trend. During FY 2001, the staff noted that consistency in examination quality

among licensees was a problem. However, the percentage of exams with a high number of

post-exam comments, which adversely reflects on examination quality, decreased from 17

percent in FY 2001 to 6 percent in FY2002 and the average number of unacceptable written

examination questions submitted by facility licensees decreased slightly (approximately 2

percent per exam on average). Also, consistency was good between the average score on

facility-prepared written examinations (~87.6 percent) and the average score (~86.9 percent)

for NRC-developed written exams. Notwithstanding the improved performance noted above, a

few licensees submitted poor exams. There have also been isolated instances of facilityprepared examinations with a problematically high number of post-examination comments. 
-6-

The staff continues to seek ways to improve the quality and consistency of facility- and NRCprepared examinations, while reducing unnecessary burden and maintaining the integrity and

reliability of the examinations. Examples of program improvements are given in the next

section. 

The following table indicates the total number of applicants who requested that NRC review 

their examination results from FY 1996 through FY 2002. During FY 2002, the staff noted a

negligible increase in the number of proposed applicant denials overturned by the review

process. One denial was overturned because the applicant provided new information that the

NRC and the facility licensee had not considered during the pre- or post-examination reviews. 

The second overturned denial involved an NRC grading error on an operating test that was

corrected immediately upon notification by the applicant. The staff believes that the low

number of overturned denials is the result of the staff’s careful pre- and post-examination

reviews of examination test items. In FY 2003 to date, the staff has received four applicant

requests for NRC review of proposed license denials. 

Power Reactor Initial Examination Denial Results

1996 1997 1998 1999 2000 2001 2002

No. of proposed

applicant denials

47 44 41 60 19 35 19

No. of applicantrequested reviews12 13 20 16 9 4 3

No. of final denials 44 35 28 47 16 34 17

No. of licenses issued

upon review

3 9 13 13 3 1 2

The following table summarizes the results of the non-power reactor initial operator licensing

examinations from FY 1998 through FY 2002. During FY 2002, the staff administered

approximately 31 site-specific initial licensing examinations to RO and SRO applicants at nonpower reactor facilities (compared to 28 in FY 2001) in accordance with the current examination

guidance in NUREG-1478, "Non-Power Reactor Operator Licensing Examiner Standards."
-7-

Non-Power Reactor Initial Examination Results

Examination

Percentage of Applicants Who Passed During the Fiscal

Year

1998 1999 2000 2001 2002

RO

Written 87 63 78 94 85

Operating 100 96 89 98 92

SRO

Written 94 100 82 89 100

Operating 100 100 100 100 93

The non-power initial operator licensing program has not been modified and remains consistent

and stable. The results continue to indicate that training programs for non-power reactor facility

operators generally produce applicants who pass the NRC’s licensing examinations. 

Operator Licensing Program Initiatives

During FY 2002 and FY 2003 to date, the NRC has continued its efforts to improve and support

the oversight of the operator licensing program and respond to stakeholder concerns. For

example: 

1. The staff issued for public comment and voluntary trial use a draft Revision 9 to

NUREG-1021. The proposed draft revision incorporates a number of

enhancements developed over the last two years in cooperation with NEI’s

operator licensing focus group. The proposed changes are to (a) reduce the

length of the reactor operator written examination (in a manner that reduces

burden in examination development with no expected reduction in examination

reliability); (b) clarify and simplify the design of the senior reactor operator written

examination; (c) better risk-inform the reactor operator and senior operator written

examinations; (d) better balance the administrative and plant systems portions of

the walk-through operating test; (e) clarify the grading criteria for the simulator

operating test; and (f) incorporate guidance on peer checks and the suppression

of inappropriate knowledge and ability catalog statements. 

2. The staff conducted a training conference at the NRC’s Region II office for all

operator licensing examiners in the second quarter of FY 2003 to ensure that

NRC examiners continue to receive appropriate training and policy direction from

senior executives and managers, to discuss pertinent topics, and to provide
-8-

feedback to the Office of Nuclear Reactor Regulation (NRR). The conference is

an effective tool for promoting consistency in the operator licensing program.

3. The staff sponsored regional workshops with facility licensees, and attended

several meetings of industry groups, such as the Middle Atlantic Nuclear Training

Group (MANTG), the Southeastern States Nuclear Training Association (SSNTA),

and NEI to promote dialog and feedback on the proposed future changes to

examination standards included in Revision 9 of NUREG-1021. The staff

participated in a 2-day industry-sponsored national workshop in February 2003

focusing on proposed Revision 9 changes to the operator licensing examination

standards and recent changes to the requalification inspection baseline

procedure, IP 71111.11, which implements the simulator rule. 

CONCLUSION:

The NRC’s licensed operator requalification inspection program continues to effectively ensure

that those individuals who are licensed to operate or supervise the reactor controls maintain the

required level of competence to safely perform their licensed duties. The NRC’s initial operator

licensing examination program continues to provide reasonable assurance that only those

applicants who have mastered the knowledge, skills, and abilities required to safely operate and

supervise the reactor controls are being licensed to do so. 

RECOMMENDATION:

The FY 2001 report noted that the consistency in examination quality among licensees and

examination developers remained an area for improvement, in part because of the turnover of

experienced facility licensee examination developers. However, based on a recently conducted

study by the staff, overall examination quality and consistency during FY 2002 appeared to be

good, with indications of an improving trend. A few licensees submitted problematic

examinations to the NRC. 

In view of the stabilization achieved in the initial operator licensing examination and the operator

requalification programs, the staff recommends that this report be discontinued. This would

result in a staff resource savings of approximately 0.3 FTE per year. The initial operator

licensing process has reached an acceptable level of maturity as demonstrated by (1) the good

consistency obtained in results between facility-prepared and NRC-prepared examinations, and

(2) the good overall quality of facility-prepared examinations (with a few exceptions). In regard

to proposed Revision 9 changes to the examination standards (NUREG-1021), the staff will

continue to work with affected stake-holders to identify, pilot, and implement the program

enhancements that improve efficiency and effectiveness or reduce unnecessary regulatory

burden provided the enhancements preserve requisite examination reliability. Similarly, the

maturity of the operator requalification programs at licensee facilities is indicated by (1) the

satisfactory results of NRC requalification inspections, (2) the effectiveness of the

requalification inspection process and ORSDP in identifying issues and ensuring their

consistent and appropriate regulatory treatment, and (3) the fact that the issues that have been 
-9-

identified are not generic industry problems. If unexpected problems arise during the piloting of

Revision 9 to the examination standards or during implementation of the simulator rule, the staff

will inform the Commission of the issues and corrective actions. 

Staff requests action within 10 days. Action will not be taken until the SRM is received. We

consider this action to be within the delegated authority of the EDO.

/RA/

William D. Travers

Executive Director 

 for Operations
-9-

identified are not generic industry problems. If unexpected problems arise during the piloting of

Revision 9 to the examination standards or during implementation of the simulator rule, the staff

will inform the Commission of the issues and corrective actions. 

Staff requests action within 10 days. Action will not be taken until the SRM is received. We

consider this action to be within the delegated authority of the EDO.

/RA/

William D. Travers

Executive Director 

 for Operations

ACCESSION #ML030420087

*See previous concurrence

To receive a copy of this document, indicate in the box: "C" = Copy without enclosures "E" = Copy with enclosures "N" = No copy

OFFICE IOHS/DIPM IOHS/IEHB IEHB/DIPM TECH ED DIR/DIPM

NAME LVick:klr/kig* DTrimble* TQuay* P.Kleene* BBoger

(CCarpenter for)

DATE 02/04/2003 02/12/2003 02/12/2003 02/06 /2003 02/20/2003

OFFICE ADIP/NRR NRR EDO

NAME RWBorchardt SCollins

(RBorchardt for)

WTravers

DATE 02/26/2003 03/03/2003 03/12/2003

OFFICIAL RECORD COPY