Operator Licensing

Examination Standards

for Power Reactors 

Draft Report for Comment

Office of Nuclear Reactor Regulation

NUREG-1021, Rev. 12
AVAILABILITY OF REFERENCE MATERIALS

IN NRC PUBLICATIONS

NRC Reference Material

As of November 1999, you may electronically access 

NUREG-series publications and other NRC records at the

NRC’s Library at www.nrc.gov/reading-rm.html. Publicly 

released records include, to name a few, NUREG-series 

publications; Federal Register notices; applicant, licensee, 

and vendor documents and correspondence; NRC 

correspondence and internal memoranda; bulletins and 

information notices; inspection and investigative reports; 

licensee event reports; and Commission papers and their 

attachments.

NRC publications in the NUREG series, NRC regulations, 

and Title 10, “Energy,” in the Code of Federal Regulations

may also be purchased from one of these two sources:

1. The Superintendent of Documents

U.S. Government Publishing Office

Washington, DC 20402-0001

Internet: www.bookstore.gpo.gov

Telephone: (202) 512-1800

Fax: (202) 512-2104

2. The National Technical Information Service

5301 Shawnee Road

Alexandria, VA 22312-0002

Internet: www.ntis.gov

1-800-553-6847 or, locally, (703) 605-6000

A single copy of each NRC draft report for comment is

available free, to the extent of supply, upon written

request as follows:

Address: U.S. Nuclear Regulatory Commission

Office of Administration

Multimedia, Graphics, and Storage &

Distribution Branch 

Washington, DC 20555-0001 

E-mail: distribution.resource@nrc.gov

Facsimile: (301) 415-2289

Some publications in the NUREG series that are posted 

at the NRC’s Web site address www.nrc.gov/reading-rm/

doc-collections/nuregs are updated periodically and may 

differ from the last printed version. Although references to 

material found on a Web site bear the date the material 

was accessed, the material available on the date cited 

may subsequently be removed from the site.

Non-NRC Reference Material

Documents available from public and special technical 

libraries include all open literature items, such as books, 

journal articles, transactions, Federal Register notices, 

Federal and State legislation, and congressional reports. 

Such documents as theses, dissertations, foreign reports 

and translations, and non-NRC conference proceedings 

may be purchased from their sponsoring organization.

Copies of industry codes and standards used in a

substantive manner in the NRC regulatory process are 

maintained at—

The NRC Technical Library

Two White Flint North

11545 Rockville Pike

Rockville, MD 20852-2738

These standards are available in the library for reference 

use by the public. Codes and standards are usually 

copyrighted and may be purchased from the originating 

organization or, if they are American National Standards, 

from—

American National Standards Institute

11 West 42nd Street

New York, NY 10036-8002

Internet: www.ansi.org

(212) 642-4900

Legally binding regulatory requirements are stated only in 

laws; NRC regulations; licenses, including technical 

specifications; or orders, not in NUREG-series publications. 

The views expressed in contractor prepared publications in 

this series are not necessarily those of the NRC.

The NUREG series comprises (1) technical and 

administrative reports and books prepared by the staff 

(NUREG–XXXX) or agency contractors (NUREG/CR–XXXX), 

(2) proceedings of conferences (NUREG/CP–XXXX),

(3) reports resulting from international agreements

(NUREG/IA–XXXX),(4) brochures (NUREG/BR–XXXX), and

(5) compilations of legal decisions and orders of the

Commission and the Atomic and Safety Licensing Boards

and of Directors’ decisions under Section 2.206 of the

NRC’s regulations (NUREG–0750).

DISCLAIMER: This report was prepared as an account 

of work sponsored by an agency of the U.S. Government. 

Neither the U.S. Government nor any agency thereof, nor 

any employee, makes any warranty, expressed or implied, 

or assumes any legal liability or responsibility for any third 

party’s use, or the results of such use, of any information, 

apparatus, product, or process disclosed in this publication, 

or represents that its use by such third party would not 

infringe privately owned rights.
Operator Licensing

Examination Standards

for Power Reactors

Draft Report for Comment

Manuscript Completed: October 2020

Date Published: November 2020

Prepared by: 

M. Scheetz

Office of Nuclear Reactor Regulation

NUREG-1021, Rev. 12
iii

ABSTRACT

The U.S. Nuclear Regulatory Commission (NRC) publishes NUREG-1021, “Operator Licensing 

Examination Standards for Power Reactors,” to establish the policies, procedures, and practices 

for examining licensees and applicants for reactor operator and senior reactor operator licenses 

at power reactor facilities in accordance with Title 10 of the Code of Federal Regulations

(10 CFR) Part 55, “Operators’ licenses.”

The agency intends these examination standards to help NRC examiners and facility licensees 

better understand the processes associated with initial and NRC-conducted requalification 

examinations. The standards also ensure the equitable and consistent administration of 

examinations for all applicants. As stated in 10 CFR 55.40, “Implementation,” “[t]he 

Commission shall use the criteria in NUREG-1021...in effect 6 months before the examination 

date to prepare the written examinations required by [10 CFR] 55.41 and [10 CFR] 55.43 and 

the operating tests required by [10 CFR] 55.45. The Commission shall also use the criteria in 

NUREG-1021 to evaluate the written examinations and operating tests prepared by power 

reactor facility licensees pursuant to paragraph (b) of this section.”

The NRC is issuing Revision 12 of NUREG-1021, which is a major rewrite of the examination 

standards, to (1) streamline information into topic-based sections for ease of use, (2) clarify 

instructions for the identification and grading of performance deficiencies on the operating test, 

and (3) introduce new instructions for the treatment of critical tasks and critical and significant 

performance deficiencies.

NUREG-1021, Revision 11, is a rule as defined in the Congressional Review Act 

(5 U.S.C. 801-808). However, the Office of Management and Budget has not found it to be 

a major rule as defined in that act. The NRC staff will submit NUREG-1021, Revision 12 

to the Office of Management and Budget for review prior to finalizing the revision.
iv

PAPERWORK REDUCTION ACT STATEMENT

This NUREG provides voluntary guidance for implementing the mandatory information 

collections in 10 CFR Part 55 that are subject to the Paperwork Reduction Act of 1995 (44 

U.S.C. 3501 et. seq.). This information collection was approved by the Office of Management 

and Budget (OMB), approval number 3150-0018. Send comments regarding this information 

collection to the FOIA, Library, and Information Collections Branch (T6-A10M), U.S. Nuclear 

Regulatory Commission, Washington, DC 20555 0001, or by e mail to 

Infocollects.Resource@nrc.gov, and to the OMB reviewer at: OMB Office of Information and 

Regulatory Affairs (3150-0018), Attn: Desk Officer for the Nuclear Regulatory Commission, 725 

17th Street, NW Washington, DC 20503; e mail: oira_submission@omb.eop.gov.

PUBLIC PROTECTION NOTIFICATION

The NRC may not conduct or sponsor, and a person is not required to respond to, a collection 

of information unless the document requesting or requiring the collection displays a currently 

valid OMB control number.

 
v

TABLE OF CONTENTS

ABSTRACT ................................................................................................................... iii

BACKFITTING AND ISSUE FINALITY ........................................................................ vii

ABBREVIATIONS AND ACRONYMS........................................................................... ix

1 GENERAL

1.1 Using the Operator Licensing Examination Standards....................................1.1-1

1.2 Guidelines for Taking NRC Examinations .......................................................1.2-1

1.3 Examination Security.......................................................................................1.3-1

2 INITIAL PREEXAMINATION ACTIVITIES

2.1 Preparing for Operator Licensing Initial Examinations ....................................2.1-1

2.2 Applications, Medical Requirements, and Waiver and Excusal of Examination 

and Test Requirements ...................................................................................2.2-1

2.3 Reviewing and Approving Operator Licensing Initial Examinations ................2.3-1

3 INITIAL OPERATING TESTS

3.1 Overview of the Operating Test for Operator Licensing Initial Examinations ..3.1-1

3.2 Developing Job Performance Measures .........................................................3.2-1

3.3 Testing Guidelines for Dynamic Simulator Scenarios .....................................3.3-1

3.4 Developing Scenarios .....................................................................................3.4-1

3.5 Administering Operating Tests ........................................................................3.5-1

3.6 Grading and Documenting Operating Tests for Operator Licensing Initial 

Examinations...................................................................................................3.6-1

3.7 Alternatives for In-Plant Job Performance Measures at Plants under 

Construction ....................................................................................................3.7-1

4 INITIAL WRITTEN EXAMINATIONS

4.1 Developing Written Examination Outlines .......................................................4.1-1

4.2 Developing Written Examinations....................................................................4.2-1

4.3 Administering Written Examinations................................................................4.3-1

4.4 Grading and Documenting the Written Examinations......................................4.4-1

5 INITIAL POSTEXAMINATION ACTIVITIES AND OTHER LICENSING ACTIONS

5.1 Issuing Operator Licenses and Postexamination Activities.............................5.1-1

5.2 Application Denials and Requests for Informal NRC Staff Review..................5.2-1

5.3 Maintaining, Changing, and Renewing Operator Licenses .............................5.3-1

6 NRC-CONDUCTED REQUALIFICATION EXAMINATIONS

6.1 Conducting NRC Requalification Examinations ..............................................6.1-1

6.2 Requalification Written Examinations..............................................................6.2-1

6.3 Requalification Walkthrough Examinations .....................................................6.3-1

6.4 Requalification Dynamic Simulator Examinations ...........................................6.4-1

7 FUEL HANDLING EXAMINATIONS

7.1 Administration of Initial Examinations for Senior Operators Limited to Fuel 

Handling ..........................................................................................................7.1-1
vi

7.2 Administration of NRC Requalification Examinations for Senior Reactor 

Operators Limited to Fuel Handling................................................................. 7.2-1 

8 GLOSSARY ...................................................................................................... 8-1

Appendix A Overview of Generic Examination Concepts..................................... A-1

Appendix B Examples of Written Examination Questions.................................... B-1

vii

BACKFITTING AND ISSUE FINALITY 

In NUREG-1021, the U.S. Nuclear Regulatory Commission (NRC) establishes the policies, 

procedures, and practices for examining licensees and applicants for reactor operator and 

senior reactor operator licenses at nuclear power reactor facilities under Title 10 of the Code of 

Federal Regulations (10 CFR) Part 55, “Operators’ licenses.” Revision 12 of NUREG-1021 

(1) streamlines information into topic-based sections for ease of use, (2) clarifies instructions for 

the identification and grading of performance deficiencies on the operating test, and 

(3) introduces new instructions for the treatment of critical tasks and critical and significant 

performance deficiencies.

Revision 12 of NUREG-1021 does not represent “backfitting” as that term is defined in 

10 CFR 50.109, “Backfitting,” and is not inconsistent with the issue finality provisions in 

10 CFR Part 52, “Licenses, certifications, and approvals for nuclear power plants.” Current 

holders of operating licenses under 10 CFR Part 50, “Domestic licensing of production and 

utilization facilities,” or combined licenses under 10 CFR Part 52 are not mandated to prepare 

the written examinations required by 10 CFR 55.41, “Written examination: Operators,” and 

10 CFR 55.43, “Written examination: Senior operators,” and the operating tests required by 

10 CFR 55.45, “Operating tests,” which must be prepared using the criteria in NUREG-1021 in 

effect 6 months before the examination date.0F

1 Because licensees under 10 CFR Part 50 and 

10 CFR Part 52 are not required to prepare the 10 CFR 55.41 and 10 CFR 55.43 examinations 

and 10 CFR 55.45 tests, changes to the criteria used to prepare the examinations and tests are 

not imposed upon them; therefore, these changes do not meet the definition of “backfitting” in 

10 CFR 50.109 and are not inconsistent with the finality provisions in 10 CFR Part 52.

Furthermore, licensees know to expect changes to NUREG-1021. The inherent structure of the 

testing regime in 10 CFR Part 55 involves updating NUREG-1021 to reflect lessons learned and 

ensure uniform conditions for licensing individuals applying for operator licenses. The language 

in 10 CFR 55.40(a)–(b) illustrates that the NRC would make changes to NUREG-1021. By 

referencing the version of NUREG-1021 that would be “in effect 6 months before the 

examination date,” the NRC shows that it anticipates that the guidance would be revised and 

that it could be revised within 6 months of the examination date. Although 10 CFR 55.40, 

“Implementation,” went into effect only in 1999,1F

2 the NRC has published revisions to 

NUREG-1021 since October 1983. Because current facility licensees under 10 CFR Part 50

have known of, and have experienced, this regime for more than three decades and because 

current facility licensees under 10 CFR Part 52 were aware of this regime at the time they 

received their respective licenses, their regulatory expectations include the possibility of 

changes. Thus, the policies underlying 10 CFR 50.109 and the issue finality provisions in

10 CFR Part 52, and the backfitting and issue finality provisions themselves, do not apply to 

these licensees.

Backfitting or issue finality regulations do not appear in 10 CFR Part 55, and the backfitting 

provisions in 10 CFR 50.109 and the finality provisions in 10 CFR Part 52 do not protect power 

reactor operator licensees because neither 10 CFR 50.109 nor 10 CFR Part 52 applies to power 

reactor operator licensees under 10 CFR Part 55.

1 See 10 CFR 55.40(a)–(b).

2 See “Initial Licensed Operator Examination Requirements; Final Rule,” in Volume 64, Number 78, of the 

Federal Register, pages 19868–19878, dated April 23, 1999.
viii

Revision 12 of NUREG-1021 could be applied to applications for 10 CFR Part 50 operating 

licenses, 10 CFR Part 52 combined licenses, or 10 CFR Part 55 operator licenses. Such action

would not constitute backfitting, as defined in 10 CFR 50.109, or would not otherwise be

inconsistent with the applicable issue finality provisions in 10 CFR Part 52, because such 

applicants are not within the scope of entities protected by 10 CFR 50.109 or by the relevant 

issue finality provisions in 10 CFR Part 52. The exception to this principle is a combined license 

applicant under 10 CFR Part 52 that references an already issued design certification or early 

site permit, but this exception does not apply to the requirements in 10 CFR Part 55.
ix

ABBREVIATIONS AND ACRONYMS

ABWR advanced boiling-water reactor

AC alternating current

ADAMS Agencywide Documents Access and Management System (NRC)

ADM administrative JPM

ADS automatic depressurization system

AFW auxiliary feedwater

ANS American Nuclear Society

ANSI American National Standards Institute

AO auxiliary operator

AOP abnormal operating procedure

AP1000® Westinghouse AP1000® pressurized-water reactor

APE abnormal plant evolution

APRM average power range monitor

ARP alarm (or annunciator) response procedure

ATC at the controls (operator)

ATWS anticipated transient without scram

BOP balance of plant (operator)

BW Babcock and Wilcox pressurized-water reactor

BWR boiling-water reactor

C Celsius

CAL confirmatory action letter

CCW component cooling water

CD-ROM compact disk, read-only memory

CE Combustion Engineering 

CFR Code of Federal Regulations

COL combined license

CPD critical performance deficiency

CRD control rod drive

CS core spray

CSF critical safety function

CT critical task

CTMT containment

CVCS chemical and volume control system

DAS dominant accident sequence

DC direct current

DG diesel generator

E/APE emergency/abnormal plant evolution

ECA emergency contingency action (procedure)

ECCS emergency core cooling system

EDG emergency diesel generator

EOP emergency operating procedure

EPIP emergency plan implementing procedure

ES examination standard

ESF engineered safety feature

ESFAS engineered safety feature actuation system
x

F Fahrenheit

FR Federal Register

GE General Electric boiling-water reactor

gpm gallon(s) per minute

GUI graphical user interface

HCL higher cognitive level

HPCI high-pressure coolant injection

HPCS high-pressure core spray

HPSI high-pressure safety injection

HVAC heating, ventilation, and air conditioning

IC initial condition

I/C instrumentation and control

ID identification

INPO Institute of Nuclear Power Operations

IP inspection procedure

IPE individual plant examination

IR importance rating

IRM intermediate range monitor

JPM job performance measure

JTA job task analysis

K/A knowledge and ability

LAN local area network

LCO limiting condition for operation

LER licensee event report

LOCA loss-of-coolant accident

LOD level of difficulty

LOK level of knowledge

LOOP loss of offsite power

LPCI low-pressure coolant injection

LPCS low-pressure core spray

LPRM local power range monitor

LSRO limited senior reactor operator (senior operator limited to fuel handling)

mmHg millimeter of mercury

MCR main control room

MSIV main steam isolation valve

N/A not applicable

NANT National Academy for Nuclear Training

NEI Nuclear Energy Institute

NNAB National Nuclear Accrediting Board

N/O not observed

NRC U.S. Nuclear Regulatory Commission

NRR Office of Nuclear Reactor Regulation (NRC)
xi

NUREG NRC technical report designation

ODCM offsite dose calculation manual

OLMC operator licensing manual chapter

OMB Office of Management and Budget (U.S.)

OPLS offsite power low signal

PCIS primary containment isolation system

PCS pressure control system

PORV power-operated relief valve

PRA probabilistic risk assessment

PRT pressurized relief tank

PRTS pressurizer relief tank system

psi pounds per square inch

psia pounds per square inch, absolute

psig pounds per square inch, gauge

PWR pressurized-water reactor

PZR pressurizer

RBMS rod block monitor system

RCIC reactor core isolation cooling

RCP reactor coolant pump

RCS reactor coolant system

RF rating factor

RG regulatory guide (NRC)

RHR residual heat removal

RMCS reactor manual control system

RO reactor operator

RPIS rod position indication system

RPS reactor protection system

RPV reactor pressure vessel

RWST refueling water storage tank

S, SAT satisfactory

SAT systems approach to training 

SG specific gravity

SGTR steam generator tube rupture

SI safety injection

SIM simulator

SLC standby liquid control

SPD significant performance deficiency

SR surveillance requirement

SRO senior reactor operator

SRO-I senior reactor operator-instant

SRO-only senior reactor operator-only

SRO-U senior reactor operator-upgrade

SRV safety relief valve

STA shift technical advisor

TDAFWP turbine-driven auxiliary feedwater pump

T/F true-false (statement/question)
xii

TPA temporary plant alteration

TRM technical requirements manual

TS technical specification(s)

U, UNSAT unsatisfactory

UPS uninterruptible power supply

U.S.C. United States Code

V volt(s) 

W Westinghouse

W/T in plant systems walk-through or JPM
ES-1.1, Page 1 of 2

1 1.1 USING THE OPERATOR LICENSING EXAMINATION 

2 STANDARDS

3 A. Purpose of the Examination Standards

4 Title 10 of the Code of Federal Regulations (10 CFR) Part 55, “Operators’ licenses,” establishes

5 procedures and criteria for the issuance of licenses to reactor operators (ROs) and senior

6 reactor operators (SROs) of utilization facilities, provides for the terms and conditions upon 

7 which the Commission will issue or modify these licenses, and provides for the terms and 

8 conditions to maintain and renew these licenses. 

9 The regulations in 10 CFR 55.33, “Disposition of an initial application,” states that the 

10 Commission will approve an initial application for a license if the Commission finds, among other 

11 things, that the applicant has passed the requisite written examination and operating test. The 

12 regulations mandate that the licensing examinations must be developed and administered in 

13 accordance with 10 CFR 55.41, “Written examination: Operators,” and 10 CFR 55.45, 

14 “Operating tests,” for ROs or 10 CFR 55.43, “Written examination: Senior operators,” and 

15 10 CFR 55.45 for SROs. The regulation at 10 CFR 55.40(a) states the following: 

16 The Commission shall use the criteria in NUREG-1021, “Operator Licensing 

17 Examination Standards for Power Reactors,” in effect six months before the 

18 examination date to prepare the written examinations required by [10 CFR] 55.41 

19 and 55.43 and the operating tests required by [10 CFR] 55.45. The Commission 

20 shall also use the criteria in NUREG-1021 to evaluate the written examinations 

21 and operating tests prepared by power reactor facility licensees.... 

22 Pursuant to 10 CFR 55.40(b), power reactor facility licensees may prepare, proctor, and grade 

23 the written examinations required by 10 CFR 55.41 and 10 CFR 55.43 and may prepare the 

24 operating tests required by 10 CFR 55.45 as long as they prepare the required examinations 

25 and tests in accordance with the criteria in NUREG-1021.

26 NUREG-1021 establishes the policies, procedures, and guidance for the development, 

27 administration, and grading of written examinations and operating tests used for examining RO 

28 and SRO licensees and applicants at power reactor facilities. It also provides procedures and 

29 guidance for maintaining operators’ licenses and for the U.S. Nuclear Regulatory Commission 

30 (NRC) to conduct requalification examinations when necessary. These standards describe the 

31 provisions of the Atomic Energy Act of 1954, as amended, and the regulations on which the 

32 operator licensing program is based. They also ensure the equitable and consistent 

33 administration of examinations to all applicants and licensed operators at all licensee facilities 

34 that are subject to the regulations. 

35 B. Format

36 Each section of the examination standards (ES) explains the policies, procedures (referred to as 

37 instructions), and guidance (or guidelines) for an element of the operator licensing program. For 

38 ease of reference, each standard is assigned a single-digit number, and related standards are 

39 grouped together such that standards beginning with the same digit apply to related aspects of 

40 the program, as follows:

41 ES-1.X: General

42 ES-2.X: Initial Preexamination Activities
ES-1.1, Page 2 of 2

1 ES-3.X: Initial Operating Tests

2 ES-4.X: Initial Written Examinations

3 ES-5.X: Initial Postexamination Activities and Other Licensing Actions

4 ES-6.X: NRC-Conducted Requalification Examinations

5 ES-7.X: Fuel Handling Examinations

6 ES-8 Glossary

7 The appendices to NUREG-1021 provide additional guidance and background information that 

8 users of NUREG-1021 may find helpful.

9 C. How To Use

10 The instructions in NUREG-1021 for developing operator licensing initial examinations are to be 

11 used in conjunction with the applicable knowledge and ability (K/A) catalogs, which are vendor 

12 or technology specific. The NRC has made a complete list of the applicable K/A catalogs 

13 available on the NRC’s Operator Licensing public Web site

14 (https://www.nrc.gov/reactors/operator-licensing.html). The latest revision of the applicable K/A 

15 catalog available at the time the facility licensee requests the written examination outline (up to 

16 18 months before the examination date based on typical initial operator licensing class length)

17 should be used.

18 NUREG-1021 contains procedures and guidance for use by both NRC and facility licensee staff

19 and operator license applicants. Procedures must be followed and are in the form of 

20 instructions and restrictions. Guidance and guidelines are based on best practices for 

21 examinations and should be followed. 

22 The instructions within an examination standard do not have to be performed in the exact order 

23 listed, and some of the steps may be performed multiple times if necessary. The NRC regional 

24 office must consult the Office of Nuclear Reactor Regulation (NRR) operator licensing program 

25 office if the instructions in NUREG-1021 cannot be met. The NRC regional office must also 

26 obtain program office approval before undertaking any initiative that could undermine 

27 examination consistency among the regions.

28 All users of NUREG-1021 should be familiar with the terms in the glossary (ES-8) to use the 

29 examination standards properly.

30 The format of the sample letters in NUREG-1021 can be adjusted as necessary.

31 Sections in NUREG-1021 reference guidance documents that are used in conjunction with 

32 these examination standards. Regulatory guides, NUREG-series reports (with the exception of 

33 the applicable K/A catalogs), industry guidance documents, and industry standards do not 

34 constitute requirements, except as specified in Commission orders or as committed to by the 

35 facility licensee. NRC examiners and licensees should consult the appropriate revisions of 

36 these guidance documents as referenced in each facility’s final safety analysis report or 

37 approved training program, as applicable. 

38 The NRC has posted a comprehensive list of regulations and guidance documents applicable 

39 to operator licensing on the NRC’s Operator Licensing public Web site 

40 (https://www.nrc.gov/reactors/operator-licensing.html).
ES-1.2, Page 1 of 6

1 1.2 GUIDELINES FOR TAKING NRC EXAMINATIONS

2 Facility licensees are encouraged to familiarize their applicants and examinees with these 

3 guidelines and distribute a copy of these guidelines to every applicant or examinee before the 

4 examination begins. All items apply to both initial and requalification examinations, except as 

5 noted.

6 A. General Guidelines

7 1. [Read Verbatim] Cheating on any part of the examination will result in a denial of your 

8 application or action against your license.

9

10 2. If you have any questions concerning the administration of any part of the examination, 

11 do not hesitate to ask them before starting that part of the test.

12

13 3. Senior reactor operator (SRO) applicants will be tested at the level of responsibility of 

14 the senior licensed shift position (i.e., shift supervisor, senior shift supervisor, or 

15 whatever the title of the position may be).

16

17 4. You must pass every part of the examination to receive a license or to continue 

18 performing licensed duties. Any applicant who demonstrates deficiencies in required 

19 knowledge and abilities during the examination may require additional training before

20 performing licensed operator duties. 

21

22 5. The U.S. Nuclear Regulatory Commission (NRC) examiner is not allowed to reveal the 

23 results of any part of the examination until NRC management has reviewed and 

24 approved them. Grades provided by the facility licensee are preliminary until approved 

25 by the NRC. You will be informed of the official examination results about 30 days after 

26 all the examinations are complete.

27

28 6. Do you have any questions?

29 B. Written Examination Guidelines

30 1. [Read Verbatim] After you complete the examination, sign the statement on the cover

31 sheet, in ink, indicating that the work is your own and you have not received or given 

32 assistance in completing the examination.

33

34 2. To pass the examination, you must achieve an overall grade of 80 percent or greater, 

35 with 70 percent or greater on the SRO-only items, if applicable. If you only take the SRO 

36 portion of the examination (as a retake or with an upgrade waiver of the reactor operator 

37 (RO) examination), you must achieve an overall grade of 80 percent or greater to pass. 

38 Grades will not be rounded up to achieve a passing score. Every question is worth 

39 1 point.

40

41 3. For the initial examination, the nominal time limit for completing the examination is 

42 6 hours for the RO examination, 3 hours for the 25-question SRO-only examination,

43 9 hours for the combined RO/SRO examination, and 4 hours for the examination for the 

44 senior operator limited to fuel handling (LSRO). The times allotted for taking each 

45 examination noted above shall not be extended except for unavoidable situations 

46 (e.g., loss of power, building evacuation, emergency response). The applicant is 
ES-1.2, Page 2 of 6

1 responsible for ensuring his or her physical capability to complete the examination in the 

2 allotted time. 

3

4 [For a requalification examination, the time limit for completing both sections of the 

5 examination is 3 hours. If both sections are administered in the simulator during a single 

6 3-hour period, you may return to a section of the examination that you already 

7 completed or retain both sections of the examination until the allotted time has expired.]

8

9 4. You may bring pens, pencils, and calculators into the examination room; however, 

10 programmable memories must be erased. Applicants shall not bring tablets, cell 

11 phones, or other communications or electronic devices or recorders into the examination 

12 room. 

13

14 5. Print your name in the blank provided on the examination cover sheet and the answer 

15 sheet. You may be asked to provide the examiner with some form of positive 

16 identification.

17

18 6. Mark your answers on the answer sheet provided, and do not leave any question blank. 

19 Use only the paper provided, and do not write on the back side of the pages. If you are 

20 recording answers on a machine-gradable form, mark your answer using ink or pencil as 

21 directed by the examination proctor. If you are using ink and decide to change your 

22 original answer, draw a single line through the error, enter the desired answer, and initial 

23 the change. If you are using pencil and decide to change your original answer, ensure 

24 that your previous choice is completely erased or indicate the change as described in 

25 the previous sentence. If you are recording your answers on a machine-gradable form 

26 that offers more than four answer choices (e.g., “a” through “e”), be careful to mark the 

27 correct column.

28

29 7. If you have any questions concerning the intent or the initial conditions of a question, do 

30 not hesitate to ask them before answering the question. Note that answers to questions 

31 you asked during the examination are documented and taken into consideration during 

32 the grading process. Only ask questions of the NRC examiner or the designated facility 

33 instructor. A dictionary is available if you need it.

34

35 8. When answering a question, do not make assumptions about conditions that are not 

36 specified in the question unless they occur as a consequence of other conditions that 

37 are stated in the question. For example, you should not assume that any alarm has 

38 activated unless the question so states or the alarm is expected to activate as a result of 

39 the conditions that are stated in the question. Similarly, you should assume that no 

40 operator actions have been taken, unless the stem of the question or the answer choices 

41 specifically state otherwise. Finally, answer all questions based on actual plant 

42 operation, procedures, and references. If you believe that the answer would be different 

43 based on simulator operation or training references, you should answer the question 

44 based on the actual plant.

45

46 9. Restroom trips are permitted, but only one applicant at a time will be allowed to leave. 

47 Avoid all contact with anyone outside the examination room to eliminate even the 

48 appearance or possibility of cheating.

49

50 10. When you complete the examination, assemble a package that includes the examination 

51 questions, examination aids, answer sheets, and scrap paper and give it to the NRC 
ES-1.2, Page 3 of 6

1 examiner or proctor. Remember to sign the statement on the examination cover sheet 

2 indicating that the work is your own and you have neither given nor received assistance 

3 in completing the examination.

4 

5 11. After turning in your examination, leave the examination area as defined by the proctor

6 or NRC examiner. If you are found in this area while the examination is still in progress, 

7 your license may be denied or revoked.

8 

9 12. Do you have any questions?

10 C. General Operating Test Guidelines

11 1. If you are asked a question or directed to perform a task that is unclear, you should not

12 hesitate to ask for clarification.

13 2. The examiner will take notes throughout the test to document your performance, and the 

14 examiner may sometimes take a short break for this reason. The amount of note-taking 

15 does not reflect your level of performance. The examiner is required to document both 

16 satisfactory and less-than-satisfactory performance.

17 3. The operating test is considered “open reference.” The reference materials normally 

18 available to operators in the facility and control room (including calibration curves, 

19 previous log entries, piping and instrumentation diagrams, calculation sheets, and 

20 procedures) are also available to you during the operating test. However, you should 

21 know from memory certain automatic actions, setpoints, interlocks, operating 

22 characteristics, and the immediate actions of emergency and other procedures, as 

23 appropriate to the facility. If you desire to use a reference, you should ask the examiner 

24 whether it is acceptable to do so for the task or question under consideration.

25 4. During job performance measures (JPMs) and during any followup question periods, you 

26 may not solicit from or provide technical information to any other person. 

27 5. To maintain test integrity and fairness, you must not discuss any aspect of your 

28 operating test with, or in the presence of, any other examinee who has not completed 

29 the applicable portion of the operating test (i.e., the administrative topics JPMs, the 

30 control room and in-plant systems JPMs, or the simulator test).

31 6. Do you have any questions?

32 D. Walkthrough (Job Performance Measure) Guidelines

33 1. The walkthrough test covers control room systems, local system operations, and 

34 administrative requirements. The examiner will evaluate these areas using JPMs and 

35 specific followup questions, as necessary.

36 2. The walkthrough for the initial examination consists of 15 JPMs for RO and SRO-instant 

37 applicants and 10 JPMs for SRO-upgrade (SRO-U) and LSRO applicants. Except for 

38 LSROs, most of the JPMs will be conducted in the control room or simulator, and the 

39 remainder will be conducted in the plant.

40 3. [The requalification walkthrough test consists of a total of five JPMs, with at least two in 

41 the control room/simulator and at least two in the plant.] 
ES-1.2, Page 4 of 6

1 4. The examiner may not be familiar with this facility. When you enter the plant, you are 

2 asked to assist the examiner as necessary to ensure that he or she complies with safety, 

3 security, and radiation protection procedures.

4 5. You should not operate plant equipment without appropriate permission from the 

5 operating crew. Nothing the examiner says or asks will be intended to violate this 

6 principle. If the JPM task requires the opening of panel doors, check with your examiner 

7 before opening the door; the examiner will let you know if the operating crew has given 

8 permission.

9 6. Before beginning each JPM, the examiner will describe the initial conditions, explain the 

10 task to be completed, indicate whether the task is time critical, and explain which steps 

11 are to be simulated or discussed. You should perform or simulate the required actions 

12 as if directed by plant procedures or shift supervision. Do not assume that the examiner 

13 will accept an oral description of the required action unless the examiner indicates 

14 otherwise.

15 7. Time-critical JPMs have been validated by your facility and must be completed within the 

16 predetermined time interval to obtain a satisfactory grade for that JPM. 

17

18 You will be permitted to take whatever time is necessary to complete those JPMs that 

19 are not time critical provided that you are making reasonable progress toward achieving 

20 the task standard. If the examiner believes that you are not making reasonable 

21 progress, he or she will ask you to explain what remains to be done and how long it 

22 should take before stopping the task. You will be permitted at least twice the validated 

23 time to complete the JPM, whether you are making progress or not.

24 8. When performing JPMs, you are expected to make decisions and take actions based on 

25 the facility’s procedural guidance and the indications available. Some of the tasks that 

26 the examiner asks you to perform will require implementation of an alternative method 

27 directed by plant procedures.

28 9. If your facility licensee’s procedures and practices require the use of procedure readers 

29 or peer checks, you may ask the NRC examiner to perform those functions. However, 

30 because the NRC examiner must evaluate your individual performance without 

31 assistance from others, he or she will simply acknowledge your request and proposed 

32 actions regardless of their accuracy or correctness or inform you that a peer-checker is 

33 not available.

34 10. As part of the examination, the examiner may ask followup questions to evaluate your

35 knowledge of an administrative topic, system, or task. There is no specific time limit for 

36 answering any question; however, you may be evaluated as unsatisfactory on a question 

37 if you are unfamiliar with the subject or reference material and are unable to answer the 

38 question in a reasonable period of time. You will not be permitted to conduct unlimited 

39 searches of the plant reference material during the examination.

40 11. To facilitate the examination and better enable the examiner to assess your level of 

41 understanding, please verbalize your actions and observations while performing the 

42 JPMs. Also, please inform the examiner when you consider your performance of each 

43 JPM and your answer to each question to be complete.

44 12. If you need a break during the test, you should ask the examiner.
ES-1.2, Page 5 of 6

1 13. Do you have any questions?

2 E. Simulator Test Guidelines

3 1. Your primary responsibility is to operate the simulator as if it were the actual plant. If you 

4 believe that the simulator is not responding properly, you should make decisions and 

5 recommendations on the basis of the indications available unless directed otherwise by 

6 the examiner.

7 2. In general, the examiner will not ask you questions while a simulator scenario is in

8 progress. If the examiner asks you a question, you should answer it only if doing so will 

9 not interfere with simulation facility operations.

10 3. Teamwork and communications are evaluated. You can enhance the evaluation 

11 process by vocalizing your observations, analyses, and the bases for your actions.

12 [Requalification examinations evaluate the crew’s ability to safely operate the plant and 

13 the performance of both the individuals and the crew.]

14 4. If you recognize, but fail to correct, an erroneous decision, response, answer, analysis, 

15 action, or interpretation made by the operating team or crew, the examiner may 

16 conclude that you agree with the incorrect item.

17 Members of the operating team or crew (whether applicants or surrogates) should 

18 perform peer checks in accordance with the facility licensee’s procedures and practices; 

19 noncrew members and NRC examiners will not perform this function. However, if you 

20 begin to make a performance deficiency that someone else corrects, you will be held 

21 accountable for the performance deficiency as if it occurred.

22 5. You should keep a rough log during each scenario that would be sufficient to complete 

23 necessary formal log entries; the examiner may consider these notes when evaluating 

24 your performance. 

25 6. A designated facility instructor will act as the auxiliary operators, radiation health and 

26 chemistry technicians, maintenance supervisors, plant management, and anyone else 

27 needed outside the control room.

28 7. A facility staff member (or examiner) will provide a shift turnover briefing before the 

29 scenario begins. The briefing will cover present plant conditions, power history, 

30 equipment that is out of service, abnormal conditions, surveillances that are due, and 

31 instructions for the shift.

32 8. Control board switches may be purposely cleared, tagged, or administratively controlled 

33 and reported on the preshift brief to enhance a scenario or transient where appropriate. 

34 If a switch is misaligned, it will be tagged or otherwise highlighted as appropriate to the 

35 facility and will be noted during the shift turnover briefing. If you find something 

36 misaligned during your initial board walkdown that was not part of the shift turnover 

37 briefing then bring it to the attention of the lead examiner. The examiners will not 

38 misalign switches during the scenario.

39 9. Time compression may be used to expedite the sequence of events in some scenarios, 

40 but it will not preclude you from performing the actions that you would typically be 
ES-1.2, Page 6 of 6

1 required to perform in response to the events. If time compression is used, you will be 

2 informed during the scenario.

3 10. You will be given sufficient time (normally about 5 to 10 minutes) to familiarize 

4 yourselves with plant conditions before starting each simulator scenario.

5 11. The operating test for initial licensing will normally consist of two or three scenarios each 

6 lasting approximately 1.5 to 2 hours. [The requalification test will normally consist of two 

7 scenarios lasting about 1 hour each.] 

8 12. SRO-U applicants who fill the role of a RO or balance-of-plant operator during a scenario 

9 will be evaluated on their ability to manipulate the controls even though an examiner 

10 may not be assigned to directly monitor their performance.

11 13. Do you have any questions?
ES-1.3, Page 1 of 8

1 1.3 EXAMINATION SECURITY

2 This examination standard contains instructions, restrictions, guidelines, and considerations for 

3 maintaining examination security and integrity for operator licensing initial examinations. 

4 A. Background

5 The NRC examination team and facility licensee personnel must be attentive to examination 

6 security measures to ensure compliance with applicable NRC regulations. 

7 Pursuant to 10 CFR 55.40(b)(2), facility licensees that elect to prepare their own examinations 

8 must establish, implement, and maintain procedures to control examination security and 

9 integrity. 

10 In accordance with 10 CFR 55.49, “Integrity of examinations and tests,” licensees, facility 

11 licensees, and applicants shall not engage in any activity that compromises the integrity of any 

12 application, test, or examination required by 10 CFR Part 55. The integrity of a test or 

13 examination is considered compromised if any activity, regardless of intent, affected, or, but for 

14 detection, would have affected the equitable and consistent administration of the test or 

15 examination. This includes activities related to the preparation and certification of license 

16 applications and all activities related to the preparation, administration, and grading of the tests 

17 and examinations required by 10 CFR Part 55.

18 The NRC Enforcement Policy, available on the NRC’s public Web site at 

19 http://www.nrc.gov/about-nrc/regulatory/enforcement/enforce-pol.html, addresses possible 

20 enforcement actions against parties subject to the requirements in the regulation 

21 (i.e., 10 CFR Part 55 license applicants and licensees, and licensees under 10 CFR Part 50, 

22 “Domestic licensing of production and utilization facilities,” and 10 CFR Part 52, “Licenses, 

23 certifications, and approvals for nuclear power plants”). 

24 B. Instructions for NRC Regional Management, Supervision, and Designees

25 Once examination arrangements are confirmed with a facility licensee, NRC regional managers 

26 ensure that an NRC examiner reviews the facility licensee’s security procedures and briefs the 

27 facility licensee contact on the examination security instructions in Section C of this examination 

28 standard.

29 If there is an indication that an examination may have been compromised, the responsible 

30 supervisor will act as necessary to ensure and restore the security and integrity of the 

31 examination process. Actions may include not giving the examination; making additional 

32 changes to the examination; voiding the results if the examination has already been given; 

33 revoking, suspending, or modifying, in whole or in part, a license pursuant to 10 CFR 55.61(b); 

34 and possibly imposing enforcement action in accordance with the Enforcement Policy. The 

35 supervisor shall keep NRC regional management and the NRR operator licensing program 

36 office informed of any concerns about examination security and integrity.

37 C. Instructions and Restrictions for Facility Licensee Personnel

38 All facility licensee and contractor personnel involved with an examination are covered by and 

39 subject to 10 CFR 55.49. This section provides instructions and restrictions for these personnel. 
ES-1.3, Page 2 of 8

1 Questions on these instructions and restrictions should be resolved with the NRC chief 

2 examiner before granting an individual access to the licensing examination.

3 1. Access to Examinations

4 Although there is no specific upper limit on the number of facility licensee personnel who have 

5 access to the NRC licensing examination, access should be limited to only those personnel with 

6 a need to know. Additionally, each person’s access should be limited to only those portions of 

7 the examination for which the individual bears responsibility (e.g., the individuals who prepare 

8 the simulator scenarios may not require access to the written examinations).

9 Facility licensees are responsible for the integrity and security of examinations prepared for 

10 them by contractor personnel.

11 2. Examination Security Agreement

12 All facility licensee personnel, including contractors, who will receive detailed knowledge of any 

13 portion of the NRC licensing examination, including the examination outline, must acknowledge 

14 their responsibilities with respect to examination security by reading and signing column 

15 “Signature (1)” of Form 1.3-1, “Examination Security Agreement,” before they obtain detailed 

16 knowledge and again by signing column “Signature (2)” after the examinations are complete.

17 The facility licensee submits the original examination security agreement forms to the NRC 

18 regional office for retention after the examinations are complete.

19 3. Prohibited Activities for Personnel on Examination Security Agreements

20 Prohibited activities for facility licensee personnel who have detailed knowledge of any portion 

21 of the NRC licensing examination include the following examples:

22 • the design and administration of any classroom and simulator instruction, including 

23 scheduled sessions, individual coaching, and remedial training, specifically for license 

24 applicants

25 – Simulator booth operation is acceptable if the individual does not select the 

26 training content or provide direct or indirect feedback. Continued participation in 

27 requalification training for groups including SRO-U applicants is also acceptable 

28 as long as this is documented on Form 1.3-1 and is limited to areas in which the 

29 instructor has no examination knowledge.

30 • all on-the-job training, practice, coaching, and signoffs

31 • the preparation, review, grading, and evaluation of periodic quizzes, examinations, and 

32 simulator exercises

33 – Signers of the examination security agreement may prepare and grade the audit 

34 examination, subject to an NRC review for test item duplication.

35 • development and addition of questions to the facility common question bank (used to 

36 create both NRC and non-NRC examinations and quizzes) if these questions are to be 

37 included in the NRC examination under development
ES-1.3, Page 3 of 8

1 – If questions are being developed to expand the size of the question bank, they 

2 are treated as any other bank item and subject to the other criteria in 

3 NUREG-1021 (e.g., repetition from the audit exam).

4 The license applicants should not be able to predict or narrow the possible scope or content of 

5 the licensing examination based on the facility licensee’s examination practices (other than 

6 those authorized by the NRC’s regulations, in NUREG-1021, or in writing by the NRC). In other 

7 words, the facility licensee personnel must not provide information to the applicants or licensed 

8 operators about examination content that would allow the test takers to either specifically or 

9 generally “predict” what test items will, or will not, be covered on the examination (e.g., the staff 

10 should not specify a certain exam bank to study if there is more than one version of the bank, or 

11 the staff should not make general statements, such as “Exam B will not overlap with Exam A”). 

12 This restriction is not intended to limit facility licensee personnel from discussing the general 

13 aspects related to an initial license examination, such as overall examination construction as 

14 described in these examination standards. However, facility licensee personnel must not 

15 disclose specific attributes of the examination’s content (e.g., how many JPMs or scenarios 

16 come directly from the bank, how many alternate path JPMs are on the examination, how many 

17 technical specification calls are in the scenarios).

18

19 4. Supervisor and Manager Interactions with Applicants

20 Supervisors and managers who have knowledge of the examination content may continue their 

21 general oversight of the training program for the license applicants, including the review of 

22 examinations, quizzes, and remedial training programs, as well as the counseling of applicants 

23 concerning nontechnical issues. Those supervisors and managers may not provide any 

24 technical guidance, training, or other direct feedback on the content of those examinations, 

25 quizzes, or programs in a manner that might compromise the integrity of the licensing 

26 examination, as defined in 10 CFR 55.49.

27 5. Examination Validation

28 The NRC discourages the use of certain individuals, such as the applicant’s supervisors or 

29 coworkers, to validate the NRC examination.

30 6. Audit Examinations

31 Implement appropriate controls to keep the comprehensive audit or screening examination that 

32 is given at or near the end of the license training class from compromising the integrity of the 

33 operator licensing initial examination. This also applies to any practice exams and quizzes 

34 created after beginning work on the licensing examination. Some examples of acceptable 

35 control measures include the following:

36

37 • Prepare the audit examination using a systematic and random sampling process that is 

38 similar to that used to prepare the NRC’s licensing examination as discussed in ES-4.1, 

39 “Preparing Written Examination Outlines.”

40 • Prepare and finalize the audit examination (and any practice exams and quizzes) before 

41 receiving the licensing written examination outline from the NRC.

42 • Develop the audit (as well as any practice exams and quizzes) and the licensing 

43 examinations using independent examination teams.
ES-1.3, Page 4 of 8

1 • Certify, as part of the examination submittal, that there is no question duplication 

2 between the facility licensee’s audit and the NRC’s licensing examinations.

3 Other methods might also be acceptable but will have to be reviewed and approved on a 

4 case-by-case basis.

5 D. Physical Examination Security Instructions and Guidelines

6 1. The NRC expects that personnel are aware of the facility licensee’s physical 

7 examination security measures and requirements (as documented in the facility 

8 licensee’s approved procedures); sign the NRC’s examination security agreement; and 

9 understand their security responsibilities, including the limits on their interaction with the 

10 license applicants (as discussed in Section C), before they are given knowledge or 

11 custody of any examination materials.

12 2. The examination outlines and final examinations shall be positively and continuously 

13 controlled and protected as sensitive information (i.e., under lock and key or in the 

14 custody of someone who has signed the examination security agreement). The number 

15 of copies of outlines and examinations should be limited, and each should be uniquely 

16 identified and controlled (e.g., with sign-out custody) at all times. Drafts, copies, and 

17 waste materials shall also be controlled and disposed of properly.

18 3. The NRC staff recommends that facility licensees consider implementing additional 

19 security measures when they are developing, storing, or printing examinations using a 

20 computer network to which license applicants or other persons who have not signed the 

21 examination security agreement could gain access. This includes any examination 

22 material that may have been stored on the simulator process computer, such as 

23 sequence of events data. Although the use of passwords should provide adequate 

24 security if normal computer security practices (e.g., selecting and changing passwords) 

25 are observed, special cases may need additional consideration. For example, if an 

26 applicant has extended access to the local area network (LAN) in his normal position, 

27 additional security measures may be appropriate.

28 4. When physically submitting examination material to the NRC regional office, the 

29 examination material should be placed in a double envelope, with the inner envelope 

30 marked “FOR OFFICIAL USE ONLY” and “TO BE OPENED BY ADDRESSEE ONLY.” 

31 Include a cover letter stating that the materials must be withheld from public disclosure 

32 until after the examinations are complete. Finally, the sender should verify with the NRC 

33 chief examiner that the package was received.

34 5. When electronically submitting examination material to the NRC, a secure means must 

35 be used, such as the NRC’s internal LAN in the resident inspector’s office, secure file 

36 sharing tools, or e-mail with password-protected electronic files. The password should 

37 be provided to the NRC chief examiner separately by mail, fax, or telephone. The 

38 password should NOT be e-mailed with the files.

39 The NRC has a checklist that NRC staff and facility licensees can use to assist in secure 

40 transmittals: “Checklist for Transmitting and Receiving NRC Exam Material over the 

41 Internet,” available at https://www.nrc.gov/reactors/operator-licensing/regs-guides42 comm.html.
ES-1.3, Page 5 of 8

1 6. The NRC expects the facility licensee to report immediately to the NRC chief examiner 

2 any indications or suggestions that examination security may have been compromised, 

3 even if the situation is identified and corrected before the examination is submitted to the 

4 NRC for review and approval. The NRC will evaluate such situations on a case-by-case 

5 basis and determine the appropriate course of action.

6 7. The facility licensee and the NRC shall determine whether examination security 

7 problems were noted in the past and ensure that corrective actions have been taken to 

8 preclude their recurrence.

9 8. The facility licensee and the NRC chief examiner shall review the simulator security 

10 considerations in Section F to ensure that the instructor station features programmer’s 

11 tools, and external interconnections do not compromise examination integrity. The 

12 primary objective is to ensure that the exam material cannot be read or recorded at other 

13 unsecured consoles, and that examination materials are either physically secured or 

14 electronically protected when not in use by individuals listed on the security agreement.

15 E. Examination Bank Limitations

16 The facility licensee and the NRC chief examiner shall ensure that written examinations and 

17 operating tests conform to the instructions in this NUREG on the use of items taken directly from 

18 the examination bank, modified items, and new items.

19

20 If the facility licensee has an open bank, it shall not place any new or modified test items 

21 (i.e., written questions, JPMs, or simulator scenarios) that will be used on the examination in its 

22 examination bank until after the last examination has been administered.

23

24 F. Simulator Security Considerations

25 Simulators present a unique set of integrity concerns during the development and administration 

26 of operating tests. NRC examiners and facility licensees should be aware of the simulator’s 

27 vulnerabilities and take appropriate measures to ensure that operating test security is 

28 maintained during use of (1) the instructor station, (2) the programmer’s tools, and (3) the 

29 external interconnections. Because facility licensees are more familiar than NRC examiners 

30 with their simulator’s unique capabilities, limitations, and vulnerabilities, the NRC expects that 

31 facility licensees will take responsibility for determining and implementing whatever measures 

32 might be necessary to ensure the integrity of the operating tests.

33 1. Instructor Station Features

34 Most of the instructor station features can be checked through the interface provided at the 

35 instructor’s console. The programmer’s tools and the external interconnections are not 

36 generally apparent to the instructor or the examiner. The simulator staff should be consulted to 

37 determine the status of those items. A typical instructor station has the following features:

38 • Snapshots. All simulators have snapshot capability. Initial conditions (ICs) are recorded 

39 for future recall.

40 • Backtrack. Backtrack files are snapshots that are automatically recorded at 

41 predetermined intervals (usually up to 1 hour of operation at intervals as frequent as 

42 1 minute). Backtrack files are usually only accessible through the BACKTRACK feature. 

43 The files typically can be overwritten by real-time operation but cannot be erased.
ES-1.3, Page 6 of 8

1 • Replay/Playback. The replay/playback feature steps through a series of snapshots and 

2 displays the output status (lights and meters) for each sequentially. Often, the replay 

3 feature uses the backtrack files, although separate replay file storage may be provided.

4 • Scripts/Computer-Assisted Exercises. Many simulators have a feature that allows 

5 preprogrammed implementation of malfunctions and remote functions based on time or 

6 logical conditions (or both). The simulator staff may use scripts to facilitate scenario 

7 administration and can typically store scripts for future use. Stored scripts can also be 

8 selected for review and editing from the instructor station.

9 • IC Summary. Snapshots are usually labeled on the IC menu of the instructor station 

10 with date/time recorded, pertinent plant parameter status, and instructor’s comments. 

11 Even if the comment field has been changed to indicate that a snapshot is available for 

12 reuse, the data (scenario initialization) may still be representative of test conditions until 

13 the snapshot is overwritten or updated.

14 • Malfunction Summary. Malfunction summary menus display the status of selected 

15 malfunctions, both active and inactive. The malfunction summary is usually IC 

16 dependent and, therefore, depicts the malfunctions that were active or staged when an 

17 IC (such as a scenario validation) was stored.

18 • Monitored Parameters. Instructors are afforded the capability to define individual or 

19 groups of parameters for display or printout. The monitored parameter group 

20 assignments can be recalled for review and editing. If used to facilitate scenario 

21 validation or examination administration, the monitored parameters can provide insight 

22 into the focus of the examination.

23 • Trend Recording. Groups of parameters can be defined and assigned to trend 

24 recorders. The recorders may be, but are not necessarily, located at the instructor 

25 station. The recording may also be in file format for presentation on the instructor station 

26 screens. Recording sessions are typically activated or deactivated at the instructor 

27 station.

28 • Student Performance Monitoring. Special groups of parameters and simulated plant 

29 operating conditions can often be assigned to a tracking and recording function that plots 

30 an individual student’s performance during training exercises. Recording sessions are 

31 typically activated or deactivated at the instructor station.

32 • Video and Audio Recording. Many simulators are equipped with video- and 

33 audio-recording capability in the control room. Video and audio controls are typically 

34 located at the instructor station.

35 • Sequence of Event Files. Many simulators have the capability to monitor and record the 

36 sequence of events during simulator scenarios. These files may stay in place and 

37 remain accessible until deleted or overwritten by subsequent scenario runs provided that 

38 examination security is maintained.

39 2. Programmer’s Tools

40 • Programmer’s tools include the following:
ES-1.3, Page 7 of 8 

1 • Software Terminals. Simulator engineers have access to real-time monitoring and

2 control of simulator and model conditions through software support terminals. These

3 terminals may be located in other work areas.

4 • Independent Executive Programs. The conditions for scenarios can sometimes be

5 replicated offline using independent executive programs. These programs should not be

6 in communication with the input/output. Independent executive programs and their

7 associated initialization files may provide an indication of planned exercises if they have

8 been used to resolve problems during scenario validation.

9 • Graphical User Interfaces (GUIs). Instructor station GUIs often display simulated plant

10 conditions and performance in real time. At remote locations, such as a programmer’s

11 desk, the GUI could display the full scenario.

12 3. External Interconnections

13 External interconnections include the following:

14

15 • Engineered Safety Feature Feeds. Many simulators have data links to the engineered 

16 safety feature and the operations management offices for emergency planning drills. 

17 These links can display the simulated plant condition to observers outside the simulated 

18 control room during scenario validation or examinations.

19 • Remote Plant Process Computer and Instructor Station Screens. Repeater screens in 

20 the training area can display scenarios in real time to observers outside the simulated 

21 control room.

22 • Modems and Remote Simulator Support Systems. Many simulators are equipped with 

23 modems from the instructor station or simulation computers for outside monitoring and 

24 control of simulator status and activities by parties off site.

25 G. Forms

26 Form 1.3-1 Examination Security Agreement
ES-1.3, Page 8 of 8 

Form 1.3-1 Examination Security Agreement

(1) Preexamination

I acknowledge that I have acquired specialized knowledge about the U.S. Nuclear Regulatory Commission (NR

examinations scheduled for the week(s) of as of the date of my signature. I agree that I will not kn

information about these examinations to any persons who have not been authorized by the NRC chief examiner

am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered 

examinations from this date until the completion of examination administration, except as specifically noted belo

the NRC. Furthermore, I am aware of the physical security measures and requirements (as documented in the 

procedures) and understand that violation of the conditions of this agreement may result in cancellation of the e

enforcement action by the NRC against me or the facility licensee, or both. I will immediately report to facility lic

or the NRC chief examiner any indications or suggestions that examination security may have been compromis

(2) Postexamination

To the best of my knowledge, I did not divulge to any unauthorized persons any information concerning the NRC 

examinations administered during the week(s) of . From the date that I entered into this examination 

until the completion of examination administration, I did not instruct, evaluate, or provide performance feedback 

who were administered these licensing examinations, except as specifically noted below or authorized by the N

NOTES:

Print Name Job Title Signature (1) Date Signature (2) Date 

1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

11.

12.

13.
ES-2.1, Page 1 of 20

1 2.1 PREPARING FOR OPERATOR LICENSING INITIAL

2 EXAMINATIONS

3 This examination standard describes activities for U.S. Nuclear Regulatory Commission (NRC) 

4 staff and facility licensee personnel to prepare for operator licensing initial examinations (written 

5 examinations and operating tests) at power reactor facilities. This standard includes instructions 

6 for scheduling and coordinating examination development, assigning NRC examiners and facility 

7 personnel, and obtaining reference and examination materials from the facility licensee.

8 A. Scheduling Examinations

9 Planning and scheduling operator licensing examinations includes the following steps:

10

11 1. The NRC issues an annual letter soliciting estimated operator licensing needs, including 

12 estimated numbers of applicants, examination dates, and the facility licensee’s intended 

13 level of participation in developing all parts of the examination.

14 2. The facility licensee should also notify the NRC regional office if its examination 

15 requirements change significantly from those stated in its response to the annual letter. 

16 3. The NRC regional office schedules the NRC’s operator licensing initial examinations and 

17 arranges for the development, administration, and grading of those examinations.

18 Note: Examinations for fewer than three applicants will only be scheduled under 

19 extenuating circumstances, such as a shortage of licensed operators at the facility or in 

20 the event of a retake examination. If a facility licensee has fewer than three applicants, 

21 the examinations may be delayed until there are more applicants.

22 Note: If the facility licensee prepares the written examinations or operating tests in 

23 conjunction with another facility, the two examinations/tests must be administered at the 

24 same time.

25 4. Each NRC regional office will prepare at least one examination per calendar year to 

26 certify new examiners, if necessary, and to maintain the proficiency of existing 

27 examiners. This will at a minimum consist of the entire written examination and the 

28 outlines for the job performance measures (JPMs) and simulator scenarios.

29 5. The NRC regional office issues a letter confirming the arrangements no later than 

30 210 days before the examination begins. This timeline does not apply to retake 

31 examinations. The letter should be addressed to the person at the highest level of 

32 corporate management who is responsible for plant operations (e.g., Vice President of 

33 Nuclear Operations). Letter 2.1-1, “Sample Corporate Notification Letter,” in this 

34 examination standard is an example of such a letter; the exact wording may be modified, 

35 as necessary to reflect the situation.

36 6. In addition, the Commission may exercise its discretion and reject a power reactor 

37 facility licensee’s election to prepare, proctor, or grade all or portions of the examinations 

38 (e.g., outline, written examination, operating test). In such a case, the Commission shall 

39 prepare, proctor, and grade the required written examinations and operating tests for 

40 that facility licensee. This may occur if the NRC regional office does not have 
ES-2.1, Page 2 of 20

1 confidence that a facility licensee will develop adequate examinations or if the region has 

2 concerns with examination security or quality.

3 7. To maintain uniform conditions, the operating test and written examination should be 

4 administered within 30 days of one another. Under certain circumstances, such as a 

5 weather or health emergency, this may not be possible. The NRC regional office must 

6 obtain concurrence from the Office of Nuclear Reactor Regulation (NRR) operator 

7 licensing program if more than 30 days will elapse between the completion of one 

8 portion and the start of the other. 

9 B. Examination Preparation Criteria for Facility Licensees

10 Form 2.1-1, “Examination Preparation Checklist,” should be used to track the examination 

11 preparations. As noted on the form, target due dates can be adjusted as necessary to 

12 accommodate a given situation. 

13 Before developing an operator licensing initial examination, facility licensees are encouraged to 

14 review the examination security considerations in Examination Standard (ES)-1.3, “Examination 

15 Security.”

16 Examination preparation for facility licensees includes the following steps:

17 Note: Items identified with an asterisk (*) DO NOT apply to NRC-authored examinations.

18 1. Designate a point of contact to work with the NRC chief examiner and assign additional 

19 personnel as required to ensure that the examinations are developed, reviewed, 

20 administered, and graded in accordance with the applicable examination standards. The 

21 facility licensee may use contractors or other outside assistance to develop the 

22 examinations, but the licensee bears full responsibility for the product, including 

23 conformance with the examination criteria and maintenance of examination security and 

24 integrity.

25 2. *When ready to begin developing the written examination, and no more than 18 months 

26 before the scheduled examination date, the facility licensee may request the NRC 

27 regional office to prepare the written examination outline (ES-4.1, “Preparing Written 

28 Examination Outlines”). The NRC will use the knowledge and abilities (K/A) catalog that 

29 has been reviewed and approved by the NRC at the time of this request to create the 

30 outline; this determines the revision of the catalog to be used for examination 

31 development (i.e., the applicable K/A catalog). The 18-month limit allows use of the 

32 current revision of the applicable K/A catalog if a revision to it is in process. 

33 3. Review the NRC-provided examination outline(s) and submit comments for any 

34 necessary changes.

35 4. Submit the required reference materials (Section F), examination outlines, and 

36 examinations, as applicable, based on the level of facility licensee participation. 

37 5. If the facility determines that it is appropriate for the purposes of operator training and 

38 examination, “freeze” the plant procedures at a specific revision number or date to 

39 facilitate examination development. If used, discuss this option with the NRC chief 

40 examiner in advance and refer to Section G of this examination standard for additional 

41 guidance on plant procedure freezes.
ES-2.1, Page 3 of 20

1 6. *Submit proposed outlines for all portions of the operating test (simulator and 

2 walkthrough) at all license levels relevant to the applicants (reactor operator (RO), senior 

3 reactor operator (SRO), and senior operator limited to fuel handling (LSRO) to be tested.

4 7. Ensure that a facility supervisor or manager independently reviews the proposed 

5 examination outline(s) and the proposed examination(s) before they are submitted to the 

6 NRC regional office. Perform this review using the instructions in ES-2.3, “Reviewing 

7 and Approving Operator Licensing Initial Examinations.”

8 8. *Ensure that an authorized representative of the facility licensee has approved the 

9 required written examinations and operating tests before they are submitted to the NRC 

10 regional office for review and approval. Submit the licensee-approved written 

11 examinations and operating tests to the NRC regional office with a cover letter signed by 

12 the facility representative. The materials must be complete and ready to use to facilitate 

13 a thorough review by the NRC region.

14 9. If the NRC staff prepared any portion of the examination, the NRC regional office will 

15 provide a copy of the applicable examination portions to the facility licensee reviewers 

16 after they sign the security agreement (Form 1.3-1, “Examination Security Agreement”). 

17 The facility licensee reviewers should make their comments directly on the 

18 examination(s), use the review worksheets in ES-2.3, or employ another method to 

19 document their comments. The reviewers should then give their comments to the NRC 

20 chief examiner and ensure that he or she understands their comments and 

21 recommendations. Simple editorial changes that do not change the intent of the 

22 question require no justification; however, every substantive change (e.g., deleting a 

23 question, replacing a distractor, or revising an answer) must be supported by approved 

24 facility reference material. The facility licensee reviewers may retain a copy of the 

25 applicable marked-up examination, subject to the physical security considerations in 

26 ES-1.3.

27 10. If the facility licensee has significant concerns with the content or difficulty of the 

28 NRC-prepared examination, the changes that the NRC has directed the facility licensee 

29 to make in its proposed examination, or the general implementation of the instructions

30 and guidelines in this standard, communicate those concerns to the NRC and, if 

31 appropriate, request a meeting with the NRC to address the concerns. The NRC chief 

32 examiner is normally the first point of contact for resolving any concerns about the 

33 examination. If the concerns are not resolved, contact NRC regional management and, 

34 if necessary, the NRR operator licensing program office Branch Chief.

35 11. Make the simulation facility available, typically during normal business hours, for NRC 

36 examiners to prepare for, validate, and administer the operating tests. The NRC will 

37 make reasonable efforts to minimize the impact on other training activities. Licensees 

38 should plan to dedicate the simulator entirely to the initial license examination for the 

39 extent of operating test administration, except for administrative JPMs. Running 

40 requalification training or simulator tests during examination week should be avoided. If 

41 an examination week is cancelled due to simulator availability, the NRC cannot 

42 guarantee that the examination will be rescheduled within the 30-day limit for time 

43 between sections of the examination.

44 12. Pursuant to Title 10 of the Code of Federal Regulations (10 CFR) 55.46(c)(1)(i) and 

45 10 CFR 55.46(d), ensure sufficient simulator fidelity to allow conduct of the evolutions 

46 listed in 10 CFR 55.45(a)(1)–(13), as applicable to the design of the reference plant. 
ES-2.1, Page 4 of 20

1 Provide the results of any uncorrected performance deficiencies that may exist at the 

2 time of operating test administration for NRC review. In addition, give the NRC a list of 

3 significant differences between the simulator facility and the reference plant.

4 13. Before administering the first operator licensing examination at a cold plant, the NRC 

5 may perform an inspection using Inspection Procedure 41502, “Nuclear Power Plant 

6 Simulation Facilities,” to verify conformance with the simulator requirements specified in 

7 10 CFR 55.46, “Simulation facilities,” and to assess the adequacy of the facility 

8 licensee’s simulation facility for use in operator licensing examinations as well as for 

9 applicant experience requirements as described in 10 CFR 55.31(a)(5).

10 14. Meet with the NRC (normally the NRC chief examiner) to review and discuss all 

11 substantive comments from the quality reviews of the examination outlines and 

12 proposed examinations. These meetings will normally be conducted by telephone but, 

13 with approval from NRC regional management and agreement of both parties, may be 

14 conducted in the regional office or at the facility. These meetings may be supported by 

15 securely sharing electronic documents.

16 15. *Make any necessary changes as agreed upon with the NRC; however, the NRC retains 

17 final authority to approve the examinations.

18 16. Submit waiver and excusal requests as early in the process as possible (ideally more 

19 than 60 days before the examination), using a preliminary NRC Form 398, “Personal 

20 Qualifications Statement—Licensee”). Early submittal of preliminary waiver and excusal 

21 requests allows the NRC to better plan resources for evaluating requests and resolve 

22 any issues before receipt of the final, signed application.

23 17. Submit the license applications along with a letter requesting the administration of

24 licensing examinations. Preliminary applications are due approximately 30 days before 

25 the examination, and final signed applications are due 14 days before the examination.

26 18. Develop a schedule for administering the operating test to optimize the efficient 

27 administration of the examination, given the number of examiners and the mix of RO and 

28 SRO applicants. The schedule should identify the crews for the simulator scenarios, 

29 state the number of scenarios that must be created to accommodate the crew

30 complement, prescribe the timing of JPMs, and propose examiners by position to 

31 evaluate each applicant (i.e., Examiner 1, Examiner 2). Follow the guidance in ES-3.4, 

32 “Developing Scenarios,” for use of surrogates, number of applicants in a crew, and other 

33 considerations. 

34 C. Examination Preparation Criteria for NRC Regional Management, Supervision, 

35 and Designees

36 Examination preparation for NRC regional management includes the following steps:

37

38 1. Approximately 7 months before the scheduled examination, the NRC regional office will 

39 assign the required number of examiners to develop, prepare for, and administer the 

40 examination. The NRC regional office will also designate an NRC chief examiner to 

41 coordinate the examination project with the facility licensee and other examiners 

42 assigned to the examination. When making assignments, the regional office should 

43 consider each examiner’s certification status, other examination commitments, possible 
ES-2.1, Page 5 of 20

1 conflicts of interest (as discussed in Section E of this examination standard), and general 

2 availability.

3 2. Once the facility licensee has begun preparing the examination, the NRC regional office 

4 will avoid changing the assigned NRC chief examiner unless necessary. If a change is 

5 unavoidable, the responsible supervisor will attempt to minimize the impact on the facility 

6 licensee.

7 3. Regional management should assign enough examiners so that no examiner will have to 

8 administer more than four complete simulator operating tests per week.

9 4. Approximately 7 months before the scheduled examinations, the NRC regional office will 

10 discuss the following examination arrangements, as applicable, with the facility licensee

11 during the examination kick-off call:

12 a. anticipated examination dates

13 b. expected number and type of applicants

14 c. expected dates for the items listed on Form 2.1-1

15 Note: The NRC regional office may agree to earlier due dates with the facility 

16 licensee contact but should refrain from advancing the dates if it is unlikely that 

17 the review will begin promptly after the regional office receives the material. The 

18 regional office should inform the facility licensee contact of the dates by which 

19 the regional office expects to provide its comments on the material.

20 d. requirements and considerations for examination security and integrity

21 e. guidance related to freezing plant procedures

22 f. the instructions and guidelines for developing, administering, and grading the 

23 written examination and operating test, including the effective revision of 

24 examination standards

25 g. for NRC-developed examinations, the need for the facility to provide the 

26 necessary reference materials for the NRC to develop the examination (see 

27 Section F)

28 h. the need for the NRC to provide the licensee with the written examination outline 

29 as early in the process as possible provided security requirements are in place 

30 (only applicable to licensee-developed examinations)

31 i. the need for the facility licensee to review the NRC-provided examination 

32 outline(s) and submit comments for any necessary changes; the NRC will make 

33 any changes to the outlines for NRC-prepared examinations

34 j. the 10 CFR 55.40(b)(3) requirement that an authorized representative of the 

35 facility licensee must approve the proposed examination outlines and 

36 examinations before they are provided to the NRC as a formal submittal 

37 (including cover letter) for review and approval
ES-2.1, Page 6 of 20

1 k. dates and simulator availability for validating examination materials on site; these 

2 materials should include all the operating test components (JPMs and simulator 

3 scenarios), and the facility licensee must make the simulator available to 

4 examiners for an ample amount of time during the validation visit

5 l. the need to ensure simulator fidelity in accordance with 10 CFR 55.46(c)(1)(i) 

6 and to have the simulator available during onsite validation and operating test 

7 administration 

8 Note: A list of uncorrected performance deficiencies and deviations from the 

9 reference plant must be made available during onsite validation and again at the 

10 beginning of administration of the operating tests.

11 m. the option to submit some sample test items (e.g., 5 to 10 written questions, 

12 1 scenario, and 1 to 2 JPMs) for preliminary NRC review and comment (this 

13 presubmittal sample could increase the efficiency of the examination review 

14 process by promoting early identification and correction of generic examination 

15 development concerns)

16 Note: As long as changes from this review are incorporated, resulting in 

17 acceptable test items, these sample test items should not count toward the 

18 20-percent threshold when calculating the quality of the submitted examination in 

19 accordance with ES-2.3.

20 n. the requirements and guidelines for submitting the license applications 

21 Note: This discussion should include a justification for waiver and excusal 

22 requests, noting that waiver and excusal requests should be submitted as early 

23 in the process as possible (ideally more than 60 days before the examination), 

24 using a preliminary NRC Form 398). The NRC’s final decision on whether to 

25 grant a waiver or excusal request will be made on the official (not preliminary) 

26 NRC Form 398 submitted for the applicant.

27 o. whether any individual applying to take the examination has previously failed an

28 initial operator licensing examination operating test at any facility

29 7. The responsible regional supervisor reviews the examination outlines and the draft 

30 examinations and evaluates any recommended changes and corrections noted during 

31 the chief (and other) examiner’s review. ES-2.3 contains additional guidance on

32 examination reviews.

33 8. If regional management determines that additional time is needed for examination 

34 development or that there is a need to delay the examination administration to address 

35 other scheduling concerns, regional management may delay either part (written 

36 examination or operating test) of an examination for up to 30 days. It is not appropriate 

37 to delay one part of an NRC examination based on license applicant performance on 

38 another part of an NRC examination that has already been administered or based on 

39 applicant performance on licensee-administered audit examinations. However, the 

40 entire NRC examination may be delayed for these reasons as agreed upon by the 

41 regional office. The regional office must consult the NRR operator licensing program 

42 office about any delay and notify the facility licensee in writing of the reasons for 

43 delaying the examination(s).
ES-2.1, Page 7 of 20

1 9. Upon receiving the preliminary license applications, the NRC regional office reviews the 

2 applications and evaluates any waiver and excusal requests in accordance with ES-2.2, 

3 “Applications, Medical Requirements, and Waiver and Excusal of Examination and Test 

4 Requirements.” The regional office will communicate any errors or missing information 

5 noted during the preliminary application review to the facility licensee to ensure that it 

6 has an opportunity to make corrections before the final signed applications are submitted 

7 14 days before the examination date. This process helps prevent unnecessary delays in 

8 approving the examination for administration.

9 10. Examinations are reviewed and approved in accordance with ES-2.3.

10 11. The region prepares and sends the examination approval letter (Letter 2.3-1, “Sample 

11 Examination Approval Letter”) approximately 7 days before the examination to notify the

12 facility licensee that the NRC has completed its review of the license applications and to 

13 confirm that both the NRC and the facility licensee agree that the examination meets the 

14 guidelines of NUREG-1021. This letter also authorizes the facility licensee to administer 

15 the written examinations, if applicable. 

16 12. Approximately 7 days before the examination, the responsible NRC supervisor shall 

17 query the facility licensee management counterpart about the licensee’s views on the 

18 examination. The following subjects should be considered for discussion, and corrective 

19 measures shall be implemented when necessary:

20 a. whether the NRC test item comments were justified and clearly explained

21 b. the licensee’s assessment of the significant test item changes

22 c. whether any of the examination changes are believed to render the test items or 

23 the examination unfair, and whether this concern was shared with the NRC chief 

24 examiner

25 d. whether the facility licensee requested and was permitted to defer the correction 

26 of test item flaws that were identified as minor in nature

27 D. Examination Preparation Criteria for NRC Chief Examiner and Assigned 

28 Examiners

29 Examination preparation for NRC chief examiners and assigned examiners includes the 

30 following steps:

31

32 1. When assigned to administer operating tests for the first time at a particular facility, the 

33 examiner should inform the NRC chief examiner and the responsible supervisor so that 

34 arrangements can be made to ensure participation in onsite validation activities as 

35 described in Section I. If the examiner is unable to attend validation activities, the NRC 

36 chief examiner and responsible supervisor should determine whether a site orientation 

37 trip to the facility is appropriate.

38 2. Examiners not involved in test development are expected to research and study the 

39 topics and systems to be examined on the operating test so that they are prepared to 

40 ask whatever performance-based followup questions might be necessary to determine 

41 applicant competence. 
ES-2.1, Page 8 of 20

1 3. NRC examiners monitor and ensure the integrity of the examination process, but they 

2 are not expected to sign onto the facility licensee’s examination security checklist. If 

3 they perceive that a compromise has occurred, caused by either facility licensee or NRC 

4 personnel, they shall immediately report it to the responsible regional supervisor so that 

5 the necessary actions can be taken to restore the security and integrity of the 

6 examination. ES-1.3 contains the examination security and integrity requirements and 

7 guidance that examiners should note when reviewing the procedures that the facility 

8 licensee has established pursuant to 10 CFR 55.40(b)(2), as applicable.

9 4. After examination arrangements are confirmed, an NRC examiner reviews the facility 

10 licensee’s security procedures and briefs the facility contact on the examination security 

11 items in ES-1.3. This may occur as part of the discussion with the facility licensee in 

12 step 4 of Section C above.

13 5. The assigned examiners review and inventory the reference materials received from the 

14 facility licensee. The purpose of this review is to determine whether the materials are 

15 complete and adequate to enable the NRC regional office to review or develop the 

16 examinations, as applicable. The reviewer(s) will inform the NRC chief examiner if the 

17 materials are incomplete or inadequate, and the responsible supervisor will request that 

18 the facility licensee send any additional materials that might be required. If necessary, 

19 an examiner may review and select additional reference materials during a site 

20 orientation trip.

21 6. The NRC chief examiner works with the assigned examiners and the designated facility 

22 licensee contact, as applicable, to ensure that the examination outlines and 

23 examinations are developed in accordance with the applicable examination standards, 

24 considering comments received from the licensee pertaining to the NRC-provided 

25 outlines. The NRC chief examiner should adapt the level of oversight and coordination 

26 based on the experience of the individuals who are preparing the examinations. 

27 7. The NRC chief examiner ensures that the examination outlines, written examinations,

28 and operating tests are independently reviewed using the instructions and guidance in 

29 ES-2.3.

30 8. The NRC chief examiner shall note and review necessary changes and forward the 

31 outlines to the responsible supervisor for review and comment before resolving any 

32 deficiencies with the author or facility licensee contact.

33 9. If the NRC staff authored any portion of the examination, the NRC regional office will 

34 provide a copy of the applicable written examination(s), operating test(s), and outlines to 

35 the facility licensee reviewers after these reviewers sign the security agreement. If the 

36 facility licensee reviewers have significant disagreements with the NRC chief examiner, 

37 the NRC chief examiner will inform the responsible regional supervisor so that the 

38 disagreements can be resolved before the examinations are administered.

39 10. After receiving approval from the responsible NRC supervisor, generally about 7 weeks 

40 before the examinations are scheduled to be given, the NRC chief examiner will review 

41 the NRC comments on the written examinations and operating tests with the facility 

42 licensee.

43 The NRC chief examiner may conduct the examination review by telephone. The review 

44 may also be conducted in the regional office or at the facility with approval from NRC 
ES-2.1, Page 9 of 20

1 regional management and the agreement of both parties. These meetings may be 

2 supported by securely sharing electronic documents.

3 11. After examination corrections have been made, the NRC chief examiner verifies that the 

4 changes are appropriate and routes the examinations and the marked-up drafts to the 

5 responsible supervisor for final approval.

6 12. As soon as possible after the responsible supervisor has approved the operating tests 

7 for administration, the NRC chief examiner will ensure that the approved scenarios and 

8 JPMs are made available to the other assigned examiners so that they can familiarize 

9 themselves with those materials and be better prepared to further evaluate the 

10 applicants’ potential deficiencies if required.

11 13. The NRC chief examiner should review the facility licensee’s schedule for administration 

12 of the operating test. The schedule should be optimized for efficient administration, 

13 given the number of examiners and the mix of RO and SRO applicants. The schedule 

14 will identify crews for the simulator scenarios and the timing of JPMs and will propose 

15 which examiners will evaluate which applicants. The NRC chief examiner may elect to 

16 change the facility licensee’s proposed schedule, including crew assignments, examiner 

17 assignments, or the order of administration of the JPMs, with justification. However, if 

18 changes to crew assignments are required, the NRC chief examiner must identify these 

19 changes as early as possible and generally will not make changes less than 2 weeks 

20 before the examination start date to allow some time for affected applicants to adapt to 

21 working as a crew before they take the simulator operating test. 

22 14. The written examinations may be administered as soon as they and the final license 

23 applications (including any applicable waiver or excusal requests) have been approved. 

24 If the facility licensee will administer the written examinations, the NRC chief examiner 

25 reviews the instructions in ES-4.3, “Administering Written Examinations” (e.g., proctoring 

26 and responding to applicant questions) and confirms the applicant’s status

27 (i.e., examination type and waivers or excusals) on Form 2.2-1, “List of Applicants,” with 

28 the facility licensee contact before the examinations are given.

29 15. The operating test will be administered within 30 days of the applicants taking the 

30 corresponding written examination. Concurrence should be obtained from the NRR 

31 operator licensing program office before exceeding 30 days between the completion of 

32 the written examination/operating test and the start of the operating test/written 

33 examination.

34 16. If the examination schedule must be changed on short notice, the NRC chief examiner

35 will work with his or her supervisor and the designated facility licensee contact to 

36 reschedule the examinations to a time when examiners are available and other 

37 examinations are not affected.

38 E. Restrictions for NRC Personnel

39 • An examiner shall not be assigned to evaluate any portion of a retake operating test for 

40 an applicant if that examiner participated in the determination of either of the following 

41 for that same applicant:

42

43 – a failure of a previous operating test
ES-2.1, Page 10 of 20

1 – a denial of a request to be excused from reexamination of any portion of the 

2 operating test (i.e., the examiner was involved in the decision to deny the 

3 excusal)

4

5 • The licensing decision associated with the retake operating test for an applicant shall be 

6 made by a supervisor other than the supervisor who made the previous determination 

7 for an operating test failure.

8

9 • If an examiner was previously employed by a facility licensee (or by one of its 

10 contractors) and was significantly involved in training any of the current license 

11 applicants, the regional office shall not assign that examiner any direct responsibilities 

12 for developing, administering, or grading written examinations or operating tests for that 

13 facility. Regional management shall control other in-office examination activities 

14 concerning the facility, such as technical consultation and quality reviews of 

15 examinations.

16

17 • If an examiner is assigned to an examination that might appear to present a conflict of 

18 interest, the examiner shall inform his or her immediate supervisor of the potential 

19 appearance of conflict. Such notifications must include the following information:

20

21 – the nature and extent of previous personal and professional relationships with the

22 applicants

23

24 – issues that could affect the administration, performance, evaluation, or results of 

25 the examination

26

27 – anything that could create the appearance of a conflict of interest

28

29 F. Reference Material for Operator Licensing Initial Examinations

30 The NRC expects facility licensees to provide reference materials for each NRC operator 

31 licensing initial examination. The NRC regional office will customize the list of reference 

32 materials to support the specific examination assignment. The regional office shall consider the 

33 administrative burden it places on the facility licensee and will request only those materials that 

34 are necessary for the NRC examiners to prepare for the examinations. The regional office may 

35 request additional materials later, if necessary, to ensure the accuracy and validity of the 

36 examinations.

37 1. Determine Need 

38 In determining the need for reference materials, the NRC regional office will consider the facility 

39 licensee’s level of participation in the examination development process. If the facility licensee 

40 will prepare the examinations, it may be sufficient to obtain only those references necessary to 

41 review and validate the items that appear on the examination, plus a set of key procedures and 

42 other documents required to prepare for the operating tests. 

43 2. Use Approved Reference Material

44 All reference materials provided for the examinations should be approved, final issuances and 

45 should be marked as such, and personal, proprietary, sensitive, or safeguards information 

46 should be marked and submitted in a separate enclosure. If any of the material is expected to 
ES-2.1, Page 11 of 20

1 change before the scheduled examination date, the facility licensee should reach agreement 

2 with the NRC chief examiner about changes before the examinations are administered.

3 For plants under construction (i.e., cold plants), the facility licensee may use plant-specific 

4 terminology (e.g., “Operational Draft” or “Draft”) to designate procedures as approved and ready 

5 for use, before they are turned over to the Operations Department, or before the facility is

6 required to implement technical specifications in accordance with the combined 

7 license. Regardless of the terminology chosen by the plant to designate the procedure status, 

8 procedures used to support examination material should receive (1) an administrative review, to 

9 verify that the procedure meets the facility licensee’s writer’s guide requirements and satisfies 

10 all technical specifications and final safety analysis report requirements, and (2) a technical 

11 review, to verify that the procedure is correct for proper operations of plant systems and 

12 equipment. Additionally, the facility licensee’s management should approve the procedures 

13 provided to the NRC for each examination in accordance with the facility licensee’s 

14 administrative procedure requirements.

15 3. Format

16 Licensees can submit reference material either electronically or in hardcopy. All procedures 

17 and reference materials should contain appropriate indices or tables of contents so that they 

18 can be used efficiently, and a master table of contents should be provided for all materials sent. 

19 Also, all electronic documents should be text searchable. Failure to provide complete and 

20 indexed/text searchable reference materials may prompt the NRC to return the materials to the 

21 person at the highest level of corporate management responsible for plant operations. The 

22 NRC will include a cover letter with the returned reference materials explaining the deficiencies 

23 in the materials and the basis for postponing or canceling the examinations, as appropriate.

24 4. Specific Reference Material To Provide

25 Unless otherwise instructed by the NRC regional office, the facility licensee is expected to 

26 provide the following reference materials for each NRC operator licensing initial examination:

27 a. materials used by the facility licensee to ensure operator competency:

28 (1) Materials used to train applicants for initial RO and SRO licensing. Material 

29 should be complete, comprehensive, and of sufficient detail to support the 

30 development of accurate and valid examinations without being redundant. Such 

31 materials include the following:

32 • learning objectives, student handouts, and lesson plans

33 • system descriptions, drawings, and diagrams of all operationally relevant 

34 flowpaths, components, controls, and instrumentation

35 • material used to clarify and strengthen understanding of normal, 

36 abnormal, and emergency operating procedures (EOPs) (including 

37 severe accident management guidelines)

38 • complete, operationally useful descriptions of all safety system 

39 interactions and, where available, balance-of-plant system interactions 

40 under emergency and abnormal conditions, including consequences of 

41 anticipated operator errors, maintenance errors, and equipment failures, 
ES-2.1, Page 12 of 20

1 as well as plant-specific risk insights based on a probabilistic risk analysis 

2 and individual plant examination

3 (2) questions and answers specific to the facility training program that may be used 

4 in the written examinations or operating tests

5 (3) copies of facility-generated simulator scenarios that expose the applicants to 

6 abnormal and emergency conditions, including degraded pressure control, 

7 degraded heat removal capability, and containment challenges, during all modes 

8 of operation, including low-power conditions (a description of the scenarios used 

9 for the training class may also be provided)

10 (4) all JPMs used to ascertain the competence of the operators in performing tasks 

11 within the control room complex and outside the control room (i.e., local 

12 operations), as identified in the facility’s job task analysis (JPMs should evaluate 

13 operator responsibilities during normal, abnormal, and emergency conditions and 

14 events, and during all modes of operation, including cold shutdown, low power, 

15 and full power)

16 b. complete index of procedures (including all categories sent)

17 c. all administrative procedures applicable to reactor operation or safety or that support a 

18 written question or operating test item

19 d. all integrated plant procedures (normal or general operating procedures)

20 e. all emergency procedures and their bases (emergency instructions and abnormal or 

21 special procedures)

22 f. standing orders (important orders that are safety related and that may modify the regular 

23 procedures)

24 g. surveillance procedures that are run frequently (i.e., weekly) or that can be run on the 

25 simulator

26 h. fuel handling and core loading procedures (if SRO applicants will be examined)

27 i. all annunciator and alarm procedures

28 j. radiation protection manual (radiation control manual or procedures)

29 k. emergency plan implementing procedures

30 l. technical specifications or similar technical requirements documents (and interpretations, 

31 if available) and their bases for all units for which licenses are sought

32 m. system operating procedures

33 n. technical data book and plant curve information used by operators, as well as the facility 

34 precautions, limitations, and setpoints document

35 o. information pertaining to the simulation facility:
ES-2.1, Page 13 of 20

1 (1) a list of all initial conditions

2 (2) a list of all malfunctions with identification numbers and cause-and-effect 

3 information, including a concise description of the expected result or range of 

4 results that will occur upon initiation and an indication of which annunciators will 

5 be actuated because of the malfunction

6 (3) a description of the simulator’s failure capabilities for valves, breakers, indicators, 

7 and alarms

8 (4) the range of severity of each variable malfunction (e.g., the size of a reactor 

9 coolant or steam leak, or the rate of a component failure such as a feed pump, 

10 turbine generator, or major valve)

11 (5) a list of modeling conditions (e.g., simplifications, assumptions, and limits) and 

12 problems that may affect the examination

13 (6) a list of any known performance test discrepancies not yet corrected

14 (7) a list of significant differences between the simulator facility and the reference 

15 plant

16 (8) the simulator instructor’s manual

17 p. additional plant-specific material that the NRC examiners have requested to develop 

18 examinations that meet the guidelines of these standards and the regulations

19 G. Guidelines for Freezing Plant Procedures

20 The NRC recognizes that facility licensees may wish to train and examine their license 

21 applicants to the same version of plant procedures. At their discretion, facility licensees may 

22 “freeze” plant procedures to a particular revision for purposes of applicant training and 

23 examination development (either for licensee-prepared examinations or as reflected in the 

24 reference materials submitted for NRC-prepared examinations). 

25 • To limit gaps between training/testing and current plant operations, freezing procedures 

26 later during the training process is preferred. Alternatively, facility licensees could 

27 choose not to freeze procedures at all but rather track any procedure changes and 

28 adjust the training and examinations as required. However, depending on the nature 

29 and volume of changes, this alternative could impose a significant additional burden on 

30 the facility licensee and NRC examiners to ensure that procedure revisions affecting test 

31 items are reconciled before examination administration. 

32

33 • Applicants will be exposed to the current version of the procedures when they spend 

34 time in the control room. Therefore, freezing procedures for the examination has the 

35 potential to confuse applicants by testing them on a different version of procedures than 

36 the version that they have seen in the control room. Such confusion has contributed to 

37 applicants’ failure on the written examination because the applicants based their 

38 answers on the wrong version of procedures. If the procedures are frozen, the 

39 applicants must be informed of the date of the procedure freeze, such that they have a 

40 complete understanding of which versions of the procedures the NRC examination is 
ES-2.1, Page 14 of 20

1 based upon. Note that freezing different procedures at different times will likely add to 

2 the applicants’ confusion.

3

4 • Examination authors and NRC reviewers need to consider the implications of the freeze 

5 during examination development; for example, the plausibility and correctness of a 

6 distractor should not hinge on a procedure change that the frozen version of the 

7 procedure does not yet incorporate. Another consideration is whether the simulator will 

8 support the implementation of both procedure versions—the new one for license holders 

9 and the old one for the applicants.

10

11 • If changes in the procedures occur after the freeze and before the licensing date, the 

12 NRC expects the facility licensee to provide training to fill the gap; if the changes are 

13 significant, the NRC could request more information about the nature of such training 

14 and testing. In at least one instance, applicants were trained and tested on a new 

15 version of the EOPs that had not yet been implemented in the plant; this eliminated the 

16 need to retrain the applicants but prompted the NRC to delay their licensing until the new 

17 EOPs went into effect. 

18

19 • Facility licensee contacts should discuss the details of, and the basis for, their plan to 

20 freeze procedures with their NRC contact when confirming the examination 

21 arrangements. The NRC chief examiner, in consultation with the regional operator 

22 licensing supervisor (and the NRR operator licensing program office, if deemed 

23 necessary), will review the facility licensee’s proposal and negotiate a mutually 

24 acceptable plan and freeze date.

25

26 H. Milestones for Plants Under Construction

27 Figure 1, “NRC and Facility Licensee Training Program Timelines and Milestones,” contains 

28 suggested milestones for planning operator licensing activities for cold plants under construction

29 under 10 CFR Part 52, “Licenses, certifications, and approvals for nuclear power plants.”
ES-2.1, Page 15 of 20

1 Figure 1 NRC and Facility Licensee Training Program Timelines and Milestones

2 I. Preparatory Site Visit

3 The NRC examination preparatory site visit, also known as preparation week or validation week, 

4 occurs before the operating test is administered at a facility licensee site. In addition to 

5 operating test validation, the preparatory site visit can also be used for examiner orientation,

6 retrieving additional reference material, auditing license applications in accordance with ES-2.2, 

7 and verifying simulator fidelity under 10 CFR 55.46(c)(1)(i). 

8 • The facility licensee must make the simulator available to examiners for an ample 

9 amount of time during the validation visit. The purpose of this visit is for the NRC 

10 examination team to review and validate the proposed operating test materials.

11

12 • The facility licensee must have available during onsite validation a list of significant 

13 differences between the simulator facility and the reference plant and a list of known 

14 uncorrected simulator performance deficiencies and deviations from the reference plant.

15

16 • The examiners will identify important plant parameters and their intervals to monitor for 

17 each simulator scenario. If available, they will review the facility licensee’s list of 

18 standard recorded parameters to determine whether additional parameters need to be 

19 recorded for simulator scenario administration. The NRC chief examiner will ask the 

20 facility licensee to record selected parameters.
ES-2.1, Page 16 of 20

1

2 • The NRC chief examiner will update Form 2.3-3, “Operating Test Review Worksheets,” if 

3 any issues are discovered during validation with an operating test developed by a facility 

4 licensee.

5

6 • All assigned examiners should attend onsite validation activities if possible. This is the 

7 most efficient and effective means for examiners to become familiar with examination 

8 materials and to provide the chief examiner with feedback on the quality of each 

9 component of the operating tests. It also serves to orient new examiners with the facility, 

10 or to refresh examiners who have previously visited the facility, with site-specific details 

11 such as plant layout and simulator operation. 

12

13 • Under some circumstances, such as the retake of operating tests, validation activities 

14 can be conducted on site just before the scheduled examination administration date. 

15 This alternative to a separate validation/preparatory week minimizes agency costs and 

16 the impact on facility licensee training activities.

17

18 • For those assigned examiners who are unable to participate in onsite validation 

19 activities, the NRC regional office should determine whether a separate preparatory site 

20 visit is necessary and appropriate. When making this determination, the regional office 

21 should carefully weigh the costs and benefits associated with each additional trip to the 

22 facility. The regional office should also consider such factors as the experience of the 

23 assigned examiners, the quality of the facility licensee’s examinations (if applicable), and 

24 the status of the simulation facility (e.g., whether it is new or has been recently 

25 upgraded).

26

27 J. Forms and Letters

28 Form 2.1-1 Examination Preparation Checklist

29 Letter 2.1-1 Sample Corporate Notification Letter

30
ES-2.1, Page 17 of 20

1 Form 2.1-1 Examination Preparation Checklist

2

Facility: ___________________________________ Date of Examination: __________

Developed by: Written: Facility NRC // Operating Facility NRC

Target 

Date* Task Description

Chief 

Examiner’s

Initials

-240 1. Examination administration date confirmed. For NRC-prepared examinations, arrangements are 

made for the facility licensee to submit reference materials. 

-210 2. NRC examiners and facility licensee contact assigned.

-210

3. Facility licensee contact briefed on examination security and other requirements. As applicable, 

the facility licensee contact submits to the NRC any prescreened K/As for elimination from the 

written examination outline, with a description of the facility’s prescreening process. 

-210 4. Reference material due for NRC-prepared examinations. 

-210 5. Examination kick-off call held. The NRC sends the corporate notification letter.

-195 6. Written examination outline developed by the NRC and sent to the facility licensee contact (must 

be on the examination security agreement).

-150 7. Operating test outlines and checklists due: Forms 1.3-1, 2.3-1, 3.2-1, 3.2-2, 3.3-1, and 3.4-1, as 

applicable. Facility licensees provides a draft operating test administration schedule to the NRC.

-136 8. Operating test outline(s) reviewed by the NRC and feedback provided to the facility licensee. 

-75

9. Proposed examinations (written, JPMs, and scenarios, as applicable) and outline forms; quality 

checklists and supporting documentation (including Forms 2.3-2 and 2.3-4 and any Form 2.3-1, 

1.3-1, and 3.4-1 updates); and reference materials due.

-75 10. NRC-prepared examinations approved by the NRC supervisor and forwarded for facility licensee 

review.

-60 11. Preliminary waiver/excusal requests due.

-50 12. Written examination and operating test reviews completed. The NRC supervisor’s authorization 

to proceed with the facility review granted.

-50 13. Examination review results discussed between the NRC and the facility licensee.

-35 14. Examination preparatory week conducted by the NRC and the facility licensee.

-30 15. Preliminary license applications, including any waiver/excusal requests, due.

-14 16. Final license applications, including any waiver/excusal requests, due and Form 2.2-1 prepared.

-7 17. Written examinations and operating tests approved by the NRC supervisor.

-7 18. Facility licensee management feedback on the examination requested by the NRC supervisor.

-7 19. Final applications reviewed; 10% of applications audited to confirm qualifications/eligibility; and 

examination approval letter and waiver/excusal letters sent.

-7 20. Proctoring/written examination administration guidelines reviewed with the facility licensee.

-7 21. Approved scenarios and job performance measures distributed to NRC examiners.

* Target dates are based on licensee-prepared examinations and the examination date identified in the corporate notification 

letter. These dates are for planning purposes and may be adjusted in coordination with the facility licensee.
ES-2.1, Page 18 of 20

1 Letter 2.1-1 Sample Corporate Notification Letter

2 NRC Letterhead

3 (Date)

4

5 (Name, Title)

6 (Name of Facility)

7 (Address)

8 (City, State ZIP Code)

9 Dear (Name):

10 In a telephone conversation on (date) between Mr./Ms. (Name, Title) and Mr./Ms. (Name, Title), 

11 arrangements were made for the administration of operator licensing examinations at (facility 

12 name) during the week(s) of (date).

13 As agreed upon during the telephone conversation, [your staff] [[the staff of the U.S. Nuclear 

14 Regulatory Commission (NRC)]] will prepare the examinations based on the guidelines in 

15 Revision 12 to NUREG-1021, “Operator Licensing Examination Standards for Power Reactors.” 

16 [The NRC’s regional office will discuss with your staff any changes that might be necessary 

17 before the examinations are administered.] [[Your staff will be given the opportunity to review 

18 the examinations during the week of (date).]]

19 [To meet the above schedule, it will be necessary for your staff to furnish the [operating test 

20 outlines by (date). The NRC staff will provide the written examination outline by (date)/The 

21 written examination outline was provided to your staff on (date). The written examinations, 

22 operating tests, and supporting reference materials identified in NUREG-1021, Examination 

23 Standard (ES)-2.1, “Preparing for Operator Licensing Initial Examinations,” will be due by (date). 

24 Pursuant to Title 10 of the Code of Federal Regulations (10 CFR) 55.40(b)(3), an authorized 

25 representative of the facility licensee shall approve the examinations and tests before they are 

26 submitted to the NRC for review and approval. All materials shall be complete and ready to 

27 use.] 

28 We request that any personal, proprietary, sensitive unclassified, or safeguards information in 

29 your response be contained in a separate enclosure and appropriately marked. Delays in 

30 receiving the required materials, or the submittal of inadequate or incomplete materials, may 

31 cause the examinations to be cancelled or rescheduled.

32 To conduct the requested written examinations and operating tests, your staff will need to 

33 provide adequate space and accommodations and to make the simulation facility available on 

34 the dates noted above. In accordance with ES-3.5, “Administering Operating Tests,” your staff 

35 should retain the original simulator performance data (e.g., system pressures, temperatures, 

36 and levels) generated during the dynamic operating tests until the NRC takes licensing action 

37 on all the applications and any adjudicatory actions on any hearing demands are complete.

38 ES-1.2, “Guidelines for Taking NRC Examinations,” contains a number of NRC guidelines for

39 the written examinations and operating tests as they are being administered.

40 To permit timely NRC review and evaluation, your staff should submit preliminary reactor 

41 operator and senior reactor operator waiver or excusal requests (if any) (Office of Management 

42 and Budget (OMB) control number 3150-0090) at least 60 days before the first examination date 

43 (if possible). Contact Mr./Ms. (Name, typically the chief examiner) to determine the method for 
ES-2.1, Page 19 of 20

1 submission of the waiver or excusal requests. Preliminary reactor operator and senior reactor 

2 operator license applications (OMB control number 3150-0090) and medical certifications 

3 (OMB control number 3150-0024) should be submitted at least 30 days before the first 

4 examination date. If the NRC does not receive the preliminary applications at least 30 days 

5 before the examination date, a postponement may be necessary. Final signed applications

6 certifying that all training has been completed and requesting any waivers or excusals, as 

7 applicable, should be submitted at least 14 days before the first examination date.

8 Although the guidelines for waiver or excusal requests call for their receipt at least 30 days 

9 before the first examination date (preliminary) and 14 days before the first examination date 

10 (final), the requests should be submitted as early as possible in the process (see the 60-day 

11 guideline above). Resolutions resulting from verbal inquiries by the licensee to the NRC are not 

12 binding. Submittals addressing waivers or excusals, or both, should be in writing (i.e., using 

13 NRC Form 398 or as directed by Mr./Ms. (Name) when contacting (him or her) to determine the 

14 method for submission). The NRC will document its final decision on whether to grant a waiver 

15 or excusal on the final (not preliminary) NRC Form 398 submitted for the applicant. The NRC 

16 will not provide its decision until the final application is submitted to the agency.

17 Paperwork Reduction Act Statement

18 This letter contains information collection requirements that are subject to the Paperwork 

19 Reduction Act of 1995 (44 U.S.C. 3501 et seq.). These information collections were approved 

20 by the OMB, approval number 3150-0018.

21 The burden to the public for these [voluntary][[mandatory]] information collections is estimated 

22 to average [2,250 hours per examination or response] [[400 hours per examination or 

23 response]], including the time for reviewing instructions, searching existing data sources, 

24 gathering and maintaining the data needed, and completing and reviewing the information 

25 collections. Send comments regarding this information collection to the FOIA, Library, and 

26 Information Collections Branch (T6-A10M), U.S. Nuclear Regulatory Commission, Washington, 

27 DC 20555 0001, or by e mail to Infocollects.Resource@nrc.gov, and to the OMB reviewer 

28 at: OMB Office of Information and Regulatory Affairs (3150-0018), Attn: Desk Officer for the 

29 Nuclear Regulatory Commission, 725 17th Street, NW Washington, DC 20503; e 

30 mail: oira_submission@omb.eop.gov.

31 Public Protection Notification

32 The NRC may not conduct or sponsor, and a person is not required to respond to, a collection 

33 of information unless the document requesting or requiring the collection displays a currently 

34 valid OMB control number.

35 In accordance with 10 CFR 2.390, “Public inspections, exemptions, requests for withholding,” of 

36 the agency’s “Rules of Practice,” a copy of this letter and its enclosures will be available 

37 electronically for public inspection through the Agencywide Documents Access and Management 

38 System (ADAMS). ADAMS is accessible on the NRC’s Web site at http://www.nrc.gov/reading39 rm/adams.html.

40 Thank you for your cooperation in this matter. (Name) has been advised of the policies and 

41 guidelines referenced in this letter. If you have any questions about the NRC’s examination 

42 procedures and guidelines, please contact (name of regional contact) at (telephone number) or 

43 (name of responsible regional supervisor) at (telephone number).
ES-2.1, Page 20 of 20

1 Sincerely,

2

3 (Appropriate regional representative, 

4 Title)

5

6 Docket No.: 50-(number) or 52-(number)

7 Distribution: Public

8 NRC Document Control System

9 Regional Distribution

10

11

12 [ ] Include only for examinations to be prepared by the facility licensee.

13 [[ ]] Include only for examinations to be prepared by the NRC.

14

15
ES-2.2, Page 1 of 22

1 2.2 APPLICATIONS, MEDICAL REQUIREMENTS, AND WAIVER AND 

2 EXCUSAL OF EXAMINATION AND TEST REQUIREMENTS

3 This examination standard provides instructions for preparing and reviewing initial license

4 applications and reapplications. It provides guidance on medical requirements. Last, it provides 

5 guidance on waivers and excusals of examination and test requirements. 

6 A. Background

7 This examiner standard addresses the following regulations from 10 CFR Part 55, “Operators’ 

8 licenses”:

9 • Subpart C—Medical Requirements

10 – 55.21 Medical examination

11 – 55.23 Certification

12 – 55.25 Incapacitation because of disability or illness

13 – 55.27 Documentation

14 • Subpart D—Applications

15 – 55.31 How to apply

16 – 55.33 Disposition of an initial application

17 – 55.35 Re-applications

18 • Subpart E—Written Examinations and Operating Tests

19 – 55.47 Waiver of examination and test requirements

20 The NRC staff, facility licensees, and license applicants should refer to these regulations as 

21 necessary when preparing and reviewing license applications. These regulations include the 

22 following requirements.

23 In accordance with 10 CFR 55.31(a)(4), a license applicant shall do the following:

24 Provide evidence that the applicant has successfully completed the facility 

25 licensee’s requirements to be licensed as an operator or senior operator and of 

26 the facility licensee’s need for an operator or a senior operator to perform 

27 assigned duties. An authorized representative of the facility licensee shall certify 

28 this evidence on Form NRC-398. This certification must include details of the 

29 applicant’s qualifications, and details on courses of instruction administered by 

30 the facility licensee, and describe the nature of the training received at the facility, 

31 and the startup and shutdown experience received. In lieu of these details, the 

32 Commission may accept certification that the applicant has successfully 

33 completed a Commission-approved training program that is based on a systems 

34 approach to training [SAT] and that uses a simulation facility acceptable to the 

35 Commission under [10 CFR] 55.45(b)....

36 As stated in the Statement of Considerations for the 1987 final rule amending 10 CFR Part 55

37 (Volume 52 of the Federal Register (FR), page 9456 (52 FR 9456); March 25, 1987), subject to 

38 continued Commission endorsement of the industry’s accreditation process under the Final 

39 Policy Statement on Training and Qualification of Nuclear Power Plant Personnel (50 FR 11147; 

40 March 20,1985), a facility licensee’s training program would be considered a 

41 “Commission-approved training program” if it is accredited by the National Nuclear Accrediting

42 Board (NNAB). To be accredited by the NNAB, a facility licensee’s training program must, 

43 among other things, implement the National Academy for Nuclear Training (NANT) education 
ES-2.2, Page 2 of 22

1 and experience guidelines (e.g., ACAD 10-001, “Guidelines for Initial Training and Qualification 

2 of Licensed Operators”). The NRC, through a memorandum of agreement with the Institute of 

3 Nuclear Power Operations (Agencywide Documents Access and Management System 

4 (ADAMS) Accession No. ML13129A093), reviews changes to these guidelines and accreditation 

5 objectives and criteria. The NRC has posted a publicly available summary of the most recent 

6 NRC-reviewed NANT operator license eligibility guidelines (i.e., ACAD 10-001) on the agency’s

7 Web site at https://www.nrc.gov/reactors/operator-licensing/licensing-process.html and in 

8 ADAMS at Accession No. ML19053A433. Therefore, one means for meeting the 

9 10 CFR 55.31(a)(4) training and qualification requirements is through NNAB accreditation and 

10 adherence to the NANT guidelines. 

11 Cold licensing of operators is a facility licensee’s method for operations personnel to acquire the 

12 knowledge and experience required for licensed operator duties during the unique conditions of 

13 new plant construction and initial operation. It provides a consistent method for operations 

14 personnel to acquire the knowledge and experience required for licensed operator duties up to 

15 the first refueling outage. Nuclear Energy Institute (NEI) 06-13A, “Template for an Industry 

16 Training Program Description,” describes the cold licensing process. The NRC has determined 

17 that this is an acceptable method for use in combined operating license applications for

18 proposed new plants. The NRC documented its endorsement of this method in a safety 

19 evaluation attached to NEI 06-13A, Revision 2, issued March 2009 (ADAMS Accession 

20 No. ML090910554).

21 When an individual applies for an operator (RO) or senior operator (SRO) license, her or his 

22 health must be sufficient to meet the minimum standards in 10 CFR 55.33(a)(1) for an 

23 unconditioned license or 10 CFR 55.33(b) for a conditional license. The regulations are 

24 intended to prevent the manipulation of the controls by an operator whose medical condition 

25 and general health would cause operational errors endangering public health and safety. When 

26 an applicant requests a conditional license, the NRC will consider the recommendations and 

27 supporting evidence of the facility licensee and the examining physician. The NRC will also use 

28 applicable guidance (American National Standards Institute (ANSI)/American Nuclear Society

29 (ANS) 3.4, “Medical Certification and Monitoring of Personnel Requiring Operator Licenses for 

30 Nuclear Power Plants,” as endorsed by Regulatory Guide (RG) 1.134, “Medical Assessment of 

31 Licensed Operators or Applicants for Operator Licenses at Nuclear Power Plants”) and may 

32 consult a qualified medical expert to review the medical evidence submitted by the applicant to 

33 determine whether the agency should issue a conditional license.

34 In accordance with 10 CFR 55.35, “Re-applications,” an applicant may request to be excused 

35 from a written examination or an operating test requirement. Specifically, the NRC may excuse 

36 an applicant from reexamination on the portions of the written examination or operating test that 

37 the applicant had passed in support of his or her immediately preceding application.

38 In accordance with 10 CFR 55.47, “Waiver of examination and test requirements,” an applicant 

39 may request waiver of a written examination or an operating test requirement. Specifically, the 

40 NRC may waive any or all of the examination and test requirements if it determines that the 

41 applicant has presented sufficient justification. 

42 Under 10 CFR 55.31(b), the Commission may at any time after an application has been filed 

43 require further information under oath or affirmation to enable it to determine whether to grant or 

44 deny the application. 
ES-2.2, Page 3 of 22

1 B. Eligibility: Qualifications and Training for Applicants 

2 Eligibility refers to the necessary qualifications and training for applicants seeking an operator 

3 license. Qualifications include education, experience, and other requirements to meet job 

4 performance criteria. Training refers to an instructional program designed to develop or improve 

5 job performance. 

6 When an NNAB-accredited facility licensee certifies that an applicant has completed a systems 

7 approach to training (SAT)-based operator training program on NRC Form 398, “Personal 

8 Qualification Statement—Licensee,” the facility licensee is also certifying that the applicant 

9 meets or exceeds the NANT guidelines for education and experience.

10 The NRC considers the eligibility guidelines for education and experience promulgated by the 

11 NANT, including those in ACAD 10-001, Revision 1, issued November 2016, as one acceptable 

12 method for meeting 10 CFR 55.31(a)(4). The NRC has reviewed the current education and 

13 experience guidelines outlined in ACAD 10-001, Revision 1, and concludes that they are 

14 equivalent to the NRC guidelines previously promulgated in RG 1.8, Revision 3, “Qualification 

15 and Training of Personnel for Nuclear Power Plants,” issued May 2000 (ADAMS Accession 

16 No. ML003706932). The NRC has posted a publicly available summary of ACAD 10-001 on the 

17 agency Web site at https://www.nrc.gov/reactors/operator-licensing/licensing-process.html and 

18 in ADAMS at Accession No. ML19053A433. Unless otherwise notified, the NRC considers the 

19 NANT guidelines to be the program requirements for NNAB-accredited training programs. The 

20 NRC cannot alter or waive education and experience requirements that are part of the

21 NNAB-accredited facility licensee training program requirements. However, an applicant may 

22 request to defer requirements in order to take the NRC examination. Substitutions allowed by 

23 NANT guidelines for qualifications and training do not require NRC approval. For example, 

24 substitution of related technical training for up to 1 year of experience for an SRO, as allowed by 

25 ACAD 10-001, Revision 1, does not require NRC approval. However, the NRC may request to 

26 review objective quality evidence to confirm that the substitution meets the requirements of the 

27 NANT guidelines. For example, if the applicant is documenting military reactor experience “in a 

28 position” equivalent (or superior) to a licensed reactor operator (e.g., propulsion plant watch 

29 officer, RO, engineering officer of the watch, propulsion plant watch supervisor, or engineering 

30 watch supervisor) as allowed by ACAD 10-001, Revision 1, the NRC may request to review 

31 objective quality evidence to confirm the duration that the applicant was “in a position” as 

32 described above and not just the duration of the applicant’s overall military service time. 

33 Facility licensees should document any deferrals of, or alternatives to, the NANT guidelines in 

34 the “Comments” section of NRC Form 398. 

35 Facility licensees seeking to use alternatives to NNAB accreditation should submit their initial 

36 licensing training program to the NRC operator licensing program office for review and 

37 acceptance as a “Commission-approved training program,” as that term is used in 

38 10 CFR 55.31(a)(4). 

39 Appendix A, “Cold License Training Plan,” to NEI 06-13A, Revision 2, is one acceptable method 

40 for licensed operator eligibility at cold plants. 

41 Neither 10 CFR Part 55 nor Section 107 of the Atomic Energy Act of 1954, as amended,

42 requires operator license applicants to be citizens of the United States; therefore, noncitizens 

43 may apply for an operator license.
ES-2.2, Page 4 of 22

1 C. Instructions for Applicant and Facility Licensee

2 To apply for an RO or SRO license in accordance with 10 CFR 55.31, “How to apply,” an 

3 applicant must submit NRC Form 398 and NRC Form 396, “Certification of Medical Examination 

4 by Facility Licensee.” The application is not complete until both forms are filled out, signed by 

5 the appropriate personnel, and received by the NRC. Each form comes with detailed 

6 instructions for completing it. The agency has posted the current versions of NRC Forms 398

7 and 396 on the NRC’s operator licensing Web page at https://www.nrc.gov/reactors/operator8 licensing/licensing-process.html.

9 The following instructions provide additional details about applications for applicants and facility 

10 licensees. 

11 1. Preliminary Application

12 The licensee should submit preliminary, uncertified operator license applications for review by 

13 the NRC regional office at least 30 days before the examination date. 

14 2. Medical Certification

15 Before licensing, every operator license applicant must have a complete medical examination 

16 certified on NRC Form 396. The facility licensee should submit medical certifications for review 

17 by the NRC regional office before the examination date (usually 30 days before). The names 

18 and license numbers of all medical practitioners (but not laboratory technicians) who were 

19 substantially involved in the examination should be entered on NRC Form 396.

20 RG 1.134 describes methods acceptable to the NRC staff for complying with those portions of 

21 the Commission’s regulations associated with approval or acceptance of the medical 

22 assessment of an applicant for an operator or senior operator license. 

23 Consistent with 10 CFR 55.33(b), if an applicant’s or operator’s general medical condition does 

24 not meet the minimum standards under 10 CFR 55.33(a)(1), the NRC may include conditions in

25 the license to accommodate the medical defect. The NRC will consider the recommendations 

26 and supporting evidence of the facility licensee and the examining physician provided on or with

27 NRC Form 396 in determining the appropriate license condition. The NRC may consult a 

28 qualified medical expert to review the submitted medical evidence to determine whether a 

29 conditional license should be issued.

30 In accordance with 10 CFR 55.31(c), an applicant whose application has been denied because 

31 of a medical condition or general health may submit a further medical report at any time as a 

32 supplement to the application. Any such supplement should be made by submitting updated 

33 NRC Forms 396 and 398.

34 Licensed operators must have medical examinations every 2 years. Licensed ROs or LSROs

35 who apply to upgrade their licenses to SRO licenses do not need to have an additional medical 

36 examination if their medical status as licensed ROs or LSROs is up to date at the time of their 

37 application, including a complete medical examination within the past 2 years.

38 If the time since the applicant’s last complete medical examination is expected to exceed 

39 2 years (use the anniversary month) before the licensing action is completed, the applicant must 

40 be reexamined by a physician and the facility licensee must recertify the applicant’s medical 

41 fitness on NRC Form 396. 
ES-2.2, Page 5 of 22

1 3. Control Manipulations

2 Under 10 CFR 55.31(a)(5), an applicant must provide the number of significant control 

3 manipulations affecting reactivity or power level that the applicant has performed. This is 

4 documented in the “Significant Control Manipulations” section of NRC Form 398. At a minimum, 

5 the applicant must have performed five manipulations on the facility for which the license is 

6 sought or on a plant-referenced simulator. 

7 Applicants should perform diverse control manipulations. The control manipulations must be 

8 performed in accordance with a station-approved procedure. RO and SRO-instant applicants

9 may perform control manipulations in either the at-the-controls or balance-of-plant position. For 

10 SRO-instant applicants, reactivity manipulations while they are in a supervisory position 

11 (i.e., supervising another operator performing the manipulations) do not count towards the five 

12 required control manipulations. 

13 For ROs applying for an SRO license (i.e., SRO-upgrade applicants), documentation of 

14 significant control manipulations on NRC Form 398 is not required; instead, certification that the 

15 operator has successfully operated the controls of the facility as a licensed operator is sufficient.

16 Control manipulations performed on the plant-referenced simulator may be chosen from a 

17 representative sampling of the control manipulations and plant evolutions described in 

18 10 CFR 55.59(c)(3)(i)(A–F), (R), (T), (W), and (X), as applicable to the design of the plant for 

19 which the license application is submitted. Power changes (Items (E) and (F)) performed on the 

20 simulator must be 10 percent or greater in magnitude. For control manipulations on a 

21 plant-referenced simulator to count towards the five required control manipulations, the 

22 plant-referenced simulator must meet the requirements of 10 CFR 55.46(c).

23 4. Certification and Submittal

24 The facility licensee’s senior management representative on site (i.e., an authorized 

25 representative of the facility licensee, such as the plant manager or site vice president) must 

26 certify when an applicant has completed all the facility licensee’s requirements and 

27 commitments for the desired license level (e.g., experience, control manipulations, training, 

28 medical). This certification is made in the “Senior Management Representative on Site” block 

29 on NRC Form 398. The senior management representative must also sign the “Facility 

30 Certification” block on NRC Form 396.

31 Consistent with 10 CFR 55.5, “Communications,” NRC Forms 398 and 396 are submitted to the 

32 NRC by mail; in person; or, where practicable, by using the Electronic Information Exchange or 

33 on CD-ROM. Electronic submissions must be made in a manner that enables the NRC to 

34 receive, read, authenticate, distribute, and archive the submission and process and retrieve it 

35 one page at a time. Detailed guidance on making electronic submissions for operator licensing 

36 documents can be obtained by visiting the NRC’s Operator Licensing public Web site at

37 https://www.nrc.gov/reactors/operator-licensing.html, calling the NRC’s Electronic Filing Help

38 Desk at (866) 672-7640, contacting the NRC at https://www.nrc.gov/site-help/e39 submittals/contact-us-eie.html, or writing to the Office of the Chief Information Officer, 

40 U.S. Nuclear Regulatory Commission, Washington, DC 20555-0001. 

41 Under 10 CFR 55.31(a)(3), an applicant must submit a written request, from an authorized 

42 representative of the facility licensee by which the applicant will be employed, that the written 

43 examination and operating test be administered to the applicant. One acceptable method for

44 satisfying this requirement is the submittal of NRC Forms 398 and 396 that include the signature 
ES-2.2, Page 6 of 22

1 of the senior management representative and a cover letter from the facility licensee listing the 

2 applicants who are scheduled to take the licensing examinations.

3 5. Application Withdrawal

4 The NRC expects an applicant to inform the NRC regional office promptly in writing if the 

5 applicant wishes to withdraw an application. As required by 10 CFR 2.107(a), if the application 

6 is withdrawn before issuance of a notice of hearing, the Commission shall dismiss the 

7 proceeding; 10 CFR 55.35(a) does not apply in this situation.

8 If a facility licensee withdraws its request in writing that the written examination or operating test

9 (or both) be administered to an applicant, the application is incomplete and will not be evaluated 

10 further by the NRC. Because the NRC did not deny the application for a license because of 

11 failure to pass the written examination or operating test (or both), the applicant may not reapply 

12 using the provisions of 10 CFR 55.35(a) as it does not apply in this situation.

13 D. Instructions for NRC Regional Office

14 The following instructions detail the activities that the NRC regional office performs regarding 

15 applications.

16

17 1. Preliminary Application

18 The NRC regional office will review preliminary applications as soon as possible once they are 

19 received. This gives the regional office ample time to process the medical certifications; 

20 evaluate any waiver, deferral, or excusal requests; and obtain any additional information that 

21 might be necessary in order to support the final eligibility determinations after the final certified

22 applications are received. 

23 2. Medical Certification

24 With regard to medical certifications, the NRC regional office shall forward the applicant’s NRC 

25 Form 396 and supporting medical evidence to the NRC’s contract physician for evaluation if any 

26 of the following is true:

27 • The examining physician recommends that the NRC issue a conditional license to the 

28 applicant (except for glasses, corrective lenses, and hearing aids).

29 • The examining physician recommends that the NRC grant the applicant a waiver 

30 (exception) of any requirement set forth in the applicable ANSI/ANS standard.

31 • The examining physician recommends that the NRC change an existing restriction (by 

32 checking applicable boxes on NRC Form 396). 

33 If, on the date of the licensing examination, the NRC’s contract physician is still reviewing an 

34 applicant’s medical certification, the applicant may take the examination, with the understanding 

35 that the NRC will not approve the license application until the health finding of 

36 10 CFR 55.33(a)(1) is made.
ES-2.2, Page 7 of 22

1 3. Request for Additional Information

2 The Commission may, at any time after the application has been filed, require further 

3 information under oath or affirmation in order to enable it to determine whether to grant or deny 

4 the application. 

5 4. Final Certified Application

6 Upon receiving the final certified license application, the NRC regional office will review the data 

7 on NRC Form 398 to ensure that it is complete.

8 The regional reviewer will determine whether the applicant meets the requirements to take the 

9 NRC examination, check the appropriate block on NRC Form 398, and sign and date the form. 

10 The applicant will not be permitted to take the licensing examination until the regional office 

11 determines that the applicant meets the requirements (with the exception of the health 

12 requirement, as discussed above). 

13 If the applicant meets the requirements, the reviewer will enter the applicant’s name, docket 

14 number, and examination requirements on Form 2.2-1. The reviewer must ensure that the list 

15 accurately reflects any examination waivers or excusals that may have been granted.

16 The NRC regional office will send the finalized Form 2.2-1 to the facility licensee separately from 

17 the examination approval letter.

18 If the NRC finds that an application does not comply with the requirements of the Atomic Energy 

19 Act of 1954, as amended, and the Commission’s regulations, then, as required by 

20 10 CFR 2.103(b), the NRC will issue a notice of denial of the application and inform the 

21 applicant in writing of (1) the nature of any deficiencies or the reason for the denial and (2) the 

22 right of the applicant to demand a hearing within 20 days from the date of the notice or such 

23 longer period as may be specified in the notice (Letter 2.2-1, “Sample Initial Application Denial 

24 from Region”). The applicant is responsible for exercising the applicant’s right to demand a 

25 hearing; hearings do not occur absent a demand for a hearing. The regional licensing authority 

26 will discuss denials with the NRR operator licensing program office.

27 Before entering the applicants’ data in the Reactor Program System—Operator Licensing, the 

28 NRC regional office shall verify that none of the applicants’ names appear on the list of 

29 “Escalated Enforcement Actions Issued to Individuals.” The regional office shall check with the 

30 appropriate contact in the Office of Enforcement to determine whether any escalated 

31 enforcement action disqualifies an applicant from receiving a license.

32 5. Application Withdrawal

33 The NRC regional office will notify the applicant if the facility licensee withdrew its request to 

34 administer the written examination or operating test (or both) to the applicant by sending an

35 application withdrawal notification letter to the applicant (see Letter 2.2-2, “Sample Application 

36 Withdrawal Notification Letter”). In this case, the application is considered incomplete.

37 The NRC regional office will issue an application withdrawal response letter (see Letter 2.2-3, 

38 “Sample Application Withdrawal Response Letter,” if an applicant withdraws his or her 

39 application in writing before completing all applicable portions of the written examination or 

40 operating test (or both) required to complete the licensing process. In this case, the applicant’s 
ES-2.2, Page 8 of 22

1 withdrawal is considered an application denial for failure to pass the written examination and 

2 operating test. 

3 E. Reapplication

4 If the applicant is reapplying under 10 CFR 55.35 following a license denial, the applicant must 

5 complete and submit a new NRC Form 398. The applicant may file the second application 

6 2 months after the date of denial of the first application; the third application 6 months after the 

7 date of denial of the second application; and successive applications 2 years after the date of 

8 denial of each subsequent denial. Each new NRC Form 398 shall describe in detail the extent 

9 of the applicant’s additional qualifications and training since the denial and shall include a 

10 certification by the facility licensee that the applicant is ready for reexamination.

11 Section F contains information about requests to be excused from a written examination or an 

12 operating test requirement upon reapplication.

13 A new NRC Form 396 is not required if the date of the original medical examination is within 

14 2 years of the anticipated licensing date (anniversary month) and the “Comments” section of 

15 NRC Form 398 certifies that the applicant has not developed any physical or mental condition 

16 that would be reportable under 10 CFR 55.25, “Incapacitation because of disability or illness.” 

17 Otherwise, a physician shall reexamine the applicant and the facility licensee shall recertify the 

18 applicant’s medical fitness on NRC Form 396. Licensed ROs or LSROs who are reapplying to 

19 upgrade their licenses to SRO licenses do not need to have an additional medical examination if 

20 their medical status as licensed ROs or LSROs is up to date at the time of their application, 

21 including a complete medical examination within the past 2 years.

22 F. Waivers and Excusals

23 The following instructions explain the process for submitting, reviewing, and granting or denying 

24 waivers and excusals.

25

26 1. How To Submit a Waiver or Excusal Request

27 The facility licensee should submit preliminary waiver and excusal requests to the NRC regional 

28 office at least 60 days before the examination date to allow the NRC ample time to review the 

29 substantiating information provided and request additional information if necessary. The facility 

30 licensee should contact the NRC chief examiner or regional office to determine the method of 

31 submission and whether the request requires advance submittal. Failure to allow adequate time 

32 for the NRC to review the justification could result in the denial of the waiver/excusal. 

33 An applicant may request a waiver or excusal of license requirements by checking the 

34 appropriate block on NRC Form 398. The applicant should provide the “category” of the waiver 

35 or excusal being requested. For the written examination, the available categories are RO-level 

36 questions, SRO-only questions, or ALL. For the operating test, the available categories are 

37 administrative topic JPMs, control room system JPMs, in-plant system JPMs, simulator 

38 operating test, or ALL.

39 The applicant should explain the basis and justification for requesting the waiver or excusal in 

40 the “Comments” section of NRC Form 398 and attach any supporting documentation. 
ES-2.2, Page 9 of 22

1 Although the NRC will begin to review waiver or excusal requests upon receipt of the 

2 preliminary request, it will only make a final determination on a waiver or excusal request in 

3 response to a final certified application.

4 If multiple applicants from the same facility request an identical waiver or excusal, the applicants 

5 may refer to a joint justification letter on their individual NRC Form 398s. Such group requests 

6 will be resolved in the same manner as individual requests.

7 If the NRC requires additional information to reach a decision on a waiver or excusal request, 

8 the NRC regional office will generally ask the facility licensee for the necessary information.

9 2. Specific Waivers of Examination and Test Requirements (10 CFR 55.47)

10 Comparable facility licensees are those that have units designed by the same nuclear steam 

11 supply system vendor and are operated at approximately the same power level, with a training 

12 program designed to evaluate training needs for differences between the units. Comparable 

13 facility licensees may request multiunit licensing for their operators at the time of initial licensing. 

14 A waiver is not required for multiunit licensing when the applicant is trained on all applicable 

15 units and the request is made at the time of initial licensing. For multiunit licensing, the 

16 applicant should request the specific units on the NRC Form 398.

17 Note: Individuals will not be examined on or issued concurrent licenses for noncomparable 

18 units, except for LSROs. 

19 For multiunit sites, the “authorized representative of the facility licensee or of a holder of an 

20 authorization by which the applicant was previously employed” (10 CFR 55.47(b)) and the 

21 “authorized representative of the facility licensee or of a holder of an authorization where the 

22 applicant’s services will be utilized” (10 CFR 55.47(c)) could be the same individual. 

23 The NRC will waive any or all of the requirements to take a written examination and operating 

24 test if it finds that an applicant meets the criteria specified in 10 CFR 55.47.

25 The NRC regional office will consult the NRR operator licensing program office for all 

26 10 CFR 55.47 requests.

27 The information in a - d below assists the NRC in making 10 CFR 55.47(a)(1), (a)(2), and (a)(3) 

28 determinations for some common waiver request types. 

29 a. Examination Waiver for Operator Previously Licensed at Comparable Facility

30 One commonly requested waiver is a waiver of licensing examinations for an operator 

31 previously licensed at a comparable facility. 

32 • For this type of waiver request, the NRC has previously found that 520 hours of 

33 operating experience as a licensed operator at a comparable facility in the previous 

34 2 years is “extensive” under 10 CFR 55.47(a)(1). 

35 • Concerning the applicant’s past performance, the following information is potentially 

36 relevant to the 10 CFR 55.47(a)(2) determination that the applicant has discharged his 

37 or her responsibilities competently and safely and is capable of continuing to do so:
ES-2.2, Page 10 of 22

1 – An authorized representative of the applicant’s previous employer makes such a 

2 certification. 

3 – If a certification is used, it must describe the applicant’s operating experience, 

4 including an approximate number of hours the applicant operated the controls of 

5 the facility, the duties performed, and the extent of the applicant’s responsibility.

6 • Concerning the applicant’s current qualifications, the following information is potentially 

7 relevant to the 10 CFR 55.47(a)(3) determination that the applicant has learned the 

8 operating procedures for and is qualified to operate competently and safely the facility 

9 designated in the application:

10 – certification from an authorized representative of the to-be-added facility licensee

11 b. Multiunit Examination Waiver

12 Waiver of examination and test requirements under 10 CFR 55.47 can also be applied to new 

13 reactor construction cases when construction of a subsequent unit is completed after an 

14 operator has received a license on an existing comparable unit. In support of this type of waiver 

15 request, the facility licensee should identify, analyze, and incorporate differences between the 

16 existing unit and the new unit(s) at the same site in its Commission-approved SAT-based 

17 training program. As part of meeting the requirements in 10 CFR 55.31, the applicant and 

18 facility licensee will complete NRC Form 398, describing the applicant’s qualifications and 

19 training or certifying that the applicant has successfully completed the facility licensee’s 

20 SAT-based training program, as required by 10 CFR 55.31(a)(4). The NRC staff will review the 

21 differences between the existing and new units and how these differences were analyzed and 

22 incorporated in the facility licensee’s SAT-based training program to determine whether 

23 examination requirements can be waived.

24 • For this type of waiver request, the NRC has previously found that 520 hours of 

25 operating experience as a licensed operator on a comparable unit in the previous 

26 2 years is “extensive” under 10 CFR 55.47(a)(1). 

27 • Concerning the applicant’s past performance, the following information is potentially 

28 relevant to the 10 CFR 55.47(a)(2) determination that the applicant has discharged his 

29 or her responsibilities competently and safely and is capable of continuing to do so:

30 – An authorized representative of the facility licensee makes such a certification.

31 – If a certification is used, it must describe the applicant’s operating experience, 

32 including an approximate number of hours the applicant operated the controls of 

33 the facility, the duties performed, and the extent of the applicant’s responsibility.

34 • Concerning the applicant’s current qualifications, the following information is potentially 

35 relevant to the 10 CFR 55.47(a)(3) determination that the applicant has learned the 

36 operating procedures for and is qualified to operate competently and safely the facility 

37 designated in the application: 

38 – details of the facility licensee training program, including how the applicant was 

39 trained on differences between the comparable units at the site 

40 – a summary of difference training for the following subjects, as applicable:
ES-2.2, Page 11 of 22

1  facility design and systems relevant to control room personnel

2  technical specifications

3  procedures (primarily abnormal and emergency operating)

4  control room design and instrument location

5  operational characteristics

6  administrative procedures related to conduct of operations at a multiunit 

7 site (e.g., shift staffing and response to accidents and fires)

8  the expected method of rotating personnel between units and the 

9 familiarization training to be conducted before an operator assumes 

10 responsibility on a new unit

11 – statement that the applicant passed a comprehensive “differences” written 

12 examination

13 – statement that the applicant passed JPMs that tested the applicant on 

14 differences in operating equipment for the comparable units

15 – statement that the simulator supports training on the additional unit(s)

16 c. Waiver of Reactor Operator Portion of Written Examination for Senior Reactor 

17 Operator-Upgrade Applicants

18 If an individual is currently licensed as an RO at a facility and applies for an SRO license at the 

19 same facility, the NRC may waive the requirement for the applicant to take the RO portion of the 

20 SRO written examination based on the following considerations:

21 • The requirement in 10 CFR 55.47(a)(1) for extensive actual operating experience within 

22 the previous 2 years would be satisfied if the SRO-upgrade applicant maintained an 

23 active RO license for at least 12 of the 24 months preceding the date of the application. 

24 • Concerning the applicant’s past performance, the following information is potentially 

25 relevant to the 10 CFR 55.47(a)(2) determination that the applicant has discharged his 

26 or her responsibilities competently and safely and is capable of continuing to do so:

27 – An authorized representative of the facility licensee makes such a certification. 

28 – If a certification is used, it must describe the applicant’s operating experience, 

29 including an approximate number of hours the applicant operated the controls of 

30 the facility, the duties performed, and the extent of the applicant’s responsibility.

31 • Concerning the applicant’s current qualifications, the following information is potentially 

32 relevant to the 10 CFR 55.47(a)(3) determination that the applicant has learned the 

33 operating procedures for and is qualified to operate competently and safely the facility 

34 designated in the application: 
ES-2.2, Page 12 of 22

1 – The applicant passed his or her most recent requalification examination and was 

2 up to date in the facility licensee’s requalification training program at the time that 

3 he or she entered the SRO-upgrade training program.

4 d. Waiver of Examination Requirements for an Operator Previously Licensed at a Facility

5 If an applicant was previously licensed at a facility and later reapplies for a license at that same 

6 facility at the same or lower license level, the NRC may, under 10 CFR 55.47, waive any or all 

7 of the requirements for a written examination and operating test based on the following 

8 considerations:

9 • The requirement in 10 CFR 55.47(a)(1) for extensive actual operating experience within 

10 the previous 2 years would be satisfied if the applicant terminated participation in the 

11 facility licensee’s requalification program less than 2 years (24 months) before the date 

12 of the license application.

13 • Concerning the applicant’s past performance, the following information is potentially 

14 relevant to the 10 CFR 55.47(a)(2) determination that the applicant has discharged his 

15 or her responsibilities competently and safely and is capable of continuing to do so:

16 – An authorized representative of the facility licensee makes such a certification. 

17 – If a certification is used, it must describe the applicant’s operating experience, 

18 including an approximate number of hours the applicant operated the controls of 

19 the facility, the duties performed, and the extent of the applicant’s responsibility.

20 • Concerning the applicant’s current qualifications, the following information is potentially 

21 relevant to the 10 CFR 55.47(a)(3) determination that the applicant has learned the 

22 operating procedures for and is qualified to operate competently and safely the facility 

23 designated in the application: 

24 – The applicant successfully completed additional training under 10 CFR 55.59(b) 

25 and a written examination and operating test prepared by the licensee that

26 ensure that the applicant is up to date in the licensed operator requalification 

27 training program (including generic fundamental topics).

28 – The applicant will successfully complete at least 40 hours of shift functions under 

29 the direction of an operator or senior operator, as appropriate, and in the position 

30 to which the applicant will be assigned before being assigned to licensed duties.

31 (This is required in 10 CFR 55.53(f)(2) for an applicant who was not actively 

32 performing the functions of an operator or senior operator.)

33 3. Specific Excusals

34 The information in a - b below assists the NRC in reviewing common types of excusal requests. 

35 a. Reapplication with Excusal

36 If an applicant failed only one of the following portions of the site-specific licensing examination, 

37 the NRC regional office may excuse the applicant from reexamination for those examination 

38 areas that the applicant passed: the written examination overall, the SRO-only section of the 

39 written examination, the simulator operating test, the walkthrough overall, or the administrative 
ES-2.2, Page 13 of 22

1 topics portion of the walkthrough. This excusal is only applicable to those portions of the 

2 licensing examination that the applicant passed as part of his or her immediately prior 

3 application, not as part of earlier applications. 

4 The facility licensee should follow its SAT-based training program to remediate applicants 

5 before a retake examination and include remediation on deficiencies identified in the portions of 

6 the examination that the applicant passed as well as the portions of the examination that the 

7 applicant failed. 

8 The applicant/facility licensee should request the excusal in writing by completing the 

9 “EXCUSAL” section on NRC Form 398 and include supporting documentation such as the 

10 following to justify the excusal request:

11 • a statement in the “Comments” section of NRC Form 398 affirming that the applicant 

12 was fully remediated in accordance with the facility licensee’s Commission-approved,

13 SAT-based training program

14 • documentation showing the content and scope of remediation and retraining efforts 

15 completed by the applicant since the previous examination failure

16 • the content of the testing and evaluations that the applicant has completed since the 

17 previous examination failure, including his or her results

18 • evidence of the applicant’s participation in the facility licensee’s licensed operator 

19 requalification training program since the previous examination failure, including the 

20 results of any evaluations of the applicant

21 The applicant/facility licensee should describe on NRC Form 398 (and in supporting 

22 documentation) the extent of the applicant’s additional qualifications and training since the 

23 denial and include a certification by the facility licensee that the applicant is ready for 

24 reexamination. Additional training should include training on weaknesses identified in any 

25 portion of the NRC examination and is not limited to only training on the failed portion of the 

26 examination.

27 The NRC regional office evaluates the request for the following:

28 • Sufficient justification is presented for approving and excusal.

29 • The request indicates that applicant has taken additional training and shows that the 

30 facility licensee made a reasonable effort to remediate the deficiencies that caused the 

31 applicant to fail the previous examination.

32 • Training on weaknesses identified on the passed portions of the examination ensures 

33 that the knowledge has been retained and justifies an excusal of those portions of the 

34 examination.

35 The office should excuse the applicant from reexamination on the portions of the examination or 

36 test that he or she passed with sufficient evidence of remediation, retesting, and retraining on all 

37 identified deficiencies, performed in accordance with the facility licensee’s SAT-based training 

38 program.
ES-2.2, Page 14 of 22

1 For example, an SRO-instant applicant who passed the operating test and achieved a score of

2 80 percent on the RO portion of the written examination, 76 percent on the SRO-only questions, 

3 and 79 percent overall would not be eligible for an excusal from the RO portion because the 

4 overall 80-percent cut score was not achieved. 

5 However, an SRO-upgrade applicant who passed the operating test and achieved a score of 

6 80 percent on the RO portion of the written examination, 76 percent on the SRO-only questions, 

7 and 79 percent overall would be eligible for a waiver of the RO portion if the applicant meets the 

8 three requirements for this type of waiver specified above. 

9 b. SRO-Instant Applicant to RO Applicant

10 An SRO-instant applicant who passed everything except the SRO-only portion of the written 

11 examination may, upon accepting the denial of the applicant’s SRO-instant application, submit a 

12 new application for an RO license and ask the NRC to consider the applicant’s passing of the 

13 SRO operating test and written examination, with the exception of the SRO-only portion of the 

14 written examination, to satisfy the requirement in 10 CFR 55.33, “Disposition of an initial 

15 application,” that the applicant pass an RO written examination and operating test. The NRC’s 

16 review of the request will determine whether the new application does the following:

17 • satisfies 10 CFR 55.31 with respect to an RO license (including the applicant’s eligibility 

18 for an RO license) 

19 • demonstrates that the applicant satisfies the “health” requirement in 10 CFR 55.33 

20 • states that the operating test that was passed provided evidence of the applicant’s 

21 control board competence and administrative duties to satisfy the requirements in 

22 10 CFR 55.45, “Operating tests,” for an RO operating test 

23 • states that the portions of the written examination that the applicant passed satisfy the 

24 requirements in 10 CFR 55.41, “Written examination: Operators” 

25 The NRC regional office must obtain concurrence from the NRR operator licensing program 

26 office before approving this type of excusal request.

27 4. Grant or Denial of Waiver or Excusal Request

28 If the NRC regional office finds that an applicant does not satisfy the requirements for waiver or 

29 excusal of examination requirements, then the region must obtain concurrence from the NRR 

30 operator licensing program office before denying the applicant’s request. 

31 The NRC regional office shall document the disposition of every waiver or excusal request, 

32 whether it is granted or denied, in the block designated “For NRC Use” on the applicant’s final, 

33 certified NRC Form 398 and by entering the data in the Reactor Program System—Operator 

34 Licensing.

35 The NRC regional office shall promptly notify the applicant in writing if the waiver or excusal 

36 request is denied and provide an explanation. If there is insufficient time to notify the applicant 

37 in writing before the examination date, the regional office shall notify the facility licensee training 

38 representative by telephone concerning the denial of the waiver or excusal request and provide 

39 a followup written notification to the applicant.
ES-2.2, Page 15 of 22

1 The region tracks all approved waivers and excusals using Form 2.2-1. 

2 G. Deferrals

3 The following instructions explain the process for submitting, reviewing, and granting or denying 

4 deferrals.

5

6 1. How To Submit

7 Deferrals allow an applicant to complete some license application requirement(s) at a later time 

8 and still take the licensing examinations. The applicant must complete any deferred 

9 requirements before the NRC will issue a license to the applicant. An applicant may request to 

10 defer a license requirement by checking the appropriate block on NRC Form 398.

11 If multiple applicants from the same facility request an identical deferral, the applicants may 

12 refer on their individual NRC Form 398s to a joint justification letter. Such group requests will be 

13 resolved in the same manner as individual requests.

14 2. Specific Deferrals

15 The NRC regional office will evaluate deferral requests on a case-by-case basis and use the 

16 criteria discussed below for granting specific deferral types. For all other deferrals, the regional 

17 office shall consult with the NRR operator licensing program office before granting or denying 

18 the request.

19 a. Deferral of Significant Control Manipulations

20 If an applicant is unable to perform the five significant control manipulations required by 

21 10 CFR 55.31(a)(5), the regional office may process the application, administer the licensing

22 examinations to the applicant, and delay any issuance of the license to the applicant until the 

23 facility licensee certifies that the required manipulations have been completed successfully. 

24 In the case of plants under construction (i.e., “cold plants”), the NRC regional office may grant 

25 an applicant’s request to defer the five significant control manipulations until they can be 

26 completed on a plant-referenced simulator.

27 b. Deferral of Specific Eligibility Items

28 The NRC regional office may grant an applicant’s request to defer completion of the specific 

29 eligibility items until after the licensing examinations are passed.

30 For cold plant operator license applicants, the NRC regional office may grant an applicant’s 

31 request to defer completion of a portion or all of the practical and meaningful work assignments 

32 detailed in NEI 06-13A.

33 The NRC regional office may administer a license examination to an applicant who has not 

34 satisfied the applicable education or experience requirements at the time of the examination but 

35 is expected to complete them.
ES-2.2, Page 16 of 22

1 3. Denial of Deferral Request

2 The NRC regional office shall obtain concurrence from the NRR operator licensing program 

3 office before denying an applicant’s request to defer license application requirements until after 

4 taking the examination.

5 H. License Application Audit

6 While on site, the NRC regional office shall audit a sample (approximately 10 percent) of the 

7 license applications (i.e., NRC Form 398) to confirm that they accurately reflect the subject 

8 applicants’ qualifications. The review should focus primarily on the applicants’ experience and 

9 on-the-job training, including reactivity manipulations, to ensure that they comply with 

10 10 CFR Part 55 and the facility’s licensing-basis documents and licensed operator training 

11 program description. For trending purposes, the regional office should inform the NRR operator

12 licensing program office of application deficiencies found during this audit.

13 When a facility licensee’s licensed operator training program description or licensing-basis 

14 documents contain education and experience requirements that are more restrictive than the 

15 current set of NRC-reviewed NANT guidelines (for NNAB-accredited training programs), the 

16 most restrictive requirements will continue to apply, pending the initiation of action by the facility 

17 licensee to amend these requirements. 

18 I. Forms and Letters

19 Form 2.2-1 List of Applicants

20 Letter 2.2-1 Sample Initial Application Denial from Region

21 Letter 2.2-2 Sample Application Withdrawal Notification Letter

22 Letter 2.2-3 Sample Application Withdrawal Response Letter

23
ES-2.2, Page 17 of 22

Form 2.2-1 List of Applicants

PRIVACY ACT INFORMATION—FOR OFFICIAL USE ONLY

Facility: Written Examination Date:

Operating Test Dates:

Applicant Name Docket 

No.

Exam 

Level

Written Operating Test

RO SRO Adm. Sys. Sim.

Instructions: For each approved applicant, enter the exam level (RO, SRO-I, or SRO-U) and an 

“X,” “E,” or “W” to indicate whether each portion of the examination is to be 

administered, excused, or waived.

Note: This form is provided separately to the facility licensee following examination approval.

PRIVACY ACT INFORMATION—FOR OFFICIAL USE ONLY
ES-2.2, Page 18 of 22

Letter 2.2-1 Sample Initial Application Denial from Region

NRC Letterhead

(date)

(Applicant’s Name)

(Street Address)

(City, State ZIP Code)

Dear (Mr./Ms. Applicant’s Name):

This letter is to inform you that your application, dated (date), for a (reactor operator, senior 

reactor operator, or limited senior reactor operator) license, submitted in connection with (facility 

name), is hereby denied.

(Region to discuss deficiencies and which part of 10 CFR 55.31, ES-2.2, or the NRC-approved 

facility training program was involved.) When you have met the requirements, you may submit 

another application.

Under Title 10 of the Code of Federal Regulations (10 CFR) 2.103(b)(2), you may demand an 

adjudicatory hearing about this denial of your application within 20 days after the date of this 

letter. Under 10 CFR 2.307(a), you may request an extension of this time limit if you can show 

good cause.

A demand for a hearing shall be filed in accordance with 10 CFR Part 2, “Agency Rules of 

Practice and Procedure,” which is accessible electronically on the U.S. Nuclear Regulatory 

Commission’s (NRC’s) Web site at http://www.nrc.gov/reading-rm/doc-collections/cfr/. 

Generally, a demand for a hearing should explain why you believe that the NRC denied your 

application in error and why you believe that you have, in fact, satisfied the requirements for 

license issuance. 

[Consult with the Office of the General Counsel/regional counsel for the most up-to-date E-Filing 

language.]

A demand for a hearing must be filed in accordance with the NRC’s E-Filing rule (Volume 72 of 

the Federal Register, page 49139; August 28, 2007). The E-Filing process requires participants 

to submit and serve all adjudicatory documents over the Internet, or in some cases to mail 

copies on electronic storage media. Detailed guidance on making electronic submissions may 

be found in the Guidance for Electronic Submissions to the NRC and on the NRC Web site at

https://www.nrc.gov/site-help/e-submittals.html. Participants may not submit paper copies of 

their filings unless they seek an exemption in accordance with the procedures described below.

To comply with the procedural requirements of E-Filing, at least 10 days before the filing 

deadline, the participant should contact the Office of the Secretary by e-mail to

hearing.docket@nrc.gov or by telephone at 301-415-1677 to (1) request a digital identification 

(ID) certificate, which allows the participant (or its counsel or representative) to digitally sign 

submissions and access the E-Filing system for any proceeding in which it is participating and 

(2) advise the Secretary that the participant will be submitting a demand for a hearing. Based 

upon this information, the Secretary will establish an electronic docket for the hearing in this 

proceeding if the Secretary has not already established an electronic docket. 
ES-2.2, Page 19 of 22 

Information about applying for a digital ID certificate is available on the NRC’s public Web site at 

https://www.nrc.gov/site-help/e-submittals/getting-started.html. Once a participant has obtained 

a digital ID certificate and a docket has been created, the participant can then submit a demand 

for a hearing. Submissions must be in Portable Document Format (PDF). Additional guidance 

on PDF submissions is available on the NRC’s public Web site at https://www.nrc.gov/sitehelp/electronic-sub-ref-mat.html. A filing is considered complete at the time the document is 

submitted through the NRC’s E-Filing system. To be timely, an electronic filing must be 

submitted to the E-Filing system no later than 11:59 p.m. eastern time on the due date. Upon 

receipt of a transmission, the E-Filing system time-stamps the document and sends the 

submitter an e-mail notice confirming receipt of the document. The E-Filing system also 

distributes an e-mail notice that provides access to the document to the NRC’s Office of the 

General Counsel and any others who have advised the Office of the Secretary that they wish to 

participate in the proceeding, so that the filer need not serve the document on those participants 

separately. Therefore, applicants and other participants (or their counsel or representative) 

must apply for and receive a digital ID certificate before adjudicatory documents are filed so that 

they can obtain access to the documents through the E-Filing system.

A person filing electronically using the NRC’s adjudicatory E-Filing system may seek assistance 

by contacting the NRC’s Electronic Filing Help Desk through the “Contact Us” link located on the 

NRC’s public Web site at https://www.nrc.gov/site-help/e-submittals.html, by e-mail to 

MSHD.Resource@nrc.gov, or by a toll-free call at 1-866-672-7640. The NRC Electronic Filing 

Help Desk is available between 9 a.m. and 6 p.m., eastern time, Monday through Friday, 

excluding Government holidays. 

Participants who believe that they have a good cause for not submitting documents 

electronically must file an exemption request, in accordance with 10 CFR 2.302(g), with their 

initial paper filing stating why there is good cause for not filing electronically and requesting 

authorization to continue to submit documents in paper format. Such filings must be submitted 

by (1) first class mail addressed to the Office of the Secretary of the Commission, U.S. Nuclear 

Regulatory Commission, Washington, DC 20555-0001, Attention: Rulemaking and 

Adjudications Staff, or (2) courier, express mail, or expedited delivery service to the Office of the 

Secretary, 11555 Rockville Pike, Rockville, Maryland 20852, Attention: Rulemaking and 

Adjudications Staff. Participants filing adjudicatory documents in this manner are responsible 

for serving the document on all other participants. Filing is considered complete by first-class 

mail as of the time of deposit in the mail, or by courier, express mail, or expedited delivery 

service upon depositing the document with the provider of the service. A presiding officer, 

having granted an exemption request from using E-Filing, may require a participant or party to 

use E-Filing if the presiding officer subsequently determines that the reason for granting the 

exemption from use of E-Filing no longer exists. 

Documents submitted in adjudicatory proceedings will appear in the NRC’s electronic hearing 

docket, which is available to the public at https://adams.nrc.gov/ehd, unless excluded pursuant 

to an order of the Commission or the presiding officer. If you do not have an NRC-issued digital 

ID certificate as described above, click “cancel” when the link requests certificates and you will 

be automatically directed to the NRC’s electronic hearing dockets where you will be able to 

access any publicly available documents in a particular hearing docket. Participants are 

requested not to include personal privacy information, such as social security numbers, home 

addresses, or personal phone numbers in their filings, unless an NRC regulation or other law 

requires submission of such information. For example, in some instances, individuals provide 

home addresses in order to demonstrate proximity to a facility or site. With respect to 

copyrighted works, except for limited excerpts that serve the purpose of the adjudicatory filings 
ES-2.2, Page 20 of 22

and would constitute a Fair Use application, participants are requested not to include 

copyrighted materials in their submission. 

If you have any questions, please contact (name) at (telephone number).

Sincerely,

(Regional Branch Chief or above)

Docket No. 55-(number)

cc: (Facility representative who signed the applicant’s NRC Form 398)

CERTIFIED MAIL—RETURN RECEIPT REQUESTED
ES-2.2, Page 21 of 22

Letter 2.2-2 Sample Application Withdrawal Notification Letter

NRC Letterhead

(Date)

(Applicant’s Name)

(Street Address) 

(City, State ZIP Code)

Dear (Mr./Ms. Applicant’s Name):

The purpose of this letter is to inform you that the facility licensee has withdrawn its request that 

the written examination/operating test for a (reactor operator/senior reactor operator/limited 

senior reactor operator) license be administered to you; therefore, your application is incomplete 

and will not be evaluated further by the U.S. Nuclear Regulatory Commission (NRC). This is not 

considered a denial under Title 10 of the Code of Federal Regulations (10 CFR) 55.35(a) 

because your application is not being denied because of failure to pass the written examination 

or operating test, or both. Therefore, you may file a new application at any time.

[Additionally, you passed the (written examination or operating test) and have not yet taken the 

(operating test or written examination). Though you did not actually fail the (written examination 

or operating test), for the purposes of 10 CFR 55.35(b), which provides the NRC the ability to 

excuse portions of an examination that have been passed, the NRC considers you to have 

failed the (operating test or written examination) that you have not yet taken. Therefore, in any 

new application that you file with the NRC, you may request to be excused from reexamination 

on the portions of the examination or test that you have passed. The NRC may, in its discretion, 

grant this request if it determines that you have presented sufficient justification.]

If you have any questions, please contact (name) at (telephone number).

Sincerely,

(Name and title of licensing official)

Docket No. 55-(XXXXX)

cc: (Facility representative who signed the applicant’s NRC Form 398)

[ ] Use only for applicants who have passed either the written examination or operating 

test.
ES-2.2, Page 22 of 22

Letter 2.2-3 Sample Application Withdrawal Response Letter

NRC Letterhead

(Date)

(Applicant’s Name)

(Street Address) 

(City, State ZIP code)

Dear (Mr./Ms. Applicant’s Name):

In response to your letter received by this office on [date], you specified that you are 

withdrawing your application for a (reactor operator/senior reactor operator/limited senior reactor 

operator) license.

[For the purposes of Title 10 of the Code of Federal Regulations (10 CFR) 55.35(a), the

U.S. Nuclear Regulatory Commission considers that your application is denied as a result of the 

failure to pass the written examination, operating test, or both. Therefore, you must comply with 

the time limits of 10 CFR 55.35(a) for the filing of a new application.]

[[You passed the (written examination or operating test) and have not yet taken the (operating 

test or written examination). For the purposes of Title 10 of the Code of Federal Regulations

(10 CFR) 55.35(a), the U.S. Nuclear Regulatory Commission (NRC) is considering your 

application as having been denied because of the failure to pass the (operating test or written 

examination). Therefore, you must comply with the time limits of 10 CFR 55.35(a) for the filing 

of a new application.

However, since the NRC is considering you to have failed the (operating test or written 

examination) that you have not yet taken, pursuant to 10 CFR 55.35(b), you may, in any new 

application that you file, request to be excused from reexamination on the portions of the 

examination or test which you have passed. The NRC may, in its discretion, grant this request if 

it determines that you have presented sufficient justification.]]

If you have any questions, please contact (name) at (telephone number).

Sincerely,

(Name and title of licensing official)

Docket No. 55-(XXXXX)

cc: (Facility representative who signed the applicant’s NRC Form 398)

[ ] Use if the applicant has taken neither the written examination nor the operating test.

[[ ]] Use if the applicant has passed either the written examination or the operating test.
ES-2.3, Page 1 of 19

1 2.3 REVIEWING AND APPROVING OPERATOR LICENSING INITIAL 

2 EXAMINATIONS

3 Pursuant to 10 CFR 55.40(a)-(b)(1), power reactor facility licensees that elect to prepare the 

4 required written examinations and operating tests shall prepare the required examinations and 

5 tests in accordance with the criteria in NUREG-1021 in effect 6 months before the examination 

6 date. Pursuant to 10 CFR 50.40(b)(3), an authorized representative of the power reactor facility 

7 licensee shall approve the required examinations and tests before they are submitted to the 

8 Commission for review and approval. Pursuant to 10 CFR 55.40(b)(4), power reactor facility 

9 licensees that elect to prepare examinations and tests must receive Commission approval of 

10 their proposed written examinations and operating tests.

11 This examination standard contains instructions for reviewing and approving examinations and 

12 operating tests.

13 A. General Guidance for Reviewing Written Examination Questions

14 When reviewing written examination questions, reviewers should try to put themselves in the 

15 position of examination applicants by attempting to answer the questions with only the 

16 information available at the time of the examination and without referring to the answer key. 

17 Reviewers of the written examination can assist examination authors by checking the technical 

18 content, level of difficulty, psychometrics, and editorial quality. The reviewer should consider 

19 each of these four areas separately and in this order. If an item needs revision during one stage 

20 of the review, the changes should be made before performing the next round of evaluation for

21 the other review criteria because the changes at each stage may affect the subsequent reviews. 

22 For example, a criticism that appears to affect only one distractor may ultimately lead to 

23 changes in other parts of the item, so time spent reviewing the item for grammar and 

24 punctuation may be wasted.

25 There are also some advantages associated with having the written questions reviewed for 

26 clarity, grammar, expression, spelling, and punctuation by someone who is not familiar with the 

27 area being tested. Such a reviewer can determine whether an item can be answered correctly 

28 by a person without knowledge of the field.

29 The examination author and reviewers should ask the following types of questions:

30 • Will the examinees clearly understand the prompt of the question?

31 • Do they have all the information they need to work with?

32 • Does answering the question correctly depend on certain assumptions that must be 

33 stated?

34 B. Instructions for Facility Licensee Management

35 These instructions provide additional detail for associated steps for review and approval listed in

36 ES-2.1, “Preparing for Operator Licensing Initial Examinations.”
ES-2.3, Page 2 of 19

1 1. Review Outlines and Examinations/Tests

2 a. Pursuant to 10 CFR 55.40(b)(1) and (3), power reactor facility licensees may prepare the 

3 written examinations required by 10 CFR 55.41 and 10 CFR 55.43, “Written 

4 examination: Senior operators,” and may prepare the operating tests required by 

5 10 CFR 55.45, in accordance with the criteria in NUREG-1021 as described in 

6 10 CFR 55.40(a). In addition, an authorized representative of the power reactor facility 

7 licensee shall approve the required examinations and tests before they are submitted to 

8 the Commission for review and approval. The NRC recommends that a facility

9 supervisor or manager familiar with the examination standards in this NUREG 

10 independently review the following items and submit feedback to the NRC regional office 

11 if any changes are necessary:

12 • the proposed written examination outline and operating test outline

13 • the proposed operating test before it is approved by the authorized 

14 representative of the facility for submittal to the NRC regional office for review 

15 and approval

16 • the proposed written examination before it is approved by the authorized 

17 representative of the facility for submittal to the NRC regional office for review 

18 and approval

19 b. If the facility licensee developed the written examination or operating test, then, as 

20 required by 10 CFR 55.40(b)(2), pursuant to 10 CFR 55.49, “Integrity of examinations 

21 and tests,” the power reactor facility licensees shall establish, implement, and maintain 

22 procedures to control examination security and integrity. The NRC chief examiner is 

23 expected to use his or her best judgment and take reasonable measures, including 

24 selective review of reference materials and past tests, to verify these attributes.

25 c. The licensee should review the final written examination outline and operating test 

26 outline using the criteria on Form 2.3-1, “Examination Outline Quality Checklist.”

27 d. The licensee should review the operating test using the criteria on Forms 2.3-2, 

28 “Operating Test Quality Checklist.”

29 e. The licensee should review the written examination using the criteria on Form 2.3-4, 

30 “Written Examination Quality Checklist.” 

31 f. As a final check of the written examination’s technical accuracy, facility licensee 

32 management should consider administering the examination (under security 

33 agreements) to one or more licensed personnel who were previously uninvolved in 

34 developing the examination. Any comments and problems identified during the trial 

35 administration should be discussed with the NRC chief examiner and resolved before 

36 the examination is administered to the license applicants. The review is intended to 

37 identify and correct deficiencies that may affect the technical accuracy of the 

38 examination.

39 g. The licensee should sign all review forms and include them (as applicable) in the outline 

40 and examination submittal package. The outline and examination package should be 

41 submitted to the NRC for review and approval in accordance with the timeline 

42 established (refer to Form 2.1-1).
ES-2.3, Page 3 of 19

1 h. Facility licensees are responsible for ensuring that contractor-prepared examinations 

2 meet the specified criteria and are encouraged to verify the origins of the questions used 

3 to construct the examination.

4 2. Meet with the NRC To Discuss Reviews

5 Facility licensee management should meet with the NRC chief examiner to review and discuss 

6 all substantive comments from the quality reviews of the examination outlines and proposed 

7 examinations. These meetings will normally be conducted by telephone, but, with approval from 

8 NRC regional management and agreement of both parties, they may be conducted in the NRC

9 regional office or at the facility.

10 C. Instructions for NRC Chief Examiner

11 The following instructions provide additional detail for associated steps for review and approval 

12 listed in ES-2.1, “Preparing for Operator Licensing Initial Examinations.”

13 1. Conduct Prompt and Independent Reviews

14 a. Ensure the examination outlines, written examination, and operating test are 

15 independently reviewed for quality in accordance with the applicable checklists included 

16 in this examination standard.

17 Note: If the NRC chief examiner wrote any portion of the outline or examination, another 

18 NRC examiner will perform the independent review of that portion(s).

19 b. Notify the regional supervisor if any of the facility-developed examination materials 

20 (written, walkthrough, or simulator) require substantive changes and cannot be made to 

21 conform with the examination standards by the time the examination is scheduled to 

22 begin.

23 Follow review timeline targets established in Form 2.1-1 or the dates established during the 

24 kick-off call. Review the examination as soon as possible after receipt so that supervisory 

25 approval can be obtained before the examination review comments are shared with the facility 

26 licensee, which is normally scheduled about 7 weeks before the administration date.

27 c. Consolidate the comments from other NRC reviewers and create one set of comments.

28 2. Review Outlines

29 a. Ensure that the examination outlines are reviewed before the facility licensee begins

30 examination development and then again before approving the written examination and 

31 operating test for administration. Use Form 2.3-1.

32 • Conduct a timely review (i.e., within 5 working days) to minimize the potential for 

33 significant problems with the examination during its development.

34 • Document any necessary changes.

35 • Indicate completion of this review by signing the bottom of Form 2.3-1.
ES-2.3, Page 4 of 19

1 b. Forward the outlines and review comments to the responsible NRC supervisor for review 

2 and comment before resolving any deficiencies with the author or facility licensee

3 contact. 

4 c. Document NRC chief examiner review/concurrence, as applicable, by signing the bottom 

5 of the form. 

6 3. Review Operating Test and Written Examination

7 a. Operating Test

8 • Review each test item using Form 2.3-3 to document the NRC’s review of each test item 

9 and subsequent comment resolution. Account for any changes between the original 

10 submittal and the final approved version of the operating test and the reason for the 

11 changes as follows:

12 a. For test items originally marked as unsatisfactory, include the following 

13 information:

14 1. how the attribute(s) was not being met

15 2. what actions the facility licensee actions took to fix or replace the 

16 unsatisfactory test item

17 b. For test items originally marked as needing enhancements, include information 

18 about the nature of the enhancements. 

19 c. For minor editorial changes that do not impact the intent or framework of the test 

20 item, include a comment such as “minor editorial changes required.”

21 • Check the items listed on Form 2.3-2.

22 • Review and document any changes that need to be made and forward the tests and 

23 completed Form 2.3-3 to the responsible NRC supervisor for review and comment in 

24 accordance with Section D before reviewing the examinations with the facility licensee. 

25 There are no minimum or maximum limits on the number or scope of changes the NRC 

26 chief examiner may direct the author or facility licensee contact to make to the proposed

27 tests, if the changes are necessary to make the tests conform to established acceptance 

28 criteria. 

29 • Upon NRC supervisor approval of the changes that need to be made, review the 

30 operating test with the facility licensee.

31 • After reviewing the operating test with the facility licensee, ensure that all comment and 

32 recommendation resolutions are documented and the necessary revisions to the test are 

33 made. Update Form 2.3-3 to reflect how the comments/recommendations were 

34 resolved. Inform the NRC supervisor if unable to resolve operating test comments with 

35 the facility licensee.

36 • Sign Form 2.3-2 and forward operating test package to the NRC supervisor for final 

37 approval after the necessary changes have been made and the examination is finalized.
ES-2.3, Page 5 of 19

1 b. Written Examination

2 • For NRC-developed examinations, independently review or assign an independent

3 reviewer to review every written examination question against the criteria in ES-4.2, 

4 “Developing Written Examination Questions,” for written examination question 

5 development. Form 2.3-5, “Written Examination Review Worksheet,” may be helpful to 

6 use for tracking purposes. Reviewers can only review the portions of the examination 

7 that they were not directly involved in developing; the NRC chief examiner may perform 

8 the independent review if he or she was not directly involved in the development of the 

9 questions reviewed.

10 • For facility licensee-developed examinations, review each written examination question 

11 using Form 2.3-5 to document the NRC’s review of each question and subsequent 

12 comment resolution. Account for any changes between the original submittal and the 

13 final approved version of the written examination and the reasons for the changes as 

14 follows:

15 a. For questions originally marked as (U) or unsatisfactory, include the following 

16 information:

17 1. how the psychometric attributes were not being met

18 2. what actions the facility licensee took to fix or replace the unsatisfactory

19 question

20 b. For questions originally marked as needing enhancements, include information 

21 about the nature of the enhancements. For example, if the enhancement 

22 involves a single implausible distractor, then a comment related to how the 

23 distractor does not meet the credible distractor attribute should be recorded, 

24 along with a summary of the resolution to the implausibility concern.

25 c. For minor editorial enhancements (i.e., question format, bulleting, punctuation, 

26 grammar, and wording changes that do not significantly impact the intent or 

27 framework of the question), include a comment such as “minor editorial changes 

28 required.”

29 d. Correct all questions identified as (U) or unsatisfactory on Form 2.3-5 by rewriting 

30 or replacing the questions before the examination is administered. Questions 

31 marked as (E) for needing editorial clarifications or enhancements must be 

32 corrected before the examination is administered.

33 e. Review the overall written examination using Form 2.3-4. The facility reviewer 

34 blocks in Column b do not apply for NRC-developed examinations.

35 • Review and document any changes that need to be made and forward the tests and 

36 completed Form 2.3-5 to the responsible NRC supervisor for review and comment in 

37 accordance with Section D before reviewing the examinations with the facility licensee. 

38 There are no minimum or maximum limits on the number or scope of changes the NRC

39 regional office may direct the author or facility contact to make to the proposed 

40 examinations if the changes are necessary to make the examinations conform to 

41 established acceptance criteria. 
ES-2.3, Page 6 of 19

1 • Do the following upon NRC supervisor approval:

2 a. For an NRC-developed examination, give the facility licensee enough time to 

3 review and comment on the examination. Review any facility licensee comments 

4 and make any necessary changes on a technical or psychometric basis.

5 Note: Examinations that the NRC has written shall be properly formatted and 

6 ready to administer before they are reviewed with the facility licensee. The 

7 region shall not rely on the facility licensee to ensure that the quality of the 

8 examination is acceptable for administration.

9 b. For a facility licensee-developed examination, provide a copy of Form 2.3-5 to 

10 the facility licensee and review comments with licensee personnel.

11 • After reviewing the examination with the facility licensee, ensure that all comments and 

12 recommendation resolutions are documented and the examination is revised as 

13 necessary. Update Form 2.3-5 to reflect how the comments/recommendations were 

14 resolved. Inform the NRC supervisor if the NRC chief examiner is unable to resolve 

15 operating test comments with the facility licensee.

16 • Sign Form 2.3-4 and forward the written examination package to the NRC supervisor for 

17 final approval after the necessary changes have been made and the examination is 

18 finalized. If the facility licensee wrote the examination, the package should include a 

19 copy of the original submittal.

20 4. Ensure Adequate Balance of Coverage

21 The NRC chief examiner should review the examination for balance of coverage at the start and 

22 then again at the end of examination development. An examination with an adequate balance 

23 of coverage contains a reasonably varied coverage of the items in the population to be sampled 

24 (i.e., 10 CFR 55.41, 10 CFR 55.43, and 10 CFR 55.45) without repeating the same concepts or 

25 overtesting similar evolutions or systems. In general, reviewing the balance of coverage of an 

26 examination involves the following steps:

27 a. Using the instructions in ES-4.1, Section B, step 8, check the written examinations for 

28 balance of coverage both within and across the sampling areas (i.e., tiers). Treat the 

29 SRO examination as a single 100-question examination for this check. 

30 b. Check the operating test for balance of coverage in three ways:

31 1. Review simulator scenarios for balance of coverage.

32 2. Review JPMs for balance of coverage.

33 3. Compare simulator scenarios and JPMs and check for overlap in their content.

34 c. Finally, compare the written examinations and operating test and check for overlap in 

35 their content. For example, overlap exists if the same pressurizer level controller 

36 concept is tested during a simulator event and in a written examination question. 
ES-2.3, Page 7 of 19

1 D. Instructions for NRC Supervisor Review and Approval

2 The responsible regional supervisor will review the examination outlines and the draft 

3 examinations and evaluate any recommended changes and corrections noted during the NRC

4 chief examiner’s review. The supervisory review is not intended to be a detailed review but 

5 rather a check to ensure that all applicable administrative requirements have been 

6 implemented. If the outlines, examinations, and recommended changes are acceptable, the 

7 supervisor will authorize the NRC chief examiner to resolve any noted deficiencies with the 

8 author or facility contact.

9 These instructions provide additional detail for reviewing and approving examinations.

10 1. Deficiencies

11 • Ensure that any significant deficiencies in the original written examination and operating 

12 test submitted by the facility licensee are evaluated to determine an appropriate course 

13 of action. Address significant deficiencies in the examination report.

14 • If any of the facility-developed examination materials (written, walkthrough, or simulator) 

15 require substantive changes, then the NRC regional office will evaluate the necessary 

16 examination changes to determine whether the examination can be made to conform 

17 with the examination standards by the time the examination is scheduled to begin. If the 

18 regional office does not have the resources to ensure that an acceptable examination 

19 can be prepared by the scheduled administration date, regional management must 

20 discuss with the facility licensee the need to reschedule the examinations as necessary.

21 • “Substantive changes” as used in the bullet above refer to the following:

22 a. For the written examination, 30 percent or more of the questions are 

23 unsatisfactory, as determined by using the instructions on Form 2.3-5.

24 b. For JPMs, one-third or more of the JPMs are unsatisfactory, as determined by 

25 using the instructions on Form 2.3-3.

26 c. For the simulator scenarios, one-third or more of the total number of scenario 

27 events are unsatisfactory, as determined by using the instructions on Form 2.3-3.

28 2. Operating Test

29 • Review the operating tests before authorizing the NRC chief examiner to proceed with 

30 the facility licensee review. After reviewing the proposed comments on the operating 

31 test items, authorize the NRC chief examiner to provide the comments to the facility 

32 licensee and proceed with reviewing comments with the facility.

33 • Following the facility licensee review, review the tests again to ensure that the concerns 

34 expressed by the facility licensee and the NRC chief examiner have been addressed

35 appropriately. If the region and the facility licensee are unable to resolve operating test

36 comments, then inform the NRR operator licensing program office.
ES-2.3, Page 8 of 19

1 3. Written Examination

2 • Review the entire written examination, with emphasis on all questions that are 

3 determined to have unacceptable flaws in accordance with Form 2.3-5, before any 

4 comments are provided to the facility licensee. 

5 • When NRC supervisor review is complete, authorize the NRC chief examiner to provide 

6 comments to the facility licensee and proceed with the facility review.

7 • Following the facility review, review the examination to ensure that the concerns 

8 expressed by the facility licensee and the NRC have been addressed appropriately. If 

9 the region and the facility licensee are unable to resolve comments on the written 

10 examinations, then consult the NRR operator licensing program office.

11 • Sign Form 2.3-4 when the examination is acceptable for administration.

12 4. Examination Approval

13 • Approve the finalized examinations for administration and sign the applicable quality 

14 checklists (Forms 2.3-2 and 2.3-4).

15 • After approving the examination, prepare and sign the examination approval letter 

16 (Letter 2.3-1) and finalize the list of applicants (Form 2.2-1) in accordance with ES-2.2.

17 • Send the examination approval letter (Letter 2.3-1) to the facility licensee. Provide the 

18 list of applicants (Form 2.2-1) to facility licensee separately.

19 E. Forms and Letters

20 Form 2.3-1 Examination Outline Quality Checklist

21 Form 2.3-2 Operating Test Quality Checklist

22 Form 2.3-3 Operating Test Review Worksheets

23 Form 2.3-4 Written Examination Quality Checklist

24 Form 2.3-5 Written Examination Review Worksheet

25 Letter 2.3-1 Sample Examination Approval Letter

26

27
ES-2.3, Page 9 of 19

1 Form 2.3-1 Examination Outline Quality Checklist

Facility: Date of Examination:

Item Task Description (Y)es / (N)o

a b* c#

WRITTEN

a. The outline was systematically and randomly prepared in accordance with the 

instructions in Section B of ES-4.1, and all knowledge and ability (K/A) categories are 

appropriately sampled.

b. The outline does not overemphasize any systems, evolutions, or generic topics.

c. Justifications for deselected or rejected K/A statements are acceptable.

SIMULATOR

a. Using Form 2.3-1, verify that the proposed scenario sets cover the required number of 

normal evolutions, reactivity evolutions, instrument and component failures, manual 

control evolutions, technical specifications, and major transients.

b. There are enough scenarios (and spares) for the projected number and mix of 

applicants in accordance with the expected crew composition and rotation schedule 

without compromising exam integrity. Ensure that scenarios will not be repeated on 

subsequent days.

c. Ensure that all scenarios are new or significantly modified in accordance with ES-3.4

and that no scenarios are duplicated from the applicants’ audit test(s).

d. To the extent possible, assess whether the outline(s) conforms with the qualitative and 

quantitative simulator set criteria specified on Form 2.3-2.

JPMS

a. Verify that the control room and in-plant systems outline meets the criteria specified

in the instructions on Form 3.2-2 and that no tasks are duplicated from the applicant’s

audit test(s).

b. Verify that the administrative outline meets the criteria specified in the instructions on 

Form 3.2-1 and that no tasks are duplicated from the applicant’s audit test(s).

c. Determine whether the number of job performance measures (JPMs) and JPM types is 

sufficient for the projected number and mix of applicants and ensure that no items are 

duplicated on subsequent days.

GENERAL

a. Assess whether the appropriate exam sections cover plant-specific priorities (including 

probabilistic risk assessment and individual plant examination insights).

b. Assess whether the 10 CFR 55.41, 10 CFR 55.43, and 10 CFR 55.45 sampling is 

appropriate.

c. Check whether K/A importance ratings (except for plant-specific priorities) are greater 

than or equal to 2.5.

d. Check for duplication and overlap across the exam and with the last two NRC exams.

e. Check the entire exam for balance of coverage.

f. Assess whether the exam fits the appropriate job level (reactor operator or senior 

reactor operator).

Printed Name/Signature Date

a. Author _________________________________________________________________

b. Facility Reviewer (*) _______________________________________________________________

c. NRC Reviewer (#) _______________________________________________________________

NRC Chief Examiner _________________________________________________________________ 

NRC Supervisor _______________________________________________________________

* The facility licensee signature is not applicable for NRC-developed tests.

# An independent NRC reviewer performs the steps in column “c.” This may be the NRC Chief Examiner if he/she did not develop 

the outline under review.

2
ES-2.3, Page 10 of 19

Form 2.3-2 Operating Test Quality Checklist

a. Each job performance measure (JPM) includes the following, as applicable:

• 0Btask standard

• 1Binitial conditions

• 2Binitiating cues

• 3Breferences and tools, including associated procedures

• 4Breasonable and validated time limits (average time allowed for completion) and specific 

designation if the facility licensee deems it to be time critical

• 5Bspecific designation if it meets the criteria of alternate path

• 6Boperationally important specific performance criteria that include the following:

– 7Bdetailed expected actions with exact criteria and nomenclature

– 8Bsystem response and other examiner cues

– 9Bstatements describing important observations to be made by the applicant

– 10Bcriteria for successful completion of the JPM task standard

– 11Bidentification of critical steps and their associated performance standards

– 12Brestrictions on the sequence of steps, if applicable

b. Ensure that any changes from the previously approved JPM outlines (Forms 3.2-1 and 3.2-2) 

have not caused the test to deviate from any of the acceptance criteria (e.g., item 

distribution, bank use, repetition from the last two NRC examinations) specified on those 

forms.

Simulator Scenario Set Criteria for Scenario Numbers: / /

QUALITATIVE ATTRIBUTES

1. The initial conditions are realistic in that some equipment and/or instrumentation may be 

out of service, but it does not cue the operators into expected events.

2. The scenarios consist mostly of related events.

3. Each event description consists of the following:

• the point in the scenario when it is to be initiated

• the malfunction(s) or conditions that are entered to initiate the event

• the symptoms/cues that will be visible to the crew

• the expected operator actions (by shift position)

• the event termination point (if applicable)

Facility: Date of Examination: Operating Test Number:

General Criteria (Y)es / (N)o

a b* c#

a. The operating test meets the criteria on the associated test outline.

b. There are enough test items so that test items will not be repeated on more than 1 day of 

the operating test.

c. The operating test does not duplicate items from the applicant’s audit test(s).

d. Overlap with the written examination and between different parts of the operating test is

minimized.

e. It appears that the operating test will differentiate between competent and 

less-than-competent applicants at the designated license level.

Walkthrough Criteria
ES-2.3, Page 11 of 19

QUALITATIVE ATTRIBUTES continued

(Y)es / (N)o

a b* c#

4. The events are valid with regard to physics and thermodynamics.

5. The sequencing and timing of events is reasonable and allows the examination team to 

observe and evaluate applicant performance.

6. If time compression techniques are used, the scenario summary clearly so indicates.

Operators have sufficient time to carry out expected activities without undue time

constraints. Cues are given.

7. The simulator modeling is not altered.

8. The scenarios have been validated. Pursuant to 10 CFR 55.46(d), any open simulator 

performance deficiencies or deviations from the referenced plant have been evaluated to 

ensure that functional fidelity is maintained while running the planned scenarios.

9. Scenarios are new or significantly modified in accordance with ES-3.4.

10. Scenarios (as grouped) allow each applicant to be significantly involved in the minimum 

number of transients, events, and evolutions specified on the version of Form 3.4-1 

submitted with the scenario set.

11. Applicants are evaluated on a similar number of preidentified critical tasks across 

scenarios, when possible.

12. The level of difficulty is appropriate to support licensing decisions for each crew position.

TARGET QUANTITATIVE ATTRIBUTES per Scenario (See ES-3.4)

Actual Attributes by 

Scenario No.

/ /

(Y)es / (N)o

a b* c#

1. Malfunctions after emergency operating procedure (EOP)

entry (1–2)

/ /

2. Abnormal events (2–4) / /

3. Major transients (1–2) / /

4. EOPs entered/requiring substantive actions (1–2) / /

5. Entry into a contingency EOP with substantive actions 

(> 1 per scenario set; set is the entire set of scenarios 

prepared for the scheduled exam)

/ /

6. Preidentified critical tasks (> 2) / /

Printed Name/Signature Date

a. Author ______________________________________________________________

b. Facility Reviewer (*) ______________________________________________________________

c. NRC Reviewer (#) ________________________________________________________________

NRC Chief Examiner _________________________________________________________________ 

NRC Supervisor _________________________________________________________________ 

* The facility licensee signature is not applicable for NRC-developed tests.

# An independent NRC reviewer performs the steps in column c. This may be the NRC Chief Examiner if he/she 

did not develop the outline under review.
ES-2.3, Page 12 of 19

Form 2.3-3 Operating Test Review Worksheet (Job Performance Measures (JPMs))

Facility: Exam Date:

1

JPM # or title

2

Type

(S/P/A)

3

ALT

(Y/N)

4

LOD

(1–5)

5

JPM Errors

6

U/E/S 

LOD REF IC TSK CUE CS TL
ES-2.3, Page 13 of 19

Form 2.3-3 Instructions for Completing the JPM Table

1. Enter the JPM number and/or title.

2. Enter the type of JPM—(S)imulator, (P)lant, or (A)dministrative.

3. Enter (Y)es or (N)o for an Alternate Path JPM.

4. Rate the level of difficulty (LOD) of each JPM using a scale of 1–5 (easy–difficult). A JPM containing less th

a JPM that tests solely for recall or memorization, or a JPM that involves directly looking up a single correct 

LOD = 1 (too easy). Conversely, a JPM with over 30 steps or a JPM that takes more than 45 minutes to com

LOD = 5 (too difficult).

5. Check the appropriate block for each JPM error type, using the following criteria:

• LOD = 1 or 5 is unsatisfactory (U).

• REF: The JPM lacks required references, tools, or procedures (U).

• IC: The JPM initial conditions are missing or the JPM lacks an adequate initial cue (U).

• CUE: The JPM lacks adequate evaluator cues to allow the applicant to complete the task, or the eva

subjective or leading (U).

• TSK: The JPM lacks a task standard or lacks completion criteria for a task standard (U).

• CS: The JPM contains errors in designating critical steps, or the JPM lacks an adequate performanc

critical step (U).

• TL: The JPM validation times are unreasonable, or a time-critical JPM lacks a completion time (U).

6. Mark the JPM as unsatisfactory (U), satisfactory (S), or needs enhancements (E). A JPM is (U) if it has one 

determined in step 5. Examples of enhancements include formatting, spelling, or other minor changes.

7. Briefly describe any JPM determined to be unsatisfactory (U) or needing enhancement (E). Save initial revi

detail subsequent comment resolution so that each exam-bound JPM is marked by a satisfactory (S) resolu
ES-2.3, Page 14 of 19

Form 2.3-3 Operating Test Review Worksheet (Scenarios)

Facility: Scenario: Exam Date:

1 

Scenario 

Event 

ID/Name:

2

Scenario event errors 3 

U/E/SExp

Realism/

Credibility

Performance 

Standards

Verifiable 

Actions Critical Task TS
ES-2.3, Page 15 of 19

Form 2.3-3 Instructions for Completing the Scenario Table

1. For each scenario, enter the scenario event names and descriptions.

2. Review the individual events contained in each scenario, and identify and mark event errors:

• The scenario guide event description is not realistic/credible—unsatisfactory (U).

• The scenario guide event description lacks adequate crew/operator performance standards—needs 

• The scenario guide event description lacks verifiable actions for a credited normal event, reactivity ev

instrument/component malfunction, or technical specification (TS) event (or a combination of these) 

• The scenario guide event description incorrectly designates an event as a critical task (i.e., a noncrit

critical or a critical task labeled as noncritical). This includes critical tasks that do not meet the critica

critical task does not have a measurable performance standard) (U).

• The scenario guide event description incorrectly designates entry into TS actions when not required 

entry into TS actions when required (U).

3. Based on the outcome in step 2, mark the scenario event as unsatisfactory (U), satisfactory (S), or needs en

event is (U) if it has one or more (U) errors as determined in step 2. Examples of enhancements include for

other minor changes.

4. Briefly describe any scenario event determined to be unsatisfactory (U) or needing enhancement (E). Save 

comments and detail subsequent comment resolution so that each exam-bound scenario event is marked b

resolution on this form.
ES-2.3, Page 16 of 19

Form 2.3-4 Written Examination Quality Checklist

Facility: Date of Exam: Exam Level: RO SRO

Item Description (Y)es / (N)o

a b* c#

1. Questions and answers are technically accurate and applicable to the facility. Each question 

includes a technical reference.

2. a. All questions reference NRC knowledge and abilities (K/As) requirements.

 b. Facility learning objectives are referenced as available.

 c. All questions include an explanation of the correct answer explanation and a distractor 

analysis.

3. Senior reactor operator (SRO) questions test at the SRO license level.

4. The sampling process was random and systematic. (If more than four reactor operator (RO) or 

two SRO questions were repeated from the last two NRC licensing exams, including full 

(100-question) retake examinations, consult the NRC Office of Nuclear Reactor Regulation

operator licensing program office.)

5. Question duplication from the licensee screening/audit exam was controlled as indicated below 

(check the item that applies) and appears appropriate:

__ The audit exam was systematically and randomly developed.

__ The audit exam was completed before the license exam was started.

__ The examinations were developed independently.

__ The licensee certifies that there is no duplication.

__ Other (explain):

6. Bank use meets limits (no more than 75% from the bank, at least 

10% new (at the comprehension/analysis level), and the rest new or 

modified); enter the actual RO/SRO-only question distribution >

Bank Modified New

/ / /

7. Between 38 and 45 questions of the questions on the RO exam and 

at least 13 of the questions on the SRO-only portion of the exam 

are written at the comprehension/analysis level; enter the actual 

RO/SRO-only question distribution >

Memory C/A

/ /

8. References/handouts provided do not give away answers or aid in the elimination of distractors.

9. Question content conforms to specific K/A statements in the previously approved examination 

outline and is appropriate for the K/A statements’ assigned tier; deviations are justified.

10. Question psychometric quality and format meet the instructions and guidelines in ES-4.2.

11. The exam contains the required number of one-point, multiple-choice items; the total is correct 

and agrees with the value on the cover sheet.

Printed Name/Signature Date

a. Author _____________________________________________________________ __________ 

b. Facility Reviewer (*) _____________________________________________________________ __________ 

c. NRC Reviewer (#) _____________________________________________________________ __________ 

NRC Chief Examiner _____________________________________________________________ __________

NRC Regional Supervisor _____________________________________________________________ __________ 

* The facility licensee signature is not applicable for NRC-developed tests.

# An independent NRC reviewer performs steps in column c. This may be the NRC Chief Examiner if he/she 

did not develop the outline under review.
ES-2.3, Page 17 of 19

Form 2.3-5 Written Examination Review Worksheet

Q#

1.

LOK

(F/H)

2.

LOD

(1–5)

3. Psychometric Flaws 4. Job Content Flaws 5. K/A Use Flaws

6. Source

(B/M/N)

7. Status

(U/E/S) 8. Exp

Stem

Focus Cues T/F Cred. Dist. Partial Job Link Minutia #/Units Logic Q–K/A License 

Level
ES-2.3, Page 18 of 19

Form 2.3-5 Instructions for Written Examination Review Worksheet

Refer to ES-4.2 for the definitions of terms used in this worksheet for the written examination. Review each que

submitted and as subsequently revised and document the following in the associated worksheet columns:

1. Enter the level of knowledge (LOK) as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) from 1 (easy) to 5 (difficult); mark direct lookup questions (applicant can dir

answer from the provided reference) as LOD 1. A question is (U)nsatisfactory if it is LOD 1 or LOD 5.

3. Check the appropriate box if a psychometric flaw is identified:

• “Stem Focus”: The stem lacks enough focus to elicit the correct answer (e.g., unclear intent, more inform

too much needless information). This is an (U)nsatisfactory question.

• “Cues”: The stem or one or more answer choices contains cues (e.g., clues, specific determiners, phras

an (U)nsatisfactory question.

• “T/F”: All of the answer choices are a collection of unrelated true/false statements. This is an (U)nsatisf

• “Cred. Dist.”: The distractors are not credible; single implausible distractors require (E)nhancement, and 

noncredible distractor in the same question results in an (U)satisfactory question. 

• “Partial”: One or more distractors are partially correct (e.g., if the applicant can make unstated assumpti

contradicted by the stem). This is an (U)nsatisfactory question.

4. Check the appropriate box if a job content flaw is identified:

• “Job Link”: The question is not linked to the job requirements (i.e., the question has a valid knowledge o

written, is not operational in content). This is an (U)nsatisfactory question.

• “Minutia”: The question requires the recall of knowledge that is too specific for the closed-reference test 

required to be known from memory). This is an (U)nsatisfactory question.

• “#/Units”: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., pan

with question in gallons). This is an (U)nsatisfactory question.

• “Logic”: The question requires backward or reverse logic or application compared to the job requiremen

(U)nsatisfactory question.

5. Check the first box if a K/A mismatch flaw exists. Check the second box if the question is flawed because it 

wrong license level. Either condition results in an (U)nsatisfactory question. 

6. Enter the question’s source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria 

7. Based on the review performed in steps 2–5, mark the question as (U)nsatisfactory, in need of (E)ditorial ch

enhancement, or (S)atisfactory. 

8. Fully explain the reason for any (U) in column 7 (e.g., how the psychometric attributes are not being met).

9. Save the initial review comments and detail subsequent comment resolution so that each exam-bound ques

(S) on this form.
ES-2.3, Page 19 of 19

Letter 2.3-1 Sample Examination Approval Letter

NRC Letterhead

(Date)

(Name, Title)

(Name of Facility)

(Address)

(City, State ZIP Code)

SUBJECT: OPERATOR LICENSING EXAMINATION APPROVAL

Dear (Name):

The purpose of this letter is to provide Commission approval of the proposed written 

examinations and operating tests under Title 10 of the Code of Federal Regulations

(10 CFR) 55.40(b)(4) and confirm the final arrangements for the upcoming operator licensing 

examination at (facility name).

The U.S. Nuclear Regulatory Commission (NRC) has approved the subject examinations, and 

you may administer the written examination on (date) in accordance with Revision 12 to 

NUREG-1021, “Operator Licensing Examination Standards for Power Reactors,” issued (issue 

date). The NRC staff will administer the operating tests during the week(s) of (date). The NRC 

has provided the list of the applicants approved to take the examination to (Name, Title). The 

examination has undergone extensive review by my staff and representatives responsible for 

licensed operator training at your facility. Based on this review, I have concluded that the 

examination meets the guidelines of NUREG-1021 for content, operational, and discrimination 

validity. By administering this examination, you also agree that it meets NUREG-1021 

guidelines and is appropriate for measuring the qualifications of licensed operator applicants at 

your facility. If you determine that this examination is not appropriate for licensing operators at 

your facility, do not administer the examination and contact me at (telephone number). 

Please contact your NRC chief examiner, (Name), at (telephone number), if you have questions 

or identify any errors or changes in the license level (reactor operator or senior reactor operator) 

or type of examination (partial or complete written examination or operating test, or both) 

specified for each applicant approved to take the examination. 

Sincerely,

(Name of Appropriate Regional 

Representative, Title)

Docket No.: 50-(number) or

Docket No.: 52-(number)

cc: Public

NRC Document Control System

Regional Distribution

ES-3.1, Page 1 of 5

1 3.1 OVERVIEW OF THE OPERATING TEST FOR OPERATING 

2 LICENSING INITIAL EXAMINATIONS

3 This examination standard contains an overview of the operating test, required by Title 10 of the 

4 Code of Federal Regulations (10 CFR) 55.45, “Operating tests,” for the initial licensing of 

5 operators, including the purpose, structure, and content of the operating test. It includes 

6 general guidelines and instructions for the operating test.

7 A. Operating Test Structure and Content

8 To the extent applicable, the operating test requires the applicant to demonstrate an 

9 understanding of, and the ability to perform, the actions necessary to accomplish a 

10 representative sampling of the 13 items identified in 10 CFR 55.45(a). Each operating test need 

11 not cover all 13 items. In addition, the content of the operating test will be identified, in part, 

12 from learning objectives contained in the facility licensee’s training program and information in 

13 the final safety analysis report, system description manuals and operating procedures, the 

14 facility license and amendments thereto, licensee event reports, and other materials that the 

15 Commission requests from the facility licensee.

16 The structure of the operating test is dictated, in part, by 10 CFR 55.45(b). Specifically, that

17 requirement states that the test will be administered in a plant walkthrough and in either a 

18 simulation facility that the Commission has approved pursuant to 10 CFR 55.46(b), a plant 

19 referenced simulator that conforms to 10 CFR 55.46(c), or the plant itself, if approved by the 

20 Commission under 10 CFR 55.46(b).

21 The operating test consists of two portions, the individual walkthrough portion, using job 

22 performance measures (JPMs), and a dynamic simulator operating test using scripted dynamic 

23 scenarios. The individual walkthrough portion, also known as the JPM portion of the operating 

24 test consists of a set of JPMs covering administrative topics, control room systems and in-plant

25 systems. Each JPM focuses on specific knowledge and abilities (K/As) required for licensed 

26 operators to safely discharge their assigned duties and responsibilities. For each applicant, 

27 depending on license level, the U.S. Nuclear Regulatory Commission (NRC) specifies the

28 number of JPMs required and the topic areas covered during this part of the operating test in 

29 Examination Standard (ES)-3.2, “Developing Job Performance Measures,” which contains 

30 specific instructions for developing JPMs for the operator licensing initial examination.

31 The simulator operating test, also known as the simulator test or simulator scenarios, is 

32 administered on a Commission-approved or plant-referenced simulator. A simulator test 

33 consists of a set of dynamic simulator scenarios, also referred to as the scenario set, developed

34 for a specific examination. Depending on license level and the number of applicants taking the 

35 operating test, one or more scenarios are administered to each applicant. 

36 Administrative Topic Job Performance Measures

37 This part of the walkthrough portion of the operating test covers K/As associated with 

38 administrative control of the plant. These JPMs can be administered in a classroom format, in 

39 the simulator, or in the plant. The scope and depth of coverage required in each administrative 

40 topic area are based on the applicant’s license level. The applicant’s competence in each topic 

41 is evaluated by administering JPMs and asking followup questions, as necessary, based on the 

42 applicant’s performance.
ES-3.1, Page 2 of 5

1 Control Room System and In-Plant System Job Performance Measures

2 This part of the walkthrough portion of the operating test is used to determine whether the 

3 applicant has an adequate knowledge of plant system design and can safely operate those 

4 systems and successfully implement facility operating procedures. This involves various system 

5 types such as primary coolant, emergency coolant, decay heat removal, auxiliary, radiation 

6 monitoring, and instrumentation and control systems. The NRC examiner evaluates the 

7 applicant’s system-related K/As by having the applicant perform selected tasks and, when 

8 necessary, based on the applicant’s performance, probing his or her knowledge of the task and 

9 its associated system with followup questions. 

10 This part of the walkthrough focuses primarily on those systems with which licensed operators 

11 are most involved (i.e., those having controls and indications in the main control room) and 

12 ensures that the applicant is familiar with the design and operation of systems located outside 

13 the main control room.

14 This part of the operating test is administered in a one-on-one walkthrough format in the control 

15 room simulator and the plant.

16 Simulator Operating Test

17 The simulator operating test is a performance-based test used to evaluate the applicant’s ability 

18 to safely and competently operate the plant’s systems under dynamic, integrated conditions.

19 Each simulator scenario is administered in a team format with up to three applicants filling the 

20 reactor operator (RO) and senior reactor operator (SRO) license positions on an operating crew. 

21 This team format enables the examiner to evaluate each applicant’s ability to function within the 

22 control room team as appropriate to the assigned position in such a way that the facility 

23 licensee’s procedures are adhered to and that the limitations in its license and amendments are 

24 not violated.

25 The simulator test is designed so that each applicant can demonstrate proficiency on the 

26 competencies applicable to his or her license level. The NRC examination team administers 

27 one or more scenarios to each team or crew of applicants in order to evaluate each applicant 

28 individually on a range of competencies applicable to the applicant’s license level. 

29 B. General Instructions and Guidelines for Developing Operating Tests

30 The following is a list of general instructions and guidelines for use during operating test 

31 development and planning:

32

33 1. To maximize efficiency, use the same operating test material to examine multiple 

34 applicants and simulator crews, if desired. The facility licensee and the NRC chief 

35 examiner shall discuss the options and reach agreement on the strategy to maximize 

36 examination preparation efficiency while maintaining test integrity before developing the 

37 operating tests.

38 2. Do NOT repeat the same JPMs and simulator scenarios on subsequent days.

39 3. Design the operating test to examine a broad range of K/As, systems and components, 

40 and operations and events. Do not duplicate material used on the walkthrough and 

41 simulator scenario portions with material covered on the written examination. Develop 
ES-3.1, Page 3 of 5

1 and review the operating test material as a package to preclude the same tasks and 

2 events from appearing on both parts of the operating test. 

3 4. Do NOT duplicate test items (simulator scenarios or JPMs) from the applicant’s audit 

4 test. For retake examinations, do not duplicate test items from the applicant’s original 

5 test. Simulator events and JPMs that are related to those tested on the audit 

6 examination are permitted, provided that the actions required to mitigate the transient or 

7 complete the task (e.g., using an alternative path) are significantly different from those 

8 required during the audit examination. The facility licensee shall identify for the NRC 

9 chief examiner those simulator events and JPMs that are related to those tested on the 

10 audit examination.

11 5. For each part of the operating test, develop new material or select testing materials 

12 (i.e., JPMs and simulator scenarios) from the facility’s examination banks (and modify 

13 them, if needed). Every selected test item must satisfy the applicable qualitative and 

14 quantitative standards and limitations specified in this NUREG or be modified 

15 accordingly. 

16 6. Consider the K/As associated with normal, abnormal, and emergency tasks and 

17 evolutions as a source of topics for use in evaluating applicant competency in each part 

18 of the operating test.

19 a. K/As associated with the tasks planned for the operating test should have 

20 importance ratings of at least 2.5. Tasks with importance ratings of less than 2.5 

21 may be used if there is a substantive reason for including them (e.g., a licensee 

22 event or a significant system modification). 

23 b. K/As should be appropriate to the plant-specific requirements for the applicant’s 

24 license level. Refer to the facility licensee’s job and task analysis (if available), 

25 learning objectives, and other reference material to confirm that the operating 

26 test is correctly oriented to the facility and the applicant’s license level. Failure to 

27 train the applicants on a particular K/A is not an acceptable basis for excluding a 

28 K/A.

29 c. The facility licensee’s site-specific task list may be used to supplement or 

30 override, on a case-by-case basis, selected individual items in the NRC’s K/A 

31 catalogs. To maintain examination consistency, the site-specific task list shall 

32 not be used in place of the entire K/A catalog.

33 7. When selecting and developing JPMs and scenarios for the operating test, ensure that 

34 the materials contribute to the test’s overall capacity to differentiate between those 

35 applicants who are competent to safely operate the plant and those who are not. 

36 Additionally, apply the three facets of test validity (i.e., content, operational, and 

37 discrimination) discussed in Appendix A, “Overview of Generic Examination Concepts.” 

38 A valid test item, when missed, facilitates subsequent justification for denying the 

39 applicant’s license; avoid test items that when missed cannot be used to justify a license 

40 denial.

41 8. The operating test for SRO applicants tests the applicant’s knowledge, skills, and 

42 abilities for any on-shift position that requires an SRO license, regardless of the position 

43 he or she will be assigned when licensed. 
ES-3.1, Page 4 of 5

1 9. The following guidelines differentiate the SRO operating test from that of the RO 

2 operating test:

3 a. In directing licensed activities, the SRO must evaluate plant performance and 

4 make operational judgments accordingly. SRO applicants should, therefore, be 

5 more knowledgeable in areas such as operating characteristics and reactor 

6 behavior.

7 b. In directing licensed activities, the SRO must have a broader and more thorough 

8 knowledge of the facility licensee’s administrative controls and methods, 

9 including limitations imposed by the regulations and the facility licensee’s 

10 technical specifications (TS) and their bases.

11 c. The SRO may be assigned responsibilities for auxiliary systems that are outside 

12 the control room (e.g., waste disposal and fuel handling systems) and are not 

13 normally operated by licensed operators. Because the SRO may have these 

14 additional responsibilities, the SRO license applicant should demonstrate 

15 knowledge of the designs of such systems as they relate to maximum 

16 permissible concentrations, effluent release rates, and other radiological 

17 considerations.

18 10. Incorporate facility-specific and industry generic operating experience into the operating 

19 test whenever possible. Documents such as licensee event reports, significant event 

20 reports, and vendor-specific operating experience reports are readily available sources

21 of operationally oriented plant anomalies.

22 11. Evaluate the dominant accident sequences for the facility to determine whether they are 

23 suitable for testing, on a sampling basis, during the dynamic simulator or walkthrough 

24 tests. Dominant accident sequences are those sequences that contribute significantly to 

25 the frequency of core damage as determined by the facility licensee’s probabilistic risk 

26 assessment or individual plant examination.

27 12. The probabilistic risk assessment/individual plant examination should also be used to 

28 identify risk-important operator actions. In determining which actions to evaluate, do not 

29 overlook actions that are relied upon for or result in specific events being driven to 

30 low-risk contribution. This will help identify those human actions that are assumed to be 

31 very reliable but might otherwise not appear in a list of risk-dominant actions.

32 13. Do the following for applicants applying for a multiunit license at a facility:

33 a. Test the applicants’ knowledge of the design, procedural, and operational 

34 differences among the units.

35 b. Divide the operating test coverage among the units and do not become 

36 predictable by conducting the walkthrough tests on only one unit. Consider 

37 examining the applicants on different units during the in-plant JPMs.

38 c. For multiunit facilities that have a simulator modeled after only one of the units, 

39 test the applicants on the different systems, control board layouts, and any other 

40 differences among the units during the in-plant JPM portion of the operating test. 
ES-3.1, Page 5 of 5

1 14. Every facet of the operating test, including the walkthrough JPMs and simulator 

2 scenarios, should be planned, researched, validated, and documented to the maximum 

3 extent possible before the test is administered.

4 15. If it becomes necessary to deviate from a test outline that the NRC has approved, 

5 discuss the proposed deviation with the NRC chief examiner and obtain concurrence 

6 before proceeding with the changes. Be prepared to explain why the original proposal 

7 could not be implemented and why the proposed replacement is considered an

8 acceptable substitute.

ES-3.2, Page 1 of 18

1 3.2 DEVELOPING JOB PERFORMANCE MEASURES

2 This examination standard contains instructions for selecting and developing JPMs for the 

3 walkthrough portion of the operating test for operator licensing initial examinations. The 

4 instructions in this examination standard can also be used for developing JPMs that can be 

5 used for NRC requalification examinations. The NRC uses JPMs to evaluate the applicant’s 

6 understanding of, and ability to safely operate, the plant. Adhering to this guidance ensures that 

7 JPMs are of appropriate substance and format and helps test creators maintain the consistency 

8 and validity of the walkthrough tests.

9 A. General Instructions

10 1. To protect the integrity and security of the examination process, limits exist for

11 examination material taken directly from the facility licensee’s testing materials or banks

12 without significant modification and the number of JPMs repeated from the last two NRC

13 licensing examinations at the facility. The outline forms (Form 3.2-1, “Administrative

14 Topics Outline,” and Form 3.2-2, “Control Room/In-Plant Systems Outline”) in this 

15 examination standard contain specific limits on JPM bank use and repetition from the 

16 previous two NRC examinations.

17 2. For JPMs, a significant modification means that at least one condition has been 

18 substantively changed in a manner that alters the course of action of the original JPM. 

19 3. If JPMs are repeated from the past two NRC examinations, they must be randomly 

20 selected from all the JPMs used on the past two examinations.

21 4. The test developer should ensure that the JPMs, both individually and as a group, have 

22 meaningful performance criteria that will provide a legitimate basis for evaluating the 

23 applicant’s understanding of the plant and ability to safely operate it. The test should not 

24 include simple, one-step JPMs or JPMs that only involve directly looking up the correct 

25 answer. JPMs that incorporate the testing of immediate action steps from memory are 

26 acceptable if they include testing additional steps or items that are not from memory. 

27 Test developers should avoid creating a walkthrough test that is heavily weighted with 

28 tasks of low difficulty; this type of test does not provide the NRC with an adequate basis 

29 to make a licensing decision.

30 B. Instructions for Selecting Administrative Topic Job Performance Measures

31 The following are instructions for how to select topics for administrative topic JPMs.

32

33 1. Determine the Number of Administrative Topic JPMs To Develop

34 Obtain the required number of administrative topic JPMs to be evaluated based on the license 

35 level of the applicants using the criteria on Form 3.2-1. Refer to the outline form for bank limits

36 and other criteria. 

37 2. Select a Knowledge and Ability Item for Each Administrative Topic JPM

38 Select a K/A associated with each administrative topic from Section 2.0, “Generic Knowledge 

39 and Abilities,” of the applicable K/A catalog:
ES-3.2, Page 2 of 18

1 a. For the SRO applicants, all administrative topic JPMs must be at the SRO level. Select 

2 K/A statements for the SRO administrative topic JPMs that allow for the evaluation of 

3 SRO applicants in a manner that is representative of the greater depth and scope of an 

4 SRO’s administrative responsibilities and of their ability to perform administrative tasks 

5 that are the responsibility of the facility’s senior licensed positions. 

6 b. For the “Emergency Plan” topic, select K/As related to the emergency plan and 

7 implementing procedures and do not use K/As associated with the emergency operating 

8 procedures (EOPs).

9 c. Select K/As with importance ratings of at least 2.5 (SRO column) for the SRO 

10 administrative topic JPMs and select K/As with importance ratings of at least 2.5 (RO 

11 column) for the RO administrative topic JPMs. Tasks with importance ratings of less 

12 than 2.5 may be used if there is a substantive reason for including them (e.g., a recent 

13 licensee event or a significant system modification). 

14 3. Select an Activity for Each Administrative Job Performance Measure Topic

15 Select a performance-based administrative activity for the JPM that meets the K/A.

16 Administrative topic JPMs are not used to evaluate the applicant’s ability to perform system or 

17 component manipulations. Administrative topic JPMs that take place in the simulator/plant can 

18 involve system or component manipulations that do not change the configuration of the plant 

19 (e.g., pulling up data on the plant computer, swapping the digital display to take different 

20 instrument readings, etc.). Administrative topic JPMs may require the applicant to identify and 

21 respond to one or more postulated administrative errors.

22 This section provides some example activities by topic area for administrative topic JPMs.

23 a. Conduct of Operations Topics

24 This topic area evaluates the applicant’s knowledge of the daily operation of the facility. The 

25 following subjects are examples of the types of administrative tasks that could be evaluated 

26 under this topic:

27 • shift turnover

28 • shift staffing requirements

29 • access controls for vital/controlled plant areas

30 • operator responsibilities and procedure usage

31 • purpose, function, and administrative controls for plant systems

32 • administrative aspects of fuel handling and refueling

33 The applicant’s knowledge of access controls for vital/controlled plant areas can also be 

34 observed during the in-plant JPMs. However, these passive observations, in and of 

35 themselves, are insufficient to justify an evaluation in that topic area.

36 The subject of fuel handling can be covered in the control room or in the fuel handling areas of 

37 the plant. The RO applicant should be aware of his or her duties in the control room during fuel 

38 handling. These duties include monitoring instrumentation and responding to alarms from the 

39 fuel handling area (with actions that do not require direct manipulation of system controls) and 

40 communicating with fuel handling personnel in support of refueling operations. The SRO 

41 applicant should be evaluated on topics such as requirements for core alterations, requirements 

42 for new and spent fuel storage and movement, the design of the fuel handling area, precautions 
ES-3.2, Page 3 of 18

1 and requirements for the use of the fuel handling tools, and administrative aspects of fuel 

2 handling casualties such as when to stop fuel movement or evacuate containment. 

3 b. Equipment Control Topics

4 This topic area addresses the administrative requirements associated with managing and 

5 controlling plant systems and equipment. The following subjects are examples of the types of 

6 administrative tasks that could be evaluated under this topic:

7 • surveillance testing (for example, recording and evaluating plant data)

8 • prestartup activities (for example, calculations for criticality)

9 • maintenance (for example, post maintenance test requirements)

10 • tagging and clearances

11 • temporary modification of systems

12 • changes to procedures and plant design

13 • TS, including plant mode

14 • processes for controlling equipment configuration/status

15 • familiarity with, and use of, piping and instrument drawings 

16 These subjects can be evaluated within the framework of a normal maintenance evolution. For 

17 example, ask the applicant to perform a clearance to take a failed system or component out of 

18 service, initiate maintenance on the system, and test the system before placing it back in 

19 service. A test that requires integrated system operation (e.g., a pump operability test) would 

20 not be considered administrative in nature. However, stroke timing a valve in a shutdown 

21 system would be administrative in nature.

22 c. Radiation Control Topics

23 This topic area evaluates the applicant’s knowledge and abilities with respect to radiation 

24 hazards and protection (of plant personnel and the public). The following subjects are examples 

25 of the types of administrative tasks that could be evaluated under this topic:

26 • use and function of portable radiation and contamination survey instruments personnel 

27 monitoring equipment

28 • knowledge of significant radiation hazards

29 • radiological safety principles and procedures

30 • radiation exposure limits under normal or emergency conditions

31 • radiation work permits

32 • control of radiation releases

33 This topic may be covered in conjunction with the JPMs prepared for the in-plant systems 

34 walkthrough. One possibility is to evaluate these subjects during the required entry into the 

35 radiologically controlled area.

36 The levels of knowledge expected of RO and SRO applicants in some radiation control subjects 

37 differ significantly. The RO’s duties generally require knowledge of radiation worker 

38 responsibilities and operation of plant systems associated with liquid and gaseous waste 
ES-3.2, Page 4 of 18

1 releases. Therefore, the depth to which RO applicants are evaluated should be limited to their 

2 responsibilities and the monitoring requirements before, during, and after the release. The 

3 SRO, however, may be involved in reviewing and approving release permits and should be 

4 cognizant of the requirements associated with those releases, as well as their potential effect on 

5 public health and safety.

6 d. Emergency Plan Topics

7 This topic area evaluates the applicant’s knowledge of the facility’s emergency plan, including, 

8 as appropriate, the responsibility of the RO or SRO to decide whether the plan should be 

9 executed, and activities/duties assigned under the plan. The following subjects are examples of 

10 the types of administrative tasks that could be evaluated under this topic:

11 • lines of authority during an emergency

12 • operator responsibilities during an emergency

13 • emergency plan procedures

14 • emergency action levels and classifications

15 • emergency facilities

16 • emergency communications

17 • emergency protective action recommendations

18 • security event procedures (nonsafeguards information)

19 The knowledge required of RO and SRO applicants in this area differs significantly. RO 

20 applicants should be familiar with the emergency plan and with their plant-specific 

21 responsibilities under the emergency plan implementing procedures (EPIPs). By contrast, SRO 

22 applicants must demonstrate additional knowledge based upon their responsibility to direct and 

23 manage the implementation of the EPIPs during the initial phases of an emergency. As a result, 

24 SRO applicants should have a more detailed understanding of the EPIPs, in general, and 

25 should be familiar with event classification procedures, protective action recommendations, and

26 communication requirements and methods. To minimize predictability, the use of JPMs that 

27 include variations of the same activity (e.g., always testing this topic with an emergency 

28 classification) should be avoided. Instead, the test items for this topic should vary from 

29 examination to examination to include items from the bulleted list above.

30 This topic may be evaluated by linking a JPM to a simulator transient that requires 

31 implementation of the emergency plan. Such a JPM can be conducted immediately following a 

32 simulator scenario or during control room or in-plant systems JPMs.

33 4. Time Allowances for Administrative Job Performance Measures

34 Administrative topic JPMs range, on average, from 15 minutes to 45 minutes in duration (per 

35 JPM). For time-critical administrative topic JPMs, the JPM task standard specifies a time limit.

36 5. Fill Out Form 3.2-1 (Administrative Topics Outline)

37 Form 3.2-1 includes instructions on how to complete the administrative topics outline. When

38 complete, the outline is reviewed and approved in accordance with ES-2.3, “Reviewing and 

39 Approving Operator Licensing Initial Examinations.”
ES-3.2, Page 5 of 18

1 6. Develop Job Performance Measures in accordance with ES-3.2, Sections D and E

2 Upon NRC approval of Form 3.2-1, the examination author develops the administrative topic 

3 JPMs in accordance with ES-3.2, Sections D and E.

4 C. Instructions for Selecting Control Room System and In-Plant System Job 

5 Performance Measures

6 Prepare Form 3.2-2 as follows:

7 1. Determine the required number of JPMs for the control room system and in-plant 

8 systems to develop based on the applicant’s license level using the criteria on 

9 Form 3.2-2. Refer to the outline form for bank limits and other criteria. 

10 2. Using the instructions on Form 3.2-2, identify the safety function, associated system and 

11 task, location, and type code for each required JPM. 

12 3. When Form 3.2-2 is complete, ensure the form is reviewed and approved in accordance 

13 with ES-2.3.

14 Upon NRC approval of the Form 3.2-2, the examination author develops the Control Room 

15 System and In-Plant System JPMs in accordance with ES-3.2, Sections D and E.

16 D. Instructions for Developing Job Performance Measures

17 This section contains instructions for developing JPMs for both operator licensing initial and 

18 requalification examinations (except as noted). The steps do not have to be performed in this 

19 exact order.

20 Although written from the perspective of developing new JPMs, these instructions also apply

21 when modifying existing JPMs for reuse and when reviewing proposed JPMs for NRC approval

22 (in accordance with ES-2.3).

23 If a JPM already exists for the selected task, review it using the criteria in this section to ensure 

24 that it is acceptable for use.

25 Document JPMs using Form 3.2-3, “Job Performance Measure Template,” or an equivalent. 

26 Form 3.2-4, “Job Performance Measure Development Job Aid,” may be used to verify that the 

27 relevant criteria are satisfied.

28 1. Develop Performance Criteria

29 Develop meaningful performance criteria that provide examiners with a basis for evaluating the 

30 applicant’s knowledge and ability. Artificially subdividing existing tasks to generate new ones 

31 may dilute the value of the JPM to a point where it becomes meaningless. The JPMs cannot 

32 test solely for simple recall or memorization. Steps a – d below explain how to develop 

33 performance criteria.

34 a. Define the Task Standard

35 For the task associated with the JPM, define the task standard. The task standard is the 

36 predetermined qualitative or quantitative outcome (or both) against which task performance will 

37 be measured. The task standard clearly describes the expected outcome (i.e., end state) for 
ES-3.2, Page 6 of 18

1 successful completion of the JPM. For alternate path JPMs, the task standard includes the end 

2 state reached by way of alternate path actions. When applicable, the task standard includes a 

3 tolerance range for acceptable performance (e.g., 4950–5050 gallons per minute flow).

4 A properly defined and detailed task standard enables consistency in determining the JPM’s 

5 critical steps and subsequently evaluating applicant performance.

6 The following are examples of task standards.

7 Administrative Topic JPMs:

8 (Example from a print-reading task): The task is satisfactorily met when the applicant has 

9 determined that, with both limit switches 2E51-N060 and 2E51-N061 closed, relay 2E51-K52A 

10 will energize, causing the following functions:

11 • valve 2E51-F029 closes

12 • valve 2E51-F031 closes

13 • “Condensate Storage Tank Low Level” annunciator alarms

14 Control Room System JPM:

15 The task is satisfactorily met when the applicant has reset and opened all main steam isolation 

16 valves at < 200 pounds per square inch differential and then has isolated a subsequent steam 

17 line break by closing valves MS V123 and MS V456 before reaching maximum safe

18 temperatures in two areas (approximately 8 minutes after receiving Group 1 System A and B 

19 alarms).

20 In-Plant System JPM:

21 The task is satisfactorily met when the diesel generator is started, synchronized to the grid, and 

22 loaded at x + y kilowatts.

23 b. Develop Performance Standards for Each Step of the Task

24 For each step of the task, identify a performance standard for successful completion of the step: 

25 • Provide details that explain what is necessary to successfully complete the given step

26 and that allow the examiner to evaluate progress toward completing the task in 

27 accordance with the procedural references.

28 • When possible, also provide details for the expected state of controls and indications 

29 (e.g., switch positions and meter readings), even if the procedural step does not specify

30 these criteria.

31 • Note any important observations that the applicant should make while performing the 

32 step.

33 • Note any specific procedural restrictions on the sequence in which the steps are 

34 performed in the JPM.

35 Facility procedures may be adapted for use as JPMs by identifying performance standards for 

36 each step.
ES-3.2, Page 7 of 18

1 c. Identify the Critical Steps and Use Verifiable Actions

2 The critical steps are the task steps that the applicant must perform correctly (i.e., accurately, in 

3 the proper sequence, and at the proper time) to accomplish the task standard. Ensure that 

4 each critical step has a performance standard as described in 1.b, above. Clearly designate the 

5 critical steps on the JPM worksheet for the NRC examiner. For tasks that involve a procedure 

6 step with multiple parts or elements, clearly identify which parts or elements of the procedure 

7 step are the critical steps.

8 Although critical steps consist of verifiable actions most of the time, sometimes a critical step 

9 does not involve a verifiable action. These critical steps must still be designated as critical. 

10 Each JPM must contain at least two critical steps. Additionally, a control room or in-plant

11 system JPM cannot consist solely of critical steps that are not verifiable actions. 

12 Example: During a control room JPM in which a system is being aligned, if a manual 

13 valve in the plant must be opened for the task to be complete, the phone call to direct a 

14 field operator to open the valve would be designated a critical step, even though a phone 

15 call is not a verifiable action.

16 Example: During an in-plant JPM that involves aligning a system, if the applicant is 

17 required to call the control room to start a pump or open a valve and the actions taken in 

18 the control room are necessary in order to accomplish the task standard, any phone call 

19 to direct those actions would also be designated a critical step, even though phone calls 

20 are not verifiable actions. Here, the applicant directs the control room operator to 

21 perform what would be considered a verifiable action. 

22 With regard to the use of verifiable actions during in-plant JPMs, write the performance standard 

23 to include what the applicant must describe for how he or she would perform the action, such as 

24 closing a valve or repositioning a switch. For example, the applicant describes how to perform 

25 the verifiable actions, such as, “I am turning the hand-wheel in the clockwise direction and 

26 observing the stem move inward until I feel resistance.” 

27 d. Specify the Job Performance Measure Termination Criteria

28 Determine at what point the JPM should be terminated, considering both applicant success and 

29 failure to meet the task standard. 

30 Include a statement that describes the conditions to be met for the NRC examiner to terminate 

31 the JPM.

32 2. Specify Initial Conditions and Write the Initiating Cue

33 Determine system and plant conditions that permit realistic operator performance of the task. 

34 Provide the applicant sufficient information about the status of the plant and system to facilitate 

35 task performance, without coaching the applicant. If the task is intended to be performed on the 

36 simulator, it is worthwhile to differentiate those specific initial conditions (ICs) and system 

37 realignments that are necessary for the task to be performed as planned from other general 

38 conditions that add realism and set the stage for performing the task but have no real bearing 

39 on its successful execution. Breaking down the ICs in such a manner will simplify the 

40 simultaneous administration of different tasks by two or more applicants. 
ES-3.2, Page 8 of 18

1 Any required operator actions preceding the starting point of the JPM should already be 

2 complete unless a given action is purposely omitted as part of an alternate path JPM.

3 Include an initiating cue that provides the stimulus for the applicant to begin performing the task. 

4 When appropriate, the cue should clearly specify the desired endpoint for the task. For 

5 example, if the applicant should start and load the emergency diesel generator, the cue should 

6 state the load at which the task will be considered complete. 

7 Note: Alternate path tasks, described in Section E, may have an actual endpoint different from 

8 that stated in the initiating cue.

9 Duplicate the ICs and initiating cue on a separate sheet of paper so that they can be handed to 

10 the applicant. This is particularly helpful for tasks with detailed ICs or those that will be 

11 performed in high-noise areas. Ensure that the ICs and initiating cue do not reveal the nature of 

12 any alternate path JPMs that are planned.

13 3. Identify References and Tools

14 Identify the plant procedures necessary for performing the JPM, as well as the procedures that 

15 provide guidance, directions, or standards for performing the task. When reviewing JPMs 

16 selected from the facility licensee’s bank, it is important to ensure that the procedures identified 

17 in the JPM are still current.

18 Identify any special tools or equipment (e.g., a stopwatch, wrench, fuse puller, or spool piece) 

19 that the applicant will need to perform the task. It is helpful to the NRC examiner if the JPM 

20 states the location(s) in which these items may be found. Any required tools should be readily 

21 available to the plant operators; they should not be staged specifically for the examination.

22 4. Develop Examiner Cues

23 Identify appropriate system response cues so that the examiner can provide the applicant with 

24 specific feedback on the component and system reactions to the applicant’s manipulations, 

25 especially those procedural steps identified as critical to task completion. The response cues 

26 are particularly important in the following situations:

27 • in-plant tasks that will be simulated because the applicant will not have available the 

28 normal indications (e.g., alarms, flow rates, temperatures, and pressures) that would be 

29 observed during actual task performance

30 • alternate path JPMs that require the applicant to perform auxiliary procedures when 

31 equipment or instrumentation fails during use

32 System response cues may not be necessary for tasks that are performed on the simulator.

33 If it is possible to anticipate incorrect actions that the applicants might take, it is beneficial to 

34 note the expected system response cues in the JPM as an aid to the NRC examiner.

35 Identify any additional cues or instructions that the examiner might need to provide to the 

36 applicant in response to procedural steps for which the applicant will not be held accountable 

37 (i.e., those steps that have either already been performed or will be performed by other 

38 personnel in remote locations).
ES-3.2, Page 9 of 18

1 5. Develop a Time Standard

2 Determine an average time for completing the task; this is referred to as the validation time, and 

3 it starts when the initiating cue is read to the applicant at the plant location where an operator 

4 would normally be when directed to perform the JPM task. 

5 JPMs that have a task standard that must be completed within a time specified in a regulation,

6 licensing basis document, or commitment to the NRC are called “time-critical” JPMs. Identify 

7 any time-critical JPMs and ensure that the timeline can be met during JPM validation. For 

8 time-critical JPMs, specify the time limit in the JPM task standard. 

9 The validation time and completion time for a time-critical JPM may differ, especially if the 

10 applicant needs to complete additional steps following the time-critical element of the assigned 

11 task.

12 6. Review and Approve Job Performance Measures Using the Instructions in ES-2.3, 

13 “Reviewing and Approving Operator Licensing Initial Examinations.”

14 7. Validate Job Performance Measures

15 Validate each JPM before exam administration; changes, other than editorial ones, should not 

16 be made to the JPM after validation activities are complete. Each JPM should be validated as 

17 early as possible before the examination is to be administered to allow time for changes to be 

18 made. The JPM should not be a surprise to the examiners or simulator operators during exam 

19 administration. 

20 E. Additional Instructions for Developing and Using Alternate Path Job 

21 Performance Measures

22 JPMs are intended to be tasks that an operator must be able to perform that relate to the 

23 operator’s job task analysis. Operators are frequently challenged to perform auxiliary 

24 procedures when equipment or instrumentation fails during use. Therefore, the NRC expects 

25 applicants to be able to use alternative methods to perform tasks. 

26 To evaluate whether the applicant has the skills and knowledge at the level needed to safely 

27 operate the system, and the ability to recognize when malfunctions have occurred and use 

28 procedures to respond, incorporate instrumentation or component malfunctions that require the 

29 applicant to perform actions other than those completed when a system responds normally. 

30 The alternate path must differ from the normal path in order to test the applicant’s ability to use 

31 an alternate operation. 

32 The alternate path JPM provides an excellent opportunity to observe how the applicant

33 recognizes and diagnoses an unexpected system response and then executes alternative paths 

34 within the wide spectrum of procedures under the applicant’s cognizance that would not 

35 otherwise be examined. Develop alternate path JPMs using the instructions in Section D and 

36 the additional instructions below for alternate path JPMs.

37 1. Success Path

38 The JPM must have a valid, facility licensee-endorsed success path. This path may require 

39 analyzing ICs to determine an alternative method for completing the task, mitigating a 

40 system-related problem that occurs during the task, or realigning the system.
ES-3.2, Page 10 of 18

1 2. Procedurally Driven

2 The JPM must be based on a procedure that addresses required actions (i.e., if the JPM 

3 requires an alternative method to complete the task, the procedure would have an exit step that 

4 directs the use of that alternative method). The applicant may be required to use some 

5 common practices endorsed by the facility licensee that are addressed through generic 

6 administrative procedures or policies (e.g., shifting controls to manual).

7 3. Logical Sequence

8 The sequence of procedurally driven actions must be logical. For example, an applicant

9 performing a normal evolution when a malfunction occurs would not be expected to enter EOPs. 

10 More realistically, the applicant would attempt to correct the problem by referring to an 

11 annunciator response procedure or abnormal operating procedure. However, an applicant

12 performing a normal evolution may encounter a situation requiring a reactor trip. A JPM should 

13 not involve a cascading sequence of malfunctions, for which several procedures must be used 

14 simultaneously, that occur while performing a task. This type of activity is better tested in the 

15 simulator operating test portion of the examination.

16 4. Independent of Crew Dynamics

17 The JPM should be designed so that the applicant can complete the task or mitigate a problem 

18 that occurs during a task without having to rely on the actions of other control room operators. 

19 This provision does not prohibit simulator operators from acknowledging irrelevant alarms or 

20 unexpected reactions of other systems that are not associated with the task. The JPM may still 

21 require the applicant to use the simulator operator to perform needed manipulations in the plant.

22 F. Forms

23 Form 3.2-1 Administrative Topics Outline

24 Form 3.2-2 Control Room/In-Plant Systems Outline

25 Form 3.2-3 Job Performance Measure Template

26 Form 3.2-4 Job Performance Measure Development Job Aid

27

28
ES-3.2, Page 11 of 18

1 Form 3.2-1 Administrative Topics Outline

2

3

Facility: _______________________________ Date of Examination: _____________

Examination Level: RO SRO Operating Test Number: __________

Administrative Topic (Step 1) Activity and Associated K/A (Step 2) Type Code 

(Step 3)

Conduct of Operations

Conduct of Operations

Equipment Control

Radiation Control

Emergency Plan
ES-3.2, Page 12 of 18

1

2

Instructions for completing Form 3.2-1, “Administrative Topics Outline”

1. For each license level, determine the number of administrative job performance measures 

(JPMs) and topic areas as follows:

Topic

Number of JPMs

RO*

SRO and 

RO 

Retakes

Conduct of 

Operations 1 (or 2) 2

Equipment Control 1 (or 0) 1

Radiation Control 1 (or 0) 1

Emergency Plan 1 (or 0) 1

Total 4 5

2. Enter the associated knowledge and abilities (K/A) statement and summarize the 

administrative activities for each JPM.

3. For each JPM, specify the type codes for location and source as follows:

Location: 

(C)ontrol room, (S)imulator, or Class(R)oom

Source and Source Criteria:

(P)revious two NRC exams (no more than one JPM that is randomly selected

from last two NRC exams

(D)irect from bank (no more than three for ROs, no more than four for senior 

reactor operators (SROs) and RO retakes)

(N)ew or Significantly (M)odified from bank (no fewer than one)

* Reactor operator (RO) applicants 

do not need to be evaluated on 

every topic (i.e., “Equipment 

Control,” “Radiation Control,” or 

“Emergency Plan” can be omitted by 

doubling up on “Conduct of 

Operations”), unless the applicant is 

taking only the administrative topics 

part of the operating test (with a 

waiver or excusal of the other 

portions).
ES-3.2, Page 13 of 18

1 Form 3.2-2 Control Room/In-Plant Systems Outline

Facility: ____________________________ Date of Examination: _____________

 Operating Test Number: ___________

Exam Level: RO SRO-I SRO-U

System/JPM Title Type Code Safety

Function

Control Room Systems

a.

b.

c.

d.

e.

f.

g.

h.

In-Plant Systems

i.

j.

k.

2
Form 3.2-2 Ins tructions for Control Room/In-Plant Sys tems Outline

ES-3.2, Page 14 of 18

1. Determine the number of control room system and in-plant systems job performance 

measures (JPMs) to develop using the following table:

License Level Control Room In-Plant Total

Reactor Operator (RO) 8 3 11

Senior Reactor 

Operator-Instant 

(SRO-I)

7 3 10

Senior Reactor 

Operator-Upgrade 

(SRO-U)

2 or 3 3 or 2 5

2. Select safety functions and systems for each JPM as follows:

Refer to Section 1.9 of the applicable knowledge and abilities (K/A) catalog for the plant 

systems organized by safety function. For pressurized-water reactor operating tests, the 

primary and secondary systems listed under Safety Function 4, “Heat Removal from 

Reactor Core,” in Section 1.9 of NUREG-1122 or NUREG-2103 may be treated as 

separate safety functions (i.e., two systems, one primary and one secondary, may be 

selected from Safety Function 4).

From the safety function groupings identified in the K/A catalog, select the appropriate 

number of plant systems by safety functions to be evaluated based on the applicant’s 

license level (see the table in step 1). 

The emergency and abnormal plant evolutions listed in Section 1.10 of the applicable K/A 

catalog may also be used to evaluate the applicable safety function (as specified for each 

emergency and abnormal plant evolution in the first tier of the written examination outlines 

in ES-4.1, “Preparing Written Examination Outlines”). 

For RO/SRO-I applicants: Each of the control room systems JPMs and, separately, each 

of the in-plant systems JPMs must evaluate a different safety function, and the same 

system or evolution cannot be used to evaluate more than one safety function in each 

location. One of the control room systems JPMs must be an engineered safety feature. 

For SRO-U applicants: Evaluate SRO-U applicants on five different safety functions. 

One of the control room systems JPMs must be an engineered safety feature, and the 

same system or evolution cannot be used to evaluate more than one safety function.

3. Select a task for each JPM that supports, either directly or indirectly and in a meaningful 

way, the successful fulfillment of the associated safety function. Select the task from the 

applicable K/A catalog or the facility licensee’s site-specific task list. If this task has an 

associated K/A, the K/A should have importance rating of at least 2.5 in the RO column. 

K/As that have importance ratings of less than 2.5 may be used if justified based on plant 

priorities; inform the NRC chief examiner if selecting K/As with an importance rating less 

than 2.5. 
Form 3.2-2 Ins tructions for Control Room/In-Plant Sys tems Outline (continued)

ES-3.2, Page 15 of 18

The selected tasks must be different from the events and evolutions conducted during the 

simulator operating test and tasks tested on the written examination. A task that is similar 

to a simulator scenario event may be acceptable if the actions required to complete the 

task are significantly different from those required in response to the scenario event.

Apply the following specific task selection criteria:

• At least one of the tasks shall be related to a shutdown or low-power condition.

• Four to six of the tasks for RO and SRO-I applicants shall require execution of 

alternative paths within the facility licensee’s operating procedures. Two to three of the 

tasks for SRO-U applicants shall require the execution of alternative paths within the 

facility licensee’s operating procedures. 

• At least one alternate path JPM must be new or modified from the bank.

• At least one of the tasks conducted in the plant shall evaluate the applicant’s ability to 

implement actions required during an emergency or abnormal condition.

• At least one of the tasks conducted in the plant shall require the applicant to enter the 

radiologically controlled area. This provides an excellent opportunity for the applicant 

to discuss or demonstrate radiation control administrative subjects.

If it is not possible to develop or locate a suitable task for a selected system, return to 

step 2 and select a different system. 

4. For each JPM, specify the codes for type, source, and location:

Code License Level Criteria

RO SRO-I SRO-U

(A)lternate path 4 - 6 4 - 6 2 - 3

(C)ontrol room

(D)irect from bank ≤ 9 ≤ 8 ≤ 4

(E)mergency or abnormal in-plant ≥ 1 ≥ 1 ≥ 1

(EN)gineered safety feature (for control room

system)

≥ 1 ≥ 1 ≥ 1 

(L)ow power/shutdown ≥ 1 ≥ 1 ≥ 1

(N)ew or (M)odified from bank (must apply to 

at least one alternate path JPM)

≥ 2 ≥ 2 ≥ 1

(P)revious two exams (randomly selected) ≤ 3 ≤ 3 ≤ 2

(R)adiologically Controlled Area ≥ 1 ≥ 1 ≥ 1

(S)imulator
ES-3.2, Page 16 of 18

Form 3.2-3 Job Performance Measure Template

Facility: Job Performance Measure No.: 

Task Title: Task No.: 

System: Knowledge/Ability Reference and Rating: 

Time-Critical Task: (Yes/No) Alternate Path: (Yes/No)

Validated Time: 

Method of Testing: Simulated Performance _____ Actual Performance ____

Location: Classroom __ Simulator ___ Plant ___

Task Standard: 

Required Materials (procedures, equipment):

General References:

Notes to Examiner:

Simulator Requirements:

Initial Conditions:

Initiating Cues: 
ES-3.2, Page 17 of 18

Form 3.2-3 Job Performance Measure Template (page 2)

Steps and Performance Standards

Note: Critical steps are denoted with a check mark  [or other symbol inserted here]

Start Time: _______ End Time: ________

STEP PERFORMANCE STANDARD CRITICAL GRADE 

(S/U)

1. Applicant... Applicant... 

Cue:

Comments:

2.

Cue:

Comments:

Note:

3.

Cue:

Comments:

4.

Cue:

Comments:

Note:

Terminating cue:
ES-3.2, Page 18 of 18

Form 3.2-4 Job Performance Measure Development Job Aid

This form can be used to verify that the relevant criteria are satisfied.

Every job performance measure (JPM) should do the following:

1. ___ Be supported by the facility licensee’s job task analysis.

2. ___ Be operationally important (meet the applicable knowledge and abilities catalog 

threshold criterion (2.5 for initial exams and 3.0 for requalification exams) or as 

determined by the facility licensee and agreed to by the U.S. Nuclear Regulatory 

Commission). JPMs shall not test only for simple recall or memorization.

3. ___ Be designed as either senior reactor (SRO)-only, reactor operator (RO)/SRO, or 

auxiliary operator/RO/SRO.

4. Include the following, as applicable:

a. ___ task standard

b. ___ initial conditions

c. ___ initiating cues

d. ___ references and tools, including associated procedures

e. ___ validated time limits (average time allowed for completion) and specific 

designation of those JPMs that are deemed to be time critical by the 

facility licensee operations department

f. ___ operationally important specific performance criteria that include:

(1) ___ expected actions with exact control and indication nomenclature 

and criteria (e.g., switch position and meter reading), even if the 

procedural step does not specify these criteria

(2) ___ system response and other cues that are complete and correct 

so that the examiner can properly cue the applicant, if asked

(3) ___ statements describing important observations that the applicant

should make

(4) ___ identification of critical steps

(5) ___ restrictions on the sequence of steps
ES-3.3, Page 1 of 17

1 3.3 TESTING GUIDELINES FOR DYNAMIC SIMULATOR SCENARIOS

2 This examination standard provides a framework for preparing and evaluating dynamic 

3 scenarios (referred to as “simulator scenarios” or “scenarios” throughout this examination

4 standard) to ensure that they are of appropriate scope, depth and complexity for the NRC’s

5 simulator operating test portion of the operator licensing initial examination and requalification 

6 examination. 

7

8 Specifically, this examination standard includes details for the following:

9

10 • a basic procedure for developing new simulator scenarios, including the associated 

11 qualitative and quantitative scenario attributes

12

13 • the critical task (CT) methodology

14

15 Adhering to the concepts and guidelines discussed herein, in association with the specific 

16 criteria cited in ES-3.4, “Developing Scenarios,” or ES-6.4, “Requalification Dynamic Simulator 

17 Examinations,” as applicable, will enhance the consistency and validity of the dynamic simulator 

18 operating tests.

19

20 A. Integrated Scenario Development

21 This section summarizes the major activities that contribute to the development of dynamic 

22 simulator scenarios. The instructions apply to both operator licensing initial and requalification 

23 examination programs, except as noted. Although they are written from the perspective of new 

24 scenario development, the instructions also apply when modifying existing scenarios for reuse 

25 and when assessing the quality of proposed scenarios.

26 1. Identify Scenario Objectives

27 The basic objective of an operating test simulator scenario is to evaluate the operator’s ability to 

28 respond to events that are most appropriately tested in a dynamic simulator environment. 

29 Specifically, such events include those that require the operators to demonstrate their 

30 knowledge of integrated plant operations, as well as their ability to diagnose abnormal plant 

31 conditions and work together to mitigate plant transients that exercise their knowledge and use 

32 of abnormal operating procedures and EOPs. Additionally, the scenario must require the 

33 operators (usually SROs) to use technical specifications (TS) and, for requalification 

34 examinations, to implement the emergency plan.

35

36 Requalification scenarios must have specific objectives derived, in part, from the facility 

37 licensee’s requalification training program objectives. A requalification examination should not 

38 be limited to topics covered in the requalification cycle.

39

40 2. Select Initial Conditions

41 The initial conditions (IC) established for a scenario must allow the scenario to commence 

42 realistically. In other words, the IC should be representative of a typical plant status with various 

43 components, instruments, and annunciators out of service. It is also realistic to have 

44 maintenance or surveillance activities in progress. All, some, or even none of these ICs may 

45 have a bearing on subsequent scenario events. ICs should also be changed frequently to 
ES-3.3, Page 2 of 17

1 prevent future events from becoming predictable. In addition, ICs (including power levels)

2 should vary among the scenarios and periodically include startup and low-power situations.

3 The IC, including any items that would normally be addressed during the shift turnover, should 

4 be described briefly in the space provided at the top of Form 3.3-1, “Scenario Outline.”

5

6 3. Select Events

7 After establishing the IC, a sequence of events designed to achieve the scenario objectives

8 should be selected. Section B discusses the qualitative and quantitative attributes to use when 

9 selecting events. For operator licensing initial examinations, ES-3.4 enumerates specific 

10 requirements for each quantitative criterion. ES-6.4 discusses NRC requalification 

11 examinations.

12

13 Each event has or contributes to an objective, whether it is to evaluate the operator’s knowledge 

14 of a recent system modification, his or her ability to respond to a safety-significant event, or his 

15 or her use of the TS for a particular safety-related component. Uncomplicated events that 

16 require no operator action beyond the acknowledgment of alarms and verification of automatic 

17 actions provide little basis for evaluating the operator’s competence. As such, the operating test 

18 should not include such events they are necessary to set the stage for subsequent events or to 

19 test the SRO applicant’s knowledge of TS actions.

20

21 Scenarios should be developed so that each type of event (i.e., normal evolutions, instrument 

22 failures, component failures, and major plant transients) affects various systems. Having one 

23 equipment failure cause or exacerbate another can also be useful to evaluate the operators’ 

24 understanding of system and component interactions. Balancing the severity of events and the 

25 demands they place on each operating position (e.g., RO and balance of plant (BOP)) will allow 

26 each operator to demonstrate his or her competence across a range of conditions.

27

28 All events do not have to be linked; one event need not occur for the next event to logically 

29 occur (although in many instances, such a relationship adds to the credibility of the scenario). 

30 However, the scenario should not consist of a series of totally unrelated events. A well-crafted 

31 scenario flows from event to event, giving the operators enough time in each event to analyze 

32 what has happened, evaluate the consequences of their action (or inaction), assign a priority to 

33 the event given the existing plant conditions, and determine a course of action. Care should be 

34 taken so that one event does not fully mask the symptoms of another, because the operators 

35 could overlook the malfunction and cause the event or competency coverage for the scenario 

36 set to be deficient.

37

38 4. Document Events

39 Each planned operation, malfunction, and transient should be recorded on Form 3.3-1 and 

40 numbered sequentially. Each event should be cross-referenced to a simulator malfunction 

41 number (or numbers), if applicable, or the simulator instructions that must be entered should be 

42 described briefly.

43

44 For each event listed on Form 3.3-1, Form 3.3-2, “Required Operator Actions” (or equivalent), 

45 should be prepared as follows:

46

47 • Enter the scenario, event, and page numbers and a brief description of the event at the 

48 top of the form. 
ES-3.3, Page 3 of 17

1 • Include information about when the event is to be initiated (e.g., by signal of the lead 

2 examiner/evaluator, timeline, or plant parameter). 

3

4 • Identify the symptoms or cues that the operators will be provided, the expected actions 

5 to be taken, communications to be made, the references to be used by each operating 

6 position (e.g., the SRO, RO, and BOP operators) on the crew, and the event terminus 

7 (i.e., the anticipated point at which the examiners or evaluators will have enough 

8 information on operator performance to move on to the next event).

9

10 • Include every required operator action; this is particularly important for the CTs (refer to 

11 Section C) and other verifiable actions and behaviors that will provide a useful basis for 

12 evaluating the operators’ competence. Include both the procedure steps and the

13 verifiable actions that the applicants are expected to take. For example, instead of only 

14 stating, “The BOP starts emergency diesel generator A in accordance with 

15 Procedure XYZ, Steps 1.10–1.25,” reference the procedure steps and include verifiable 

16 actions for that set of procedure steps that the BOP is expected to perform.

17

18 • List expected actions in chronological order. Certain actions may be required throughout 

19 the event (e.g., if a safety or relief valve fails open, the operators should continually 

20 monitor pressure and water level). Flag these actions to show that they are continuous.

21

22 • Space the expected actions on Form 3.3-2 to allow room for the examiner to document 

23 the operator’s performance during the simulator test. Leave the far-left column of the 

24 form blank so that the examiners can record the time that key actions occurred during 

25 the test.

26

27 • Flag all CTs in a manner that makes them apparent to the individuals who will administer 

28 the operating test (e.g., by using underlines, asterisks, or bold type). Identify the 

29 measurable performance standard for each CT (refer to Section C).

30

31 • Identify and document events that impact TS functions (such as for inoperable 

32 instruments).

33

34 • When possible, include setpoints and other parameters to provide an objective method 

35 for evaluating the operator’s performance. Statements such as, “performs actions in 

36 accordance with Procedure XXXXX,” generally do not provide sufficient guidance and 

37 are inadequate. However, the statement “performs actions of steps XXX of 

38 Procedure XXX (attached)” is acceptable.

39

40 5. Determine the Scenario Endpoint

41 The endpoint of the scenario should be specified on the last operator action sheet (Form 3.3-2) 

42 by identifying a particular plant condition, procedural step, or other point that is clearly 

43 recognizable. For requalification examinations, the scenario should not be terminated until the 

44 stated objectives have been achieved.

45

46 6. Validate the Scenario

47 Every scenario should be validated to ensure that it will run as intended.

48
ES-3.3, Page 4 of 17

1 B. Scenario Attributes

2 All valid scenarios contain common attributes that make them useful as evaluation tools. A 

3 properly constructed scenario provides for an accurate test of each individual applicant or

4 operator’s skills and abilities, as well as an opportunity to evaluate the crew members’ 

5 team-dependent skills and abilities. Each scenario must be of sufficient scope and complexity 

6 to demonstrate the difference between competent operators and crews and those who are not 

7 performing at an acceptable level. Scenarios also must require the crew to demonstrate its 

8 ability as a team to adequately protect public health and safety in emergency conditions, using 

9 the facility licensee’s EOPs.

10

11 A scenario consists of both qualitative and quantitative attributes. No single qualitative or 

12 quantitative attribute or group of attributes can be used to determine the acceptability of a 

13 scenario. However, a trained examiner should be able to assess the adequacy of a scenario or 

14 develop a new scenario, using both sets of attributes. This assessment, combined with 

15 validation of the scenario on a real-time basis, should be enough to determine whether a 

16 scenario is an acceptable tool for use in measuring the competency of a crew or its individual 

17 members.

18

19 1. Qualitative Attributes

20 a. Realism/Credibility

21 Introducing unrealistic or incredible events into a scenario can affect the validity of the scenario. 

22 Piping, component, and instrument failures often occur in such a way that deterioration can be 

23 tracked over a discrete time period (e.g., a small leak that propagates over time or a pump 

24 failure preceded by a high-vibration condition). Including such precursors in scenarios is 

25 important, where appropriate. A great deal of evaluative feedback can be obtained by 

26 observing how an operator or crew responds to a gradually worsening condition. One technique 

27 is to insert an event precursor (e.g., a small steam generator tube leak) and maintain the plant 

28 at a slightly degraded condition to observe how the crew incorporates that condition into its 

29 conduct of subsequent plant operations.

30

31 Although scenarios may include faults that occur with little or no warning (e.g., valve operators 

32 fail, fires occur in breakers or transformers, undetected pipe erosion results in piping failures), 

33 such faults often provide minimal evaluative benefits because they happen so suddenly that 

34 operators have little to do but watch the event unfold. These events are most useful when trying 

35 to establish a plant condition for subsequent evaluation goals or to assess the ability of an 

36 operator or crew to use procedures in a symptom-based, rather than an event-based, mode.

37

38 Mechanistic component failures are well-documented events that occur each year and 

39 sometimes in multiple numbers. However, nonmechanistic failures (e.g., pipe breaks) generally 

40 occur singularly; therefore, unless there is a connective precursor, such as a seismic event, it 

41 would not be realistic or credible to have several piping systems fail during any one scenario.

42

43 Simulated events that appear to violate the laws of physics and thermodynamics contribute to 

44 negative training and are unacceptable. Time-compression techniques, discussed further 

45 below, may also contribute to negative training. However, if a scenario is intended to evaluate a 

46 crew’s ability to execute procedural steps that may take a long time to reach during an event 

47 (e.g., hydrogen generation during a core uncovering event), such a technique may be useful. In 

48 such instances, the scenario must contain a cue that, when the crew detects the indications for 
ES-3.3, Page 5 of 17

1 such events, they are informed that the parameters are not responding as expected for the 

2 actual plant and that time is being compressed. This cue should be presented at the first 

3 opportunity that does not distract the crew from responding to available indications and before 

4 the crew challenges the validity of those indications.

5

6 Time compression can also include a “jump” forward in time to test important tasks that occur 

7 after a prolonged period of time. An example of such time compression is a jump more than 

8 21 hours forward in time following a loss of all alternating-current power on an AP1000 plant so 

9 that actions to be taken before a loss of direct-current power can be evaluated. If this “jump” is 

10 used, the crew must receive with a turnover or cue addressing any relevant plant conditions that 

11 changed due to the time compression. The crew should also be allowed an opportunity to brief 

12 before resuming the scenario.

13

14 b. Event Sequencing

15 The sequence of events has a major effect on establishing the complexity of a simulator 

16 scenario. The pace at which malfunctions are entered can also adversely affect the way an 

17 operator or a crew responds.

18

19 Malfunctions may be entered simultaneously at separate control panel locations provided that 

20 an individual applicant can handle each event without requiring extensive assistance. Too short 

21 a time between malfunctions may mask the effects of a particular malfunction and divert the 

22 operators’ attention. This limits the observers’ ability to evaluate the operators’ response to the 

23 earlier malfunction and may be prejudicial to a fair evaluation. Conversely, extending the time 

24 between malfunctions so that no operator activity is in progress may cause undue stress. 

25 During an examination, the operators expect something to occur; too much time between events 

26 should be avoided.

27

28 Therefore, the insertion of malfunctions in the scenario should be carefully timed. Rigorously 

29 following a planned time sequence of events is often less valid than initiating malfunctions on 

30 the basis of plant parameters or operator actions. The appropriate sequencing of events relates 

31 directly to the objectives of the scenario.

32

33 Event sequencing may involve time compression to speed up the response of key parameters 

34 so that the scenario can proceed to the next event within a reasonable time. Time compression 

35 may be accomplished by adjusting parameter indications or accelerating plant behavior 

36 characteristics so that plant indications trigger an event more quickly than would typically occur 

37 in reality (e.g., opening a drain path from a steam generator that is not noticeable to the 

38 operator so that the simulation reaches the entry conditions for a loss of heat sink.) This 

39 method is acceptable as long as the time compression gives the operators time to perform tasks 

40 that they would typically perform during the period in which time is compressed. To avoid 

41 wasting the operators’ time determining the validity of the plant indications, the examiner should 

42 inform the crew before the scenario begins that time compression may be used during an event 

43 and should debrief the crew after the scenario to minimize the potential for negative training.

44

45 Important evaluative benefit is gained in terms of safety significance by having key components 

46 or instruments fail after entering the EOPs. This process compels the operators to respond 

47 immediately to a safety-related situation by taking alternative actions to mitigate the event. This 

48 process also allows for a better evaluation of the operators’ overall knowledge of plant 

49 procedures and systems because the event must be incorporated into the mitigation strategy for 

50 the remainder of the scenario. Exam developers should be careful when selecting failures 
ES-3.3, Page 6 of 17

1 because some instrument and component failures that are initiated after the major transient 

2 require little action and may provide little insight into the operator’s competence.

3

4 c. Simulator Modeling

5 The scenario shall not exceed the limits of the facility licensee’s configuration management 

6 system by altering a simulator model to obtain a desired effect. For example, it is not 

7 appropriate to increase the posttrip decay heat input in order to maximize internal core 

8 temperatures during a loss-of-cooling event; the simulator model must be allowed to perform as 

9 designed. The scenario may simulate events for which a simulator malfunction does not exist 

10 by using overrides or remote functions for local operator actions. An example would be failing 

11 indicators to simulate an inoperable component.

12

13 d. Evaluating Competencies

14 ES-3.6, “Grading and Documenting Operating Tests for Operator Licensing Initial 

15 Examinations,” describes the individual competencies that apply to the RO and SRO license 

16 levels during initial licensing examinations. ES-6.4 describes competencies that apply to 

17 individuals and crews during NRC requalification examinations.

18

19 The scenario set needs to allow for the evaluation of each applicant in all the rating factors 

20 within each competency. Scenarios that require little analysis or problem solving and few 

21 verifiable actions may not provide an adequate basis to evaluate the required rating factors.

22

23 NRC examiners depend on verifiable actions to evaluate competencies and make conclusions 

24 about an applicant’s ability to operate the plant.

25

26 e. Level of Difficulty

27 The simulator operating test must differentiate between those examinees who have adequately 

28 obtained the minimum level of knowledge, skills, and abilities required to be licensed operators

29 and those who have not. Simulator scenarios that are either too easy or too difficult are not 

30 effective differentiators.

31

32 In general, the level of difficulty of a scenario will increase with an increase in the scenario’s

33 quantitative attributes, such as the number of malfunctions or CTs. However, the number of 

34 quantitative attributes does not always indicate the scenario’s level of difficulty (two scenarios 

35 having the same quantitative attributes can vary significantly in the level of difficulty). 

36 Unfortunately, there are no definitive minimum or maximum attribute values that can be used to 

37 identify inappropriate scenarios that will not discriminate because they are too easy or difficult.

38

39 The two most important determinants of the level of difficulty of a simulator scenario are the 

40 amount of analysis and problem solving and the number of operator actions required to mitigate 

41 the events in the scenario. Malfunctions that require analysis or problem solving increase the 

42 level of difficulty because they require the examinees to integrate a number of system 

43 conditions, evaluate their interrelationships, and take actions that demonstrate an understanding 

44 of the underlying concepts. Scenarios that consist of a number of unrelated malfunctions that 

45 require little or no operator analysis or response are generally less challenging.

46
ES-3.3, Page 7 of 17

1 2. Quantitative Attributes

2 Those attributes discussed in the previous section provide for a qualitative assessment of the 

3 complexity of a simulator scenario. However, some characteristics of a scenario can be 

4 quantified and generally have a bearing on the complexity and level of difficulty of the scenario. 

5

6 These characteristics are described below, and ES-3.4 and ES-6.4 enumerate a target range for 

7 each attribute that is applicable to the operator licensing initial and requalification examination, 

8 respectively. The ranges are not absolute limitations; some scenarios may provide an excellent 

9 evaluation tool but may not fit within the ranges. A scenario that does not fit into these ranges 

10 should be evaluated to ensure that it is appropriate.

11

12 a. Normal Evolutions

13 Normal evolutions include activities, such as a feed pump startup, turbine loading, generator 

14 synchronization, and reactivity manipulations, that involve evolutions such as a reactor startup 

15 or a change in power with boron concentration, control rods, or core flow. 

16

17 Normal evolutions can be used as a backdrop on which to stage the abnormal or emergency 

18 situations. For example, a main feedwater control valve may fail passively (i.e., as is) before the 

19 operators conduct a normal power change.

20

21 Time-consuming normal evolutions (such as a power escalation from low power) can provide an 

22 opportunity to evaluate the SRO’s supervisory or resource management skills. Events such as 

23 component or instrument failures may be added to challenge the operators while continuing the 

24 power escalation.

25

26 Short surveillances (e.g., exercising safety rods or paralleling the emergency diesel generator 

27 with the grid) may be used to examine the operator’s dexterity on the control panels or to 

28 involve operators who are not engaged in other activities.

29

30 b. Reactivity Manipulations

31 Reactivity manipulations include activities that produce a clearly observable plant response, 

32 such as bringing the reactor critical from a substantially subcritical state, raising power to the 

33 point at which reactivity feedback from nuclear heat addition is noticeable and a heatup rate is 

34 established, changing reactor power manually with control rods or recirculation flow, or borating 

35 or diluting the reactor coolant.

36

37 c. Total Events

38 Total events are the number of instrument (e.g., nuclear, control, or process) and component 

39 failures (e.g., pump, motor, valve, or pipe) that constitute a scenario, including those initiated 

40 after EOP entry. To count as a separate event, they must involve a significant system response 

41 and require operator action to correct. For example, an anticipated transient without scram 

42 (ATWS) is a single event, regardless of how many instructions a facility licensee simulator 

43 operator must program to produce it.

44

45 Components that are placed out of service at the beginning of a scenario as part of the shift 

46 turnover conditions, and of which the crew is made aware, are not considered events nor 

47 counted for TS evaluations. 
ES-3.3, Page 8 of 17

1 Component or instrument failures that require no verifiable actions do not count toward the 

2 recommended total number of events.

3

4 d. Events after Emergency Operating Procedure Entry

5 A scenario must include at least one event that involves failure of instruments or components 

6 after the EOPs have been entered. The malfunction(s) for this event should influence the 

7 operators’ choice of mitigation strategy. For example, failing a high-pressure safety injection 

8 pump to start on a large-break loss-of-coolant accident (LOCA) does not affect the mitigation 

9 strategy; however, this would have an effect if it were the only available high-pressure safety 

10 injection pump on a small-break LOCA. These malfunctions may have been initiated at the 

11 beginning of the scenario or before EOP entry. 

12

13 An instrument or component failure occurring before the major event/EOP entry can only be

14 counted again, as a separate event, after the major event/EOP entry if the operator actions 

15 differ distinctly from the actions before the major event/EOP entry for the same malfunction, as 

16 in the following examples:

17 • Event 4 of a scenario had a loss of an electrical bus with a failure of the bus auto 

18 transfer, resulting in a loss of normal letdown. As a result, the operators had to place 

19 excess letdown in service.

20 • Event 5 of the same scenario was the major event, with a steam generator tube rupture

21 (SGTR) occurring on the reactor trip. Because of the reactor trip and the previous bus 

22 loss, normal letdown cannot be established, and the operator at the controls (ATC) has 

23 to establish excess letdown again. The actions of the ATC operator do not differ from

24 the ATC actions taken before the major event and thus cannot be counted as a separate 

25 event for this applicant position.

26 e. Abnormal Events

27 Each scenario shall evaluate the operator’s ability to implement abnormal operating procedures. 

28 This attribute also includes alarm response procedures that involve significant and verifiable 

29 actions. An abnormal event may or may not be a precursor to the major transient, although it 

30 can add to the credibility of a scenario, such as preceding a total loss of feedwater with a single 

31 feed pump trip. However, certain events may cue the operators about subsequent events. 

32 Therefore, for a scenario derived from the facility licensee’s bank, the developer should vary or 

33 modify the precursor events that lead to the major transient. The scenario should include

34 abnormal events that are not always predictive of the same major transient (e.g., a steam 

35 generator tube leak does not always lead to a subsequent tube rupture).

36

37 Some abnormal events for each scenario should require the operators to recognize and apply 

38 TS. 

39

40 Components or instrument failures that occur following EOP entry do not count toward the 

41 recommended total number of abnormal events, but they do count as post-EOP entry 

42 malfunctions.

43
ES-3.3, Page 9 of 17

1 f. Major Transient

2 A major transient is one that has a significant effect on plant safety and leads to an automatic 

3 (or manual, if initiated by an operator) protective system actuation, such as a reactor trip or an 

4 engineered safety system actuation. A single major transient that actuates more than one 

5 automatic protective system will count as a single major transient. Examples include a loss of 

6 offsite power, LOCA, steam or feed line break, SGTR, and loss of feedwater. A major transient 

7 should normally involve activation of the facility licensee’s emergency plan.

8

9 g. Emergency Operating Procedures Used

10 Operators use EOPs to respond to events that challenge plant safety functions. Scenarios

11 should require significant operator action to maintain plant safety and, where a success path 

12 exists, prevent further degradation to plant safety.

13

14 A scenario that requires the operators to refer to many different EOPs may not be as complex 

15 as a scenario for which only one EOP is used but that requires the use of alternative decision 

16 paths and prioritization of actions within the EOP to deal with the situation. Therefore, this 

17 attribute reflects the EOPs that have measurable actions that the crew must take. Moreover, 

18 the primary scram response procedure that serves as the entry point for the EOPs is not 

19 counted. For AP1000 plants, for which the EOP entry procedure is prolonged, the primary 

20 reactor trip procedure can be counted if there are significant operator actions and the NRC 

21 approves doing so as part of the exam approval process. 

22

23 For boiling-water reactors (BWRs), the number of EOPs used shall be counted consistent with 

24 the four top-level guidelines for emergency procedures: 

25

26 1. reactor pressure vessel (RPV) control 

27 2. primary containment control 

28 3. secondary containment control 

29 4. radioactivity release control 

30

31 The use of multiple control sections of these guidelines does not count separately as EOPs 

32 used. For example, use of RPV level control and RPV pressure control should be counted as 

33 one EOP used (RPV control).

34

35 h. Emergency Operating Procedure Contingency Procedures Used

36 Contingency procedures are used when there is a significant challenge to a critical safety 

37 function or if plant conditions have become severely degraded. Therefore, using them in a 

38 scenario provides an opportunity to observe the operators’ attempt to execute a mitigation 

39 strategy that clearly has substantial safety significance to the plant and public health and safety. 

40 For initial licensing examinations, the scenario set must include at least one event that requires

41 the applicants to enter and perform safety-related tasks within an EOP contingency procedure.

42

43 Scenario developers and reviewers should consider the following list of contingency procedures, 

44 which is neither unique nor all inclusive, as a set of general guides that may not fully apply to all 

45 scenarios sets:

46
ES-3.3, Page 10 of 17

1 1. Westinghouse pressurized-water reactor (PWR)

2 Emergency contingency action procedures include the following:

3

4 • loss of all alternating current power with or without safety injection required

5 • loss of emergency coolant recirculation

6 • LOCA outside containment

7 • uncontrolled depressurization of all steam generators

8 • SGTR with loss of reactor coolant—subcooled recovery

9 • SGTR with loss of reactor coolant—saturated recovery

10 • SGTR without pressurizer pressure control

11 Functional recovery procedures entered as a result of red or orange conditions on the 

12 critical safety function status trees include the following:

13

14 • response to nuclear power generation/ATWS

15 • response to inadequate core cooling

16 • response to degraded core cooling

17 • response to loss of secondary heat sink

18 • response to imminent pressurized thermal shock conditions

19 • response to high containment pressure

20 • response to containment flooding

21

22 2. Westinghouse AP1000

23 Emergency contingency action procedures include the following:

24

25 • LOCA outside of containment

26

27 Functional recovery procedures entered as a result of red or orange conditions on the 

28 critical safety function status trees include the following:

29

30 • response to nuclear power generation/ATWS

31 • response to inadequate core cooling

32 • response to degraded core cooling

33 • response to loss of heat sink

34 • response to imminent pressurized thermal shock conditions

35 • response to high containment pressure

36 • response to containment flooding

37 • response to high pressurizer level

38

39 3. Combustion Engineering PWR

40 EOP contingency procedures for Combustion Engineering plants include:

41

42 • entry into functional recovery procedures 

43 • transition among functional recovery safety function success paths
ES-3.3, Page 11 of 17

1 • transition from one safety function to another within the functional recovery 

2 procedures

3 4. Babcock and Wilcox PWR

4 The Babcock and Wilcox EOP structure does not identify procedures that can be easily 

5 recognized as contingency procedures. However, the descriptions above for the 

6 Westinghouse contingency procedures provide guidance on the types of events to 

7 consider.

8

9 5. General Electric BWR

10 EOP contingency procedures for General Electric plants include:

11

12 • alternative level control

13 • emergency RPV depressurization

14 • level/power control (ATWS)

15 • RPV flooding

16

17 i. Simulator Run Time

18 A scenario should be designed to run approximately 60 to 90 minutes. However, this does not 

19 preclude scenarios taking more or less time. The nominal run time of 60 minutes may not 

20 provide sufficient time to conduct a scenario that progresses through several EOPs or requires 

21 the performance of fairly involved procedural steps. Very meaningful and involved scenarios 

22 can be conducted in less time, but care should be taken not to place an undue burden on the 

23 operators by initiating malfunctions at too rapid a pace. This parameter is one of many that 

24 should be considered in assessing the overall quality of a scenario. As long as the scenario 

25 meets the other criteria stated here, the scenario run time is a secondary concern.

26

27 j. Emergency Operating Procedure Run Time

28 The time during which the operators are involved in EOPs has a strong relationship to the 

29 complexity of the scenario because most CTs occur in the EOPs and the actions the operators 

30 take have the most potential to affect public health and safety. Therefore, a significant 

31 percentage of the time a scenario is progressing should be spent in the EOPs. Usually, more 

32 time is required when contingency procedures are in effect because it generally takes some 

33 time for the plant to degrade such that critical safety functions are jeopardized. However, 

34 operators should be evaluated in EOP activities beyond the point at which an event is 

35 diagnosed and initial mitigation actions are taken. Many of the actions taken to stabilize the 

36 plant and recover from a transient are safety significant. Therefore, scenarios should be 

37 allowed to progress so that these operations can be observed.

38

39 However, scenarios should not be solely EOP oriented. Valuable assessments can be made 

40 within abnormal operating procedures with the plant at power because of the level of safety 

41 significance associated with transients in these conditions.

42

43 k. Critical Tasks

44 CTs contain actions that must be performed correctly, either by operators individually or 

45 together with other crew members, to ensure the safety of the plant and the public. Failure to 
ES-3.3, Page 12 of 17

1 successfully complete CTs, either by improper performance or omission, significantly degrades

2 the level of safety afforded to the plant and the public and therefore will result in automatic 

3 failure of the simulator operating test. 

4

5 CTs give the examination team an opportunity to evaluate whether an individual or crew has 

6 successfully completed those actions that are of high significance to the safety of both the plant 

7 and the public. CTs range between simple safety-significant tasks (e.g., starting the standby 

8 liquid control system during an ATWS condition or tripping a reactor coolant pump during a 

9 small-break LOCA) and tasks that require a higher level of skill and involve several crew 

10 members (e.g., executing a rapid cooldown within predefined limits using steam generator 

11 power operated relief valves or using low-pressure injection systems to maintain the vessel level 

12 while cooling the suppression pool). 

13

14 CTs are used during operator licensing initial and requalification examination scenarios in the 

15 following manner:

16

17 • Initial licensing exam scenarios use CTs for the individual operator competency 

18 evaluations because they help the examiner focus on tasks that significantly impact the 

19 safety of the plant or the public. 

20

21 • Requalification exam scenarios use CTs to evaluate crew performance on tasks that are 

22 significant to the safety of the plant or the public. CTs are objective measures for 

23 determining whether the performance of an individual or crew is satisfactory or 

24 unsatisfactory. ES-6.4 includes specific instructions on CTs in dynamic requalification 

25 examinations. 

26

27 When determining the number of CTs designed into a scenario and reviewing the number of 

28 CTs for a scenario set, scenario developers and reviewers should consider the difficulty level 

29 and need for an equitable administration of the operating test. 

30

31 Preidentified CTs are part of scenario design and are included on the scenario outline forms 

32 (Form 3.3-1). Because not all operator actions can be predicted, either an individual or the crew 

33 may “create” a new CT. Additional events may be determined to be CTs after the scenario is 

34 administered. These are considered “postscenario CTs” and are identified by the NRC 

35 examination team after scenario administration. Both preidentified and postscenario CTs are 

36 identified and designated using the same criteria, which Section C explains in detail. 

37

38 C. Critical Task Methodology

39

40 1. Identifying Scenario-Specific Critical Tasks

41 If a facility maintains a CT list derived from guidance provided by its vendor owner’s group, the 

42 scenario developer should reference this list as an aid in identifying CTs. However, the NRC 

43 has not reviewed such CT lists, which may contain tasks that are not sufficiently differentiating 

44 for the purposes of an NRC operating test. Furthermore, scenario CTs depend on both specific 

45 equipment configurations and malfunctions, while owner’s group CTs are based upon specific 

46 accident sequences that may not match those of a given scenario. 

47

48 The developer should apply the following guidance to identify and designate CTs in conjunction 

49 with facility CT lists (or in the absence of such a list):

50
ES-3.3, Page 13 of 17

1 • Where a success path exists, applicants must prevent significant safety challenges; this 

2 includes preventing conditions that warrant the initiation of emergency depressurization 

3 (BWR specific), result in orange or red path critical safety functions (Westinghouse and 

4 AP1000 specific), warrant transitioning to functional recovery guidelines (Combustion 

5 Engineering specific), or adversely impact the implementation of those emergency 

6 operating procedure actions essential to the mitigative strategy for the event in question 

7 (Babcock and Wilcox specific).

8

9 • Applicants must properly implement procedural actions for mitigating significant safety 

10 challenges when those actions directly lead to restoring safety functions.

11 • Applicants must properly implement procedural actions of EOPs when those actions are 

12 essential to an event’s overall mitigative strategy.

13 • Applicants must avoid unnecessarily creating situations that would result in emergency 

14 action level entry or escalation on the loss or potential loss of more than one fission 

15 product barrier in accordance with the facility’s emergency action levels.

16

17 Those applying this guidance should carefully consider the specific equipment 

18 configurations, malfunctions, and accident sequences of a given scenario to ensure that 

19 CTs are designated and bounded in a manner that is reasonable for evaluation 

20 purposes. 

21

22 2. Elements of a Critical Task

23 A CT must possess each of the following elements:

24

25 • Initiating Cue: An initiating cue is an expected signal or notice (i.e., indication, alarm, 

26 communication, or procedure step) that designates when a CT should be performed.

27 The cue need not indicate that the action is a CT.

28

29 • Measurable Performance Standard: The measurable performance standard for a CT 

30 consists of observable actions taken by at least one member of the crew. The 

31 measurable performance standard for a CT includes both an expected action(s) and 

32 boundary conditions that clearly identify at what point a CT must be accomplished; such 

33 conditions must be objective in nature. The boundary conditions for each preidentified 

34 CT should be documented on the scenario outline form (Form 3.3-1) and operator action 

35 form (Form 3.3-2).

36 

37 • Performance Feedback: During the time span of a CT, performance feedback must be 

38 available to at least one member of the crew. This feedback provides the crew members

39 with information about the effect of the crew’s actions or inaction related to or because of 

40 the CT. The crew must be able to determine that its action had an impact or that its 

41 inaction is causing plant conditions to degrade.

42

43 The CT should use objective performance standards that include clear boundary conditions for 

44 when a CT must be accomplished. For example, “if pressure falls below 1,400 pounds per 

45 square inch (psi), start pump XYZ,” is not objective because there is not a lower bound for 

46 system pressure after which the operator performance would be considered deficient. The 

47 operator performing this task could start the pump when pressure reaches 0 psi and still not 
ES-3.3, Page 14 of 17

1 violate the performance standard, even though the operator is expected to start the pump 

2 sooner.

3

4 The following are examples of performance standards that cannot be measured objectively 

5 during a simulator scenario and, therefore, are not suitable for CT performance standards:

6 • understanding (such as an applicant must understand the significance of a certain plant 

7 response)

8 • observing that an expected response has occurred

9 • observing the performance of a system

10

11 The following guidance applies to the establishment of objective boundary conditions for 

12 performance standards:

13

14 • preferred boundary conditions:

15

16  thresholds at which safety functions are severely challenged or lost

17

18  thresholds that result in changes to the mitigative strategy for an event, such as 

19 transitions to contingency procedures or functional recovery procedures

20

21  time-critical operator actions that are applicable to the facility and necessary to 

22 mitigate the event (ensure that the time-critical operator action assumptions and 

23 consequences are still applicable in the specific scenario)

24

25  TS safety limits

26

27 • alternative boundary conditions:

28

29  exiting or transitioning from the procedure that first directs CT accomplishment

30

31  exceeding a parameter value (i.e., limits from the facility’s final safety analysis 

32 report or design documentation), as agreed upon by the NRC chief examiner and 

33 the facility licensee

34

35  the expiration of a reasonable amount of time, as agreed upon by the NRC chief 

36 examiner and the facility licensee

37

38  the boundary conditions for postscenario CTs during major transients, as

39 determined by the lead examiner

40

41 One method for determining CT boundary conditions during scenario development is to allow 

42 the related simulator malfunction(s) to continue with no operator action in the simulator and 

43 evaluate the consequences for useful boundary information.

44
ES-3.3, Page 15 of 17

1 3. What to Avoid when Designating Critical Tasks

2

3 The “CT” designation should not be assigned to generic tasks that have safety significance but 

4 do not meet the elements of a CT.

5

6 Although a crew is not performing optimally if it fails to anticipate a scripted automatic action 

7 given sufficient time to assess plant behavior, crew members are not required to anticipate an 

8 automatic action. A crew member may, at any time, take manual action in advance of an 

9 automatic action if, in the crew member’s judgment, manual action is needed to place the 

10 reactor in a safe condition. If an operator takes an action that the examiners did not expect, the 

11 examiners must further evaluate the individual’s rationale for taking that action. Such 

12 preemptive actions may indicate a misunderstanding of plant conditions or a weakness in 

13 integrated plant knowledge that should be clarified with followup questions. Taking a 

14 preemptive manual action when an automatic action is imminent because of an incorrect action 

15 or inaction does not mitigate the initial incorrect action/inaction.

16

17 Taking manual control of an automatic safety system qualifies as a CT only if the automatic 

18 initiation feature failed to work. It is then safety significant for the crew to take manual actions, 

19 as plant conditions clearly indicate that an automatic action should have occurred and did not. 

20 Moreover, during scenario development and validation, CTs are identified based on those 

21 actions that, if performed incorrectly or omitted, degrade the mitigation strategy needed in the 

22 scenario. If the manual system has also failed and no action will be effective, this should not be 

23 identified as a CT. However, if an operator or the crew significantly deviates from, or fails to 

24 follow, procedures that affect the maintenance of basic safety functions, those actions may form 

25 the basis of a CT identified in the postscenario review.

26

27 Emergency event classification during a simulator scenario does not meet all aspects of the CT 

28 criteria. Emergency event classifications have safety significance, an initiating cue, and a 

29 measurable performance standard, but they do not provide the applicant performance feedback 

30 for the correct or incorrect classification.

31

32 D. Forms

33 Form 3.3-1 Scenario Outline

34 Form 3.3-2 Required Operator Actions
ES-3.3, Page 16 of 17

1 Form 3.3-1 Scenario Outline

2 

Facility: Scenario #:

Scenario Source: Op. Test #:

Examiners: Applicants/

Operators:

Initial Conditions:

Turnover:

Critical Tasks:

Event 

No.

Malf. 

No.

Event 

Type*

Event

Description

* (N)ormal, (R)eactivity, (I)nstrument, (C)omponent, (M)ajor, (TS)Tech Spec, (MC)Manual Control
ES-3.3, Page 17 of 17

Form 3.3-2 Required Operator Actions

Op. Test No.: _____ Scenario No.: _____ Event No.: _____ Page ___ of ___

Event Description: 

Symptoms/Cues: _

_________________________________________________________________

Time Position Applicant’s Actions or Behavior

ES-3.4, Page 1 of 9 

1 3.4 DEVELOPING SCENARIOS 

2 This examination standard contains specific criteria and instructions for selecting and 

3 developing simulator scenarios for the NRC’s simulator operating test portion of the operator 

4 licensing initial examination. This section is designed to be used in conjunction with ES-3.3, 

5 “Testing Guidelines for Dynamic Simulator Scenarios.” Adherence to the concepts and 

6 instructions in these two sections ensures simulator scenarios are of appropriate scope, depth, 

7 and complexity and maintains the consistency and validity of this part of the operating test.

8 

9 Examination developers should develop new scenarios in accordance with these instructions 

10 and modify existing scenarios to conform to these instructions. If a facility licensee encounters 

11 difficulty meeting a requirement in these instructions (e.g., because of large class sizes requiring 

12 the generation of more scenarios than normal), it should coordinate with the NRC chief 

13 examiner to meet the intent of these instructions to the extent possible. 

14

15 A. General Instructions

16 The following are general instructions for developing simulator scenarios for operator licensing 

17 initial examinations.

18

19 • The scenario set consists of new scenarios and scenarios from existing facility licensee 

20 or NRC scenario banks that are then modified. 

21

22 • To maintain test integrity, every scenario shall be new or significantly modified to ensure 

23 that the applicant has not had the opportunity to rehearse or practice the scenario. A 

24 significant modification means that, for each scenario, at least two events have not been 

25 used on the previous two NRC operator licensing initial examinations at that facility. 

26 However, if the associated operator actions in response to an event are different, this 

27 event is not considered to be the same event. Events found in spare scenarios will 

28 count as previously used events if they were made publicly available in the NRC’s 

29 records management system (Agencywide Documents Access and Management 

30 System). 

31

32 – Likewise, the NRC examination can only repeat events that appear on the

33 applicant audit examination if the operator actions in response to the event are

34 significantly different from those required during the audit examination.

35

36 – Because of a limited number of methods for adding reactivity, reactivity

37 manipulation events can be repeated from past examinations.

38

39 • Scenarios extracted from the facility licensee’s bank must be altered to the degree 

40 necessary to prevent the applicants from immediately recognizing the scenarios based 

41 on the ICs, the sequence and repetition of events used, or other cues.

42

43 • The examination author should vary the initial conditions (including power levels), normal 

44 operations, malfunctions, and major transients among the scenario set and periodically

45 include startup and low-power situations. 

46

47 • If any major event is repeated from either of the previous two NRC operator licensing 

48 initial examination operating tests, the examination author should change the major 
ES-3.4, Page 2 of 9

1 event, the ICs, or subsequent malfunctions (or a combination) to alter the course of 

2 action (within the emergency procedures) for the given scenario(s). 

3

4 – The NRC expects that all major events would be broadly sampled over the 

5 course of several operating tests and that no major event will be omitted routinely 

6 without justification. 

7

8 • It may be possible to significantly reduce the number of simulator scenarios required to 

9 examine a large group of applicants by administering the same scenario on the same 

10 day to multiple applicant crews. Provisions must be made to sequester the crews and to 

11 prevent them from discussing the simulator scenario until all of the crews have 

12 completed that scenario.

13

14 • SRO-upgrade (SRO-U) applicants are evaluated on manipulating the controls only if 

15 they are put in the ATC or BOP position to prevent the need for a surrogate to complete 

16 the crew.

17

18 • The scenario should not duplicate operator tasks that appear on the JPM portion of the 

19 operating test or on the written examination unless the operator actions for the same 

20 task are different for the related simulator event.

21

22 • If the facility licensee normally operates with and is required by TS to have more than 

23 two ROs in the control room, the NRC chief examiner may authorize the use of 

24 additional surrogates to fill out the crews. In such cases, plans for the scenarios should

25 ensure that the additional operators do not reduce the examiners’ ability to evaluate 

26 each applicant on the required number of events.

27

28 • The scenario developer should refer to Form 3.6-5, “Reactor Operator Competency 

29 Grading Worksheet for the Simulator Operating Test,” and Form 3.6-6, “Senior Reactor 

30 Operator Competency Grading Worksheet for the Simulator Operating Test, in ES-3.6 to 

31 ensure that each scenario allows the examiner to evaluate the applicant’s performance 

32 on each competency and rating factor. The scenario could give each applicant multiple 

33 opportunities to demonstrate competence in each rating factor.

34

35 B. Specific Instructions for Developing Scenarios

36 The following are specific instructions for developing simulator scenarios for operator licensing 

37 initial examinations.

38

39 1. Determining the Number of Scenarios To Develop

40 Based on the anticipated crew compositions, plan the crews and determine the number of 

41 scenarios to satisfy the following:

42

43 • Each SRO applicant (upgrade and instant) has at least one scenario in the unit 

44 supervisory position (i.e., the shift supervisor, control room supervisor, or unit supervisor 

45 position).

46

47 • Each RO applicant has at least one scenario in the BOP position and one scenario in the 

48 lead reactor operator position (i.e., the ATC position).
ES-3.4, Page 3 of 9

1

2 • Each SRO-instant (SRO-I) applicant has at least one scenario in either the BOP or the 

3 ATC position.

4

5 For example, a crew consisting of two ROs and one SRO-I will normally require three scenarios 

6 to evaluate each applicant’s performance on the primary plant component controls; however, a 

7 surrogate SRO may have to fill the supervisory role while the SRO-I applicant is in the lead 

8 operator position. 

9

10 Include one “spare” or additional scenario for use in case one of the planned scenarios does not 

11 work as intended.

12

13 2. Use of Surrogate Operators

14 • Use a surrogate operator only when necessary to complete an applicant crew for the 

15 simulator test.

16

17 • Do not replace license applicants with surrogates solely because the applicants have 

18 performed the minimum number of events or scenarios. If an applicant would be 

19 exposed to only one additional scenario above the minimum required, do not use a 

20 surrogate operator in place of a license applicant.

21

22 • Do not require or permit an applicant to participate in more than one scenario above the 

23 minimum required. In such a case, use a surrogate operator.

24

25 • If the facility licensee normally operates with, and is required by its TS to have, more 

26 than two ROs in the control room, the NRC chief examiner can allow the use of 

27 additional surrogates to fill out the crews. In such cases, NRC examiners must ensure

28 that the presence of additional operators does not limit the examiners’ ability to evaluate 

29 each applicant on the required number of events.

30

31 The NRC chief examiner must consult the NRC Office of Nuclear Reactor Regulation (NRR)

32 operator licensing program office before examination administration if he or she desires to use 

33 surrogate operators contrary to the instructions above. 

34

35 3. Integrated Scenario Development Steps 1–6 in ES-3.3

36 On the scenario outline Form 3.3-1, provide information about the source of each scenario 

37 (i.e., new or significantly modified). In addition, for modified scenarios, provide information 

38 about the original scenario (e.g., the NRC examination year and name of the facility).

39

40 To minimize the amount of rework that might be required as a result of changes in the planned 

41 scenario events, complete Form 3.3-2 after the NRC chief examiner has reviewed and 

42 commented on the proposed simulator operating test outlines.

43

44 4. Criteria for Specific Events and Evolutions for Operator Licensing Initial Examination 

45 Scenarios and Scenario Sets

46 Select and sequence events and evolutions for each scenario using the testing guidelines in 

47 ES-3.3, along with the targets in Table 3.4-1, “Target Quantitative Attributes per 
ES-3.4, Page 4 of 9

1 Scenario/Scenario Set,” for quantitative attributes per scenario and the required number of

2 events and evolutions for each applicant.

3 Table 3.4-1 Target Quantitative Attributes per Scenario/Scenario Set

4

Quantitative Attribute Target per Scenario

Malfunctions after EOP entry 1–2

Abnormal events 2–4

Major transients 1–2

EOPs entered/requiring substantive actions 1–2

Entry into a contingency EOP with substantive actions 1 per scenario set

Preidentified CTs 2 or more

5

6 The quantitative attribute target ranges specified in Table 3.4-1 are not absolute limitations; 

7 some scenarios may be excellent evaluation tools but may not fit within the ranges. The NRC 

8 chief examiner can evaluate a scenario that does not fit into these ranges to ensure that the 

9 level of difficulty is appropriate. If this allowance is used, the NRC chief examiner must

10 document his or her qualitative evaluation on Form 2.3-3, “Operating Test Review Worksheets.” 

11

12 The only exception to the above allowance is that each scenario must have at least two 

13 preidentified CTs. To assist in promoting exam consistency among applicants, NRC chief 

14 examiners and exam writers should carefully assign scenarios to applicant teams so that, 

15 whenever possible, the applicants are evaluated on a similar number of preidentified CTs.

16

17 5. License Level Criteria

18 Additionally, each applicant is required to respond to a specified minimum number, based on 

19 applicant license level, of the types of evolutions, failures, TS evaluations, and transients in 

20 accordance with Table 3.4-2, “Events and Evolutions for License Level,” and the instructions for 

21 using Table 3.4-2 below. This table provides the minimum number of events/evolutions by 

22 applicant, based on the license level for an applicant’s entire scenario portion of the operating 

23 test; it is not the number required per scenario. 

24

25 Use Form 3.4-1, “Events and Evolutions Checklist,” to track how each applicant will be tested on 

26 the minimum number of planned events and evolutions.

27

28 Table 3.4-2 Events and Evolutions for License Level

29

Event/Evolution Type

Applicant License Level

RO SRO-I SRO-U

Reactivity Manipulation 1 1 0

Normal Evolution 1 1 1

Instrument/Component (I/C) Failure 4 4 2

Major Transient 2 2 1

Manual Control of Automatic Function 1 1 0 

TS Evaluation 0 2 2

30
ES-3.4, Page 5 of 9

1 Use the following instructions for meeting the required number of events and evolutions in Table 

2 3.4-2:

3 a. For SRO-U applicants, the requirement for a minimum number of manual control of 

4 automatic functions does not apply because SRO-U applicants do not need to be 

5 evaluated in the ATC/BOP position.

6

7 b. For SRO-I applicants, at least two of the four I/C failures and one of the two major 

8 transients must be while they are in the ATC/BOP position. One reactivity manipulation 

9 and one manual control of an automatic function must be while the SRO-I is in the 

10 ATC/BOP position. 

11

12 c. For the minimum required number of reactivity manipulations, normal evolutions, and I/C

13 failures, only count events/evolutions that require the applicant to perform verifiable 

14 actions that provide insight to the applicant’s competence. This means that the applicant 

15 must perform some action, not just make a telephone call to an operator to take some 

16 action in the field. An applicant on the telephone directing an operator to take some 

17 action in the field while he or she is observing control room indications is not performing 

18 a verifiable action; instead, the applicant is directing. Although such an action may 

19 provide insight as to whether the applicant understands the system, it does not provide 

20 insight for the examiner to be able to determine whether the applicant can operate the 

21 equipment controls and control the system response.

22

23 To count the performance of a calculation as a verifiable action, the performance should 

24 relate directly to the competency that is being examined and not just credit the activity

25 because a calculation was performed. In these cases, it is only acceptable to give credit 

26 for performing a verifiable action if the applicant diagnoses a malfunction or event and 

27 then performs a calculation (i.e., shutdown-margin or leak-rate calculation) that can be 

28 graded by the examiner against acceptable grading criteria. Additionally, an applicant 

29 may calculate the leak rate for the primary coolant system to determine entry into TS or 

30 to determine a possible location for the leak (primary or secondary), in which case credit 

31 could be given. However, the applicant should only receive credit for performing the 

32 calculation if that calculation is used for some amount of diagnosis of the event by the 

33 applicant or for decisionmaking. Therefore, unless the applicant manipulates some 

34 piece of equipment/equipment controls or performs a calculation that includes diagnosis 

35 of the event/malfunction (including entry into TS), the applicant should not receive credit 

36 for the event/malfunction.

37

38 d. Each TS evaluation must be tied to a separate event, and each TS evaluation must 

39 involve entry into TS action statements. The offsite dose calculation manual cannot be 

40 used to meet the minimum TS evaluation requirement.

41

42 e. TS events can be standalone events for the SRO applicant and are not required to also 

43 count as an I/C failure event requiring verifiable action by the control board operators.

44

45 f. With the exception of manual control of automatic functions and the SRO TS

46 evaluations, count each evolution, failure, or transient only once per applicant. For 

47 example, a power change can be counted as a normal evolution or as a reactivity 

48 manipulation, and, similarly, a component failure that immediately results in a major 

49 transient counts as one or the other, but not both. 

50
ES-3.4, Page 6 of 9

1 g. Any normal evolution, component failure, or abnormal event (other than a reactor trip or 

2 other automatic power reduction) that requires the applicant to perform a controlled 

3 power or reactivity change will qualify as a reactivity manipulation. This includes events 

4 such as an emergency boration, a dropped rod recovery, a significant rod bank 

5 realignment, or a manual reactor power reduction in response to a secondary system 

6 upset. Such events may produce a timelier operator and plant response than a normal 

7 power change. 

8

9 Note: Count an event as either an I/C failure or a reactivity manipulation for a single 

10 applicant position (such as the ATC) but not both. Applicants must perform verifiable 

11 actions to receive credit for the I/C failure. For example, if a component fails and a

12 power reduction is the only verifiable action, count this as either an I/C failure or as a 

13 reactivity manipulation. If both the ATC and the BOP perform the power reduction, the 

14 event may be counted as an I/C failure or reactivity manipulation for either position

15 (i.e., the ATC gets the reactivity manipulation and the BOP gets the I/C failure—the ATC 

16 may NOT receive credit for both a reactivity manipulation and an I/C failure).

17

18 h. Replace reactivity manipulations and normal evolutions with an additional I/C failure on a 

19 one-for-one basis, if desired.

20

21 i. Develop scenarios so that the required TS evaluations and I/C failures are completed in 

22 events that occur before the major transient. Take care when using TS evaluations and 

23 I/C failures after the major transient, since they may require little applicant action and 

24 therefore provide little insight on applicant performance. For some plant types, it may be 

25 necessary to have I/C failures, used to meet the minimum number of I/C failures, after 

26 the major transient. This is acceptable if the applicants can be evaluated properly. 

27

28 j. Record the planned events and evolutions, using the event number (i.e., a “2” for 

29 Event “2”), for each applicant using Form 3.4-1.

30

31 k. If the facility prepared the operating test outline, a facility supervisor or manager shall 

32 review the proposed outline in accordance with ES-2.3 before submitting the outline to 

33 the NRC for approval.

34

35 6. Forwarding Outlines to the NRC Chief Examiner for Approval

36 When the proposed simulator operating test outline, Form 3.3-1, is complete, forward it, along 

37 with Form 3.4-1, to the NRC chief examiner so that it is received by the date agreed upon with 

38 the NRC regional office at the time the examination arrangements were confirmed; outlines are 

39 normally due approximately 150 days before the scheduled examination date. Refer to ES-2.1, 

40 “Preparing for Operator Licensing Initial Examinations,” for additional instructions on the review 

41 and submittal of the examination outlines.

42

43 7. Submitting the Presubmittal Sample to the NRC Chief Examiner

44 Facility licensees have the option to submit up to two JPMs and one simulator scenario to the 

45 NRC chief examiner for preliminary review and comment.

46
ES-3.4, Page 7 of 9

1 8. Detailing Operator Actions for Each Scenario

2 After the NRC approves the operating test outlines, use the instructions in ES-2.3 to prepare the 

3 final simulator test materials by revising Form 3.3-1 as requested by the NRC chief examiner 

4 and completing a detailed operator action form (Form 3.3-2 or equivalent) for each event. All 

5 required operator actions (e.g., opening, closing, and throttling valves; starting and stopping 

6 equipment; raising and lowering level, flow, and pressure; making decisions and giving 

7 directions; acknowledging or verifying key alarms and automatic actions) shall be documented, 

8 and CTs shall be identified. 

9

10 Update Form 3.4-1 as necessary.

11

12 9. Reviewing Simulator Scenarios

13 Review each scenario and the complete simulator operating test using the instructions in 

14 ES-2.3. Perform this review in conjunction with review of the associated JPMs to minimize 

15 duplication.

16

17 10. Validating Scenarios

18 Before administering the simulator operating test, validate each scenario on the simulator to 

19 ensure that it will run as intended. Typically, the facility licensee performs this validation during 

20 examination development, and the examination team validates it again during the preparatory 

21 site visit.

22

23 In some cases, such as for retake examinations and other exigent circumstances 

24 (i.e., examination security issue), the scenarios can be validated while the applicants are taking 

25 another portion of the examination (i.e., the written examination).

26

27 After validation, ensure that any necessary revisions are made to the scenario outline form 

28 (Form 3.3-1) and operator actions form (Form 3.3-2). 

29

30 11. Finalize Scenarios

31 After incorporating the necessary revisions, review and approval the simulator operating test 

32 using the instructions in ES-2.3.

33

34 C. Forms

35 Form 3.4-1 Events and Evolutions Checklist
ES-3.4, Page 8 of 9

Form 3.4-1 Events and Evolutions Checklist

E

A V

P E

P N T

L T O

I T

C T A

A Y S A B S A B S A B S A B L

N P R T O R T O R T O R T O

T E O C P O C P O C P O C P RO I U

RO

SRO-I NOR 1 1 1

I/C 4 4 2

SRO-U MAJ 2 2 1

Man. Ctrl 1 1 0

TS 0 2 2

RO

SRO-I NOR 1 1 1

I/C 4 4 2

SRO-U MAJ 2 2 1

Man. Ctrl 1 1 0

TS 0 2 2

RO

SRO-I NOR 1 1 1

I/C 4 4 2

SRO-U MAJ 2 2 1

Man. Ctrl 1 1 0

TS 0 2 2

RO

SRO-I NOR 1 1 1

I/C 4 4 2

SRO-U MAJ 2 2 1

Man. Ctrl 1 1 0

TS 0 2 2

RO

SRO-I NOR 1 1 1

I/C 4 4 2

SRO-U MAJ 2 2 1

Man. Ctrl 1 1 0

TS 0 2 2

RX 1 1 0

RX 1 1 0

1 1 0

RX

RX

1 1 0

RX 1 1 0

 U

 M*

POSITION POSITION POSITION POSITION

Facility: Date of Exam: Operating Test No.:

Scenarios

1 2 3 4

 M

 I

 N

 I

 M
ES-3.4, Page 9 of 9

Form 3.4-1 Instructions for the Events and Evolutions Checklist

Instructions for filling out Form 3.4-1, “Events and Evolutions Checklist”:

1. Mark the applicant license level for each simulator operating test number.

2. For the set of scenario columns, fill in the associated event number from Form 3.3-1, 

“Scenario Outline,” to show the specific event types being used for the applicant while in 

the assigned crew position for that scenario.

* Minimums are subject to the instructions in Section B, Step 4.B, “License Level Criteria.” 

KEY: RX = Reactivity Manipulation; NOR = Normal Evolution; I/C = Instrument/Component 

Failure; MAJ = Major Transient; Man. Ctrl = Manual Control of Automatic Function;

TS = Technical Specification Evaluation; RO = Reactor Operator; SRO-I or I = Instant Senior 

Reactor Operator; SRO-U or U = Upgrade Senior Reactor Operator; Senior Reactor Operator; 

ATC = At the Controls; and BOP = Balance of Plant

ES-3.5, Page 1 of 13 

1 3.5 ADMINISTERING OPERATING TESTS 

2 This examination standard contains the instructions for NRC examiners on how to administer 

3 operating tests to initial license applicants at power reactor facilities. It includes instructions and 

4 guidelines for administering both the walkthrough and simulator portions of the operating test. 

5 

6 Regardless of whether the facility licensee or the NRC prepared the operating test, the NRC

7 administers every operating test for initial licensing in accordance with the instructions contained 

8 in this section.

9 

10 The NRC regional office and the facility licensee coordinate the schedule for administering the 

11 operating test, with a goal of maximizing efficiency while maintaining examination security. The 

12 facility licensee provides administrative and logistics support (e.g., personnel to operate the 

13 simulation facility, surrogate operators, monitors, copies of the approved operating test 

14 materials as arranged with the NRC chief examiner) to facilitate the administration of the 

15 operating tests.

16

17 All participants in the testing process must also be mindful of their responsibilities for

18 examination security and integrity pursuant to 10 CFR 55.49, “Integrity of examinations and 

19 tests.” 

20

21 A. General Instructions for the NRC Chief Examiner

22 1. Operating tests will normally be administered on regular workdays. If weekend, holiday, 

23 or shift work is necessary to administer the operating tests, coordinate arrangements 

24 with the assigned examiners and the facility licensee and obtain permission from the 

25 NRC regional Branch Chief. 

26

27 2. To maintain uniform conditions for licensing operators, administer the operating test and 

28 written examination within 30 days of each another. This may not be possible under 

29 certain circumstances, such as a weather or health emergency. Concurrence from the 

30 NRR operator licensing program office more than 30 days will elapse between the 

31 completion of one and the start of the other. 

32

33 3. Assign a single NRC examiner (the examiner of record) to individually evaluate an 

34 applicant during the scenario(s) that an applicant takes as part of his or her operating 

35 test. The following exceptions to this assignment are permitted:

36

37 a. For a three-person crew consisting only of SRO-U applicants (who do not need

38 to be evaluated on the control boards), the region may assign two examiners to

39 evaluate the crew. Note: SRO-U applicants will still be held accountable for any

40 errors that occur because of their action(s) or inaction(s), and they will be graded

41 on their ability to “operate the controls.”

42

43 b. For the purpose of examination efficiency or minimizing surrogate use, an

44 examiner other than the examiner of record can administer a scenario to an

45 applicant, provided that the examiner of record is present during the scenario

46 (e.g., examining one of the other applicants) and the scenario is in addition to the

47 minimum required for that applicant. This exception requires approval from the

48 NRR operator licensing program office.
ES-3.5, Page 2 of 13

1 4. Assign the same examiner to an applicant for each JPM, or divide the JPMs among 

2 different examiners for each applicant.

3

4 5. Arrange for any NRC examiners who are not familiar with the facility to obtain a tour of 

5 the facility before they administer any operating tests. Such tours shall not be conducted 

6 or observed by any of the applicants. In addition, the tours should, at a minimum, cover 

7 areas of the plant that will be used during the examination process, such as the control 

8 room, the simulation facility, and planned walkthrough locations.

9

10 6. Ensure that any JPMs that the facility licensee or the NRC did not previously validate 

11 during the preparatory site visit are validated before administering them.

12

13 7. Review the facility licensee’s list of differences between the simulator and the reference 

14 plant and the list of uncorrected simulator performance deficiencies and confirm that any 

15 uncorrected simulator performance deficiencies will not interfere with the conduct of the 

16 planned operating tests.

17

18 8. Confirm with the facility licensee that the simulator instructor’s station, programmers’ 

19 tools, and external interconnections do not compromise operating test security while 

20 conducting examinations. The primary objective is to ensure that the examination

21 material cannot be read or recorded at other unsecured consoles and is either physically 

22 secured or electronically protected when not in use by individuals listed on the security 

23 agreement. Refer to ES-1.3, “Examination Security,” for simulator security 

24 considerations.

25

26 9. In the interest of examination security, limiting the number of observers, and maintaining 

27 uniform conditions for licensing operators, do not permit the facility licensee to use video 

28 and audio recording during the administration of the operating test.

29

30 10. Before beginning the operating test, brief the applicant(s) using Parts A, C, D, and E of 

31 ES-1.2, “Guidelines for Taking NRC Examinations.” To save time, an examiner may 

32 brief the applicants as a group. This briefing may be conducted during the NRC 

33 preparatory site visit.

34

35 11. Administer the operating test to each applicant identified on Form 2.2-1, “List of 

36 Applicants,” as indicated on the form.

37

38 12. Do not repeat the same simulator scenarios and JPMs on subsequent days. One 

39 approach for efficiency is to divide the operating test into segments and administer the 

40 same operating test segment to all relevant applicants on the same day. Ensure 

41 examination security is maintained.

42

43 13. Verify that each applicant performed the required number of JPMs and simulator 

44 scenario events (and types of events) before leaving the facility licensee site. Use the 

45 spare scenario when necessary to ensure that each applicant performs the required 

46 simulator scenario events and evolutions. It is acceptable to run an abbreviated version 

47 of the spare scenario if necessary to ensure that all applicants are evaluated on the 

48 required number of events. For example, run the spare scenario up to the major event 

49 to ensure that applicants are evaluated on the minimum number of required I/C failures

50 or TS evaluations. 

51
ES-3.5, Page 3 of 13

1 14. The NRC chief examiner will discuss with the facility licensee how long it plans to retain 

2 the operating test materials to support the NRC licensing action on all the applications 

3 and adjudicatory actions on any hearing demands.

4

5 15. Conduct an exit briefing with the facility licensee after the operating tests are complete. 

6 Address any generic weaknesses noted during the operating tests, as well as any other 

7 significant issues (e.g., problems with the reference materials, the simulation facility, or 

8 the plant) that the examination report might address. The individual operating test 

9 results are predecisional until approved by NRC management in accordance with 

10 ES-5.1, “Issuing Operator Licenses and Postexamination Activities.” Do not share 

11 results with the facility licensee during the exit briefing.

12

13 16. Retain all examination material generated during administration of the operating test to 

14 support the NRC licensing action on all the applications and adjudicatory actions on any 

15 hearing demands. When those actions are complete, consult Management Directive 

16 3.53, “NRC Records and Document Management Program,” to determine the record 

17 status and disposition the operating test examination material accordingly. 

18

19 B. General Instructions for the NRC Examination Team

20 1. Meet as a group with the NRC chief examiner to review the examination materials after 

21 the responsible supervisor has approved them for administration. The meeting should 

22 focus on those test items that might require extensive cuing by the examiner and those 

23 that are unique to the facility and require a response different than what the examiner 

24 might expect based on experience.

25

26 2. Administer the operating test in accordance with the planned and approved outlines and 

27 simulator scenario guides (operator actions). Substitute or replace planned material only 

28 if an item is determined to be invalid or impossible to perform or simulate because of 

29 unanticipated access restrictions, equipment failures, or examination security concerns. 

30 Operating test changes require NRC chief examiner and regional Branch Chief approval. 

31 If the regional Branch Chief is not available, the NRC chief examiner can authorize a 

32 change and later inform the regional Branch Chief that a change occurred. 

33

34 3. During administration, take detailed notes to facilitate a thorough documentation of an 

35 applicant’s performance deficiencies. The examiner must be able to cross-reference 

36 each comment and performance deficiency to a specific JPM, simulator event, or 

37 followup question.

38

39 4. Take reasonable measures to ensure that any notes documenting the applicant’s 

40 performance on the operating test are not accessible to the facility licensee staff. These 

41 notes are predecisional; do not leave examination notes unattended or unsecured in the 

42 simulator or examination room to which the facility licensee staff has access.

43

44 C. Limits on Operating Test Observers

45 Limit the number of people present during an operating test to maintain the integrity of the test 

46 and to minimize distractions: 

47

48 1. Except for the simulation facility operators, no other member of the facility licensee’s 

49 staff are allowed to observe an operating test without the NRC chief examiner’s 
ES-3.5, Page 4 of 13

1 permission. Facility management and other personnel deemed necessary by the facility 

2 licensee will generally be allowed access to the examination (under security 

3 agreements, as appropriate), provided that the simulation facility can accommodate 

4 them and there is no impact on the applicants.

5

6 2. Under no circumstances will another applicant be allowed to observe an operating test. 

7 Operating tests are not to be used as training for future applicants.

8

9 3. Other examiners may observe an operating test as part of their training or to audit the 

10 performance of the examiner(s) administering the operating test.

11

12 4. The NRC chief examiner may permit other NRC employees, such as resident inspectors, 

13 regional personnel, researchers, or NRC supervisors, to observe an operating test.

14

15 5. Non-NRC employees (e.g., representatives from the Institute of Nuclear Power 

16 Operations) may observe an operating test with prior approval from the NRR operator 

17 licensing program office. The NRC chief examiner will control the observer’s activities in 

18 accordance with guidance provided by NRR. 

19

20 6. When possible, observers should monitor the examination from the simulator control 

21 booth or a separate room using a secured video feed (while maintaining examination 

22 security).

23

24 7. An applicant may request that the administration of his or her operating test without 

25 extraneous observers.

26

27 D. Instructions for the Use of Surrogate Operators during Administration

28 Surrogate operators are used when necessary to complete an applicant crew for the simulator 

29 test in accordance with the instructions in Section B.2 of ES 3.4, “Developing Scenarios.” 

30

31 Prior to administering a scenario with a surrogate operator, the NRC chief examiner ensures

32 that the surrogate operator(s) is briefed on the content of the scenario(s) and his or her

33 expected actions in response to every event. The surrogate operator’s activities should not be 

34 restricted to such an extent that the applicants being evaluated are required to assume 

35 responsibilities beyond the scope of their respective positions.

36

37 Surrogate operators should be licensed operators or facility licensee staff who are certified to 

38 have the knowledge and abilities required to assume the full responsibilities of the roles that 

39 they take in the operating test. Examiners are not assigned to evaluate surrogate operators.

40

41 Surrogate operators shall not take a proactive role in assisting or coaching the applicants,

42 because such interventions would hinder the examiners’ ability to evaluate the applicants’

43 competence. The NRC will run additional scenarios, if necessary, to ensure that every applicant 

44 can demonstrate his or her competence for the NRC to make a licensing decision.

45

46 E. Instructions for the Use of Followup Questions

47 Examiners use followup questions during operating test administration to assist in their 

48 assessment of the applicant’s knowledge, skills, and abilities and to clarify their observations. 
ES-3.5, Page 5 of 13

1 Followup questions must be performance based, meaning that they must be related to observed 

2 performance during the operating test. 

3

4 Examiners should use followup questions to do the following:

5 1. Clarify or confirm an applicant’s understanding of the administrative topic or system as it 

6 relates to a JPM task when the applicant fails to accomplish the task standard for the 

7 JPM or demonstrates a lack of understanding with regard to the administrative topic or 

8 system, equipment, and procedures, such as having difficulty locating information, 

9 control board indications, or controls.

10

11 2. Clarify actions taken or not taken during simulator scenarios.

12

13 3. Clarify statements (written or verbal) made by an applicant during simulator scenarios 

14 and JPMs.

15

16 4. Determine whether a performance deficiency exists when an applicant takes an 

17 unexpected action, does not take an expected action, or says something during the 

18 simulator scenario that reveals a potential lack of understanding.

19

20 5. Determine the appropriate competency or the primary cause of a performance deficiency 

21 in support of rating factor selection.

22

23 6. Evaluate the SRO applicant (while in the SRO position) on applicable TS actions in 

24 accordance with the simulator scenario guide. Do this by first asking the applicant to 

25 explain the TS implications for each event of the scenario without providing any cues as 

26 to which events should be considered. Then, use more detailed questions if a 

27 performance deficiency is identified to determine which rating factor to cite during 

28 grading activities. 

29

30 Examiners administer followup questions during a brief question-and-answer period after the 

31 completion of each simulator scenario or JPM and before starting the next JPM or the applicant 

32 is released to a facility licensee staff member for sequestration. Followup questions can be 

33 asked, with examiner discretion, during a JPM. 

34

35 Examiners must document the question asked and the applicant’s response in their examiner 

36 notes for each followup question for use during grading activities. 

37

38 Performance-based followup questions during any part of the operating test may include a 

39 combination of open- and closed-reference items. Open-reference items require applicants to 

40 apply their knowledge of the plant to postulated normal, abnormal, and emergency situations. 

41 Closed-reference items may be used to evaluate the immediate actions of emergency and other 

42 procedures, certain automatic actions, operating characteristics, interlocks, set points, and 

43 routine administrative activities, as appropriate to the facility.

44

45 Applicants may use reference information such as diagrams and procedures for answering 

46 open-reference followup questions.

47

48 The examiner may encourage the applicants to sketch diagrams, flowpaths, or other illustrations 

49 to aid in answering any followup questions that might be necessary. The examiner retains this 

50 examination material because it provides additional documentation to support a pass or fail 
ES-3.5, Page 6 of 13

1 decision. After completion of the licensing action and adjudicatory actions on any hearing 

2 demands, the examiner should consult Management Directive 3.53 to determine the record 

3 status and disposition the examination material accordingly. 

4

5 F. Simulator Performance during the Operating Test

6 If the simulation facility becomes inoperative or presents hardware, software, or modeling issues

7 that cannot be resolved in a timely manner, the NRC chief examiner should discuss the situation 

8 with the facility licensee and the responsible regional supervisor so that management can make 

9 a decision about the conduct of the operating test. The simulator examinations may need to be 

10 rescheduled for a later date.

11

12 The simulator should be considered inoperative under any of the following conditions:

13

14 • The simulator exhibits a mass/energy imbalance, erratic logic, or inexplicable panel 

15 indications during evolution execution.

16

17 • The simulator exhibits unplanned and unexplained events or malfunctions that cause the 

18 applicants to divert from the expected responses and success path of the planned scenario.

19

20 • The simulator automatically goes to the “freeze” state during a scenario, or the instructor’s 

21 station receives a “beyond simulated limits” alarm.

22

23 • The simulator operator informs the NRC examination team that a software module has 

24 halted or “kicked out.”

25

26 The occurrence of any of these abnormal simulator operating conditions during an examination 

27 is a reason to stop the scenario. Additionally, the NRC chief examiner must determine whether

28 the simulator condition impacted applicant performance or an examiner’s ability to assess

29 applicant performance. 

30

31 When the simulator has been restored to full operation, the NRC chief examiner will determine 

32 whether to replace the scenario, run the scenario from a specific event, or restart the scenario 

33 from the beginning.

34

35 When resuming or restarting the scenario, the “backtrack” function should not be used; 

36 “backtracking” may introduce new computer coding errors and affect data recording displays. 

37 Additionally, the simulator must be in a relatively stable plant condition, at a definitive procedural 

38 step, before turning over to the applicants. The applicants should have a reasonable amount of 

39 time to refamiliarize themselves with plant status.

40

41 Any events or malfunctions that did not function as expected or were not useful in evaluating the 

42 applicants (e.g., a surveillance test that required a long time to perform) should be noted on the 

43 master copy of the scenarios to aid in future scenario preparation.

44

45 G. Specific Instructions for Administering Simulator Scenarios

46 Simulator scenarios should be administered according to the following instructions: 

47

48 1. The only senior operator position that can be filled by an SRO applicant during the 

49 simulator operating test is that of the senior licensed operator immediately responsible 
ES-3.5, Page 7 of 13

1 for control of the unit. This position is typically referred to as the shift supervisor, control 

2 room supervisor, or unit supervisor.

3

4 2. One or more facility licensee staff members normally assume the role of plant personnel 

5 that the applicants direct or notify about plant operations (e.g., shift technical advisor 

6 (STA), shift manager, work control supervisor, assist/third control room operator). In 

7 rare circumstances, and with NRC chief examiner approval, a surrogate crew member 

8 may fill such a position (only if the facility’s license requires that position). These role 

9 players do not need to have a license. SRO applicants still need to be able to perform 

10 the tasks normally carried out by these role players, but they are not required to 

11 demonstrate those duties as part of the simulator scenario portion of the operating test. 

12 SRO applicants are tested and evaluated on these types of SRO-level tasks during the 

13 administrative topics part of the walkthrough portion of the operating test. This includes 

14 the principal duties of the shift manager position (i.e., assuming the role of the 

15 emergency director, performing emergency classifications, and making protective action 

16 recommendations). (Reference: SECY-98-266, “Final Rule—Requirements for Initial 

17 Operator Licensing Examinations,” dated November 13, 1998, Attachment 1, Section II.) 

18

19 3. Before the operating test begins, the NRC chief examiner will brief or participate in the 

20 briefing of all nonapplicant operating test participants on their responsibilities, reporting 

21 requirements, duties, and level of participation. If the STA position is implemented 

22 during a simulator scenario, the briefing must include the STA’s expected actions in 

23 response to every event. 

24

25 4. Applicant crew interactions with the STA must be conducted in accordance with the 

26 facility licensee’s normal control room practice (e.g., an STA shall not be stationed in the 

27 simulator if he or she is on call at the site). The STA cannot proactively assist or coach 

28 the applicants; this behavior hinders the NRC examination team’s ability to evaluate the 

29 applicants’ competence. 

30

31 5. The NRC expects applicants to perform “peer checks” in accordance with the facility 

32 licensee’s operations and training procedures and practices; however, additional 

33 personnel may not be stationed or called upon for this purpose. If an applicant begins to 

34 make an error that is corrected by a peer checker, the applicant will still be assessed a 

35 performance deficiency as if he or she had taken the action without correction.

36

37 6. Just before beginning the simulator scenarios, the examiners review the scenario events 

38 with the facility licensee simulator operator and ensure that he or she has the most 

39 current version of the scenario guide and is familiar with the sequence of events. The 

40 purpose of this review is to refamiliarize the NRC examination team with the sequence of 

41 events and ensure that the scenario will proceed as planned. Examiners should also 

42 discuss the required procedures, TS, and special circumstances related to the 

43 scenarios. The NRC chief examiner will ensure that the facility licensee simulator 

44 operator playing the role of other plant personnel is aware of the timescale for 

45 responding to the applicants’ requests for information. For example, the NRC chief 

46 examiner may specify the use of time compression for field operations such as valve 

47 lineups to prevent long delays in simulated operations, and the use of real-time delays 

48 for giving applicants simulated reports from maintenance or chemistry personnel, to 

49 present applicants with the same analysis problems that they will face as operators.

50
ES-3.5, Page 8 of 13

1 7. The facility licensee simulator operator can only provide information that the applicants

2 specifically request and that does not compromise the integrity of the examination. 

3 When the simulator operator is briefing the applicants or communicating with them on 

4 the telephone, the examiners should monitor the conversations to ensure that the 

5 information provided is appropriate and does not cue the applicants.

6

7 8. The NRC chief examiner ensures that the facility licensee simulator operator is ready to 

8 record the plant parameters at specified intervals as identified by the NRC examination 

9 team during the examination preparation week. Parameter readings should be collected 

10 at meaningful intervals, depending on the parameter, the nature of the event, and the 

11 capability of the simulation facility. If necessary, based on applicant performance, the 

12 NRC chief examiner will retain the recordings as backup documentation to augment the 

13 notes taken by the examiners during the simulator test until the NRC takes its licensing 

14 action on all the applications and adjudicatory actions on any hearing demands are 

15 complete. Then, the NRC chief examiner should consult Management Directive 3.53 to 

16 determine the record status and disposition the simulator recordings accordingly.

17

18 9. The examiner in charge of the scenario should have a suitable communication system

19 with the facility licensee simulator operator so that the examiner can prompt him or her

20 to insert the malfunctions without cuing the applicants. Malfunctions may be planned for 

21 a predetermined time or power level so that the examiners and the facility licensee 

22 simulator operator are aware of the event that is occurring or pending.

23

24 10. If necessary, the examiners may use time compression to speed up the response of key 

25 parameters so that the scenario can proceed to the next event within a reasonable time. 

26 Time compression is acceptable if used judiciously, and the operators receive enough

27 time to perform the tasks that they would typically perform in real time. The examiners 

28 should inform the applicants during the scenario if time compression is used. 

29

30 11. Before beginning each scenario, the facility licensee simulator operator will advance any 

31 control room strip chart recorders that may prove useful in recreating the sequence of 

32 events. Additionally, the facility simulator operator will ensure that digital recorders are 

33 appropriately reset so that the applicants cannot observe the history from previous 

34 simulator runs. 

35

36 12. Before the simulator test begins, the facility licensee staff or NRC examiner will provide a 

37 shift turnover briefing. The briefing will cover present plant conditions, power history, 

38 equipment out of service, abnormal conditions, surveillances due, and instructions for 

39 the shift, and the applicants will have time to familiarize themselves with the plant status. 

40

41 13. After the shift turnover briefing and before beginning the scenario, the facility licensee 

42 staff or NRC examiner will advise the applicants to conduct any standard shift briefings 

43 (e.g., reactivity management) and any necessary normal evolution briefings before the 

44 scenario commences. 

45

46 14. Examiners should use the expected actions and behaviors listed on Form 3.3-2 as a 

47 guide while administering the simulator tests. If an applicant performs as expected, the 

48 examiner may simply note on the form the time when the expected actions occurred. 

49 However, if an applicant does not perform as expected, the examiner should note the 

50 applicant’s actions (or lack thereof) next to or below the expected action and follow up 

51 with appropriate questions after the simulator scenario is completed.
ES-3.5, Page 9 of 13

1

2 Note: Each examiner must determine the best way to document the applicant’s actions

3 and develop a documentation technique that facilitates an assessment of the applicant’s 

4 competencies and substantiates a subsequent licensing decision. Some examiners 

5 record a minute-by-minute account of all key plant events and the applicant’s actions as 

6 they occur; other examiners record only the applicant’s significant actions. Precise

7 examiner notes are an essential aspect of examination administration because these 

8 notes provide an account of observed performance deficiencies.

9

10 15. Examiners should limit discussions with the applicants during the scenarios both to 

11 maintain realism and to avoid distracting the applicants from operating the plant. 

12 Questions during the scenarios should be limited to those necessary to assess the 

13 applicants’ understanding of plant conditions and the required operator actions. 

14 Whenever possible, defer questioning the applicant until a time when the applicant is not 

15 operating or closely monitoring the plant (preferably after the simulator has been placed 

16 in “freeze”).

17

18 16. Immediately after completing the scenario, the examiners assemble to do the following:

19

20 a. Verify that each examiner observed that his or her applicant perform the required 

21 number of transients and events to allow adequate evaluation of all required 

22 competencies. 

23

24 b. Determine the status of the CTs during the scenario and identify any critical 

25 performance deficiencies, including whether an applicant’s action(s) or 

26 inaction(s) resulted in a postscenario CT (refer to the CT methodology in

27 Section C of ES-3.3).

28

29 c. Identify any significant performance deficiencies made during the scenario. 

30 Significant performance deficiencies include any of the following:

31

32  performance deficiencies that either cause an automatic reactor 

33 protection system (RPS)/engineered safety feature (ESF) actuation or 

34 that warrant or involve a manual RPS/ESF actuation that should have 

35 otherwise been avoidable had the applicant responded to the event as 

36 expected

37

38 Note: Subsequent RPS/ESF actuations that do not alter equipment 

39 alignments are not treated as additional significant performance 

40 deficiencies.

41

42  performance deficiencies that would result in an avoidable emergency 

43 action level entry or escalation in accordance with the facility’s emergency 

44 plan emergency action level matrix

45

46  performance deficiencies resulting in unplanned power changes of more 

47 than 10-percent rated thermal power

48

49 d. An applicant who is corrected by another crew member will still be held 

50 accountable for what would have transpired if he or she had taken the action 

51 without correction. Analyze those corrected actions to determine whether they 
ES-3.5, Page 10 of 13

1 would have resulted in an event that reaches the threshold for classification as a 

2 significant performance deficiency or postscenario CT.

3

4 e. Determine if the as-run scenario invalidated any preidentified CTs. If necessary, 

5 run an additional scenario to ensure that all required events and evolutions are 

6 completed. 

7

8 f. Cross-check examiner notes and observations for performance deficiencies 

9 “shared” by more than one applicant. Both evaluating examiners must note the 

10 deficiency and formally document it during grading.

11

12 g. Determine what followup questions need to be asked.

13

14 17. If a simulator scenario includes emergency plan event classification, because the 

15 simulator operating tests for the initial licensing examination are conducted with only one 

16 applicant in the SRO position, the NRC does not require the SRO applicant to complete 

17 an emergency classification within the normal event classification period of time. The 

18 scenario does not need to include event classification. 

19

20 18. If an applicant demonstrated potential performance deficiencies during the simulator 

21 scenario, before permitting cleanup and reset of the simulator, the examiners should ask 

22 the facility licensee simulator operator to provide copies of the logs, charts, data, or other 

23 materials that may be required to evaluate and document the applicant’s performance. 

24

25 H. Specific Instructions for Administering Job Performance Measures

26 JPMs should be administered according to the following instructions: 

27

28 1. JPMs that are conducted in the control room cannot interfere with normal shift 

29 operations. The NRC chief examiner should ask the facility licensee training manager to 

30 notify the shift supervisor when the NRC will conduct examination activities in the control 

31 room. If the number of persons or the noise level in the control room is excessive, the 

32 examiner should, if possible, move to a quieter location, modify the sequence of the 

33 JPMs, and return when the level of activity in the control room has abated, or ask the 

34 facility licensee training manager to address the issue.

35

36 2. JPM administration should be coordinated with the facility licensee in order to optimize

37 the use of the simulator. The NRC chief examiner may allow simultaneous 

38 administration of different JPMs (in a staggered or parallel fashion) to multiple applicants

39 in the simulator; the NRC examiners must ensure that applicant interference is 

40 minimized, and that test integrity is not compromised. This can be accomplished 

41 successfully through various techniques. A simulator control room of average size can

42 often support two JPMs conducted in parallel. If one JPM contains significant 

43 annunciator response or an alternate path, simultaneous administration may occur by 

44 staggering the start of the second JPM to ensure that every applicant who is taking the 

45 second JPM has already taken the JPM with an alternate path or significant 

46 annunciators. A back panel JPM and a main board JPM are compatible for parallel 

47 administration. If available, equipment such as room dividers are useful to separate the 

48 JPMs. 

49
ES-3.5, Page 11 of 13

1 3. JPMs may be administered in “station keeping” mode, in which the examiners remain in 

2 position at designated operating stations and the applicants, under escort, rotate through 

3 the various stations. Such arrangements would have to be agreed to by, and 

4 coordinated with, the facility licensee; moreover, examination security must be 

5 maintained.

6

7 4. If a discrepancy exists between the simulator setup and the conditions specified in a 

8 JPM, the examiner should stop the JPM and correct the situation, as necessary. If the 

9 task can be completed with different values (e.g., wind direction when determining a 

10 protective action recommendation during an emergency), the examiner should document 

11 the differences in the associated as-administered JPM and coordinate with the facility 

12 licensee contact and the NRC chief examiner to validate the applicant’s response under 

13 the actual conditions.

14

15 5. The examiner should encourage the applicant to use such materials as facility licensee 

16 forms, schedules, and procedures if they are relevant to the tasks to be performed.

17

18 6. If the applicant requests a “peer check,” the examiner can acknowledge the applicant’s 

19 request and proposed actions regardless of their accuracy or correctness or inform the 

20 applicant that a peer checker is not available; NRC examiners do not provide peer 

21 checks during JPMs. The applicant will be held accountable for errors made during the 

22 JPMs. Similarly, the examiner will not permit an applicant to obtain assistance from a 

23 “procedure reader” when performing JPMs.

24

25 7. For in-plant systems JPMs, the evaluator should not credit the applicant pointing to a 

26 valve or switch and stating that “the valve is closed” or “the breaker is closed” as 

27 completing the step successfully. The applicant must describe how he or she would 

28 perform a verifiable action, such as, “I am turning the hand-wheel in the clockwise 

29 direction and observing the stem move inward until I feel resistance.”

30

31 8. If the applicant exceeds twice the validated time estimate for a JPM because he or she 

32 has selected an incorrect procedure or operated the wrong equipment (despite being 

33 presented with sufficient plant feedback to correct the error), the examiner should stop 

34 the JPM, document the circumstances, and proceed with the next JPM. However, if the

35 applicant is on the correct path but has simply stopped making progress toward 

36 completing a nontime-critical JPM, the examiner should ask the applicant to describe the 

37 work to be done and how long it should take to complete the JPM. If the applicant does 

38 not then make timely progress toward completing the described actions, the examiner 

39 should inform the applicant that the allowed time for the JPM has elapsed and the 

40 applicant will be evaluated on the work completed. The examiner should then proceed 

41 with the next JPM.

42

43 9. If the applicant volunteers additional or corrected information after completing a task but 

44 before starting the next JPM or release to a facility licensee staff member for 

45 sequestration, the examiner should offer the applicant the opportunity to take whatever 

46 actions would be required in a similar situation in the plant. The examiner should record 

47 any revisions to previously performed tasks or answers for consideration when grading 

48 the operating test.

49

50 10. If an applicant demonstrated potential performance deficiencies during a control room 

51 system JPM, the examiner should ask the facility licensee simulator operator to provide
ES-3.5, Page 12 of 13

1 copies of the logs, charts, data, or other materials that may be required after leaving the 

2 facility to evaluate and document the applicant’s performance. 

3

4 I. Walkthrough Evaluation Techniques

5 NRC examiners should be familiar with and practice the walkthrough evaluation techniques 

6 described in this section. This guidance is intended to assist NRC examiners in administering 

7 JPMs by illustrating good and bad examples of walkthrough examination techniques.

8

9 1. Providing Cues

10 Cuing refers to the information that an examiner provides to an applicant when conducting a 

11 JPM. When conducting JPMs on the simulator, the simulator provides most of the required 

12 cues. However, when conducting JPMs outside of the simulator, the examiner must provide 

13 realistic and timely information to the applicant.

14

15 a. Verbal and Visual Cues

16

17 Verbal cues are often required to provide relevant system information, such as valve position, 

18 meter deflection, or indicating light status. Visual cues can also be used, such as employing a 

19 pointing device to indicate a gauge value. The examiner should take care to provide the 

20 applicant with the indications that should be readily observed (e.g., “the red light just 

21 illuminated,” “the valve position indicator does not move”). An examiner can give too much 

22 information or inappropriate information (e.g., providing indications that are not visible or audible 

23 to the applicant) that could invalidate the JPM. The examiner should keep in mind what the 

24 applicant would see and hear while performing the JPM and provide consistent cues.

25

26 b. Nonverbal Cues

27

28 Examiners should take care to avoid body language that would suggest to an applicant that he 

29 or she provided an incorrect response or performed the wrong procedural step. Voice 

30 inflections indicating something has been performed incorrectly or a change in how cues are 

31 given (e.g., talking more methodically or rapidly) are examples of nonverbal communication that 

32 should be avoided.

33

34 Thorough preparation and familiarity with the JPM are vital to providing proper cuing. 

35 Knowledge of the indications that will be available and how they will respond to the applicant’s 

36 actions allow an examiner to give accurate and timely cues when an applicant is incorrectly 

37 performing the task.

38

39 2. Evaluation Skills

40 When evaluating an applicant, an examiner must have the ability to differentiate between what 

41 he or she knows or believes to be true about an applicant’s ability and how the applicant

42 performs on the JPM. As previously discussed, an examiner must be familiar with the JPM to 

43 be able to accurately evaluate performance. The examiner may not see errors made by the 

44 applicant performing the JPM, or may not ask pertinent questions, if the examiner has not 

45 prepared for the examination. An examiner must remain attentive to the applicant’s actions at 

46 all times. This will ensure that the examiner provides timely cues and detects errors in 

47 performance.

48
ES-3.5, Page 13 of 13

1 3. Examination Administration

2 When conducting JPMs in the simulator, the examiner should not manipulate any controls or 

3 silence/acknowledge any alarms. The examiner must take a “hands-off” approach to maintain 

4 the proper testing environment. The examiner should take care to shield any notes or grading 

5 from the applicant to prevent giving an indication of performance, which may either provide a 

6 false sense of security or increase stress levels. If an applicant’s actions are not clear, the 

7 examiner should be prepared to ask appropriate followup or clarifying questions. Documenting 

8 these questions and the subsequent answers is important as they may have a bearing on an 

9 applicant’s overall grade.

ES-3.6, Page 1 of 28

1 3.6 GRADING AND DOCUMENTING OPERATING TESTS FOR 

2 OPERATOR LICENSING INITIAL EXAMINATIONS

3 This examination standard applies to the operating test for initial licensing examinations and 

4 contains instructions for grading and documenting all parts of the operating test, collating the 

5 data to arrive at a pass or fail recommendation, and reviewing the documentation to ensure

6 completeness.

7

8 The procedures contained in this examination standard require the NRC examiners to evaluate 

9 each applicant’s performance on the operating test, document performance deficiencies (PDs),

10 and determine whether the applicant’s level of knowledge and understanding meets the 

11 minimum requirements to safely operate the facility for which the license is sought. 

12

13 A. Responsibilities

14 1. Facility Licensee

15 The facility licensee’s responsibilities include providing the NRC examiners with additional 

16 reference materials and information they may require to evaluate an applicant’s performance on 

17 the operating tests. Such materials might include simulator strip chart recordings, captured data 

18 that document plant status during the simulator scenarios, and procedures that document the 

19 expected operator actions.

20

21 Facility licensees must submit any formal applicant or facility comments (or both) about the 

22 operating test to the NRC chief examiner as part of the postexamination package (see ES-4.4, 

23 “Grading and Documenting the Written Examinations”) for consideration during grading 

24 activities.

25

26 2. NRC Examiner of Record

27 The NRC examiner of record has the following responsibilities for the operating tests for 

28 operator licensing initial examinations:

29

30 • After administering the operating test, review and evaluate each applicant’s performance

31 and finalize the operating test documentation in accordance with the instructions in 

32 Sections B and C of this examination standard.

33

34 • Thoroughly explain and document the basis for any operating test failure.

35

36 3. NRC Chief Examiner

37 The NRC Chief Examiner assigned to the initial licensing examination, or a 

38 management-assigned designee, has the following responsibilities:

39

40 • Review any operating test comments received as part of the postexamination package 

41 and determine whether they need to be incorporated in the grading of the operating test.

42

43 • Work with the other examiners on the team to resolve any technical questions that might 

44 arise during the grading process.

45
ES-3.6, Page 2 of 27

1 • Communicate the need for any additional reference material to the facility licensee 

2 contact.

3

4 • Review the grading of each applicant’s operating test to verify that the examiner’s 

5 comments appropriately support his or her recommendation.

6

7 • Ensure that PDs are graded and documented consistently for each applicant who took

8 the simulator operating test. 

9

10 • Ensure that grading is accurate for PDs that involve more than one applicant in the same 

11 operating crew.

12

13 • Review the examiner of record’s recommendation for the operating test in the examiner 

14 recommendations section of each applicant’s Form 5.1-2, “Individual Examination 

15 Report,” and document a final recommendation by signing and dating next to the 

16 examiner signature. Confer with the examiner before opposing the examiner’s pass/fail 

17 recommendation. Such disagreements are not common and usually happen when the

18 justification for an unsatisfactory grade is not adequately documented. Provide specific 

19 reasons for opposing the examiner of record’s recommendation on the applicant’s 

20 Form 5.1-2.

21

22 Note: If the NRC chief examiner is the examiner of record for an applicant, a 

23 management-assigned designee must perform this step for that applicant in lieu of the

24 NRC chief examiner. The same person cannot sign both blocks (recommendation and 

25 final recommendation) on an applicant’s Form 5.1-2.

26

27 B. Grading and Documenting Instructions

28 Use these instructions to grade and document operating test performance for each applicant.

29

30 1. Identify Performance Deficiencies

31 For each applicant, review the JPMs and simulator scenarios that were administered and the 

32 responses to any followup questions that were asked. Evaluate all notes and documentation 

33 generated during operating test administration to determine the areas for which the applicant 

34 had PDs. 

35

36 Material (such as figures, drawings, flowcharts, or forms) that the applicant generated or used 

37 during the operating test can be used as an aid to document the applicant’s performance. Mark 

38 and reference any material used to support the documentation of a PD and attach it to the 

39 applicant’s examination results package.

40

41 Evaluate the validity and technical accuracy of any followup questions asked during the 

42 operating test, as well as any unexpected events or actions that occurred during the simulator 

43 operating test. If necessary, work through the NRC chief examiner to obtain any additional 

44 reference material that might be required to resolve any technical questions.

45

46 Evaluate every item that may constitute a PD and determine whether it meets the definition of a 

47 performance deficiency which is defined in ES-8, “Glossary,” as follows: 

48
ES-3.6, Page 3 of 27

1 An observed action or inaction (including operational tasks, procedure/process 

2 implementation, communications, and administrative functions), or a statement of 

3 understanding or intent, which demonstrates a lack of ability or understanding as 

4 outlined by an established standard for operator performance (e.g., facility 

5 procedure, policy, learning objective, regulatory requirement, etc.). 

6 For simulator scenarios, hold applicants accountable for PDs corrected by other members of the 

7 control room team.

8 2. Evaluate the Applicant’s Performance on Job Performance Measures

9 The following is a list of steps for how to evaluate the applicant’s performance on JPMs:

10

11 a. On the applicant’s Form 3.6-1, “Job Performance Measure Grade Report,” enter the 

12 titles of the administrative topic JPMs and control room/in-plant systems JPMs that the 

13 applicant took for the walkthrough portion of the operating test.

14

15 b. Evaluate each PD highlighted in the examiner notes. Grade each JPM listed on the form 

16 as satisfactory (S) or unsatisfactory (U) using the following criteria:

17

18 • If the applicant did not complete a time-critical JPM in the allotted time, grade the 

19 JPM as unsatisfactory. 

20

21 • If the applicant did not complete the task standard, grade the JPM as 

22 unsatisfactory.

23

24 • The applicant must accomplish the JPM task standard by correctly completing all 

25 the critical steps. If the applicant initially missed a critical step but later 

26 performed it correctly and accomplished the task standard without degrading the 

27 condition of the system or the plant, grade the applicant’s performance on that 

28 JPM as satisfactory. 

29

30 • The responses to any performance-based followup questions asked during 

31 operating test administration must confirm that the applicant’s understanding of 

32 the administrative topic or plant system is satisfactory. If the responses to any of 

33 the followup questions reveal that the applicant’s understanding of the 

34 administrative topic or plant system is severely deficient, the applicant may 

35 receive an unsatisfactory grade for the JPM even though the applicant 

36 successfully completed the task standard for the JPM. Thoroughly document the

37 reasons for this type of unsatisfactory grade. 

38

39 c. Document the grade for each JPM by placing an S or U in the “Evaluation” column for 

40 that JPM on Form 3.6-1.

41

42 d. Using the instructions in Section C of this examination standard, document all PDs for 

43 each JPM, regardless of the applicant’s grade, on Form 3.6-4, “Operating Test

44 Comments.” If applicable, also document the reason for terminating any JPMs before 

45 the applicant could complete the task standard. Reference the comment page number 

46 on Form 3.6-1. 

47
ES-3.6, Page 4 of 27

1 3. Document the Applicant’s Score on Job Performance Measures

2 Document the applicant’s results for both the overall walkthrough and administrative topic JPMs

3 by placing an S or a U in the appropriate blocks in the operating test summary section on 

4 Form 5.1-2. Enter W or E if any part of the walkthrough was waived or excused, respectively.

5

6 a. Overall Score for Walkthrough

7 Determine the score for the overall walkthrough by calculating the percentage of satisfactory 

8 grades for the administrative topic and systems JPMs. If the applicant has an S on fewer than 

9 80 percent of the administrative topics and systems combined (i.e., 12/15 for RO and SRO-I 

10 applicants and 8/10 for SRO-U applicants), the applicant fails the walkthrough portion of the 

11 operating test and receives a U overall.

12

13 b. Score for Administrative Topic Job Performance Measures

14 To determine if the applicant demonstrated minimal competence on the administrative topics, a 

15 separate score is calculated for the applicant’s administrative topic JPMs. Calculate the 

16 percentage of satisfactory grades for the administrative topic JPMs. If an SRO applicant has an 

17 S on fewer than 60 percent (i.e., 3/5) or an RO applicant has an S on fewer than 50 percent 

18 (i.e., 2/4) of the administrative topic JPMs, the applicant fails this part of the walkthrough. 

19

20 Retake applicants who were previously granted an excusal from the systems walkthrough must 

21 achieve a satisfactory grade on at least 80 percent of the administrative topic JPMs (i.e., 4/5 for 

22 RO and SRO applicants) to pass.

23

24 4. Evaluate the Applicant’s Performance on Simulator Scenarios

25 The following (a - c ) is a list of steps for how to evaluate the applicant’s performance on 

26 simulator scenarios:

27

28 a. Types of Performance Deficiencies for Simulator Scenarios

29 For the simulator scenarios, two subcategories of PDs, under specific circumstances, allow for 

30 more stringent grading than that which would otherwise be applied to a PD:

31

32 • A critical performance deficiency (CPD) is a PD associated with the failure to complete a 

33 CT. The CT must meet the elements of a CT in ES-3.3. 

34

35 Applicants will be held accountable for CPDs corrected by other members of the control 

36 room team. If an applicant neglects to take an action or takes an incorrect action and is 

37 subsequently corrected by a team member, the examination team will determine the 

38 impact of that lack of action or incorrect action on the scenario as it relates to a CT. The 

39 measurable performance standard for this type of CT depends on the consequence of 

40 the applicant’s lack of action or incorrect action if the crew had not corrected it.

41

42 A single CPD in a competency other than the communications competency results in a 

43 direct failure of the simulator operating test for the applicant. The examiner must still 

44 document and grade the applicant’s performance independent of any CPDs to provide a 

45 complete assessment of the applicant’s competence and an account of all PDs. 

46
ES-3.6, Page 5 of 27

1 • A significant performance deficiency (SPD) results in a larger rating factor (RF) point 

2 deduction than a regular PD. The examination team identifies SPDs after administering 

3 each simulator scenario. SPDs include any of the following:

4

5 – PDs that either cause an automatic RPS/ESF actuation or warrant a manual 

6 RPS/ESF actuation that should have otherwise been avoidable had the applicant 

7 responded to the event as expected

8

9 Note: Subsequent RPS/ESF actuations that do not alter equipment 

10 alignments are not treated as additional SPDs.

11

12 – PDs that would result in an avoidable emergency action level entry or escalation 

13 in accordance with the facility’s emergency plan emergency action level matrix

14

15 – PDs resulting in unplanned power changes of more than 10-percent rated 

16 thermal power

17

18 b. Code and Grade Performance Deficiencies

19 Review each PD for the simulator operating test. Using the applicable competency and RF 

20 worksheet (Form 3.6-5 for RO applicants and Form 3.6-6 for SRO applicants) and the 

21 instructions and guidance in this step, code each PD with an RF (number and letter) from the

22 competency area that most accurately reflects the deficiency. 

23

24 Note that Competency 3, “Control Board Operations,” may be “not observed” for SRO-U 

25 applicants. However, if the SRO-U applicant was observed in this area, then grade this 

26 competency and factor it into the applicant’s overall grade.

27

28 No more than two RFs can be assigned to each PD. If a second RF is assigned, it must be 

29 from a different competency area.

30

31 Assess all PDs using the competency RF descriptions listed in Forms 3.6-5 and 3.6-6 in 

32 conjunction with the criteria in items 1–4 below. If the examiner concludes a “No” response 

33 exists after comparing the PD against the RF questions on Forms 3.6-5 and 3.6-6 and 

34 assessing the performance using the criteria below, the examiner must lower the applicant’s 

35 score in the corresponding RF(s) by the appropriate number of points. Table 3.6-1, “Summary 

36 of RF Scores by PD Type,” summarizes how to reduce RF scores.

37
ES-3.6, Page 6 of 27

1

2 Table 3.6-1 Summary of RF Scores by PD Type

CPD SPD RF Score 

Point Deduction

All Other PDs

RF Score Point Deduction

Competency/RF

RFs other than 

those in the 

communications 

competency

Mark RF as UNSAT

(Failure of simulator 

operating test)

2 points 1 point

Communications 

competency

RFs

Reduce RF score by 

1 point

1 point

First non-CPD or non-SPD 

per RF: none

Second/third PD in the 

same RF: 1 point total 

Four or more PDs in the 

same RF: 2 points total 

(minimum RF score of 1)

3

4 Use the following criteria when determining whether the PD affects the applicant’s simulator 

5 operating test score:

6

7 • For a PD related to the applicant’s ability to operate (including operating controls, 

8 directing operations, and implementing procedures), reduce the score in the applicable 

9 RF(s) if either of the following is observed: 

10

11 – The applicant’s action or inaction fails to meet an expectation established by a 

12 standard for operator performance.

13

14 – The applicant’s inaction or intent to perform an action would have failed to meet 

15 an expectation established by a standard for operator performance, but another 

16 crew member corrected for the error.

17 • For a PD related to the applicant’s understanding (including diagnosing plant conditions 

18 and understanding system operation), reduce the score in the applicable RF(s) if any of 

19 the following are observed:

20

21 – The applicant exhibits a lack of understanding by providing an erroneous 

22 response to a followup question related to an observed potential PD during the 

23 scenario.

24 – The applicant’s statement (verbal or written) reveals a lack of understanding 

25 (related to required operator knowledge) that the applicant does not correct, 

26 using his or her own knowledge, before taking improper action based on the 

27 misunderstanding or before the initiation of the next scenario event. The 

28 applicant may use a reference to correct himself/herself. 

29
ES-3.6, Page 7 of 27

1 – The applicant’s delay in taking a required action reveals a lack of understanding

2 that the applicant does not correct, using his or her own knowledge, and the 

3 resulting inaction complicates the crew’s response to the event. 

4

5 Note: Typically, an unsatisfactory delay in taking required action should be 

6 assessed in RF 3.a as lack of ability to manipulate controls in a timely and 

7 accurate manner. However, if misunderstanding is the primary cause of the 

8 applicant’s delay, then RF 3.b can be considered. The NRC does not expect the 

9 applicant to exhibit an immediate and unwavering understanding of plant 

10 conditions at all times. The applicant will need a period of time to evaluate plant 

11 conditions and come to a state of understanding. The applicant may pause while 

12 operating the plant to establish, improve, or confirm this understanding. This 

13 behavior is consistent with industry expectations and standards for operator 

14 human performance techniques (e.g., stop-think-act-review or STAR). The 

15 examiner must provide evidence of the applicant’s lack of understanding while 

16 operating plant controls in order to cite RF 3.b and reduce the RF 3.b score. The 

17 following are examples of a lack of understanding:

18

19  The applicant relies on another crew member’s knowledge to effectively 

20 assess and operate the plant.

21  The applicant does not successfully exhibit the understanding required to 

22 take prompt and prudent action to avoid more complicated plant 

23 conditions.

24  The applicant provides an erroneous response to followup questions.

25

26 • For a PD related to the applicant’s ability to communicate, reduce the score in the 

27 associated RF if both of the following are observed:

28

29 – Communication made by the applicant is inaccurate, incomplete, or not in 

30 accordance with the licensee’s established means for ensuring precise 

31 communications.

32

33 – The communication is not self-corrected, and the communication is needed to 

34 support effective plant operation.

35 Note: Temporary miscommunication that stems simply from one applicant 

36 mishearing another, if it is corrected by licensee’s established means for 

37 ensuring precise communications, does not result in a point deduction. 

38

39 • If the examiner concludes a “Yes” response exists after comparing the PD against the 

40 RF questions in Forms 3.6-5 and 3.6-6 or if items 1–3 above do not apply to the PD,

41 then the PD does not impact any RF score. Document the PD in the comments section 

42 for the applicable grade report. 

43

44 c. Specific Instructions for Performance Deficiencies Involving Technical Specifications

45 Use the following instructions when evaluating SRO applicant performance against the RF 

46 areas in the TS competency. Treat each TS event separately. A TS event can be either a 
ES-3.6, Page 8 of 27

1 predetermined TS-related event as designated in the scenario Forms 3.3-1 or 3.3-2 or a TS2 related condition that occurs as a result of the flow of the scenario or actions of the applicants. 

3 Additionally, if, during followup questioning, the applicant corrects a TS determination that was 

4 made incorrectly during the scenario (provided that time limits associated with any required 

5 actions had not yet expired by the end of the scenario), then do not penalize the applicant for 

6 the original error in the TS competency.

7

8 Every missed TS entry represents a PD, except for missed TS entries that fall under RF 6.a, 

9 which are limited to one PD per TS event. For example, if a single event during a scenario has 

10 three associated TS entries, each missed TS from that single event constitutes a separate PD. 

11 Similarly, if an applicant incorrectly determines that an inoperability exists for an operable 

12 component or identifies and enters a TS that does not apply, these represent PDs that must be 

13 graded accordingly. However, PDs related to recognition (RF 6.a) should not be “carried 

14 forward” as PDs under location (RF 6.b) or TS compliance (RF 6.c). Similarly, an applicant who 

15 recognizes that an inoperability exists but who does not locate the correct TS (RF 6.b) cannot 

16 have that PD “carried forward” as a PD under TS compliance (RF 6.c) unless the applicant’s 

17 deficient knowledge about TS compliance is substantiated by followup questioning. 

18

19 Rating Factor 6.a

20 Did the applicant RECOGNIZE when instruments/components were inoperable and when 

21 conditions were covered by the TS?

22

23 Evaluate RF 6.a at the event level; evaluate TS-related conditions that arise as a result of the 

24 flow of the scenario or due to unexpected operator action as separate TS events. For example, 

25 if the condition arose from the flow of the scenario, enter a TS because the departure from

26 nucleate boiling limit was exceeded, and this condition represents a separate TS event. 

27 Similarly, if a crew takes an unexpected action that causes TS-related equipment to be 

28 considered inoperable, then this condition represents a separate TS event.

29

30 For a given TS event, determine whether the applicant recognized that TS entry is required. 

31 The applicant demonstrates competence by identifying that the event has TS implications and 

32 making a TS determination. An applicant’s TS determination may include announcing entry into 

33 one or more limiting conditions for operation (LCOs), logging the LCO entry in the crew’s log, 

34 discussing the LCO entry during followup questioning, or a combination of these. It is 

35 insufficient for an applicant to state that he or she is “evaluating TS.” The applicant must make 

36 a clear and definitive TS determination. Additionally, an applicant demonstrates competence for 

37 an event that has no TS implications by not making a TS determination. Once the applicant has 

38 made a TS determination associated with the event, then refer to the descriptions in RF 6.b and 

39 RF 6.c to assess the accuracy of the applicant’s TS determination.

40

41 PDs in RF 6.a. are limited to one PD per TS event whether the applicant failed to make a TS 

42 determination when required or made a TS determination when TS entry was not required.

43

44 Rating Factor 6.b

45 Did the applicant demonstrate an ability to LOCATE the appropriate TS for the equipment he or 

46 she determined was inoperable, covered by TS, or both?

47

48 Evaluate RF 6.b at the LCO level. An applicant demonstrates competence in RF 6.b by 

49 entering the required LCOs, and only the required LCOs, for a given event. For each LCO, the 
ES-3.6, Page 9 of 27

1 entry must be considered correct in terms of any functional units that are applicable for the 

2 inoperable component. Functional units are not to be evaluated independently for separate 

3 PDs; either the LCO entry is entirely correct or it is not. 

4

5 Deduct 1 point in this RF for each required LCO that is not entered correctly. If an applicant 

6 enters an LCO that is not required, this error represents a separate PD only if it can be 

7 determined through followup questioning that the LCO entry was not made in lieu of a correct 

8 LCO entry. If it is determined that an applicant had the choice between multiple LCOs and 

9 simply selected the wrong one, then this error is considered a single PD. Each required LCO 

10 entry represents a separate opportunity for a PD in RF 6.b. The number of 1-point deductions 

11 for failure to enter a required LCO is limited to the number of LCOs that are required to be 

12 entered for a given event. Limit additional 1-point reductions for entry into LCOs that are not 

13 required to only those incorrect LCO entries that were made due to separate and independent 

14 misconceptions that did not contribute to an applicant’s failure to enter a correct LCO.

15

16 Rating Factor 6.c

17 Did the applicant correctly INTERPRET and ensure COMPLIANCE with TS and LCO action

18 statements?

19

20 Evaluate RF 6.c at the LCO level. For each required LCO entry, the applicant demonstrates

21 competence in RF 6.c by identifying and complying with the correct conditions and action 

22 statements, and only the correct conditions and action statements. For any action statement 

23 with a time requirement that fits within the scenario runtime, an applicant further demonstrates 

24 compliance by ensuring that the necessary action is taken within the required timeframe. If the 

25 applicant does not identify the correct conditions and action statements, and only the correct 

26 conditions and action statements, for a given LCO, then deduct 1 point in RF 6.c for the specific 

27 LCO. For example, if the correct TS determination is LCO 3.1.1, Conditions A, B, and C, and 

28 instead the applicant enters LCO 3.1.1, Conditions D, E, and F, then this would constitute a 

29 1-point deduction in RF 6.c. The PD is that the applicant failed to identify the correct 

30 conditions/action statements for the LCO that was entered. Evaluate each required LCO that is 

31 entered in this manner. 

32

33 Therefore, for each TS event, limit the number of PDs in RF 6.c to the number of LCOs required 

34 to be entered.

35

36 5. Document the Applicant’s Rating Factor Scores for Simulator Scenarios

37 Document the grade for each RF on the applicant’s Form 3.6-2, “Reactor Operator Simulator 

38 Scenario Grade Report,” or Form 3.6-3, “Senior Reactor Operator Simulator Scenario Grade 

39 Report,” as follows:

40

41 • If there is no basis upon which to grade an RF (i.e., it is “not observed” (N/O)), enter “0” 

42 under “Weighting Factors,” and enter “N/O” under “RF Grades.” Depending on which RF 

43 is “N/O,” circle the appropriate weighting factors for each remaining RF applicable to that 

44 competency; the weighting factors for each competency must always add up to one. If 

45 more than one RF per competency or more than two RFs overall are not observed, 

46 inform the NRC regional office management and consult with the NRR operator licensing 

47 program office to determine whether the test supports a licensing decision.

48
ES-3.6, Page 10 of 27

1 – On Form 3.6-4, document the reason for grading an RF as “N/O” (e.g., the 

2 simulator malfunctioned, an event did not take place as planned, another 

3 applicant interceded).

4

5 • If an applicant performed an activity related to an RF and had no PDs, enter an RF score 

6 of “3” for that RF.

7

8 • For PDs not categorized as critical or significant and not related to RFs under the 

9 communications and crew interactions (communications) competency—

10

11 – If an applicant has a single PD related to an RF, enter an RF score of “2” for that 

12 RF. 

13

14 – If an applicant has two PDs related to an RF, enter an RF score of “1” for that 

15 RF.

16

17 – If an applicant has three or more PDs related to an RF, circle an RF score of “0.” 

18

19 • For PDs not categorized as critical or significant but related to RFs under the 

20 communications and crew interactions (communications) competency—

21

22 – Do not deduct points for the first PD in any RF under the communications 

23 competency. If an applicant has a second or a third PD in an RF under the 

24 communications competency, deduct 1 point, resulting in an RF score of “2.”

25

26 – If an applicant has four or more PDs in the same RF under the communications 

27 competency, deduct 2 points, resulting in an RF score of “1.” Note: The 

28 minimum score for PDs in RFs under the communications competency is “1.”

29 • For SPDs—

30

31 – A SPD results in a 2-point deduction for that RF unless it is in the 

32 communications competency. SPDs coded with RFs in the communications 

33 competency result in a 1-point deduction for that RF.

34

35 • For CPDs—

36

37 – A CPD coded with a RF in the communications competency results in a 1-point 

38 deduction for that RF.

39

40 – For a CPD coded with a RF in all other competency areas (except 

41 communications), mark the associated RF score(s) for the CPD as “UNSAT” on 

42 the applicant’s Form 3.6-2 or 3.6-3. If the applicant has additional PDs in the 

43 same RF as the CPD, document these PDs and their associated RF in the 

44 comment section on the applicant’s Form 3.6-4 and reference the comment page 

45 number for the associated RFs in the comment column on Form 3.6-2 or 3.6-3.

46

47 Using the instructions in Section C of this examination standard, document all PDs on Form 3.6-

48 4 regardless of impact to the applicant’s simulator scenario score. Include reasons for coding 

49 each PD with the specific RF(s). For each comment, reference the comment page number on 

50 Form 3.6-2 or Form 3.6-3. 
ES-3.6, Page 11 of 27

1

2 6. Determine and Document RF Grades, Competency Grades, and Overall Simulator 

3 Test Scores for Simulator Scenarios

4 An “UNSAT” RF score results in an “UNSAT” RF grade, and an “UNSAT” competency grade. 

5 Subsequently, an “UNSAT” competency grade equates to a failure of the simulator operating 

6 test. Even though the applicant has failed the simulator operating test, document all other RF 

7 scores, RF grades, and competency grades, following the instructions in this step.

8

9 For each RF on Form 3.6-2/3, multiply the RF score by its associated weighting factor to obtain 

10 a numerical measure (the RF grade).

11

12 For each competency on Form 3.6-2 and Form 3.6-3, sum the RF grades to obtain a 

13 competency grade and enter the resulting competency grade in the designated column. (The 

14 grades should range between 0 and 3.)

15

16 Using the following evaluation criteria, determine whether the applicant’s overall performance on 

17 the simulator test is satisfactory (S) or unsatisfactory (U) and document the grade by placing an 

18 S or a U in the “Simulator Operating Test” block of the “Operating Test Summary” on 

19 Form 5.1-2. Enter W or E if this part of the operating test was waived or excused, respectively.

20

21 If the applicant has a CPD, mark the overall simulator performance as unsatisfactory.

22

23 If the applicant does not have any CPDs, determine the overall performance as follows:

24 • The applicant’s performance is satisfactory if the grade for all competencies is greater 

25 than 1.8.

26

27 • The applicant’s performance is satisfactory if the grade for Competency 4, 

28 “Communications and Crew Interactions,” is less than or equal to 1.8 but greater than 

29 1.0 and the individual grades for all other competencies are 2.0 or greater.

30

31 • The applicant’s performance is unsatisfactory if the grade for Competency 4 is 1.0 or the 

32 grade for any other competency is 1.8 or less.

33

34 7. Finalize the Documentation

35 Review and finalize the simulator scenarios that were run during the operating test.

36

37 Complete Form 3.3-1 by entering the applicants’ names, the positions they occupied during the 

38 scenario, and the facility’s name on the top of the form. Enter on Form 3.3-1 any scenario 

39 revisions that were made during the test, so that each form accurately shows all the events that 

40 actually occurred during each scenario. Change the event numbers, malfunction numbers, 

41 malfunction types, and descriptions, as necessary, to reflect the “as-run” conditions. These 

42 changes may be made either by using pen and ink or by retyping the scenario as long as the 

43 final form is clear and legible. 

44

45 Update each Form 3.3-2 to reflect the “as-run” conditions. Discard or mark as “not used” any 

46 events that were not run and fill out new forms for any events that were run but not originally 

47 planned. Neatly enter notes, comments, and additional actions in the spaces between the 

48 expected operator actions.
ES-3.6, Page 12 of 27

1

2 The final Forms 3.3-1 and 3.3-2 must be a clear, legible, and sequential record of the actual 

3 events and actions that occurred during the simulator operating test. The forms sent to the 

4 applicant shall not contain any rough notes or irrelevant comments.

5

6 Explain deviations from the nominal grading criteria in detail. For example, an examiner may 

7 conclude that an applicant’s performance is acceptable despite exhibiting deficiencies that 

8 would normally result in an unsatisfactory grade. Conversely, an examiner may conclude that 

9 an applicant’s performance is unacceptable even though the documented deficiencies would 

10 normally result in a passing grade. In either case, the examiner shall document the basis for 

11 concluding that the applicant is, in fact, (un)acceptably proficient in that area, why the nominal 

12 grading criteria might be too (lenient) severe, or how a flaw in the test item might have 

13 contributed to the applicant’s deficient performance (or a combination of these). The NRC

14 regional office shall obtain written concurrence from the NRR operator licensing program office 

15 before completing the licensing action.

16

17 Retain notes and documentation until the NRC takes its licensing action on all the applications 

18 and adjudicatory actions on any hearing demands are complete. When those actions are 

19 complete, consult Management Directive 3.53, “NRC Records and Document Management 

20 Program,” to determine the record status and disposition the notes and documents accordingly. 

21 Such notes may be subject to disclosure if requested under the Freedom of Information Act.

22

23 8. Make a Final Recommendation

24 After grading and documenting the operating test, in the “Examiner Recommendation” area of

25 the applicant’s Form 5.1-2, make an overall recommendation for the operating test by checking 

26 the “Pass” or “Fail” (or “Excuse/Waive” if the entire operating test was excused or waived in 

27 accordance with ES-2.2, “Applications, Medical Requirements, and Waiver and Excusal of 

28 Examination and Test Requirements”) block and signing and dating in the row for the operating 

29 test recommendation. Make a “Pass” recommendation only if all “Operating Test Summary” 

30 blocks of Form 5.1-2 contain satisfactory (S) grades or the letters W or E, indicating that the 

31 applicant did not have to be examined in that area.

32

33 If, during the JPM portion of the operating test, an applicant demonstrated a PD with serious 

34 safety consequences, the examiner may recommend an overall walkthrough failure even if the 

35 grading instructions in this examination standard would normally result in a passing grade. 

36 Conversely, if, on any part of the operating test, an applicant demonstrated several PDs with 

37 minimal or no safety consequences, the examiner may recommend a “Pass” if the grading 

38 instructions would normally result in a failing grade. In either case, the examiner shall 

39 thoroughly justify and document the basis for the recommendation on Form 3.6-4. The NRC

40 regional office shall obtain written concurrence from the NRR operator licensing program office 

41 before completing the licensing action.

42

43 Assemble the operating test results package (including Forms 5.1-2, 3.6-1, 3.6-2 or 3.6-3, 3.6-4,

44 3.3-1, and 3.3-2 and all supporting documentation, such as strip chart recordings and applicant 

45 notes and drawings) for each applicant and forward the package to the NRC chief examiner to 

46 review.

47
ES-3.6, Page 13 of 27

1 C. Specific Instructions for Documenting Performance Deficiencies

2 For each applicant, document every PD. The level of detail required for documenting a PD

3 depends on whether the PD contributed to the applicant’s failure of that part of the operating 

4 test.

5

6 1. For Performance Deficiencies that Contribute to a Failure:

7 To document a PD that contributes to a failure of that part of the operating test (overall 

8 walkthrough, administrative topics, simulator operating test), include the following details (as 

9 applicable):

10

11 a. the task administered (i.e., describe the JPM or simulator scenario and event, as well as 

12 the applicant’s position on the operating crew)

13

14 b. the applicant’s incorrect action and an indication of whether the action was a JPM critical 

15 step or a simulator scenario CPD or SPD

16

17 c. for a CPD or SPD on the simulator operating test, an explanation of how the associated 

18 criteria are met

19

20 d. the lack of knowledge or ability that the applicant demonstrated

21

22 e. the potential or actual consequences of the applicant’s incorrect action

23

24 f. the associated followup questions asked and the applicant’s responses

25

26 g. the correct answer or action, with an appropriate facility licensee reference (e.g., lesson 

27 plan, system description, procedure name and number)

28

29 h. the K/A number and its importance rating from the applicable K/A catalog, the facility 

30 licensee’s learning objectives, or both

31

32 i. the item from 10 CFR 55.45(a) that the applicant did not understand or was unable to 

33 perform

34

35 Whenever possible, substantiate comments with printouts, strip chart recordings, or other 

36 collected parameter data generated during the simulator operating test, as well as with figures, 

37 drawings, flowcharts, or forms generated or used by the applicant. For a PD related to a 

38 followup question, document the followup question and the applicant’s response. 

39

40 Provide specific supporting details (and avoid generalizing the PD) so that the licensing official 

41 can make a logical decision about the examiner’s recommendation (ES-5.1, “Issuing Operator 

42 Licenses and Postexamination Activities”).

43

44 2. Performance Deficiencies that Do Not Contribute to a Failure

45 To document a PD that does not contribute to an operating test failure, include a brief statement 

46 describing the PD and the expected action. For a PD related to a followup question, document 

47 the followup question and the applicant’s response. 

48
ES-3.6, Page 14 of 27

1 Provide enough detail (and avoid generalizing the PD) so that the independent reviewer, 

2 responsible supervisor, and licensing official can make a logical decision in support of the 

3 examiner’s recommendation with regard to issuing the operator license.

4

5 D. Forms

6 Form 3.6-1 Job Performance Measure Grade Report

7 Form 3.6-2 Reactor Operator Simulator Scenario Grade Report

8 Form 3.6-3 Senior Reactor Operator Simulator Scenario Grade Report

9 Form 3.6-4 Operating Test Comments

10 Form 3.6-5 Reactor Operator Competency Grading Worksheet for the Simulator Operating 

11 Test

12 Form 3.6-6 Senior Reactor Operator Competency Grading Worksheet for the Simulator 

13 Operating Test

14
ES-3.6, Page 15 of 28

Form 3.6-1 Job Performance Measure Grade Report

PRIVACY ACT INFORMATION—FOR OFFICIAL USE ONLY

Applicant Docket Number: 55- Page of

Walkthrough Grading Details Evaluation

(S or U)

Comment Page

Number

Administrative Topics

a. 

b. 

c. 

d. 

e. 

Systems—Control Room

a.

b.

c.

d.

e.

f.

g.

h.

Systems—In Plant

i.

j.

k.

PRIVACY ACT INFORMATION—FOR OFFICIAL USE ONLY
ES-3.6, Page 16 of 27

Form 3.6-2 Reactor Operator Simulator Scenario Grade Report 

PRIVACY ACT INFORMATION—FOR OFFICIAL USE ONLY

Applicant Docket Number: 55- Page of

Reactor Operator Simulator Operating Test Grading Details

Competencies/

Rating Factors (RFs)

RF

Weights

RF

Scores

RF

Grades

Comp.

Grades

Comment 

Page No.

1. Interpretation/Diagnosis

a. Recognize and Verify 

Status

b. Interpret and Diagnose 

Conditions

c. Prioritize Response

_____

_____

_____

_____

_____

_____

_____

_____

_____

_____

_____

_____

_____

2. Procedures/Technical

Specifications

a. Reference

b. Procedure Compliance

c. Technical Specification

Entry

_____

_____

_____

_____

_____

_____

_____

_____

_____

_____

_____

_____

_____

3. Operate Plant Component 

Controls

a. Locate and Manipulate

b. Understanding

c. Manual Control

_____

_____

_____

_____

_____

_____

_____

_____

_____

_____

_____

_____

_____

4. Communications

a. Provide Information

b. Receive Information

c. Carry Out Instructions

_____

_____

_____

_____

_____

_____

_____

_____

_____

_____

_____

_____

_____

Note: Enter RF weights (nominal, adjusted, or “0” if not observed (N/O)); RF scores (0, 1, 2, 3, 

or N/O); and RF grades from Form 3.6-5, “Reactor Operator Competency Grading Worksheet 

for the Simulator Operating Test,” and sum to obtain competency grades.

PRIVACY ACT INFORMATION—FOR OFFICIAL USE ONLY
ES-3.6, Page 17 of 27

Form 3.6-3 Senior Reactor Operator Simulator Scenario Grade Report 

PRIVACY ACT INFORMATION—FOR OFFICIAL USE ONLY

Applicant Docket Number: 55- Page of

Senior Reactor Operator Simulator Operating Test Grading Details

Competencies/

Rating Factors (RFs)

RF

Weights

RF

Scores

RF

Grades

Comp.

Grades

Comment 

Page No.

1. Interpretation/Diagnosis

a. Recognize and Attend

b. Ensure Accuracy

c. Understanding

d. Diagnose

_____

_____

_____

_____

_____

_____

_____

_____

_____

_____

_____

_____

____

_____

_____

_____

_____

2. Procedures

a. Reference

b. Emergency Operating 

Procedure Entry

c. Correct Use

_____

_____

_____

_____

_____

_____

_____

_____

_____

____

_____

_____

_____

3. Operate Plant Component 

Controls

a. Locate and Manipulate

b. Understanding

c. Manual Control

_____

_____

_____

_____

_____

_____

_____

_____

_____ ____

_____

_____

_____

4. Communications

a. Clarity

b. Crew and Others Informed

c. Receive Information

_____

_____

_____

_____

_____

_____

_____

_____

_____

____

_____

_____

_____

5. Directing Operations

a. Timely and Decisive 

Action

b. Oversight

c. Solicit Crew Feedback

d. Monitor Crew Activities

_____

_____

_____

_____

_____

_____

_____

_____

_____

_____

_____

_____

____

_____

_____

_____

_____

6. Technical Specifications

a. Recognize 

b. Locate 

c. Compliance

_____

_____

_____

_____

_____

_____

_____

_____

_____

____

_____

_____

_____

Note: Enter RF weights (nominal, adjusted, or “0” if not observed (N/O)); RF scores (0, 1, 2, 3, 

or N/O); and RF grades from Form 3.6-6, “Senior Reactor Operator Competency 

Grading Worksheet for the Simulator Operating Test,” and sum to obtain competency 

grades.

PRIVACY ACT INFORMATION—FOR OFFICIAL USE ONLY
ES-3.6, Page 18 of 28

Form 3.6-4 Operating Test Comments

Use this form or an equivalent and the instructions in Section C of ES-3.6, “Grading and 

Documenting Operating Tests for Operator Licensing Initial Examinations,” to document the 

applicant’s PDs. Items marked with a * do not apply if the applicant has a passing score for the 

associated part of the operating test (overall walkthrough, administrative topics, simulator 

operating test).

PRIVACY ACT INFORMATION—FOR OFFICIAL USE ONLY

Reference: [page, JPM No.]

JPM: [topic or name]

Simulator Scenario [No. or name], Event [No. or name] 

Assigned Competency Area(s) and Rating Factor(s)

Expected Action/Response and Associated References:

Applicant Action/Response:

*Lack of Knowledge/Ability:

*Potential/Actual Consequences:

Followup Questions and Response:

*Knowledge/Ability and Importance Rating and/or Learning Objective:

*Reference to 10 CFR 55.45(a):

PRIVACY ACT INFORMATION—FOR OFFICIAL USE ONLY
ES-3.6, Page 19 of 28

Form 3.6-5 Reactor Operator Competency Grading Worksheet for the Simulator 

Operating Test

Reactor Operator (RO) Competency 1:

Interpret/Diagnose Events and Conditions Based on Alarms, Signals, and Readings

This competency does not include knowledge of system operation such as setpoints,

interlocks, or automatic actions nor the understanding of how one’s actions affect the plant and 

system conditions—this is evaluated in Competency 3.

Rating Factors (RFs) Weighting Factors RF 

Scores

RF 

Grades

Comp.

Grade

(a) Did the applicant RECOGNIZE 

and VERIFY off-normal trends 

and status accurately and 

promptly in order to identify 

conditions that are out of 

specification? 

Not observed = 0

(N/O) 3

_______

Nominal = 0.33 2

(b) N/O = 0.50 1

(c) N/O = 0.50 0

(b) Did the applicant correctly 

INTERPRET/DIAGNOSE plant 

conditions based on control room 

indications to guard against and 

mitigate conditions that are out of 

specification and to verify that 

signals are consistent with plant 

and system conditions?

N/O = 0 3

Nominal = 0.34 2

(a) N/O = 0.50 1

(c) N/O = 0.50 0

(c) Did the applicant prioritize and 

ATTEND TO annunciators, alarm 

signals, and instrument readings 

in order of importance and 

severity?

N/O = 0 3

Nominal = 0.33 2

(a) N/O = 0.50 1

(b) N/O = 0.50 0
ES-3.6, Page 20 of 28

RO Competency 2: 

Comply with and Use Procedures, References, and Technical Specifications

Rating Factors (RFs) Weighting Factors RF Scores RF 

Grades

Comp.

Grade

(a) Did the applicant REFER TO 

and/or VERIFY the appropriate 

normal, alarm/annunciator, 

abnormal, emergency, and 

administrative procedure or 

reference in sufficient time to 

avoid adverse impacts on plant 

status?

N/O = 0 3

_______

Nominal = 0.33 2

(b) N/O = 0.50 1

(c) N/O = 0.50

0

(b) Did the applicant COMPLY WITH 

procedures (including 

precautions and limitations) and 

references in an accurate 

manner and in sufficient time to 

avoid adverse impacts on plant 

status?

This includes recognizing 

emergency operating procedure

entry conditions, carrying out 

immediate actions without 

assistance, and using control 

room reference materials, such 

as prints, books, and charts to 

aid in the response to events and 

plant conditions.

N/O = 0 3

Nominal = 0.34 2

(a) N/O = 0.50 1

(c) N/O = 0.50 0

(c) Did the applicant RECOGNIZE 

plant conditions that are 

addressed in technical 

specifications and COMPLY 

WITH required limiting conditions 

for operation and action 

statements? 

This includes the use of control 

room reference materials, such 

as prints, books, and charts, to 

aid in the diagnosis and 

classification of events and 

conditions.

N/O = 0 3

Nominal = 0.33 2

(a) N/O = 0.50 1

(b) N/O = 0.50 0
ES-3.6, Page 21 of 28

RO Competency 3:

Operate Plant Component Controls

Rating Factors (RFs) Weighting Factors RF Scores RF 

Grades

Comp.

Grade

(a) Did the applicant LOCATE AND 

MANIPULATE controls in an 

accurate manner to attain a 

desired plant and system 

response in sufficient time to 

avoid adverse impacts on plant 

conditions?

This includes the ability to locate

plant and system instruments 

and indications.

N/O = 0 3

_______

Nominal = 0.34 2

(b) N/O = 0.50 1

(c) N/O = 0.50 0

(b) Did the applicant’s actions 

demonstrate UNDERSTANDING 

OF SYSTEM OPERATION, 

including set points, interlocks, 

and automatic actions? 

This includes knowledge of 

system operation AND the

understanding of how one’s 

actions affect plant and system 

conditions.

N/O = 0 3

Nominal = 0.33 2

(a) N/O = 0.50 1

(c) N/O = 0.50 0

(c) Did the applicant demonstrate 

the ability to take MANUAL 

CONTROL of automatic 

functions when appropriate 

based on plant conditions?

N/O = 0 3

Nominal = 0.33 2

(a) N/O = 0.50 1

(b) N/O = 0.50 0
ES-3.6, Page 22 of 28

RO Competency 4:

Communicate and Interact with Other Crew Members

Rating Factors (RFs) Weighting Factors RF 

Scores

RF 

Grades

Comp.

Grade

(a) Did the applicant PROVIDE 

clear, pertinent, and accurate 

INFORMATION, oral and written, 

on system status to others for the 

performance of their jobs? This 

includes providing information to 

other crew members about

conditions affecting safe plant 

operation, regardless of which 

applicant’s control board is 

directly affected.

N/O = 0 3

_______

Nominal = 0.34 2

(b) or (c) N/O = 0.50 1

(b) Did the applicant effectively 

RECEIVE INFORMATION from 

others? This includes 

requesting, acknowledging, and 

attending to information from 

other crew members about

conditions affecting safe plant 

operation, regardless of which 

applicant’s control board is 

directly affected. 

N/O = 0 3

Nominal = 0.33 2

(a) or (c) N/O = 0.50 1

(c) Did the applicant successfully 

CARRY OUT THE 

INSTRUCTIONS of the 

supervisor?

N/O = 0 3

Nominal = 0.33 2

(a) or (b) N/O = 0.50 1
ES-3.6, Page 23 of 28

Form 3.6-6 Senior Reactor Operator Competency Grading Worksheet for the Simulator 

Operating Test

Senior Reactor Operator (SRO) Competency 1:

Interpret/Diagnose Events and Conditions Based on Alarms, Signals, and Readings

Rating Factors (RFs) Weighting Factors RF 

Scores

RF 

Grades

Comp.

Grade

(a) Did the applicant 

RECOGNIZE AND ATTEND 

TO off-normal trends and 

status, annunciators, and 

alarms in order of their 

importance and severity and 

in sufficient time to avoid 

adverse impacts on plant 

status?

This includes correctly 

interpreting the significance 

of each alarm and verifying 

that it is consistent with plant 

and system conditions.

N/O = 0 3

_______

Nominal = 0.20 2

(b) N/O = 0.25 1

(c) or (d) N/O = 0.29

0

(b) Did the applicant ensure the

collection of CORRECT,

ACCURATE, and

COMPLETE information and 

reference material on which 

to base diagnoses?

N/O = 0 3

Nominal = 0.20 2

(a) N/O = 0.25 1

(c) or (d) N/O = 0.28 0

(c) Did the applicant’s directives

and actions demonstrate an

UNDERSTANDING of how 

the PLANT, SYSTEMS, and

COMPONENTS OPERATE 

AND INTERACT (including 

set points, interlocks, and 

automatic actions)?

This does NOT include 

knowledge of system 

operation that is evaluated 

under SRO Competency 3.

N/O = 0 3

Nominal = 0.30 2

(a) or (b) N/O = 0.38 1

(d) N/O = 0.43

0

(d) Did the applicant correctly

INTERPRET/DIAGNOSE

plant conditions based on 

control room indications to 

guard against and mitigate

conditions that do not meet 

specifications?

N/O = 0 3

Nominal = 0.30 2

(a) or (b) N/O = 0.37 1

(c) N/O = 0.43

0
ES-3.6, Page 24 of 28

SRO Competency 2:

Comply with and Use Procedures and References

Rating Factors (RFs) Weighting Factors RF 

Scores

RF 

Grades

Comp.

Grade

(a) Did the applicant REFER TO 

and/or VERIFY correct 

alarm/annunciator, abnormal, 

emergency, and 

administrative procedures, 

procedural steps, and 

references when appropriate 

in sufficient time to avoid 

adverse impacts on plant 

status? 

This includes the use of 

control room reference 

materials, such as prints, 

books, and charts, to aid in 

the diagnosis and 

classification of events and 

conditions.

N/O = 0 3

______

Nominal = 0.33 2

(b) N/O = 0.50 1

(c) N/O = 0.50

0

(b) Did the applicant 

RECOGNIZE ABNORMAL 

OPERATING PROCEDURE/

EMERGENCY OPERATING 

PROCEDURE ENTRY 

CONDITIONS in sufficient 

time to avoid adverse impacts 

on plant status?

N/O = 0 3

Nominal = 0.33 2

(a) N/O = 0.50 1

(c) N/O = 0.50 0

(c) Did the applicant USE 

PROCEDURES CORRECTLY 

and ensure correct 

implementation by the crew in 

sufficient time to avoid 

adverse impacts on plant 

status? 

This includes following 

procedural steps in the correct 

sequence, abiding by 

procedural cautions and 

limitations, selecting correct 

paths on decision blocks, and 

correctly transitioning between 

procedures.

N/O = 0 3

Nominal = 0.34 2

(a) N/O = 0.50 1

(b) N/O = 0.50

0
SRO Competency 3:

Operate Plant Component Controls

Note: For SRO-upgrade (SRO-U) applicants, include grading in this competency if the 

applicant performed a scenario in a position that operated plant controls. If the SRO-U 

applicant was only tested in the SRO position, mark this competency as “N/O” (not 

observed).

Rating Factors (RFs) Weighting Factors RF Scores RF 

Grades

Comp.

Grade

(a) Did the applicant LOCATE 

AND MANIPULATE 

CONTROLS to attain a desired 

plant and system response or 

condition in sufficient time to 

avoid adverse impacts on plant 

status?

N/O = 0 3

______

Nominal = 0.34 2

(b) N/O = 0.50 1

(c) N/O = 0.50 0

(b) Did the applicant’s control 

manipulations demonstrate an 

UNDERSTANDING OF 

SYSTEM OPERATION, 

including set points, interlocks, 

and automatic actions?

This includes the ability to 

locate plant and system 

instruments and indications 

and to understand how one’s 

actions affect plant and system 

conditions.

N/O = 0 3

Nominal = 0.33 2

(a) N/O = 0.50 1

(c) N/O = 0.50 0

(c) Did the applicant demonstrate 

the ability to take MANUAL 

CONTROL of automatic 

functions when appropriate 

based on plant conditions?

N/O = 0 3

Nominal = 0.33 2

(a) N/O = 0.50 1

(b) N/O = 0.50 0
ES-3.6, Page 26 of 28

SRO Competency 4:

Communicate and Interact with the Crew and Other Personnel

Rating Factors (RFs) Weighting Factors RF Scores RF 

Grades

Comp.

Grade

(a) Did the applicant PROVIDE 

communications in a clear, 

easily understood manner?

N/O = 0 3

______

Nominal = 0.40 2

(c) N/O = 0.50

1

(b) N/O = 0.67

(b) Did the applicant KEEP crew 

members and those outside the 

control room INFORMED of 

plant status? 

N/O = 0 3

Nominal = 0.40 2

(c) N/O = 0.50

1

(a) N/O = 0.67

(c) Did the applicant ENSURE 

RECEIPT of clear, easily 

understood communications 

from the crew and others?

N/O = 0 3

Nominal = 0.20 2

(a) or (b) N/O = 0.33 1
ES-3.6, Page 27 of 28

SRO Competency 5:

Direct Shift Operations

Rating Factors (RFs) Weighting Factors RF 

Scores

RF 

Grades

Comp.

Grade

(a) Did the applicant take TIMELY 

AND DECISIVE ACTION in 

response to problems during 

both normal and off-normal 

situations? 

This includes well-thought-out 

directions that demonstrated 

appropriate CONCERN for the 

SAFETY of the plant, staff, and 

public. 

Timely actions are those actions 

that are performed in sufficient 

time to avoid adverse impacts 

on plant status.

N/O = 0 3

_______

Nominal = 0.30 2

(c) or (d) N/O = 0.38 1

(b) N/O = 0.43

0

(b) Did the applicant remain 

ATTENTIVE to control room 

indications, stay in a position of 

OVERSIGHT, and provide an 

APPROPRIATE AMOUNT of 

DIRECTION and GUIDANCE 

that facilitated CREW 

PERFORMANCE?

This includes maintaining focus 

on the “big picture.”

N/O = 0 3

Nominal = 0.30 2

(c) or (d) N/O = 0.37 1

(a) N/O = 0.43 0

(c) Did the applicant SOLICIT and 

INCORPORATE FEEDBACK 

from the crew to foster an 

effective, team-oriented 

approach to problem solving and 

decisionmaking?

N/O = 0 3

Nominal = 0.20 2

(d) N/O = 0.25 1

(a) or (b) N/O = 0.29 0

(d) Did the applicant ensure that 

THE CREW carried out 

CORRECT AND TIMELY 

ACTIVITIES (including 

diagnosis, procedural 

implementation, and operation of 

the control boards)?

Timely actions are those that are 

performed in sufficient time to 

avoid adverse impacts on plant 

status.

N/O = 0 3

Nominal = 0.20 2

(c) N/O = 0.25 1

(a) or (b) N/O = 0.28 0
ES-3.6, Page 28 of 28

SRO Competency 6:

Comply with and Use Technical Specifications

Use this worksheet in conjunction with the specific instructions for grading performance 

deficiencies involving technical specifications (TS) in ES-3.6, Section B.4.c.

Rating Factors (RFs) Weighting Factors RF 

Scores

RF 

Grades

Comp.

Grade

(a)* Did the applicant 

RECOGNIZE when 

instruments/components 

were inoperable and when 

conditions were covered by 

the TS? 

*If the applicant does not 

address TS at all, this 

weighting factor becomes 1.0.

N/O = 0 3

_______

Nominal = 0.33 2

(b) N/O = 0.50 1

(c) N/O = 0.50 0

(b) Did the applicant demonstrate 

an ability to LOCATE the 

appropriate TS for the 

equipment he or she

determined was inoperable 

and/or covered by TS?

This includes the functional 

units for instrument TS.

N/O = 0 3

Nominal = 0.33 2

(a) N/O =0.50 1

(c) N/O = 0.50 0

(c) Did the applicant correctly 

INTERPRET and ensure 

COMPLIANCE with TS and 

limiting condition for operation

action statements?

N/O = 0 3

Nominal = 0.34 2

(a) N/O = 0.50 1

(b) N/O = 0.50 0
ES 3.7, Page 1 of 1

1 3.7 ALTERNATIVES FOR IN-PLANT JOB PERFORMANCE

2 MEASURES AT PLANTS UNDER CONSTRUCTION

3 At plants under construction, facility licensees may desire to use alternatives for the in-plant 

4 JPMs for initial licensing examination purposes.

5 This examination standard provides guidance for a facility licensee of a reactor under 

6 construction that requests and is granted an exemption from the plant walkthrough requirement 

7 and proposes alternatives for in-plant JPMs in the actual plant. This guidance is intended to be 

8 used in addition to the instructions in ES-3.2, “Developing JPMs.”

9 A. Developing Alternative Job Performance Measures

10 When developing in-plant JPMs using alternative methods such that the actual plant is not 

11 entered, the JPM should contain specific performance standards that allow the NRC examiner 

12 to evaluate the applicant’s knowledge or ability (or both) to do the following:

13 1. Locate tools and equipment using plant layout diagrams, equipment diagrams, maps, or 

14 a combination of these, including how to get to the location in the plant where the task 

15 would be performed.

16 2. Describe the expected status/configuration of equipment before performing task.

17 3. Operate the equipment and explain how the applicant expects equipment and systems 

18 to respond to his or her actions, using props such as pictures of equipment or mockup 

19 equipment in lieu of actual equipment in the plant. 

20 4. Use a simulated radiologically controlled area entry checkpoint with electronic dosimetry, 

21 radiation work permits, and other radiologically controlled area entry equipment

ES-4.1, Page 1 of 28

1 4.1 DEVELOPING WRITTEN EXAMINATION OUTLINES

2 This examination standard contains the instructions for preparing written examination outlines

3 for the written examination portion of the U.S. Nuclear Regulatory Commission (NRC) operator

4 licensing initial examinations for reactor operator (RO) and senior reactor operator (SRO) 

5 license applicants at power reactor facilities. Preparing written examination outlines using the 

6 instructions in this examination standard, in conjunction with the applicable knowledge and 

7 abilities (K/A) catalog, ensures that the written examination includes a representative sample of 

8 the items specified in Title 10 of the Code of Federal Regulations (10 CFR) Part 55, “Operators’ 

9 licenses.”

10

11 A. Responsibilities

12 1. Facility Licensee

13 The facility licensee performs the following activities, as applicable, depending on the 

14 examination arrangements confirmed with the NRC regional office in accordance with the 

15 instructions in Examination Standard (ES)-2.1, “Preparing for Operator Licensing Initial 

16 Examinations,” before the scheduled examination date:

17

18 a. If available, provide the NRC regional office any prescreened K/As for elimination from 

19 the written examination outline, with a description or basis for eliminating any K/A. 

20

21 b. Establish examination security in accordance with ES-1.3, “Examination Security,”

22 before receiving the written examination outline from the NRC. 

23

24 c. Review the proposed written examination outline and provide feedback to the NRC 

25 regional office in accordance with ES-2.3, “Reviewing and Approving Operator Licensing 

26 Initial Examinations.”

27

28 d. Once the written examination outline is approved, discuss any deviations with the NRC 

29 chief examiner and follow the instructions in ES-4.2, “Developing Written Examinations.”

30

31 2. NRC Regional Office

32 The NRC regional office performs the following activities:

33

34 a. Prepare the initial written examination outline in accordance with the instructions in 

35 Section B of this examination standard.

36

37 b. Verify that the facility licensee has established examination security (ES-1.3) and then 

38 submit the written examination outline to the facility licensee point of contact for review. 

39

40 c. Review and approve the final written examination outline in accordance with the 

41 instructions in ES-2.3.

42

43 d. Review and approve any site-specific item or topic substitutions in accordance with the 

44 instructions in ES-4.2 for deviations.

45

46
ES-4.1, Page 2 of 28

1 B. Instructions for Preparing Written Examination Outlines

2 The examination author will prepare the applicable technology-specific outline (Form 4.1-BWR, 

3 “Boiling-Water Reactor Examination Outline”; Form 4.1-PWR, “Pressurized-Water Reactor 

4 Examination Outline”; Form 4.1-AP, “AP1000 Examination Outline”; or Form 4.1-ABWR, 

5 “Advanced Boiling-Water Reactor Examination Outline”) and the common examination outline

6 (Form 4.1-COMMON, “Common Examination Outline”) according to the following instructions:

7

8 1. Use the latest revision of the applicable K/A catalog available at the time that the facility 

9 licensee requests the written examination outline. The applicable K/A catalog is defined 

10 in ES-8, “Glossary” as follows:

11 The revision of the technology or vendor specific (PWR, BWR, AP1000, 

12 etc.,) Knowledge and Abilities catalog reviewed by the NRC and used to 

13 develop the written examination outline and operating test outline for 

14 initial operator licensing examinations

15

16 2. Although use of the applicable K/A catalog ensures examination consistency, because

17 several of the K/A catalogs are based on generic job and task analyses and not all 

18 facilities are the same, eliminate inapplicable or inappropriate K/A statements by 

19 (1) discarding randomly selected K/As during the outline development process or 

20 (2) prescreening the entire K/A catalog to eliminate inappropriate K/As before beginning 

21 the random selection process. Facility licensees can provide the NRC regional office 

22 with a list of “prescreened K/As” that do not apply for their plants, along with a basis for 

23 eliminating each K/A, before the NRC develops the written examination outline. Refer to 

24 the remainder of this section for specific requirements and guidance on K/A elimination. 

25 Facility licensees can also provide the NRC regional office with operationally important 

26 emergency/abnormal plant evolutions or plant systems if these topics are not 

27 represented on the examination outlines in this examination standard. The NRC 

28 regional office, in consultation with the Office of Nuclear Reactor Regulation (NRR)

29 operator licensing program office, will determine the tier/group assignments for the

30 topics before sampling. 

31

32 3. Use a systematic sampling method to select K/As for the outline. A systematic sampling 

33 methodology is any methodology that meets the following criteria: 

34

35 a. reproducible, comprehensible, and yields an examination outline that is free of 

36 bias

37 b. adheres to the applicable examination model

38 c. minimizes the number of K/As related to any system or evolution (i.e., sample 

39 every system or evolution in the group once before selecting a second K/A for 

40 any system or evolution)

41 d. samples at the specific K/A statement level.

42 Refer to Section C of this examination standard for an example of a systematic sampling 

43 method. 

44
ES-4.1, Page 3 of 28

1 4. Systematically and randomly select specific K/A statements (e.g., K1.03 or A2.11) from 

2 the applicable K/A catalog to complete each of the four tiers:

3

4 • Tier 1, “Emergency and Abnormal Plant Evolutions”

5 • Tier 2, “Plant Systems”

6 • Tier 3, “Generic Knowledge and Abilities”

7 • Tier 4, “Theory”

8

9 Document the selected Tier 1 and Tier 2 K/As on the appropriate technology-specific 

10 outline form (Forms 4.1-PWR, 4.1-BWR, 4.1-AP, or 4.1-ABWR). Document the selected 

11 Tier 3 and Tier 4 K/As on the common examination outline form (Form 4.1-COMMON). 

12

13 5. Observe the following when selecting individual K/As:

14 a. Select only those K/As having an importance rating (IR) of 2.5 or higher. K/As 

15 that have IRs of less than 2.5 may be used if justified based on plant priorities; 

16 inform the NRC chief examiner if selecting K/As with an IR less than 2.5. Use 

17 the RO and SRO ratings for the RO and SRO-only portions, respectively. 

18

19 b. For the SRO-only portion of the outline, include K/A statements that relate to the 

20 topics in 10 CFR 55.43(b) to ensure that the SRO examination tests at the 

21 appropriate license level and also do the following: 

22

23 • For Tiers 1 and 2, select from the shaded systems and K/A categories 

24 (i.e., the “A2” and “G” K/A categories, which are linked to 10 CFR 55.43, 

25 “Written examination: Senior operators,” and all the K/A categories for 

26 the fuel handling equipment, which is specifically identified for sampling in 

27 10 CFR 55.43(b)(7)).

28 • For Tier 3, select K/As linked to 10 CFR 55.43.

29 c. Ensure that at least two topics from every applicable K/A category are sampled 

30 within Tier 1 and Tier 2 of the RO and SRO-only outline sections.

31

32 d. Select topics from as many systems and evolutions as possible and sample 

33 every system or evolution in the group before selecting a second topic for any 

34 system or evolution. Treat the SRO examination, which consists of 75 RO 

35 questions and 25 SRO-only questions, as a 100-question test overall and adjust 

36 to ensure balance of coverage and appropriate sampling. 

37

38 e. For the generic K/A category in Tier 1 (i.e., Column “G” on the outline), select 

39 topics from Section 2, “Generic Knowledge and Abilities,” of the applicable K/A 

40 catalog that are relatable/relevant to the selected abnormal or emergency event. 

41

42 f. For the generic K/A category in Tier 2 (i.e., Column “G” on the outline), select 

43 topics from Section 2 and Section 5, “Components,” of the applicable K/A catalog

44 that are relatable/relevant to the selected system. 

45

46 g. For Tier 3 of the examination outline, randomly select K/As from Section 2 of the 

47 applicable K/A catalog so that the K/A categories “Conduct of Operations” and 

48 “Equipment Control” have two items each and the K/A categories “Radiation 
ES-4.1, Page 4 of 28

1 Control” and “Emergency Procedures/Plan” have one item each on the RO 

2 portion of the examination. For the SRO portion, randomly select two K/As for 

3 each category except for one category, which will have only one K/A.

4

5 h. For Tier 4 of the examination outline, randomly select K/As from Section 6, 

6 “Theory,” of the applicable K/A catalog so that the K/A categories “Reactor 

7 Theory” and “Thermodynamics” have three items each. Tier 4 topics selected 

8 must sample different theory subjects located within Section 6, “Theory,” of the 

9 applicable K/A catalog (e.g., only one question may test Reactor Theory subject 

10 292001, “Neutrons”). Note that Tier 4 is limited to the RO portion of the 

11 examination. 

12

13 i. Enter the K/A statement numbers, a brief description of each topic, the topics’ 

14 IRs for the license level of the examination (use the RO and SRO ratings for the 

15 RO and SRO-only portions, respectively), the question number, and the point 

16 totals (system, category, group, and tier) on the examination outline. 

17

18 j. If fuel handling equipment is sampled in a category other than Category A2 or G 

19 on the SRO-only examination, enter it on the left side of Column A2 for Tier 2, 

20 Group 2 (step 5.c does not apply.) 

21

22 k. Ensure that the point total for each group and tier in the proposed outline 

23 matches that specified in the table. The final point total for each group and tier 

24 may deviate by ±1 from that specified in the table based on NRC revisions. The 

25 final RO examination must total 75 points, and the SRO-only examination must 

26 total 25 points.

27

28 8. After completing the outline, check the selected K/As for balance of coverage within and 

29 across the four tiers as follows:

30

31 • Ensure that every applicable K/A category is sampled at least twice within Tier 1 

32 and Tier 2 so that a valid sample will likely be maintained if some questions are 

33 deleted as a result of postexamination comments.

34

35 • Ensure that the K/A category totals are met for Tier 3 and Tier 4 in accordance 

36 with steps 5.g and 5.h.

37

38 • Ensure that the same generic K/A is only used once for either Tier 1 or Tier 2. 

39 Generic K/As used in Tier 1 or Tier 2 can be reused for Tier 3.

40

41 • Ensure that no emergency/abnormal plant evolution, system, or K/A category is 

42 oversampled (i.e., avoid selecting more than two K/A topics from a given system 

43 unless they relate to plant-specific priorities). 

44

45 • Make any adjustments that might be necessary by systematically and randomly 

46 selecting replacement K/A statements. 

47

48 • Document and justify all changes on Form 4.1-1, “Record of Rejected Knowledge 

49 and Abilities,” and submit the documentation with the completed/revised outline.

50
ES-4.1, Page 5 of 28

1 9. Review the written examination outline using the instructions in ES-2.3 before sending it 

2 to the facility licensee to begin question development.

3

4 10. Consult the instructions for deviations from the approved examination outline in ES-4.2

5 as needed.

6

7 C. Example of a Systematic Sampling Methodology

8 This section gives an example of an acceptable systematic sampling methodology for randomly 

9 selecting K/A within the defined structure of the examination outline to achieve as broad a 

10 sample as possible. Other methodologies may be used as long as they meet all of the above 

11 criteria.

12

13 The following process uses Form 4.1-BWR for illustration. Tokens are used to randomly select 

14 from the first column of items listed in each tier/group on Form 4.1-BWR. A computer-based 

15 random number generator could be used instead of tokens.

16

17 1. Sequentially number the items in the group and sequentially mark numbers on the same 

18 number of tokens. Assuming that 1 of the 21 emergency/abnormal plant evolutions in 

19 Tier 1, Group 1, was deleted before sampling because it was not applicable to the 

20 facility, 20 tokens, numbered from 1 to 20, should remain.

21

22 a. Because the number of items remaining in the group (in this case 20) is the same 

23 as the required number of points for the group specified in the right-hand column 

24 of the examination outline, each item in the group would be sampled one time.

25

26 b. If the number of items remaining in the group is smaller than the required number 

27 of points for the group (e.g., Tier 2, Group 1, has 23 items but requires 

28 26 points), sample each item once and determine the rest of the sample by 

29 randomly selecting and removing tokens (in this case 3 of 23) until the required 

30 total number of points (26) is reached. Update Form 4.1-BWR to note the 

31 selected items.

32

33 c. If the number of items remaining in the group is larger than the required number 

34 of points for the group (e.g., Tier 1, Group 2, has 20 items but only requires 

35 7 points), randomly select and remove the required number of tokens (7) and 

36 note them on Form 4.1-BWR.

37

38 2. After selecting the topics to be sampled in each group as described in step 1, count the 

39 number of K/A categories in the group (e.g., six for each group in Tier 1 (i.e., K1, K2, K3, 

40 A1, A2, and G)) and sequentially annotate the same number of tokens (in this case six). 

41 For each emergency/abnormal plant evolution (and system) selected in step 1, randomly 

42 select and remove a token and note the K/A category on Form 4.1-BWR. If the 

43 emergency/abnormal plant evolution (or system) was sampled more than once in 

44 accordance with step 1.a, randomly select a second K/A category. If the selected K/A 

45 category contains no K/A statements that have an IR above 2.5, systematically select 

46 another K/A category, unless the lower importance is justified based on plant-specific 

47 priorities. Then replace all tokens in the container and repeat the process for every 

48 selected item in each group.

49
ES-4.1, Page 6 of 28

1 3. Use a similar method to randomly select from among the K/A statements under each 

2 selected K/A category.

3 D. Forms

4 Form 4.1-BWR Boiling-Water Reactor Examination Outline

5 Form 4.1-PWR Pressurized-Water Reactor Examination Outline

6 Form 4.1-AP AP1000 Examination Outline

7 Form 4.1-ABWR Advanced Boiling-Water Reactor Examination Outline

8 Form 4.1-COMMON Common Examination Outline 

9 Form 4.1-1 Record of Rejected Knowledge and Abilities

10
ES-4.1, Page 7 of 28

1 Form 4.1-BWR Boiling-Water Reactor Examination Outline

2

3 Note: CO = Conduct of Operations; EC = Equipment Control; RC = Radiation Control;

4 EM = Emergency Procedures/Plan

5

Facility: Date of Exam:

Tier Group RO K/A Category Points SRO-Only Points

K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G Total A2 G Total

1.

Emergency 

and 

Abnormal 

Plant 

Evolutions

1

N/A N/A

20 7

2 6 3

Tier 

Totals 26 10

2.

Plant

Systems

1 26 5

2 11 3

Tier 

Totals 37 8

3.

Generic 

Knowledge 

and Abilities 

Categories

CO EC RC EM

6

CO EC RC EM

7 2 2 1 1

4. Theory

Reactor Theory Thermodynamics

6

3 3
ES-4.1, Page 8 of 28 

Form 4.1-BWR BWR Examination Outline Page 2 

Emergency and Abnormal Plant Evolutions—Tier 1/Group 1(RO/SRO)

E/APE # / Name / Safety Function K1 K2 K3 A1 A2 G K/A IR #

295001 (APE 1) Partial or Complete 

Loss of Forced Core Flow Circulation / 1 

295003 (APE 3) Partial or Complete 

Loss of AC Power / 6

295004 (APE 4) Partial or Total Loss of 

DC Power / 6

295005 (APE 5) Main Turbine Generator 

Trip / 3

295006 (APE 6) Scram / 1

295016 (APE 16) Control Room 

Abandonment

295018 (APE 18) Partial or Complete 

Loss of CCW / 8

295019 (APE 19) Partial or Complete 

Loss of Instrument Air / 8

295021 (APE 21) Loss of Shutdown 

Cooling / 4

295023 (APE 23) Refueling Accidents / 8

295024 (EPE 1) High Drywell Pressure / 5

295025 (EPE 2) High Reactor Pressure / 3

295026 (EPE 3) Suppression Pool High 

Water Temperature / 5 

295027 (EPE 4) High Containment 

Temperature (Mark III Containment 

 

295028 (EPE 5) High Drywell 

Temperature (Mark I and Mark II only) / 

295030 (EPE 7) Low Suppression Pool 

Water Level / 5 

295031 (EPE 8) Reactor Low Water Level / 

295037 (EPE 14) Scram Condition 

Present and Reactor Power Above APRM 

Downscale or Unknown / 1

295038 (EPE 15) High Offsite 

Radioactivity Release Rate / 9

600000 (APE 24) Plant Fire On Site / 8

700000 (APE 25) Generator Voltage 

and Electric Grid Disturbances / 6

K/A Category Totals: Group Point Total: 20/7
ES-4.1, Page 9 of 28 

Form 4.1-BWR BWR Examination Outline Page 3 

Emergency and Abnormal Plant Evolutions—Tier 1/Group 2(RO/SRO)

E/APE # / Name / Safety Function K1 K2 K3 A1 A2 G K/A IR #

295002 (APE 2) Loss of Main 

Condenser Vacuum / 3

295007 (APE 7) High Reactor Pressure / 3

295008 (APE 8) High Reactor Water Level / 

295009 (APE 9) Low Reactor Water Level / 

295010 (APE 10) High Drywell Pressure / 5

295011 (APE 11) High Containment 

Temperature (Mark III Containment only) 

295012 (APE 12) High Drywell 

Temperature / 5

295013 (APE 13) High Suppression Pool 

Water

295014 (APE 14) Inadvertent Reactivity 

Addition / 1

295015 (APE 15) Incomplete Scram / 1

295017 (APE 17) High Offsite Release 

Rate / 9

295020 (APE 20) Inadvertent Containment 

Isolation / 5 & 7

295022 (APE 22) Loss of Control Rod 

Drive Pumps / 1

295029 (EPE 6) High Suppression Pool 

Water Level / 5

295032 (EPE 9) High Secondary 

Containment Area Temperature / 5

295033 (EPE 10) High Secondary 

Containment Area Radiation Levels 

295034 (EPE 11) Secondary 

Containment Ventilation High Radiation 

295035 (EPE 12) Secondary 

Containment High Differential Pressure 

295036 (EPE 13) Secondary 

Containment High Sump/Area Water 

 

500000 (EPE 16) High Containment 

Hydrogen

K/A Category Point Totals: Group Point Total: 6/3
ES-4.1, Page 10 of 28 

Form 4.1-BWR BWR Examination Outline Page 4 

Plant Systems—Tier 2/Group 1 (RO/SRO)

System # / Name K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G K/A Topic(s) IR # 

203000 (SF2, SF4 RHR/LPCI) 

RHR/LPCI: Injection Mode

205000 (SF4 SCS) Shutdown Cooling

206000 (SF2, SF4 HPCI) 

High-Pressure Coolant Injection

207000 (SF4 IC) Isolation 

(Emergency) Condenser

209001 (SF2, SF4 LPCS) 

Low-Pressure Core Spray

209002 (SF2, SF4 HPCS) 

High-Pressure Core Spray

211000 (SF1 SLCS) Standby Liquid 

Control

212000 (SF7 RPS) Reactor Protection

215003 (SF7 IRM) 

Intermediate-Range Monitor

215004 (SF7 SRMS) Source-Range 

Monitor

215005 (SF7 PRMS) Average Power 

Range Monitor/Local Power Range 

Monitor

217000 (SF2, SF4 RCIC) Reactor 

Core Isolation Cooling

218000 (SF3 ADS) Automatic 

Depressurization

223002 (SF5 PCIS) Primary 

Containment Isolation/Nuclear Steam 

Supply Shutoff

239002 (SF3 SRV) Safety Relief 

Valves

259002 (SF2 RWLCS) Reactor Water 

Level Control

261000 (SF9 SGTS) Standby Gas 

Treatment

262001 (SF6 AC) AC Electrical 

Distribution

262002 (SF6 UPS) Uninterruptable 

Power Supply (AC/DC)

263000 (SF6 DC) DC Electrical 

Distribution

264000 (SF6 EGE) Emergency 

Generators (Diesel/Jet)

300000 (SF8 IA) Instrument Air 

400000 (SF8 CCW) 

Component Cooling Water

510000 (SF4 SWS) Service Water 

(Normal and Emergency)

K/A Category Point Totals: Group Point Total: 26/5 
ES-4.1, Page 11 of 28 

Form 4.1-BWR BWR Examination Outline Page 5 

Plant Systems—Tier 2/Group 2 (RO/SRO)

System # / Name K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G K/A Topic(s) IR #

201001 (SF1 CRDH) CRD Hydraulic

201002 (SF1 RMCS) Reactor Manual Control

201003 (SF1 CRDM) Control Rod and 

Drive Mechanism

201004 (SF7 RSCS) Rod Sequence Control

201005 (SF1, SF7 RCIS) Rod Control and 

Information

201006 (SF7 RWMS) Rod Worth Minimizer

202001 (SF1, SF4 RS) Recirculation

202002 (SF1 RSCTL) Recirculation Flow Control

204000 (SF2 RWCU) Reactor Water Cleanup

214000 (SF7 RPIS) Rod Position Information

215001 (SF7 TIP) Traversing In-Core Probe

215002 (SF7 RBMS) Rod Block Monitor

216000 (SF7 NBI) Nuclear Boiler Instrumentation

219000 (SF5 RHR SPC) RHR/LPCI:

Torus/Suppression Pool Cooling Mode

223001 (SF5 PCS) Primary Containment 

and Auxiliaries

226001 (SF5 RHR CSS) RHR/LPCI: Containment

Spray Mode

230000 (SF5 RHR SPS) RHR/LPCI:

Torus/Suppression Pool Spray Mode

233000 (SF9 FPCCU) Fuel Pool Cooling/Cleanup

234000 (SF8 FH) Fuel Handling Equipment

239001 (SF3, SF4 MRSS) Main and Reheat Steam

239003 (SF9 MSIVLC) Main Steam Isolation Valve 

Leakage Control

241000 (SF3 RTPRS) Reactor/Turbine Pressure 

Regulating

245000 (SF4 MTGEN) Main 

Turbine Generator/Auxiliary

256000 (SF2 CDS) Condensate

259001 (SF2 FWS) Feedwater

268000 (SF9 RW) Radwaste

271000 (SF9 OG) Offgas

272000 (SF7, SF9 RMS) Radiation Monitoring

286000 (SF8 FPS) Fire Protection

288000 (SF9 PVS) Plant Ventilation

290001 (SF5 SC) Secondary Containment

290003 (SF9 CRV) Control Room Ventilation

290002 (SF4 RVI) Reactor Vessel Internals

51001 (SF8 CWS) Circulating Water

K/A Category Point Totals: Group Point Total: 11/3
ES-4.1, Page 12 of 28 

Form 4.1-PWR Pressurized-Water Reactor Examination Outline

Note: CO = Conduct of Operations; EC = Equipment Control; RC = Radiation Control;

 EM = Emergency Procedures/Plan

Facility: Date of Exam:

Tier Group RO K/A Category Points SRO-Only Points

K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G Total A2 G Total

1.

Emergency 

and 

Abnormal 

Plant 

Evolutions

1 

N/A N/A

18 6 

2 8 4 

Tier 

Totals 26 10

2.

Plant

Systems

1 28 5 

2 9 3

Tier 

Totals 37 8 

3.

Generic 

Knowledge 

and Abilities 

Categories

CO EC RC EM

6 

CO EC RC EM

7 2 2 1 1 

4. Theory

Reactor Theory Thermodynamics

6 3 3 
ES-4.1, Page 13 of 28 

Form 4.1-PWR PWR Examination Outline Page 2

Emergency and Abnormal Plant Evolutions—Tier 1/Group 1 (RO/SRO) 

E/APE # / Name / Safety Function K1 K2 K3 A1 A2 G K/A Topic(s) IR #

000007 (EPE 7; BW E02&E10; CE E02) 

Reactor Trip, Stabilization, Recovery / 1

000008 (APE 8) Pressurizer Vapor Space 

Accident / 3

000009 (EPE 9) Small-Break LOCA / 3

000011 (EPE 11) Large-Break LOCA / 3

000015 (APE 15) Reactor Coolant Pump 

Malfunctions / 4

000022 (APE 22) Loss of Reactor Coolant 

Makeup / 2

000025 (APE 25) Loss of Residual Heat 

Removal System / 4

000026 (APE 26) Loss of Component 

Cooling Water / 8

000027 (APE 27) Pressurizer Pressure 

Control System Malfunction / 3

000029 (EPE 29) Anticipated Transient 

Without Scram / 1

000038 (EPE 38) Steam Generator Tube 

Rupture / 3

000040 (APE 40; BW E05; CE E05; W E12)

Steam Line Rupture—Excessive Heat 

Transfer / 4

000054 (APE 54; CE E06) Loss of Main 

Feedwater /4

000055 (EPE 55) Station Blackout / 6

000056 (APE 56) Loss of Offsite Power / 6

000057 (APE 57) Loss of Vital AC 

Instrument Bus / 6

000058 (APE 58) Loss of DC Power / 6

000062 (APE 62) Loss of Service Water / 

4

000065 (APE 65) Loss of Instrument Air / 8

000077 (APE 77) Generator Voltage and 

Electric Grid Disturbances / 6

(W E04) LOCA Outside Containment / 3

(W E11) Loss of Emergency Coolant 

Recirculation / 4

(BW E04; W E05) Inadequate Heat 

Transfer—Loss of Secondary Heat Sink / 4

K/A Category Totals: Group Point Total: 18/6
ES-4.1, Page 14 of 28 

Form 4.1-PWR PWR Examination Outline Page 3 

Emergency and Abnormal Plant Evolutions—Tier 1/Group 2 (RO/SRO) 

E/APE # / Name / Safety Function K1 K2 K3 A1 A2 G K/A Topic(s) IR #

000001 (APE 1) Continuous Rod Withdrawal / 1

000003 (APE 3) Dropped Control Rod / 1 

000005 (APE 5) Inoperable/Stuck Control Rod / 1

000024 (APE 24) Emergency Boration / 1 

000028 (APE 28) Pressurizer Level Control 

Malfunction / 2

000032 (APE 32) Loss of Source Range Nuclear 

Instrumentation / 7

000033 (APE 33) Loss of Intermediate Range Nuclear 

Instrumentation / 7

000036 (APE 36; BW/A08) Fuel Handling Incidents / 8

000037 (APE 37) Steam Generator Tube Leak / 3

000051 (APE 51) Loss of Condenser Vacuum / 4

000059 (APE 59) Accidental Liquid Radwaste Release / 9

000060 (APE 60) Accidental Gaseous Radwaste Release / 9

000061 (APE 61) Area Radiation Monitoring System Alarms 

/ 7

000067 (APE 67) Plant Fire On Site / 8 

000068 (APE 68; BW A06) Control Room Evacuation / 8

000069 (APE 69; W E14) Loss of Containment Integrity / 5 

000074 (EPE 74; W E06 & E07) Inadequate Core Cooling / 

4

000076 (APE 76) High Reactor Coolant Activity / 9

000078 (APE 78*) RCS Leak / 3

(W E01 & E02) Rediagnosis & SI Termination / 3

(W E13) Steam Generator Overpressure / 4

(W E15) Containment Flooding / 5

(W E16) High Containment Radiation /9

(BW A01) Plant Runback / 1

(BW A02 & A03) Loss of NNI-X/Y/7

(BW A04) Turbine Trip / 4

(BW A05) Emergency Diesel Actuation / 6

(BW A07) Flooding / 8 

(BW E03) Inadequate Subcooling Margin / 4

(BW E08; W E03) LOCA Cooldown—Depressurization / 4

(BW E09; W E09 & E10) Natural Circulation/4 

(BW E13 & E14) EOP Rules and Enclosures

(W E08) RCS Overcooling—Pressurized Thermal Shock / 4 

(CE A16) Excess RCS Leakage / 2

(CE E09) Functional Recovery 

(CE E13) Loss of Forced Circulation/LOOP/Blackout / 4 

K/A Category Point Totals: Group Point Total: 8/4
ES-4.1, Page 15 of 28 

Form 4.1-PWR PWR Examination Outline Page 4 

Plant Systems—Tier 2/Group 1 (RO/SRO)

System # / Name K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G K/A Topic(s) IR #

003 (SF4P RCP) Reactor Coolant 

Pump

004 (SF1; SF2 CVCS) Chemical and 

Volume Control

005 (SF4P RHR) Residual Heat 

Removal

006 (SF2; SF3 ECCS) Emergency 

Core Cooling

007 (SF5 PRTS) Pressurizer 

Relief/Quench Tank

008 (SF8 CCW) Component Cooling 

Water

010 (SF3 PZR PCS) Pressurizer 

Pressure Control

012 (SF7 RPS) Reactor Protection

013 (SF2 ESFAS) Engineered 

Safety Features Actuation

022 (SF5 CCS) Containment Cooling

025 (SF5 ICE) Ice Condenser

026 (SF5 CSS) Containment Spray

039 (SF4S MSS) Main and Reheat 

Steam

059 (SF4S MFW) Main Feedwater

061 (SF4S AFW) 

Auxiliary/Emergency Feedwater

062 (SF6 ED AC) AC Electrical 

Distribution

063 (SF6 ED DC) DC Electrical 

Distribution

064 (SF6 EDG) Emergency Diesel 

Generator

073 (SF7 PRM) Process Radiation 

Monitoring

076 (SF4S SW) Service Water

078 (SF8 IAS) Instrument Air

103 (SF5 CNT) Containment

053 (SF1; SF4P ICS) Integrated 

Control

K/A Category Point Totals: Group Point Total: 28/5
ES-4.1, Page 16 of 28 

Form 4.1-PWR PWR Examination Outline Page 5 

Plant Systems—Tier 2/Group 2 (RO/SRO)

System # / Name K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G K/A 

( )

IR # 

001 (SF1 CRDS) Control Rod Drive

002 (SF2; SF4P RCS) Reactor 

Coolant 

011 (SF2 PZR LCS) 

Pressurizer Level Control 

014 (SF1 RPI) Rod 

Position Indication 

015 (SF7 NI) 

Nuclear 

Instrumentation

016 (SF7 NNI) 

Nonnuclear 

Instrumentation

017 (SF7 ITM) In-Core 

Temperature Monitor 

027 (SF5 CIRS) Containment 

Iodine Removal 

028 (SF5 HRPS) Hydrogen 

Recombiner and Purge 

Control

029 (SF8 CPS) Containment Purge 

033 (SF8 SFPCS) Spent Fuel 

Pool Cooling 

034 (SF8 FHS) Fuel 

Handling Equipment 

035 (SF 4P SG) Steam Generator 

041 (SF4S SDS) Steam 

Dump/Turbine Bypass Control 

045 (SF 4S MT/G) Main 

Turbine Generator

055 (SF4S CARS) Condenser 

Air Removal 

056 (SF4S CDS) Condensate 

068 (SF9 LRS) Liquid Radwaste 

071 (SF9 WGS) Waste 

Gas Disposal 

072 (SF7 ARM) Area 

Radiation Monitoring 

075 (SF8 CW) Circulating Water 

086 (SF8 FP) Fire Protection 

050 (SF 9 CRV) Control Room 

Ventilation 

K/A Category Point Totals: Group Point Total: 9/3 
ES-4.1, Page 17 of 28 

Form 4.1-AP AP1000 Examination Outline

Note: CO = Conduct of Operations; EC = Equipment Control; RC = Radiation Control;

 EM = Emergency Procedures/Plan

Facility: Date of Exam:

Tier Group

RO K/A Category Points SRO-Only Points

K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G Total A2 G Total

1.

Emergency 

and 

Abnormal 

Plant 

Evolutions

1 

N/A N/A

18 6 

2 8 4 

Tier 

Totals 26 10

2.

Plant

Systems

1 28 5 

2 9 3 

Tier 

Totals 37 8 

3.

Generic 

Knowledge 

and Abilities 

Categories

CO EC RC EM

6 

CO EC RC EM

7 2 2 1 1 

4. Theory

Reactor Theory Thermodynamics

6 3 3 
ES-4.1, Page 18 of 28 

Form 4.1-AP AP1000 Examination Outline Page 2 

Emergency and Abnormal Plant Evolutions—Tier 1/Group 1 (RO/SRO)

E/APE # / Name / Safety Function K

1

K

2

K

3

A

1

A

2 G K/A Topic(s) IR # 

E-0, Reactor Trip or Safeguards

Actuation / 1, 2, 3, 4

ES-0.1, Reactor Trip Response / 1, 2, 

3, 4 

ES-1.3, ADS Stage 1–3 Actuation 

Response / 3 

ES-1.4, ADS Stage 4 Actuation 

Response / 3 

A-313, Uncontrolled Cooldown / 4

A-336, Malfunction of Protection and

Safety Monitoring System / 7

E-1, Loss-of-Coolant Accident / 2, 3

A-342, Reactor Coolant Pump

Malfunction / 1, 2, 3, 4

A-337, Passive RHR Heat Exchanger

Leak / 4

A-343, Loss of Normal Residual Heat

Removal / 4

A-317, Loss of Component Cooling

Water / 8

ES-0.2, Natural Circulation Cooldown 

/ 4 

FR-S.1, Response to Nuclear Power 

Generation / 1 

E-3, Steam Generator Tube Rupture /

3

E-2, Faulted Steam Generator

Isolation / 4

A-301, Rapid Power Reduction / 1

A-307, DAS Operations at Local

Cabinets / 7

FR-C.1, Response to Inadequate 

Core Cooling / 4 

A-323, Loss of 6.9-kV, 4,160-V, or

480-V Bus Power / 6

ES-1.1, Passive Safety System 

Termination / 3 

A-345, Loss of Nuclear Service Water

/ 4

A-329, Loss of Instrument Air / 8

ECA-1.1, Loss-of-Coolant Accident 

Outside Containment / 3 

FR-H.1, Response to Loss of Heat 

Sink / 4 

SDP-1, Response to Loss of RCS 

Inventory During Shutdown / 2 

SDP-2 Response to Loss of RNS 

During Shutdown / 4 

K/A Category Totals: Group Point Total: 18/6
ES-4.1, Page 19 of 28 

Form 4.1-AP AP1000 Examination Outline Page 3

Emergency and Abnormal Plant Evolutions—Tier 1/Group 2 (RO/SRO)

E/APE # / Name / Safety Function K

1

K

2

K

3

A

1

A

2 G K/A Topic(s) IR # 

A-311, Rod Control System Malfunction / 1

A-308, Loss of Control Room AC / 8

A-320, Loss of Circulating Water / 8

A-302, Emergency Boration / 1

A-327, Startup Feedwater System Malfunction /

4

A-328, Malfunction of Feedwater Heaters and

Extraction Steam / 4

FR-I.1 Response to High Pressurizer Level / 2 

A-314, Fuel Handling Incident / 8

A-304, Steam Generator Tube Leak / 3

A-333, Main Turbine Malfunction / 4

FR-Z.1, Response to High Containment 

Pressure / 5

SDP-4, Response to Rising Nuclear Flux During 

Shutdown / 1

SDP-5, Response to RCS Cold Overpressure 

During Shutdown / 3

SDP-6 Response to Unexpected RCS 

Temperature Changes During Shutdown / 4

A-306, Evacuation of Control Room / 8

A-318, Condensate System Malfunctions / 4

FR-C-2, Response to Degraded Core Cooling / 4 

FR-C.3, Response to Saturated Core Cooling / 4 

FR-H.2, Response to Steam Generator 

Overpressure / 4

FR-Z.2, Response to Containment Flooding / 5 

FR-Z.3, Response to High Containment 

Radiation / 9 

FR-Z.4, Response to Low Containment Pressure 

/ 5

A-332, Turbine Trip Without Reactor Trip / 4

ES-1.2, Post-LOCA Cooldown and 

Depressurization / 4

A-321, Loss of Data Display and Processing

System / 7

FR-P.1, Response to Imminent Pressurized 

Thermal Shock Condition / 3

A-340, Reactor Coolant Leak / 2

FR-1.2, Response to Low Pressurized Level / 2 

FR-1.3, Response to Voids in Reactor Vessel / 2 

A-326, Feedwater System Malfunctions / 4

A-303, Loss of Plant DC Electric Power or AC

Instrument Power / 6

A-348, Degraded Grid / 6

K/A Category Totals: Group Point Total: 8/4 
ES-4.1, Page 20 of 28 

Form 4.1-AP AP1000 Examination Outline Page 4

Plant Systems—Tier 2/Group 1 (RO/SRO)

System Name / Safety Function K

1

K

2

K

3

K

4

K

5

K

6

A

1

A

2

A

3

A

4 G K/A Topic(s) IR # 

Reactor Coolant / 2, 4 

Steam Generator / 4 

Normal Residual Heat Removal / 

4 

Passive Residual Heat Removal /

4 

Passive Core Cooling / 2 

Component Cooling Water / 8 

Pressurizer Pressure Control / 3 

Automatic Depressurization / 3 

Reactor Trip System / 7 

Engineered Safeguards Actuation 

/ 2

Diverse Actuation / 7 

Passive Containment Cooling / 5 

Main Steam / 4 

Main and Startup Feedwater / 4 

AC Electrical Distribution / 6 

Class 1E and Non 1E DC and 

UPS / 6

Onsite Standby Power System / 6 

Service Water / 4 

Compressed Air / 8 

Containment System / 5 

Reactor Coolant Pump / 4 

Chemical and Volume Control / 1, 

2

K/A Category Point Totals: Group Point Total: 28/5 
ES-4.1, Page 21 of 28 

Form 4.1-AP AP1000 Examination Outline Page 5

Plant Systems—Tier 2/Group 2 (RO/SRO)

System Name / Safety Function K

1

K

2

K

3

K

4

K

5

K

6

A

1

A

2

A

3

A

4 G K/A Topic(s) IR # 

Digital Rod Control / 1 

Pressurizer Level Control / 2 

Rod Position Indication / 1 

In-Core Instrument System / 7 

Containment Air Filtration / 8 

Containment Hydrogen Control / 

5 

Main Control Room HVAC / 8 

Spent Fuel Pool Cooling / 8 

Condensate / 4 

Condenser Air Removal / 4 

Main Turbine and Main Turbine 

Control / 4 

Fuel Handling / 8 

Gaseous Radwaste / 9 

Radiation Monitoring / 7 

Circulating Water / 8 

Fire Protection / 8 

Steam Dump Control System / 4 

Nuclear Instrumentation System / 

7 

Liquid Radwaste System / 9 

K/A Category Point Totals: Group Point Total: 9/3 
ES-4.1, Page 22 of 28 

Form 4.1-ABWR Advanced Boiling-Water Reactor Examination Outline

Note: CO = Conduct of Operations; EC = Equipment Control; RC = Radiation Control;

 EM = Emergency Procedures/Plan

Facility: Date of Exam:

Tier Group RO K/A Category Points SRO-Only Points

K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G Total A2 G Total

1.

Emergency 

and 

Abnormal 

Plant 

Evolutions

1 

N/A N/A

20 7 

2 6 3 

Tier 

Totals 26 10

2.

Plant

Systems

1 26 5 

2 11 3

Tier 

Totals 37 8 

3.

Generic 

Knowledge 

and Abilities 

Categories

CO EC RC EM

6 

CO EC RC EM

7 2 2 1 1 

4. Theory

Reactor Theory Thermodynamics

6 3 3 
ES-4.1, Page 23 of 28 

Form 4.1-ABWR ABWR Examination Outline Page 2 

Emergency and Abnormal Plant Evolutions—Tier 1/Group 1 (RO/SRO) 

E/APE # / Name / Safety Function K

1

K

2

K

3

A

1

A

2 G K/A Topic(s) IR # 

APE2001 Partial or Complete Loss of 

Forced Core Flow Circulation / 1 & 4

APE2003 Partial or Complete Loss of AC / 6 

APE2004 Partial or Total Loss of DC Power 

/ 6

APE2005 Main Turbine Generator Trip / 3 

APE2006 Reactor Scram / 1 

APE2015 Control Room Evacuation / 7 

APE2017 Partial or Total Loss of CCW 

Reactor Building Cooling Water / 8

APE2018 Partial or Total Loss of Instrument 

Air / 8

APE2020 Loss of Shutdown Cooling / 4 

APE2023 Plant Fire Onsite / 8 

APE2024 Generator Voltage and Electric 

Grid Disturbances / 6

APE2022 Refueling Accidents / 8 

EPE1001 High Drywell Pressure / 5 

EPE1002 High Reactor Pressure / 3 

EPE1003 Suppression Pool High Water 

Temperature / 5

EPE1004 High Drywell Temperature / 5 

EPE1006 Low Suppression Pool Water 

Level / 5

EPE1007 Reactor Low Water Level / 2 

EPE1008 High Secondary Containment 

Area Temperature / 5

EPE1009 High Secondary Containment 

Area Radiation Levels / 9

EPE1010 Reactor Building HVAC Exhaust 

High Radiation / 9

EPE1013 Scram Condition Present and 

Reactor Power above APRM Downscale or 

Unknown / 1

EPE1014 High Offsite Release Rate / 9 

K/A Category Totals: Group Point Total: 20/7 
ES-4.1, Page 24 of 28 

Form 4.1-ABWR ABWR Examination Outline Page 3

Emergency and Abnormal Plant Evolutions—Tier 1/Group 2 (RO/SRO)

E/APE # / Name / Safety Function K

1

K

2 

K

3

A

1 

A

2 G K/A Topic(s) IR # 

APE2002 Loss of Main Condenser 

Vacuum / 3 

APE2007 High Reactor Pressure / 3 

APE2008 High Reactor Water Level / 2 

APE2009 Low Reactor Water Level / 2 

APE2010 High Drywell Pressure / 5 

APE2011 High Drywell Temperature / 5 

APE2012 High Suppression Pool 

Temperature / 5 

APE2013 Inadvertent Reactivity Addition / 

1 

APE2014 Incomplete Scram / 1 

APE2016 High Offsite Release Rate / 9 

APE2019 Inadvertent Containment 

Isolation / 5 & 7 

APE2021 Loss of CRD Pumps / 1 

EPE1005 High Suppression Pool 

Water Level / 5 

EPE1011 Secondary Containment High 

Differential Pressure / 5 

EPE1012 Secondary Containment High 

Floor Drain Sump/Area Water Level / 5 

K/A Category Point Totals: Group Point Total: 6/3 
ES-4.1, Page 25 of 28 

Form 4.1-ABWR ABWR Examination Outline Page 4

Plant Systems—Tier 2/Group 1 (RO/SRO)

System # / Name K

1

K

2

K

3

K

4

K

5

K

6

A

1

A

2

A

3

A

4 G K/A Topic(s) IR # 

SF1RRS & SF4RRS Reactor 

Recirculation System

SF1RFC Recirculation Flow 

Control

SF1SLC Standby Liquid Control

SF2RHRLPFL RHR: 

Low-Pressure Flooder Mode

SF2HPCF High-Pressure Core 

Flooder

SF2RCIC & SF4RCIC Reactor 

Core Isolation Cooling

SF2FWC Feedwater Control

SF3ADS Automatic 

Depressurization System

SF3SRV Safety Relief Valves

SF4RHRSDC RHR: Shutdown 

Cooling Mode

SF5RHRSPC RHR: Suppression 

Pool Cooling Mode

SF5RHRSPR RHR: 

Drywell/Wetwell Spray Mode

SF5LDIS Leak Detection and 

Isolation System

SF6EPDS AC Electrical 

Distribution

SF6VAC Vital AC Power Supply

SF6DC Direct Current Power 

Supply

SF6DGCTG Emergency 

Generators (Diesel/CTG)

SF7RTIS Reactor Trip and 

Isolation System 

SF7SRNM Startup Range 

Neutron Monitor 

SF7ELCS ESF Logic and Control 

System

SF7APRM Average Power Range 

Monitor/Local Power Range 

Monitor

SF8IAS Instrument Air

SF8RBCW Reactor Building 

Cooling Water

SF8RSW Reactor Service Water

SF9SGTS Standby Gas 

Treatment System

K/A Category Point Totals: Group Point Total: 26/5
ES-4.1, Page 26 of 28 

Form 4.1-ABWR ABWR Examination Outline Page 5

Plant Systems—Tier 2/Group 2 (RO/SRO)

System # / Name K

1

K

2

K

3

K

4

K

5

K

6

A

1

A

2

A

3

A

4 G K/A Topic(s) IR #

SF1CRD Control Rod Drive

SF1FMCRD Fine Motion Control 

Rod Drive Mechanism

SF1RCIS Rod Control and 

Information System

SF2RWCU Reactor Water Cleanup

SF2CD Condensate

SF2FW Feedwater

SF2AFI Alternate Feedwater 

Injection

SF3NBS & SF4NBS Main and 

Reheat Steam

SF3EHC Steam Bypass and 

Pressure Control/Turbine Pressure 

Control

SF4MT Main Turbine Generator 

and Auxiliaries

SF5SEC Secondary Containment

SF5PCS Primary Containment and 

Auxiliaries

SF5RPV & SF9RPV Reactor Vessel 

Internals

SF6I&C Instrumentation and 

Control Power Supply

SF7ATIP Automated Traversing 

In-Core Probe

SF7MRBM Multichannel Rod Block 

Monitor

SF7NBI Nuclear Boiler 

Instrumentation 

SF7PICS Plant Information and 

Control System

SF7SPTM Suppression Pool 

Temperature Monitoring

SF7RSS Remote Shutdown System

SF7APR Automatic Power 

Regulator

SF7ATLM Automated Thermal Limit 

Monitor

SF7RMS & SF9RMS Radiation 

Monitoring

SF8FPS Fire Protection

SF8FH Fuel Handling 

SF9FPC Fuel Pool Cooling/Cleanup

SF9RD Radwaste

SF90G Offgas

SF9CRHVAC Control Room 

Habitability Area HVAC

SF9HVAC Plant Ventilation 

Systems

K/A Category Point Totals: Group Point Total: 11/3
ES-4.1, Page 27 of 28 

Form 4.1-COMMON Common Examination Outline

Facility: Date of Exam:

Generic Knowledge and Abilities—Tier 3 (RO/SRO)

Category K/A # Topic RO SRO-Only

IR # IR #

1.

Conduct of 

Operations

2.1.

2.1.

2.1.

2.1.

Subtotal N/A N/A

2.

Equipment 

Control

2.2.

2.2.

2.2.

2.2.

Subtotal N/A N/A

3.

Radiation Control

2.3.

2.3.

2.3.

Subtotal N/A N/A

4.

Emergency 

Procedures/

Plan

2.4.

2.4.

2.4.

Subtotal N/A N/A

Tier 3 Point Total 6 7 

Theory—Tier 4 (RO)

Category K/A # Topic

RO

IR #

Reactor Theory

6

6

6

Subtotal N/A

Thermodynamics

6

6

6

Subtotal N/A

Tier 4 Point Total 6 
ES-4.1, Page 28 of 28 

Form 4.1-1 Record of Rejected Knowledge and Abilities

Refer to Examination Standard (ES)-4.2, “Developing Written Examinations,” Section B.3, for 

deviations from the approved written examination outline.

Tier/Group Randomly 

Selected K/A

Reason for Rejection
ES-4.2, Page 1 of 29

1 4.2 DEVELOPING WRITTEN EXAMINATIONS

2 This examination standard contains the instructions and guidelines for developing written 

3 examination questions and assembling the examination for the written examination portion of 

4 the NRC operator licensing initial examinations for RO and SRO license applicants at power 

5 reactor facilities. Preparing written examinations using the instructions and guidelines in this 

6 examination standard, in conjunction with the applicable K/A catalog and the approved written 

7 examination outline (ES-4.1, “Preparing Written Examination Outlines”), ensures that the written 

8 examination includes a representative sample of the items specified in the regulations in 

9 10 CFR Part 55.

10

11 A. Responsibilities

12 1. Facility Licensee

13 The facility licensee performs the following activities, as applicable, depending on the 

14 examination arrangements confirmed with the NRC regional office in accordance with the 

15 instructions in ES-2.1 before the scheduled examination date:

16

17 a. Submit the reference materials necessary for the NRC regional office to prepare or 

18 review the written examination(s) in accordance with ES-2.1.

19

20 b. Prepare the proposed written examination(s) in accordance with the instructions in 

21 Section B of this examination standard.

22

23 c. (Optional) Submit up to 10 written examination questions to the NRC chief examiner as 

24 part of the presubmittal sample for preliminary review and comment.

25

26 d. Review the proposed written examination(s) in accordance with ES-2.3.

27

28 e. Submit the proposed written examination(s) to the NRC regional office for review and 

29 approval.

30

31 f. Submit the final written examination to the NRC.

32

33 g. Assemble the final, approved written examinations:

34

35 1. Format the examinations using a one-question-per-page layout by placing one 

36 complete question on each page, if possible.

37

38 2. Use a cover sheet in the format shown in Form ES-4.2-3, “Reactor Operator

39 Written Examination Cover Sheet,” or Form ES-4.2-4, “Senior Reactor Operator

40 Written Examination Cover Sheet,” as applicable, for all RO and SRO written 

41 examinations. Fill out all items in the upper section of the cover sheet, except 

42 the name of the applicant, when assembling the examinations.

43 2. NRC Regional Office

44 The NRC regional office performs the following activities:

45

46 a. Verify that the questions are prepared in accordance with this examination standard. 
ES-4.2, Page 2 of 29

1 b. Review the proposed written examination(s) in accordance with ES-2.3.

2 c. Approve the finalized written examination(s) in accordance with ES-2.3.

3

4 B. Instructions for Preparing Written Examinations

5 The examination author should use a combination of existing, modified, and new questions that 

6 match the specific K/A statements in the approved written examination outline and the criteria 

7 listed in this section to prepare the written examination. 

8

9 1. License Level Criteria

10 a. In order to maintain examination quality and consistency, 50–60 percent of the questions 

11 on the RO written examination (38–45 questions) and at least 50 percent of the 

12 questions on the SRO-only portion of the written examination (13 questions) shall be 

13 written at the comprehension/analysis level. 

14

15 b. A question at the RO level should test 1 (or more) of the 14 items listed under 

16 10 CFR 55.41(b) that the K/A is linked to, or it should test at a RO level as determined 

17 from the facility licensee’s learning objectives. An SRO-level question should test one 

18 (or more) of the seven items listed under 10 CFR 55.43(b) that the K/A is linked to, or it 

19 should test at a level that is unique to the SRO job position as determined from the 

20 facility licensee’s learning objectives. 

21

22 c. Additionally, the author should use the instructions and guidance for developing and 

23 reviewing SRO-level questions in Section E of this examination standard.

24

25 2. Specific Criteria for Tier 1 and Tier 3 Questions

26 a. Ensure that Tier 1 questions test an applicant’s knowledge of how to safely operate the 

27 plant during emergency and abnormal conditions. Satisfactory ways of achieving this 

28 objective include, but are not limited to, (1) information contained in the site’s 

29 procedures, including alarm response procedures, abnormal operating procedures 

30 (AOPs), emergency operating procedures (EOPs), and their associated bases 

31 documents, (2) diagnosis that leads to selection of the procedures that should be used 

32 to respond to the evolution, (3) the progression of an event, and (4) the assessment of 

33 the integrated plant response to emergency or abnormal situations crossing several 

34 plant systems or safety functions, or both.

35

36 b. Ensure that Tier 3 questions maintain their focus on plantwide generic K/As and not 

37 system-specific knowledge or knowledge of specific emergency/abnormal plant 

38 evolutions. 

39 3. Deviations from the Approved Written Examination Outline

40 When selecting and developing questions, it may be necessary to deviate from the previously 

41 approved written examination outline. The following are common examples of reasons to 

42 deviate:

43 • The subject K/A is not relevant at the subject facility.

44
ES-4.2, Page 3 of 29

1 • It is not possible (without expending an unreasonable amount of resources) to prepare a 

2 psychometrically sound question related to the subject K/A.

3

4 • It is not possible to prepare a question at the correct license level related to the subject 

5 K/A.

6

7 • The subject K/A is more appropriately tested on the operating test than on the written 

8 examination. (A K/A only associated with an “ability” is not a sufficient reason to reject 

9 the K/A from the written examination. Therefore, Form 4.1-1 should be used to 

10 document why the operating test is a better evaluation tool.)

11

12 • It is not possible to prepare a question that is appropriate for its assigned tier (e.g., for 

13 K/As selected for Tier 3 questions, the subject K/A does not lend itself to developing a 

14 question that focuses on plantwide, generic concepts). 

15 The fact that a K/A does not have a corresponding facility learning objective, was not covered in 

16 training, or is subject to selection in multiple tiers is not sufficient to eliminate the K/A from any 

17 tier of the outline.

18

19 Any time it becomes necessary to deviate from the previously approved written examination 

20 outline, the author should randomly select a replacement K/A and document the eliminated K/A 

21 statements on Form 4.1-1, or equivalent, with an explanation of why the K/A was rejected and 

22 why the proposed replacement is an acceptable substitute. 

23

24 4. Limits on Use of Bank Questions and Criteria for New Questions and Significantly 

25 Modified Bank Questions

26 A bank question is a written examination question taken from any facility licensee collection of 

27 questions that have previously appeared on any operator training-related examination at the 

28 facility. This definition includes NRC examination questions used at other facility licensee sites.

29 To avoid compromising the integrity and security of the examination and to enhance 

30 consistency, authors should observe the following limits on using bank questions when 

31 preparing the written examination:

32

33 • Limit the use of bank questions to no more than 75 percent of the questions for the 

34 examination (56 questions for the RO examination and 19 questions for the SRO-only

35 portion). A bank question may have to be edited or replaced if it conflicts with another 

36 question on the examination or if it is necessary to meet the criteria on the written 

37 examination quality checklist (Form 2.3-4, “Written Examination Quality Checklist”). 

38 Correct technical and psychometric flaws that cause the question to have no or multiple 

39 correct answers regardless of when these flaws are identified. 

40

41 • If the bank contains more than one question that fits a specific K/A statement, randomly 

42 select from among the available questions unless there is an appropriate basis for 

43 selecting a specific question (e.g., a particular question has a higher cognitive level, has 

44 better discrimination validity, is more operationally oriented, or addresses site-specific 

45 priorities).

46

47 • Write at least 10 new questions (8 for the RO examination and 2 for the SRO-only

48 portion) at the comprehension/analysis level. Generally, if a question is created without 

49 referring to a bank question, it can be considered a “new” question.
ES-4.2, Page 4 of 29

1 • Select the remaining questions for the examination by significantly modifying bank 

2 questions so that an applicant does not recognize the question as a bank question. 

3 Significant modification can be met by any of the following:

4

5 – Change at least one pertinent condition in the stem and at least one distractor. 

6 OR

7 – Change the conditions in the stem such that one of the three distractors in the 

8 original question becomes the correct answer. 

9 OR

10 – The NRC chief examiner agrees that the bank question is significantly modified,

11 and an applicant would not be able to arrive at the correct answer because he or 

12 she recognizes the question from the bank.

13

14 • When significantly modifying a bank question, the intent or objective of the question 

15 does not have to change. Adding or deleting irrelevant information and making minor 

16 changes (e.g., the unit number, component train, or power level when it makes no 

17 difference) does not qualify as a significant modification to the question.

18

19 5. Length of the Written Examination

20 The applicants should be able to complete and review the RO examination within 4 hours and 

21 the SRO-only portion within 2 hours. (Refer to ES 4.3, “Administering Written Examinations,” for 

22 examination administration time limits, which are longer.)

23

24 6. Instructions for Documenting Questions and for Reference Material

25 The author should use Form 4.2-1, “Written Examination Question Worksheet,” or an equivalent 

26 form to document the required information that accompanies each proposed question submitted 

27 to the NRC for review and approval:

28

29 a. For every question, provide a technical reference, including the reference’s revision or 

30 version number (if applicable) and a cross-reference to the facility licensee’s 

31 examination question bank, if applicable. If the facility licensee has a learning objective 

32 applicable to the question, reference it as well. However, the absence of a learning 

33 objective does not invalidate the question if it has an appropriate K/A and technical 

34 reference. 

35

36 b. For every question, explain briefly why the answer is correct and why each distractor is 

37 plausible but incorrect. This increases the efficiency of the examination review process 

38 and promotes the detection and correction of invalid questions before the examinations 

39 are approved for administration.

40

41 c. For every question, provide applicable source information as follows: 

42

43 • new question

44 • bank question: Identify bank items that were used on an NRC licensing 

45 examination at the facility by indicating the examination location and year that it 

46 was administered. 
ES-4.2, Page 5 of 29

1 • modified bank question: Explain how the bank question was significantly 

2 modified or include a copy of the original bank question. 

3 d. The written examination for initial licensing is a closed-reference examination to test and 

4 evaluate the applicants’ initial mastery of the required knowledge, skills, and abilities. 

5 Reference materials, such as diagrams, sketches, mimics, pictures, and portions of 

6 facility procedures, may be used on a selective basis, either embedded within a specific 

7 question or as attachments to the written examination. Ensure that all the following 

8 criteria are met when using reference material in the examination:

9 • Material is easy to read and clearly marked.

10 • Material provides an effective and objective way for the applicant to demonstrate 

11 knowledge of the topic or concept.

12 • Material does not give away the answers to other questions on the examination 

13 or improve the applicant’s chances of guessing the correct answer by eliminating 

14 incorrect distractors.

15 • RO reference material cannot cue or give away an answer to an SRO question 

16 and vice versa. (In accordance with ES-4.3, the SRO and RO written 

17 examinations must be given at the same time.)

18 C. General Guidelines Applicable to All Written Examination Questions

19 The following guidelines are applicable to all written examination questions, including 

20 open-reference-style questions that are used on an NRC-developed requalification examination.

21 The guidelines are based in part on psychometrics—the process of applying sound qualitative 

22 processes to mental measurements. It is important to minimize the number of psychometric 

23 errors in NRC examinations because test items that are free of psychometric errors yield greater 

24 measurement validity.

25

26 Form 4.2-2, “Question Development Checklist,” summarizes these guidelines and can be used 

27 as a job aid during the process of developing and reviewing written examination questions. 

28 Appendix B, “Examples of Written Examination Questions,” provides examples that illustrate the

29 psychometric errors that should be avoided. Additionally, Appendix A, “Overview of Generic 

30 Examination Concepts,” contains an overview of generic concepts related to preparing and 

31 formatting content-valid written examinations and should be referred to as necessary while 

32 preparing the examination. 

33

34 Examination authors should follow these guidelines:

35

36 1. Ensure that the concept being measured has a direct, important relationship to the ability 

37 to perform the job. Although Appendix A stresses the importance of relevant K/As and 

38 testing objectives, it is equally important that the construction of the question itself clearly 

39 reflects the importance of the topic. 

40

41 It is not always necessary to establish a direct, word-for-word match between a question 

42 and a facility learning objective. A broadly stated learning objective may support any 

43 number of related questions.

44
ES-4.2, Page 6 of 29

1 Similarly, the absence of a facility learning objective does not preclude the development 

2 of a valid, K/A-based question. This is consistent with the concept of the NRC 

3 examination providing checks and balances on the facility licensee’s training program, 

4 thereby alerting the facility licensee that it may need to develop such a learning 

5 objective.

6

7 Although it is appropriate to develop questions regarding knowledge that is embedded 

8 in, or covered by, procedures, do not test knowledge that is trivial in nature.

9

10 2. Ensure that the question matches the intent of the K/A.

11

12 It is very easy to wind up with a question that tests a relatively trivial aspect of an 

13 important K/A topic. When reviewing the draft question, consider whether it is likely that 

14 someone could answer the question correctly and still not meet the objective or intent of 

15 the K/A or perform the responsibilities or tasks for which the K/A is needed.

16

17 To facilitate the translation of a K/A into a test question, consider the following prompts

18 to help generate ideas for potential test questions:

19

20 a. What are the common misconceptions about this topic?

21 b. Why is this topic important to satisfactory job performance?

22 c. Under which circumstances would it be important to understand this topic?

23 d. What might an individual do if he or she does not understand this topic?

24 e. What might be the consequences of a lack of knowledge about this topic?

25 f. How can the individual demonstrate his or her knowledge of this topic?

26

27 When selecting or writing questions for K/As that test coupled K/As (e.g., the A.2 K/A 

28 statements in Tiers 1 and 2 and a number of generic K/A statements, such as 2.4.1, in 

29 Tier 3), try to test both aspects of the K/A statement. If that is not possible without 

30 expending an inordinate amount of resources, limit the scope of the question to that 

31 aspect of the K/A statement requiring the highest cognitive level (e.g., the (b) portion of 

32 the A.2 K/A statements) or substitute another randomly selected K/A. 

33

34 3. State the question unambiguously, precisely, and as concisely as possible, but provide 

35 all necessary information.

36

37 Often the individuals who develop a question assume that certain stipulations or 

38 conditions are inherent in the question when, in fact, they are not. It is very difficult for 

39 the person who wrote a question to review it impartially or through the eyes of a new 

40 reader. Therefore, it is very important to have others review the questions to ensure that 

41 they include all necessary information and exclude all extraneous or superfluous 

42 information. For example, it is not necessary to provide a status for each annunciated 

43 parameter that is in its normal (nonalarming) state. ES-2.3 provides additional guidance 

44 on examination reviews, and ES-1.2, “Guidelines for Taking NRC Examinations,” gives 

45 the instructions to applicants on question clarity and assumptions.

46

47 However, as discussed in Appendix A, keep in mind that the key purpose of any test 

48 item is to assess important K/As at a level that distinguishes between safe and unsafe 

49 applicants. A test item’s ability to make that distinction is referred to as its discrimination 

50 validity. For a question to differentiate at the appropriate level, the test author must 

51 exercise judgment in establishing the initial conditions posed in the stem of the question. 
ES-4.2, Page 7 of 29

1 Providing too much information may “lead the applicant to the answer” and decrease the 

2 validity of the question because the answer is obvious to all applicants.

3

4 For closed-reference questions related to a specific plant procedure, it is generally 

5 desirable for the question to cite the number and title of the subject procedure, thereby 

6 limiting the possibility of an alternative correct answer if another procedure happens to 

7 relate to the same activity. For open-reference questions, use caution to ensure that the 

8 question does not become a direct lookup, with a pointer to help the applicant find the 

9 answer.

10

11 4. Write the question at the highest level of knowledge reflected in the testing objective.

12

13 One of the most challenging aspects of question development is capturing the 

14 appropriate level of knowledge. The reference benchmark that the NRC uses to classify 

15 the levels of knowledge of test items is Bloom’s taxonomy, a classification scheme that 

16 permits the grouping of items by the level (depth) of mental thought and performance 

17 required to answer the items. (Refer to Appendix B for references related to Bloom’s 

18 taxonomy.) Although Bloom’s taxonomy is most pertinent to written examination 

19 questions, it can also be applied to simulator scenarios and job performance measures. 

20 In ascending order, the three levels (depths) of mental thought and performance are the 

21 following (refer to Appendix B for examples of each level):

22

23 a. Level 1 (i.e., fundamental knowledge or simple memory) tests the recall or 

24 recognition of discrete bits of information. Examples include knowledge of 

25 terminology, definitions, set points, patterns, structures, procedural steps and 

26 cautions, and other specific facts.

27

28 b. Level 2 (i.e., comprehension) involves the mental process of understanding the 

29 material by relating it to its own parts or to some other material. Examples 

30 include rephrasing information in different words, describing or recognizing 

31 relationships, showing similarities and differences among parts or wholes, and 

32 recognizing how systems interact, including consequences or implications.

33

34 c. Level 3 (i.e., analysis, synthesis, or application) testing is a more active and 

35 product-oriented testing approach, which involves the multifaceted mental 

36 process of assembling, sorting, or integrating the parts (information bits and their 

37 relationships) to predict an event or outcome, solve a problem, or create 

38 something new. This level requires mentally using the knowledge and its 

39 meaning to solve problems.

40

41 Although test questions should be written to reflect the level of knowledge that is most 

42 appropriate for a specific K/A, the written examination should avoid high percentages of 

43 questions on fundamental knowledge. When choosing between two levels of 

44 knowledge, try to write the question to reflect the higher level. In general, test items at 

45 the comprehension and analysis levels are the most operationally oriented and, 

46 therefore, tend to be the most valid and discriminatory measure of operator knowledge 

47 and safe performance. Questions that require only memorization or recall are not 

48 acceptable for use on open-reference examinations. 

49
ES-4.2, Page 8 of 29

1 5. Establish a level of difficulty that distinguishes between applicants who have and have 

2 not mastered the required K/As and skills. Avoid unnecessarily difficult or irrelevant 

3 questions. 

4

5 As discussed conceptually in Appendix A, both the level of knowledge and the difficulty 

6 of an item are at the heart of examination discrimination. Authors should develop 

7 examinations that are estimated to center around the 80-percent cut score level, with 

8 individual item difficulty estimated to fall in the 70- to 90-percent difficulty range. (These 

9 parameters should not be viewed as precise benchmarks but rather as approximate end 

10 points.) Authors of written examinations should consider the results of past 

11 examinations when preparing a new one. Past performance on individual test questions 

12 may provide a basis for generating new questions and for estimating the level of difficulty 

13 of the examination. For example, questions that everyone answered incorrectly may 

14 indicate that the topic did not receive sufficient emphasis in training or that the item was 

15 poorly worded. Conversely, questions that everyone answered correctly may indicate 

16 that the item was written at too low a level or that the distractors were not very plausible.

17

18 Because item difficulty can usually be decreased or increased through revision, the 

19 examination author need not be overly preoccupied with difficulty when writing the items. 

20 Instead, the author should focus on achieving a valid measure of the concept he or she 

21 is attempting to evaluate.

22

23 When determining the appropriate level of difficulty, it may be helpful to think of two 

24 groups of individuals, one composed of experienced operators and the other of typical 

25 applicants, and evaluate the likelihood that each group of individuals will be able to 

26 answer the question. If at least 80 percent of the job incumbents or license applicants 

27 should be able to answer the question as written based on the expected knowledge 

28 levels for the position (RO or SRO), the item is likely written at an appropriate difficulty 

29 level. Authors and reviewers of written examinations may also consider the following 

30 factors in an effort to identify questions that are unnecessarily difficult or irrelevant:

31

32 a. Could someone do the job safely and effectively even without being able to 

33 answer the question? If so, is it because (1) the content is inappropriate, (2) the 

34 wording is unclear, or (3) the level of understanding is too great?

35

36 b. What aspects of the item or option might cause the most difficulty? Has the item 

37 been made artificially difficult? Can a person understand the principle being 

38 tested and still miss the item?

39

40 Estimates of difficulty made by the written examination author and reviewers may vary 

41 somewhat but should not vary widely. Unless there is some reason to doubt the 

42 estimates of some reviewers, the average estimate may be taken as a basis for 

43 assessing the suitability of item difficulty for the examination. Items should be revised if 

44 estimates fall well below or above the 70- to 90-percent target range.

45

46 Research has shown that when authors write test items in their own area of 

47 specialization, they tend to underestimate the difficulty of a concept or principle being 

48 tested. This tendency can manifest itself in two ways: (1) the author will view items of 

49 average difficulty as being easy or (2) in an effort to include plausible misleads among 

50 distractors in a multiple-choice test item, the author may make the item even more 
ES-4.2, Page 9 of 29

1 difficult. For this reason, an estimate of item difficulty made by the reviewers will 

2 probably be more accurate than one made by the author of the item.

3

4 Authors of written examinations should take care not to develop an examination with 

5 wide swings of individual item difficulty. For example, writing half the items at a 

6 60-percent difficulty level with the other half at a 100-percent difficulty level would yield 

7 an average of 80 percent; however, this approach has numerous flaws. The items at the 

8 100-percent level, by design, would be meaningless, since they would fail to discriminate 

9 at any level because the expectation is that nearly everyone would answer the question 

10 correctly. On the other hand, those written at the 60-percent difficulty level, by design, 

11 would also not discriminate and would likewise be unfair because 40 percent of the 

12 examinees would not be expected to answer those items correctly.

13

14 6. Limit the question to one concept or topic, unless the synthesis of the concepts is being 

15 tested. There is a common misconception that testing for multiple K/A topics in one 

16 question is a time-efficient way to examine. However, questions containing a variety of 

17 topics and issues only serve to confuse the examinee about the purpose of the question 

18 and what is expected in terms of a correct response. Develop each individual question 

19 to test the selected K/A topic and make that topic (as well as the intent of the question) 

20 clear to both the reviewer and the examinee. 

21

22 7. Avoid copying text directly from training or other reference material. Another common 

23 tendency among written examination developers is to copy sentences directly from 

24 reference material and turn them into test questions. Unfortunately, questions written in 

25 this way generally encourage rote memorization. Furthermore, copying from reference 

26 material can cause ambiguity or deficiency in questions because the replicated material 

27 often draws its meaning and importance from its surrounding context. Therefore, the 

28 test question often omits important assumptions or stipulations stated elsewhere in the 

29 material. Finally, such questions can frequently be answered correctly by examinees 

30 who do not really understand the concept but do remember the specific wording on a 

31 page of reference material. Conversely, examinees who understand the topic, but not in 

32 the exact way it was written in the material, may miss the question because of unstated 

33 assumptions or other missing information.

34

35 8. Avoid “backward logic” questions that ask for what should be provided in the question 

36 and give what should be required in the examinee’s response. Appendix B provides 

37 examples of backward logic questions.

38

39 In addition to testing on valid topics, it is important to test on those topics in a way that is 

40 consistent with how the K/A should be remembered and used. Do not test on the topic 

41 in a backward way.

42

43 9. Ensure that each question is technically accurate.

44

45 10. Ensure that each question is free of confusing or ambiguous language. 

46

47 D. Instructions for Constructing Multiple-Choice Questions

48 1. All written examination questions must be in the multiple-choice format. Use four 

49 answer options. Each question is worth one point. The four-distractor multiple-choice 
ES-4.2, Page 10 of 29

1 item with only one correct answer is the only style that is considered acceptable for NRC 

2 examinations. 

3

4 Do not use test items with multiple correct answers from which examinees must select 

5 the “most correct” answer because such items significantly reduce the reliability of 

6 examination results by increasing the effect of examiner subjectivity in the examination 

7 development and grading processes.

8

9 The five-answer option contributes nothing to the question, and any format with fewer 

10 than four distractors makes guessing correctly more probable. 

11

12 2. The following four basic models are acceptable and may be used in combination with 

13 one another:

14

15 Model A:

16 A. correct answer

17 B. plausible incorrect answer

18 C. plausible incorrect answer

19 D. plausible incorrect answer

20

21 This model depicts the traditional multiple-choice design format with one correct 

22 single-word/phrase answer followed by three incorrect single-word/phrase options. Note 

23 that all options are of similar length.

24

25 Model B:

26 A. correct answer

27 B. plausible misconception

28 C. plausible incorrect answer

29 D. plausible incorrect answer

30

31 This variation of Model A uses a plausible misconception as one of the three incorrect

32 answers. Again, note that all options are of similar length.

33

34 Model C:

35 A. correct answer with correct condition (e.g., because, since, when, if, and other 

36 such conditions)

37 B. correct answer with plausible incorrect condition

38 C. plausible incorrect answer with incorrect condition

39 D. plausible incorrect answer with incorrect condition

40 Model C depicts an acceptable design that uses answers with conditions (i.e., a setting, 

41 event, cause, or effect) that may make the answer correct or incorrect. Note that

42 Model C shows only one correct answer with its correct condition, and all options are

43 similar in length.

44
ES-4.2, Page 11 of 29

1 Model D:

2 A. correct answer

3 B. plausible incorrect answer

4 C. correct answer with plausible incorrect condition

5 D. plausible incorrect answer with incorrect condition

6

7 Model D is useful when it is not possible to create four options of similar length. This 

8 model shows paired lengths (two long and two short options). This approach prevents 

9 any one option from standing apart (by being either too long or too short) from the 

10 remaining options.

11

12 When using Model C or D, it is particularly important to maximize the plausibility of any 

13 incorrect conditions that appear in multiple distractors to minimize the chances that 

14 examinees will be able to eliminate those distractors by detecting one piece of 

15 implausible information.

16

17 3. Do not use “all of the above” or “none of the above.”

18

19 “All of the above” questions provide inadvertent clues to the examinee. When the “all of 

20 the above” option is the correct response, the examinee must simply recognize that two 

21 options are correct to answer the question correctly. Similarly, when “all of the above” is 

22 used as a distractor, the examinee only needs to be able to determine that one option is 

23 incorrect in order to eliminate this option. “None of the above” responses should not be 

24 used with “best answer” multiple-choice questions, since it may always be defensible as 

25 a response. Note that a question might fall into this category even if the words “all of the 

26 above” or “none of the above” do not appear in the answer choices.

27

28 4. Do not present a collection of true/false (T/F) statements as a multiple-choice item.

29

30 As previously discussed, each item should focus on one K/A topic that is determined by 

31 the stem of the question. A question containing answer options related to many 

32 separate issues does not increase the efficiency of the question. To the contrary, 

33 questions with multiple topics confuse the examinee about the meaning and purpose of 

34 the question.

35

36 As a way of determining whether a test item is a collection of T/F statements, check 

37 whether the answer can be determined, or the distractors can be rejected, without the 

38 information contained in the stem. If so, the question is likely a T/F collection. 

39 Appendix B provides sample questions that illustrate this psychometric deficiency.

40

41 5. Define the question, task, or problem in the stem of the question. In designing 

42 multiple-choice questions that are operationally based and require an application/use 

43 scenario, consider providing the conditions in the first part of the question, separated by 

44 a double space from the body of the question and blocked to the left column with each 

45 condition bulleted, as in the following example:

46

47 Given the following conditions:

48

49 • Both main feed pumps tripped.

50 • Auxiliary feedwater (AFW) automatically started.
ES-4.2, Page 12 of 29

1 • AFW valves reset to control steam generator water level.

2 • AFW suction pressure decreased to 7 psig.

3 Which ONE of the following describes the AFW pump response for the given 

4 conditions?

5

6 A. Suction will automatically shift to the nuclear service water system.

7 B. Suction will automatically shift to the upper surge tank.

8 C. Trip when suction pressure decreases to 5 psig.

9 D. Trip after a 6-second time delay.

10

11 6. Include as much necessary information as possible about the problem or situation in the 

12 stem, leaving only the solution, action, or effect for the answer options. Consider the 

13 following “poor” and “better” examples:

14

15 (Poor) At 50-percent power:

16

17 A. The equilibrium xenon reactivity worth is approximately equal to the 

18 equilibrium xenon worth at 100-percent power.

19 B. The equilibrium xenon reactivity worth is approximately one-half the 

20 equilibrium xenon worth at 100-percent power.

21 C. The equilibrium xenon reactivity worth is approximately two-thirds the 

22 equilibrium xenon worth at 100-percent power.

23 D. The equilibrium xenon reactivity worth is approximately three-fourths the 

24 equilibrium xenon worth at 100-percent power.

25 (Better) How does the equilibrium xenon reactivity worth at 50-percent power 

26 compare to the equilibrium xenon reactivity worth at 100-percent power?

27

28 A. equal to

29 B. one-half

30 C. two-thirds

31 D. three-fourths

32

33 7. When possible, avoid using negatively stated stems. Try to write the question in a 

34 positive manner (e.g., instead of “do not open the valve,” use “keep the valve closed”). If 

35 a negative stem is necessary, highlight the negative word (e.g., not, never, least).

36

37 It is very tempting to write negatively stated questions, since they can be constructed by 

38 picking three true statements out of the reference material and changing a fourth 

39 statement to make it false. However, studies have shown that examinees do not do as 

40 well on negatively stated questions because they overlook the negative word or because 

41 negatively stated questions require examinees to select an answer that is not true or 

42 characteristic, which can be somewhat confusing. In addition, these questions tend to 

43 emphasize negative learning. For example, consider the following stem of a 

44 multiple-choice question:

45
ES-4.2, Page 13 of 29

1 During 100-percent power operation, the feedwater heater 2A high-level dump 

2 valve opens inadvertently. The condensate pumps will not do which of the 

3 following:

4

5 This stem can be made to read positively:

6

7 During 100-percent power operation, the feedwater heater 2A high-level dump 

8 valve opens inadvertently. The condensate pumps will do the following:

9

10 A. Increase flow to maintain feedwater flow rate.

11 B. Trip because of a runout condition.

12 C. Have no response.

13 D. Trip because of low suction pressure.

14

15 Although a negatively stated question is sometimes unavoidable, never use a negatively 

16 stated stem with a negatively stated answer option, as illustrated by example E.3 in 

17 Appendix B.

18

19 8. Provide sufficient counterbalance in questions with multipart answers.

20

21 Multiple-choice questions can legitimately contain multipart answer options. However, if 

22 the answers contain too many parts or too many options for each part, cues indicating 

23 the correct answer may be unavoidable. Consider the following example:

24

25 The reactor coolant system (RCS) is in hot standby with no reactor coolant 

26 pumps (RCPs) running. If the once-through steam generator pressure is 

27 decreased, according to the plant verification procedure, which of the following 

28 temperature responses indicate the presence of natural circulation?

29

30 A. T-H increases; T-C remains the same.

31 B. T-H increases; T-C decreases.

32 C. T-H decreases; T-C decreases.

33 D. T-H remains the same; T-C decreases.

34

35 The examinee could choose the correct answer (c) without knowing about the T-C 

36 temperature response in this situation, since “T-H decreases” only occurs in option “C.”

37

38 Notice that two-part answers, with each part containing a two-option response, can 

39 provide complete counterbalance, since all contingencies can be covered in four 

40 responses (provided that the responses meet plausibility requirements), as in the 

41 following example:

42

43 Which of the following is a definition of quadrant power tilt ratio?

44

45 A. minimum upper detector output divided by average upper detector output

46 B. maximum upper detector output divided by average upper detector output

47 C. minimum upper detector output divided by average lower detector output

48 D. maximum upper detector output divided by average lower detector output
ES-4.2, Page 14 of 29

1 If a multipart question format is used, the recommended format is one in which the 

2 two-part answer options consist of a two-level response (e.g., yes/no, off/on) and a 

3 reason, as in the following example:

4

5 Which of the following describes the behavior of equilibrium xenon reactivity over 

6 core life?

7

8 A. It decreases because of the increased fuel burnup.

9 B. It decreases because of the decrease in plutonium-xenon yield.

10 C. It increases because of the increase in thermal flux.

11 D. It increases because of the decrease in boron concentration.

12

13 Note: For multipart questions, both parts of the question must test a topic/concept that is 

14 related to the associated K/A.

15

16 Sometimes, to improve their plausibility, distractors may include secondary pieces of 

17 information that have lower relative importance and discriminatory value than the key 

18 point of the distractor. However, those secondary pieces of information are not 

19 irrelevant; the value of the question should be considered as a whole and not discounted 

20 because the answer choices contain information of lower importance.

21

22 9. When possible, include common misconceptions as distractors. Since the purpose of 

23 the examination is to differentiate between competent and less-than-competent 

24 examinees, consider questions involving topics for which there are common 

25 misconceptions about important K/A topics. For example, the following question was 

26 based on a common misconception about a loss of subcooling margin:

27

28 During a small-break loss-of-coolant accident with a resultant loss of subcooling 

29 margin, why are the RCPs secured?

30

31 A. to prevent pump damage resulting from operation under two-phase

32 conditions

33 B. to prevent core damage resulting from rapid phase separation upon 

34 subsequent loss of RCS flow

35 C. to reduce RCS pressure by removing the pressure head developed by the 

36 RCPs

37 D. to remove the heat being added to the RCS by the operating RCPs

38 10. Make all answer options homogeneous. Ensure there is only one correct answer and 

39 that the distractors are highly plausible. Consider the following “poor” and “better” 

40 examples:
ES-4.2, Page 15 of 29

1 On a loss of condenser circulating water intake canal, the upper surge tank, 

2 hotwell, and condensate storage tank will supply sufficient feedwater to allow 

3 decay heat removal for approximately:

4

5 Poor Better

6

7 A. 15 minutes A. 8 hours

8 B. 8 hours B. 24 hours

9 C. 48 hours C. 48 hours

10 D. 3 months D. 72 hours

11

12 Notice how one method of changing the difficulty level of a question is to vary the 

13 similarity of the answer options. The distractors should be similar enough to be chosen 

14 by those who do not meet the testing objective, yet different enough so they do not test 

15 trivial issues or distinctions. Also note how the answer options in each case are listed in 

16 order of magnitude.

17

18 11. If the answer options have a logical or a timeline sequence, put them in order (as in 

19 no. 10 above). 

20

21 12. Avoid overlapping answer options, as in the following example:

22

23 The self-powered neutron detector uses rhodium, which decays with a half-life of 

24 42 seconds. How long will it take for a detector to indicate approximately 

25 95 percent of an instantaneous power-level change?

26

27 Poor Better

28

29 A. 2 to 4 minutes A. 1 to 2 minutes

30 B. 4 to 6 minutes B. 3 to 4 minutes

31 C. 6 to 8 minutes C. 5 to 6 minutes

32 D. 8 to 10 minutes D. 7 to 8 minutes

33

34 13. Do not include trivial distractors with more important distractors. In the search for 

35 distractors, it is very tempting to include relatively trivial facts along with options that 

36 focus on more important issues or concepts, as in the following example:

37

38 Which of the following is true concerning the turbine?

39

40 A. The turbine is rotated at low speed when shut down to prevent distortion 

41 of the turbine casing.

42 B. Turbine eccentricity is the measure of turbine speed.

43 C. The turbine blades are cooled by hydrogen gas.

44 D. Technical specifications require that at least one turbine overspeed 

45 protection system must be operable in Mode 2.

46 Relative to the other options, option “C” could be considered a trivial distractor. Even if 

47 included as an incorrect answer, relatively unimportant information jeopardizes the 
ES-4.2, Page 16 of 29

1 content validity of the question. Also, note that this question consists of a collection of 

2 T/F statements as described in no. 4 above.

3

4 14. Vary the position of the correct answer; avoid a pattern. Make sure that the position of 

5 the correct answer is randomized throughout the examination. This means that options 

6 “A,” “B,” “C,” and “D” should be correct about an equal number of times, but in no 

7 specific order.

8

9 15. Avoid “specific determiners” that give clues as to the correct answer. Specific 

10 determiners include the following:

11

12 a. distractors that do not follow grammatically from the stem, as in the following 

13 example:

14

15 During 100-percent normal power operation, a single steam flow element 

16 in the steam generator feedwater control system fails high. This will 

17 cause the following:

18

19 A. the feedwater valves to increase steam generator level slightly 

20 before returning the level to normal

21 B. before returning the level to slightly above normal, the feedwater 

22 valves to increase the steam generator level significantly

23 C. the feedwater valves to increase the steam generator level to the 

24 level of a reactor trip

25 D. the feedwater valves to increase the steam generator level slightly 

26 and maintain the increased level

27 Note the improvement when distractor “B” is reworded as follows:

28

29 B. the feedwater valves to increase the steam generator level 

30 significantly before returning the level to slightly above normal

31

32 b. options that can be judged correct or incorrect without reading the stem

33

34 c. equivalent or synonymous options, which rule out both options for an examinee 

35 who recognizes the equivalence

36

37 d. an option that includes another option, also called a “subset” (e.g., (A) less than 

38 5; (B) less than 3)

39

40 e. implausible distractors

41

42 f. a correct answer that is longer or shorter than the distractors

43

44 g. qualifiers in the correct answer (e.g., probably and ordinarily) unless they are 

45 also used in the distractors

46

47 h. words such as “never” or “always” that suggest an incorrect option
ES-4.2, Page 17 of 29

1 i. a correct option that differs from the distractors in favorableness, style, or

2 terminology, as in the following example:

3

4 Which action or occurrence is likely to cause water hammer?

5

6 A. maintaining the discharge line from an auto starting pump filled 

7 with fluid

8 B. water collecting in a steamline

9 C. prewarming of steam lines

10 D. slowly closing the discharge valve of an operating pump

11 In the above question, all options except for “B” (the correct answer) describe 

12 preventive actions, while option “B” describes a condition that occurs as a result

13 of negligence or oversight. A test-wise examinee would only need to know that 

14 water hammer is not a desired occurrence to determine that “B” is the least 

15 favorable and, therefore, the correct answer.

16

17 j. When appropriate, use distractors that are generically correct statements, but do 

18 not correctly answer the question, as illustrated in the following example:

19

20 Preparations are being made for refueling, and the following plant 

21 conditions exist:

22

23 • The refueling cavity is filled with the transfer tube gate valve open.

24 • The SFP LO LEVEL and CTMT SUMP HI LEVEL annunciators 

25 are in alarm.

26

27 Which ONE of the following is the required IMMEDIATE ACTION in 

28 response to these conditions?

29

30 A. Verify alarms by checking the containment sump level recorder 

31 and spent fuel level indication.

32 B. Sound the containment evacuation alarm.

33 C. Initiate containment ventilation isolation.

34 D. Initiate control room ventilation isolation.

35 Answer “A” is a generic good practice, but it is not responsive to the conditions 

36 specified in the stem of the question. It is not a required immediate action, nor is 

37 it an appropriate response in light of the mutually confirmatory annunciators that 

38 are in alarm.

39
ES-4.2, Page 18 of 29

1 E. Instructions for Developing and Reviewing Senior Reactor Operator-Level

2 Questions 

3 1. The 25 questions for the SRO-level written examination, also known as “SRO-only 

4 questions,” help the NRC evaluate the additional knowledge, skills, and abilities required 

5 for performing licensed senior operator duties. SRO-level questions must be consistent 

6 with the cognitive level of their associated K/A statement.

7

8 Several generic K/As in Section 2 of the applicable K/A catalogs are specifically linked to 

9 one or more topics specified in 10 CFR 55.43(b), and all of the Category A2, AA2, and 

10 EA2 K/A statements are (or, in the case of NUREG-1123, “Knowledge and Abilities 

11 Catalog for Nuclear Power Plant Operators: Boiling Water Reactors,” should be) 

12 similarly linked. Consequently, the K/As for the SRO-portion of the written examination 

13 will be drawn from those K/A categories (denoted by Columns “A2” and “G” in the 

14 SRO-only section of the applicable written examination outline) and from all K/A 

15 categories related to the fuel handling facilities, which are specifically identified for 

16 sampling in 10 CFR 55.43(b)(7). The fact that a K/A is linked to both 10 CFR 55.41, 

17 “Written examination: operators,” and 10 CFR 55.43, “Written examination: Senior 

18 operators,” does not mean that the K/A cannot be used to develop an SRO-level

19 question, nor does it exclude the K/A from sampling on the RO examination. However, 

20 to be used on the SRO-portion of a written examination, a question developed from a 

21 K/A linked to both 10 CFR 55.41 and 10 CFR 55.43 must either test at the level of the 

22 10 CFR 55.43(b) item number(s) that the K/A is linked to, or test at a level that is unique 

23 to the SRO job position as determined by the facility licensee training program. 

24 Similarly, K/A topics linked to 10 CFR 55.41(b) might also be appropriate for developing 

25 SRO-level questions if the questions developed evaluate K/As at a 10 CFR 55.43(b) 

26 level or at a level that is unique to the SRO job position as determined from the facility

27 licensee’s learning objectives. The fact that a K/A does not reference 10 CFR 55.43 

28 does not, in and of itself, disqualify the K/A from testing on the SRO-portion of a written 

29 examination. 

30

31 2. Each SRO-level question must be tied to one of the 10 CFR 55.43(b) items. If a facility 

32 licensee desires to evaluate a K/A that is not tied to one of the 10 CFR 55.43(b) items, 

33 then the licensee must classify the K/A as “unique to the SRO position” and provide

34 documentation that ties the K/A to the facility licensee’s SRO job position duties in 

35 accordance with the systematic approach to training, as in the following examples:

36

37 • The question is linked to a learning objective that is specifically labeled in the 

38 lesson plan as being SRO-only (e.g., some facility licensee lesson plans have 

39 columns in the margin that differentiate auxiliary operator, RO, and SRO learning 

40 objectives).

41

42 • The question is linked to a task that is labeled as an SRO-only task, and the task

43 is not listed in the RO task list. Note: The fact that a facility licensee trains its 

44 ROs to master certain 10 CFR 55.43 K/As and skills does not mean that they can 

45 no longer be used as a basis for SRO-only questions.

46

47 3. Examination authors should use the 10 CFR 55.43(b) topic-based guidance and 

48 examples (a–g below) when developing SRO-level questions.

49
ES-4.2, Page 19 of 29

1 a. Conditions and Limitations in the Facility License [10 CFR 55.43(b)(1)]

2 Examples of SRO-only examination items for this topic include the following:

3

4 • reporting requirements when the maximum licensed thermal power output is exceeded

5 • administration of fire protection program requirements, such as compensatory actions 

6 associated with inoperable sprinkler systems and fire doors

7 • required actions necessary when a facility does not meet the administrative controls listed 

8 in Technical Specifications (TS) Section 5 or 6, depending on the facility (e.g., shift staffing 

9 requirements)

10 • National Pollutant Discharge Elimination System requirements, if applicable

11 • processes for TS and final safety analysis report changes

12 Note: The analysis and selection of required actions for TS Sections 3 and 4 may be more 

13 appropriately listed for the 10 CFR 55.43(b)(2) topic.

14

15 b. Facility Operating Limitations in the Technical Specifications and Their Bases

16 [10 CFR 55.43(b)(2)]

17 Examples of SRO-only examination items for this topic include the following:

18

19 • application of required actions (TS Section 3) and surveillance requirements (SRs) (TS 

20 Section 4) in accordance with rules of application requirements (TS Section 1)

21

22 • application of generic limiting condition for operation (LCO) requirements (LCOs 3.0.1 

23 through 3.0.7; SRs 4.0.1 through 4.0.4)

24

25 • knowledge of TS bases that are necessary to analyze TS-required actions and 

26 terminology; such knowledge should be beyond that associated with “above the line” TS 

27 information or plant systems knowledge, as shown in the following examples:

28

29 – [TS bases required to analyze TS-required actions] For a given facility, 

30 TS 3.7.14, “Spent Fuel Pool Water Level,” Action A.1, requires suspending 

31 movement of irradiated fuel assemblies in the spent fuel pool immediately if 

32 spent fuel pool water level is not within limits. However, the basis information for 

33 this TS action clarifies that this does not preclude movement of a fuel assembly 

34 to a safe position. Thus, despite the wording of the TS action, the correct 

35 application (taking the basis into account), would be to first complete an 

36 in-progress fuel movement before suspending movement.

37

38 – [terminology] For another facility, TS 3.4.7, “Residual Heat Removal (RHR) 

39 Shutdown Cooling System—Hot Shutdown,” Action A.2, requires verifying that 

40 an “alternate method of decay heat removal” is available for each inoperable 

41 RHR shutdown cooling subsystem within 1 hour of one or two RHR shutdown 

42 cooling subsystems becoming inoperable. However, Action A.2 does not detail 

43 the composition of these methods. The basis information for this TS action 

44 clarifies that decay heat removal by ambient losses, the condensate/main steam 
ES-4.2, Page 20 of 29

1 systems, the reactor water cleanup system, and a combination of an emergency 

2 core cooling system pump and a safety relief valve are all possible means of 

3 satisfying this requirement.

4 • the same items listed above for the technical requirements manual (TRM) and offsite 

5 dose calculation manual (ODCM)

6

7 SRO-only knowledge generally cannot be claimed for questions that can be answered solely

8 based on knowledge of ≤ 1-hour action statements and the safety limits since ROs are typically 

9 required to know these items. SRO-only knowledge generally cannot be claimed for questions 

10 that can be answered solely based on expected RO TS knowledge. ROs are typically expected 

11 to know the LCO statements and associated applicability information (i.e., the information above 

12 the double line separating the ACTIONS from the LCO and associated applicability statements 

13 in the standardized TS, as shown in the example in Figure 4.2-1).

14

15 Figure 4.2-1 Example of “Above the Line” TS Information

16

17

18

19

20

21

22

23

24

RO knowledge

Above this 

line
ES-4.2, Page 21 of 29

1 Figure 4.2-2 Screening for SRO-Only Questions Linked to 10 CFR 55.43(b)(2) (TS)

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26 

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48 c. Facility Licensee Procedures Required To Obtain Authority for Design and Operating 

49 Changes in the Facility [10 CFR 55.43(b)(3)]

50 Some examples of SRO-only examination items for this topic include the following:

RO question Yes

Can the question be answered solely by 

knowing ≤ 1-hour TS/TRM action?

No

Can the question be answered solely by

knowing the LCO/TRM information listed 

“above the line?”

No

Can the question be answered solely by 

knowing the TS safety limits?

No

Does the question involve one or more of the following for TS, 

TRM, or ODCM:

• application of required actions (TS Section 3) and SRs (TS 

Section 4) in accordance with rules of application 

requirements (TS Section 1)

• application of generic LCO requirements (LCOs 3.0.1 

through 3.0.7 and LCOs 4.0.1 through 4.0.4)

• knowledge of TS bases that is required to analyze TS 

required actions and terminology

SROlevel 

question

Yes

No

Question might not be linked to 

10 CFR 55.43(b)(2) for SRO-only

Can the question be answered solely by 

knowing the TS bases information associated 

with the “above-the-line” LCO information or 

general systems knowledge?

No

RO question Yes

RO question Yes

RO question Yes
ES-4.2, Page 22 of 29

1 • screening and evaluation processes under 10 CFR 50.59, “Changes, tests and 

2 experiments”

3 • administrative processes for temporary modifications

4 • administrative processes for disabling annunciators

5 • administrative processes for the installation of temporary instrumentation

6 • processes for changing the plant or plant procedures

7 Appendix B provides an example of a satisfactory SRO-level question related to this topic.

8

9 d. Radiation Hazards that May Arise during Normal and Abnormal Situations, including 

10 Maintenance Activities and Various Contamination Conditions [10 CFR 55.43(b)(4)]

11 Some examples of SRO-only examination items for this topic include the following:

12

13 • process for gaseous/liquid release approvals (i.e., release permits)

14 • analysis and interpretation of radiation and activity readings as they pertain to the 

15 selection of administrative, normal, abnormal, and emergency procedures

16 • analysis and interpretation of coolant activity, including comparison to emergency plan 

17 criteria or regulatory limits (or both)

18 • process for authorizing emergency exposure limits

19 SRO-only knowledge should not be claimed for questions that can be answered solely based on 

20 RO knowledge of radiological safety principles (e.g., radiation work permit requirements, stay 

21 time, and derived air concentration hours).

22

23 e. Assessment of Facility Conditions and Selection of Appropriate Procedures during Normal, 

24 Abnormal, and Emergency Situations [10 CFR 55.43(b)(5)]

25 This topic involves both (1) assessing plant conditions (normal, abnormal, or emergency) and 

26 then (2) selecting a procedure or section of a procedure to mitigate or recover, or with which to 

27 proceed. One area of SRO-level knowledge (with respect to selecting a procedure) is 

28 knowledge of the content of the procedure versus knowledge of the procedure’s overall 

29 mitigative strategy or purpose.

30

31 The applicant’s knowledge can be evaluated at the level of 10 CFR 55.43(b)(5) by ensuring that 

32 the additional knowledge of the procedure’s content is required to correctly answer the written 

33 test item, as in the following examples:

34 • knowledge of when to implement attachments and appendices, including how to 

35 coordinate these items with procedure steps

36

37 • knowledge of diagnostic steps and decision points in the EOPs that involve transitions to 

38 event-specific subprocedures or emergency contingency procedures

39
ES-4.2, Page 23 of 29

1 • knowledge of administrative procedures that specify hierarchy, implementation, or 

2 coordination of plant normal, abnormal, and emergency procedures

3

4 Another area of SRO-level knowledge (with respect to selecting a procedure) is knowledge of 

5 decision points within the facility’s emergency plan. These include emergency action level 

6 declarations and protective action recommendations since these would subsequently lead to the 

7 implementation of various subprocedures within the emergency plan.

8

9 SRO-only knowledge should not be claimed for questions that can be answered solely using 

10 “systems knowledge,” such as the following:

11

12 • how the system works

13 • system flowpath

14 • component locations

15

16 SRO-only knowledge should not be claimed for questions that can be answered solely using 

17 fundamental knowledge of the following:

18

19 • the basic purpose of a procedure, the overall sequence of events that will occur, or the 

20 overall mitigative strategy of a procedure

21 • any AOP entry condition

22 • plant parameters that require entry into major EOPs:

23 – major EOPs for Westinghouse: E0, E1, E2, E3, ECA-0.0, and red/orange

24 functional restoration procedures

25 – major EOPs for General Electric: reactor vessel control, primary containment 

26 control, secondary containment control, and radioactive release control

27 – major EOPs for Combustion Engineering: standard posttrip actions and optimal

28 recovery procedures

29 – major EOPs for Babcock and Wilcox: EOP entry, loss of subcooling margin, lack 

30 of heat transfer, excessive heat transfer, steam generator tube rupture, and 

31 inadequate core cooling

32 – major EOPs for AP1000: E0, E1, E2, E3, and red/orange functional restoration 

33 procedures

34 • immediate operator actions of a procedure

35 Appendix B provides several satisfactory and unsatisfactory examples of test items related to 

36 10 CFR 55.43(b)(5).

37
ES-4.2, Page 24 of 29

1 Figure 4.2-3 Screening for SRO-Only Questions Linked to 10 CFR 55.43(b)(5)

2 (Assessment and Selection of Procedures)

3

4

Can the question be answered solely from 5

systems knowledge (i.e., how the system works, 

flowpath, logic, location)? RO question Yes

No

Can the question be answered solely by knowing 

immediate operator actions? Yes

Can the question be answered solely by knowing 

the entry conditions for AOPs or plant parameters 

that require entry to major EOPs?

Yes

No

Does the question require one or more of the following?

• assessment of plant conditions (normal, abnormal, or 

emergency) and then prescribing a procedure or section of 

a procedure to mitigate or recover or with which to proceed

• knowledge of when to implement attachments and 

appendices, including how to coordinate these items with 

procedure steps

• knowledge of diagnostic steps and decision points in the 

EOPs that involve transitions to event-specific 

subprocedures or emergency contingency procedures

• knowledge of administrative procedures that specify 

hierarchy, implementation, or coordination of plant normal, 

abnormal, and emergency procedures

SRO-only 

question

Yes

Can the question be answered solely by knowing 

the purpose, overall sequence of events, or 

overall mitigative strategy of a procedure?

No

No

Yes

Question might not be linked to 

10 CFR 55.43(b)(5) for SRO-only

No

RO question

RO question

RO question
ES-4.2, Page 25 of 29

1 f. Procedures and Limitations Involved in Initial Core Loading, Alterations in Core 

2 Configuration, Control Rod Programming, and Determination of Various Internal and 

3 External Effects on Core Reactivity [10 CFR 55.43(b)(6)]

4 Some examples of SRO-only examination items for this topic include the following:

5

6 • evaluation of core conditions and emergency classifications based on core conditions

7 • administrative requirements associated with low-power physics testing processes

8 • administrative requirements associated with refueling activities, such as approvals 

9 required to amend core loading sheets or administrative controls of potential dilution 

10 paths or activities

11 • administrative controls associated with the installation of neutron sources

12 • knowledge of TS bases for reactivity controls

13

14 g. Fuel Handling Facilities and Procedures [10 CFR 55.43(b)(7)]

15 Some examples of SRO-only examination items for this topic include the following:

16

17 • refuel floor SRO responsibilities

18 • assessment of fuel handling equipment SR acceptance criteria

19 • prerequisites for vessel disassembly and reassembly

20 • decay heat assessment

21 • assessment of SRs for the refueling mode

22 • reporting requirements

23 • emergency classifications

24

25 F. Forms

26 Form 4.2-1 Written Examination Question Worksheet

27 Form 4.2-2 Question Development Checklist 

28 Form 4.2-3 Reactor Operator Written Examination Cover Sheet

29 Form 4.2-4 Senior Reactor Operator Written Examination Cover Sheet
ES-4.2, Page 26 of 29

Form 4.2-1 Written Examination Question Worksheet

Examination Outline Cross-Reference: Level RO SRO

Tier # _____ _____

Group # _____ _____

K/A # _________________

Importance Rating _____ _____

Knowledge and Ability (K/A) Statement:

Proposed Question:

Proposed Answer: ____

Explanation:

Technical Reference(s): _______________________________________________

(Attach if not previously provided, _______________________________________________

including version/revision number.) _______________________________________________

Proposed references to be provided to applicants during examination: _________________

Learning Objective: _________________________ (required, if available)

Question Source: Bank # _______

Modified Bank # _______ (note changes or attach parent)

New _______

Question History: Last NRC Exam ____________ (if applicable)

Question Cognitive Level: Memory or Fundamental Knowledge _____

Comprehension or Analysis _____

10 CFR Part 55 Content: 10 CFR 55.41 _____

10 CFR 55.43 _____

Comments:
ES-4.2, Page 27 of 29

Form 4.2-2 Question Development Checklist

1. Does the concept being measured have a direct, important relationship to the ability to 

perform the job?

2. Does the question match the testing objective and intent of the knowledge or ability?

3. Is the question clear, concise, and easy to read? Could it be stated more simply and still 

provide the necessary information? Should it be reworded or split into more than one 

question?

4. Is each question stated positively, unless the intent is to test knowledge of what not to 

do?

5. Does the question provide all necessary information, stipulations, and assumptions 

needed for a correct response? Does the stem include as much information as 

possible?

6. Is the question written at the highest appropriate level of knowledge or ability for the job 

position of the person being tested?

7. Is the question free of unnecessary difficulty, trickiness, or irrelevance?

8. Is the question limited to one concept or topic? Is it something other than a collection of 

true/false items?

9. Does the question have face validity?

10. Are key points underlined or highlighted?

11. Is each question separate and independent of all other questions?

12. Are the answer options homogeneous and are the distractors highly plausible? Is there 

only one correct answer? Are common misconceptions used as distractors? Is the 

question free of trivial distractors and overlapping answer options?

13. Does the question avoid the use of “none of the above” and “all of the above”?

14. Does each question have four answer options?

15. Are the answer options for the questions ordered sequentially?

16. Is the question free of “specific determiners” (e.g., logical or grammatical 

inconsistencies, incorrect answers that are consistently different, verbal associations 

between the stem and the answer options)?
ES-4.2, Page 28 of 29

Form 4.2-3 Reactor Operator Written Examination Cover Sheet

U.S. Nuclear Regulatory Commission

Reactor Operator Written Examination

Applicant Information

Name:

Date: Facility/Unit

Region: I II III IV Reactor Vendor/Type: 

Start Time: Finish Time:

Instructions

Use the answer sheets provided to document your answers. Staple this cover sheet on top 

of the answer sheets. To pass the examination, you must achieve a final grade of at least 

80 percent. You have 6 hours to complete this portion of the examination. 

Applicant Certification

All work done on this examination is my own. I have neither given nor received aid.

_______________________________

Applicant’s Signature

Results

Examination Points ________ Points

Applicant’s Points ________ Points

Applicant’s Grade ________ Percent
ES-4.2, Page 29 of 29

Form 4.2-4 Senior Reactor Operator Written Examination Cover Sheet

U.S. Nuclear Regulatory Commission

Senior Reactor Operator Written Examination

Applicant Information

Name:

Date: Facility/Unit

Region: I II III IV Reactor Vendor/Type: 

Start Time: Finish Time:

Instructions

Use the answer sheets provided to document your answers. Staple this cover sheet on top 

of the answer sheets. To pass the examination, you must achieve a final grade of at least 

80 percent overall, with 70 percent or better on the senior reactor operator (SRO)-only items 

if given in conjunction with the reactor operator (RO) exam; SRO-only exams given alone 

require a final grade of 80 percent to pass. You have 9 hours to complete the combined 

examination and 3 hours if you are only taking the SRO-only portion.

Applicant Certification

All work done on this examination is my own. I have neither given nor received aid.

 __________________________________

 Applicant’s Signature

Results

RO/SRO-Only/Total Examination Points ______ / ______ / ______ Points

Applicant’s Points ______ / ______ / ______ Points

Applicant’s Grade ______ / ______ / ______ Percent

ES-4.3, Page 1 of 6

1 4.3 ADMINISTERING WRITTEN EXAMINATIONS

2 This examination standard contains the instructions and guidelines for administering the written 

3 examination portion of the NRC operator licensing initial examinations for RO and SRO license 

4 applicants at power reactor facilities. It includes instructions for proctoring the written 

5 examinations and conducting postexamination reviews of the written examinations.

6

7 Generally, the facility licensee administers the written examination in accordance with the 

8 instructions contained herein, although the NRC regional office may choose to administer the 

9 examination at its discretion.

10

11 All participants in the examination administration process must also be mindful of their 

12 responsibilities for examination security and integrity. 

13

14 A. Responsibilities

15 1. Facility Licensee

16 The facility licensee has the following responsibilities with regards to administering the written 

17 examination:

18

19 a. Establish, implement, and maintain procedures to control examination security and 

20 integrity, as required under 10 CFR 55.49, “Integrity of examinations and tests,” and 

21 10 CFR 55.40(b)(2).

22

23 b. Provide a single room suitable for administering the written examination. To ensure 

24 examination integrity, the room must be large enough so that there is only one applicant 

25 per table, with a 1-meter (3-foot) space between tables.

26

27 c. The examination room and supporting restroom facilities (i.e., the examination area) 

28 must be located to prevent the applicants from having contact with all other facility and 

29 contractor personnel during the written examination.

30

31 d. If desired and compatible with examination security requirements, arrange for the 

32 applicants to have lunch, coffee, or other refreshments during the examination.

33

34 e. Before the scheduled examination date, familiarize the applicants with the examination 

35 guidelines contained in ES-1.2.

36

37 f. Provide the necessary number of copies of the approved examinations, answer sheets, 

38 and handouts for each applicant, as directed and approved by the NRC chief examiner. 

39 Make an English dictionary, steam tables, and the generic fundamentals examination

40 equation sheet available in the examination room. Use machine-gradable answer 

41 sheets if desired, but this is not required.

42

43 g. As discussed in Section C, document and collect any comments for NRC consideration 

44 during the grading process. Submit these comments to the NRC as part of the

45 postexamination package (refer to ES-4.4, “Grading and Documenting the Written 

46 Examinations”). The facility licensee may also request an informal meeting with the 

47 NRC chief examiner to discuss the examination questions and resolve concerns.
ES 4.3, Page 2 of 6

1 2. NRC Regional Office 

2 The NRC regional office has the following responsibilities with regards to administering the 

3 written examination:

4

5 a. The NRC regional office may, at its discretion, administer the examination in accordance 

6 with the instructions in Section B. 

7

8 b. An NRC examiner inspects the examination area to ensure its adequacy. This can 

9 occur during the preparatory site visit, during operating test administration, or, if the NRC 

10 is on site, during written examination administration. Additionally, if NRC examiners are 

11 on site during administration, the NRC examiners should periodically monitor the 

12 examination to ensure that the proctor is appropriately addressing the applicants’ 

13 questions. 

14

15 c. If NRC examiners are not on site during administration, an NRC point of contact must be 

16 available to respond to facility licensee questions while the examinations are being 

17 given. For NRC-authored examinations, either the NRC chief examiner or another

18 examiner familiar with the examination content must be available to respond to the 

19 applicants’ questions by telephone.

20

21 B. Written Examination Administration Instructions

22 The written examination is administered after it is approved by the NRC for administration and 

23 when all license applications have been received and reviewed by the NRC regional office.

24

25 1. Prepare

26 a. A proctor must be present at all times to monitor the applicants while they are taking the 

27 written examination. Ensure that all proctors clearly understand their responsibilities and 

28 the instructions in this section before distributing examinations.

29

30 For NRC-administered examinations, the NRC chief examiner should consider using an 

31 NRC examiner or other NRC employee to ensure that enough people are available to 

32 perform proctoring duties, as needed. The NRC may arrange for facility licensee 

33 employees to proctor the examination for brief periods if it is necessary for the NRC staff

34 to take a short break.

35

36 b. Ensure at least one individual who is familiar with the questions (i.e., an NRC examiner 

37 or facility licensee examination author) is available to clarify examination questions for 

38 the applicants during the examination.

39

40 c. Remove from the examination area, or otherwise remove from the applicants’ view, any 

41 wall charts, models, or other training materials that might compromise examination 

42 integrity.

43

44 d. Verify each applicant’s identity and examination level against Form 2.2-1, “List of 

45 Applicants,” before beginning the examination. Resolve any errors or absences and 

46 update the form if necessary.

47
ES 4.3, Page 3 of 6

1 e. Document the seating arrangement of the applicants during the examination on a chart 

2 or equivalent diagram.

3

4 f. If the applicants will use machine-gradable answer forms that offer more than four 

5 answer choices (e.g., “a” through “e”), use a straight edge to line out the inapplicable 

6 column(s) before distributing the forms.

7

8 g. Clear the memory on any programmable calculators available for applicant use.

9

10 h. Ensure that the applicants’ electronic devices are secured such that applicants cannot 

11 use them during the examination.

12

13 2. Start the Examination

14 a. Remind the applicants that they may use calculators to complete the examination and 

15 that only the reference materials provided with the examination are allowed in the 

16 examination area.

17

18 b. Pass out the examinations, blank answer sheets, and all required handouts approved by 

19 the NRC chief examiner (e.g., steam tables, equation sheets, and all approved reference 

20 material for each portion of the examination). 

21

22 c. Ensure that RO applicants only get references associated with the RO examination and 

23 that SRO applicants get references associated with the examination sections that they 

24 are taking (e.g., an SRO-upgrade examinee who is only taking the SRO-only portion of 

25 the examination would only receive references associated with the SRO-only portion of 

26 the examination). 

27

28 For SRO applicants taking both sections of the SRO examination (i.e., RO and 

29 SRO-only), provide both portions of the examination, including approved reference 

30 material, at the start of the examination. The NRC does not require examinees to 

31 complete the RO portion of the examination before starting the SRO-only portion of the 

32 examination. 

33

34 d. Instruct the applicants not to review the examination until told to do so.

35

36 e. Provide each applicant with a copy of ES-1.2 and brief the applicants using Parts A and 

37 B of ES-1.2, which contains the guidelines that will be in effect during the written 

38 examination.

39

40 f. Instruct the applicants to verify that they have a complete examination by checking their 

41 cover sheet and the number of pages in the examination. RO applicants should have a 

42 75-question examination and SRO applicants should have a 100-question examination, 

43 unless they have obtained a waiver (ES-2.2, “Applications, Medical Requirements, and 

44 Waiver and Excusal of Examination and Test Requirements”) to upgrade their RO 

45 licenses with a 25-question SRO-only examination or they are taking the 40-question 

46 SRO examination limited to fuel handling.

47

48 g. Answer any questions that the applicants may have about the examination policies. 

49

50 h. Start the examination and record the start time.
ES 4.3, Page 4 of 6

1

2 i. Follow the allotted times for the examination:

3

4 • 3 hours for the 25-question SRO-upgrade examination

5 • 4 hours for the SRO examination limited to fuel handling

6 • 6 hours for the RO examination

7 • 9 hours for the combined RO/SRO examination

8

9 3. Monitor the Examination

10 a. When proctoring the examination, remain fully attentive to the applicants during the 

11 examination. The proctor cannot engage in any activity that diverts attention from the 

12 applicants and may possibly cause the examination to be compromised.

13

14 b. When responding to questions raised by applicants during the examination, be 

15 extremely careful not to lead the applicants or give away answers when clarifying 

16 questions. If there is any doubt about how to respond to an applicant’s question, 

17 withhold additional guidance and instruct the applicant to do his or her best with the 

18 information that is provided.

19

20 c. Provide any question changes or clarifications unambiguously to all applicants in the 

21 room (use a chalk board or white board, etc.) and announce this information to all the 

22 applicants. 

23

24 d. Document any changes to questions made during the examination in ink on the NRC’s 

25 master copy and a copy that is retained by the facility licensee staff after the examination 

26 is administered. The NRC chief examiner will review and approve any changes during

27 the grading process (ES-4.4).

28

29 e. Document all applicant questions and associated answers or clarifications about the test

30 items on the written examination made during examination administration (verbatim if 

31 possible) and provide these to the NRC chief examiner after the examination. 

32

33 f. Periodically advise the applicants of the time that remains to complete the examination. 

34 Typically, a chalk board or white board is available and can be used for this purpose.

35

36 4. Complete the Examination

37 a. As the applicants complete the examination, ensure that they sign the examination cover 

38 sheet and staple it on top of their answer sheets. Record the official start time and the 

39 time at which each applicant completed the examination in the space provided on the 

40 examination cover sheet. Collect the examination packages, including the questions,

41 answer sheets, cover sheets, and any reference materials provided with the 

42 examination. Verify that all applicants have entered their names on both the answer and 

43 cover sheets.

44

45 b. Remind the applicants to leave the examination area, as previously defined.

46

47 c. When the allotted time has elapsed, instruct the remaining applicants to stop work, sign 

48 their examination cover sheets, and turn in their examinations. 

49
ES 4.3, Page 5 of 6

1 d. Do not exceed the allotted time for taking each examination (above), except for 

2 unavoidable situations (e.g., loss of power, building evacuation, emergency response). 

3 If a time extension is necessary, the facility licensee must first notify the NRC regional 

4 office to ensure that a point of contact remains available to respond to questions. The 

5 applicant is responsible for ensuring his or her physical capability to complete the 

6 examination in the allotted time.

7

8 e. Inform the NRC when all the applicants have completed the examination.

9

10 f. Deliver the completed examination packages, the marked-up master examinations, the 

11 list of applicant questions and answers, and the seating chart to the NRC chief examiner 

12 or the appropriate facility licensee representative, as applicable, for review and grading 

13 in accordance with the instructions in ES-4.4.

14

15 C. Postexamination Reviews and Comments

16 This section contains instructions for collecting examination comments from applicants and the 

17 facility licensee. The facility licensee must collect all comments from the applicants during the 

18 examination administration and postexamination reviews and submit them to the NRC. If on

19 site during exam administration, the NRC may participate in the postexamination review and 

20 collect comments in person. The postexamination comments are submitted to the NRC regional 

21 office as part of the postexamination package. ES-4.4 contains the instructions for that 

22 submittal.

23

24 Comment collection involves the following steps: 

25

26 1. Document all applicant questions and answers or clarifications about items on the 

27 written examination made during any postexamination reviews with the facility licensee 

28 training staff (verbatim if possible).

29

30 2. If the NRC administered the examination, then the NRC chief examiner ensures that the 

31 master copy of the examination reflects all changes made to questions during the 

32 administration of the examination. The NRC chief examiner also provides a copy of the 

33 master examination and answer key to the facility licensee staff.

34

35 3. Evaluate all questions posed by the applicants during the examination, any pen-and-ink 

36 changes made on the master examination during administration, and any 

37 postexamination review comments from the facility licensee or applicants.

38

39 4. Document all comments in the following:

40

41 • Indicate the source of the comment (i.e., facility licensee or applicant). If the 

42 applicant submits the comment, indicate the docket number associated with the 

43 comment. Note that the NRC examination report will not identify examination 

44 comments by applicant docket number.

45

46 • List the question, answer, and reference.

47

48 • State the comment and recommend whether the answer key should be changed

49 (i.e., delete the question, accept a different answer, or accept multiple answers). 
ES 4.3, Page 6 of 6

1 Explain why the recommendation is being made. Include the facility licensee’s

2 position for each applicant comment.

3

4 • Support the comment with a reference and provide a copy if it was not included 

5 in the original reference material submittal. (Note: The NRC will not change the 

6 examination without a reference to support the comment.)

7 5. Encourage facility licensees to discuss their postexamination comments with the NRC 

8 chief examiner before formally submitting any comments in writing.

9 6. Submit the comments to the NRC regional office as part of the postexamination package 

10 in accordance with the instructions in ES-4.4. 

11 7. The NRC will review all post-examination comments submitted by the facility licensee

12 and individual applicants. Commenters should consider the instructions in ES-4.4, 

13 Section C, before submitting postexamination comments to the NRC. 
ES-4.4, Page 1 of 6

1 4.4 GRADING AND DOCUMENTING THE WRITTEN EXAMINATIONS

2 This examination standard contains the instructions and guidelines for grading the written 

3 examination portion of the NRC operator licensing initial examinations for RO and SRO license 

4 applicants at power reactor facilities. This examination standard also includes instructions for 

5 evaluating and revising written examinations after they are administered and for the NRC review 

6 of proposed examination changes and grading.

7

8 The facility licensee grades the written examinations, evaluates the outcome, and submits the 

9 examination results to the responsible NRC regional office for review. The NRC reviews the 

10 grading and the examination results and considers any examination changes recommended by 

11 the facility licensee and individual applicants before making licensing decisions.

12

13 A. Responsibilities

14 1. Facility Licensee

15 The facility licensee performs the following activities:

16

17 • Review and resolve any questions and comments that arose during or after the 

18 administration of the examination (refer to ES-4.3).

19

20 • Grade the examinations using the instructions in Section B.

21

22 • A facility licensee manager or supervisor must confirm the quality of the grading and 

23 inform the NRC that he or she concurs with the individual and collective examination 

24 results, including the justification for any proposed examination changes in the 

25 postexamination package submitted to the NRC regional office.

26

27 • Submit the postexamination package to the NRC regional office for review.

28

29 2. NRC

30 The NRC performs the following activities:

31

32 • The NRC may opt to perform steps in the initial grading (Section B), particularly if the

33 NRC examiners participated in the examination’s development or administration. 

34

35 • The NRC reviews the examination grading in accordance with the instructions in 

36 Section C and ensures that the examinations were graded in accordance with the 

37 instructions in Section B. 

38

39 • The NRC reviews any subsequent examination changes.

40

41 B. Written Examination Grading and Evaluation Instructions

42 The primary developer of the examination should perform the steps in this section; however,

43 another equally qualified individual can perform the grading if the examination developer is not 

44 available, the number of applicants is large, or if there is a need to expedite the grading 

45 process. 

46
ES-4.4, Page 2 of 6

1 The examination should be graded as soon as possible after it was administrated, in 

2 accordance with the following steps:

3

4 1. Make two copies of each applicant’s answer sheet (save one on site in the event that the 

5 original is lost and give the other to the NRC in the postexamination package) or, if 

6 submitting the postexamination package electronically to the NRC, scan the original 

7 answer sheet before grading. Ensure that the marks on the machine-graded answer 

8 sheets are legible to support copying or scanning. Perform the grading on the original 

9 answer sheet.

10

11 2. If the examinations are graded by machine, attach a copy of each applicant’s profile 

12 report to his or her answer sheet; ensure that the report or answer sheet contains the 

13 information listed in step 3. 

14

15 3. If the examinations are graded manually, document the following on the original answer 

16 sheet for each applicant:

17

18 • indicate which questions were answered incorrectly 

19 • indicate which questions (if any) were deleted 

20 • note the final number of incorrect answers by RO/SRO/Total section (if the 

21 answer sheet is more than one page long, it is helpful to note the total number of 

22 incorrect answers on each page to aid in tabulating the final grade)

23 • note the ratio of the final number of correct answers on the answer sheet to the 

24 total number of questions by RO/SRO/Total section

25 Note: These are preliminary grades and subject to change based on the NRC’s

26 acceptance of changes to the examination or examination answer key. Do not 

27 communicate these preliminary grades to the NRC staff until the operating test has been 

28 administered to each applicant.

29

30 4. If it is necessary to change a grade during the grading process, line out the original 

31 grade on the answer sheet in such a way that it remains legible. Briefly explain the 

32 reason for the change on the applicant’s answer sheet and initial the change. Do not

33 use “white-out” or other methods that obscure the change.

34

35 5. Evaluate the overall group performance on each examination question to identify any 

36 indications of a problem with the question or a deficiency in the facility licensee training 

37 program. Consider using a table to summarize the applicants’ answers on each 

38 question or including a machine automated analysis to identify items with which the 

39 applicants had problems.

40 Based on this evaluation, determine, for a faulty question, whether the question should 

41 be deleted, the answer key changed, or the question revised before reuse (or a 

42 combination of these).

43 6. For changes to the examination or answer key made during examination administration 

44 or as a result of an applicant or facility licensee comment, annotate the changes on the 
ES-4.4, Page 3 of 6

1 master examination and answer key and document the reason. Then, regrade the 

2 examinations as necessary.

3

4 7. Submit the following examination documentation (also known as the postexamination 

5 package) to the NRC chief examiner (marked “addressee only”) as soon as possible, but 

6 not more than 20 calendar days after all parts of the operator licensing initial 

7 examination have been administered:

8

9 • the graded written examinations (i.e., each applicant’s original answer and 

10 examination cover sheets) plus a clean copy of each applicant’s answer sheet 

11

12 • the master written examinations and answer keys, annotated to indicate any 

13 changes made while administering and grading the examinations

14

15 • any questions asked by the applicants and the answers given to the applicants 

16 during the written examination (ES-4.3) 

17

18 • all examination administration or postexamination review comments made by the 

19 facility licensee and the applicants after the written examination and operating 

20 tests (ES-4.3)

21

22 • the seating chart for the written examination (ES-4.3)

23

24 • documentation (through a cover letter or other correspondence) of facility 

25 licensee management or supervisor concurrence

26

27 • the results of any performance analysis for the written examination, with 

28 recommended substantive changes

29

30 • Form 1.3-1, “Examination Security Agreement,” with a pre- and post-examination 

31 signature by every individual who had detailed knowledge of any part of the 

32 operating tests or written examinations before they were administered

33

34 Note: If the completion of Form 1.3-1 is delayed due to issues associated with 

35 the collection of postexamination signatures, it is acceptable for the facility 

36 licensee to forward this form to the NRC chief examiner at a later date, as agreed 

37 to by the NRC chief examiner. Any delay in submitting the postexamination

38 package will likely result in a delay in the final licensing actions.

39

40 8. Evaluate any applicant questions and facility licensee and applicant comments and 

41 recommendations that do not result in answer key changes or question deletions to 

42 determine whether the associated test questions might benefit from editorial changes 

43 before they are used on another examination.

44

45 9. Before depositing the questions in any examination bank, revise the questions to 

46 incorporate all changes, comments, and enhancements, as appropriate.
ES-4.4, Page 4 of 6

1 C. Instructions for NRC Review of Grading and Examination Changes

2 1. Upon receipt, immediately inventory the postexamination package to ensure that all 

3 required materials have been submitted. Inform the responsible NRC supervisor of any 

4 obvious deficiencies and contact the facility licensee to determine the status of any 

5 missing documentation.

6

7 2. Discuss all grading discrepancies with the grader or previous reviewer before making 

8 any changes and document any changes by carefully lining out the original entry so that 

9 it remains legible, entering the revision with a brief explanation, and initialing the change. 

10 These changes may be documented electronically provided that the original entry is 

11 apparent, the individual making the change is identified, and an explanation for the 

12 change is included. Do not use “white-out” or other methods that obscure the original 

13 entry.

14

15 3. Review proposed examination changes and perform an independent grading review as 

16 follows:

17

18 a. If the written examination was graded by machine or using a template, ensure 

19 that the template accurately parallels the approved, finalized answer key.

20

21 b. Independently analyze each examination and answer key change made or 

22 recommended by the facility licensee or a license applicant to determine whether 

23 it is justified. During the analysis, keep in mind that both the facility licensee and 

24 the NRC had previously agreed that the written examination met the 

25 requirements of NUREG-1021 before the examination was administered (refer to

26 the examination approval letter). 

27

28 c. Despite the extensive reviews performed by both the NRC and the facility 

29 licensee before examination administration, it is possible that errors may be 

30 discovered only after an examination has been administered. The NRC will 

31 consider examination changes for the following types of errors, if identified and 

32 adequately justified by the facility licensee or an applicant:

33

34 • a question with an unclear stem that confused the applicants or did not 

35 provide all the necessary information (to assist in determining whether an

36 unclear stem confused the applicants, closely evaluate any applicant 

37 questions asked during the examination; also evaluate the question stem 

38 to determine whether the information provided could reasonably result in

39 the applicant misunderstanding the intent of the question or the validity of 

40 the answer choices)

41

42 • unintended typographical errors in a question or on the answer key

43

44 • newly discovered technical information that supports a change in the 

45 answer key

46

47 • a question testing the wrong license level (RO versus SRO) or not linked 

48 to job requirements
ES-4.4, Page 5 of 6

1 d. The NRC will not accept examination changes for the following types of question 

2 errors identified after examination administration:

3

4 • a question that does not exactly match its referenced K/A statement

5

6 • a question for which references would be needed to provide the correct 

7 answer, even though the facility licensee and the NRC previously agreed 

8 that the question should be closed-reference

9

10 • a question that tests minutiae, even though the facility licensee and the 

11 NRC previously agreed that the question did not test minutiae

12

13 • a question that contains psychometric errors that do not increase its 

14 difficulty or make the question confusing (e.g., a question with two 

15 implausible distractors or a collection of true or false answers would be 

16 unsatisfactory during preexamination review, but neither problem would 

17 justify deleting the question after examination administration)

18

19 e. If a question is determined to have two correct answers, the NRC will accept both 

20 answers as correct. However, if both answers contain conflicting information, the 

21 question will be deleted. Conflicting information is present when two answers 

22 contain plant information that cannot be true or exist within the same timeframe. 

23 For example, if a part of one answer states that operators are required to insert a 

24 manual reactor scram and a part of another answer states that a manual scram 

25 is not required, then the NRC will not accept both answers as correct and the 

26 question will be deleted, as the answers conflict. A facility cannot have a manual 

27 scram both be required and not be required at the same time.

28

29 f. If a question is determined to have three or more correct answers or there is no 

30 correct answer, the NRC will delete the question. 

31

32 g. Ensure that a valid facility licensee reference exists to support examination 

33 changes.

34

35 h. Annotate NRC-approved changes on the master examination and answer key.

36

37 i. Independently review and regrade each applicant’s examination.

38

39 j. Calculate and enter the following in the applicable Results Area on each 

40 applicant’s coversheet (Form 4.2-3, “Reactor Operator Written Examination 

41 Cover Sheet,” and Form 4.2-4, “Senior Reactor Operator Written Examination 

42 Cover Sheet”): 

43

44 • Examination Points = original test point total minus one point for each 

45 deleted question for the RO, SRO-only, and Total examination sections

46 • Applicant’s Points = the applicant’s total correct answers for the RO, 

47 SRO-only, and Total examination sections

48 • Applicant’s Grade = the applicant’s points divided by the examination 

49 points for each part of the examination (RO, SRO, and Total) 
ES-4.4, Page 6 of 6

1 For the Applicant’s Grade, do not round up or down, but instead truncate the 

2 examination score in the tenths place. For example, if an applicant correctly 

3 answers 59 of 75 questions, the score is documented as a 78.6.

4 k. Record the scores in the Written Examination Summary of the applicant’s

5 Form 5.1-2, “Individual Examination Report,” and record the name of the NRC

6 independent reviewer (person who performed step 3 of Section C).

7

8 l. In the Examiner Recommendation area of the applicant’s Form 5.1-2, “Individual 

9 Examination Report,” make a recommendation for the written examination by 

10 checking the “Pass” or “Fail” (or “Excuse/Waive” if the written examination was 

11 excused or waived in accordance with ES-2.2) block and signing and dating in 

12 the row for the written examination recommendation.

13

14 To pass the examination, the applicant must achieve an overall grade of at least 

15 80 percent, with a 70 percent or better on the SRO-only items, if applicable. 

16

17 Retake and SRO-upgrade applicants who take only the SRO exam must achieve 

18 a grade of 80 percent or higher to pass.

19

20 4. The NRC chief examiner performs an additional independent grading review using the 

21 final, approved answer key and the original applicant answer sheet (or a clean copy of 

22 the answer sheet, if necessary) for each applicant with a borderline grade. Borderline 

23 grades are defined as those between 78 and 82 percent overall, between 66 and 

24 74 percent on the SRO-only portion, and between 76 and 84 percent if the RO portion 

25 was waived.

26

27 5. The NRC chief examiner reviews the written examination results and the facility 

28 licensee’s performance analysis (if applicable) for indications of the following:

29 • deficiencies in the applicants’ training program, so that they may be addressed in 

30 the examination report

31 • poor question construction, so that the applicants are not graded unfairly, any 

32 significant problems can be addressed in the examination report, and the 

33 questions can be corrected before reuse

34 • any indications that the examination was compromised

35 6. Once the NRC grading review is complete, the NRC chief examiner makes a final 

36 recommendation and then signs and dates the NRC chief examiner block on each 

37 applicant’s Form 5.1-2.

38

39 7. The NRC chief examiner forwards the examination results to the licensing official in 

40 accordance with ES-5.1, “Issuing Operator Licenses and Postexamination Activities.”
ES-5.1, Page 1 of 20

1 5.1 ISSUING OPERATOR LICENSES AND POSTEXAMINATION 

2 ACTIVITIES

3 This examination standard describes the activities that the U.S. Nuclear Regulatory Commission 

4 (NRC) regional office performs after all parts of the operator licensing initial examination have 

5 been administered and graded in accordance with Examination Standard (ES)-3.5, 

6 “Administering Operating Tests”; ES-3.6, “Grading and Documenting Operating Tests for 

7 Operator Licensing Initial Examinations”; ES-4.3, “Administering Written Examinations”; and 

8 ES-4.4, “Grading and Documenting Written Examinations,” of this NUREG. This examination 

9 standard also includes instructions for notifying the facility licensee and applicants of the 

10 examination results and issuing licenses.

11 The NRC regional office is responsible for the activities in this examination standard. The 

12 regional office may use Form 5.1-1, “Postexamination Check Sheet,” to track completion of the 

13 administrative activities after the examinations are administered.

14 A. Compile Examination Results and Reports

15 The NRC chief examiner ensures that any operating test licensing recommendation that 

16 deviates from the nominal grading instructions in ES-3.6 (e.g., recommending a simulator test 

17 failure based on a single error with serious safety consequences or a passing grade despite 

18 multiple errors related to the same rating factor) receives written concurrence from the Office of 

19 Nuclear Reactor Regulation (NRR) operator licensing program office before completing the 

20 licensing or denial action.

21 The NRC chief examiner ensures that the examination results package is complete and 

22 contains the following items for each applicant: 

23 • Form 5.1-2, “Individual Examination Report”

24 • operating test: 

25 – Form 3.3-1, “Scenario Outline,” reflecting the “as-run” scenarios (pen-and-ink 

26 markups of the original scenario outlines are acceptable)

27 – if the applicant failed the simulator operating test: Form 3.3 2, “Required 

28 Operator Actions,” reflecting the “as-run” scenarios but without any examiner 

29 notes on applicant performance. (Pen-and-ink markups of the original/approved 

30 scenarios are acceptable)

31 – Form 3.6-1, “Job Performance Measure Grade Report”

32 – Form 3.6-2, “Reactor Operator Simulator Scenario Grade Report,” or Form 3.6-3, 

33 “Senior Reactor Operator Simulator Scenario Grade Report”

34 – Form 3.6-4, “Operating Test Comments,” and any other supporting 

35 documentation

36 – if the applicant failed the overall walkthrough or administrative topics job 

37 performance measures (JPMs): a copy of all JPMs that the applicant failed, 

38 reflecting the “as-administered” JPM conditions but without any examiner notes 
ES-5.1, Page 2 of 20

1 on applicant performance (pen-and-ink markups of the original/approved JPMs 

2 are acceptable)

3 • written examination: 

4 – written examination cover sheet

5 – original answer sheet

6 – if the applicant failed the written examination: a copy of the master written 

7 examination and answer key 

8 The NRC chief examiner records the overall and individual results of the operating test and 

9 written examination on Form 5.1-3, “Power Plant Examination Results Summary.”

10 The NRC chief examiner provides the examination results packages to the NRC regional 

11 operator licensing assistant to prepare the respective applicant letters—preliminary results 

12 letter, denial letter, pass letter, or license—and then forwards the examination results packages 

13 and letters to the regional licensing official. 

14 B. Licensing Action

15 The NRC Regional Administrator or his or her designee, who must be at or above the Branch 

16 Chief level, makes the licensing decision for each applicant who took the examination; 

17 short-term designees shall not make licensing decisions unless specifically authorized in writing, 

18 with any limitations, by the Regional Administrator. This individual is referred to as the licensing 

19 official throughout this examination standard. 

20 The licensing official performs the following activities:

21 • Independently review each applicant’s examination results and associated examiner 

22 recommendations.

23 • Check the “Issue License” or “Deny License” block in the “Licensing Action” section of on 

24 each applicant’s Form 5.1-2, and sign and date each form.

25 • Sign each applicant’s license, denial letter, preliminary results letter, or pass letter (as 

26 applicable).

27 If the licensing official does not believe that the operating test or written examination 

28 documentation supports the final recommendation, he or she must consult with the examiner of 

29 record and the NRC chief examiner to discuss and resolve any disagreements.

30 If the licensing official overturns the examiner’s recommendation, the licensing official must 

31 annotate the associated examination material as follows:

32 • For the written examination, line out and initial the master written examination and 

33 answer key and provide an explanation of the change on the answer key. 

34

35 • For the operating test, line out and initial the affected performance deficiency evaluations 

36 and ensure that the new performance deficiency evaluations and an explanation of the 
ES-5.1, Page 3 of 20

1 changes are entered on to the Form 3.6-4 and attach the comment form to the 

2 applicant’s Form 5.1-2.

3 C. Issue Licenses 

4 Final license issuance involves the following activities:

5 1. Before issuing a license, the NRC regional office shall ensure that the applicant has 

6 satisfied the health requirement of Title 10 of the Code of Federal Regulations

7 (10 CFR) 55.33, “Disposition of an initial application,” through either (1) a certification by 

8 the facility licensee that the applicant’s medical condition and general health will not 

9 adversely affect the performance of assigned operator job duties or cause operational 

10 errors endangering public health and safety or (2) a Commission determination, 

11 considering the recommendations and supporting evidence of the facility licensee and of 

12 the examining physician, that specific license conditions will accommodate any medical 

13 defects of the applicant. The instructions in ES-2.2, “Applications, Medical 

14 Requirements, and Waiver and Excusal of Examination and Test Requirements,” 

15 provide additional information. 

16

17 2. Before issuing a license, the NRC regional office must ensure that the applicant has 

18 completed any deferred eligibility items.

19

20 3. Before issuing a license, the NRC regional office must ensure that senior reactor 

21 operator (SRO)-upgrade applicants and individuals who initially received a pass letter 

22 are up to date in the facility licensee’s requalification training program. 

23

24 4. The NRC regional office issues a license to the applicant in the form of a letter. 

25 Letter 5.1-1, “Sample Reactor Operator License,” and Letter 5.1-2, “Sample Senior 

26 Reactor Operator License,” of this examination standard provide sample licenses.

27 D. Notify Applicants of Examination Results

28 The NRC regional office expects to complete the actions for initial operator license applications 

29 within 30 days of receiving the postexamination package from the facility licensee. 

30 The NRC regional office notifies the applicants of the examination results, after the licensing 

31 official reviews and approves them, by sending the applicable letter (described below) along 

32 with the following materials (sent either electronically using the Electronic Information

33 Exchange, a CD-ROM, or an electronic storage device or in hard copy):

34 • a copy of Form 5.1-2, Form 3.6-1, Form 3.6-2/3, and Form 3.6-4

35 • a copy of Form 3.3-1

36 • a copy of Form 3.3-2 (if the applicant failed the simulator operating test), reflecting the 

37 “as-run” scenario conditions but without any examiner notes on the applicant’s 

38 performance (pen-and-ink markups of the original, approved scenarios are acceptable)

39 • a copy of all JPMs that the applicant failed (if the applicant failed the operating test in the 

40 overall walkthrough or administrative topics JPMs), reflecting the “as-administered” JPM 

41 conditions but without any examiner notes on the applicant’s performance (pen-and-ink 

42 markups of the original, approved JPMs are acceptable)
ES-5.1, Page 4 of 20

1 • a copy of the applicant’s written examination cover and answer sheets

2 • a copy of the master written examination and answer key (if the applicant failed the 

3 written examination)

4 1. Preliminary Results Letter

5 Under 10 CFR 55.31(b), the Commission may at any time after the application has been filed 

6 require further information under oath or affirmation in order to enable it to determine whether to 

7 grant or deny the application. A preliminary results letter is one such method of gathering 

8 further information (see Letter 5.1-3, “Sample Preliminary Results Letter”). The NRC issues a

9 preliminary results letter when the NRC regional office grading results indicate that the applicant 

10 failed any portion of the operator licensing initial examination. This letter gives the applicant the 

11 option to provide further information to be used in an informal NRC staff review. The applicant 

12 should respond to the 10 CFR 55.31(b) request (i.e., the preliminary results letter) within 

13 20 days from the date of the request, or within such other time as the letter may specify. 

14 Inquiries, communications, and reports concerning the application must be filed as specified in 

15 10 CFR 55.5, “Communications.” To account for mail delivery, if applicable, the region should 

16 delay issuing further correspondence for an additional 5 calendar days to ensure that the 

17 20 days have expired. 

18 2. Denial Letter

19 The NRC issues a denial letter when the Commission is unable to make the findings in 

20 10 CFR 55.33(a) (addressing health and passing the written examination and operating test) or, 

21 if applicable, the Commission is unable to approve a conditional license under 10 CFR 55.33(b). 

22 As a prerequisite to a denial letter, the staff will typically wait until, as applicable, an applicant 

23 does not request an informal NRC staff review within 20 days (or within such other time as may 

24 be specified), or the NRC staff upholds the preliminary results following an informal NRC staff 

25 review.

26 A denial letter is required under 10 CFR 2.103(b). If applicable, this letter may provide 

27 instructions on how the applicant may reapply under 10 CFR 55.35, “Re-applications” (for

28 failures of the written examination or operating test, or both) or under 10 CFR 55.31(c) (for 

29 medical conditions). Because 10 CFR 55.35(a) sets the permissible earliest time for filing a

30 reapplication based on “the date of the denial,” some applicants may elect to respond to a 

31 10 CFR 55.31(b) request for informal NRC staff review by asking the NRC to immediately send 

32 the denial letter. The NRC should promptly send the denial letter and firmly set the date of the 

33 denial to be used with regard to 10 CFR 55.35(a), as well as the date of denial to demand a 

34 hearing under 10 CFR 2.103(b). An applicant may reapply under 10 CFR 55.35 or supplement 

35 under 10 CFR 55.31(c) (or both) during the pendency of a hearing on a previous application. 

36 An applicant is not required to waive or decline to exercise the right to demand a hearing as a 

37 prerequisite to reapplying.

38 A sample denial letter is available as Letter 5.1-4, “Sample Denial Letter,” of this examination 

39 standard. Note: If the NRC staff upholds the preliminary results following an informal NRC staff 

40 review, the agency sends a slightly different version of this denial letter to the applicant (see

41 Operator Licensing Manual Chapter (OLMC) 500, “Processing Requests for Administrative 

42 Reviews and Demands for Hearings” (Agencywide Documents Access and Management 

43 System (ADAMS) Accession No. ML20230A201) or the most current revision of this OLMC).
ES-5.1, Page 5 of 20

1 3. Pass Letter

2 The NRC issues a pass letter (see Letter 5.1-5, “Sample Pass Letter) if an applicant has passed 

3 the requisite written examination and operating test in accordance with 10 CFR 55.41, “Written 

4 examination: Operators,” and 10 CFR 55.45, “Operating tests,” or 10 CFR 55.43, “Written 

5 examination: Senior operators,” and 10 CFR 55.45, and the applicant’s general medical 

6 condition meets the minimum standards under 10 CFR 55.33(a)(1) or may be accommodated 

7 with appropriate conditions under 10 CFR 55.33(b), but to date the applicant has not completed 

8 all of the elements of 10 CFR 55.31, “How to apply.” This letter notifies the applicant that the 

9 NRC will issue his or her license when the incomplete (i.e., deferred) items are resolved. The 

10 NRC regional office will issue a license when the applicant or facility licensee, as appropriate, 

11 completes the deferred items.

12 E. Notify Facility Licensee of Examination Results

13 The NRC regional office notifies the facility licensee of the examination results after the 

14 licensing official reviews and approves them.

15 The NRC regional office should notify the facility licensee’s designated representative of the 

16 examination results by telephone and confirm the results by mailing or sending electronically a 

17 copy of Form 5.1-3 under a separate cover letter. For each applicant who failed or 

18 demonstrated deficiencies, the regional office will also send the facility licensee a copy of the 

19 applicant’s Form 5.1-2, Form 3.6-1, Form 3.6-2/3, Form 3.6-4, and written examination answer 

20 sheet. 

21 The NRC expects the facility licensee to use its systems approach to training process to analyze 

22 and determine the need for additional training for any applicant who passed the NRC 

23 examination but had knowledge or performance deficiencies, or both. 

24 NRC regional management, or the licensing official, should consider calling the facility licensee 

25 management counterpart to discuss the examination outcome and lessons learned. Any 

26 pertinent feedback on the examination process should be forwarded to the NRR operator 

27 licensing program office for consideration.

28 F. Special Circumstances for Senior Reactor Operator-Upgrade Applicants

29 If an SRO-upgrade applicant did not complete the SRO-upgrade training program or failed the 

30 SRO-upgrade examination, NRC regional management shall ensure that the reactor operator 

31 (RO) licensee complies with the requirements of 10 CFR 55.53(e), (f), and (h) and 

32 10 CFR 55.59(a) before resuming active duties as an RO.

33 Similarly, the NRC regional office shall ensure that SRO-upgrade applicants who passed and 

34 did not participate in RO requalification training while they were enrolled in the SRO-upgrade 

35 training program comply with 10 CFR 55.59(a). If an applicant missed the annual operating test 

36 or the comprehensive requalification written examination required by 10 CFR 55.59(a)(2) and 

37 then did not take the RO portion of the licensing written examination, the applicant must 

38 complete additional training in accordance with 10 CFR 55.59(b) and must make up the missed 

39 requalification examination to verify proficiency in the topics under 10 CFR 55.41 before 

40 resuming licensed duties as an RO or commencing duties as an SRO (which requires testing on 

41 items under 10 CFR 55.41 and 10 CFR 55.43). The NRC would consider the requirements of 

42 10 CFR 55.59(a)(2)(i) to be satisfied if the applicant repeats the applicable portions (to be 
ES-5.1, Page 6 of 20

1 determined using a systems approach to training) of the license training program and passes a 

2 comprehensive audit examination covering the topics required by 10 CFR 55.41.

3 G. Determine Quality of Submitted Examination

4 After examination administration and once all postexamination comments have been resolved, 

5 the NRC regional office will determine the quality of the submitted written examination and 

6 operating test material based on the following for documentation in the examination report (refer 

7 to the most current revision of OLMC 510, “Operator Licensing Examination Reports” (Revision 

8 0 is available at ADAMS Accession No. MLXXXXXXX):

9 1. For the submitted written examination questions (RO and SRO questions shall be 

10 considered separately), following the written examination review performed in ES-2.3, 

11 “Reviewing and Approving Operator Licensing Initial Examinations,” Section C.3, and 

12 documented on Form 2.3-5, “Written Examination Review Worksheet,” for each 

13 submitted question, determine the percentage of submitted questions that were 

14 determined to be unsatisfactory. Include any questions that were deleted during the 

15 grading process, or for which the answer key had to be changed, in the count of 

16 unsatisfactory questions.

17 Note: As long as changes were incorporated to result in satisfactory test items, do not 

18 count any presubmittal sample test items reviewed by the NRC before the final licensee 

19 examination submittal. 

20 Note: Do not count questions from past NRC examinations (at any facility) unless the 

21 facility licensee’s current use caused the current unsatisfactory status after the last NRC 

22 approval of the test item. (For example, the question’s reference changed, but the 

23 question was not revised accordingly, or the question was previously associated with a 

24 different knowledge and abilities (K/A) statement and its current K/A pairing results in a 

25 K/A mismatch flaw.)

26 Note: Do not count flaws of a less serious nature (e.g., editorial clarifications or 

27 enhancements, single implausible distractors) that were corrected before the 

28 examination was administered.

29 2. To determine the total percentage of unsatisfactory operating test items, total the 

30 number of JPMs and scenario events that were marked unsatisfactory following the 

31 operating test review performed in ES-2.3, Section C.4, and divide by the total number of 

32 JPMs and scenario events (i.e., 4 of 15 JPMs were marked unsatisfactory (U), and 3 of 

33 33 scenario events were marked U, resulting in 7 of 48 or 14.6 percent). Update line 

34 items on Form 2.3-3, “Operating Test Review Worksheets,” with postexamination 

35 changes and include those changes in this calculation.

36

37 Note: If the review indicated that a scenario event did not require verifiable action(s), do 

38 not include this event as part of the total number of events unless the event was used to 

39 meet one of the minimum required for an applicant according to Form 3.4-1, “Events and 

40 Evolutions Checklist.” 

41 Note: Do not count JPMs and scenario events that were previously used on past NRC 

42 examinations and required replacement or substantial modification unless the facility 

43 licensee caused the current unacceptable flaw after the last NRC approval of the test 
ES-5.1, Page 7 of 20

1 item. (For example, the JPM reference changed, but the JPM was not revised 

2 accordingly.)

3 Note: Do not count flaws of a less serious nature (e.g., editorial clarifications or 

4 enhancements) that were corrected before the examination was administered.

5 H. Reference Material

6 If an applicant was denied a license based on an examination failure, the reference materials 

7 should be retained during the period in which the applicant may request an informal NRC staff 

8 review, demand a hearing, or both. If an applicant requests an informal NRC staff review,

9 demands a hearing, or both, the NRC chief examiner shall consult with the assigned attorney 

10 before returning or destroying any document related to the examination.

11 I. Examination Report and NRC Recordkeeping

12 The NRC should prepare the examination report using the instructions in the current revision of 

13 OLMC 510 and follow the instructions for NRC examination recordkeeping activities in the 

14 current revision of OLMC 520, “Operator Licensing Examination Records Documentation”

15 (OLMC 510, Revision 0 is available at ADAMS Accession No. MLXXXXXXX and OLMC 520, 

16 Revision 0 is available at MLXXXXXXXX).

17 J. Forms and Letters

18 Form 5.1-1 Postexamination Check Sheet

19 Form 5.1-2 Individual Examination Report

20 Form 5.1-3 Power Plant Examination Results Summary

21 Letter 5.1-1 Sample Reactor Operator License

22 Letter 5.1-2 Sample Senior Reactor Operator License

23 Letter 5.1-3 Sample Preliminary Results Letter

24 Letter 5.1-4 Sample Denial Letter

25 Letter 5.1-5 Sample Pass Letter

26
ES-5.1, Page 8 of 20

1 Form 5.1-1 Postexamination Check Sheet

2

3

4

Postexamination Check Sheet

Facility: Dates of Examination:

Activity Description Date 

Complete

1. *Received postexamination package from facility licensee and 

verified complete. (ES-4.4)

2. *Reviewed and incorporated any necessary facility and applicant 

written examination comments. NRC grading of written examination 

completed. (ES-4.4)

3. *Reviewed and incorporated any necessary facility and applicant 

operating test comments. NRC grading of operating test completed. 

(ES-3.6)

4. *Completed NRC chief examiner review of operating test and written 

examination grading. (ES-3.6 and ES-4.4)

5. Completed licensing official review.

6. Mailed licenses, preliminary results, and pass letters.

7. Notified facility licensee of results.

8. Issued the examination report (refer to Operator Licensing Manual 

Chapter (OLMC) 510, “Operator Licensing Examination Records”).

9. Returned reference material after final resolution of any informal 

NRC staff reviews, hearing demands, or both.

10. Performed examination recordkeeping activities (refer to OLMC 520, 

“Operator Licensing Examination Records Documentation,” and 

Management Directive 3.53, “NRC Records and Document 

Management Program”).

* Activity may not be applicable for a retake examination. If this activity does not apply, 

place an “N/A” in the Date Complete column.
ES-5.1, Page 9 of 20

1 Form 5.1-2 Individual Examination Report

2 PRIVACY ACT INFORMATION—FOR OFFICIAL USE ONLY

U.S. Nuclear Regulatory Commission 

Individual Examination Report

Applicant’s Name: Docket Number: 55-

I R Examination Type (Initial or Retake) Facility Name: 

Reactor Operator

Facility

Description

Hot

Senior Reactor Operator—Instant Cold

Senior Reactor Operator—Upgrade BWR

Senior Reactor Operator—Limited to Fuel Handling

PWR

______

Written Examination Summary

Name—NRC Independent Reviewer: Overall SRO-only

Applicant 

Points

Examination 

Points

Applicant 

Grade (%)

Name—NRC Chief Examiner: 

Date Administered: 

Operating Test Summary

Name—Examiner of Record: Dates Administered: 

Overall Walkthrough (S)atisfactory, (U)nsatisfactory, (E)xcused, or (W)aived

Administrative Topics (S, U, E, or W)

Simulator Operating Test (S, U, E, or W)

Examiner Recommendations

Pass Fail Excuse/Waive Signature and Date

Written Examination, 

Independent Reviewer

Operating Test, 

Examiner of Record

Final Recommendation,

Chief Examiner/Designee

Licensing Action

Issue License Licensing Official’s Signature Date

Deny License

3 PRIVACY ACT INFORMATION—FOR OFFICIAL USE ONLY

4
ES-5.1, Page 10 of 20

1 Form 5.1-3 Power Plant Examination Results Summary

2 PRIVACY ACT INFORMATION—FOR OFFICIAL USE ONLY

Power Plant Examination Results Summary

Facility: Plant Status: Hot Cold 

Written Examination Date:

Prepared by: Facility NRC 

Operating Test Date(s):

Prepared by: Facility NRC 

NRC Examiners:

Overall Results

# Total Applicants # Passed % Passed # Failed % Failed

RO

SRO

Individual Results

Name Docket # 

55-( )

Type(1) Written Grade

Overall/SRO-Only

Operating Test(2)

W-T ADM SIM

NOTES:

(1) 1=RO; 2=SRO-I; 3=SRO-U; 4=RO-Retake; 5=SRO-I-Retake; 6=SRO-U-Retake; 7=SRO-Fuel Handling

(2) P=Passed; F=Failed; W=Waived; E=Excused

3 PRIVACY ACT INFORMATION—FOR OFFICIAL USE ONLY

4
ES-5.1, Page 11 of 20

1 PRIVACY ACT INFORMATION—FOR OFFICIAL USE ONLY

Power Plant Examination Results Summary

(Continuation Sheet)

Facility:

Written Examination/Operating Test Date(s):

Individual Results

Name Docket # 

55-( )

Type(1) Written Grade

Overall/SRO-Only

Operating Test(2)

W-T ADM SIM

NOTES:

(1) 1=RO; 2=SRO-I; 3=SRO-U; 4=RO-Retake; 5=SRO-I-Retake; 6=SRO-U-Retake; 7=SRO-Fuel Handling

(2) P=Passed; F=Failed; W=Waived; E=Excused

2 PRIVACY ACT INFORMATION—FOR OFFICIAL USE ONLY

3
ES-5.1, Page 12 of 20

1 Letter 5.1-1 Sample Reactor Operator License

2

3 NRC Letterhead

4

5 (Date)

6

7 LICENSE

8

9

10

11 (Applicant’s Name)

12 (Street Address)

13 (City, State ZIP Code)

14

15 Under the Atomic Energy Act of 1954, as amended, and the Energy Reorganization Act of 1974, 

16 as amended, and subject to the conditions and limitations incorporated herein, the U.S. Nuclear 

17 Regulatory Commission hereby licenses you to manipulate all controls of the (Name of facility, 

18 facility license number).

19

20 Your license number is OP-(number), and your docket number is 55-(number). The effective 

21 date is (date). Unless this license is terminated, renewed, or upgraded sooner, it shall expire 

22 6 years from the effective date.

23

24 This license is subject to the provisions of Title 10 of the Code of Federal Regulations

25 (10 CFR) 55.53, “Conditions of licenses,” with the same force and effect as if fully set forth 

26 herein.

27

28 While performing licensed duties, you shall observe the operating procedures and other 

29 conditions specified in the facility license authorizing operation of the facility. [You shall also 

30 comply with the following condition(s):

31

32 • You shall wear corrective lenses while performing the activities for which you are 

33 licensed.]

34

35 The issuance of this license is based on an examination of your qualifications, including the 

36 representations and information contained in your application for this license.

37

38 A copy of this license has been made available to the facility licensee.

39

40 For the U.S. Nuclear Regulatory Commission,

41

42

43 (Name and title of licensing official)

44

45 Docket No.: 55-(number)

46

47 cc: (Facility representative who signed the applicant’s NRC Form 398)

48

49 [ ] Include for any license conditions necessary to accommodate medical defects.

50
ES-5.1, Page 13 of 20

1 Letter 5.1-2 Sample Senior Reactor Operator License

2 NRC Letterhead

3

4 (Date)

5

6 LICENSE

7

8 (Applicant’s Name)

9 (Street Address)

10 (City, State ZIP Code)

11

12 Under the Atomic Energy Act of 1954, as amended, and the Energy Reorganization Act of 1974, 

13 as amended, and subject to the conditions and limitations incorporated herein, the U.S. Nuclear 

14 Regulatory Commission hereby licenses you to direct the [licensed] [[fuel handling]] activities of 

15 [licensed] operators at, and to manipulate [all] [[fuel handling]] controls of, the (Name of facility, 

16 facility license number).

17

18 Your license number is SOP-(number), and your docket number is 55-(number). The effective 

19 date is (date). Unless this license is terminated, renewed, or upgraded sooner, it shall expire 

20 6 years from the effective date.

21

22 This license is subject to the provisions of Title 10 of the Code of Federal Regulations

23 (10 CFR) 55.53, “Conditions of licenses,” with the same force and effect as if fully set forth 

24 herein.

25

26 While performing licensed duties, you shall observe the operating procedures and other 

27 conditions specified in the facility license authorizing operation of the facility. [[[You shall also 

28 comply with the following condition(s):

29

30 • You shall wear corrective lenses while performing the activities for which you are 

31 licensed.]]]

32

33 The issuance of this license is based on an examination of your qualifications, including the 

34 representations and information contained in your application for this license.

35

36 A copy of this license has been made available to the facility licensee.

37

38 For the U.S. Nuclear Regulatory Commission,

39

40

41 (Name and title of licensing official)

42

43 Docket No.: 55-(number)

44

45 cc: (Facility representative who signed the applicant’s NRC Form 398)

46

47 [ ] Include only for unrestricted senior operators.

48 [[ ]] Include only for senior operators limited to fuel handling under 10 CFR 55.53(c).

49 [[[ ]]] Include for any license conditions necessary to accommodate medical defects.

50
ES-5.1, Page 14 of 20

1 Letter 5.1-3 Sample Preliminary Results Letter

2

3 NRC Letterhead

4 [Date]

5 [Applicant’s Name]

6 [Street Address]

7 [City, State ZIP Code]

8

9

10 Dear [Mr./Ms. Applicant’s Name]:

11

12 This is to inform you that the preliminary results of the [operating test, written examination, or 

13 operating test and written examination] taken by you on [date(s)], in connection with your 

14 application for a [reactor operator, senior reactor operator, or limited senior reactor operator] 

15 license for the [facility name], indicate that you did not pass that [test, examination, or test and 

16 examination]. Enclosed is a copy of the preliminary results of your [operating test, written 

17 examination, or operating test and written examination], indicating the areas in which you 

18 exhibited deficiencies. [A copy of the master answer key is also provided.]

19

20 You have three options: (1) you may take no action, in which case the preliminary results will 

21 become the final results and the U.S. Nuclear Regulatory Commission (NRC) will issue you a 

22 letter denying your license application; (2) you may, within 20 days of the date of this letter, 

23 provide additional information to the NRC and request that the NRC staff conduct an informal 

24 review of the preliminary results in light of that information; or (3) you may request that the NRC 

25 issue you a letter denying your license application without delay. The letter denying your 

26 license application will inform you of your right to demand a hearing on the denial and provide

27 you with instructions on how to demand a hearing. The letter will also explain when you may file 

28 a new application.

29

30 A request for an NRC staff informal review must identify the portions of your [operating test, 

31 written examination, or operating test and written examination] that you believe were graded 

32 incorrectly. In addition, you must provide the basis, including supporting documentation (such 

33 as procedures, instructions, computer printouts, and chart traces), in as much detail as possible, 

34 for your arguments that certain of your responses were graded incorrectly. The NRC will review 

35 your arguments and finalize the grading of your [operating test, written examination, or 

36 operating test and written examination]. Depending on the final results, the NRC will send you 

37 either (1) a notification of passing the licensing examination or (2) a letter denying your license 

38 application, as described above. Responses to this preliminary results letter must be submitted 

39 to

40

41 [the Regional Administrator of Region I. Submissions by mail or hand delivery must be 

42 addressed to the Administrator at U.S. Nuclear Regulatory Commission, 2100 Renaissance 

43 Boulevard, Suite 100, King of Prussia, PA 19406–2713; where email is appropriate it should be 

44 addressed to RidsRgn1MailCenter.Resource@nrc.gov.]

45

46 [the Regional Administrator of Region II. Submissions by mail or hand delivery must be 

47 addressed to the Regional Administrator at U.S. Nuclear Regulatory Commission, 245 

48 Peachtree Center Avenue, NE., Suite 1200, Atlanta, Georgia 30303–1257; where e-mail is 

49 appropriate, it should be addressed to RidsRgn2MailCenter@nrc.gov.]

50
ES-5.1, Page 15 of 20

1 [the Regional Administrator of Region III. Submissions by mail or hand delivery must be 

2 addressed to the Administrator at U.S. Nuclear Regulatory Commission, 2443 Warrenville 

3 Road, Suite 210, Lisle, IL 60532–4352; where e-mail is appropriate it should be addressed to 

4 RidsRgn3MailCenter@nrc.gov.]

5

6 [the Regional Administrator of Region IV. Submission by mail or hand delivery must be 

7 addressed to the Administrator at U.S. Nuclear Regulatory Commission, 1600 E. Lamar Blvd., 

8 Arlington, TX 76011–4511; where email is appropriate, it should be addressed to 

9 RidsRgn4MailCenter@nrc.gov.]

10

11 If you have any questions, please contact me at [number]. 

12

13

14

15

16 Sincerely,

17

18

19

20 [Name], Chief 

21 [Branch]

22 [Division]

23

24 Docket No.: 55-[number]

25

26 Enclosure:

27 As stated

28

29 cc w/o enclosure: Senior Facility Licensee Representative

30 cc w/enclosure: Facility Licensee Training Manager

31

32

33 CERTIFIED MAIL—RETURN RECEIPT REQUESTED

34
ES-5.1, Page 16 of 20

1 Letter 5.1-4 Sample Denial Letter

2

3 NRC Letterhead

4

5 [Date]

6 [Applicant’s Name]

7 [Street Address]

8 [City, State Zip Code]

9

10

11 Dear [Mr./Ms. Applicant’s name]:

12

13 [By letter dated [date], the U.S. Nuclear Regulatory Commission (NRC) informed you of the 

14 preliminary results of the [operating test, written examination, or operating test and written 

15 examination] taken by you on [date(s)], in connection with your application for a [reactor 

16 operator, senior reactor operator, or limited senior reactor operator] license for the [facility 

17 name] that indicated that you did not pass that [test, examination, or test and examination]. In 

18 response, [you did not, within 20 days of the date of that letter, provide additional information to 

19 the NRC and request that the NRC staff conduct an informal review of the preliminary results in 

20 light of that information] [you requested that the NRC issue you a letter denying your license 

21 application without delay]. Consequently, the preliminary results of your [operating test, written 

22 examination, or operating test and written examination] now become the final results and a 

23 denial of your license application.]

24

25 Under Title 10 of the Code of Federal Regulations (10 CFR) 55.33, “Disposition of an initial 

26 application,” the NRC will approve an initial application for [a reactor operator license/a senior 

27 reactor operator license] if it finds that (1) the applicant’s medical condition and general health 

28 will not adversely affect the performance of assigned operator job duties or cause operational 

29 errors endangering public health and safety and (2) the applicant has passed the requisite 

30 written examination and operating test. Your application was denied because the NRC 

31 determined that[, considering the recommendations and supporting evidence of the facility 

32 licensee and of the examining physician, you do not satisfy the health requirement/you did not 

33 pass [the written examination/the operating test/either the written examination or the operating 

34 test].

35

36 * Under 10 CFR 55.31(c), you may submit a further medical report at any time as a supplement 

37 to your application.

38

39 ** Under 10 CFR 55.35, “Re-applications,” you may file a new application [2/6/24] months after 

40 the date of this letter.

41

42 [Because you passed the written examination but did not pass the operating test, you may 

43 request, in the new application, to be excused from reexamination on the written examination. 

44 The NRC may grant this request at the agency’s discretion if it determines that sufficient 

45 justification is presented.]

46

47 [Because you passed the operating test but did not pass the written examination, you may 

48 request, in the new application, to be excused from reexamination on the operating test. The 

49 NRC may grant this request at the agency’s discretion if it determines that sufficient justification 

50 is presented.]
ES-5.1, Page 17 of 20

1

2 [Because you passed the written examination and the (administrative topics, control 

3 room/in-plant systems, simulator) part of the operating test but did not pass the remainder of the 

4 operating test, you may request, in the new application, to be excused from reexamination on 

5 the written examination and the (administrative topics, control room/in-plant systems, simulator) 

6 part of the operating test. The NRC may grant this request at the agency’s discretion if it 

7 determines that sufficient justification is presented.]

8

9 [Because you did not pass either the written examination or the operating test, you will be 

10 required to retake both the written examination and the operating test.]

11

12 Under 10 CFR 2.103(b)(2), you may demand a hearing on this denial of your application within 

13 20 days after the date of this letter. Under 10 CFR 2.307(a), you may request an extension of 

14 this time limit if you can show good cause.

15

16 A demand for a hearing shall be filed in accordance with 10 CFR Part 2, “Agency Rules of 

17 Practice and Procedure,” which is accessible electronically from the NRC Library on the NRC’s 

18 Web site at https://www.nrc.gov/reading-rm/doc-collections/cfr/. Generally, a demand for a 

19 hearing should explain why you believe that the NRC’s denial of your application was in error 

20 and why you believe that you have, in fact, satisfied the requirements for license issuance.

21

22 [Consult with the Office of the General Counsel/regional counsel for the most up-to-date E-Filing 

23 language.]

24

25 A demand for a hearing must be filed in accordance with the NRC’s E-Filing rule (Volume 72 of 

26 the Federal Register, page 49139; August 28, 2007). The E-Filing process requires participants 

27 to submit and serve all adjudicatory documents over the Internet, or in some cases to mail 

28 copies on electronic storage media. Detailed guidance on making electronic submissions may 

29 be found in the Guidance for Electronic Submissions to the NRC and on the NRC Web site at

30 https://www.nrc.gov/site-help/e-submittals.html. Participants may not submit paper copies of 

31 their filings unless they seek an exemption in accordance with the procedures described below.

32

33 To comply with the procedural requirements of E-Filing, at least 10 days before the filing 

34 deadline, the participant should contact the Office of the Secretary by e-mail at 

35 hearing.docket@nrc.gov or by telephone at 301-415-1677 to (1) request a digital identification 

36 (ID) certificate, which allows the participant (or its counsel or representative) to digitally sign 

37 submissions and access the E-Filing system for any proceeding in which it is participating and 

38 (2) advise the Secretary that the participant will be submitting a demand for a hearing. Based 

39 upon this information, the Secretary will establish an electronic docket for the hearing in this 

40 proceeding if the Secretary has not already established an electronic docket. 

41

42 Information about applying for a digital ID certificate is available on the NRC’s public Web site at 

43 https://www.nrc.gov/site-help/e-submittals/getting-started.html. Once a participant has obtained 

44 a digital ID certificate and a docket has been created, the participant can then submit a demand 

45 for a hearing. Submissions must be in Portable Document Format (PDF). Additional guidance 

46 on PDF submissions is available on the NRC’s public Web site at https://www.nrc.gov/site47 help/electronic-sub-ref-mat.html. A filing is considered complete at the time the document is 

48 submitted through the NRC’s E-Filing system. To be timely, an electronic filing must be 

49 submitted to the E-Filing system no later than 11:59 p.m. Eastern Time on the due date. Upon 

50 receipt of a transmission, the E-Filing system time-stamps the document and sends the 

51 submitter an e-mail notice confirming receipt of the document. The E-Filing system also 
ES-5.1, Page 18 of 20

1 distributes an e-mail notice that provides access to the document to the NRC’s Office of the 

2 General Counsel and any others who have advised the Office of the Secretary that they wish to 

3 participate in the proceeding, so that the filer need not serve the document on those participants 

4 separately. Therefore, applicants and other participants (or their counsel or representative) 

5 must apply for and receive a digital ID certificate before adjudicatory documents are filed so that 

6 they can obtain access to the documents through the E-Filing system.

7

8 A person filing electronically using the NRC’s adjudicatory E-Filing system may seek assistance 

9 by contacting the NRC’s Electronic Filing Help Desk through the “Contact Us” link located on the 

10 NRC’s public Web site at https://www.nrc.gov/site-help/e-submittals.html, by e-mail to 

11 MSHD.Resource@nrc.gov, or by a toll-free call at 1-866-672-7640. The NRC Electronic Filing 

12 Help Desk is available between 9 a.m. and 6 p.m., Eastern Time, Monday through Friday, 

13 excluding Federal Government holidays. 

14

15 Participants who believe that they have a good cause for not submitting documents 

16 electronically must file an exemption request, in accordance with 10 CFR 2.302(g), with their 

17 initial paper filing stating why there is good cause for not filing electronically and requesting 

18 authorization to continue to submit documents in paper format. Such filings must be submitted 

19 by (1) first-class mail addressed to the Office of the Secretary of the Commission, U.S. Nuclear 

20 Regulatory Commission, Washington, DC 20555-0001, Attention: Rulemaking and 

21 Adjudications Staff, or (2) courier, express mail, or expedited delivery service to the Office of the 

22 Secretary, 11555 Rockville Pike, Rockville, MD 20852, Attention: Rulemaking and 

23 Adjudications Staff. Participants filing adjudicatory documents in this manner are responsible 

24 for serving the document on all other participants. Filing is considered complete by first-class 

25 mail as of the time of deposit in the mail, or by courier, express mail, or expedited delivery 

26 service upon depositing the document with the provider of the service. A presiding officer, 

27 having granted an exemption request from using E-Filing, may require a participant or party to 

28 use E-Filing if the presiding officer subsequently determines that the reason for granting the 

29 exemption from use of E-Filing no longer exists. 

30

31 Documents submitted in adjudicatory proceedings will appear in the NRC’s electronic hearing 

32 docket, which is available to the public at https://adams.nrc.gov/ehd, unless excluded pursuant 

33 to an order of the Commission or the presiding officer. If you do not have an NRC-issued digital 

34 ID certificate as described above, click “cancel” when the link requests certificates and you will 

35 be automatically directed to the NRC’s electronic hearing dockets where you will be able to 

36 access any publicly available documents in a particular hearing docket. Participants are 

37 requested not to include personal privacy information, such as social security numbers, home 

38 addresses, or personal phone numbers in their filings, unless an NRC regulation or other law 

39 requires submission of such information. For example, in some instances, individuals provide 

40 home addresses in order to demonstrate proximity to a facility or site. With respect to 

41 copyrighted works, except for limited excerpts that serve the purpose of the adjudicatory filings 

42 and would constitute a Fair Use application, participants are requested not to include 

43 copyrighted materials in their submission. 

44

45
ES-5.1, Page 19 of 20

1 If you have any questions, please contact me at [telephone number].

2

3

4

5 Sincerely,

6

7

8

9 [Name], Chief 

10 [Branch]

11 [Division]

12

13

14 Docket No.: 55-[number]

15

16 Enclosure: As stated

17

18 cc w/o enclosure: Senior Facility Licensee Representative

19 cc w/enclosure: Facility Licensee Training Manager

20

21

22 CERTIFIED MAIL—RETURN RECEIPT REQUESTED

23

24

25

26 * Use for application denials for failure to meet the health requirement.

27

28 ** Use for application denials for failure to pass the written examination or operating test, or 

29 both.

30

31
ES-5.1, Page 20 of 20

1 Letter 5.1-5 Sample Pass Letter

2

3 NRC Letterhead

4

5

6 (Date)

7

8

9 (Applicant’s Name)

10 (Street Address) 

11 (City, State ZIP Code)

12

13 Dear (Mr./Ms. Applicant’s Name):

14

15 The purpose of this letter is to forward to you the results of the site-specific operating test and 

16 written examination administered to you during the week of (date) in connection with your 

17 application for a (reactor operator, senior reactor operator, limited senior reactor operator)

18 license for the (facility name). Copies of your operating test and written examination answer 

19 sheets are enclosed.

20

21 However, we will not issue your license [until your employer certifies in writing that you have 

22 acquired all of the training and experience for which you were previously granted a waiver] 

23 [[until we determine that your medical condition and general health are satisfactory for 

24 licensing]].

25

26 If you have any questions, please contact (name) at (telephone number).

27

28 Sincerely,

29

30

31

32

33 (Name and title of licensing official)

34

35 Docket No. 55-(number)

36

37 Enclosures: As stated

38

39 cc w/o enclosure: Senior Facility Licensee Representative

40 cc w/enclosure: Facility Licensee Training Manager

41

42

43

44 [ ] Use only for applicants who need to complete training or experience before licensing.

45 [[ ]] Use only for applicants whose medical condition is still under review.
ES-5.2, Page 1 of 3

1 5.2 APPLICATION DENIALS AND REQUESTS FOR INFORMAL NRC 

2 STAFF REVIEW

3 This examination standard describes the options and associated responsibilities with respect to 

4 license application denials and informal NRC staff reviews.

5 A. Background

6 An applicant who, upon initial NRC review, fails to pass the written examination or operating 

7 test, or both, is notified of his or her results in accordance with ES-5.1, “Issuing Operator 

8 Licenses and Postexamination Activities,” through a preliminary results letter. This letter

9 describes the nature of the deficiencies noted and informs the applicant of his or her available 

10 options. Before a denial, an applicant may request an informal NRC staff review of the license 

11 examination grading results. After the denial, applicants may reapply for a license in 

12 accordance with 10 CFR 55.35, demand a hearing on the denial under 10 CFR 2.103(b), or 

13 both. 

14 Under 10 CFR 55.31(c), an applicant whose application has been denied because of a medical 

15 condition or his or her general health may submit a further medical report at any time as a 

16 supplement to the application.

17 B. Applicant Responsibilities

18 The applicant has the following responsibilities:

19 1. An applicant who receives a preliminary results letter and receives the opportunity to 

20 provide additional information in accordance with 10 CFR 55.31(b) may request 

21 reconsideration of the preliminary results through an informal NRC staff review. 

22 Applicants must submit such requests following the instructions of 10 CFR 55.5. The 

23 applicant’s submittal must clearly state the basis for the request and the item(s) for 

24 which he or she requests additional review, and it must include documentation 

25 supporting the item(s) in contention.

26 2. An applicant who receives a preliminary results letter and receives the opportunity to 

27 provide additional information in accordance with 10 CFR 55.31(b) may respond using 

28 the methods described in 10 CFR 55.5 stating that he or she will not submit additional 

29 information and that the NRC should set the application denial date. The NRC will then 

30 promptly issue a denial letter acknowledging the waiver of an informal NRC staff review. 

31 The denial will be accompanied by a description of hearing rights, as applicable, and 

32 provide guidance for reapplication under 10 CFR 55.35 or medical supplementation 

33 under 10 CFR 55.31(c).

34 3. An applicant whose application for a license has been denied because of failure to pass 

35 the written examination or operating test, or both, may file a new application 2 months 

36 after the date of denial. The application must be submitted on Form NRC-398,

37 “Personal Qualifications Statement—Licensee,” and include a statement signed by an 

38 authorized representative of the facility licensee by whom the applicant will be employed 

39 that states in detail the extent of the applicant’s additional training since the denial and 

40 certifies that the applicant is ready for reexamination. An applicant may file a third 

41 application 6 months after the date of denial of the second application, and he or she 

42 may file further successive applications 2 years after the date of denial of each prior 
ES-5.2, Page 2 of 3

1 application. The applicant shall submit each successive application on Form NRC-398 

2 and include a statement of additional training. An applicant who has passed either the 

3 written examination or operating test and failed the other may request in a new 

4 application on Form NRC-398 to be excused from reexamination on the portions of the 

5 examination or test that the applicant has passed. The Commission may in its discretion 

6 grant the request, if it determines that the applicant has presented sufficient justification. 

7 4. An applicant whose application for a license has been denied because of failure to pass 

8 the written examination or operating test, or both, may demand a hearing within 20 days 

9 from the date of the notice or such longer period as may be specified in the notice under 

10 10 CFR 2.103(b)(2). The application denial letter (see Letter 5.1-4 for a sample) 

11 provides detailed guidance related to the process of demanding a hearing. Under 

12 10 CFR 2.307(a), the applicant may request an extension of this time limit if he or she 

13 can show good cause. A demand for a hearing shall be filed in accordance with 

14 10 CFR Part 2, “Agency Rules of Practice and Procedure,” which is accessible 

15 electronically from the NRC Library on the agency’s Web site at 

16 https://www.nrc.gov/reading-rm/doc-collections/cfr/. Generally, a demand for a hearing 

17 should explain why the applicant believes that the NRC’s denial of his or her application 

18 was in error and why the applicant believes that he or she has, in fact, satisfied the 

19 requirements for license issuance. Applicants must submit such requests electronically 

20 in accordance with the requirements of 10 CFR 2.302, “Filing of documents.” Detailed 

21 guidance on making electronic submissions may be found on the NRC Web site at 

22 https://www.nrc.gov/site-help/e-submittals.html.

23 C. Facility Licensee Responsibilities

24 The facility licensee has the following responsibilities:

25 1. The NRC may ask the facility licensee to provide reference materials, technical support, 

26 and a confirmation of the validity of the test items to allow the agency to resolve any 

27 concerns raised by an applicant.

28 2. The facility licensee should ensure that any written examination questions that are 

29 determined to be invalid (e.g., those that have no correct answer or multiple correct 

30 answers) as a result of NRC staff review are retrieved from any examination bank into 

31 which they have been deposited and corrected or discarded.

32 D. NRC Responsibilities

33 The NRC has the following responsibilities:

34 1. The NRC conducts informal NRC staff reviews of preliminary examination results for a 

35 license under 10 CFR Part 55, “Operators’ licenses,” if requested by an applicant, in 

36 accordance with the current revision of OLMC 500, “Processing Requests for 

37 Administrative Reviews and Demands for Hearings” (OLMC 500, October 2020 revision 

38 is available at Agencywide Documents Access and Management System (ADAMS) 

39 Accession No. ML20230A201).

40 2. The NRC conducts hearings related to 10 CFR Part 55 license examination results, if 

41 demanded by an applicant, in accordance with 10 CFR Part 2.
ES-5.2, Page 3 of 3

1 3. When the NRR operator licensing program office has concurred in the results of an 

2 informal NRC staff review that new information showed that the requirements of 

3 10 CFR 55.33 were met or when ordered as a result of a hearing, the NRC regional 

4 office will (1) issue a license, (2) update the master examination file to reflect any test 

5 item deletions or answer key changes, and (3) consider the need to correspond with the 

6 facility licensee about the quality of the examination.

7 4. When the NRR operator licensing program office has concurred on the results of an 

8 informal NRC staff review that new information did not show that the requirements of 

9 10 CFR 55.33 were met or when ordered as a result of a hearing, the NRC regional 

10 office will inform the applicant of the denial of the application, as necessary.

ES-5.3, Page 1 of 12

1 5.3 MAINTAINING, CHANGING, AND RENEWING OPERATOR

2 LICENSES

3 This examination standard provides instructions for maintaining an NRC operator license and 

4 the procedures for changing operator license information, processing license renewal 

5 applications, and denying applications for license renewal. 

6 A. Maintaining an Operator License

7 1. Requalification Training and Testing

8 a. The regulation in 10 CFR 55.53(h) imposes a condition that requires licensed operators 

9 to complete a requalification program, as described by 10 CFR 55.59, “Requalification.” 

10 The requirement applies to all licensed operators, even if they do not maintain 

11 watch-standing proficiency under 10 CFR 55.53(e). The regulations in 

12 10 CFR 55.59(a)(1) require licensed operators to successfully complete a requalification 

13 program that is conducted for a continuous period not to exceed 24 months in duration. 

14 Under 10 CFR 55.59(c)(1), the facility licensee is required to conduct the requalification 

15 program for a continuous period not to exceed 2 years. 

16 To avoid exceeding the 24-month/2-year duration requirement, a requalification program 

17 must be completed within the anniversary month of the second year. For example, if a 

18 licensed operator requalification program began on June 1, 2018, the facility licensee 

19 would have until June 30, 2020, to complete the program to ensure compliance with 

20 10 CFR 55.59(a)(1) and (c)(1).

21 Under 10 CFR 55.59(a)(2), each licensed operator must pass a comprehensive 

22 requalification written examination and an annual operating test as part of the 24-month 

23 requalification program; therefore, the examination must occur during the requalification 

24 program rather than after its completion. Although the comprehensive written 

25 examinations are generally conducted on the same 24-month frequency, their timing can 

26 be adjusted somewhat near the end of the 24-month program to account for outages 

27 and other events, thereby resulting in some longer testing intervals if an examination is 

28 advanced during one 24-month program cycle and returned to its normal timing during 

29 the following cycle. Thus, the interval between the administrations of successive 

30 comprehensive requalification written examinations may exceed 24 months for individual 

31 licensed operators. As long as a licensed operator successfully completes the facility 

32 licensee’s Commission-approved requalification program, including its required 

33 comprehensive written examination, within 24 months, as required by 

34 10 CFR 55.59(a)(1), the operator’s comprehensive requalification written examination 

35 can be administered more than 24 calendar months from the administration of his or her 

36 last comprehensive written examination without requesting an exemption in accordance 

37 with 10 CFR 55.11, “Specific exemptions.”

38 For example, consider a licensed operator who took a comprehensive requalification 

39 written examination on August 18, 2012, for a facility licensee requalification training 

40 program that ran for 24 months, according to 10 CFR 55.59(a)(1) and 

41 10 CFR 55.59(c)(1), from October 1, 2010, through September 30, 2012. If that 

42 operator’s next comprehensive requalification written examination is scheduled for 

43 September 27, 2014, the operator will exceed 24 calendar months between successive 

44 comprehensive requalification written examinations, but the licensed operator is still in 
ES-5.3, Page 2 of 12

1 compliance with 10 CFR 55.59(a)(1) and (2) and 10 CFR 55.59(c)(1). The licensed 

2 operator will have successfully completed two consecutive requalification training 

3 programs, including comprehensive written examinations, within the 24-month 

4 requalification program time limit according to 10 CFR 55.59(a)(1) and 

5 10 CFR 55.59(c)(1).

6 b. Newly licensed operators must enter the requalification training and examination 

7 program promptly upon receiving their licenses. As they just passed the initial licensing 

8 examination and have received none of the requalification training, new operators may 

9 be excused from taking any annual operating test or comprehensive written examination 

10 that is scheduled to be administered during the first requalification training cycle 

11 (nominally lasting about 6 weeks) in which the operators participate. However, 

12 operators who complete one or more training cycles before the scheduled annual test or 

13 comprehensive examination should take the test or examination to ensure that they do 

14 not exceed the allowed testing intervals.

15 c. If an operator has not met the requirements of 10 CFR 55.59(a)(1) and (2), the 

16 Commission may require the operator to complete additional training in accordance with 

17 10 CFR 55.59(b) and to submit evidence to the Commission of his or her successful 

18 completion of this training before returning to licensed duties. The following examples 

19 are the most common extenuating circumstances that result in an operator not meeting 

20 the requalification requirements:

21 • temporary assignment to the Institute of Nuclear Power Operations

22 • participation in a foreign interchange program

23 • college attendance

24 • military assignment

25 The facility licensee should notify the NRC regional office when any licensed operator is 

26 suspended from the requalification program with the details of the facility licensee’s plan 

27 to ensure that the operator’s qualifications and status are acceptable before he or she 

28 resumes licensed duties.

29 The NRC regional office will confirm its expectations for the operator’s return to licensed 

30 duties and the need for the facility licensee to certify when the actions have been 

31 completed in accordance with 10 CFR 55.59(b). The agency will document this in a 

32 letter to the facility licensee with a copy to the operator.

33 2. Proficiency Watches

34 a. NRC regulations include minimum requirements for the number of shifts and general 

35 types of functions that ROs and SROs must complete in order to maintain active status. 

36 In 10 CFR 55.53(e), the NRC states that “[t]o maintain active status, the licensee shall 

37 actively perform the functions of an operator or senior operator on a minimum of seven 

38 8-hour or five 12-hour shifts per calendar quarter.” This requirement may be completed 

39 with a combination of complete 8- and 12-hour shifts (in a position appropriately credited 

40 for watch-standing proficiency as discussed below) at sites having a mixed-shift 

41 schedule, and watches shall not be truncated when the operator satisfies the minimum 

42 quarterly requirement (56 hours). Overtime may be credited if the overtime work is in a 

43 position appropriately credited for watch-standing proficiency. Working overtime as an 
ES-5.3, Page 3 of 12

1 extra “helper” after the official watch has been turned over to another watch-stander 

2 does not count toward proficiency time.

3 b. In accordance with 10 CFR 55.4, “Definitions,” “actively performing the functions of an 

4 operator or senior operator” means “that an individual has a position on the shift crew 

5 that requires the individual to be licensed as defined in the facility technical 

6 specifications, and that the individual carries out and is responsible for the duties 

7 covered by that position.” Therefore, to meet these regulatory requirements, ROs and 

8 SROs will be responsible for actively performing the functions of an operator or senior 

9 operator, for the required number of quarterly shifts, and while in a position on a shift 

10 crew that requires an operator or senior operator, as appropriate, license under the 

11 facility technical specifications.

12 c. Watch-standing proficiency credit may also be appropriate for certain licensed RO or 

13 SRO shift crew positions that exceed those required by a facility’s technical 

14 specifications. However, to credit watch-standing proficiency for such excess positions, 

15 the facility licensee should have in place the following procedural administrative controls:

16 • a list of all the licensed shift crew positions, including title, description of duties, 

17 and indication of which positions are required by technical specifications

18 • for shift crew positions in excess of those required by technical specifications, a 

19 description of how the position is meaningfully and fully engaged in the functions 

20 and duties of the analogous minimum licensed position(s) required by technical 

21 specifications

22 For example, technical specifications for a dual-unit facility with a common control room 

23 require two SROs per shift. The facility licensee could credit watch-standing proficiency

24 for three SROs per shift, with one SRO responsible for overall plant operation and the 

25 other two SROs each responsible for the command and control of a single unit. In this 

26 case, the third SRO would be entitled to watch-standing proficiency credit because he or 

27 she is performing duties analogous to the second SRO (who is required by technical 

28 specifications). Similarly, a dual-unit facility with a common control room could credit 

29 watch-standing proficiency for four ROs (two per unit) per shift even if technical 

30 specifications require only three ROs if the fourth RO is performing duties analogous to 

31 the third RO (who is required by technical specifications).

32 If a facility cannot justify, as explained above, crediting watch-standing proficiency for 

33 shift crew positions in excess of technical specifications or does not implement 

34 administrative controls as described above, an individual who stands watch in an excess 

35 position shall not receive proficiency credit. In order to maintain an active license under 

36 such circumstances, each licensed individual would have to rotate into a licensed shift 

37 crew position required by technical specifications for the minimum of seven 8-hour or 

38 five 12-hour shifts per calendar quarter, with sufficient administrative controls to 

39 document those activities.

40 Facility licensees that are uncertain whether shift crew positions in excess of those 

41 required by technical specifications qualify for watch-standing proficiency credit should 

42 contact their NRC regional office.

43 d. An individual with an SRO license could maintain only the RO portion of his or her 

44 license in an active state by performing the functions of an RO for a minimum of seven 
ES-5.3, Page 4 of 12

1 8-hour or five 12-hour shifts per calendar quarter under 10 CFR 55.53(e). Moreover, an 

2 inactive SRO may reactivate only the RO portion of his or her license under 

3 10 CFR 55.53(f)(2) by completing a minimum of 40 hours of shift functions, including a 

4 plant tour, under the direction of an operator and in the position to which the individual 

5 will be assigned. However, the fact that an SRO license holder is routinely standing 

6 watches only as an RO does not maintain his or her proficiency as an SRO. Therefore, 

7 before such an SRO can resume duties that require an SRO license, he or she must 

8 reactivate that portion of the license under 10 CFR 55.53(f)(2) by completing a minimum 

9 of 40 hours of shift functions, including a plant tour, under the direction of a senior 

10 operator and in the SRO position to which the individual will be assigned.

11 e. To maintain the supervisory portion of an SRO license active, an SRO must stand at 

12 least one complete watch (8- or 12-hour shift) per calendar quarter in a shift crew 

13 position credited for SRO-only supervisory licensed duties. The remainder of complete 

14 watches (to meet the required minimum of seven 8-hour or five 12-hour shifts per 

15 calendar quarter) may be performed in either a credited SRO or RO position. An SRO 

16 may stand all of his or her required watches in credited SRO-only supervisory positions, 

17 and the RO portion of the license will still be considered active. Similarly, for an SRO to 

18 reactivate the supervisory portion of his or her SRO license under 10 CFR 55.53(f)(2), 

19 he or she must complete a minimum of 40 hours of shift functions, including a complete 

20 plant tour and required shift turnover procedures, under the direction of an SRO in a 

21 credited SRO-only supervisory position. An SRO who reactivates his or her license in 

22 this manner automatically reactivates the RO portion of the license; an additional 

23 40 hours of under-direction watches in a credited RO position are not required.

24 f. Individuals who are licensed on two (or more) comparable units at a facility are not 

25 required to establish proficiency on each of the comparable units unless they hold a 

26 separate license for each unit. Performing the required seven 8-hour or five 12-hour 

27 shifts of watch-standing per calendar quarter on a single unit maintains the license active 

28 for all comparable units identified in the license. Similarly, individuals who are licensed 

29 on two (or more) comparable units at a facility are not required to reactivate their license 

30 on each of the comparable units identified in the license. Performing the required 

31 40 hours of under-direction watches on a single unit, including a plant tour, reactivates 

32 the license for all comparable units identified in the license.

33 g. In addition to the under-direction watch requirements discussed above, the following 

34 clarifies license reactivation under 10 CFR 55.53(f):

35 • The 40 hours of under-direction watches required by 10 CFR 55.53(f)(2) shall 

36 only be credited for standing watches in an RO or SRO position appropriately 

37 credited for maintaining license proficiency. It is not appropriate to credit 

38 reactivation watch hours while under the direction of an active license holder who 

39 is standing watch in an “extra” or noncredited position.

40 • When performing under-direction watches, only one under-direction 

41 watch-stander shall be assigned to an active license holder. Given that the 

42 inactive operator must complete (not just observe) 40 hours of shift functions, it 

43 would not be appropriate to divide under-direction watch functions among 

44 multiple individuals.
ES-5.3, Page 5 of 12

1 • The 40 hours of under-direction watches for license reactivation do not need to 

2 occur in complete shifts or to be completed on consecutive days. All 40 hours 

3 should occur within a reasonable timeframe (e.g., 30 days), and at least one 

4 complete on-coming shift turnover and one complete off-going shift turnover must 

5 be performed while under the direction of the active license holder. Once the 

6 operator completes all the requirements for license reactivation, the license is 

7 considered active for the remainder of the current calendar quarter, with 

8 proficiency watches (i.e., seven 8-hour or five 12-hour shifts) required to maintain 

9 the license in an active state during subsequent calendar quarters.

10 • The 40 hours of under-direction watches do not need to occur in the control 

11 room; they may be performed wherever the duties of the credited licensed 

12 position are performed.

13 • The 40 hours of under-direction watches must include at least one complete 

14 plant tour. Since it is a part of the 40 hours of under-direction watches, the plant 

15 tour must be performed under the direction of an active license holder. Although 

16 the regulations do not define the scope of a complete plant tour, the NRC 

17 expects that this tour will include all readily accessible major areas of the plant 

18 that are routinely toured by in-plant operators and that contain safety-related 

19 equipment. If a facility has developed a checklist of areas to tour, it is generally 

20 inappropriate to skip plant areas and mark the items as “nonapplicable” unless 

21 there is sufficient justification (e.g., personnel or radiation hazard).

22 h. Senior operators limited to fuel handling under 10 CFR 55.53(c) (i.e., limited SROs or 

23 LSROs) would generally be unable to maintain an active status as defined in 

24 10 CFR 55.53(e). Therefore, under 10 CFR 55.53(f), an authorized representative of the 

25 facility licensee must certify that, among other things, the LSRO has completed one shift 

26 under the direction of an active senior operator before resuming activities authorized by 

27 the license. Ideally, such a watch should be performed primarily in the fuel handling 

28 area during refueling operations (i.e., at a time when the presence of a senior operator is 

29 required under 10 CFR 50.54(m)(2)(iv)). This would clearly meet the requirements of 

30 10 CFR 55.53(f)(2), which mandates that the licensee must complete one shift of shift 

31 functions under the direction of a senior operator in the position to which the licensee will 

32 be assigned. It also meets the definition of “actively perform the functions of [a] senior 

33 operator” in 10 CFR 55.4, which requires the licensee to fill a position on the shift crew 

34 that requires the individual to be licensed and to carry out and be responsible for the 

35 duties covered by that position. This also ensures that the inactive LSRO’s activities are 

36 adequately supervised. However, given the infrequency and short duration of shift 

37 functions that require the presence of an LSRO on the refueling floor, it may not always 

38 be practical for a facility licensee to delay its LSRO reactivations until those shift 

39 functions are actually underway. In such instances, the facility licensee can satisfy the 

40 intent of the regulation by implementing a reactivation program that specifies, in detail, 

41 the refueling tasks and activities that an LSRO must satisfactorily complete or walk

42 through on-station in order to demonstrate watch-standing proficiency.

43 To properly reactivate an LSRO license in accordance with 10 CFR 55.53(f), the 

44 individual should stand a watch under the direction and in the presence of an active 

45 SRO or LSRO, who will directly oversee the inactive LSRO’s activities and enable an 

46 authorized representative of the facility licensee to certify that the operator’s 

47 qualifications are current and valid, as required by 10 CFR 55.53(f)(1). The NRC gives 
ES-5.3, Page 6 of 12

1 its requirements for the conduct of under-instruction or training watches in 

2 10 CFR 55.13, “General exemptions,” which allows trainees to manipulate the controls of 

3 a facility “[u]nder the direction and in the presence of a licensed operator or senior 

4 operator.” The responses to Questions 252 and 276 in NUREG-1262, “Answers to 

5 Questions at Public Meetings Regarding Implementation of Title 10, Code of Federal 

6 Regulations, Part 55 on Operators’ Licenses,” issued November 1987 (ADAMS 

7 Accession No. ML15198A217), state that the responsible person should closely monitor 

8 a trainee’s activities.

9 If a facility licensee needs to reactivate an SRO license as an LSRO (strictly for the 

10 purpose of supervising refueling activities), the operator must complete one shift under 

11 direction performing refueling activities, as discussed above, and the facility licensee 

12 must ensure that the operator is administratively restricted from performing full SRO 

13 duties.

14 If a facility licensee is unable to comply with the LSRO license reactivation requirements 

15 in 10 CFR 55.53(f)(2) even under the conditions discussed above, the licensee may, 

16 under 10 CFR 55.11, request an exemption from the requirements in 10 CFR 55.53(e) 

17 and propose alternative criteria for maintaining active LSRO licenses. The Commission 

18 may grant such exemptions from the regulatory requirements as it determines are 

19 authorized by law and will not endanger life or property and are otherwise in the public 

20 interest. Such requests should provide the following information:

21 • the reason why the facility licensee is unable to comply with the requirements of 

22 10 CFR 55.53(f)(2), as clarified above, for reactivating its LSRO licenses to 

23 supervise fuel handling

24 • the nature of the fuel handling activities that a licensee will have to complete to 

25 remain “active” and an explanation of how those activities would maintain an 

26 operator’s proficiency to supervise actual core alterations (identify those activities 

27 that must be performed and those that may be simulated and explain how the 

28 simulation will be accomplished)

29 • the minimum duration and frequency of the fuel handling activities required to 

30 remain “active”

31 • the nature, duration, and frequency of the training related to fuel handling that is 

32 given to its licensed fuel handlers

33 3. Medical Standards

34 a. In accordance with Subpart C, “Medical Requirements,” of 10 CFR Part 55 and 

35 10 CFR 55.33(a)(1), the medical condition and general health of licensed operators must 

36 be such that it will not adversely affect the performance of assigned operator duties or 

37 cause operational errors endangering public health and safety. Therefore, licensed 

38 operators must be examined by a physician and determined to be fit every 2 years 

39 (measured from the date of the last physical examination rather than from the date of 

40 licensing). In addition, under 10 CFR 55.57(a)(6), their fitness must be certified on NRC 

41 Form 396, “Certification of Medical Examination by Facility Licensee,” every time the 

42 license is renewed. As noted on NRC Form 396, the physician and facility licensee may 

43 use the 1983, 1996, 2013, or most recently endorsed version of American National 

44 Standards Institute (ANSI)/American Nuclear Society (ANS) 3.4, “Medical Certification 
ES-5.3, Page 7 of 12

1 and Monitoring of Personnel Requiring Operator Licenses for Nuclear Power Plants,” 

2 when making their fitness determinations. These versions of the standard include 

3 provisions for those cases in which the operator can demonstrate complete capacity to 

4 perform licensed duties and conditional licenses for those cases in which compensatory 

5 measures may be required to ensure public health and safety (refer to Section C.3.c 

6 below). However, in all cases, the examining physician and facility licensee must submit 

7 a recommendation and supporting evidence on or with NRC Form 396 to enable the 

8 NRC to make a licensing decision.

9 b. If, during the term of the license, an operator is temporarily unable to meet medical 

10 standards but is expected to meet those standards again in the future, the facility 

11 licensee may administratively classify that operator’s license as “inactive” or require 

12 compensatory measures, such as taking any medications as prescribed during the 

13 temporary period to maintain medical qualifications, or impose other operating 

14 restrictions to accommodate the operator’s medical condition until the operator is once 

15 again certified to meet all medical standards by the facility licensee. Similarly, if the 

16 operator’s medical condition precludes the operator from completing the requalification 

17 training program under 10 CFR 55.59(a), the facility licensee shall administratively 

18 control the operator’s activities until he or she completes the additional training 

19 requirements of 10 CFR 55.59(b), including notification of the NRC.

20 The facility licensee does not need to notify the NRC nor request a conditional license 

21 concerning an operator’s temporary disability, including the temporary use of prescribed 

22 medications, provided that the facility licensee administratively prevents the operator 

23 from performing licensed duties or otherwise compensates for or restricts the operator, 

24 as appropriate, throughout the period of his or her temporary disability. If the disability 

25 extends beyond the date of license expiration, the operator may apply for timely license 

26 renewal in accordance with 10 CFR 55.55(b) and 10 CFR 55.57(a). In that event, the 

27 facility licensee should document the nature of the operator’s temporary disability on the 

28 medical certificate and submit a revised certificate to the NRC after the physician 

29 determines that the operator meets the requirements of 10 CFR 55.33(a)(1). The NRC 

30 will not renew the operator’s license until the staff finds that all the conditions specified in 

31 10 CFR 55.57(b) are satisfied.

32 c. If the facility licensee determines that an operator’s medical condition is permanently

33 disqualifying in accordance with ANSI/ANS 3.4, the facility licensee shall notify the NRC 

34 within 30 days of learning of the diagnosis (see 10 CFR 50.74, “Notification of change in 

35 operator or senior operator status,” and 10 CFR 55.25, “Incapacitation because of 

36 disability or illness”). If an operator develops a permanent medical condition that is not 

37 identified in ANSI/ANS 3.4, but the examining physician believes that it could affect the 

38 operator’s performance or cause operator errors, it would be prudent to report the 

39 condition to the NRC or at least contact the appropriate NRC regional office to ask 

40 whether the condition should be reported.

41 While most of the medical conditions/disabilities identified in ANSI/ANS 3.4, including 

42 those that result in failure to meet the minimum requirements for medical qualification, 

43 are likely to be permanent, the examining physician is responsible for evaluating each 

44 operator’s medical condition on a case-by-case basis and assessing whether the 

45 operator will be capable of meeting medical standards in the foreseeable future. For 

46 example, the facility licensee should report to the NRC a condition for an operator who 

47 takes medication to meet the minimum standard for blood pressure (i.e., less than or 
ES-5.3, Page 8 of 12

1 equal to 160/100 millimeters of mercury (mmHg)), unless the physician has reasonably 

2 determined that the condition will be controllable without medication in the foreseeable 

3 future. In addition, many physicians prescribe blood pressure medication before an 

4 individual reaches the 160/100 mmHg limit, and facility licensees should consider

5 reporting this to the NRC as well.

6 When reporting a permanent disqualifying medical condition, if a conditional license is 

7 requested, the facility licensee shall provide medical certification and evidence on NRC 

8 Form 396 and recommend the exact wording of any license restriction that might be 

9 necessary. A permanent disqualifying condition is always reportable, even if it is being 

10 controlled and regardless of whether the compensatory measures are recognized in the 

11 applicable version of ANSI/ANS 3.4.

12 d. In accordance with 10 CFR 55.33(b), if an operator’s general medical condition does not 

13 meet the minimum standards under 10 CFR 55.33(a)(1), the NRC may condition the 

14 license to accommodate the medical defect. The NRC will consider the 

15 recommendations and supporting evidence provided on or with NRC Form 396 in 

16 determining the appropriate license condition. The following medical restrictions and 

17 conditions are illustrative but not all-inclusive:

18 • An operator may be required to wear corrective lenses while performing licensed 

19 duties if his or her vision does not meet medical standards.

20 • An operator may be required to wear a hearing aid while performing licensed 

21 duties if his or her hearing does not meet medical standards.

22 • An RO who is at risk of sudden incapacitation may have a no-solo restriction that 

23 requires another licensed operator to be in view when the restricted operator is 

24 performing control manipulations, and someone capable of summoning 

25 assistance must be present at all other times while the restricted operator is 

26 performing licensed duties. The analogous SRO restriction would require 

27 another licensed operator to be in view when the restricted operator is performing 

28 control manipulations and another senior operator to be present on site at all 

29 other times while the restricted operator is performing SRO licensed duties, or 

30 someone capable of summoning assistance must be present at all other times 

31 while the restricted operator is performing RO licensed duties. For LSROs, the 

32 no-solo restriction would require someone capable of summoning assistance to 

33 be in view when the restricted LSRO is performing licensed LSRO duties.

34 • An operator may be required to take medication as prescribed, if an operator’s 

35 medical qualification is contingent on taking a prescription medication.

36 • An operator may be required to use a therapeutic device, such as a continuous 

37 positive airway pressure therapy machine, as prescribed by a doctor, if an 

38 operator’s medical qualification is contingent on using that device.

39 • An operator whose medical condition is acceptable but unstable may be required 

40 to submit followup medical status reports (i.e., prognosis, treatment, and ability to 

41 perform licensed duties) at 3-, 6-, or 12-month intervals.
ES-5.3, Page 9 of 12

1 • An operator with respiratory problems may be restricted from performing licensed 

2 activities that require the use of a respirator.

3 e. With regard to prescription medications, it is important that the examining physician 

4 understand what medical conditions are contained in the applicable version of 

5 ANSI/ANS 3.4. For example, the fact that a licensed operator is diagnosed with 

6 gastroesophageal reflux disease and placed on the appropriate prescription medication 

7 would, in all likelihood, not be reportable to the NRC, since this condition is not 

8 addressed in ANSI/ANS-3.4. However, when assessing any prescription medication, the 

9 examining physician needs to consider (1) the possible side effects of the medication, 

10 drug interactions, and dosages to ensure that they will not cause operational errors or 

11 affect the operator’s capacity to safely perform licensed duties and (2) any delay in 

12 taking a medication that might be expected to result in the incapacity of the operator.

13 In addition, the actual wording of the license condition on medication will not specify a 

14 particular medical condition or medication, but it will simply state that the operator must 

15 “take medication as prescribed.” Therefore, physician-prescribed changes in medication 

16 or dosing for an existing medical condition are not required to be reported to the NRC 

17 unless the examining physician believes that the operator’s medical condition has 

18 become unstable (therefore requiring followup medical status reports to the NRC) or that 

19 the operator requires a no-solo license restriction. However, any new permanently 

20 disqualifying medical condition(s), requiring new medication(s), must be reported to the 

21 NRC.

22 B. Changes to Operator License

23 1. Downgrading a Senior Reactor Operator License

24 If a facility licensee and SRO desire to permanently downgrade the SRO’s license, the licensee

25 may do so by submitting a written request to the NRC regional office. In such instances, the 

26 NRC regional office will (1) amend the license to restrict the operator’s activities to those 

27 authorized for an RO under 10 CFR Part 55, (2) condition the license to prohibit the operator 

28 from directing the licensed activities of licensed operators, and (3) inform the operator and 

29 facility licensee in writing that the license will not be subject to renewal under 10 CFR 55.57, 

30 “Renewal of licenses,” and that a new application (NRC Form 398) will be required under 

31 10 CFR 55.31 if the operator desires to maintain an RO license upon expiration of the amended 

32 SRO license. The expiration date of the original license will not change, and the operator may 

33 transition to the RO requalification program upon receipt of the amended license.

34 2. Amendments

35 An amendment is required to change information on the operator license. Some examples of 

36 changes that require an amendment include the following:

37

38 • legal name (such as first or last name)

39 • type of license (such as the downgrade of an SRO license to an RO license)

40 • permanent medical condition or restriction
ES-5.3, Page 10 of 12

1 • addition of another unit at a multiunit site (requested using NRC Form 398; also 

2 see ES-2.2, “Applications, Medical Requirements, and Wavier and Excusal of 

3 Examination and Test Requirements”)

4 The following types of changes do not require an amendment to the operator license:

5

6 • mailing address

7 • e-mail address

8 • education

9 • additional industry experience

10 • editorial issues (such as a typographical error)

11 • new facility medical contact

12 • use of a new ANS/ANSI standard (ANSI/ANS 3.4 or ANSI/ANS 15.4, “Selection 

13 and Training of Personnel for Research Reactors”)

14 If there is a question as to whether a change requires a license amendment, the facility licensee 

15 should contact the NRC regional office for clarification.

16

17 A request to amend an operator license requires a signature from the affected licensed operator 

18 in accordance with 10 CFR 55.31(b); 10 CFR 55.31(d); 10 CFR 55.9, “Completeness and 

19 accuracy of information”; 10 CFR 55.61, “Modification and revocation of licenses”; and 

20 Sections 107 and 182a of the Atomic Energy Act of 1954, as amended. The NRC will not 

21 amend the operator’s license in response to the amendment request without a signed statement 

22 of fact from the affected licensed operator. For the purpose of medical condition license 

23 amendments, the affected licensed operator can satisfy this requirement by signing the NRC 

24 Form 396. To meet this requirement, the facility licensee may e-mail or submit using the NRC’s

25 Electronic Information Exchange the operator-signed license amendment request to the NRC 

26 regional office.

27

28 C. License Renewal

29 The license renewal application differs in some respects from the initial license application. An 

30 operator who wishes to renew a license must comply with the requirements of 10 CFR 55.57(a), 

31 as follows:

32

33 1. The operator will complete and sign NRC Form 398, including the operator’s experience 

34 under the current license, the approximate number of hours that the operator spent on 

35 operating shifts, and the date and results of the applicant’s most recent requalification 

36 written examination and annual operating test. The senior management representative 

37 on site shall provide evidence that the operator has safely and competently discharged 

38 his or her license responsibilities and has satisfactorily completed the facility’s approved 

39 requalification program by checking the corresponding certification box and signing in 

40 the designated space on NRC Form 398.
ES-5.3, Page 11 of 12

1 2. The facility licensee must certify on NRC Form 396 that a physician has performed a 

2 medical examination within the previous 2 years, as required by 10 CFR 55.21, 

3 “Medical examination,” and submit that form along with NRC Form 398.

4

5 3. The operator must submit NRC Forms 396 and 398 not less than 30 days before the 

6 expiration date of the license. In accordance with 10 CFR 55.55(b), if the operator files 

7 an application for renewal at least 30 days before the date of expiration, the license 

8 shall not expire until the NRC has dispositioned the application for renewal. If the 

9 application is received more than 60 days in advance, the NRC regional office should 

10 contact the facility licensee to determine whether it would prefer to have the license 

11 renewed immediately with a new effective date (the license will not be predated, nor will 

12 it exceed a 6-year license term) or to resubmit the application within the 60- to 30-day 

13 window preceding the expiration date.

14

15 If an operator is waiting to take a reexamination after failing a requalification 

16 examination, the operator should still make timely application for license renewal under 

17 the provisions of 10 CFR 55.55(b).

18

19 Under 10 CFR 55.55(a), each operator license and senior operator license expires 

20 6 years after the date of issuance, upon termination of employment with the facility 

21 licensee, or upon determination by the facility licensee that the licensed individual no 

22 longer needs to maintain a license. Under 10 CFR 55.55(b), if a licensee files an 

23 application for renewal or an upgrade of an existing license on NRC Form 398 at least 

24 30 days before the expiration of the existing license, it does not expire until the 

25 Commission has finally determined the disposition of the application for renewal or for 

26 an upgraded license. Filing by mail will be deemed to be complete at the time the 

27 application is deposited in the mail. Under 10 CFR 2.109, “Effect of timely renewal 

28 application,” if at least 30 days before the expiration of an existing license, the licensee 

29 files an application for a renewal or for a new license for the same activity, then the 

30 existing license will not be deemed to have expired until the application has been finally 

31 determined.

32

33 4. If the license for a RO expires while he or she is participating in the facility licensee’s 

34 SRO-upgrade training program, NRC Forms 396 and 398 should still be submitted for 

35 timely renewal of the RO license. However, if the RO is not current in the facility’s 

36 requalification training and testing program because he or she is attending 

37 SRO-upgrade training, NRC Form 398 must note the exception in the “Comments” 

38 section, and the operator must be administratively restricted from performing licensed 

39 duties until the individual is up to date in the requalification program.

40

41 5. The regulations at 10 CFR 55.5 set forth requirements on where to submit any 

42 application for a license or license renewal and any related inquiry, communication, 

43 information, or report.

44

45 Upon receipt of a renewal application, the NRC regional office may take the following actions, 

46 as appropriate:

47

48 1. Under 10 CFR 55.31(b), the Commission may at any time after the application has been 

49 filed, and before the license has expired, require further information under oath or 

50 affirmation in order to enable it to determine whether to grant or deny the application or 
ES-5.3, Page 12 of 12

1 whether to revoke, modify, or suspend the license. After reviewing the renewal 

2 application, the NRC’s regional office may ask the licensee or facility licensee to provide 

3 supplemental information. The Regional Administrator may deny an application if an 

4 applicant fails to respond to a request for additional information within 30 days from the 

5 date of the request or within such other period as may be specified. (Requirements 

6 concerning denials appear in 10 CFR 2.108, “Denial of application for failure to supply 

7 information,” and 10 CFR 2.103(b).)

8

9 2. The NRC regional office will review the application and issue the license renewal if the 

10 staff finds that the applicant satisfies the requirements of 10 CFR 55.57(b). The operator 

11 does not have to operate the facility for any minimum number of hours to qualify for 

12 license renewal (i.e., inactive licenses are also renewable).

13

14 3. If the renewal applicant does not satisfy the requirements of 10 CFR 55.57, the NRC 

15 regional office shall inform the applicant and the facility licensee of the deficiencies and 

16 request any supplemental information that the staff might require to make a renewal 

17 decision. If, after evaluating the supplemental information, the NRC regional office still 

18 concludes that the applicant does not meet the requirements for license renewal, the 

19 staff will issue a notice of denial of the application and inform the applicant in writing of 

20 the right of the applicant to demand a hearing within 20 days from the date of the notice 

21 or such longer period as may be specified in the notice in accordance with 

22 10 CFR 2.103(b).

23

24 The applicant may file a demand for a hearing following the denial of his or her license renewal 

25 application. A demand for a hearing shall be filed in accordance with 10 CFR Part 2, which is 

26 accessible electronically from the NRC Library on the NRC’s Web site at 

27 https://www.nrc.gov/reading-rm/doc-collections/cfr/. Generally, a demand for a hearing should 

28 explain why the applicant believes that the NRC’s denial of his or her application was in error 

29 and why the applicant believes that he or she has, in fact, satisfied the requirements for license 

30 renewal. Applicants must submit such requests electronically in accordance with the 

31 requirements of 10 CFR 2.302. The NRC has published detailed guidance on making electronic 

32 submissions on the agency’s Web site at https://www.nrc.gov/site-help/e-submittals.html.
ES-6.1, Page 1 of 33 

1 6.1 CONDUCTING NRC REQUALIFICATION EXAMINATIONS

2 Title 10 of the Code of Federal Regulations (10 CFR) 55.59(a) requires licensed operators and 

3 senior operators to complete a requalification program developed by the facility licensee and to 

4 pass a comprehensive requalification written examination and an annual operating test. In lieu 

5 of accepting the facility licensee’s certification that the operator has passed the required 

6 examinations and tests administered within the facility licensee’s Commission-approved 

7 program, the U.S. Nuclear Regulatory Commission (NRC) may administer a comprehensive 

8 requalification written examination and an annual operating test.

9 

10 This standard provides guidance and instructions for conducting NRC requalification 

11 examinations. In addition, this standard offers guidance and instructions for evaluating the 

12 facility licensee’s requalification training program for licensed operators to ensure that it is 

13 effectively maintaining the competency of the licensed operators. Examination Standard 

14 (ES)-6.2, “Requalification Written Examinations”; ES-6.3, “Requalification Walkthrough 

15 Examinations”; and ES-6.4, “Dynamic Simulator Requalification Examinations,” provide specific 

16 guidance and instructions for conducting the comprehensive requalification written examinations 

17 and the annual operating tests (including both the plant walkthrough and dynamic simulator 

18 sections). 

19

20 This standard also contains instructions to follow for an operator who fails one or more 

21 NRC-conducted requalification examinations, including licensed operators’ requests for informal

22 reviews. 

23

24 A. Background

25 Section 306 of the Nuclear Waste Policy Act of 1982 authorized and directed the NRC to issue 

26 regulations, or other appropriate guidance, for training and qualifying nuclear power plant 

27 operators. Those regulations were to include requirements governing the administration of 

28 requalification examinations and operating tests at nuclear power plant simulators. The NRC’s 

29 requalification evaluation program consists primarily of periodic, onsite requalification 

30 inspections. The oversight program requires the NRC to actively oversee each facility licensee’s 

31 requalification training programs, and the Commission’s regulations will continue to contain 

32 legally binding requirements that apply to the conduct of operator requalification examinations by 

33 facility licensees.

34

35 When determining the scope of a facility’s requalification inspection and examination activities, 

36 regional managers will consider overall facility performance; the results of the NRC’s inspection 

37 programs (e.g., requalification, emergency operating procedure, and resident); the results of 

38 routine initial and requalification examinations; and other factors. When necessary, the NRC 

39 can initiate augmented activities in accordance with program office guidance to ensure safe 

40 plant operation. Those activities could include a training program inspection in accordance with 

41 Inspection Procedure (IP) 41500, “Training and Qualification Effectiveness,” operational 

42 evaluations of onshift crews, or NRC examinations conducted in accordance with this series of 

43 examination standards.

44

45 The NRC will conduct requalification examinations when it has lost confidence in the facility 

46 licensee’s ability to do so or when the staff believes that the inspection process will not provide 

47 the needed insight. Regional management should consider conducting requalification 

48 examinations or operational evaluations when any of the following conditions exist:
ES-6.1, Page 2 of 33 

1 • Requalification inspection results indicate an ineffective operator requalification program.

2 • Operator errors are a major contributor to operational problems.

3 • Allegations have been raised about significant training program deficiencies.

4 The decision to conduct NRC examinations should be implemented through the normal 

5 resource planning system because an inspection activity will be replaced with examinations that 

6 are more resource intensive. Using the existing inspection planning process ensures that the 

7 regional office and the NRC’s Office of Nuclear Reactor Regulation (NRR) consider the need to 

8 conduct examinations, as well as the alternative expanded inspection tools, when allocating the 

9 required resources. Operational evaluations should be considered as a reactive effort based on 

10 immediate safety concerns.

11

12 B. Scope

13 The NRC-conducted requalification examinations measure the effectiveness of a facility 

14 licensee’s requalification program by evaluating the licensee’s ability to adequately prepare 

15 written examination questions, job performance measures (JPMs), and simulator scenarios, as 

16 well as its ability to properly evaluate its operators’ performance. The examination procedures 

17 are based on a systems approach to training (SAT) program, as defined in 10 CFR 55.4, 

18 “Definitions.” To the extent possible, these procedures rely on existing requalification program 

19 standards for developing and implementing the NRC’s examinations. The SAT approach allows 

20 the NRC to conduct requalification examinations that are fundamentally consistent with existing 

21 facility-developed programs. As such, this approach reduces the impact on the facilities and 

22 improves the reliability of the NRC’s assessment of requalification training programs.

23

24 The NRC-conducted requalification examination normally consists of three parts, including a 

25 two-section open-reference written examination, a walkthrough evaluation, and a dynamic 

26 simulator evaluation. ES-6.2, ES-6.3, and ES-6.4 further describe the three examination parts. 

27 The NRC will consider preferentially using the facility licensee’s requalification examination 

28 structure or methodology if it differs from that described here if it complies with 10 CFR 55.59, 

29 “Requalification,” and is free of significant flaws. The regional office shall consult with the NRR 

30 operator licensing program office to determine the appropriate examination procedure.

31

32 To the extent practical, the examination will be based on the facility licensee’s requalification 

33 program and learning objectives. The NRC expects the facility licensee to use the plant-specific 

34 job task analyses (JTAs) as the basis for developing the examination materials and 

35 substantiating the importance rating factors for each task. The facility licensee may also refer to 

36 the applicable knowledge and abilities (K/A) catalog for additional guidance on identifying 

37 job-specific importance rating factors. The use of a JTA will result in more technically sound 

38 and operationally oriented examinations.

39

40 An examination team composed of NRC examiners and facility representatives will develop, 

41 review, and conduct each requalification examination. Parallel evaluation of operator 

42 performance by NRC examiners and facility evaluators will enhance the NRC’s ability to assess 

43 both individual and program performance.

44

45 C. Examination Preparations

46 Form 6.1-1, “Examination Timeline,” outlines an example timeline for conducting an NRC 

47 requalification examination. 

48
ES-6.1, Page 3 of 33

1 1. Communication

2 a. When the NRC determines that it is necessary to conduct a requalification examination, 

3 the regional office will notify the facility licensee to be evaluated at least 90 but 

4 preferably 120 days before the examination start date using the corporate notification 

5 letter shown in Letter 6.1-1, “Sample Corporate Notification Letter.” If possible, the NRC 

6 will schedule the site visits to coincide with the facility’s requalification training cycle. 

7 Depending on the number of operators and crews at the facility, it may be necessary to 

8 conduct the examinations over a period of 2 or more weeks to attain the required sample 

9 size. The requalification training cycle, referenced here and throughout NUREG-1021, is 

10 that continuous period (not to exceed 24 months) within which the facility licensee 

11 conducts its operator requalification training program.

12

13 If the purpose of the examination is to retest operators who previously failed an 

14 NRC-conducted requalification examination, the regional office should modify the 

15 corporate notification letter, as appropriate.

16

17 b. The NRC expects the facility licensee to respond to the corporate notification letter at 

18 least 60 days before the evaluation by submitting the materials and information 

19 requested in the letter. The facility licensee may request that the NRC chief examiner or 

20 another NRC representative meet with appropriate facility licensee managers and the 

21 operators to be examined. Such a meeting should be scheduled during the examination 

22 preparation week as discussed in Section C.5.

23

24 c. At least 30 days before the examination, the NRC will confirm with the facility licensee 

25 which operators have been selected to participate in the evaluation.

26

27 2. Selection of Operators

28 a. The NRC expects facility licensees to train and examine their operators in the same 

29 crew configurations with which they normally operate the plant. Generally, the NRC 

30 expects the crew to include no more than five operators, but the agency will consider 

31 larger crews on a case-by-case basis.

32

33 At times, to ensure an adequate sample size, the examination team may configure 

34 crews that do not routinely work together to perform shift duties. Mixed crews of shift 

35 and nonshift operators should not be configured unless the facility licensee routinely 

36 evaluates mixed crews in its requalification training program, or the facility licensee’s 

37 normal crew size is so large that it is necessary to separate a normal crew for 

38 examination purposes.

39

40 b. All crew members for requalification dynamic simulator examinations must be currently 

41 licensed on the facility and up to date in the facility licensee’s requalification program.

42

43 c. The selections will be made to minimize disruption of the facility licensee’s schedules 

44 and plant operations. Operating crew(s) in training will be given priority during the 

45 examination week(s). If the NRC is reevaluating the facility’s program after an 

46 unsatisfactory evaluation, the selection process should favor operators who either failed 

47 their previous NRC-conducted examinations or were not previously examined.

48
ES-6.1, Page 4 of 33

1 d. During retake examinations, the dynamic simulator crew evaluation may include 

2 operators who have passed an NRC requalification examination. However, these 

3 operators will not be required to take the written or walkthrough portions of that 

4 examination. The operators’ performance on the simulator examination will be 

5 evaluated in accordance with the guidance of ES-6.4.

6

7 e. A shift technical advisor (STA) may be added to the crew if the facility normally uses an 

8 STA during requalification training. In such instances, the NRC expects the STA’s duties 

9 and responsibilities to be the same as those assigned during requalification training and 

10 plant operations. 

11

12 f. The NRC will review the list of crews and operators submitted by the facility licensee and 

13 will recommend any necessary changes.

14

15 3. Reference Material

16 a. The NRC expects the facility licensee to supply the reference materials requested in the 

17 corporate notification letter (see Enclosure 1, “Reference Material Guidelines,” to 

18 Letter 6.1-1). The NRC will evaluate the facility’s reference materials for adequacy 

19 before the scheduled preparation week, using Form 6.1-4, “Evaluation Checklist for 

20 Facility Reference Material.”

21

22 b. The NRC reserves the right to prepare the requalification examinations using the 

23 facility’s background reference materials if the facility licensee’s test items are 

24 inadequate for examination preparation. If the NRC prepares the examination, the staff 

25 may require reference materials comparable to those listed in Section F of ES-2.1, 

26 “Preparing for Operator Licensing Initial Examinations.”

27

28 c. The NRC expects the facility licensee to provide a sample plan that meets the guidelines 

29 of Form 6.1-2, “Examination Sample Plan,” for the NRC’s use in developing the 

30 examination.

31

32 4. Examination Team Selection

33 a. The NRC will contribute no fewer than two examiners to the examination team. The 

34 regional office should consider assigning additional examiners if the operating crews for 

35 the dynamic simulator examinations contain five or more operators. To promote 

36 consistency in requalification program administration, regional office management 

37 should try to assign an examiner who participated in a prior requalification inspection or 

38 examination at the facility to be part of the NRC’s examination team.

39

40 In most cases, the NRR operator licensing program office will send a representative to 

41 observe the examination process or an examiner to participate as an additional member 

42 of the examination team. The program office will work with the responsible regional 

43 supervisor to make the necessary arrangements.

44

45 b. The facility licensee is expected to provide an employee to work with the NRC as part of 

46 the requalification examination team. The employee should be drawn from the 

47 operations staff and must be an active senior reactor operator (SRO) as defined in 

48 10 CFR 55.53(e) or (f). The NRC encourages the facility licensee to designate another 

49 employee from the training staff to be a member of the examination team. This 
ES-6.1, Page 5 of 33 

1 employee should also be a licensed SRO but may be a certified instructor. If the facility 

2 licensee desires and the NRC chief examiner agrees, the facility licensee may also 

3 include additional employees from the operations or training staffs who have 

4 qualifications comparable to the facility’s other examination team members.

5 

6 The function of these examination team members is to provide facility-specific technical 

7 assistance to the NRC in developing and reviewing the written examination items, plant 

8 walkthrough topics, and dynamic simulator scenarios. If necessary, the facility 

9 representatives may participate as facility evaluators in conducting the operating test or 

10 written examination. However, the facility representatives should be used as evaluators 

11 only if they routinely perform that function during the administration of the facility 

12 licensee’s requalification program.

13 5. Examination Development

14 a. The facility licensee may develop proposed written examinations and operating tests and 

15 forward them to the NRC as part of its reference material submittal. In accordance with 

16 10 CFR 55.59(a)(2)(ii), the facility licensee must ensure that the operating tests require 

17 the operators to demonstrate an understanding of and ability to perform the actions 

18 necessary to accomplish a comprehensive sample of the items specified in 

19 10 CFR 55.45(a)(2)–(13), inclusive, to the extent applicable to the facility.

20

21 b. Approximately 2 weeks before the scheduled examinations, the NRC examiners will visit 

22 the facility to make final preparations for the examination. The written, walkthrough, and 

23 dynamic simulator examinations will be developed in accordance with ES-6.2, ES-6.3, 

24 and ES-6.4, respectively. The examination should distinguish between reactor operator 

25 (RO) and SRO K/As to the extent that the facility training materials allow the examiners 

26 to make these distinctions. The NRC examiners will rely on the facility licensee’s 

27 examination team members for site-specific technical assistance in developing, 

28 reviewing, and validating the written examination static scenarios and items, plant 

29 walkthrough topics (JPMs), and dynamic simulator scenarios.

30

31 c. The NRC chief examiner and the responsible regional supervisor will determine the 

32 required length of time on site and the required number of examiners. This 

33 determination will be based on the experience of the examiners, the quality of the facility 

34 licensee’s testing material, and the level of effort required to develop new test items.

35

36 d. If requested by the facility licensee, the NRC chief examiner will brief the operators and 

37 managers about the requalification examination process. The NRC chief examiner will 

38 use this time to explain the examination and grading processes and to respond to any 

39 questions that the operators may ask about the NRC’s examination procedures. If the 

40 schedule does not allow them to meet during the preparation week, they may meet at 

41 any mutually agreeable time.

42

43 6. Examination Security

44 a. To ensure examination security, each facility representative who acquires knowledge of 

45 the content of the NRC’s requalification examination before it is administered will be 

46 subject to the security restrictions described below from the time he or she first acquires 

47 the specific knowledge until the examination exit meeting.

48
ES-6.1, Page 6 of 33 

1 b. To the maximum extent possible, only the examination team members and a simulator

2 operator should be given specific knowledge about the content of the examination. The

3 facility evaluators should receive the package of simulator scenarios and JPMs the week

4 before the examination to allow them to prepare for their evaluation, including

5 coordinating the use of the simulator to perform JPMs and scenarios. If the facility

6 licensee submits a proposed examination, those who participate in developing the

7 examination become subject to the security restrictions when their involvement begins.

8 Also, if facility representatives other than the examination team members are used to

9 time validate the written examination, they too become subject to the security restrictions

10 as soon as they are exposed to the examination questions.

11

12 c. Facility representatives who acquire specific knowledge of the NRC’s examinations will

13 sign Form 6.1-3, “Examination Security Agreement,” before their examination

14 involvement begins and again after the examination process is complete (i.e., following

15 the exit meeting).

16

17 D. Operator and Program Evaluation Procedures

18 1. Examination Administration

19 a. Each selected operator will take a requalification examination using ES-6.2, ES-6.3, and 

20 ES-6.4 for the written, walkthrough, and simulator portions of the requalification 

21 examination, respectively. Operator performance will be documented on Form 6.1-6, 

22 “Individual Requalification Examination Report.”

23

24 b. The number of persons present during an operating test should be limited to ensure the 

25 integrity of the test and to minimize distractions to the operators. Under no 

26 circumstances will another operator be allowed to witness an operating test. Exam 

27 security considerations preclude using operating tests as training vehicles for potential 

28 future applicants.

29

30 c. Other NRC examiners may observe an operating test as part of their training or to audit 

31 the performance of the examiner administering the operating test. The NRC chief 

32 examiner may permit others (such as resident inspectors, regional personnel, 

33 researchers, or NRC supervisors) to observe an operating test if the applicant does not 

34 object to the observers’ presence. The NRR operator licensing program office must 

35 approve deviations from this policy in advance.

36

37 d. Other non-NRC personnel (e.g., representatives from the Institute of Nuclear Power 

38 Operations or the Nuclear Energy Institute) may observe the operating tests with prior 

39 approval from the NRR operator licensing program office. The NRC chief examiner will 

40 control the observers’ activities in accordance with guidance provided by the program 

41 office.

42

43 2. Examination Grading

44 a. The NRC expects the facility licensee to grade the written examinations and operating 

45 tests in parallel with the NRC’s examiners.

46

47 b. The agency expects the facility evaluators to provide preliminary pass/fail results for the 

48 simulator and walkthrough portions of the examination by the end of each day and the 
ES-6.1, Page 7 of 33 

1 final results before the exit briefing or at the end of each examination week for multiweek 

2 examinations.

3 

4 c. The NRC will notify the facility licensee immediately if any operator’s performance on the 

5 examination is sufficiently poor to require immediate removal from licensed activity. The 

6 NRC will also notify the facility licensee of the results of the examination. 

7 

8 d. The facility licensee will provide the NRC with the final results of the written 

9 examinations and an overall summary of the examination results within 2 weeks after the 

10 exit meeting.

11

12 3. Evaluation of Requalification Programs

13 A requalification program evaluation requires a minimum sample size of 12 operators. The 

14 sample size is determined by counting the number of operators taking the dynamic simulator 

15 examination. This total includes those operators who participate in the simulator examination 

16 only for the purpose of meeting crew composition requirements but excludes those operators 

17 who are being reexamined after failing a previous NRC-conducted examination.

18

19 If less than one-half of the operators taking the dynamic simulator examination complete the 

20 entire examination, the NRC regional supervisor will determine whether a valid program 

21 evaluation can be made. In these instances, the regional supervisor will contact the NRR 

22 operator licensing program office.

23

24 A satisfactory requalification program meets each of the following criteria:

25

26 1. At least 75 percent of the operators must pass all portions of the examination in

27 which they participate. The pass rate is determined by dividing the number of

28 operators who pass all portions of the examination in which they participate by

29 the total number of operators in the sample.

30

31 In the event of a crew failure, only those operators who receive a satisfactory

32 evaluation in the individual followup evaluation will be counted when calculating

33 the operator pass rate.

34

35 When calculating the pass rates, fractions should be rounded up to the next

36 highest whole number. For example, if 15 operators are evaluated, 75 percent

37 passing would be 11.25 operators; thus, 11 of 15 passing would not meet the

38 75-percent requirement, but 12 would.

39

40 2. At least two-thirds (66 percent) of the crews must pass the simulator

41 examination.

42

43 For requalification examinations with more than three crews participating, three

44 out of four, or four out of five crews, must pass to satisfy this requirement.

45

46 The NRC will consider the following areas in the overall program evaluation and may use the 

47 related findings to identify facility weaknesses that will be documented in the examination report:

48

49 1. The facility evaluators do not concur with the NRC examiners on all

50 unsatisfactory crew evaluations.
ES-6.1, Page 8 of 33 

1 

2 2. More than one facility evaluator is determined to be unsatisfactory. Section I,

3 “Walkthrough Evaluation Techniques,” of ES-3.5, “Administering Operating

4 Tests,” provides guidance that examiners should use to assess evaluator

5 competence.

6 

7 3. The facility licensee failed to train and evaluate an operator in all positions

8 permitted by the individual’s license. (For example, the facility is required to train

9 and evaluate an SRO in the RO position, as well as in directing operators.) An

10 SRO will not be required to perform RO activities during the simulator portion of

11 the operating test; however, his or her performance will be evaluated if the facility

12 normally places the SRO in a shift RO position during the simulator examination.

13 Otherwise, RO skills will be evaluated during the performance of JPMs.

14

15 4. The facility licensee has insufficient administrative controls to preclude an RO or

16 SRO with an inactive license from performing licensed duties. Operators must

17 meet the requirements of 10 CFR 55.53, “Conditions of licenses,” to restore an

18 inactive license to active status.

19

20 5. The facility licensee has insufficient quality control of its examination bank. The

21 NRC will evaluate the facility’s performance in this area if postexamination

22 changes to facility-developed test items result in significant modifications or

23 deletions of more than 10 percent of the questions on the written examination.

24

25 6. The number of test items duplicated from any past examination or combination of

26 examinations administered during the current requalification training cycle (as

27 described in 10 CFR 55.59(a)(1)) or the number of operating test items repeated

28 on successive days of an examination period is such that the discrimination

29 validity and integrity of the examination could be affected. When test items are

30 repeated, they should be selected in a distributed manner and approximately

31 equally over all previous examinations to reduce predictability (if a large number

32 of items were taken from the most recent examination).

33

34 7. The facility licensee’s failure decisions are not as conservative as the NRC’s. To

35 ensure that the rationale for the evaluation is fully understood, the NRC will

36 review with the facility managers any case in which the facility licensee passed

37 an operator whom the NRC failed. In addition, the NRC will assess whether the

38 facility licensee’s evaluations are conducted in accordance with documented

39 facility guidance and whether facility managers periodically assess their

40 evaluation process.

41

42 The NRC also expects the facility program to explicitly link an operator’s

43 examination failure with unsafe performance. In this way, all facility failures and

44 NRC failures will agree. In certain instances, the facility licensee’s program may

45 have operator performance standards that are not explicitly linked to unsafe

46 performance and thus do not meet the threshold stated in these standards for the

47 operator to fail the examination. In such instances, the facility licensee is

48 expected to differentiate failures in which the operator performed at an unsafe

49 level from those in which the operator failed for reasons other than safety

50 (i.e., not meeting higher facility established performance standards). In these
ES-6.1, Page 9 of 33 

1 instances, operators identified as failing for safety reasons would also be 

2 considered NRC failures.

3 

4 4. Evaluation of Operator Performance

5 To pass the NRC-conducted requalification examination, the operator must pass a written 

6 examination and an operating test consisting of a walkthrough examination and a dynamic 

7 simulator examination. These examinations are developed and administered in accordance 

8 with ES-6.2, ES-6.3, and ES-6.4, respectively, unless the NRR operator licensing program office 

9 authorizes the regional office to use the facility licensee’s alternative examination methodology. 

10 To pass the operating test, the operator must also be a member of a crew that passes the 

11 dynamic simulator examination.

12

13 E. Unsatisfactory Operator or Program Evaluation

14 1. Actions Following an Unsatisfactory Operator Evaluation

15 In all cases, a facility licensee’s administrative procedures should ensure that an operator who 

16 fails a requalification examination is removed from licensed duties, given remedial training, and 

17 reexamined before being allowed to return to licensed duties. This also applies to an SRO who 

18 performs only RO-level duties at the facility when the failure is caused solely by activities 

19 involving SRO responsibility. Section H contains the instructions for notifying the operator about 

20 his or her performance on the NRC-conducted requalification examination, as well as guidance 

21 about the actions to be taken for an operator to return to licensed duty.

22

23 The NRC has deleted 10 CFR 55.57(b)(2)(iv)), which required an operator to pass an 

24 agency-administered requalification examination as a prerequisite for license renewal. 

25 Nonetheless, it would be inappropriate to renew the license of any operator who failed to pass 

26 any NRC-conducted requalification examination without some level of agency involvement in 

27 the retesting process. The amount of NRC involvement may include conducting the retest in 

28 accordance with the appropriate examination standard(s); inspecting the facility licensee in 

29 accordance with IP 71111.11, “Licensed Operator Requalification Program and Licensed 

30 Operator Performance,” as it retests the operator; or reviewing the reexamination prepared by 

31 the facility licensee. The regional office, in consultation with the NRR operator licensing 

32 program office, will determine the appropriate level of involvement on a case-by-case basis 

33 depending on the quality of the facility licensee’s program. As long as the operator submits a 

34 timely renewal application, the term of the license will continue until the renewal requirements 

35 are satisfied or the operator fails three NRC-conducted examinations as discussed in Section H. 

36

37 If an operator who failed a requalification examination is not prepared for a reexamination after 

38 6 months of remedial training, the regional office will request the following information from the 

39 facility licensee:

40

41 • confirmation that the facility licensee still has a need for the individual’s license

42

43 • the expected completion date of the operator’s remedial training and when the facility 

44 licensee will be ready to administer its retake examination 

45

46 • assurance that the operator will not return to licensed duties until he or she successfully 

47 retakes the examination (or portion thereof) administered by the facility licensee with a 

48 satisfactory requalification program or in accordance with the provisions of the 
ES-6.1, Page 10 of 33 

1 confirmatory action letter (CAL) if the facility licensee has an unsatisfactory program and 

2 the NRC has not determined it to be “provisionally satisfactory”

3 

4 The NRC will inform the facility licensee that a comprehensive requalification examination may 

5 be necessary if the operator is not ready to take a retest within 1 year after failing the 

6 examination.

7 

8 2. Actions Following an Unsatisfactory Requalification Program Evaluation

9 The NRC will take the following actions for all requalification programs that the agency 

10 evaluates as unsatisfactory:

11

12 a. The NRC expects the facility licensee to identify program deficiencies and corrective 

13 actions to improve operator performance. The NRC will use a CAL to establish a formal 

14 dialogue and to document the facility licensee’s corrective action commitments.

15

16 An operator who fails the requalification examination, as determined by the NRC, will be 

17 subject to an NRC-administered reexamination before resuming licensed duties.

18

19 The NRC expects the facility licensee to remediate and reevaluate an operator whose 

20 performance does not meet facility standards, as determined by the facility licensee, in 

21 accordance with the provisions of the facility licensee’s requalification program. The 

22 NRC will review or monitor the reexamination, or both, to ensure the adequacy of the 

23 facility licensee’s requalification program.

24

25 b. The NRC will schedule a meeting with senior facility managers to review the examination 

26 results, as well as the identified deficiencies and their root causes, the proposed 

27 corrective actions and the schedule for their implementation, and the need for followup 

28 inspections and examinations. (Section E.3 contains additional guidance on conducting 

29 augmented inspections.)

30

31 The Regional Administrator will evaluate the examination and inspection results and 

32 make a decision as to the continued operation of the facility and possible enforcement 

33 action against the facility licensee. At a minimum, the Regional Administrator should 

34 consider the following factors when making this determination:

35

36 • the results of previous program evaluations, including corrective actions

37

38 • the significance of generic performance deficiencies identified during the program

39 evaluation

40

41 • recent facility events that relate to licensed operator performance

42

43 • recommendations by the NRC staff (including the results of any operational

44 evaluations and inspections)

45

46 c. If operator performance deficiencies cause the unsatisfactory program evaluation, an 

47 operational evaluation is required. The operational evaluation is intended to help the 

48 Regional Administrator determine whether the facility’s remaining operating crews are 

49 suitably qualified to continue to operate the facility. In this case, the facility licensee 

50 identifies the individual operators and shift crews it proposes to use to continue plant 
ES-6.1, Page 11 of 33 

1 operations. The regional office may choose not to evaluate those operators who passed 

2 their most recent NRC-conducted initial or requalification examination within the past 

3 12 months. However, the regional office will evaluate all other operators in those areas 

4 noted as operational deficiencies during the requalification examination regardless of 

5 whether they have already passed or not yet taken the facility-administered 

6 requalification examination. The regional office will conduct the operational evaluations 

7 in accordance with applicable guidance in ES-6.3 and ES-6.4. 

8 

9 If the facility licensee proposes to use a shift crew that is significantly different from its 

10 normal configuration, even though all the operators may have recently passed an 

11 NRC-conducted examination, the regional office may perform an operational evaluation 

12 of this crew.

13

14 The regional office should schedule the operational evaluation as soon as possible after 

15 determining that the facility licensee’s requalification program is unsatisfactory. The 

16 evaluation should not be delayed to accommodate the facility’s operating schedule, the 

17 completion of programmatic corrective actions, or the completion of remedial training for 

18 operators who failed the requalification examination. The operational evaluation may 

19 identify further program deficiencies that may need to be reflected in the CAL discussed 

20 in Section E.2(a) or may warrant additional inspection by the NRC. Additional operator 

21 weaknesses that require remediation may also be identified.

22

23 d. The NRC will review the corrective actions the facility is to perform, the expected 

24 followup actions by the NRC, and the schedule for each.

25

26 As part of the followup activities, the NRC may conduct additional operational 

27 evaluations, requalification retake examinations, and augmented inspections (as 

28 necessary). Before these activities, the NRC will verify that the facility licensee has 

29 completed the applicable corrective actions and will obtain a certification of crew 

30 readiness from the facility managers. Regional managers should consider using a new 

31 NRC chief examiner and having examiners from other regional offices participate in 

32 those operational evaluations and requalification retake examinations that have potential 

33 implications for restart approval.

34

35 e. The Regional Administrator will incorporate into the decision on followup activities any 

36 extraordinary circumstances surrounding the examination that may affect the validity of 

37 the examination results.

38

39 f. When the NRC determines that a requalification program is unsatisfactory, the program 

40 will remain unsatisfactory until the facility licensee completes all identified corrective 

41 actions agreed on by the NRC for restoring the program to satisfactory status and the 

42 NRC completes all related followup activities. For purposes of allowing facility 

43 examiners to perform reexamination functions, however, a facility may attain a status of 

44 “provisionally satisfactory” provided that the facility has completed to the NRC’s 

45 satisfaction all short- and intermediate-term corrective actions agreed on with the NRC.

46

47 Once the NRC determines that the facility licensee has satisfactorily implemented these 

48 corrective actions, the Regional Administrator or designee will determine whether to 

49 permit the facility to reexamine all operators who failed the NRC-conducted 

50 requalification examination for the purpose of returning the operators to licensed duties. 

51 Any operator who fails the NRC-conducted examination still needs to pass a future 
ES-6.1, Page 12 of 33

1 NRC-administered (i.e., conducted, inspected, or approved, as appropriate) 

2 requalification examination to renew the license. Long-term corrective actions are 

3 expected to be completed before the NRC’s next requalification program evaluation 

4 (IP 71111.11).

5

6 To attain a satisfactory rating following an unsatisfactory evaluation, the subsequent 

7 requalification program evaluation, with a sample size of at least 12 operators, must 

8 satisfy the passing criteria in Section D.3.

9

10 g. The Regional Administrator or designee may specify additional actions, as appropriate. 

11 The specific sequence of actions is not critical; however, this sequence of events 

12 corresponds to a typical regional response to an unsatisfactory program evaluation. The 

13 Regional Administrator or designee should defer determining whether a plant shutdown 

14 is required until he or she reviews all factors listed in Section E.2(b).

15

16 3. Augmented Inspection Guidelines

17 If the NRC determines that an augmented requalification program inspection is required, 

18 regional management should define its scope and depth based on the nature of the 

19 deficiencies.

20

21 The regional office should consider the following activities in addition to those specified in 

22 Section E.2:

23

24 a. The regional office may conduct augmented inspection coverage of all shifts. The 

25 inspection procedures for shift coverage should be used as appropriate. Inspection 

26 activities should devote attention to the following areas:

27

28 • operator performance and attitude

29 • operator overtime

30 • management oversight

31 • shift staffing

32

33 b. The regional office may develop a long-term training program inspection plan based on 

34 IP 41500. Such an inspection plan may include the following activities:

35

36 • ongoing status reviews of requalification training effectiveness, with an emphasis 

37 on known program deficiencies and implementation of short-term corrective 

38 actions

39

40 • an inspection to determine the root cause(s) for the unsatisfactory requalification 

41 program evaluation and to verify that the facility licensee’s proposed corrective 

42 action plan should preclude or minimize the probability of recurrence

43

44 • an inspection to evaluate the adequacy of the facility licensee’s corrective actions 

45 and to determine the effectiveness of the facility licensee’s SAT-based 

46 requalification program

47

48 c. The regional office may convene an enforcement panel to determine whether action is 

49 warranted on the basis of the requirements of 10 CFR 50.54(i-1). Potentially, a 
ES-6.1, Page 13 of 33 

1 requalification program rated unsatisfactory on two successive NRC evaluations does 

2 not meet the minimum requirements of 10 CFR 55.59(c) as required by 

3 10 CFR 50.54(i-1). The basis for any proposed enforcement action will be the 

4 inadequate corrective action or requalification program element deficiencies (identified 

5 by the inspections related to Section E.3(b)) that led to the successive requalification 

6 examination failures.

7 

8 F. Requalification Program Evaluation Report

9 After the Regional Administrator or designee approves the requalification examination results, 

10 the regional office will prepare a final requalification program evaluation report. The program 

11 evaluation report needs to include a copy of the written examination only if the report addresses 

12 written examination problems. The regional office will issue the report within 30 days following 

13 receipt of the facility licensee’s final results or the examination exit meeting, whichever is later, 

14 and will place a complete copy of the report in the facility’s requalification file.

15

16 The NRC chief examiner is responsible for completing Form 6.1-5, “Power Plant Requalification 

17 Results Summary Sheet.” The examiner will enter each operator’s scores in the appropriate 

18 columns and add a continuation sheet to document scores for all the operators. Under the 

19 “Simulator” column, the examiner will enter the results of the operator’s individual followup 

20 evaluation. If the operator did not receive an individual followup evaluation, the examiner will 

21 enter a passing score. If an operator was a member of a crew that failed the dynamic simulator 

22 examination, but the operator passed or did not receive an individual followup evaluation, the 

23 examiner will enter a pass in the “Simulator” column for that operator. Crew failures will be 

24 summarized in the overall results at the top of Form 6.1-5. 

25

26 The regional office will send a copy of the summary (and continuation) sheet(s) to the NRC 

27 Headquarters operator licensing assistant. The NRR operator licensing program office uses the 

28 results summary to verify the data in the Reactor Program System—Operator Licensing, so that 

29 statistics can be maintained on operator performance. The regional office will not include the 

30 results summary in the examination report as the summary contains information subject to the 

31 Privacy Act. 

32

33 If a small number of operators are given retake examinations, the regional office may issue an 

34 addendum to the original requalification evaluation report instead of issuing a new report. If the 

35 reexaminations are conducted concurrently with initial examinations or inspected during a 

36 requalification program evaluation in accordance with IP 71111.11, the results may be reported 

37 as part of the initial examination or inspection report.

38

39 G. Individual Requalification Examination Report

40 After the regional office completes the requalification evaluation, it will keep a copy of each 

41 operator’s NRC-conducted written, walkthrough, and simulator examination results and return 

42 the original documents to the facility licensee. The regulation in 10 CFR 55.59 requires the 

43 facility licensee to maintain records of these examination results, along with a copy of the written 

44 examination, until the operator’s license is renewed or 2 years after the license expires.

45

46 The NRC chief examiner will ensure the completion of Form 6.1-6 for each operator who takes 

47 an NRC-conducted requalification examination. The report will include the following information 

48 for each individual:

49
ES-6.1, Page 14 of 33

1 • written examination grade

2 • the crew evaluation from the dynamic simulator examination

3 • the individual followup results (pass or fail) from the dynamic simulator examination

4 • the number (and percentage) of JPMs performed correctly, if JPMs were conducted

5

6 The regional office will send a copy of this report to the facility’s training manager and the 

7 operator with a letter notifying the operator of the examination results. The regional office will 

8 also file a copy in the operator’s docket file.

9

10 H. NRC-Conducted Requalification Examination Results

11 1. Passing an NRC-Conducted Requalification Examination

12 a. An operator who passes all portions of the requalification examination, including being a 

13 member of a crew that passes the dynamic simulator examination, will receive written 

14 notification from the NRC regional office.

15

16 2. Failing an NRC-Conducted Requalification Examination

17 a. The NRC regional office will notify the operator in writing of a failure of the requalification 

18 examination. On receiving the failure notification, the operator can request an informal 

19 review of the failed portion of the examination. The request must be made as described 

20 in the failure notification letter.

21

22 b. If an operator fails any part of an NRC-conducted requalification examination, the facility 

23 licensee is expected to remove the operator from licensed duty and take corrective 

24 action consistent with the provisions of its requalification program before returning the 

25 operator to licensed duty. If the facility licensee’s requalification program is 

26 unsatisfactory, Section E.2 of this examination standard lists other recommended 

27 actions, including those actions the facility licensee is expected to complete before 

28 attaining a “provisionally satisfactory” requalification program status.

29

30 c. The NRC will normally administer a second (first retake) examination approximately 

31 6 months after issuing the first failure. That examination will concentrate on the areas in 

32 which the operator exhibited deficiencies.

33

34 d. The NRC will normally administer a third (second retake) examination approximately 

35 6 months after issuing the second failure. The third examination will be a 

36 comprehensive requalification examination.

37

38 Regardless of the status of the facility licensee’s requalification program, if an operator 

39 fails a third requalification (second retake) examination, the NRC will thoroughly review 

40 the operator’s examination performance and may conduct a complete review of the 

41 facility licensee’s training program. The third failure may be grounds for suspending or 

42 revoking the operator’s license. If an operator has an application pending for license 

43 renewal with the NRC at the time of a third requalification failure, that failure will provide 

44 the basis for denying the application. Notification of the operator will be handled on a 

45 case-by-case basis and coordinated through the NRR operator licensing program office.

46
ES-6.1, Page 15 of 33 

1 I. Operator Licensing Renewal Policy

2 Operators are not required to take an NRC-conducted requalification examination to renew their 

3 licenses. However, if an operator takes, but fails to pass, an NRC-conducted examination, the 

4 NRC will not renew the license until the operator passes a retake examination conducted by the 

5 NRC, passes a retake examination administered by the facility licensee and inspected by the 

6 NRC in accordance with IP 71111.11, or passes an examination approved by the NRC. The 

7 regional office, in consultation with the NRR operator licensing program office, will determine the 

8 appropriate level of involvement on a case-by-case basis depending on the quality of the facility 

9 licensee’s requalification program.

10

11 ES-5.3, “Maintaining, Changing, and Renewing Operator Licenses,” contains the instructions for 

12 processing license renewal applications. 

13

14 J. Records Retention

15 1. Facility Licensee Requalification Examination File

16 The NRC’s regional office shall ensure that the original (whenever possible) or a copy of the 

17 following items is electronically available through the Agencywide Documents Access and 

18 Management System (ADAMS): 

19

20 a. examination standard attachments and forms:

21 • Letter 6.1-1

22 • Form 6.1-3, “Examination Security Agreement”

23 • Form 6.1-5, “Power Plant Requalification Results Summary Sheet”

24 • Form 6.4-2, “Simulator Crew Evaluation Form”

25

26 b. a master list of all JPMs administered and the operators to whom they were

27 administered

28

29 c. a master list of all scenarios conducted and operators to whom they were administered

30 (facility-generated forms or Form 3.3-1, “Scenario Outline,” may be used to meet this

31 requirement)

32

33 d. a copy of the written examination and answer key

34

35 e. a copy of the requalification examination report

36

37 The regional office may require that additional documents be retained in the facility’s 

38 requalification examination file. (Note that paper copies of examinations administered before 

39 the implementation of ADAMS may be discarded after confirming that the examination report is 

40 available in ADAMS.)

41

42 2. Operator Docket Files

43 The regional office will retain the following records in each operator’s docket file until the license 

44 is renewed or 2 years after the license expires or is terminated:

45

46 • Form 6.1-6 
ES-6.1, Page 16 of 33 

1 • results notification letter

2 • a copy of all failed portions of the NRC-graded examination

3 

4 3. Other Files

5 The regional office will retain reference materials used to develop each examination until the 

6 NRC has resolved with the facility licensee all failures associated with the examination and has 

7 sent a notification letter to each operator.

8 

9 K. Feedback on Requalification Examination Stress

10 The level of stress perceived by operators and facility personnel can affect their overall 

11 performance on the requalification examination. Therefore, the NRR operator licensing program 

12 office is interested in monitoring the stress of operators and facility personnel participating in the 

13 requalification examination. Regional examiners and other personnel who participate in an 

14 NRC requalification examination should assume the following responsibilities: 

15

16 • Monitor the level of stress in operators and facility representatives and be alert for 

17 examination techniques that may be causing examination stress.

18

19 • Recommend to the program office any changes to NUREG-1021 that would further 

20 alleviate operator stress. Document recommendations and forward them to NRC 

21 Headquarters using report on interaction forms.

22

23 L. Forms and Letters

24 Form 6.1-1 Examination Timetable

25 Form 6.1-2 Examination Sample Plan

26 Form 6.1-3 Examination Security Agreement

27 Form 6.1-4 Evaluation Checklist for Facility Reference Material

28 Form 6.1-5 Power Plant Requalification Results Summary Sheet

29 Form 6.1-6 Individual Requalification Examination Report

30 Letter 6.1-1 Sample Corporate Notification Letter

31
ES-6.1, Page 17 of 33 

1 Form 6.1-1 Examination Timetable

2 

3 Date* Activity

4 

5 -120/90 The U.S. Nuclear Regulatory Commission (NRC) notifies the facility licensee.

6 

7 -60 The facility licensee sends the NRC the materials requested for developing the 

8 examination (including written examination questions, simulator scenario banks, 

9 and job performance measures (JPMs)).

10

11 The facility licensee proposes composition of the crews to be evaluated and 

12 identifies facility examination team members.

13

14 The facility licensee may ask the NRC chief examiner to review the examination 

15 process with operators and managers.

16

17 -45 The facility licensee submits its proposed requalification written examination and 

18 operating test.

19

20 -30 The NRC concurs on the operating crews to be evaluated.

21

22 -14 The NRC examiners visit the facility to review the JPMs to be administered, 

23 observe the static and dynamic simulator examinations, and validate the test 

24 items (as needed). The chief examiner and the regional Branch Chief determine 

25 the length of time on site and the number of examiners required on the basis of 

26 the examiners’ experience and the quality of the facility licensee’s testing 

27 materials.

28

29 The facility licensee designates a simulator operator. 

30

31 If requested, the chief examiner briefs the operators and managers about the 

32 requalification examination process.

33

34 -7 The facility examination team members finalize the examinations based on 

35 preparation week activities. Evaluators review reference material to prepare for 

36 the JPMs and simulator scenarios.

37

38 0 The NRC administers the examinations to selected crews and operators. The 

39 facility licensee notifies the NRC of its final results for crews and individuals at 

40 the end of each examination week. 

41

42 +7 The NRC finalizes the examination results.

43

44 +14 The facility licensee transmits the written examination grades and a final 

45 summary to the NRC.

46

+30# 47 The NRC issues operator results and the final requalification examination report.

48

49 * Number of days before (-) or after (+) the examination, except as noted.

50 # Number of days after receipt of facility results or examination exit meeting, whichever is later.
ES-6.1, Page 18 of 33 

1 Form 6.1-2 Examination Sample Plan

2 A. Introduction

3 

4 An examination sample plan provides a systematic approach to selecting and developing test 

5 items to determine whether a student has mastered the knowledge and abilities (K/As) and skills 

6 to be covered in a training program. The sample plan should provide an explicit, documented 

7 link between the learning objectives associated with the training program and the test items 

8 used to perform the evaluation and to verify the relevance to the job task analysis (JTA) 

9 associated with the operator’s position.

10

11 Examination Standard (ES)-4.1, “Preparing Written Examination Outlines,” gives explicit 

12 guidance for developing a sample plan for initial examinations using the applicable K/A catalog. 

13 A similar methodology may be applied to any training program. With respect to a requalification 

14 program, the scope of topics is necessarily limited because less material is covered during a 

15 requalification program than in an initial licensing training program. However, the U.S. Nuclear 

16 Regulatory Commission (NRC) permits and encourages reserving 10 to 20 percent of test items 

17 for topics that have high importance ratings and contain K/As that operators should retain 

18 because of their safety significance but were not necessarily covered during the requalification 

19 cycle. 

20

21 B. Requalification Test Outline

22

23 The facility licensee is expected to develop a test outline for all NRC-administered 

24 requalification examinations. At least 80 percent of the test outline must reflect the training 

25 curriculum of the most recent requalification cycle in a manner consistent with the distribution of 

26 emphasis in the curriculum.

27

28 The curriculum of the requalification training cycle for which the examination is being developed 

29 should identify the following:

30

31 • requalification lecture/classroom topics indicating the percentage of the cycle devoted to 

32 each

33

34 • concentration of training exercises using the simulation facility, including the types of 

35 scenarios trained for (e.g., accident, abnormal, normal) and the number of times each 

36 scenario was run

37

38 • special focus of the training, such as plant modifications, licensee event reports, and 

39 major changes to operating practices or policy

40

41 • practical training, such as operation of individual systems or components for 

42 requalification training purposes, using either the simulation facility, mockups, or actual 

43 systems and components

44

45 The format of the sample plan is a matter of training department preference as long as the plan 

46 results in a thorough and accurate assessment of the facility’s training program and its intended 

47 objectives. The NRC expects the sample plan to contain the following information for use in 

48 developing or selecting the test items to be used in the requalification examination:

49
ES-6.1, Page 19 of 33

1 • identification of the subjects to be evaluated (system, component, procedure, or other 

2 training subject)

3

4 • the preferred testing medium for evaluating each subject (written, simulator, or 

5 walkthrough examination); more than one testing method may be used to evaluate a 

6 subject

7

8 • the learning objectives intended to be evaluated

9

10 • a list of references used to develop the test items

11

12 • the specific K/A topic or facility JTA K/As and skills that are closely linked to the learning 

13 objectives for each subject and the importance factors for each (the facility licensee may 

14 use a site-specific K/A if it exists)

15

16 • a K/A value of 3 or greater for all test items used in the examination; the facility licensee 

17 may propose the use of test items with NRC K/A values less than 3 with appropriate 

18 justification

19

20 • the percentage or number of points of the examination that should be devoted to the 

21 topic area (e.g., 3 points for technical specification interpretation or 5 percent on reactor 

22 coolant pumps)

23

24 • whether the subject is identified as safety related in the facility’s JTA

25

26 • whether the subject was covered in the cycle for which the examination is being 

27 developed

28

29 • the identification code or number for previously developed test items that evaluate the 

30 subject

31

32 • recent safety-related issues and events (e.g., relevant licensee event reports)

33
ES-6.1, Page 

2

0 of 33

Form 6.1-3 Examination Security Agreement

1. Preexamination

I acknowledge that I have acquired specialized knowledge about the U.S. Nuclear Regulatory Commission (NRC) requalification 

for the week(s) of as of the date of my signature. I agree that I will not knowingly divulge any information about these ex

persons who have not been authorized by the NRC chief examiner. I understand that I am not to instruct, evaluate, or provide pe

those operators scheduled to be administered these examinations from this date until completion of examination administration, e

noted below and authorized by the NRC (e.g., acting as a simulator booth operator or communicator is acceptable if I do not sele

provide direct or indirect feedback). Furthermore, I am aware of the physical security measures and requirements (as documente

procedures) and understand that violation of the conditions of this agreement may result in cancellation of the examinations or an 

against me or the facility licensee (or both actions). I will immediately report to facility management or the NRC chief examiner an

suggestions that examination security may have been compromised.

2. Postexamination

To the best of my knowledge, I did not divulge to any unauthorized persons any information concerning the NRC requalification e

administered during the week(s) of . From the date that I entered into this security agreement until the completion of e

administration, I did not instruct, evaluate, or provide performance feedback to those operators who were administered these exa

specifically noted below and authorized by the NRC.

PRINTED NAME JOB TITLE/RESPONSIBILITY SIGNATURE (1) DATE SIGNATURE 

1. ____________________ ______________________________ _______________________ ____________ _____________

2. ____________________ ______________________________ _______________________ ____________ _____________

3. ____________________ ______________________________ _______________________ ____________ _____________

4. ____________________ ______________________________ _______________________ ____________._____________

5. ____________________ ______________________________ _______________________ ____________ _____________

6. ____________________ ______________________________ _______________________ ____________ _____________

7. ____________________ ______________________________ _______________________ ____________ _____________

8. ____________________ ______________________________ _______________________ ____________ _____________

9. ____________________ ______________________________ _______________________ ____________ _____________

10. ____________________ ______________________________ _______________________ ____________ _____________

11. ____________________ ______________________________ _______________________ ____________ _____________

12. ____________________ ______________________________ _______________________ ____________ _____________

13. ____________________ ______________________________ _______________________ ____________ _____________

14. ____________________ ______________________________ _______________________ ____________ _____________

15. ____________________ ______________________________ _______________________ ____________ _____________

16. ____________________ ______________________________ _______________________ ____________ _____________

NOTES:
ES-6.1, Page 21 of 33

Form 6.1-4 Evaluation Checklist for Facility Reference Material

This checklist represents the minimum content of facility-generated reference material. Items 

marked “optional” should be checked if requested from the facility licensee by the chief 

examiner. The chief examiner or designee may use this checklist to make a quick, general 

evaluation of the completeness and adequacy of the facility licensee’s references. The chief 

examiner may resolve any specific questions about the references with the facility staff as 

necessary.

I. Quantity

Actual

Reference Material Required Minimum Submitted

A. Open-reference written 350 per section; bank

examination items is to be dynamic, with at

least 150 revised, reviewed,

or newly generated 

questions per year

B. Simulator scenarios 25; plus 5 per year following

the initial requalification exam

until at least 30 scenarios

covering all aspects of the 

emergency operating procedures

are developed

C. Job performance 95; plus 10 per year following

measures (JPMs) the initial requalification

exam until the job task analysis

is fully covered

D. Technical specifications 1 copy

E. Applicable 1 set (optional)

plant procedures

F. Emergency plan 1 copy

G. Applicable 1 copy (optional)

administrative procedures

H. Sample plan 1 copy 

I. Requalification cycle 1 set (optional)

training reference material

(e.g., lesson plans and handouts)

J. Appropriate sections 1 set (optional)

of the job task analysis or 

facility-specific

knowledge and ability (K/A) 

catalog

Reviewed by: ___________________________________ Date: _______________
ES-6.1, Page 22 of 33

II. Usability

Circle one

A. The reference material is legible. Yes No

B. The reference material is properly arranged and labeled for its function. Yes No

C. The reference material indicates a systems approach to training program. Yes No

D. Reference material is available to verify that test items are appropriate,

job relevant, and technically accurate. Yes No

E. Reference material is available to adequately support the examination 

topics. Yes No

Comments

Reviewed by: _______________________________ Date: __________________
ES-6.1, Page 23 of 33

III. Quality

Exam Section Required Standards Comments

A. Sample Plan Subjects covered in requalification cycle

are identified.

The test outline incorporates:

• time spent on topic

• relative importance 

• frequency of performance

• job level (reactor operator (RO)

or senior reactor operator (SRO))

The test outline identifies K/As (or facility

equivalent) of sufficient importance.

Plant-specific priorities are identified

(license event reports (LERs), procedure changes,

system modifications, risk-dominant accident 

scenarios, risk-important systems and operator 

actions0F

1 identified in, for example, the facility 

licensee’s probabilistic risk assessment 

(PRA)/individual plant examination (IPE)).

Appropriate testing methods are indicated for 

each K/A (i.e., JPM, written exam, and/or simulator).

Applicable learning objectives are associated with K/As.

A methodology exists to tie test items to a learning 

objective and a K/A.

Sample plan includes important topics not covered in the 

requalification cycle.

Test areas appropriate to ROs and SROs only 

are identified.

Reviewed by: ______________________________ Date: _______________

1 Chapter 13 of NUREG-1560, Revision 3, “Individual Plant Examination Program: Perspectives on Reactor 

Safety and Plant Performance,” issued December 1997, identifies important human actions that may be 

appropriate for evaluation.
ES-6.1, Page 24 of 33

III. Quality (continued)

Exam Section Required Standards Comments

C. Walkthrough At least 10 percent of the JPM bank were

reviewed using Form 3.2-4, “Job Performance 

Measure Development Job Aid”

Test outline identifies applicable plant systems:

• systems covered in requalification cycle

• new or recently modified systems

• systems in recent facility LERs

or vendor notices

• PRA-identified risk-dominant systems

• systems in U.S. Nuclear Regulatory Commission

(NRC) generic communications

Tasks/abilities for identified systems:

• are applicable to the facility

• are at the auxiliary operator (AO)/RO/SRO level

• have a K/A value of 3 or greater

• include JPMs pertinent only to SROs

Some JPMs are performed under

low-power or shutdown operating

conditions.

Some JPMs require the operator

to implement alternative paths

within the facility licensee’s

procedures.

Facility JPMs contain the information found on 

Form 3.2-4.

Reviewed by: ______________________________ Date: ___________________
ES-6.1, Page 25 of 33

III. Quality (continued)

Exam Section Required Standards Comments

D. Simulator At least 10 percent of the scenarios are

reviewed using Form 6.4-1, “Simulator 

Scenario Review Checklist.”

Scenarios are an appropriate measure of the 

material covered in the sample plan.

Scenarios are based on the following:

• lessons covered in the requalification

cycle 

• recent industry events

• LERs

• emergency and abnormal procedures

• design and procedural changes

Scenarios exercise the crew’s ability to use 

facility procedures in accident prevention and 

mitigation.

Scenario events have a K/A of 3 or greater.

Some scenarios are based on low-power1F

2 operations.

Some scenarios are based on the dominant accident 

sequences for the facility as determined by a PRA/IPE.

The scenario identifies critical tasks that meet the criteria

of ES-3.3, “Testing Guidelines for Dynamic Simulator Scenarios.”

Proposed examination scenarios that were used for 

training during the most recent training cycle

have been reviewed by the NRC and replaced or 

modified, if appropriate, to ensure the validity of the 

examination and to minimize the potential 

for examination compromise.

Reviewed by: _____________________________ Date: __________________

2 NUREG-1449, “Shutdown and Low-Power Operation at Commercial Nuclear Power Plants in the United 

States,” issued September 1993, defines “low power” to include the range from criticality to 5-percent power.
ES-6.1, Page 

2 of 3

3

Form 6.1-5 Power Plant Requalification Results Summary Sheet

Privacy Information—For Official Use Only

Facility: Overall Results Total Pa

Exam Date: # 

NRC Examiners:

Reactor Operator:

Senior Operator:

Total:

Crews:

Operator

Docket

55-( )

Grader JPMs

%

Written

(A & B)

Resu

Written

S

Crew 

NRC %

FAC %

NRC %

FAC %

NRC %

FAC %

NRC %

FAC %

NRC %

FAC %

NRC %

FAC %

NRC %

FAC %

Privacy Information—For Official Use Only
ES-6.1, Page 27 of 33

Form 6.1-6 Individual Requalification Examination Report

Privacy Information—For Official Use Only

U.S. Nuclear Regulatory Commission

Individual Requalification Examination Report

Facility: Operator’s Name:

Docket No: 55- License No: Expiration Date:

Exam Type: RO/SRO Retake: 1st / 2nd / No Date of Last Exam:

Written Examination Results

Date(s) of Exam: NRC Examiner (Print): Facility Evaluator (Print):

Section A (Points)

NRC Grading Facility Grading

of of

Section B (Points) of of

Overall Score (%) % %

Simulator Examination Results

Date(s) of Exam: NRC Examiner(s) (Print): Facility Evaluator(s) (Print):

Crew Evaluation Pass / Fail Pass / Fail

Individual Followup Pass / Fail / N/A Pass / Fail / N/A

Walkthrough Examination Results

Date(s) of Exam: NRC Examiner(s) (Print): Facility Evaluator(s) (Print):

No. of Successful JPMs of 5 of 5

Exam Results (%) % %

NRC Examiner Recommendations

Category Results Signature/Date

Written Pass / Fail

Simulator Pass / Fail

Walkthrough Pass / Fail

NRC Supervisor Review

Date: Pass / Fail Signature:

Privacy Information—For Official Use Only
ES-6.1, Page 28 of 33

Letter 6.1-1 Sample Corporate Notification Letter

NRC Letterhead

(Date)

(Name, Title)

(Name of Facility)

(Street Address)

(City, State ZIP Code)

SUBJECT: REQUALIFICATION PROGRAM EVALUATION

Dear (Name):

In a telephone conversation on (date), (Name, title) and (Name, title) arranged to evaluate the 

requalification program and licensed personnel at the (facility name). The evaluation is 

scheduled for the week of (date). U.S. Nuclear Regulatory Commission (NRC) examiners and 

evaluators from your facility will conduct requalification examinations, and the NRC will evaluate 

your requalification program in accordance with NUREG-1021, “Operator Licensing Examination 

Standards for Power Reactors.” You are encouraged to ensure that your training staff and 

proposed examinees are familiar with these standards.

For the NRC to adequately prepare for this evaluation, the facility licensee will need to furnish 

the NRC with the approved items listed in Enclosure 1, “Reference Material Guidelines.” The 

agency also asks you to submit, at your option, a proposed examination for use during the 

examination week. However, if you do submit a proposed examination, the personnel 

participating in its development will become subject to the security restrictions described in this 

letter.

Please review the guidance in NUREG-1021 concerning the content and scope of simulator 

examination scenarios. The scenario examination bank should cover the entire spectrum of 

emergency operating procedures (EOPs), including alternative decision paths within the EOPs, 

and it should incorporate a range of failures with various degrees of severity for the same type 

of event. Each scenario should contain simultaneous events that require the senior reactor 

operators (SROs) to prioritize their actions and to assign particular tasks to other crew 

members. Each scenario should also require the SROs to decide when to make the transition 

between EOPs and which actions to take within EOPs.

You are requested to designate at least one employee to be a member of a joint NRC-facility 

examination team. The NRC expects that employee to be an active SRO as defined by Title 10 

of the Code of Federal Regulations (10 CFR) 55.53(e) or (f) from the (facility name) operations 

department. You are encouraged to designate a second employee from the training staff to be 

a member of the examination team. This employee should also be a licensed SRO but may be 

a certified instructor. If desired and agreed to by the chief examiner, you may designate one 

additional employee from the training staff who has appropriate qualifications to be a member of 

the examination team. In addition to these individuals, you will need to designate a simulator 

operator for scenario preview and validation during the onsite examination preparation week. In 

some cases, you may also need to designate a simulator operator during the test item review 

period. All of these individuals will be subject to the examination security agreement.
ES-6.1, Page 29 of 33

The NRC restricts any facility licensee representatives under the security agreement from 

knowingly communicating (by any means) the content or scope of the examination to 

unauthorized persons or participating in any facility licensee programs (such as instruction, 

examination, or tutoring) in which an identified requalification examinee will be present. These 

restrictions apply from the day that the facility licensee representative signs the examination 

security agreement indicating that the representative understands that he or she has specialized 

knowledge of the examination. The chief examiner will determine when a facility licensee 

representative has received specialized knowledge concerning the examination and will execute 

an examination security agreement. In most cases, the examination team members will not be 

required to enter into an examination security agreement more than 60 days before the 

examination week. The simulator operator will normally become subject to the security 

restrictions during the examination preparation and validation week; however, this may occur as 

many as 45 days before the examination week.

Sixty days before the examination administration date, please provide the NRC’s regional office 

with a proposed list of operators, including crew composition, for the examination. The list 

should include at least 12 operators composing three or more crews and the current mailing 

address for each proposed operator, if different from that listed on the most recent Form 398, 

“Personal Qualification Statement—Licensee,” submitted to the NRC. Your training staff should 

send this information directly to the NRC chief examiner, ensuring that each operator’s address 

is sent in a manner to ensure privacy.

The facility licensee may request that the NRC chief examiner or another NRC representative 

meet with the licensee managers and the operators to be examined during the examination 

preparation week, normally 2 weeks before the examination. However, if the schedule does not 

allow them to meet during the preparation week, they may meet at any mutually agreeable time. 

The NRC examiner will explain the examination and grading processes and will respond to any 

questions that the operators may have about the NRC’s examination procedures. If such a 

meeting is desired, your training staff should schedule it with the NRC chief examiner.

The facility licensee staff is responsible for providing adequate space and accommodations to 

properly develop and conduct the examinations. Enclosure 2, “Administration of Requalification 

Examinations,” describes our requirements for developing and conducting the examinations. 

Also, a facility operations management representative above a shift supervisor level should 

observe the simulator examination process at the site.

Paperwork Reduction Act Statement

This letter contains information collection requirements that are subject to the Paperwork 

Reduction Act of 1995 (44 U.S.C. 3501 et seq.). The Office of Management and Budget (OMB)

approved these information collections under approval number 3150-0018.

The burden to the public for these [voluntary][mandatory] information collections is estimated to 

average 25 hours per response, including the time for reviewing instructions, searching existing 

data sources, gathering and maintaining the data needed, and completing and reviewing the 

information collection. Send comments regarding this information collection to the FOIA, 

Library, and Information Collections Branch (T6-A10M), U.S. Nuclear Regulatory Commission, 

Washington, DC 20555 0001, or by e mail to Infocollects.Resource@nrc.gov, and to the OMB 

reviewer at: OMB Office of Information and Regulatory Affairs (3150-0018), Attn: Desk Officer 

for the Nuclear Regulatory Commission, 725 17th Street, NW Washington, DC 20503; e 

mail: oira_submission@omb.eop.gov.
ES-6.1, Page 30 of 33

Public Protection Notification

The NRC may not conduct or sponsor, and a person is not required to respond to, a collection 

of information unless the document requesting or requiring the collection displays a currently 

valid OMB control number.

In accordance with 10 CFR 2.390, “Public inspections, exemptions, requests for withholding,” of 

the NRC’s “Rules of Practice,” a copy of this letter and its enclosures will be available 

electronically for public inspection in the Publicly Available Records System component of the 

NRC’s Agencywide Documents Access and Management System (ADAMS). ADAMS is 

accessible from the Electronic Reading Room page of the NRC’s public Web site at 

http://www.nrc.gov/reading-rm/adams.html.

Thank you for your cooperation in this matter. (Name) has been advised of the NRC guidelines 

and policies addressed in this letter. If you have any questions on the evaluation process, 

please contact (Name, regional section chief) at (telephone number).

Sincerely,

(Appropriate Regional Title)

Docket No.: 50-(Number) or

52-(Number)

Enclosures:

1. Reference Material Guidelines

2. Administration of Requalification Examinations 

DISTRIBUTION:

Public

NRC Document Control System

Regional Office Distribution
ES-6.1, Page 31 of 33

Enclosure 1

Reference Material Guidelines

1. Sixty days before the examination date, the facility licensee should provide test items to 

the U.S. Nuclear Regulatory Commission (NRC) to support all aspects of the 

requalification examination.

2. The facility licensee is expected to submit the following reference materials for all 

NRC-conducted requalification examinations:

• an examination sample plan that meets the requirements of Attachment 3 to this 

examination standard

• the facility’s examination banks (written, simulator, and job performance 

measures (JPMs)) and associated reference materials (including, at a minimum, 

technical specifications, abnormal and emergency operating procedures, and 

emergency plan procedures used in requalification training)

• additional reference materials requested by the NRC chief examiner

3. The NRC expects the facility licensee’s examination banks to contain the following 

information:

• a minimum of 700 test items equally divided for use in the two sections of the 

written examination and covering all safety-related elements of the facility’s job 

task analysis

The NRC expects the facility licensee to maintain a dynamic bank by reviewing, 

revising, or generating at least 150 questions a year. New questions should 

cover equipment and system modifications, as well as recent industry and 

licensee events and procedural changes.

• JPMs that meet the criteria in Examination Standard (ES)-6.3, “Requalification 

Walkthrough Examinations,” for evaluating each reactor operator’s and senior 

reactor operator’s safety-related tasks identified in the facility’s job task analysis

The JPM bank should expand at a rate of at least 10 JPMs per year until this 

goal is reached. An estimated 125 to 150 JPMs will be the final result.

• a bank of at least 30 simulator scenarios reflecting all abnormal and emergency 

situations to which an operator is expected to respond or control

The facility licensee should generate at least five scenarios per year until all 

aspects of the emergency operating procedures are covered with sufficient 

variation in the type and scope of initiating events and level of degradation. 

Scenarios that include applicable industry events should be emphasized.
ES-6.1, Page 32 of 33

Enclosure 2

Administration of Requalification Examinations

1. The U.S. Nuclear Regulatory Commission (NRC) must evaluate at least 12 operators to 

perform a program evaluation. Section C.2 of Examination Standard (ES)-6.1, 

“Conducting NRC Requalification Examinations,” and ES-6.4, “Dynamic Simulator 

Requalification Examinations,” describe the guidelines on crew composition in the 

simulator.

2. The simulator and simulator operators need to be available for examination 

development. The chief examiner and the facility representatives will agree on the dates 

and amount of time needed to develop the examinations.

3. The NRC chief examiner will review the reference materials used in the simulator. The 

NRC will not authorize for use during the simulator test any reference material that is not 

normally used for plant operation in the control room.

4. The facility licensee will provide a single room for completing Section B of the written 

examination. The licensee will select the locations of the examination room and 

supporting restroom facilities to prevent the examinees from having contact with any 

other facility or contractor personnel during the examination.

5. The NRC chief examiner will inspect the examination room to see that it meets the 

minimum standard that will ensure examination integrity. The minimum spacing 

standard consists of one examinee per table and a 1-meter (3-foot) space between 

tables. No wall charts, models, or other training materials are allowed in the examination 

room.

6. The NRC expects the facility licensee to provide a copy of each reference document for 

each examinee for Section B of the written examination. The material should include 

documents that are normally available to the operators in the control room (such as the 

technical specifications, operating and abnormal procedures, administrative procedures, 

and emergency plans). The NRC chief examiner will review the reference materials 

before the examination begins.

7. The NRC requalification examination will attempt to distinguish between reactor operator 

and senior reactor operator knowledge and abilities to the extent that the facility training 

materials allow the developers to make these distinctions.

8. Prudent scheduling of examination week activities is important to help alleviate undue 

stress on the operators. The facility training staff and the NRC chief examiner should 

attempt to formulate a schedule that will minimize delays while conducting the 

examination. The following suggestions will help to structure the examination activities 

to achieve this objective:

• Consider allowing operators to stay at home until their scheduled examination 

times.

• Segregate the group of operators who are completing their examination, instead 

of the group of operators who are scheduled to start their examination.
ES-6.1, Page 33 of 33

2 Enclosure 2

• Following simulator scenarios, the facility evaluators and NRC examiners should 

quickly determine whether followup questioning is required so that the crew 

members may be released to talk among themselves about the scenario.

• Ensure that time validation of job performance measures, particularly those 

performed in the simulator, is accurate. Establish a reasonable schedule to 

prevent operators from waiting for simulator availability to complete their job 

performance measures.

ES-6.2, Page 1 of 24

1 6.2 REQUALIFICATION WRITTEN EXAMINATIONS

2 The NRC staff uses this examination standard to conduct written requalification examinations in 

3 accordance with 10 CFR 55.59(a)(2)(iii). NRC examiners are to follow this examination 

4 standard in conjunction with ES-6.1, “Conducting NRC Requalification Examinations,” to 

5 prepare and administer all NRC-conducted written requalification examinations.

6

7 A. Scope

8 The written examination is useful for evaluating the K/As and skills of licensed operators that are 

9 difficult to infer from behavior alone but can be tested readily through written responses to 

10 questions that value interpretation and allow the examinee to use references. Additionally, an 

11 individual’s knowledge of factual information and his or her ability to perform “paper and pencil” 

12 tasks are best evaluated through a written examination.

13 The written examination consists of two sections for which the examinee may refer to 

14 references (i.e., “open-reference examinations”). Section A, “Plant and Control Systems,” is 

15 administered using a static simulator. Section B, “Administrative Controls/Procedural Limits,” 

16 may be administered in a classroom. Each section should be designed to last a minimum of 

17 1 hour, including time for the operator to review his or her work. Combined, the two sections of 

18 the written examination should be designed to last 3 hours. The facility licensee will determine 

19 the exact number of questions and time allowed to complete each section, on the basis of the 

20 requalification sample plan and the license level of the operators taking the examination (RO or 

21 SRO).

22 Although the examination is designed so that examinees may use references, an examinee 

23 should not expect to have time to complete the examination by consulting references to 

24 determine each answer. A good mix of test items will contain some questions that evaluate the 

25 operator’s abilities to determine a correct response without delving into reference material and 

26 others that require the use of reference material to select the correct response. By combining 

27 test items that require references with those that do not, the written examination can test a 

28 broader sample of operator knowledge within a given period.

29 On both sections of the written examination, certain questions will test the K/As of an RO, while 

30 others will test those of an SRO. In developing the examination, the examiner should consult 

31 the facility’s JTA and the applicable K/A catalog to help identify the most suitable topics for an 

32 RO or SRO. Additionally, 10 CFR 55.41, “Written examination: Operators,” and 10 CFR 55.43, 

33 “Written examination: Senior operators,” provide further guidance on item selection for RO and 

34 SRO written examinations, respectively.

35 1. Section A, “Plant and Control Systems” (Static Simulator)

36 This section of the written examination is designed for using the simulator as a reference tool to 

37 visually provide realistic information and to give the operators an environment that is as close as 

38 possible to their normal control room. During this section, the simulator will be “frozen” in the 

39 middle of an evolution, transient, or accident. Test items in this section should allow the use of 

40 references that can be related to plant systems and components, control room indications, 

41 instrumentation and controls, and technical specification (TS) limiting conditions for operation.

42 Section A is designed to evaluate the operators’ knowledge of plant systems, integrated plant 

43 operations, and instrumentation and controls. In addition, it evaluates the operators’ abilities to 

44 recognize TS limiting conditions for operation and to determine the effects of postulated events. 
ES-6.2, Page 2 of 24

1 The NRC encourages facility licensees to include questions that test the operators’ abilities to 

2 use their facility curves and charts.

3

4 While administering Section A, the examination team will use one “frozen” simulator condition or 

5 setup. The condition places the simulator in a “snapshot” of the plant following a major transient 

6 that resulted in an engineered safety feature initiation or a steady-state situation at power. 

7 Some equipment should be frozen in an abnormal or failed condition to provide adequate 

8 material for test items.

9

10 2. Section B, “Administrative Controls/Procedural Limits”

11 Section B of the written examination is designed to evaluate the operator’s abilities to analyze a 

12 given set of conditions and determine the proper procedural and administrative guidance to use. 

13 This section may include theory-related questions that are appropriate to sample the topics 

14 listed in 10 CFR 55.41 and 10 CFR 55.43 as long as the questions are operational in nature or 

15 test unique facility characteristics.

16

17 Section B is designed to test the operator’s knowledge and use of plant procedures and 

18 administrative controls while allowing the use of references. As sources of test items for this 

19 section of the examination, the NRC uses administrative, operating, normal, abnormal, and 

20 emergency procedures; the TS; and the emergency plan. The test items focus on how 

21 direction, guidance, and information found in these procedures are used or interpreted, rather 

22 than focusing on finding the procedure containing the necessary information. Additionally, the 

23 test items for Section B of the SRO examination assess the operator’s understanding of the 

24 reasons and bases for procedural requirements. The use of graphs, charts, tables, and 

25 drawings is appropriate. The simulator may be made available to the examinees to make the 

26 examination more operationally oriented.

27

28 B. Examination Development Responsibilities

29 1. Facility Examination Team Members’ Responsibilities

30 The facility licensee examination team is responsible for the following activities:

31

32 a. The NRC expects the facility to provide a bank of test items that are developed using the 

33 guidance in Section C of this examination standard; ES-4.2, “Developing Written 

34 Examinations”; and Appendix B, “Examples of Written Examination Questions,” as 

35 applicable. The number of test items should meet the submittal guidelines of 

36 Enclosure 1, “Reference Material Guidelines,” of Letter 6.1-1, “Sample Corporate 

37 Notification Letter.” Form 6.1-4, “Evaluation Checklist for Facility Reference Material,”

38 provides information that facility personnel may use to evaluate reference material sets 

39 before submitting them to the NRC. 

40

41 The facility licensee should keep its examination question bank up to date by reviewing, 

42 modifying, or creating at least 150 questions each year to expand the bank and reflect 

43 procedure or system changes, new lesson plans, and recent licensee and industry 

44 events.

45

46 If the facility question bank contains at least 700 items that meet the format guidance of 

47 Form 6.2-4, “Example Formats for Open-Reference Questions,” in this examination 

48 standard, the facility may release the bank to its operators for review.
ES-6.2, Page 3 of 24

1 b. The following items should be provided for each test question:

2 • applicable K/A reference and values (RO/SRO)

3 • reference JTA (if applicable)

4 • estimated time to answer

5 • appropriate learning objectives

6 • applicable references (e.g., lesson plan and emergency operating procedures)

7

8 c. The facility is expected to provide a sample plan that meets the guidelines of Form 6.1-2, 

9 “Examination Sample Plan,” and may submit a proposed examination that conforms to 

10 the facility’s sample plan. The proposed examination should contain a total of 30 to 40 

11 test items, depending on the time validation (maximum of 3 hours) of the individual 

12 questions selected. Sections A and B should each contain 15 to 20 questions, and each 

13 section must be designed to last a minimum of 1 hour, with the total examination 

14 designed to last 3 hours.

15

16 The facility licensee will determine the number of questions in each section based on the 

17 requalification sample plan and the license level of the operators taking the examination 

18 (RO or SRO) and subject to the quantitative constraints of the previous paragraph. Plant 

19 systems questions that do not directly relate to the static scenario can be included in 

20 Section A to meet the facility’s sample plan and the requirements of 10 CFR Part 55, 

21 “Operators’ licenses.” In addition, up to 20 percent of the test items may be from topics 

22 outside the sample plan as long as the information stated in Section B.1.b. of this 

23 examination standard is provided.

24

25 If the facility licensee submits a proposed examination, those individuals involved in its 

26 development become subject to the security restrictions of ES-6.1 once examination 

27 development starts. These restrictions remain in effect until the NRC examination is 

28 given. If, after developing a proposed examination, the facility decides not to submit it 

29 for use in the NRC-conducted examination, the developers are released from the 

30 security restrictions of ES-6.1.

31

32 d. After the NRC has reviewed the facility’s examination bank and commented on the test 

33 items selected for the examination, the facility team members are expected to prepare 

34 the examination for final NRC review and approval. The examination may be finalized 

35 before or during the preparation week.

36

37 e. The facility team representative will evaluate each test item that the NRC revised to 

38 assess the following criteria:

39 • appropriateness

40 • time required to answer, given the operational context

41 • technical accuracy

42 • clarity

43 • K/A and objective references

44

45 Following this evaluation, the facility examination team representatives and the NRC chief 

46 examiner need to agree on the final form of the examination. They also need to complete a 

47 time validation of the proposed examination. A variety of methods have proven effective in 

48 accomplishing this task.
ES-6.2, Page 4 of 24

1 Any individual involved in time validating the examination is required to sign Form 6.1-3, 

2 “Examination Security Agreement.” The examination team may add or delete items from 

3 the examination based on the results of this time validation and the experiences. If any 

4 test items are added, it is not necessary to time validate the entire examination again as 

5 long as a subject matter expert has reviewed the added questions, indicating the 

6 approximate time that an operator should take to answer each question.

7

8 f. The NRC expects the facility licensee to provide sufficient copies of each reference so 

9 that each examinee can use the references during the examination and, immediately 

10 upon completion of the examination, to compile the examinations and reproduce 

11 sufficient copies for its own use and that of the NRC.

12

13 g. To help relieve the burden of providing a complete set of references to each operator, 

14 the examination may be assembled so that a different sequence of questions appears 

15 on each operator’s examination. Alternatively, handouts of relevant information 

16 (e.g., plant curves, blank forms) may be provided with the test.

17

18 2. NRC Examination Team Members’ Responsibilities

19 The NRC examination team is responsible for the following activities:

20

21 a. The NRC will begin its evaluation of the sample plan, the bank of test items, and the 

22 proposed examination as soon as possible after receiving the facility’s materials. The 

23 NRC will promptly evaluate the materials to allow sufficient time for the agency or the 

24 facility to develop the test items and for the facility to revise them to meet NRC 

25 standards, if required. The NRC examiners should review the proposed test items using 

26 Form 6.2-1, “NRC Checklist for Open-Reference Test Items,” to ensure appropriateness, 

27 clarity, and importance to safety as described in Section C of this examination standard.

28

29 If the facility licensee intends to administer both sections of the examination during a 

30 single 3-hour period as noted in Section B.1.c, the examination team members must 

31 review the examination as a whole to ensure that the items in either section do not 

32 compromise those in the other.

33

34 b. A minimum of 80 percent of the test items will be chosen in accordance with the sample 

35 plan. The examination team may substitute the remaining 20 percent by using facility 

36 examination bank questions or new questions that the team develops. Should it be 

37 necessary to develop additional items to satisfy the sample plan, the NRC will ask the 

38 facility to do so.

39

40 c. If, after reviewing at least 75 percent of the bank, the examination team finds that test 

41 items are insufficient to develop an NRC examination that meets the sample plan, the 

42 NRC staff will declare the bank of test items inadequate. In that event, the regional 

43 managers may either cancel the scheduled examination or administer an examination 

44 using NRC-developed test items without consideration of the 20-percent substitution

45 constraints.

46

47 d. If the sample plan does not include topics from outside the requalification cycle, the 

48 examination team should consider incorporating 10 to 20 percent test items specific to 

49 the nonrequalification cycle.

50
ES-6.2, Page 5 of 24

1 e. If a test item does not have a clear tie to the JTA, the examiner will discuss the 

2 applicability of the test item with the facility representatives.

3 C. Guidelines for Developing and Reviewing Open-Reference Examinations

4 Those who are involved in developing or reviewing test items for the written portion of the NRC’s 

5 requalification examination should use the following guidelines. As described in ES-6.1, the written 

6 examination consists of two sections. Section A uses a static simulator to provide the context for 

7 questions on plant and control systems, while Section B focuses on plant procedures and 

8 administrative controls. Examinees may use references, including simulator displays, for both 

9 sections. Open-reference written examinations are used for two reasons:

10

11 1. examination validity

12

13 By permitting the use of references that are available to the control room operators, the 

14 conditions and requirements of the written examination more closely approximate those 

15 of the actual job. The information provided to the operators in the test items should 

16 closely parallel the information typically available to them, while the responses elicited by 

17 the questions should be related to the decisions, solutions, and actions required for 

18 effective job performance. In other words, consulting references more closely correlates 

19 job demands and test demands—a cornerstone of examination validity.

20

21 2. level of knowledge

22

23 Use of references enhances examination validity by elevating the level of knowledge of 

24 the test items. As described later in these guidelines, operator access to references 

25 precludes the use of questions that test for the mere recall of facts and specifics. 

26 Instead, open-reference test items require test takers to demonstrate that they can find, 

27 apply, analyze, evaluate, or otherwise use knowledge to handle the problems and issues 

28 they may encounter on the job.

29

30 Most principles for effective test item construction apply equally to all types of written questions, 

31 regardless of format. Therefore, those who develop and review open-reference test items should 

32 consult Appendix A, “Overview of Generic Examination Topics,” to this NUREG, in addition to the 

33 guidelines in this section.

34

35 1. Selection of Test Topics

36 Use the following criteria to select test item topics for the NRC’s requalification examination:

37

38 a. requalification training program curriculum

39

40 Base the test topics on the curriculum of the most recent operator requalification 

41 program training cycle. However, the NRC may substitute up to 20 percent of the 

42 examination topics selected by the facility with subjects not emphasized during the 

43 requalification cycle. These test items should emphasize knowledge that is of high 

44 importance to safety.

45
ES-6.2, Page 6 of 24

1 b. performance basis

2

3 Like the requalification program itself, draw the test topics from a JTA for an RO and an

4 SRO. The facility licensee should validate each test item by demonstrating a link 

5 between each item and the following JTA products:

6 • important operator tasks, as identified by the JTA

7

8 • important K/As (rated 3.0 or higher), as identified in NUREG-1122, “Knowledge 

9 and Abilities Catalog for Nuclear Power Plant Operators: Pressurized Water 

10 Reactors”; NUREG-2103, “Knowledge and Abilities Catalog for Nuclear Power 

11 Plant Operators: Westinghouse AP1000 Pressurized-Water Reactors”; 

12 NUREG-1123, “Knowledge and Abilities Catalog for Nuclear Power Plant 

13 Operators: Boiling Water Reactors”; and NUREG-2104, “Knowledge and 

14 Abilities Catalog for Nuclear Power Plant Operators: Advanced Boiling-Water 

15 Reactors”; or a facility-specific K/A catalog

16

17 • facility learning objectives identified as important to safety

18

19 c. adequacy of test coverage

20

21 Check the facility’s proposed sample plan (or test outline) to ensure that it provides 

22 balanced, comprehensive coverage of the requalification training cycle topics. Revise 

23 the distribution of proposed facility test items on the examination if the topics 

24 underrepresent or overrepresent the material covered in the requalification program. In 

25 the sample plan, address recent safety-related issues and events (e.g., those in relevant 

26 LERs). Form 6.1-2 provides further information on sample plan development.

27

28 2. General Guidelines for Sections A and B

29 Use the guidelines in this section to construct and review test items for both parts of the written 

30 examination. The NRC intends these guidelines to supplement, rather than replace, the good 

31 practices stated in Appendices A and B to this NUREG.

32

33 a. Operational Orientation

34

35 As previously discussed, examination validity is enhanced to the extent that the demands of the 

36 test match the demands of the job. Therefore, in addition to being derived from important K/As 

37 and testing objectives, the context and stipulations of test items should mirror the situations 

38 encountered in the work setting. The following example illustrates effective and ineffective ways 

39 to design test items from K/As and learning objectives:

40

41 K/A: Knowledge of the design attributes of the turbine-driven auxiliary 

42 feedwater pump (TDAFWP) differential pressure controller.

43 Task: Operate the TDAFWP controls during all modes of plant 

44 operation.

45 Learning Objective: The student will be able to operate the TDAFWP differential 

46 pressure controller without error during a loss-of-feedwater event.
ES-6.2, Page 7 of 24

1 Enabling Objective: After completing this lesson, the student will be able to explain the 

2 operation of the TDAFWP differential pressure controller.

3

4 Poor Test Item: State the parameters used by the TDAFWP differential pressure 

5 controller.

6 Better Test Item: Before isolating the “C” steam generator (in accordance with EPP11), 

7 an operator noted that the transducer-fed auxiliary feed flow 

8 indicators for the “C” steam generator were reading greater than the 

9 flow indicators to the “A” and “B” steam generators. What is the 

10 reason for this flow deviation?

11 Notice that the “better” test item requires the operator to demonstrate mastery of the knowledge 

12 by applying it to an actual job situation. In developing items, ask “Why is the K/A important to 

13 satisfactory job performance?” and “In what situation will the operator need this K/A?” The 

14 answers to these questions can provide a basis and context for a test item.

15

16 b. Level of Knowledge

17

18 The operational orientation required of test items on the open-reference examination, as well as 

19 the operators’ access to controlled documents, precludes the use of questions that test for mere 

20 recall or memorization. Rather than requiring operators to simply recognize or recall facts and 

21 specifics, open-reference test items have the operators demonstrate understanding by using the 

22 knowledge to address real-life situations and problems. A test item at the higher level of 

23 knowledge requires operators to determine or identify the appropriate fact, rule, or principle and 

24 then correctly apply it to a novel situation. Appendix B describes each level of knowledge. 

25 Together with Form 6.2-3, “Examples of Different Types of Questions,” Appendix B also 

26 provides sample questions that illustrate the various levels of knowledge.

27

28 c. Realistic Context

29

30 For additional assurance of examination validity, make the situation or problem posed in the 

31 open-reference test item as similar as possible to the actual situations that operators encounter 

32 on the job. Situations described in the questions should be realistic and should also be free of 

33 common “context” problems, including “backward logic” and “window dressing.”

34

35 Backward-logic questions give operators information they normally must produce while asking 

36 them for information they normally receive, as illustrated by the following example:

37

38 K/A: Ability to calculate shutdown margins

39

40 Backward-Logic Item: If the shutdown margin is 5.5 percent, how long has the unit been 

41 shut down?

42

43 Better Item: The unit has been shut down for x hours. Which of the following is 

44 the shutdown margin?

45

46 Questions with window dressing have additional, unnecessary information, typically in an 

47 attempt to make a memory-level item more operationally oriented, as in the following example:
ES-6.2, Page 8 of 24

1

2 Window Dressing Item: The plant has tripped from the effects of a tornado crossing the 

3 site boundary. You, as shift supervisor, direct the phone talker to 

4 complete the 15-minute notification. He informs you that the 

5 normal notification network is inoperable. Which of the following

6 do you direct him to use for completing the 15-minute notification?

7 Better Item: If the normal notification network is inoperable, which of the 

8 following methods do you use to complete the 15-minute 

9 notification after the plant has tripped?

10

11 Another common problem when constructing a question with a realistic context is that 

12 “real-world” situations often have more than one correct solution or response. Carefully check 

13 the question and references to ensure that each test item has only one correct answer.

14

15 d. Question Novelty

16

17 One of the most effective ways to ensure that an operator has a high level of knowledge is to 

18 present novel situations and require the operator to realize both what information is relevant and 

19 how to apply it. If a test question does not contain unique or varied circumstances different from 

20 those presented in training, the item will merely elicit simple recall.

21

22 When candidates are able to memorize test items and answers (in their static state) to respond 

23 to test items, the test cannot really determine whether they can truly solve the problems or 

24 whether they have merely memorized the answers. Once a test item and its answer have been 

25 seen and rehearsed, the item ceases to be a viable discriminator of safe operator performance. 

26 It is no longer challenging or testing problem-solving ability; rather, it is simply testing recall. 

27 Therefore, test items must be dynamic, replacing or substituting items of like kind and difficulty 

28 to preserve integrity in the test discrimination process.

29

30 Because an infinite number of combinations of plant or equipment parameters and malfunctions 

31 may exist at any given time, a true test will compensate for this variation and will become 

32 dynamic so that the test can adapt to the infinite number of combinations and still test the same 

33 kinds of responses but to different situations.

34

35 Review the training material to ensure that questions do not include overly familiar conditions. 

36 Keep in mind, however, that all conditions and situations should be reasonable, realistic, and 

37 safety related.

38

39 e. Relationship of Open-Reference Examinations and Direct Lookup Questions

40

41 Direct lookup questions are associated with open-reference examinations. The key phrase 

42 here, “direct lookup,” conveys the meaning that little mental activity is involved other than simply 

43 copying an answer that is readily available in a reference (i.e., simple recall of where to find the 

44 information). Merely omitting from the item stem any mention of where to find the answer does 

45 not make it an acceptable open-reference question.

46

47 Do not use direct lookup questions for two reasons. First, these items only test memory, in that 

48 the information is readily available; this is an inefficient and less valid means of testing 

49 candidate knowledge. Second, other than demonstrating that a candidate knows where to find 

50 information, this type of question does not test the understanding or analysis of the information 
ES-6.2, Page 9 of 24

1 that can be applied on the job. Consequently, this type of question will not discriminate the safe 

2 operator from the unsafe operator.

3

4 The other option is an “open-reference” question. Use an open-reference examination to test 

5 candidate knowledge for the following purposes: 

6 • Does the candidate know which reference to use and where to find it?

7 • Does the candidate know how to apply the information in the reference to the problem?

8 For an open-reference question, the kind and amount of information required to solve the 

9 problem would exceed that which could normally be committed to memory. In other words, the 

10 NRC does not expect candidates to remember the information needed to solve the problem.

11

12 In regular closed-reference questions, the candidate is expected to know and understand how 

13 systems operate so that they can answer a question with the information provided in the stem of 

14 the question. For a closed-reference question, the candidate would not need to consult a 

15 reference. In other words, the NRC expects the candidate to solve the problem by knowing and 

16 understanding how the systems work, given various conditions set in the problem.

17

18 Whether an examination is open- or closed-reference, the test should, to the extent possible, 

19 assess problem-solving or decisionmaking because, at this more complex level of thought, the 

20 test can more closely approximate the job and achieve a valid assessment.

21

22 Memory types: Understanding how memory operates relates to understanding why an 

23 open-reference question is preferable. Obviously, all that one knows or does involves memory. 

24 Operationally, however, memory falls into two categories: simple memory and complex 

25 memory.

26

27 Simple memory can be viewed as recall or recognition of simple bits and chunks of information. 

28 Simple memory may still be involved when the volume of information increases (i.e., the amount 

29 of information is large, but the process is basically simple memorization of more bits of 

30 information). Visualize the type of memory required to memorize 5 letters of the alphabet 

31 versus 26 letters or the recitation of a short poem (or procedure) versus a long one, and so 

32 forth. This memorization process does not involve analysis, integration of facts, or 

33 problem-solving.

34

35 Rather, the process requires repetition, practice, and rehearsal. The difference lies in the 

36 amount of information to be recalled, not the level of mental processing.

37

38 By contrast, complex memory, as the term suggests, involves a higher level of cognitive 

39 processing. The bits and chunks of information must now be combined or integrated to create 

40 something new, solve a problem, predict a response, or make a decision. Therefore, both the 

41 amount of information and what is to be done with it make the cognitive mental processes 

42 complex. Naturally, some questions will involve greater complexity than others, but the mental 

43 processes will be the same—integrating bits of information, combining and sorting them, and 

44 distinguishing the relevant from the irrelevant to arrive at an answer to the question. This is the 

45 essence of an analysis or synthesis process.

46

47 As previously stated, the NRC should evaluate candidates at this complex level, because this 

48 level of thought processing most closely approximates that needed on the job. The complex, 
ES-6.2, Page 10 of 24

1 problem-solving level subsumes knowledge of the bits and chunks of information frequently 

2 tested at the simple memory level. Therefore, by testing at the complex level, the examination 

3 also implicitly tests at the simple memory level. As a prerequisite to solving the problem, the 

4 candidate recalls and integrates these bits and chunks of information. Therefore, testing at the 

5 analysis level is more efficient than testing at the simple memory level.

6

7 A Final Note: Undue emphasis is placed on the term “immediately” in the definition of a direct 

8 lookup. Speed of knowing where to locate a reference is irrelevant to direct lookup. The NRC 

9 expects candidates, who have been trained, to quickly locate the appropriate reference. The 

10 speed issue is relevant to whether the stem of the question contains unnecessary cues to the 

11 candidate about where to find the reference. If the open-reference question is intended to 

12 assess whether the candidate knows where to find the information, the stem should not contain 

13 a cue about the location of the information. Part of the value of an open-reference exam is to 

14 test the candidate’s evaluative knowledge of where to look. If the stem provides unnecessary 

15 cues to the reference, a candidate can immediately go to the reference and some value of the 

16 open-reference test is lost.

17

18 Speed in answering the question properly is a function of the level of difficulty and the thought 

19 processes or steps required to answer the question. Obviously, if the question is a direct 

20 lookup, by definition, it assesses only simple memory and will be quickly and easily answered. 

21 This type of question should never be asked.

22

23 References should be considered “tools” that operators use to solve problems. The 

24 open-reference examination should test the correct use of these “tools,” not just the recall of 

25 facts and specifics. As previously stated, “direct lookup” questions should generally be avoided 

26 and should not be included in the examination; rather, questions should be structured to 

27 determine whether operators can identify, locate, or select correct reference information to 

28 produce organized responses and satisfactory solutions to job-related problems and issues. 

29 The following is an example of a lookup question, which should generally be avoided:

30

31 Based on the “Alarm Response Procedure” 1ZZ-040-3, what is the setpoint of the 

32 high-high containment pressure alarm (PK25) on VB3?

33 a. 10 psig

34 b. 15 psig

35 c. 20 psig

36 d. 25 psig

37

38 This question should be rejected because a candidate can easily find the setpoint in the alarm 

39 response procedure. Some may argue that knowing how to look up this data in the procedure 

40 makes the item valid; however, no higher order cognitive skills requiring analysis or synthesis of 

41 information were required to determine the correct response. Avoid similar questions on 

42 precautions or prerequisites that are listed in procedures. The following is a better question 

43 using reference material:

44

45 Using the current plant conditions (assume emergency core cooling system (ECCS) and 

46 containment spray flow rates REMAIN CONSTANT), how much time is available before 

47 switchover to containment recirculation?
ES-6.2, Page 11 of 24

1 a. 3.6 hours

2 b. 4.2 hours

3 c. 4.8 hours

4 d. 5.2 hours

5

6 This is a “lookup” question in a sense, but it certainly requires gathering data from the control 

7 boards (e.g., ECCS flow, CS flow, and refueling water storage tank level) and then identifying 

8 the correct emergency procedure and locating and selecting the correct graph to determine how 

9 much time is left before the tank reaches a specific level. It requires use of both the simulator 

10 and the plant procedures as references.

11

12 The following is another appropriate question using facility references:

13

14 Following a loss-of-coolant accident, automatic actions have occurred as follows:

15

16 • The reactor has tripped and is shut down.

17 • Auxiliary feedwater has actuated, and steam generator pressure is being 

18 controlled at 1,005 psig, using steam dumps to the condenser.

19 • Containment pressure has risen to 15 psig, and no additional automatic actions 

20 have occurred.

21

22 Which of the following functional recovery procedures should be implemented 

23 IMMEDIATELY?

24 a. FR-C1

25 b. FR-Z1

26 c. FR-P1

27 d. FR-I1

28

29 This question requires identifying which systems should have actuated based on the engineered 

30 safety feature actuation signal setpoints and which critical safety functions are compromised. 

31 The operator should refer to the functional recovery procedures to verify which critical safety 

32 functions have been compromised. Knowing where to look and what to look for are factors 

33 required to answer this question in a reasonable time.

34

35 The item could also be used in the simulator section by requiring the operator to look at the 

36 control board in the “frozen” simulator to determine the plant conditions and deduce what critical 

37 safety functions were not met. Naturally, the more integration and evaluation required, the more 

38 time must be given to answer the question.

39

40 Another question that makes effective use of reference material is as follows:

41

42 While operating at 100-percent power, volume control tank and pressurizer alarms and 

43 indications show decreasing pressurizer level with two charging pumps operating. Also, 

44 the blowdown and main steam radiation monitors have alarmed. While following the 

45 appropriate abnormal and emergency procedures, you, as the shift supervisor, must 

46 evaluate the existing conditions. Which emergency classification would you declare on 

47 the basis of this information?
ES-6.2, Page 12 of 24

1 a. Notification of Unusual Event

2 b. Alert

3 c. Site Area Emergency

4 d. General Emergency

5

6 This question requires the operator to consult references to classify an event. It also requires 

7 analyzing the situation, finding the correct part of the emergency plan implementing procedures, 

8 and selecting the appropriate classification.

9

10 f. Difficulty Level versus Discriminatory Value

11

12 Test developers sometimes believe, erroneously, that open-reference questions should be more 

13 difficult to compensate for the operators’ access to reference material. Frequently, this 

14 increased difficulty is in the form of requiring knowledge of more obscure or otherwise 

15 unnecessary information. Both open- and closed-reference questions should have the same 

16 standard of difficulty; that is, difficulty should be based on the job demands and responsibilities 

17 of operators. A question should be constructed so that it effectively discriminates a competent 

18 operator from one who is not. A high K/A value should not be confused with the difficulty or 

19 discriminating ability of a question.

20

21 g. Time Limits

22

23 Operators take considerably longer to answer open-reference questions than closed-reference 

24 questions. (Weaker operators especially have been found to spend an inordinate amount of 

25 time consulting references rather than writing responses.) Provide the operators ample time to 

26 complete the examination, although not so much time that less-than-competent operators have 

27 the opportunity to locate answers without prior familiarization with the topic. Use the following 

28 four guidelines to determine the appropriate length of the examination:

29

30 1. A competent operator should complete the combination of Sections A and B in 3 hours. 

31 Give the operators an appropriate amount of time to review Sections A and B based on 

32 the number of questions assigned to each section. For example, if Section A has 

33 15 questions and is validated for 45 minutes, allow operators 15 minutes for review. 

34 Likewise, if Section B has 20 questions and is validated for 90 minutes, allow 30 minutes 

35 for review. The 3-hour time limit must include the time allocated to review Sections A 

36 and B.

37

38 2. Develop questions so that Sections A and B each have approximately 15 to 20 points for 

39 a total test value of 30 to 40 points. Use the examination sample plan to determine the 

40 exact number of questions to be asked in each section. As noted in Appendix B, 

41 multiple-choice questions are preferred, but other formats are acceptable. No question 

42 will be worth more than 2 points.

43

44 3. In an open-reference examination, every answer need not require the operator to use a 

45 reference. When developing the questions, make a reasonable estimate of the time 

46 required to answer each question and identify any references needed to respond.

47

48 Whether and to what extent references are needed affect what constitutes a reasonable 

49 amount of time to respond. For example, if the static scenario is set up for an abnormal 

50 plant transient that requires relatively rapid operator analysis or response, the time 

51 allowed to respond to the question should be similar to that required to react to the 
ES-6.2, Page 13 of 24

1 transient. The NRC does not expect an operator to answer a question as quickly as he 

2 or she would react in the plant but does expect that the operator would consult few 

3 references.

4

5 Conversely, questions involving scenarios for which an operator would have time to 

6 consult many references would allow similar time to respond to the question.

7

8 4. The NRC expects each proposed examination to be time validated. In the best method,

9 a representative cross section of plant operators would take the examination in near-test 

10 conditions. Then, by taking the average of the time it took each individual to answer 

11 each question, a reasonable time may be established for the test. However, if a large 

12 deviation occurs among test takers on particular questions, ask them why they took 

13 either an excessive or relatively short amount of time to answer the question (compared 

14 to that anticipated). Responses may lead to eliminating certain operators’ times from the 

15 averaging process and, thereby, eliminating anomalies associated with individuals 

16 (rather than eliminating the test items themselves). However, logistics dictate that 

17 sometimes only one or a few individuals can participate in validating the time to 

18 complete the test. In any case, the results need to be carefully evaluated for any 

19 unanticipated deviations from the amount of time anticipated to complete each item.

20

21 The NRC expects facility managers responsible for validating the examination to validate 

22 the time for each question similarly. When performing time validation of the 

23 examination, make these expectations clear to the facility representatives validating the 

24 examination so that a reasonable estimate can be obtained.

25

26 h. Correct Mode of Measurement

27

28 No matter how high their importance ratings or operational relevance, certain operator K/As and 

29 skills are not amenable to written testing, as in the following example:

30

31 Arrange the major steps in the proper sequence to start, parallel, and load diesel 

32 generator DG-2:

33 _____ Use governor control to increase kilowatt power to DG-2.

34 _____ Raise DG speed to 900 revolutions per minute.

35 _____ Match voltage with bus 1A2 voltage.

36 _____ Close breaker 1AD2.

37

38 Despite its operational orientation, the underlying skill addressed in this test item would be 

39 better assessed by having the operator simulate or perform the steps during either the simulator 

40 or walkthrough portions of the operating examination.

41

42 3. Specific Guidelines for Section A

43 The guidelines in the section are specific to Section A of the written examination as performed 

44 on a static simulator. 

45

46 a. Question Development

47

48 To ensure that the operator’s knowledge of systems and integrated plant response is 

49 adequately evaluated, Section A of the written examination should incorporate the behavior of 
ES-6.2, Page 14 of 24

1 systems and controls in normal, abnormal, and emergency plant conditions. To the extent 

2 possible, questions should require the operators to refer to control room indications in 

3 formulating their responses, as in the following example:

4

5 Which one of the following describes the location of the steam break?

6 a. inside containment, upstream of the steamline flow transmitters

7 b. inside containment, downstream of the steamline flow transmitters

8 c. outside containment, between “C” main steam isolation valve (MSIV) and “C” 

9 main steamline check valve

10 d. outside containment, between “C” MSIV and “C” main steamline containment 

11 penetration

12

13 The scenario used should put the plant at some point in a major plant transient

14 (e.g., loss-of-coolant accident, steam generator tube rupture, loss of all alternating current 

15 power) with several passive or active failures incorporated. However, the number of 

16 malfunctions or failures included in the scenario should be limited. No more than four minor 

17 failures should be used (e.g., failure of a safety-related pump to start, failed pressurizer 

18 pressure indication, nuclear instrumentation failure). Four failures will provide sufficient effects 

19 to test a wide range of objectives. Such a scenario would provide sufficient visual cues to 

20 develop a good percentage (at least 50 percent) of questions directly related to the existing 

21 plant conditions.

22

23 Questions may be used that do not relate to the transient but use the simulator as a frame of 

24 reference only provided that the operators are aware of this lack of relationship to the transient.

25

26 Exam authors should carefully ensure that multiple questions stemming from one event do not 

27 give each other away. The operator should be able to understand and correctly answer each 

28 question, based only on the information given in the question rather than on the answer to a 

29 previous question.

30

31 Use of plant diagnostic questions for which the examinee attempts to identify the transient that 

32 has occurred is generally not suitable given the purpose of this section of the examination. 

33 Having the operator attempt to identify what took place may limit the number of questions the 

34 test may ask about the transient. The author should indicate which symptoms or events have 

35 occurred, which procedure has been implemented, and which point in the procedure was 

36 reached at the time the simulator was “frozen.”

37

38 The operator’s response should either determine the root cause of the actual system or 

39 component failure or (by using “what if” questions) propose a future event and ask for the 

40 expected response.

41

42 b. Simulator Setup

43

44 Before the test, the simulator recorders should be advanced to provide clean readings and the 

45 recorders checked for proper operation. All indications (e.g., bulbs, meters, manual loader 

46 indications) should be checked to ensure they are in proper working order.

47
ES-6.2, Page 15 of 24

1 When the simulator has been frozen, the chart recorder drive power should be secured, if 

2 necessary.

3

4 Before administering the test, the test team should verify that the simulator indications are as 

5 expected in order for operators to arrive at the correct answer.

6

7 Any “first-out” annunciators that would normally blink to announce first-out conditions and 

8 should be frozen and their readings provided to the operators.

9

10 If plant procedures are used to stabilize a transient, the step at which the simulator is frozen

11 should be noted and recorded on the simulator operations summary sheet. As necessary, the 

12 examinees should receive the progress of the procedure step in effect.

13

14 4. Ideas for Open-Reference Formats

15 Form 6.2-4, “Example Formats for Open-Reference Questions,” lists of sample formats to assist 

16 individuals who are developing performance-based, open-reference questions.

17

18 Form 6.2-5, “Developing Open-Reference Test Items,” provides additional guidance on the 

19 process of developing open-reference questions.

20

21 D. Examination Administration and Evaluations

22 1. Written Examination Conduct

23 a. An NRC examiner or knowledgeable facility representative who has signed the security 

24 agreement will proctor each section of the examination. At a minimum, an NRC 

25 examiner will observe the examination briefing as the operators begin the examination to 

26 ensure that all administrative aspects of the examination are followed. If an NRC 

27 examiner does not continuously proctor the examination, an examiner will periodically 

28 visit the examination room to ensure that the proctor appropriately addresses questions 

29 on the content or administration of the examination that may have arisen.

30

31 b. Section A is administered on the facility’s simulator or an approved simulation facility.

32

33 c. Section B may be administered in the simulator or in a classroom setting as the facility 

34 staff and the NRC chief examiner deem appropriate. If both sections of the examination 

35 are administered in the simulator during a single 3-hour period, operators may return to 

36 a section of the examination that they have already completed or retain both sections of 

37 the examination until the time has expired.

38

39 d. For Section A of the examination, the facility licensee is responsible for giving the group 

40 of examinees at least one copy of all controlled reference materials available in the 

41 control room. Examination reference materials will not include material that is intended 

42 for training use only. The licensee controls all reference materials in accordance with its 

43 procedure revision control program under Appendix B, “Quality Assurance Criteria for 

44 Nuclear Power Plants and Fuel Reprocessing Plants,” to 10 CFR Part 50, “Domestic 

45 licensing of production and utilization facilities.” The materials should be authorized for 

46 use in operating the power plant, agreed on by the facility and the NRC chief examiner, 

47 and in effect at the time of the examination validation (i.e., the preparation week).

48
ES-6.2, Page 16 of 24

1 e. During the administration of Section B, each examinee will have available for use the 

2 following materials (complete, current issue):

3

4 • TS

5 • plant procedures (emergency operating procedures, abnormal operating 

6 procedures, and normal operating procedures)

7 • emergency plan (as available in the control room)

8 • administrative procedures applicable to operations

9 • other controlled plant reference materials that are normally available in the 

10 control room (e.g., curves and data book, forms, plant drawings, flow charts) and 

11 authorized for use in operating the plant

12

13 Note that “noncontrolled” reference materials, such as the emergency operating procedure 

14 owner’s group basis documents, will not be provided unless they are authorized to be used by 

15 the control room operators during plant operations.

16

17 2. Examination Administration Procedures

18 The written examinations will begin only after the NRC chief examiner has verified the adequacy 

19 of the examination facilities and arranged for continuous proctoring of the examination as 

20 discussed in Section D.1.a of this examination standard. An NRC examiner may act as proctor 

21 during this examination. However, the chief examiner is responsible for ensuring that the

22 actions described in paragraphs D.2.b through D.2.i (below) are complete.

23

24 Each section of the written examination will be administered as follows:

25

26 a. An NRC examiner will verify each examinee’s identity and examination level against the 

27 list provided by the facility licensee. If possible, the ROs and SROs should sit at 

28 alternating tables. Any errors or no shows will be resolved with the facility staff, and the 

29 list will be updated as required.

30

31 b. The proctor will remind the operators that they may use calculators to complete the 

32 examination and that no reference material other than that provided is allowed in the 

33 examination area. The proctor will define the examination area for the examinees.

34

35 c. The proctor will pass out the examinations and answer sheets and instruct the 

36 examinees not to turn over the examination until told to do so. The examinees will be 

37 informed that pads of scrap paper are available from the proctor upon request.

38

39 d. The proctor will brief the examinees about the rules and guidelines in effect during the 

40 written examination using ES-1.2, “Guidelines for Taking NRC Examinations.” The 

41 proctor should inform the examinees that they may refer to the instructions directly 

42 beneath their examination cover sheet. The proctor will read the indicated policies 

43 verbatim.

44
ES-6.2, Page 17 of 24

1 e. The proctor will ask the examinees to verify the completeness of their copies by 

2 checking each page of the examination. The proctor should also have the examinees 

3 check to ensure that their examinations include an examination sheet, if required.

4

5 f. After answering any questions that the examinees may have about the examination 

6 policies, the proctor will start the examination and record the time.

7

8 g. The proctor will periodically advise the examinees of the time that remains to complete 

9 the examination. Normally, a chalk board or white board is available for this purpose.

10

11 h. As the examinees complete the examination, the proctor will ensure that they sign the 

12 examination cover sheet and staple it on top of their answer sheets. The proctor will 

13 collect the examination packages, including the questions and answer sheets, any 

14 references used with the examination, and all scrap and unused paper. The NRC 

15 examiner will keep the cover and answer sheets, dispose of the scrap paper, and give 

16 the packages of questions to the facility licensee for subsequent use.

17

18 i. The proctor will remind the examinees to leave the examination area, as defined by the 

19 examination team.

20

21 3. Written Examination Evaluations

22 Using the examination and answer key, the facility and NRC will independently grade each 

23 section of the written examination and will complete the grading of all written examinations 

24 within 10 working days of the examination administration date. NRC examiners will record the 

25 grades on the written examination cover sheet (Form 6.2-2, “Written Requalification 

26 Examination Cover Sheet”).

27

28 An individual’s grade will be obtained by summing the points credited to the examinee on both 

29 sections of the examination and dividing by the total points available (i.e., compensatory grading 

30 methodology).

31

32 To pass the written portion of the examination, operators must achieve an overall score of 

33 80 percent on the written examination.

34

35 4. Test Item Evaluation

36 If a number of test items require significant modification during the grading of the examination 

37 (e.g., more than 10 percent of the items are deleted or the answer is changed from the original 

38 key), the NRC will determine the root cause and reflect it in the examination report. As 

39 discussed in ES-6.1, if significant deficiencies exist in the facility’s quality control of its 

40 examination bank, the NRC will consider them as part of the program evaluation.

41

42 If technical flaws that have some safety significance are found in procedures while analyzing the 

43 answers to the written examination, the facility may institute an immediate procedural change 

44 and inform all operators of the change.

45

46 E. Forms

47 Form 6.2-1 NRC Checklist for Open-Reference Test Items

48 Form 6.2-2 Written Requalification Examination Cover Sheet
ES-6.2, Page 18 of 24

1 Form 6.2-3 Examples of Different Types of Questions

2 Form 6.2-4 Example Formats for Open-Reference Questions

3 Form 6.2-5 Developing Open-Reference Test Items

4
ES-6.2, Page 19 of 24

1 Form 6.2-1 NRC Checklist for Open-Reference Test Items

2 Test Item Level

3

4 ____ 1. Does each test item have a documented link to important operator tasks, 

5 knowledge and abilities (K/As), and/or facility learning objectives?

6

7 ____ 2. Is each question operationally oriented (i.e., is there a correlation between job 

8 demands and test demands)?

9

10 ____ 3. Is the question at least at the comprehension level of knowledge?

11

12 ____ 4. Is the context of the questions realistic and free of window dressing and 

13 backward logic?

14

15 ____ 5. Does the item require an appropriate use of references (i.e., use of analysis skills 

16 or synthesis of information either to discern what procedures are applicable or to 

17 consult the procedures to obtain the answer)?

18

19 ____ 6. Is the question a “direct lookup” question, or does one question on the 

20 examination compromise another? A “direct lookup” question is defined as a 

21 question that only requires the examinee to recall where to find the answer.

22

23 ____ 7. Does the question possess a high K/A importance factor (3.0 or greater) for the 

24 job position?

25

26 ____ 8. Does the question discriminate a competent operator from one who is not?

27

28 ____ 9. Is the question appropriate for the written examination and the selected format 

29 (e.g., short answer or multiple choice)?

30

31 ____ 10. Do questions in Section A take advantage of the simulator control room setting?

32

33 ____ 11. Is the question clear, precise, and easy to read and understand?

34

35 ____ 12. Is there only one correct answer to each question?

36

37 ____ 13. Does the question pose situations and problems other than those presented 

38 during training?

39

40 ____ 14. Does the question have a reasonable estimated response time?

41
ES-6.2, Page 20 of 24

1 Form 6.2-2 Written Requalification Examination Cover Sheet

U.S. Nuclear Regulatory Commission

Written Requalification Examination

Operator Information

Name:

Date: Region: I / II / III / IV

Facility/Unit: Reactor Type: W / CE / BW / GE

AP1000 / ABWR

Start Time: Stop Time:

Instructions

Use the answer sheets provided to document your answers. Staple this cover sheet on top 

of the answer sheets. Points for each question are indicated in parentheses after the 

question. The passing grade is 80 percent. You have a total of 3 hours to complete both 

sections of the examination.

Operator Certification

All work done on this examination is my own. I have neither given nor received aid.

______________________________________

Operator’s Signature

Results

Test Value (Points) Section A: __________

Section B: __________

TOTAL: __________

Operator’s Score (Points) Section A: __________

Section B: __________

TOTAL: __________

Operator’s Grade (Combined) __________ Percent

2

3
ES-6.2, Page 21 of 24

1 Form 6.2-3 Examples of Different Types of Questions

2 1. Memory-level questions are not to be used on open-reference examinations.

3

4 2. Comprehension-level questions require the operator to demonstrate an understanding of 

5 a concept without necessarily relating it to other material or fully comprehending it in 

6 depth:

7

8 A spurious safety injection signal resulted in high head safety injection flow to the 

9 loop cold legs when the plant was in Mode 4. After completing corrective actions 

10 for the inadvertent safety injection initiation, you must do the following:

11 a. Stroke test the cold-leg motor-operated stop valves within 24 hours.

12 b. Test the cold-leg injection check valves for leakage within 48 hours.

13 c. Stroke test the cold-leg motor-operated stop valves before entering 

14 Mode 3.

15 d. Test the cold-leg injection valves for leakage before entering Mode 2.

16

17 3. Questions at the analysis, synthesis, and application levels require higher order 

18 cognitive thought processes.

19

20 a. Application-level questions may require the operator to apply his or her 

21 knowledge to various concrete situations:

22

23 Given the following conditions—

24 − Both main feed pump turbines tripped.

25 − Auxiliary feedwater (AFW) automatically started.

26 − AFW valves reset to control steam generator water level.

27 − AFW suction pressure decreases to 7 psig.

28

29 Which ONE of the following describes AFW pump response for the given

30 conditions?

31 A. The pump suction will automatically shift to nuclear service water.

32 B. The pump suction will automatically shift to upper surge tank.

33 C. The pump will trip when suction pressure decreases to 5 psig.

34 D. The pump will trip after a 6-second delay.

35

36 b. Analysis questions require the operator to mentally integrate a number of 

37 conditions, analyze their interrelationships, sort through and discriminate among 

38 distractors, and finally choose the correct answer:

39

40 Which answer below correctly indicates the posting required for a room 

41 using the results of the following radiological survey?

42 SURVEY RESULTS:

43 AIRBORNE ACTIVITY: 6.44 E-9 uci/cc (Co-60)

FLOOR SMEAR: Beta-610 dpm/cm2

; Alpha-4 dpm/cm2 44
ES-6.2, Page 22 of 24

EQUIPMENT SMEAR: Beta-1,800 dpm/cm2

; Alpha-16 dpm/cm2 1

2 GENERAL RADIATION LEVEL: 110 mr/hr

3 A. Radiation Area, Airborne Area, and Full Anti-Cs

4 B. High Radiation Area, Airborne Area, and Full Anti-Cs

5 C. High Radiation Area, Full Anti-Cs

6 D. Locked High Radiation Area, Airborne Area, Double Anti-Cs

7

8 c. Problem-solving questions require putting together elements to demonstrate an 

9 understanding of the underlying knowledge:

10

11 The plant is operating at 100-percent power when a loss-of-coolant 

12 accident occurs. The reactor trips automatically, but fast transfer fails,

13 and buses 1A1 and 1A2 become deenergized. Pressurizer pressure low 

14 signal and containment pressure high signal initiate, and all equipment 

15 operates as designed.

16 Which ONE of the following is the expected system response?

17 a. Offsite power low signal (OPLS) initiates load shed and starts both 

18 emergency diesel generators.

19 b. OPLS does NOT actuate; the emergency diesel generators start 

20 and reenergize buses 1A1 and 1A2.

21 c. OPLS does NOT actuate; the emergency diesel generators do 

22 NOT start, and the sequencers start safeguards motors.

23 d. OPLS does NOT actuate; the emergency diesel generators run at 

24 idle speed, and the sequencers start safeguards motors.

25
ES-6.2, Page 23 of 24

1 Form 6.2-4 Example Formats for Open-Reference Questions

2 1. Given plant, system, or component condition or problem—

3 • Diagnose the cause of the problem.

4 • Identify the location of the problem.

5 • Predict the effect on the plant/system/component.

6 • Identify the precipitating events/actions.

7 • Classify and indicate whether the conditions meet the specified criteria.

8 • Indicate and use the proper procedures/references.

9 • Identify the appropriate recuperative actions.

10

11 2. Given plant conditions and operator actions or procedural steps implemented—

12 • Indicate the purpose/reason behind taking these actions.

13 • Determine whether the correct actions were taken, given available cues.

14 • Indicate what further actions are required to achieve a desired effect.

15

16 3. Given a proposed or hypothetical course of action or recommendation—

17 • Determine its appropriateness or acceptability.

18 • Predict the expected plant/system/component response.

19 • Predict the effect on other systems/components.

20

21 4. Given data on plant conditions or parameters—

22 • Compute or determine the status or change in other parameters.

23 • Use, for example, charts, curves, and graphs to perform calculations or estimations.

24
ES-6.2, Page 24 of 24

1 Form 6.2-5 Developing Open-Reference Test Items

2 Analysis-level open-reference questions should be developed according to the following 

3 decision steps and mental model:

4

5 1. Determine the purpose of the test. Do you want to test knowledge where and 

6 knowledge what/how?

7

8 2. Determine the information needed to respond to the question. Is the volume and kind of 

9 information such that you would not normally expect candidates to recall the information 

10 from memory to answer the question?

11

12 3. If the answer is yes to both Questions 1 and 2, develop an open-reference question.

13

14 4. Construct the question as two tiers:

15

Tier Purpose Process Criteria Outcome

1. Knowledge where Evaluate reference 

sources

Avoid clues in stem Locate reference 

sources

2. Knowledge 

what/how

Integrate multiple 

variables/events

Information volume 

and detail high (not 

in memory)

Identify correct 

answer

16

17 Question Stem

18

19 bits, chunks of stem information

20 (conditions, setpoints, components, etc.)

21

 Mental Processes Processes

* Analyzing a. Answer * Integrate mental 

processes with stem 

information, reference 

data, and distractors.

* Sorting b. Distractor

* Eliminating ←────→ c. Distractor ←──→

* Differentiating d. Distractor

* Evaluating

Determine Answer

22
ES-6.3, Page 1 of 7

1 6.3 REQUALIFICATION WALKTHROUGH EXAMINATIONS

2 NRC examiners, working with facility evaluators, follow this examination standard to administer 

3 walkthrough requalification examinations as authorized by 10 CFR 55.59(a)(2)(iii). The 

4 walkthrough examination is an effective tool for evaluating the ability of a licensed operator to 

5 manipulate system components and controls, interpret references, use administrative 

6 procedures, and demonstrate knowledge of component locations.

7 A. Scope

8 This standard provides specific guidance and requirements for NRC examiners to use in 

9 preparing, reviewing, and administering walkthrough requalification examinations in which each 

10 operator performs five JPMs. Each operator’s walkthrough examination is designed to test the 

11 operator on plant systems that are important to the safe operation of the reactor. NRC 

12 examiners and facility evaluators jointly approve the JPMs for each examination. Each JPM 

13 consists of several steps, one or more of which is designated as “critical.” An operator must 

14 properly complete each critical step to pass the JPM.

15

16 The examination team will agree on five JPMs so that at least two are conducted in the 

17 simulator (or the control room) and at least two are conducted in the plant. To the maximum 

18 extent practical, control room JPMs will be conducted using the simulator. When operators 

19 perform JPMs in the control room or the plant, they will be cautioned not to manipulate the 

20 reactor controls. To successfully complete these JPMs, operators will demonstrate to the 

21 examiners the steps or actions they would take to complete the task.

22

23 B. Development of Job Performance Measures

24 The following are the activities conducted by the facility licensee and the NRC examination team 

25 for a NRC conducted requalification walkthrough examination.

26

27 1. Facility Examination Team Members’ Responsibilities

28 a. The NRC staff expects the facility licensee to identify the plant systems that are critical to 

29 protecting public health and safety. The systems that are selected for the examination 

30 should meet the following criteria:

31

32 • systems covered during the current requalification cycle (the facility’s sample 

33 plan should identify the systems and appropriate learning objectives; see 

34 Form 6.1-2, “Examination Sample Plan”)

35

36 • new or recently modified systems

37

38 • systems that are the subject of recent facility LERs or vendor notices

39
ES-6.3, Page 2 of 7

• risk-important systems, components, and operator actions0F

1 1 for plant or vendor 

2 generic systems as identified through probabilistic risk assessment (PRA)

3 systems that are the subject of NRC information notices

4 • systems that are important to safety during low-power or shutdown operations

5

6 b. For those systems that are identified as being important to safety, the facility 

7 representatives are expected to review the JTA, learning objectives, and applicable K/A 

8 catalog. The facility representative should highlight for use as JPMs the tasks, abilities, 

9 and learning objectives that fulfill the following criteria:

10

11 • apply to the facility

12

13 • are at the appropriate level for the operator being examined (i.e., the RO is 

14 responsible for AO/RO tasks, and the SRO is responsible for AO/RO/SRO tasks)

15

16 • have a K/A rating of 3.0 or higher (tasks and abilities selected for use may have 

17 ratings below 3.0 if proper facility justification exists for such ratings)

18

19 c. Some tasks important to safety are unique to a specific plant and are not referenced in 

20 the applicable K/A catalog. The NRC staff expects each facility to maintain a list of 

21 these plant-specific tasks and develop JPMs that test the operator’s K/A in these areas. 

22 Before submitting the JPMs to the NRC for review, the facility is responsible for ensuring 

23 that the tasks are appropriate to the applicable license level and have a safety 

24 importance rating of at least 3.0.

25

26 If a facility-specific K/A is used in lieu of those specified in the applicable K/A catalog, the 

27 importance ratings must be based on protecting public health and safety.

28

29 d. JPMs should meet the development instructions in ES-3.2, “Developing Job 

30 Performance Measures,” and Form 3.2-4, “Job Performance Measure Development Job

31 Aid.” The JPMs should indicate which steps are “critical” to successful completion of the 

32 task. Critical steps are those steps that when not performed correctly, in the proper 

33 sequence, or at the proper time, will prevent the system from functioning properly or 

34 preclude successful completion of the task. Form 3.2-3, “Job Performance Measure 

35 Template,” or an equivalent facility form should be used to construct and format the 

36 JPMs.

37

38 In accordance with 10 CFR 55.59(a)(2)(ii), requalification operating tests require 

39 operators and senior operators to demonstrate an understanding of, and ability to, 

40 perform necessary actions. Therefore, JPMs selected for the walkthrough examination 

41 shall not test solely for simple recall or memorization. Although it addresses written 

42 examinations, ES-6.2, Section C, “Guidelines for Developing and Reviewing Open43 Reference Examinations,” should be used when preparing JPMs. Although an operating 

44 test does not require every JPM to be an alternate path or demonstrate detailed system 

1 Chapter 13 of NUREG-1560, “Individual Plant Examination Program: Perspectives on Reactor Safety and 

Plant Performance,” issued December 1977, identifies important human actions that might be appropriate 

for the operating test. In determining important operator actions, do not overlook actions that are relied on 

or result in specific events being driven to lower risk contribution. This will help identify those human 

actions, assumed to be very reliable, that might otherwise not show up on a list of risk-dominant actions.
ES-6.3, Page 3 of 7

1 understanding, simple one-step JPMs or JPMs that require only directly look up of the 

2 correct answer are not appropriate. JPMs that incorporate the testing of immediate 

3 action steps from memory are acceptable. However, JPMs should not solely test 

4 immediate action steps and should include testing additional steps or items that are not 

5 from memory.

6

7 The majority of the JPMs selected for the walkthrough examination will cover topics from 

8 the most recent requalification training cycle. In addition, the facility is expected to 

9 create at least 10 new JPMs each year until it has a JPM bank that is representative of 

10 Sections B.1.a and B.1.b of this examination standard. The NRC anticipates that a 

11 facility’s bank will comprise approximately 125–150 JPMs; however, the exact number 

12 will depend on the facility’s JTA. New JPMs should generally be based on recent 

13 requalification training, industry events, facility changes, and tasks for safety-significant 

14 systems.

15

16 e. The NRC staff expects each facility to develop “time-critical” JPMs to evaluate 

17 time-critical tasks identified in the facility’s JTA for each licensed position. To facilitate 

18 the selection of time-critical JPMs for the requalification examination, the facility licensee 

19 is expected to uniquely identify these JPMs. To successfully complete a time-critical 

20 JPM, the operator must perform the “time-critical” steps within a prespecified time 

21 period, in addition to successfully performing all of the critical steps that are not time 

22 critical. The time period identified in the time-critical JPM should be based on a 

23 regulatory requirement or a facility commitment to the NRC.

24

25 f. The NRC staff also expects each facility to develop “alternate-path JPMs” and include 

26 them in the JPM bank. To facilitate the selection of alternate-path JPMs for the 

27 requalification examination, the NRC staff expects the facility licensee to uniquely 

28 identify these JPMs. ES-3.2 provides guidance for use in developing these JPMs.

29

30 2. NRC Examination Team Members’ Responsibilities

31 a. The NRC’s examination team will review and approve the JPMs selected by the facility. 

32 The majority of the selected JPMs should be based on the systems covered during the 

33 most recent requalification cycle. However, the facility should also select JPMs in 

34 systems that are important to safety, regardless of when they were reviewed in 

35 requalification training. NRC examiners will review the JPMs submitted by the facility to 

36 ensure that the most recent training cycle did not cover 20 percent of the selected JPMs, 

37 because this examination is intended to sample skills and abilities that operators should 

38 always be able to display. In general, examiners should select systems in Groups I and 

39 II of the appropriate written examination outline form in ES-4.1, with Group I comprising 

40 at least 50 percent of the selected systems.

41

42 b. The NRC staff will discuss with the facility representatives the selected JPMs that are 

43 not identified in the applicable K/A catalog to ensure that the system or task meets the 

44 site-specific importance criteria. The staff will also discuss any modifications to the 

45 selection of JPMs with the facility representatives. The NRC may substitute up to 

46 20 percent of the facility-proposed JPMs with NRC-developed JPMs. The NRC will give 

47 facility representatives sufficient time to review any substituted JPMs.

48
ES-6.3, Page 4 of 7

1 c. The NRC chief examiner has the authority to decide the content of each examination 

2 set. NRC examiners should review the proposed JPMs using the criteria in ES-3.2 for 

3 developing JPMs.

4

5 d. The NRC chief examiner will ensure that enough different JPMs are scheduled during 

6 the examination week to avoid compromising the examination.

7

8 e. The NRC chief examiner will ensure that the time validation of each JPM is reasonable 

9 and will verify that each JPM is identified as “time critical” or “not time critical.”

10

11 C. Examination Administration

12 1. Conducting Job Performance Measure Walkthrough Examinations

13 a. The facility evaluator is responsible for conducting the walkthrough examination while 

14 the NRC examiner observes. The NRC examiner and the facility evaluator may ask the 

15 operator questions to clarify his or her performance of the JPM after the measure is 

16 completed. In most instances, the NRC examiner will ask the facility evaluator to 

17 question the operator about the appropriateness of an action or a response that does not 

18 follow the actions specified in the JPM.

19

20 b. The facility evaluator will brief the operator using ES-1.2. If desired, the evaluator may 

21 brief the operators as a group before starting the walkthrough examination.

22

23 c. Operators should not be informed of the expected completion time before commencing 

24 the JPM. Informing operators of the expected completion time may increase tension as 

25 operators approach the time limit. However, the evaluator will inform operators that a 

26 JPM is time critical.

27

28 d. Time should be allotted during the operating test for evaluating each operator’s 

29 performance of five JPMs.

30

31 Each walkthrough examination should last approximately 2 hours. This time includes 

32 the validated times associated with each planned JPM plus any administrative tasks 

33 required to conduct the examination.

34

35 Administrative tasks include the following examples:

36

37 • transit time to and from the plant site

38

39 • time spent complying with facility security and radiological administrative 

40 requirements (unless this is part of the JPM being performed)

41

42 • transit time within the plant after a JPM is completed to arrive at the location 

43 where the initiating cue for the next JPM is to be given

44

45 Note: The JPM sample size will be constrained to the requirements of this examination 

46 standard for NRC-conducted examinations. The facility may perform an additional 

47 evaluation of its operators outside the timeframe designated for the NRC examination.

48
ES-6.3, Page 5 of 7

1 However, the final requalification evaluation of the operator by the NRC will not factor in 

2 any additional evaluation by the facility. The criteria for determining requalification 

3 program status remain the same.

4

5 e. JPMs that directly relate to the operator’s job functions are preferable, particularly for 

6 SROs. For example, if an SRO will not perform an emergency action level classification 

7 during the dynamic simulator or written examinations, the examination team may choose 

8 to have the operator perform one JPM that involves classifying an emergency.

9

10 f. The NRC examiner will ensure that the facility evaluator conducts an appropriate 

11 examination. ES-3.5, “Administering Operating Tests,” provides examples of good 

12 evaluation techniques to use during the walkthrough examination. If the NRC examiner 

13 observes improper evaluation techniques that may render the examination invalid, the 

14 NRC examiner will stop the walkthrough and counsel the facility evaluator. If the facility 

15 evaluator continues to display poor evaluation techniques, the NRC examiner will stop 

16 the examination and request that another facility evaluator continue the examination. If 

17 necessary, the NRC examiner may conduct the walkthrough with the original facility 

18 evaluator observing and co-evaluating.

19

20 g. If an evaluator believes that followup questioning is required and is not sure how to 

21 phrase the question, he or she should consult the NRC examiner. This will avoid 

22 inadvertent prompting of the operator and enhance communication between the facility 

23 evaluator and the NRC examiner.

24

25 h. The examiner will document the operator’s performance using the applicable portions of 

26 Form 3.2-3 or the facility equivalent for each JPM. The examiner will document any 

27 questions asked to clarify the operator’s performance and also fill out Form 6.3-1, “Job 

28 Performance Measure Summary Matrix,” to maintain operators’ scores during the 

29 examination; document which JPM each operator performed; and fulfill the record 

30 retention requirements of ES-6.1.

31

32 i. After completing an operator’s JPM set, the NRC and facility evaluators shall discuss 

33 and resolve any outstanding issues that may result in the operator failing the 

34 walkthrough examination or any individual JPM. A discussion of their observations will 

35 often correct a difference of opinion. Unresolved differences should be brought to the 

36 attention of the NRC chief examiner.

37

38 2. Grading the Examination

39 a. To pass the walkthrough examination, each operator must successfully complete at least 

40 four of the five JPMs. To successfully complete a JPM, the operator must complete all 

41 critical steps and satisfy the completion criteria specified in the given JPM.

42

43 b. The NRC expects an operator to complete each JPM within the validated time period. 

44 For a JPM that is not time critical, an operator may exceed the validated time if the 

45 facility evaluator and the NRC examiner agree that the operator is making acceptable 

46 progress toward completing the JPM.

47

48 For time-critical JPMs, the facility representatives should identify a period that they 

49 consider to be the absolute maximum time in which they would expect an operator to 

50 perform the given task (e.g., locally opening reactor trip breakers on an anticipated 
ES-6.3, Page 6 of 7

1 transient without scram or locally starting an auxiliary feedwater pump on a loss of all 

2 feedwater). An operator who fails to meet the time criteria will receive an unsatisfactory 

3 evaluation for the given JPM.

4

5 D. Forms

6 Form 6.3-1 Job Performance Measure Summary Matrix

7
ES-6.

3, Page 

7 of 7

Form 6.3-1 Job Performance Measure Summary Matrix

Operators’ Names >>>

Job Performance Measure Number/Brief Description

1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

11.

12.

13.

14.

15.

ES-6.4, Page 1 of 19

1 6.4 DYNAMIC SIMULATOR REQUALIFICATION EXAMINATIONS

2 NRC examiners use this standard in preparing and administering dynamic simulator 

3 requalification operating tests in accordance with the provisions of 10 CFR 55.59(a)(2)(iii).

4 By simulating actual plant operation, the dynamic simulator test provides a comprehensive 

5 evaluation of the integrated plant knowledge and skills required of operating crews. It is 

6 effective in evaluating a crew’s communication skills and team behavior and in identifying any 

7 areas in which the licensed operators should be retrained to improve their K/As in accordance 

8 with the requalification program developed by the facility licensee.

9 A. Scope

10 The dynamic simulator test consists of two scenarios. Each scenario is constructed to last 

11 approximately 45 to 60 minutes. The actual time needed to complete the scenarios will depend 

12 on the specific events within the scenarios but should allow the crew the time necessary to 

13 perform the actions required to respond to each event. To successfully complete this portion of 

14 the operating test, the crew must demonstrate the ability to operate effectively as a team while 

15 completing a series of critical tasks (CTs) that measure the crew’s ability to safely operate the 

16 plant during normal, abnormal, and emergency situations.

17

18 The NRC examiners evaluate the performance of each crew, using standard competency rating 

19 scales. Each competency is rated according to the crew’s ability to satisfactorily complete the 

20 tasks that have been designated as “critical” within that crew’s scenario set. Each CT must 

21 meet the methodology specified in ES-3.3, “Testing Guidelines for Dynamic Simulator 

22 Scenarios.” If the crew fails to correctly perform a CT, that failure would indicate a critical 

23 performance deficiency (CPD) in the knowledge, skill, or ability of that crew to demonstrate 

24 team behavior and will be evaluated using the behavioral anchors on Form 6.4-2, “Simulator 

25 Crew Evaluation Form.”

26

27 Facility evaluators will assess the performance of the operators during the dynamic simulator 

28 test. Because the primary purpose of the dynamic simulator test is to evaluate crews, each 

29 individual is not required to perform a specific number of CTs and may not necessarily receive 

30 an individual evaluation by an NRC examiner. However, NRC examiners will follow up on 

31 individual CPDs and significant performance deficiencies (SPDs) observed during the simulator 

32 test in a manner and setting compatible with the deficiency. This followup evaluation will be 

33 graded as a component of the individual’s operating test. To meet the requirements of 

34 10 CFR 55.59(a)(2), it is the facility licensee’s responsibility to conduct its annual operator 

35 performance evaluations on the dynamic simulator in accordance with the requirements of its 

36 requalification program. The facility licensee may use the NRC-conducted operating test to 

37 meet this requirement if the conditions of 10 CFR 55.59(a)(2)(ii) are satisfied (the operating test 

38 will require the operator or senior operator to demonstrate an understanding of and the ability to 

39 perform the actions necessary to accomplish a comprehensive sample of items specified in 

40 10 CFR 55.45(a)(2) through (13) inclusive, to the extent applicable to the facility).

41

42 If an operator demonstrates SPDs or CPDs during the dynamic simulator portion of the 

43 operating test, the facility and NRC examination team members should discuss those 

44 deficiencies at the end of the dynamic simulator test.

45

46 If the operating crew performs satisfactorily and NRC examiners observe no individual CPDs or 

47 SPDs, the individual would pass the dynamic simulator test. In the case of operators who 
ES-6.4, Page 2 of 19

1 demonstrate SPDs or CPDs, the facility evaluators and NRC examiners will decide whether the 

2 operator would pass or fail by asking the operator followup questions about his or her 

3 performance to determine the extent of the knowledge or ability deficiency demonstrated. The

4 NRC examiners and facility evaluators will agree to number and scope of followup questions to 

5 be asked, which will be based on the individual’s demonstrated K/A performance deficiencies. 

6 The followup questions and the individual’s answers will be documented and used along with 

7 the individual’s performance as the basis for a pass or fail decision. Section D.2 of this standard 

8 describes the method for evaluating and documenting individual performance.

9

10 In the rare event that the only way to evaluate the scope and depth of the individual’s 

11 performance deficiency is by conducting another scenario to gain additional information, the 

12 examination team (the NRC and facility) will determine the content, CTs, operator actions, and 

13 crew position rotation necessary to complete the evaluation of the individual’s performance. 

14 Conducting another scenario is time consuming and may adversely affect the examination 

15 process. If an individual operator exhibits only minor deficiencies in performance and 

16 satisfactorily completes the testing requirements of 10 CFR 55.59(a), then remedial retraining 

17 and reevaluation will be conducted in accordance with the facility licensee’s requalification 

18 program.

19

20 B. Examination Development

21 Developing the NRC’s dynamic simulator requalification examination is a combined effort 

22 between the facility representatives and the NRC examiners on the examination team. The 

23 responsibilities of the examination team members are outlined below.

24

25 1. Facility Team Member Responsibilities

26 The facility licensee examination team is responsible for the following activities:

27

28 a. The facility licensee develops the dynamic simulator scenarios with identified CTs that 

29 meet the methodology specified in ES-3.3 and Form 6.4-1, “Simulator Scenario Review 

30 Checklist.” The facility licensee will submit each proposed dynamic simulator test to the 

31 NRC chief examiner 45 days before the scheduled examination.

32

33 b. The NRC expects the facility licensee to provide a qualified simulator operator to assist 

34 in developing and administering the simulator examinations. The simulator operator 

35 must be available to support the examination team during the examination preparation 

36 week, normally 2 weeks before the examination.

37

38 The simulator operator needs to sign a security agreement at the time that the chief 

39 examiner determines that he or she has access to specialized knowledge of any part of 

40 the examination.

41

42 c. The scenarios should be based on the training conducted during the requalification 

43 cycle, recent industry events, LERs, emergency and abnormal procedures, and design 

44 and procedural changes. The scenarios should demonstrate the crew’s ability to use 

45 facility procedures to prevent and mitigate accidents. Some scenarios should be based 

46 on the dominant accident sequences for the facility or actual events that have occurred 

47 at that or a similar facility. Dominant accident sequences are those that contribute 

48 significantly to the frequency of core damage as determined by the facility licensee’s 

49 probabilistic risk assessment (PRA) or individual plant examination (IPE). The PRA/IPE 
ES-6.4, Page 3 of 19

should also be used to identify risk-important operator actions.0F

1 1 In identifying those 

2 actions, actions that are relied on or result in specific events being driven to lower risk 

3 contribution should not be overlooked. This will help identify those human actions that 

4 are assumed to be very reliable and might not otherwise show up in a list of 

5 risk-dominant actions.

6

7 d. The facility representatives on the examination team will have the opportunity to review 

8 any modifications that the NRC makes to the scenarios. The representatives may 

9 recommend changes to events that are critical to plant safety, but they must substantiate 

10 the reasons for those changes. The examination team has to agree on the validity and 

11 content of each scenario before the examination.

12

13 e. The NRC encourages each utility to have its management discuss with the agency any 

14 problems with examination complexity. Utility managers engaged in the examination 

15 review will be required to sign a security agreement. Responsibility rests with the utility 

16 to resolve any issues before administering the examination. This review is to ensure 

17 that the final scenarios are (1) consistent with the facility’s requalification requirements 

18 for operators licensed at the facility, (2) within the capability of the simulation facility, and 

19 (3) within the scope of the facility’s procedures.

20

21 This utility’s senior manager or representative should communicate any significant 

22 concerns about scenario validity to the chief examiner. If adequate resolution is not 

23 reached, the concerns should be brought to the attention of the NRC’s regional 

24 managers and then, if necessary, to managers in the NRR operator licensing program 

25 office.

26

27 2. NRC Team Member Responsibilities

28 The NRC examination team is responsible for the following activities:

29

30 a. At least 2 weeks before the preparation week, the NRC chief examiner or a designee will 

31 complete a draft of Form 6.4-1 for each scenario that the facility proposes to use during 

32 the examination, along with any proposed changes to be validated during the 

33 preparation week. During the review of each scenario that the facility selected for the 

34 examination, the chief examiner or designee will consider the quantitative and qualitative 

35 factors described in ES-3.3, as summarized on Form 6.4-1.

36

37 b. If the proposed scenarios require major changes to follow the guidance in Form 6.4-1, 

38 the NRC chief examiner will inform the NRC regional managers and determine the 

39 appropriate actions. The NRC may revise the scenarios, as appropriate, or develop new 

40 scenarios to add to the facility’s existing scenarios, if required. The NRC will 

41 communicate all scenario changes to the appropriate facility representative early enough 

42 to allow for scenario validation before the preparation week. During the preparation 

43 week, the examiners may make minor changes to ensure that the scenario objectives 

44 are properly accomplished. The NRC staff will review the final scenarios with the 

45 facility’s examination team representatives before the examination is administered. The 

1 Chapter 13 of NUREG-1560 identifies important human actions that might be appropriate for evaluation on 

the dynamic simulator operating test.
ES-6.4, Page 4 of 19

1 NRC has the final authority to determine the content of the scenarios and decide 

2 whether a task is critical for evaluating the competency of the crew.

3

4 c. A key element of the examination team’s resolution of concerns about the scope, depth, 

5 and complexity of simulator scenarios involves a senior utility manager observing the 

6 proposed examination scenarios (subject to signing an appropriate examination security 

7 agreement) during examination preparation. If necessary, this executive would raise 

8 specific concerns to appropriate NRC regional management for resolution before the 

9 examination is administered.

10

11 C. Examination Administration

12 1. Administrative Requirements

13 a. A facility manager or representative with responsibilities for conducting plant operations 

14 (at a minimum, a manager at the first level above shift supervisor) should be present 

15 while the simulator examinations are administered. The NRC’s chief examiner or a 

16 designee will also be present during the administration of each dynamic simulator 

17 examination. The NRC chief examiner is the principal point of contact between the 

18 facility manager and the NRC.

19

20 b. The examination team briefs the operating crews before the start of the simulator 

21 scenarios, using the information in ES-1.2.

22

23 c. Crews should have adequate time to respond to all planned and unplanned events. A 

24 scenario’s contact time should be approximately 45 to 60 minutes. Contact time means 

25 the actual time the operators spend in the scenario; it does not include time spent on 

26 briefings, simulator setup, or investigating simulator performance problems.

27

28 d. Under no circumstance will any member of the examination team modify the sequence 

29 of events and transients during the scenario. If the scenario is not properly administered 

30 as a result of a simulator operator error or an unexpected simulator response, the 

31 examination team will confer immediately after the scenario set to determine whether the 

32 crew has performed enough transients and events to justify an evaluation of the required 

33 competencies. If necessary, the examination team can run an additional scenario to 

34 ensure that the required competencies are addressed.

35

36 e. Crew rotation practices shall be discussed and agreed to during the preparation week, 

37 and any problems shall be resolved before the administration of the operating test.

38

39 f. The members of the operating crew should maintain the same operating positions as 

40 during facility requalification evaluations. The crew members should rotate between 

41 positions in the manner identical to the facility’s rotation practices for evaluations 

42 specified in the facility’s requalification program.

43

44 g. SROs must be evaluated in at least one scenario in an SRO licensed crew position. 

45 More than two simulator scenarios may be required to examine crews that consist of 

46 more than four SROs.

47
ES-6.4, Page 5 of 19

1 2. Postscenario Activities

2 a. If the NRC examiners and facility evaluators observe actions that are unexpected during 

3 the simulator scenario, they should question the crew members as necessary to 

4 completely document the crew’s performance during the scenario. Questions should be 

5 factual and should clarify performance related to observations.

6

7 b. If an examiner observes an individual who demonstrates performance deficiencies in 

8 performing a CT, the NRC examiner and the facility evaluator will discuss those 

9 performance deficiencies at the completion of the scenario. If they determine that the 

10 operator’s performance deficiencies cannot be assessed because of a lack of 

11 information, the examination team has the option to conduct an additional scenario or a 

12 JPM to obtain the necessary information.

13

14 During the postscenario discussion, the NRC expects the facility evaluator to describe 

15 the operator’s deficiencies to the NRC examiner and suggest a series of followup 

16 questions designed to identify the cause of the deficiency. The NRC examiner will 

17 assess the facility evaluator’s ability to diagnose the operator’s deficiency and document 

18 it in the examination report, if applicable. The NRC examiner has the option to augment 

19 the followup questions proposed by the facility evaluator, if necessary.

20

21 The examination team should minimize the time needed to conduct this review of crew 

22 and individual performance to minimize the impact on the operators. However, the 

23 examination team’s responsibility is to ensure that the review is thorough and complete.

24

25 The facility evaluator will conduct an individual evaluation of the operator in accordance 

26 with Section D.2 of this examination standard. The NRC examiner has the option to ask 

27 additional followup questions.

28

29 c. Upon completing any followup questioning, the NRC examiners and facility evaluators 

30 will dismiss the crew to await the next scenario and inform the crew members that they 

31 may discuss the completed scenario among themselves.

32

33 d. The NRC examiners and facility evaluators will meet separately to compare observations 

34 and determine whether the crew omitted or incorrectly performed any CTs (CPDs) or 

35 had any SPDs.

36

37 e. The facility evaluators will discuss the crew’s performance with the NRC examiners after 

38 each scenario to clarify any performance deficiencies noted. The examination team will 

39 determine whether the as-run scenario has invalidated any predesignated CTs or 

40 whether any new CTs should be designated to evaluate unpredicted events or actions 

41 taken by the crew during the scenario. The examination team will then revalidate the 

42 CTs in each scenario, using the methodology presented in ES-3.3.

43

44 f. After the crew completes the last scenario, the NRC examiners and facility evaluators 

45 will independently complete Form 6.4-2 as discussed in Section D. The facility 

46 evaluators will also evaluate individual operator performance in accordance with their 

47 requalification program requirements and Section D.2. In addition, the NRC examiners 

48 will review the facility’s evaluations of individual operator performance after completing 

49 each crew evaluation.

50
ES-6.4, Page 6 of 19

1 D. Performance Evaluations

2 Two separate evaluations will be conducted based on the information obtained during the 

3 dynamic simulator examination. The first is a crew simulator evaluation. For the second, the 

4 examination team uses individual simulator performance to determine whether followup 

5 questioning of the operator is necessary. After observing the operator’s performance in the 

6 dynamic simulator and evaluating the responses to followup questions, the examination team 

7 may conclude that additional performance information about the operator must be obtained to 

8 make an individual evaluation. In this case, an additional scenario or JPM would be conducted. 

9 The individual followup would then be documented along with the individual’s crew evaluation 

10 on Form 6.1-6.

11

12 Each operator will be subject to failure based on a competency evaluation of his or her 

13 performance on the dynamic simulator and the required followup evaluation if he or she 

14 exhibited deficient performance in executing a crew CT.

15

16 1. Crew Simulator Evaluations

17 After administering the dynamic simulator scenario set discussed in Section C, the NRC 

18 examiners and facility evaluators will independently evaluate the crew’s performance by 

19 completing Form 6.4-2. The facility is expected to provide its final crew evaluations to the NRC 

20 examiners before the crew members return to licensed duties or the end of the examination 

21 week, whichever is sooner. Specific guidance for completing Form 6.4-2 appears on the first 

22 page of the form.

23

24 Each individual’s examination results and the facility requalification program evaluation will 

25 factor in the results of the crew evaluations. Members of a crew that receives an unsatisfactory 

26 crew evaluation are expected to receive remedial training from the facility licensee and to be 

27 reevaluated in accordance with the facility licensee’s NRC-approved requalification program 

28 before returning to licensed duties. Although operators are not required to take an 

29 NRC-conducted requalification examination for purposes of license renewal, those who fail to 

30 pass (individually or as a member of a crew) an examination conducted by the NRC must be 

31 reevaluated by the agency before their license will be renewed. The level of NRC involvement 

32 during the reevaluation will be determined on a case-by-case basis (see Section E.1 of ES-6.1).

33

34 NRC examiners will document the results of each operator’s crew performance in the “Simulator 

35 Examination Results” section of Form 6.1-6.

36

37 2. Individual Operating Evaluations

38 The facility licensee is primarily responsible for individual operating evaluations on the dynamic 

39 simulator examination and the resulting remedial training. Unsatisfactory operator performance 

40 of a crew CT will be followed up after the simulator scenario and documented on Form 6.1-6.

41

42 The NRC expects facility evaluators to document and grade individual operator performance 

43 during the dynamic simulator examination in accordance with the requirements of the facility 

44 licensee’s requalification program. The NRC also expects the facility’s grading methodology to 

45 identify operator deficiencies, and for the facility evaluators to discuss those deficiencies with 

46 the NRC examiners during the meetings after the scenarios as described in Section C. The 

47 facility evaluators should document the deficiencies and remediate and retest the operators for 

48 the identified deficiencies in accordance with the facility licensee’s requalification training 
ES-6.4, Page 7 of 19

1 program. At a minimum, the NRC expects the facility evaluators to identify any operator on the 

2 crew who was directly responsible for the omission or incorrect performance of validated CTs.

3 Individual followup is conducted if an operator has SPDs or CPDs. As described in 

4 Section C.2.b of this examination standard, the NRC examiner will assist in developing and 

5 administering followup questions specific to the performance deficiencies that the operator 

6 displayed. The examination team will determine the number and scope of the followup 

7 questions that will be asked based on a review of the operator’s performance deficiencies at the 

8 completion of the scenario. The examination team has the option to gather additional 

9 information about an operator who displays SPDs or CPDs by either running an additional 

10 scenario or using JPMs if the dynamic simulator examination and followup questioning are 

11 inconclusive.

12

13 Upon completion of the individual followup questions, the NRC examiner will complete an 

14 individual competency evaluation using the appropriate sections of Form 6.4-2 or the facility’s 

15 equivalent form. Only those competencies that deal with the operator’s individual performance 

16 deficiencies should be filled out. If the NRC examiner gives the operator a rating factor score of 

17 “1” in either of the following cases, the individual fails this portion of the examination:

18

19 • any two rating factors in any one competency

20

21 • any one rating factor in any one competency if, in the judgment of the examination team, 

22 the operator’s performance deficiency jeopardizes the safety of the plant or has a 

23 significant safety impact on the public

24

25 NRC management will make the final decision concerning all operator failures resulting from a 

26 single rating factor evaluation of “1.”

27

28 When conducting the evaluation described here, NRC examiners will not assign rating factor 

29 scores of “1” based solely on performance in the dynamic simulator. The examiners will ask 

30 followup questions and record the operator’s responses to evaluate and document CPDs and 

31 SPDs.

32

33 The NRC examiner will then apply the individual’s responses to the questions asked to evaluate 

34 and justify individual performance deficiencies that warrant a rating factor score of “1.” The 

35 examiner will document and include the followup questions asked and the responses given by 

36 the operator. The results of the operator’s simulator examination will include written comments 

37 describing the operator’s performance and the as-run simulator scenario set.

38

39 The NRC examiner will document the pass or fail determination for each operator’s individual 

40 followup under “Individual Followup” in the “Simulator Examination Results” section of 

41 Form 6.1-6.

42

43 If an operator demonstrates no performance deficiencies and, therefore, does not require any 

44 additional followup questioning, regardless of whether the crew passes or fails the dynamic 

45 simulator examination, the NRC examiner will record an “N/A” for “Individual Followup” in the 

46 “Simulator Examination Results” section of Form 6.1-6.

47

48 E. Forms

49 Form 6.4-1 Simulator Scenario Review Checklist
ES-6.4, Page 8 of 19

1 Form 6.4-2 Simulator Crew Evaluation Form

2
ES-6.4, Page 9 of 19

1 Form 6.4-1 Simulator Scenario Review Checklist

2 Note: Attach a separate copy of this form to each scenario reviewed. The examination team 

3 uses this form as guidance in reviewing the proposed scenarios.

4

5 SCENARIO IDENTIFIER: REVIEWER: 

6

7 Qualitative Attributes

8

9 1. The scenario summary clearly states the objectives of the scenario.

10

11 2. The initial conditions are realistic in that some equipment or instrumentation, or both,

12 may be out of service, but the conditions do not cue the crew to expected events.

13

14 3. The scenario consists mostly of related events.

15

16 4. Each event description consists of the following:

17 • the point in the scenario when it is to be initiated

18 • the malfunction(s) that is entered to initiate the event

19 • the symptoms and cues that will be visible to the crew

20 • the expected operator actions (by shift position)

21 • the event termination point

22

23 5. No more than one nonmechanistic failure (e.g., pipe break) is incorporated into the 

24 scenario without a credible preceding incident, such as a seismic event.

25

26 6. The events are valid with regard to physics and thermodynamics.

27

28 7. Sequencing and timing of events are reasonable and allow the examination team to 

29 obtain complete evaluation results commensurate with the scenario objectives.

30

31 8. The scenario summary clearly indicates whether time compression techniques are 

32 used. Operators have sufficient time to carry out expected activities without undue 

33 time constraints. Cues are given.

34

35 9. The simulator modeling is not altered.

36

37 10. All crew competencies can be evaluated.

38

39 11. The scenario has been validated.

40
ES-6.4, Page 10 of 19

1 Form 6.4-1 Simulator Scenario Review Checklist (Continued)

2

3 Note: The “Quantitative Attributes” criteria address scenario traits that are numerical in nature. 

4 A second set of numbers indicates a range to be met for a set of two scenarios. 

5 Therefore, to complete this part of the review, the set of scenarios must be available. 

6 This page should be completed once per scenario set.

7

8 Scenario Set Consists of Scenario: and Scenario: 

9

10 Quantitative Attributes

11

12 12. If the sampling plan indicates that the scenario was used for training during the 

13 requalification cycle, the need to modify or replace the scenario has been evaluated.

14

15 13. Total malfunctions inserted: 5 to 8 / 10 to 14

16

17 14. Malfunctions that occur after emergency operating procedure (EOP) entry:

18 1 to 4 / 3 to 6

19

20 15. Abnormal events: 2 to 3 / 4 to 5

21

22 16. Major transients: 1 to 2 / 2 to 3

23

24 17. EOPs used beyond the primary scram response EOP: 1 to 3 / 3 to 5

25

26 18. EOP contingency procedures used: 0 to 3 / 1 to 3

27

28 19. Approximate scenario run time: 45 to 60 minutes (One scenario may approach 

29 90 minutes.)

30

31 20. EOP run time: 40 to 70 percent of scenario run time

32

33 21. Crew critical tasks: 2 to 5 / 5 to 8

34

35 22. Technical specifications are exercised during the test.

36

37

38 COMMENTS: 

39
ES-6.4, Page 11 of 19

1 Form 6.4-2 Simulator Crew Evaluation Form

2 The examination team should use this evaluation form during the dynamic simulator component 

3 of the requalification examination. The rating scales on this form are for evaluating the crew as 

4 a whole, rather than the individual operators. Use the following instructions when rating team 

5 performance on the simulator examination:

6

7 1. Review the rating scales before the simulator examination so that you are familiar with 

8 each competency to be evaluated.

9

10 2. Use Form 3.3-2, “Required Operator Actions,” or an equivalent facility form to make 

11 notes during the examination, as described in ES-3.5, “Administering Operating Tests.”

12

13 3. Complete this form immediately after the simulator examination. Evaluate the crew’s 

14 performance on each applicable rating factor by comparing the actions of the crew 

15 against the associated behavioral anchors and selecting the appropriate grade. The 

16 tasks planned and performed during the crew’s scenario set may not permit you to 

17 evaluate every rating factor for every crew. Annotate those rating factors that are not 

18 used in the evaluation.

19

20 The examination team should pay particular attention to the completion of tasks that they 

21 identified as critical to plant safety. The crew may compensate for actions that individual 

22 operators performed incorrectly as long as the critical task was completed satisfactorily. 

23 The rating factor evaluations should also account for other less significant deficiencies to 

24 provide information for crew remedial training during subsequent requalification training.

25

26 4. Justify all rating factor grades of “1” and document each justification in the space for 

27 “Comments” on the form. Rating factor grades of “1” must be linked to the performance 

28 of at least one critical task.

29

30 5. Complete the simulator examination summary sheet, recording for each scenario the 

31 scenario name (or identifier) and the critical tasks performed by the crew. Annotate 

32 whether the critical task was performed satisfactorily or unsatisfactorily. Complete the 

33 crew’s overall evaluation using the criteria listed in the next paragraph. Space is 

34 provided for additional comments about the crew’s performance.

35

36 6. The threshold for failing the simulator portion of the examination is to receive a 

37 (behavioral anchor) score of “1” in either of the following:

38

39 a. any two rating factors in any one competency

40

41 b. any one rating factor in any one competency if, in the judgment of the 

42 examination team, the crew’s performance deficiency jeopardizes the safety of 

43 the plant or has a significant safety impact on the public (U.S. Nuclear Regulatory 

44 Commission management will make the final decision concerning all crew 

45 failures resulting from a single rating factor evaluation of “1”)

46

47
ES-6.4, Page 12 of 19

1 Form 6.4-2 Simulator Crew Evaluation Form (page 2)

2 Simulator Examination Summary Sheet

3

4 Facility: _______________________ Examination Date: _________________

5

6 Overall Dynamic Simulator Crew Evaluation: SAT or UNSAT

Crew Members Docket No. Scenario #1 Scenario #2

Position Position

1. _____________________ 55-_________ _____________ _____________

2. _____________________ 55-_________ _____________ _____________

3. _____________________ 55-_________ _____________ _____________

4_____________________ 55-_________ _____________ _____________

5. _____________________ 55-_________ _____________ _____________

6. _____________________ 55-_________ _____________ _____________

7

Scenario #1: [Enter scenario descriptor]

Crew Critical Tasks SAT UNSAT

1. [Enter critical task descriptor]

2.

3.

4.

5.

8

Scenario #2:

Crew Critical Tasks SAT UNSAT

1.

2.

3.

4.

5.

9 Comments:

10
ES-6.4, Page 13 of 19

1 Form 6.4-2 Simulator Crew Evaluation Form (page 3)

2 Diagnosis of Events and Conditions Based on Signals or Readings

3

4 Did the crew—

5

6 (a) Recognize off-normal trends and status?

7

3

Recognized status and 

trends quickly and 

accurately.

2

Recognized status and 

trends at the time of, but 

not before, exceeding 

established limits.

1

Did not recognize adverse 

status and trends, even 

after alarms and 

annunciators sounded.

8 (b) Use information and reference material (e.g., prints, books, charts, emergency plan 

9 implementation procedures) to aid in diagnosing and classifying events and conditions?

10

3

Made accurate diagnosis 

by using information and 

reference material 

correctly and in a timely 

manner.

2

Committed minor errors in 

using or interpreting 

information and reference 

material.

1

Failed to use, misused, or 

misinterpreted information 

or reference material that 

resulted in improper 

diagnosis.

11 (c) Correctly diagnose plant conditions based on control room indications?

12

3

Performed timely and 

accurate diagnosis.

2

Committed minor errors or 

had minor difficulties in 

making diagnosis.

1

Made incorrect diagnosis, 

which resulted in incorrect 

manipulation of any safety 

control.

13

14 Grade for diagnosis of events and conditions based on signals and readings: SAT or UNSAT

15

16 Comments:

17
ES-6.4, Page 14 of 19

1 Form 6.4-2 Simulator Crew Evaluation Form (page 4)

2 Understanding of Plant and System Responses

3

4 Did the crew—

5

6 (a) Locate and interpret control room indications correctly and efficiently to ascertain and 

7 verify the status and operation of plant systems?

3

Each crew member 

located and interpreted 

instruments or displays 

accurately and efficiently.

2

Some crew members 

committed minor errors in 

locating or interpreting 

instruments or displays. 

Some crew members 

required assistance.

1

The crew members made 

serious omissions, delays, 

or errors in interpreting 

safety-related parameters.

8 (b) Demonstrate an understanding of the manner in which the plant, systems, and 

9 components operate, including setpoints, interlocks, and automatic actions?

3

Crew members 

demonstrated thorough 

understanding of how 

systems and components 

operate.

2

The crew committed minor 

errors because of 

incomplete knowledge of 

the operation of the 

system or component. 

Some crew members 

required assistance.

1

Inadequate knowledge of 

safety system or 

component operation 

resulted in serious 

mistakes or plant 

degradation.

10 (c) Demonstrate an understanding of how the crew’s actions (or inaction) affected systems 

11 and plant conditions?

12

3

All members understood 

the effect that actions or 

directives had on the plant 

and systems.

2

Actions or directives 

indicated minor 

inaccuracies in individuals’ 

understanding, but the 

crew corrected the actions.

1

The crew appeared to act 

without knowledge of, or 

with disregard for, the 

effects on plant safety.

13 Grade on understanding of the response of plant and systems: SAT or UNSAT

14

15 Comments:

16
ES-6.4, Page 15 of 19

1 Form 6.4-2 Simulator Crew Evaluation Form (page 5)

2 Adherence to and Use of Procedures

3

4 Did the crew—

5

6 (a) Refer to and/or verify the appropriate procedures in a timely manner?

3

The crew used procedures 

as required and knew what 

conditions were covered 

by procedures and where 

to find them.

2

The crew committed minor 

failures to refer to and/or 

verify procedures without 

prompting, which affected 

the plant’s status.

1

The crew failed to correctly 

refer to and/or verify 

procedure(s) when 

required, resulting in faulty 

safety system operation.

7 (b) Correctly implement procedures, including following procedural steps in correct 

8 sequence, abiding by cautions and limitations, selecting correct paths on decision 

9 blocks, and transitioning between procedures when required?

3

The crew followed the 

procedural steps 

accurately and in a timely 

manner, demonstrating a 

thorough understanding of 

the procedural purposes 

and bases.

2

The crew misapplied 

procedures in minor 

instances but made 

corrections in sufficient 

time to avoid adverse 

effects.

1

The crew failed to follow 

procedures correctly, 

which impeded recovery 

from events or caused 

unnecessary degradation 

in the safety of the plant.

10 (c) Recognize abnormal operating procedure (AOP) and emergency operating procedure 

11 (EOP) entry conditions and perform appropriate actions without the aid of references or 

12 other forms of assistance?

3

The crew recognized plant 

conditions and 

implemented AOPs and 

EOPs consistently, 

accurately, and in a timely 

manner.

2

The crew had minor 

lapses or errors. 

Individual crew members 

needed assistance from 

others to implement 

procedures.

1

The crew failed to 

accurately recognize a 

degraded plant 

condition(s) or execute an 

efficient mitigating 

action(s), even with the 

use of aids.

13 Grade on adherence to and use of procedures: SAT or UNSAT

14

15 Comments:

16
ES-6.4, Page 16 of 19

1 Form 6.4-2 Simulator Crew Evaluation Form (page 6)

2 Operate Plant Component Controls

3

4 Did the crew—

5

6 (a) Locate controls efficiently and accurately?

3

Individual operators 

located controls and 

indications without 

hesitation.

2

One or more operators 

hesitated or had difficulty 

in locating controls.

1

The crew failed to locate a 

control(s), which 

jeopardized a system(s) 

important to safety.

7 (b) Manipulate controls in an accurate and timely manner?

3

The crew manipulated 

plant controls smoothly 

and maintained 

parameters within 

specified bounds.

2

The crew demonstrated 

minor shortcomings in 

manipulating controls, but 

recovered from errors 

without causing problems.

1

The crew made mistakes 

manipulating a control(s) 

that caused safety system 

transients and related 

problems.

8 (c) Take manual control of automatic functions, when appropriate?

3

All operators took control 

and smoothly operated 

automatic systems 

manually without 

assistance, thereby 

averting adverse events.

2

Some operators delayed 

or required prompting 

before overriding or 

operating automatic 

functions but avoided plant 

transients where possible.

1

The crew failed to 

manually control automatic 

systems important to 

safety, even when ample 

time and indications 

existed.

9

10 Grade on operation of plant component controls: SAT or UNSAT

11

12 Comments:

13
ES-6.4, Page 17 of 19

1 Form 6.4-2 Simulator Crew Evaluation Form (page 7)

2 Crew Operations

3

4 Did the crew members—

5

6 (a) Maintain a command role?

3

The crew took early 

remedial action when 

necessary.

2

In minor instances, the 

crew failed to take action 

within a reasonable period 

of time.

1

The crew failed to take 

timely action, which 

resulted in the 

deterioration of plant 

conditions.

7

8 (b) Provide timely, well planned directions to each other that facilitated their performance 

9 and demonstrated appropriate concern for the safety of the plant, staff, and public?

3

The supervisor’s directives 

allowed for safe and 

integrated performance by 

all crew members.

2

In minor instances, the 

supervisors gave orders 

that were incorrect, trivial, 

or difficult to implement.

1

The supervisor’s 

directive(s) inhibited safe 

crew performance. Crew 

members had to explain 

why an order(s) could not 

or should not be followed.

10

11 (c) Maintain control during the scenario with an appropriate amount of direction and 

12 guidance from the crew’s supervisors?

3

Crew members stayed 

involved without creating a 

distraction, the crew 

members anticipated each 

other’s needs, and the

supervisors provided 

guidance when necessary.

2

Crew members had to 

solicit assistance from 

supervisors or each other, 

interfering with their ability 

to carry out critical 

action(s).

1

Crew members had to 

repeatedly request 

guidance. The crew failed 

to verify successful 

accomplishment of orders.

13

14

15 Crew Operations Continued on Next Page

16
ES-6.4, Page 18 of 19

1 Form 6.4-2 Simulator Crew Evaluation Form (page 8)

2 Crew Operations (Continued)

3

4 Did the crew members—

5

6 (d) Use a team approach to problem-solving and decisionmaking by soliciting and 

7 incorporating relevant information from all crew members?

3

Crew members were 

involved in the 

problem-solving and 

decisionmaking processes 

for effective team 

decisionmaking.

2

At times, crew members 

failed to get involved in the 

decisionmaking process 

when they should have, 

detracting from the 

team-oriented approach.

1

The crew was not involved 

in making a decision(s). 

The crew was divided over 

the scenario’s progress, 

and this behavior was 

counterproductive.

8

9 Grade on crew operations: SAT or UNSAT

10

11 Comments:

12
ES-6.4, Page 19 of 19

1 Form 6.4-2 Simulator Crew Evaluation Form (page 9)

2 Communications

3

4 Did the crew—

5

6 (a) Exchange complete and relevant information in a clear, accurate, and attentive manner?

3

Crew members provided 

relevant and accurate 

information to each other.

2

Crew communications 

were generally complete 

and accurate, but the crew 

sometimes needed 

prompting or failed to 

acknowledge the 

completion of evolutions or 

to respond to information 

from others.

1

Crew members did not 

inform each other of an 

abnormal indication(s) or 

action(s). Crew members 

were inattentive when 

important information was 

requested.

7 (b) Keep key personnel outside the control room informed of plant status?

3

Crew members provided 

key personnel outside the 

control room with 

accurate, relevant 

information throughout the 

scenarios.

2

In minor instances, the 

crew needed to be 

prompted for information 

and/or provided some 

incomplete or inaccurate 

information.

1

The crew failed to provide 

needed information.

8 (c) Ensure receipt of clear, easily understood communications from the crew and others?

3

The crew requested 

information or clarification 

when necessary and 

understood 

communications from 

others.

2

In minor instances, the 

crew failed to request or 

acknowledge information 

from others.

1

The crew failed to request 

needed information or was 

inattentive when 

information was provided; 

serious misunderstandings 

occurred among crew 

members.

9

10 Grade on communications: SAT or UNSAT

11

12 Comments:

ES-7.1, Page 1 of 26

1 7.1 ADMINISTRATION OF INITIAL EXAMINATIONS FOR SENIOR 

2 OPERATORS LIMITED TO FUEL HANDLING

3 This standard provides specific instructions for use in preparing, administering, grading, and 

4 documenting initial examinations for senior operators limited to fuel handling (i.e., limited senior 

5 reactor operators (SROs) or LSROs).

6 A. Background

7 Pursuant to Title 10 of the Code of Federal Regulations (10 CFR) 55.41, “Written examination: 

8 Operators,” and 10 CFR 55.43, “Written examination: Senior operators,” the U.S. Nuclear 

9 Regulatory Commission’s (NRC’s) written LSRO examinations must contain a representative 

10 selection of questions concerning the specific knowledge and abilities (K/As) and skills needed 

11 to perform licensed fuel handling duties. Similarly, to the extent applicable, the operating tests 

12 must require the applicant to demonstrate an understanding of and the ability to perform the 

13 actions necessary to accomplish a representative sample of the items in 10 CFR 55.45, 

14 “Operating tests.” The regulations also stipulate that the content of the examinations and tests 

15 will be identified, in part, from learning objectives derived from a systematic analysis of the 

16 operators’ duties performed by the facility licensee. Therefore, the facility licensee’s job task 

17 analysis (JTA) for fuel handlers would provide an excellent source of information for developing 

18 the written examination and operating test.

19 Except as noted herein, the applicable instructions and guidance in the following examination 

20 standards for administering unrestricted initial licensing examinations at power reactors also 

21 applies to the LSRO examination:

22 • Examination Standard (ES)-2.1, “Preparing for Operator Licensing Initial Examinations”

23 • ES-2.2, “Applications, Medical Requirements, and Waiver and Excusal of Examination 

24 and Test Requirements”

25 • ES-2.3, “Reviewing and Approving Operator Licensing Initial Examinations”

26 • ES-3.1, “Overview of the Operating Test for Operator Licensing Initial Examinations”

27 • ES-3.2, “Developing Job Performance Measures”

28 • ES-3.5, “Administering Operating Tests”

29 • ES-3.6, “Grading and Documenting Operating Tests for Operator Licensing Initial 

30 Examinations”

31 • ES-4.1, “Developing Written Examination Outlines”

32 • ES-4.2, “Developing Written Examinations”

33 • ES-4.3, “Administering Written Examinations”

34 • ES-4.4, “Grading and Documenting the Written Examinations”

35 • ES-5.1, “Issuing Operator Licenses and Postexamination Activities”
ES-7.1, Page 2 of 26

1 • ES-5.2, “Application Denials and Requests for Informal NRC Staff Review”

2 B. Responsibilities

3 1. Facility Licensee

4 The facility licensee is responsible for the same activities specified in the examination standards 

5 for unrestricted licenses, with the following exceptions and modifications:

6 a. As an exception to ES-2.2, the facility licensee may request LSRO licenses that are valid 

7 for more than one site. To do so, the facility licensee shall document the differences in 

8 the design, procedures, technical data, and administrative controls of the separate 

9 facilities for which the license is being sought.

10 b. The scope, content, administration, and grading of the written examination and operating 

11 test shall be as described in Sections C and D below.

12 c. In accordance with 10 CFR 55.46(b), the facility licensee shall request the Commission’s 

13 approval to use the plant or a simulation facility, other than a plant-referenced simulator, 

14 in administering the operating test under 10 CFR 55.45(b)(1) or (3).

15 2. NRC Regional Office

16 The NRC regional office is responsible for the same activities specified in the unrestricted 

17 examination standards, with the following exceptions and modifications:

18 a. The NRC regional office should generally conduct the LSRO examinations during a time 

19 when the fuel handling equipment will be available for the operating tests.

20 b. With the concurrence of the Office of Nuclear Reactor Regulation (NRR) operator 

21 licensing program office, the NRC regional office may issue LSRO licenses that are valid 

22 for units at more than one site if the units are manufactured by the same vendor and are 

23 of similar design. The applicant must pass an examination that addresses the 

24 differences in the design, procedures, technical data, and administrative controls of the 

25 separate facilities for which the license is being sought.

26 c. The scope, content, administration, and grading of the written examination and operating 

27 test shall be as described in Sections C and D below.

28 d. The NRC regional office shall coordinate with the NRR operator licensing program office 

29 to approve to use the plant or a simulation facility, other than a plant-referenced 

30 simulator, in administering the operating test under 10 CFR 55.45(b)(1) or (3).

31 C. Written Examination Instructions

32 1. Preparation

33 The NRC’s written LSRO examination should meet all the guidelines and requirements for 

34 question construction, quality, and facility reviews specified in ES-4.2, except as noted below:

35 a. Develop the examination outline as described in Section B of ES-4.1, with the following 

36 exceptions and clarifications:
ES-7.1, Page 3 of 26

1 • Use the applicable LSRO outline form (i.e., Form 7.1-BWR, “Written Examination 

2 Outline for Senior Operators Limited to Fuel Handling for Boiling-Water 

3 Reactors”; Form 7.1-PWR, “Written Examination Outline for Senior Operators 

4 Limited to Fuel Handling for Pressurized-Water Reactors”; Form 7.1-ABWR, 

5 “Written Examination Outline for Senior Operators Limited to Fuel Handling for 

6 Advanced Boiling-Water Reactors”; or Form 7.1-AP, “Written Examination 

7 Outline for Senior Operators Limited to Fuel Handling for the AP1000”) and 

8 Form 7.1-GEN, “Generic Knowledge and Abilities Outline (Tier 3) for Senior 

9 Operators Limited to Fuel Handling,” to develop the examination outline. As with 

10 the unrestricted examinations, eliminate topics that are not applicable to LSROs 

11 at the subject facility in accordance with Section B.3 of ES-4.2. Given the large 

12 number of K/A statements that will not apply to LSROs, it may be advantageous 

13 to prescreen the K/As, as discussed in ES-4.1. When reviewing K/As for 

14 elimination, do not focus only on the fuel handling equipment; rather, focus more 

15 broadly on the K/As that an LSRO would need to support safe operation during 

16 fuel handling. If the facility licensee’s JTA identified other LSRO-relevant 

17 components, systems, and evolutions that are not included on the applicable 

18 LSRO outline form, add those items to the appropriate tier of the outline before 

19 beginning the random selection process. Additional instructions are noted on the 

20 forms.

21 • The applicable LSRO outline forms do not have a separate category for sampling 

22 SRO-only questions.

23 • Use Form 7.1-3, “Examination Outline Quality Checklist for Senior Operators 

24 Limited to Fuel Handling,” instead of Form 2.3-1, “Examination Outline Quality 

25 Checklist,” when reviewing the examination outline.

26 b. Select and develop questions as described in ES-4.2, with the following exceptions: 

27 • Construct the LSRO written examination so that a competent applicant can 

28 complete the examination in 2.5 hours. (The applicants will be allowed 4 hours 

29 to complete and review the examination.)

30 • Write between 50 and 60 percent of the LSRO examination questions (20 to 

31 24 questions) at the comprehension/analysis level.

32 • Reactor theory, component, and thermodynamic questions that directly relate to 

33 the LSRO JTA may be selected from prior generic fundamentals examinations.

34 • ES 4.2, Section E, is not applicable to the LSRO examination.

35 • Limit the use of bank questions to no more than 30 and include at least 4 new 

36 questions on every examination; the remaining 6 examination questions may be 

37 new or significantly modified from the facility licensee’s or any other bank. All 

38 questions developed must be relevant to the LSRO function. To be considered a 

39 significantly modified question, at least one pertinent condition in the stem and at 

40 least one distractor must be changed from the original bank question. Changing 

41 the conditions in the stem such that one of the three distractors in the original 

42 question becomes the correct answer would also be considered a significant 

43 modification. 
ES-7.1, Page 4 of 26

1 • If the examination will be used to license the applicants at more than one facility, 

2 ensure that it adequately covers all of the applicable units. An examination 

3 developed for the purpose of cross-qualifying a licensed LSRO at another similar 

4 facility may focus exclusively on the differences between the facilities.

5 c. Review and assemble the examination as described in Form 7.1-4, “Written Examination 

6 Quality Checklist for Senior Operators Limited to Fuel Handling,” and Form 7.1-6, 

7 “Written Examination Cover Sheet for Senior Operators Limited to Fuel Handling,” 

8 instead of the equivalent forms in ES-2.3 and ES-4.2.

9 2. Administration and Grading

10 The NRC’s written LSRO examination shall be administered and graded in accordance with

11 applicable instructions in ES-4.3 and ES-4.4. The examination may be administered 

12 concurrently and in the same room with full-scope, initial license examinations. However, in 

13 such instances, the proctor should minimize any disturbance to those applicants taking the 

14 longer examination.

15 D. Operating Test Instructions

16 The LSRO operating test shall generally be prepared, administered, and documented in 

17 accordance with the applicable instructions in ES-3.1, ES-3.2, ES-3.5, and ES-3.6, except as 

18 noted below and in the specific criteria at the bottom of Form 7.1-2, “Operating Test Outline for 

19 Senior Operators Limited to Fuel Handling.”

20 The operating test shall be performance based to the maximum extent possible; however, given 

21 the nature of an LSRO’s duties, it is neither practical nor appropriate to administer the test on 

22 the plant-referenced simulator. Therefore, pursuant to 10 CFR 55.45(b), the test shall be 

23 administered in a plant walkthrough and in either the plant or a simulation facility, as approved 

24 by the Commission under 10 CFR 55.46(b). The facility licensee is encouraged to permit the 

25 actual use of equipment to handle dummy fuel elements, assemblies, or modules during the 

26 operating test whenever feasible. This may require careful coordination with the facility licensee 

27 to establish a schedule and to ensure that a licensed SRO is available, if needed. When actual 

28 equipment is not available or accessible (e.g., because of high radiation), the test should be 

29 administered using walkthrough methods near the actual equipment or by using mockup 

30 equipment. If the facility licensee has a refueling machine simulator, it should be used to the 

31 extent possible during the administration of the operating test.

32 The operating test shall assess the applicant’s ability to execute normal, abnormal, and 

33 emergency procedures associated with fuel handling. Each applicant will be required to 

34 simulate or perform tasks related to fuel handling and, if necessary based on his or her 

35 performance, to answer questions associated with the refueling equipment and associated 

36 systems. The applicant shall not be held accountable for duties that are performed exclusively 

37 by the control room staff or shift supervisor.

38 1. Preparation

39 The operating test shall consist entirely of job performance measures (JPMs) covering those 

40 administrative topics, systems, and emergency/abnormal plant evolutions (E/APEs) related to 

41 refueling. No distinction between control room and facility systems/evolutions is required 

42 because most (if not all) of the test will be conducted outside the control room. The dynamic 

43 simulator operating test requirements, instructions, and guidelines in ES-3.1, ES-3.3, “Testing 
ES-7.1, Page 5 of 26

1 Guidelines for Dynamic Simulator Scenarios,” and ES-3.4, “Developing Scenarios,” do not apply 

2 to the LSRO license examination.

3 Part of the operating test may be conducted in the control room so that those controls, 

4 instruments, and other materials or equipment related to fuel handling (e.g., procedures and 

5 diagrams) are available for reference. Although LSROs will not operate any systems from the 

6 control room, they must be aware of the effects (e.g., alarms) that fuel handling operations will 

7 have in the control room. They must also be familiar with the methods and requirements for 

8 communicating with the control room staff and shift supervisor. At least two of the JPMs must 

9 require the applicant to use the facility’s technical specifications.

10 The following additional guidelines clarify the expectations for each part of the LSRO operating 

11 test:

12 a. Develop the administrative portion of the operating test in accordance with the 

13 instructions for administrative topic JPMs in ES-3.2; however, given the reduced scope 

14 of the LSRO’s responsibilities, the required number of tasks is reduced from five to 

15 three, distributed among the four administrative topics. Note that some “Conduct of 

16 Operations” subjects (e.g., reactor plant startup requirements) may not apply; however, 

17 most can be adapted for use during the LSRO operating test. The “Equipment Control” 

18 subjects all lend themselves to evaluating the required refueling maintenance and 

19 surveillance actions that the LSRO should be able to supervise or perform. All of the 

20 “Radiation Control” subjects apply to refueling operations and should be evaluated on a 

21 sampling basis. The “Emergency Plan” topic shall be evaluated to the extent that the 

22 applicant is required to respond to a declared event and the knowledge required of a 

23 radiation worker.

24 b. Develop the systems portion of the operating test as follows:

25 • Develop two JPMs that require the applicant to manipulate the facility’s fuel

26 handling equipment.

27 • Develop two JPMs related to systems other than fuel handling equipment 

28 (i.e., systems other than System No. 234000 (boiling-water reactor (BWR)) or 

29 System No. 034 (pressurized-water reactor (PWR)) listed in Tier 2 of the 

30 appropriate LSRO written examination outline (i.e., Form 7.1-BWR/PWR/ABWR/

31 AP).

32 • Include two JPMs that require the applicant to execute alternative paths within 

33 the facility’s operating procedures.

34 • Follow the instructions in ES-3.2, Sections D and E, for developing JPMs. (Note 

35 that the specific instructions for selecting safety functions and systems on 

36 Form 3.2-2, “Control Room/In-Plant Systems Outline,” in ES-3.2 do not apply.)

37 c. Develop the E/APE portion of the operating test as follows:

38 • Develop three JPMs based on the evolutions listed in Tier 1 of the appropriate 

39 LSRO written examination outline (i.e., Form 7.1-BWR/PWR/ABWR/AP as 

40 modified in Section C.1.a above); one of the JPMs must involve a refueling 

41 accident.
ES-7.1, Page 6 of 26

1 • Include one JPMs that requires the applicant to execute alternative paths within 

2 the facility’s operating procedures.

3 d. The operating test should normally take between 4 and 6 hours, depending on whether 

4 the LSRO actually operates refueling equipment.

5 e. Use Form 7.1-2 to document the selection of administrative, system, and E/APE JPMs to 

6 be performed; insert the applicable type codes and adhere to the specific criteria noted 

7 at the bottom of the form. Review the outline using Form 7.1-3, “Examination Outline 

8 Quality Checklist for Senior Operators Limited to Fuel Handling.”

9 f. Review the final operating test in using Form 7.1-5, “Operating Test Quality Checklist for 

10 Senior Operators Limited to Fuel Handling.” 

11 2. Administration

12 The operating test should be administered in accordance with the applicable sections in ES-3.5.

13 3. Grading

14 The applicant’s performance on the operating test should be graded and documented in 

15 accordance with ES-3.6, “Grading and Documenting Operating Tests for Operator Licensing 

16 Initial Examinations,” as applicable, with the following clarifications:

17 a. Use Form 7.1-3 to document a grade for each JPM. On Form 5.1-2, “Individual 

18 Examination Report,” write “N/A” for “Simulator Operating Test” under “Operating Test 

19 Summary.”

20 b. The applicant must achieve a satisfactory grade on at least 80 percent of the JPMs (8 

21 out of 10) overall and at least 60 percent (2 out of 3) of the administrative topic JPMs. 

22 E. Forms

23 Form 7.1-BWR Written Examination Outline for Senior Operators Limited to Fuel 

24 Handling for Boiling-Water Reactors

25 Form 7.1-PWR Written Examination Outline for Senior Operators Limited to Fuel 

26 Handling for Pressurized-Water Reactors

27 Form 7.1-ABWR Written Examination Outline for Senior Operators Limited to Fuel 

28 Handling for Advanced Boiling-Water Reactors

29 Form 7.1-AP Written Examination Outline for Senior Operators Limited to Fuel 

30 Handling for the AP1000

31 Form 7.1-GEN Generic Knowledge and Abilities Outline (Tier 3) for Senior Operators 

32 Limited to Fuel Handling

33 Form 7.1-2 Operating Test Outline for Senior Operators Limited to Fuel Handling

34 Form 7.1-3 Examination Outline Quality Checklist for Senior Operators Limited to 

35 Fuel Handling

36 Form 7.1-4 Written Examination Quality Checklist for Senior Operators Limited to 

37 Fuel Handling

38 Form 7.1-5 Operating Test Quality Checklist for Senior Operators Limited to Fuel 

39 Handling

40 Form 7.1-6 Written Examination Cover Sheet for Senior Operators Limited to Fuel 

41 Handling
ES-7.1, Page 7 of 26

1 Form 7.1-BWR Written Examination Outline for Senior Operators Limited to Fuel 

2 Handling for Boiling-Water Reactors

3

Facility: Date of Exam:

Tier

K/A Category Points

K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G Total

1. Emergency and 

Abnormal Plant 

Evolutions

10

2. Plant

Systems 20

3. Generic 

Knowledge and 

Abilities Categories

1 2 3 4 Fundamentals

10

Notes:

1. Ensure that at least one topic from every knowledge and ability (K/A) category is sampled 

within each tier.

2. The point total for each tier in the proposed outline must match that specified in the table. 

The final point total for each tier may deviate by ±1 from that specified in the table. The 

final exam must total 40 points.

3. Select topics from many systems and evolutions; avoid selecting more than two K/A topics 

from a given system (except fuel handling equipment) or evolution (except refueling 

accident).

4. The shaded areas are not applicable to the category/tier.

5. The generic (G) K/As in Tiers 1 and 2 shall be selected from Section 2 of the applicable K/A 

catalog, but the topics must be relevant to the applicable evolution/system. 

6. Systems/evolutions within each tier are identified on the associated outline. Enter the K/A 

numbers, a brief description of each topic, importance ratings (IRs) for the senior reactor 

operator license level, and the point totals (#) for each system and category. Enter the tier 

totals for each category in the table above.

7. For Tier 3, select topics from Sections 2, 5, and 6 of the applicable K/A catalog. A 

minimum of four Tier 3 questions shall include basic reactor theory, component, and 

thermodynamic topics from Sections 5 and 6 of the applicable K/A catalog that apply to fuel

handling operations (place these items in the “Fundamental” category). Enter the K/A 

numbers, descriptions, IRs, and point totals (#) on Form 7.1-GEN.

8. Inapplicable or inappropriate K/A statements can be eliminated by (1) discarding randomly 

selected K/As during the outline development process or (2) prescreening the entire K/A 

catalog to eliminate inappropriate K/As before beginning the random selection process. 

Use the facility licensee’s job task analysis for fuel handlers as the basis for eliminating or 

adding testable topics.
ES-7.1, Page 8 of 26

ES-7.1 Form 7.1-BWR

Written Examination Outline for Senior Operators Limited to Fuel Handling for Boiling-Water Reactors

Emergency and Abnormal Plant Evolutions—Tier 1

E/APE #/Name/Safety Function K

1

K

2

K

3

A

1

A

2

G K/A Topic(s) IR #

295003 Partial or Complete Loss of AC

295004 Partial or Total Loss of DC

295014 Inadvertent Reactivity Addition

295018 Partial or Total Loss of CCW

295021 Loss of Shutdown Cooling

295023 Refueling Accidents

295033 High Secondary Containment 

Area Radiation Levels

295034 Secondary Containment 

Ventilation High Radiation

295006 Scram

295008 High Reactor Water Level

295009/295031 Reactor Low Water 

Level

295017/295038 High Offsite Release 

Rate

295019 Partial or Total Loss of 

Instrument Air

295020 Inadvertent Containment 

Isolation

295030 Low Suppression Pool Water 

Level

295035 Secondary Containment High 

Differential Pressure

600000 Plant Fire On Site

K/A Category Totals: Tier Point Total: 10

1

2
ES-7.1, Page 9 of 26

ES-7.1 Form 7.1-BWR

Written Examination Outline for Senior Operators Limited to Fuel Handling for Boiling Water Reactors

Plant Systems—Tier 2

System #/Name K

1

K

2

K

3

K

4

K

5

K

6

A

1

A

2

A

3

A

4

G K/A Topic(s) IR #

205000 Shutdown Cooling

215004 Source Range Monitor

233000 Fuel Pool 

Cooling/Cleanup

234000 Fuel Handling 

Equipment

262001 AC Electrical Distribution

263000 DC Electrical Distribution

290002 Reactor Vessel Internals

201002 RMCS

201003 Control Rod and Drive 

Mechanism

203000 RHR/LPCI: Injection 

Mode

204000 RWCU

211000 SLC

212000 RPS

214000 RPIS

215001 Traversing In-Core 

Probe

215003 IRM

215005 APRM/LPRM

223001 Primary CTMT and Aux.

223002 PCIS/Nuclear Steam 

Supply Shutoff

261000 SGTS

264000 EDGs

272000 Radiation Monitoring

286000 Fire Protection

288000 Plant Ventilation

290001 Secondary CTMT

300000 Instrument Air

400000 Component Cooling 

Water

K/A Category Totals: Tier Point Total: 20
ES-7.1, Page 10 of 26

1 Form 7.1-PWR Written Examination Outline for Senior Operators Limited to Fuel Handling 

2 for Pressurized-Water Reactors

3

Facility: Date of Exam:

Tier

K/A Category Points

K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G* Total

1. Emergency and 

Abnormal Plant 

Evolutions

10

2. Plant

Systems

20

3. Generic 

Knowledge and 

Abilities Categories

1 2 3 4 Fundamentals

10

Notes:

1. Ensure that at least one topic from every knowledge and ability (K/A) category is sampled 

within each tier.

2. The point total for each tier in the proposed outline must match that specified in the table. 

The final point total for each tier may deviate by ±1 from that specified in the table. The 

final exam must total 40 points.

3. Select topics from many systems and evolutions; avoid selecting more than two K/A topics 

from a given system (except fuel handling equipment) or evolution (except refueling 

accident).

4. The shaded areas are not applicable to the category/tier.

5. Select the generic (G*) K/As in Tiers 1 and 2 from Section 2 of the applicable K/A catalog, 

but the topics must be relevant to the applicable evolution/system. 

6. Systems/evolutions within each tier are identified on the associated outline. Enter the K/A 

numbers, a brief description of each topic, importance ratings (IRs) for the senior reactor 

operator license level, and the point totals (#) for each system and category. Enter the tier 

totals for each category in the table above.

7. For Tier 3, select topics from Sections 2, 5, and 6 of the applicable K/A catalog. A 

minimum of four Tier 3 questions shall include basic reactor theory, component, and 

thermodynamic topics from Sections 5 and 6 of the applicable K/A catalog that apply to fuel

handling operations (place these items in the “Fundamental” category). Enter the K/A 

numbers, descriptions, IRs, and point totals (#) on Form 7.1-GEN.

8. Eliminate inapplicable or inappropriate K/A statements by (1) discarding randomly selected 

K/As during the outline development process or (2) prescreening the entire K/A catalog to 

eliminate inappropriate K/As before beginning the random selection process. Use the 

facility licensee’s job task analysis for fuel handlers as the basis for eliminating or adding 

testable topics.
ES-7.1, Page 11 of 26

1

ES-7.1 Form 7.1-PWR

Written Examination Outline for Senior Operators Limited to Fuel Handling for Pressurized-Water Reactors

Emergency and Abnormal Plant Evolutions—Tier 1

E/APE #/Name/Safety Function K

1

K

2

K

3

A

1

A

2

G K/A Topic(s) IR #

000025 Loss of RHR System

000026 Loss of Component 

Cooling Water

000032 Loss of Source-Range 

Nuclear Instrumentation

000036 (BW/A08) Fuel Handling 

Accident

000061 ARM System Alarms

000033 Loss of 

Intermediate-Range Nuclear 

Instrumentation

000055 Station Blackout

000056 Loss of Offsite Power

000057 Loss of Vital AC Instrument 

Bus

000058 Loss of DC Power

000062 Loss of Nuclear Service 

Water

000065 Loss of Instrument Air

000067 Plant Fire on Site

000069 (W/E14) Loss of CTMT 

Integrity

W/E16 High Containment Radiation

K/A Category Totals: Tier Point Total: 10
ES-7.1, Page 12 of 26

1

2

ES-7.1 Form 7.1-PWR

Written Examination Outline for Senior Operators Limited to Fuel Handling for Pressurized-Water Reactors

Plant Systems—Tier 2

System #/Name K

1

K

2

K

3

K

4

K

5

K

6

A

1

A

2

A

3

A

4

G K/A Topic(s) IR #

005 Residual Heat Removal

015 Nuclear Instrumentation

033 Spent Fuel Pool Cooling

034 Fuel Handling Equipment

103 Containment 

062 AC Electrical Distribution

063 DC Electrical Distribution

002 Reactor Coolant

004 Chemical and Volume 

Control 

008 Component Cooling 

Water

013 Engineered Safety 

Features Actuation

064 Emergency Diesel 

Generator

072 Area Radiation Monitoring

076 Service Water

078 Instrument Air

079 Station Air 

086 Fire Protection

K/A Category Totals: Tier Point Total: 20
ES-7.1, Page 13 of 26

1 Form 7.1-ABWR Written Examination Outline for Senior Operators Limited to Fuel 

2 Handling for Advanced Boiling-Water Reactors

3

Facility: Date of Exam:

Tier

K/A Category Points

K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G Total

1. Emergency and 

Abnormal Plant 

Evolutions

10

2. Plant

Systems

20

3. Generic 

Knowledge and 

Abilities Categories

1 2 3 4 Fundamentals

10

Notes:

1. Ensure that at least one topic from every knowledge and ability (K/A) category is sampled 

within each tier.

2. The point total for each tier in the proposed outline must match that specified in the table. 

The final point total for each tier may deviate by ±1 from that specified in the table. The 

final exam must total 40 points.

3. Select topics from many systems and evolutions; avoid selecting more than two K/A topics 

from a given system (except fuel handling equipment) or evolution (except refueling 

accident).

4. The shaded areas are not applicable to the category/tier.

5. Select the generic (G) K/As in Tiers 1 and 2 from Section 2 of the applicable K/A catalog, 

but the topics must be relevant to the applicable evolution/system. 

6. Systems/evolutions within each tier are identified on the associated outline. Enter the K/A 

numbers, a brief description of each topic, importance ratings (IRs) for the senior reactor 

operator license level, and the point totals (#) for each system and category. Enter the tier 

totals for each category in the table above.

7. For Tier 3, select topics from Sections 2, 5, and 6 of the applicable K/A catalog. A 

minimum of four Tier 3 questions shall include basic reactor theory, component, and 

thermodynamic topics from Sections 5 and 6 of the applicable K/A catalog that apply to fuel

handling operations (place these items in the “Fundamental” category). Enter the K/A 

numbers, descriptions, IRs, and point totals (#) on Form 7.1-GEN.

8. Eliminate inapplicable or inappropriate K/A statements by (1) discarding randomly selected 

K/As during the outline development process or (2) prescreening the entire K/A catalog to 

eliminate inappropriate K/As before beginning the random selection process. Use the 

facility licensee’s job task analysis for fuel handlers as the basis for eliminating or adding 

testable topics.
ES-7.1, Page 14 of 26

ES-7.1 Form 7.1-ABWR

Written Examination Outline for Senior Operators Limited to Fuel Handling for Advanced Boiling-Water Reactors

Emergency and Abnormal Plant Evolutions—Tier 1

E/APE #/Name/Safety Function K

1

K

2

K

3

A

1

A

2 G

K/A Topic(s) IR #

APE2003 Partial or Complete Loss of AC / 

6

APE2004 Partial or Total Loss of DC 

Power / 6

APE2006 Reactor Scram

APE2008 High Reactor Water Level

APE2013 Inadvertent Reactivity Addition / 

1

APE2009 Low Reactor Water Level / 2

APE2016High Offsite Release Rate / 9

APE2017 Partial or Total Loss of CCW 

Reactor Building Cooling Water / 8

APE2018 Partial or Total Loss of 

Instrument Air / 8

APE2019 Inadvertent Containment 

Isolation / 5 & 7

APE2020 Loss of Shutdown Cooling / 4

APE2022 Refueling Accidents / 8

APE2023 Plant Fire On Site / 8

EPE1006 Low Suppression Pool Water 

Level / 5

EPE1009 High Secondary Containment 

Area Radiation Levels / 9

EPE1010 Reactor Building HVAC Exhaust 

High Radiation / 9

EPE1011 Secondary Containment High 

Differential Pressure / 5

K/A Category Totals: Tier Point Total: 10

1

2
ES-7.1, Page 15 of 26

ES-7.1 Form 7.1-ABWR

Written Examination Outline for Senior Operators Limited to Fuel Handling for Advanced Boiling-Water Reactors

Plant Systems—Tier 2

System #/Name K

1

K

2

K

3

K

4

K

5

K

6

A

1

A

2

A

3

A

4

G K/A Topic(s) IR #

SF2RHRLPFL RHR: 

Low-Pressure Flooder Mode

SF4RHRSDC RHR: Shutdown 

Cooling Mode

SF5LDIS Leak Detection and 

Isolation System

SF6EPDS AC Electrical 

Distribution

SF6VAC Vital AC Power Supply

SF6DC Direct Current Power 

Supply

SF6DGCTG Emergency 

Generators (Diesel/CTG)

SF7SRNM Startup Range Neutron 

Monitor 

SF7ELCS ESF Logic and Control 

System

SF7APRM Average Power Range 

Monitor/Local Power Range 

Monitor

SF9SGTS Standby Gas Treatment 

System

SF8IAS Instrument Air

SF8RBCW Reactor Building 

Cooling Water

SF8RSW Reactor Service Water

SF2RWCU Reactor Water 

Cleanup

SF5SEC Secondary Containment

SF5PCS Primary Containment and 

Auxiliary

SF5RPV & SF9RPV Reactor 

Vessel Internals

SF6I&C Instrumentation and 

Control Power Supply

SF9HVAC Plant Ventilation 

Systems

SF7RMS & SF9RMS Radiation 

Monitoring

SF8FPS Fire Protection

SF8FH Fuel Handling

1
ES-7.1, Page 16 of 26

ES-7.1 Form 7.1-ABWR

Written Examination Outline for Senior Operators Limited to Fuel Handling for Advanced Boiling-Water Reactors

Plant Systems—Tier 2

System#/Name K

1

K

2

K

3

K

4

K

5

K

6

A

1

A

2

A

3

A

4

G K/A Topic(s) IR #

SF9FPC Fuel Pool 

Cooling/Cleanup

SF9RD Radwaste

SF1 Standby Liquid Control

SF1 Control Rod Drive

SF1 Fine Motion Control Rod 

Drive Mechanism

SF1 Rod Control and Information 

System

SF7 Automated Traversing 

In-Core Probe

K/A Category Totals: Tier Point Total: 20

1

2
ES-7.1, Page 17 of 26

1 Form 7.1-AP Written Examination Outline for Senior Operators Limited to Fuel Handling 

2 for the AP1000

3

Facility: Date of Exam:

Tier

K/A Category Points

K1 K2 K3 K4 K5 K6 A1 A2 A3 A4 G* Total

1. Emergency and 

Abnormal Plant 

Evolutions

10

2. Plant

Systems

20

3. Generic 

Knowledge and 

Abilities Categories

1 2 3 4 Fundamentals

10

Notes:

1. Ensure that at least one topic from every knowledge and ability (K/A) category is sampled 

within each tier.

2. The point total for each tier in the proposed outline must match that specified in the table. 

The final point total for each tier may deviate by ±1 from that specified in the table. The 

final exam must total 40 points.

3. Select topics from many systems and evolutions; avoid selecting more than two K/A topics 

from a given system (except fuel handling equipment) or evolution (except refueling 

accident).

4. The shaded areas are not applicable to the category/tier.

5. Select the generic (G*) K/As in Tiers 1 and 2 from Section 2 of the applicable K/A catalog, 

but the topics must be relevant to the applicable evolution/system. 

6. Systems/evolutions within each tier are identified on the associated outline. Enter the K/A 

numbers, a brief description of each topic, importance ratings (IRs) for the senior reactor 

operator license level, and the point totals (#) for each system and category. Enter the tier 

totals for each category in the table above.

7. For Tier 3, select topics from Sections 2, 5, and 6 of the applicable K/A catalog. A 

minimum of four Tier 3 questions shall include basic reactor theory, component, and 

thermodynamic topics from Sections 5 and 6 of the applicable K/A catalog that apply to fuel

handling operations (place these items in the “Fundamental” category). Enter the K/A 

numbers, descriptions, IRs, and point totals (#) on Form 7.1-GEN.

8. Eliminate inapplicable or inappropriate K/A statements by (1) discarding randomly selected 

K/As during the outline development process or (2) prescreening the entire K/A catalog to 

eliminate inappropriate K/As before beginning the random selection process. Use the 

facility licensee’s job task analysis for fuel handlers as the basis for eliminating or adding 

testable topics.
ES-7.1, Page 18 of 26

ES-7.1 Form 7.1-AP

Written Examination Outline for Senior Operators Limited to Fuel Handling for the AP1000

Emergency and Abnormal Plant Evolutions—Tier 1

E/APE #/Name/Safety Function K

1

K

2

K

3

A

1

A

2 G K/A Topic(s) IR #

A-337, Passive RHR Heat Exchanger Leak / 

4

A-343, Loss of Normal Residual Heat

Removal / 4

A-317, Loss of Component Cooling Water / 8

FR-S.1, Response to Nuclear Power 

Generation / 1

A-323, Loss of 6.9-kV, 4,160-V, or 480-V 

Bus Power / 6

A-345, Loss of Nuclear Service Water / 4

A-329, Loss of Instrument Air / 8

ECA-1.1, Loss-of-Coolant Accident Outside 

Containment / 3

FR-H.1, Response to Loss of Heat Sink / 4

SDP-1, Response to Loss of RCS Inventory 

during Shutdown / 2

SDP-2, Response to Loss of RNS during 

Shutdown / 4

A-308, Loss of Control Room AC / 8

A-320, Loss of Circulating Water / 8

A-302, Emergency Boration / 1

A-314, Fuel Handling Incident / 8

SDP-4, Response to Rising Nuclear Flux 

during Shutdown / 1

SDP-5, Response to RCS Cold 

Overpressure during Shutdown / 3

SDP-6, Response to Unexpected RCS 

Temperature Changes during Shutdown / 4

A-306, Evacuation of Control Room / 8

FR-Z.2, Response to Containment Flooding / 

5

FR-Z-3, Response to High Containment 

Radiation / 9 

A-321, Loss of Data Display and Processing 

System / 7

A-340, Reactor Coolant Leak / 2

K/A Category Totals: Tier Point Total: 10

1
ES-7.1, Page 19 of 26

ES-7.1 Form 7.1-AP

Written Examination Outline for Senior Operators Limited to Fuel Handling for the AP1000

Plant Systems—Tier 2

System Name/Safety Function K

1

K

2

K

3

K

4

K

5

K

6

A

1

A

2

A

3

A

4 G K/A Topic(s) IR #

Reactor Coolant / 2, 4 

Normal Residual Heat Removal / 4

Passive Residual Heat Removal /4

Passive Core Cooling / 2

Component Cooling Water / 8

Engineered Safeguards Actuation / 2

Passive Containment Cooling / 5

AC Electrical Distribution / 6

Class 1E and Non-Class 1E DC and 

UPS / 6

Onsite Standby Power System / 6

Service Water / 4

Compressed Air / 8

Containment System / 5

Containment Air Filtration / 8

Containment Hydrogen Control / 5

Main Control Room HVAC / 8

Spent Fuel Pool Cooling / 8

Fuel Handling / 8

Gaseous Radwaste / 9

Radiation Monitoring / 7

Circulating Water / 8

Fire Protection / 8

K/A Category Point Totals: Tier Point Total: 20

ES-7.1, Page 21 of 26

Form 7.1-GEN Generic Knowledge and Abilities Outline (Tier 3) for Senior Operators 

Limited to Fuel Handling

ES-7.1 Form 7.1-GEN

Generic Knowledge and Abilities Outline (Tier 3) for Senior Operators Limited to Fuel 

Handling

Facility: Date of Exam:

Category K/A # Topic IR #

1. Conduct of 

Operations

2.1.

2.1.

2.1.

2.1.

Subtotal

2. Equipment 

Control

2.2.

2.2.

2.2.

2.2.

Subtotal

3. Radiation 

Control

2.3.

2.3.

2.3.

2.3.

Subtotal

4. Emergency 

Procedures/ 

Emergency 

Plan

2.4.

2.4

2.4.

2.4.

Subtotal

5.

Fundamentals

Subtotal 4

Tier 3 Point Total 10
ES-7.1, Page 22 of 26

Form 7.1-2 Operating Test Outline for Senior Operators Limited to Fuel Handling

Applicant Docket Number: 55-

Facility: Date of Examination:

Title/Description of Tasks (Job Performance 

Measures)

Type

Codes*

Evaluation

(S or U)

Comment Page

Number

Administrative

1.

2. 

3.

Systems

1.

2.

3.

4.

Emergency/Abnormal Plant Evolutions

1.

2.

3. 

Type Codes & Criteria: (A)lternative path (2 systems; 1 emergency/abnormal plant 

evolution))

(C)ontrol room

(D)irect from bank (≤ 7)

(I)n-plant

(N)ew or (M)odified from bank, including 1(A) (≥ 1/section) 

(L)ast NRC exam (≤ 1/section)

(R)efueling accident (1)

(T)echnical specification (≥ 2)

(S)atisfactory or (U)nsatisfactory
ES-7.1, Page 23 of 26

Form 7.1-3 Examination Outline Quality Checklist for Senior Operators Limited to Fuel 

Handling

Facility: Date of Examination: 

Item Task Description (Y)es / (N)o

a b* c#

1.

W

R

I

T

T

E

N

a. Verify that the outline fits the model in accordance with ES-7.1.

b. Assess whether the outline was systematically and randomly prepared in accordance with 

Section B of ES-4.1 and whether all knowledge and ability (K/A) categories are sampled at 

least once.

c. Assess whether the outline overemphasizes any systems, evolutions, or generic topics.

d. Assess whether the justifications for deselected or rejected K/A statements are appropriate.

2.

O

P

E

R

A

T

I

N

G

a. Verify that the overall operating test—

(1) 13Bincludes at least two tasks that require the use of technical specifications and

(2) does not duplicate any tasks from the applicants’ audit test(s).

b. Verify that the administrative tasks—

(1) are distributed among the four administrative topics described in ES-3.2,

(2) include no more than one repeat from the last NRC licensing examination, and

(3) include at least one task that is new or significantly modified.

c. Verify that the systems walkthrough includes—

(1) two tasks requiring the manipulation of fuel handling equipment,

(2) two additional tasks related to Tier 2 systems other than fuel handling equipment,

(3) two tasks requiring implementation of alternative-path procedures,

(4) no more than one repeat from the last NRC licensing examination, and

(5) at least one task that is new or significantly modified.

d. Verify that the emergency/abnormal plant evolution walkthrough includes—

(1) three job performance measures based on the Tier 1 evolutions, including a refueling 

accident;

(2) one task requiring implementation of an alternative-path procedure;

(3) no more than one repeat from the last NRC licensing examination; and

(4) at least one task that is new or significantly modified.

e. Determine whether there are enough different outlines to test the projected number of 

applicants and ensure that no items are duplicated on subsequent days.

3.

G

E

N

E

R

A

L

a. Assess whether plant-specific priorities (including probabilistic risk assessment and 

individual plant examination insights) are covered in the appropriate exam section.

b. Assess whether the 10 CFR 55.41, 10 CFR 55.43, and 10 CFR 55.45 sampling is 

appropriate.

c. Assess whether the sampling process adequately considered plant-specific refueling 

components, systems, and procedures that are not included in the generic models.

d. Ensure that K/A importance ratings (except for plant-specific priorities) are at least 2.5.

e. Check for duplication and overlap among exam sections.

f. Check the entire exam for balance of coverage.

g. Assess whether the proposed sample is consistent with the job responsibilities for a senior 

operator limited to fuel handling.

Printed Name/Signature Date

a. Author __________________________________________________________ _______

b. Facility Reviewer (*) __________________________________________________________ _______

c. NRC Chief Examiner (#)_________________________________________________________ _______

d. NRC Supervisor __________________________________________________________ _______

* The facility licensee signature is not applicable for NRC-developed tests.

# An independent NRC reviewer performs the steps in column c. This may be the NRC chief examiner if he/she 

did not develop the outline under review.
ES-7.1, Page 24 of 26

Form 7.1-4 Written Examination Quality Checklist for Senior Operators Limited to Fuel 

Handling

Facility: Date of Exam:

Item Description

 (Y)es / (N)o

a b* c#

1. Questions and answers are technically accurate and applicable to the facility.

2. NRC knowledge and ability (K/A) statements are referenced for all questions (as 

applicable). Facility learning objectives are referenced as available.

3. Questions are appropriate for applicants for senior operators limited to fuel handling.

4. The sampling process was random and systematic. (If more than three questions were

repeated from the last two NRC licensing exams, consult the NRC Office of Nuclear Reactor 

Regulation operator licensing program office.)

5. Question duplication from the license screening/audit exam was controlled as indicated 

below (check the item that applies) and appears appropriate:

__ the audit exam was systematically and randomly developed, or

__ the audit exam was completed before the license exam was started, or

__ the examinations were developed independently, or

__ the licensee certifies that there is no duplication, or

__ other (explain).

6. Bank use meets limits (no more than 30 questions

from the bank, at least 4 new, and the rest modified); 

enter the actual question distribution at right.

Bank Modified New

7. Between 50 and 60 percent (20 and 24) of the 

questions on the exam are written at the 

comprehension/analysis (C/A) level; enter the actual 

question distribution at right.

Memory C/A

8. References/handouts provided do not give away answers or aid in eliminating distractors.

9. Question content conforms to specific K/A statements in the previously approved

examination outline and is appropriate for the tier to which the questions are assigned; 

deviations are justified.

10. Question psychometric quality and format meet the guidelines in Appendix B to 

NUREG-1021.

11. The exam contains 40 one-point, multiple-choice items; the total is correct and agrees 

with the value on the cover sheet.

Printed Name/Signature Date

a. Author ________________________________________________________ _______

b. Facility Reviewer (*) ________________________________________________________ _______

c. NRC Chief Examiner (#) ________________________________________________________ _______

d. NRC Supervisor ________________________________________________________ _______

* The facility licensee signature is not applicable for NRC-developed tests.

# An independent NRC reviewer performs the steps in column c. This may be the NRC chief examiner if he/she 

did not develop the outline under review.
ES-7.1, Page 25 of 26

Form 7.1-5 Operating Test Quality Checklist for Senior Operators Limited to Fuel 

Handling

Facility: Date of Examination: Operating Test Number:

Item Description (Y)es / (N)o

a b* c#

1. The operating test conforms to the job responsibilities of a senior operator limited to fuel 

handling and the previously approved outline.

2. Any changes from the previously approved outline have not caused the test to deviate from 

any of the acceptance criteria (e.g., item distribution, bank use, repetition from the last two 

NRC examinations) specified on the outline.

3. There is no day-to-day repetition between this and other operating tests to be administered 

during this examination.

4. The operating test does not duplicate items from the applicants’ audit test(s).

5. Overlap between the written examination and the operating test is within acceptable limits.

6. It appears that the operating test will differentiate between competent and 

less-than-competent applicants.

7. Each job performance measure includes the following, as applicable:

• 0Btask standard

• 1Binitial conditions

• 2Binitiating cues

• 3Breferences and tools, including associated procedures

• 4Breasonable and validated time limits (average time allowed for completion) and specific 

designation if deemed to be time critical by the facility licensee

• 5Bspecific designation if meets criteria of alternate path

• 6Boperationally important specific performance criteria that include the following:

– 7Bdetailed expected actions with exact criteria and nomenclature

– 8Bsystem response and other examiner cues

– 9Bstatements describing important observations to be made by the applicant

– 10Bcriteria for successful completion of the job performance measure task standard

– 11Bidentification of critical steps and their associated performance standards

– 12Brestrictions on the sequence of steps, if applicable

Printed Name/Signature Date

a. Author __________________________________________________________ _______

b. Facility Reviewer (*) __________________________________________________________ _______

c. NRC Chief Examiner (#) ________________________________________________________ _______

d. NRC Supervisor __________________________________________________________ _______

* The facility licensee signature is not applicable for NRC-developed tests.

# An independent NRC reviewer performs the steps in column c. This may be the NRC chief examiner if he/she 

did not develop the outline under review.
ES-7.1, Page 26 of 26

Form 7.1-6 Written Examination Cover Sheet for Senior Operators Limited to Fuel 

Handling

U.S. Nuclear Regulatory Commission

Written Examination for Senior Operators Limited to Fuel Handling

Applicant Information

Name:

Date: Region: I II III IV 

Facility/Unit: Reactor Type: W CE BW GE 

AP1000 ABWR 

Start Time: Stop Time:

Instructions

Use the answer sheets provided to document your answers. Staple this cover sheet

on top of the answer sheets. Passing the examination requires a final grade of at least 

80 percent. Examination papers will be picked up 4 hours after the examination begins.

Applicant Certification

All work done on this examination is my own. I have neither given nor received aid.

__________________________

Operator’s Signature

Results

Test Value __________ Points

Applicant’s Score __________ Points

Applicant’s Grade __________ Percent
ES-7.2, Page 1 of 6

1 7.2 ADMINISTRATION OF NRC REQUALIFICATION EXAMINATIONS 

2 FOR SENIOR OPERATORS LIMITED TO FUEL HANDLING

3 This examination standard provides general guidance for facility licensees and instructions for 

4 NRC examiners to use in preparing, administering, grading, and documenting NRC-conducted

5 requalification examinations for LSROs. It is designed to be used in conjunction with ES-6.1, 

6 “Conducting NRC Requalification Examinations,” ES-6.2, “Requalification Written 

7 Examinations,” and ES-6.3, Requalification Walkthrough Examinations.”

8 A. Background

9 The NRC’s requalification examinations for LSROs are administered under this standard in 

10 accordance with the provisions of 10 CFR 55.59(a)(2)(iii).

11 B. General Differences

12 The LSRO examinations will be conducted in accordance with the methodology outlined in 

13 ES-6.1, with the following exceptions:

14 1. The dynamic simulator operating test described in ES-6.4, “Requalification Dynamic 

15 Simulator Examinations,” is not applicable to LSRO requalification examinations.

16 2. The NRC will coordinate with the facility licensee to schedule the NRC-conducted LSRO 

17 examinations concurrent with the facility licensee’s LSRO requalification examination 

18 schedule. If practical, the examination team will conduct the LSRO examination shortly 

19 before or after an outage to facilitate access to refueling equipment because some of the 

20 equipment is not accessible during plant operations.

21 The NRC may administer LSRO requalification examinations concurrent with full-scope 

22 initial license or operator requalification examinations.

23 3. The facility licensee’s LSRO requalification program, LSRO job task analysis, and 

24 associated learning objectives will provide the basis for the examination if they are of 

25 sufficient scope and depth. The items in 10 CFR 55.43 and 10 CFR 55.45 will be 

26 sampled as appropriate to the LSRO’s limited responsibilities.

27 4. The LSRO requalification examination will consist of a written examination and a

28 walkthrough operating test, which are administered and evaluated individually. 

29 References to the crew-based dynamic simulator test and the associated crew 

30 evaluation criteria and forms do not apply to LSROs.

31 5. Whenever possible, the facility licensee should include an LSRO on the examination 

32 team.

33 6. The requirement to examine at least 12 operators to arrive at a program evaluation is not 

34 applicable to LSRO examinations. The region and, if necessary, the NRR operator 

35 licensing program office will determine the appropriate sample size based on the number 

36 of LSROs licensed at the facility.

37 7. The NRC will revise the sample corporate notification letter (Letter 6.1-1, “Sample 

38 Corporate Notification Letter”) as necessary to reflect the examination arrangements and 

39 to specify a modified list of reference material requirements associated with LSRO fuel 
ES-7.2, Page 2 of 6

1 handling activities. The NRC regional office will review the reference material using the 

2 applicable portions of Form 6.1-4, “Evaluation Checklist for Facility Reference Material.”

3 8. The NRC staff expects the facility licensee to maintain JPMs and written examination 

4 banks for use in evaluating LSROs. Facility licensees should periodically update these 

5 examination banks to reflect areas of emphasis in training and to ensure that they 

6 represent all applicable knowledge and skills. There is no minimum threshold or ceiling 

7 for these banks.

8 9. The NRC regional office will document the agency’s LSRO requalification examination 

9 results using Form 7.2-1, “Individual Requalification Examination Report for Senior 

10 Operators Limited to Fuel Handling,” and Form 7.2-2, “Power Plant Requalification 

11 Results Summary for Senior Operators Limited to Fuel Handling,” instead of Form 6.1-5, 

12 “Power Plant Requalification Results Summary Sheet,” and Form 6.1-6, “Individual 

13 Requalification Examination Report.”

14 10. This standard does not provide for a formal LSRO requalification program evaluation; 

15 however, if more than one-third of the examined LSROs at a facility fail, the NRC may 

16 need to inspect the LSRO requalification program. The NRC regional office is 

17 responsible for determining whether such an inspection should be conducted. If the 

18 NRC performs such an inspection, the staff should assess at least the following:

19 a. the content of the training program, the development of examination materials, 

20 and the quality controls

21 b. the administrative controls for maintaining training material current with 

22 procedural revisions and design changes

23 c. the training and evaluation techniques of the facility licensee’s evaluators

24 d. the evaluation techniques that the facility licensee uses to determine whether it 

25 has effectively implemented and assessed its training

26 e. the frequency, scope, and depth of the training provided to the operators

27 Section C of this examination standard discusses specific exceptions related to each category 

28 of the examination. Any questions about the NRR operator licensing program office’s 

29 expectations for the conduct of LSRO requalification examinations should be referred to the 

30 NRR operator licensing program office for resolution.

31 C. Examination Differences

32 1. Written Examination

33 The written examination will be developed, administered, and evaluated as described in ES-6.2, 

34 with the following exceptions:

35 a. The written examination will be “open reference” and will contain a minimum of 25 points 

36 in a single section; static simulator scenarios do not apply to the LSRO examination. 

37 The time limit for completing the examination shall be 2 hours, but the examination 

38 should be constructed so that a competent LSRO can complete it in 1.5 hours. The 

39 examination should emphasize refueling procedures, administrative controls, and 
ES-7.2, Page 3 of 6

1 abnormal and emergency procedures. The examination should include questions 

2 associated with industry and licensee event reports and recent plant modifications that 

3 affected refueling operations and systems that apply to the facility.

4 b. Form 7.2-3, “Written Requalification Examination Cover Sheet for Senior Operators 

5 Limited to Fuel Handling,” will be used as a cover sheet rather than Form 6.2-1, “NRC 

6 Checklist for Open-Reference Test Items.

7 2. Walkthrough Operating Test

8 The walkthrough operating test will be developed, administered, and evaluated as described in 

9 ES-6.3, with the following exceptions:

10 a. Each LSRO will be administered an operating test consisting of five JPMs. Whenever 

11 possible, these JPMs should include the use of refueling equipment to manipulate 

12 dummy fuel only or the use of a refueling machine simulator if one is available at the 

13 facility. If dummy fuel manipulation or the use of a simulator is not possible, the refueling 

14 tasks should be simulated. The requirement to conduct a minimum number of JPMs in 

15 the control room/simulator is not applicable to LSRO examinations.

16 b. Each JPM will consist of a task that is normally performed by fuel handling personnel 

17 and will include tasks performed both before and after refueling and for maintenance, 

18 surveillance, or testing of systems or equipment. The examination team may evaluate 

19 the LSRO’s ability to perform normal fuel handling administrative tasks, including 

20 documenting clearances, maintenance activities, and surveillances. The operating test 

21 should also evaluate the LSRO’s response to abnormal or emergency events associated 

22 with fuel handling.

23 c. If sufficient facility-developed JPMs are not available, the NRC can conduct a 

24 walkthrough examination of the type administered to an initial LSRO applicant, as 

25 discussed in ES-7.1, “Administration of Initial Examinations for Senior Operators Limited 

26 to Fuel Handling.”

27 D. Forms

28 Form 7.2-1 Individual Requalification Examination Report for Senior Operators Limited to 

29 Fuel Handling

30 Form 7.2-2 Power Plant Requalification Results Summary for Senior Operators Limited to 

31 Fuel Handling

32 Form 7.2-3 Written Requalification Examination Cover Sheet for Senior Operators Limited to 

33 Fuel Handling

34
ES-7.2, Page 4 of 6

1 Form 7.2-1 Individual Requalification Examination Report for Senior Operators Limited to 

2 Fuel Handling

3 PRIVACY ACT INFORMATION—FOR OFFICIAL USE ONLY

4

U.S. Nuclear Regulatory Commission

Individual Requalification Examination Report for Senior Operators 

Limited to Fuel Handling

Operator’s Name: Facility:

Docket No.: 55- Retake Exam: 1st / 2nd / # Date of Last Exam:

License No.: SOP- Expiration Date:

Written Examination Results

Date of Exam: NRC Examiner: Facility Evaluator:

Overall Grade (%)

NRC Facility

% %

Operating Test Results

Date of Test: NRC Examiner: Facility Evaluator:

No. of Job Performance 

Measures Correct of of

Final Grade (%) % %

NRC Examiner Recommendations

Category Results Signature

Written Pass/Fail

Operating Pass/Fail

NRC Supervisor Review

Date: Pass/Fail

5

6 PRIVACY ACT INFORMATION—FOR OFFICIAL USE ONLY

7

8
ES-7.2, Page 5 of 6

1 Form 7.2-2 Power Plant Requalification Results Summary for Senior Operators Limited to 

2 Fuel Handling

3 PRIVACY ACT INFORMATION—FOR OFFICIAL USE ONLY

Power Plant Requalification Results Summary

for Senior Operators Limited to Fuel Handling

Facility: Exam Date:

Examiners:

Overall

Results

--->

Total # of

Operators

Passed

(# / %)

Failed

(# / %)

Individual Results

Operator’s

Name

Docket

No. 55-

Grader JPM %

Overall

Written

(%)

Results ((P)ass/(F)ail)

Written Operating

NRC

Facility

NRC

Facility

NRC

Facility

NRC

Facility

NRC

Facility

NRC

Facility

4 PRIVACY ACT INFORMATION—FOR OFFICIAL USE ONLY

5
ES-7.2, Page 6 of 6

1 Form 7.2-3 Written Requalification Examination Cover Sheet for Senior Operators Limited 

2 to Fuel Handling

U.S. Nuclear Regulatory Commission

Written Requalification Examination for Senior Operators Limited to Fuel 

Handling

Operator Information

Name:

Date: Region: I II III IV 

Facility/Unit: Reactor Type: W CE BW GE 

AP1000 ABWR 

Start Time: Stop Time:

Instructions

Use the answer sheets provided to document your answers. Staple this cover sheet on top of 

the answer sheets. Points for each question are indicated in parentheses after each 

question. Passing the examination requires a final grade of at least 80 percent. Examination 

papers will be picked up 2 hours after the examination begins.

Operator Certification

All work done on this examination is my own. I have neither given nor received aid.

____________________________

Operator’s Signature

Results

Test Value __________ Points

Operator’s Score __________ Points

Operator’s Grade __________ Percent

3

4

5

6

7

8

9
ES-8, Page 1 of 7

1 8 GLOSSARY

2 Annual: In most instances, a period of time equal to 365 days calculated from any point in a

3 calendar year to the same point in the following calendar year. However, annual requirements 

4 in successive years can reach a period of nearly 2 years. That is, “annual” could encompass a 

5 range extending to 729 days, depending on when an event occurred in the first calendar year 

6 and viewing December 31 of the following calendar year as meeting the annual requirement.

7 Applicable K/A catalog: The revision of the technology- or vendor-specific 

8 (e.g., pressurized-water reactor, boiling-water reactor, AP1000) knowledge and abilities (K/A)

9 catalog reviewed by the U.S. Nuclear Regulatory Commission (NRC) and used to develop the 

10 written examination outline and operating test outline for initial operator licensing examinations.

11 Applicant: Any individual who has submitted an NRC Form 398, “Personal Qualification 

12 Statement—Licensee,” in pursuit of a reactor operator (RO) or senior reactor operator (SRO)

13 license. For the purposes of the NRC’s other examination standards, “applicant” is 

14 synonymous with “candidate.”

15 Applicant license level: The level of operator license (i.e., RO or SRO) for which the 

16 applicant has applied.

17 Average: A score that indicates the typical performance of a group of scores. The mean, 

18 median, and mode of a distribution of scores are all commonly used as averages.

19 Bank question: A written examination question taken from any facility licensee collection of 

20 questions that have previously appeared on any operator training-related examination at the 

21 facility. This definition includes NRC examination questions used at other facility licensee sites.

22 Biennial: In most instances, a period of time equal to 730 days and synonymous with 2 years. 

23 Biennial requirements can extend beyond 730 days if the requirement is met during the

24 anniversary month of the second year. For example, a biennial medical examination last

25 performed on January 10, 2020, would be due again by January 31, 2022. In this case, 

26 January is seen as the anniversary month, and the biennial requirement is satisfied even though

27 the period of time between the two examinations is longer than 730 days.

28 Bloom’s taxonomy: A classification system that depicts knowledge and information 

29 processing in a hierarchy from lowest to highest as fundamental knowledge, comprehension, 

30 analysis, synthesis, and evaluation.

31 Calendar quarter: One of four parts of a calendar year, each consisting of a 3-month 

32 segment. In any calendar year, the first quarter is from the first day of January to the last day 

33 of March, the second quarter is from the first day of April to the last day of June, the third 

34 quarter is from the first day of July to the last day of September, and the fourth quarter is from

35 the first day of October to the last day of December.

36 Central tendency: A term referring to the most typical performance of a group of individuals,

37 generally the mean, median, or mode.

38 Cold-plant operator licensing: Also called the operator licensing process for cold plants. A

39 term that refers to a facility licensee’s method for operations personnel to acquire the knowledge 

40 and experience required for licensed operator duties during the unique conditions of new plant 
ES-8, Page 2 of 7

1 construction and initial operation. This method usually applies during plant construction and 

2 ends with the completion of the first refueling outage.

3 Cognitive: Aspects of a person or test level that refer to knowledge or understanding.

4 Comparable unit: An identical or nearly identical second or subsequent unit operated by the 

5 same facility licensee at the same location.

6 Content validity: The degree to which a test measures the specific objectives or content.

7 Correlation coefficient: A numerical value, ranging from -1 to +1, that indicates the

8 relationship between two sets of scores or other measures of each individual in a group. A

9 value of 0 indicates no relationship; +1 or -1 indicates a perfect relationship (either positive or 

10 negative).

11 Criterion: A characteristic or combination of characteristics used as the basis for assessing 

12 performance.

13 Criterion-referenced test: An examination based upon mastery of objectives of content that

14 was or should have been taught and mastered and one that uses an established standard or cut

15 score as a measure of acceptable performance.

16 Critical task: A task used by the NRC during simulator scenarios to evaluate whether an 

17 individual or crew can complete actions that are of significant importance to the safety of the 

18 plant and the public. These tasks must meet specific criteria in order to be designated as 

19 critical tasks. See Examination Standard (ES)-3.3, “Testing Guidelines for Dynamic Simulator 

20 Scenarios.”

21 Critical performance deficiency: A performance deficiency associated with a critical task.

22 Cut score: The score at which a trainee is deemed to have met the criteria for an examination.

23 Defer: To postpone completion of a license application requirement(s) until a later date, 

24 typically after the applicant passes an initial NRC licensing examination. An applicant’s request 

25 to defer a requirement(s) is documented as a “deferral” on NRC Form 398. The applicant shall 

26 complete the deferred item(s) before the NRC issues a license.

27 Diagnostic test: An instrument that is designed to identify an individual’s strengths and

28 weaknesses in a given content area.

29 Different units: Two or more units owned or managed by a single facility licensee and any of 

30 the following:

31 • units having the same vendor but significantly different age or power level (e.g., Nine 

32 Mile Point Nuclear Station, Units 1 and 2)

33 • units having the same vendor and similar design but different locations (e.g., Sequoyah 

34 Nuclear Plant and Watts Bar Nuclear Plant, Byron Station and Braidwood Station)

35 • units having different vendors (pressurized-water reactors only) but located on the same 

36 site (e.g., Arkansas Nuclear One, Units 1 and 2; Millstone Power Station, Units 2 and 3)
ES-8, Page 3 of 7

1 Difficulty index: A numerical index, ranging from 0.00 to 1.00, that indicates the percentage 

2 of trainees who correctly answer a test item. An index of 0.00 indicates that no one correctly

3 answered the test item, while an index of 1.00 indicates that all trainees correctly answered the 

4 item.

5 Discrimination index: A measure of a test item’s ability to differentiate between good and

6 poor trainees. A high discrimination index indicates that more high performers than low

7 performers correctly answered the item. (“High” and “low” are typically determined by overall 

8 test scores, but they may also be established by external criteria.)

9 Discrimination validity: Setting the item difficulty at an estimated level around the cut score.

10 Distractor: An incorrect alternative among the possible answers for a test item.

11 Error of measurement: Any difference between an obtained score and a true score on a test. 

12 The actual error of measurement can only be estimated, since it is impossible to know the true

13 score.

14 Event: Any normal evolution, instrument or component failure, equipment malfunction, 

15 reactivity manipulation, and major plant transient when used in the context of the simulator 

16 scenario portion of the operating test.

17 Excuse: To excuse a reapplicant from the requirement to complete portions of a 

18 reexamination or test in accordance with 10 CFR 55.35(b). Granting such a release is referred 

19 to as an “excusal.”

20 Frequency distribution: A graphic display listing scores or score intervals on one axis of a

21 graph and the number of trainees at that score or in that interval on the other.

22 Item analysis: A set of procedures performed on test items to determine their difficulty and 

23 discriminating power.

24 Item bank: A group of test items covering a defined area. Items for a test can be chosen 

25 from this source.

26 Item stem: The part of a test item that presents the problem or situation to be solved. The

27 item stem may be a question requiring a response, or a statement that is followed by the

28 alternatives from which the trainee must choose the best answer.

29 Job performance measure: An evaluation tool that is based on tasks contained in the

30 facility’s job task analysis or the applicable NRC K/A catalogs (NUREG-1122, “Knowledge and 

31 Abilities Catalog for Nuclear Power Plant Operators: Pressurized Water Reactors”; 

32 NUREG-1123, “Knowledge and Abilities Catalog for Nuclear Power Plant Operators: Boiling 

33 Water Reactors”; NUREG-2103, “Knowledge and Abilities Catalog for Nuclear Power Plant 

34 Operators: Pressurized-Water Reactors Westinghouse AP1000”; and NUREG-2104, 

35 “Knowledge and Abilities Catalog for Nuclear Power Plant Operators: Advanced Boiling Water 

36 Reactors”) and requires the applicant to perform (or simulate) a task that is applicable to the 

37 license level of the examination.

38 Job task analysis: A systematic analysis of the knowledge, skills, and abilities required to 

39 perform a particular occupation.
ES-8, Page 4 of 7

1 Learning objective: A statement of the behavior a trainee is expected to exhibit following

2 instruction.

3 Low power: In accordance with NUREG-1449, “Shutdown and Low-Power Operation at 

4 Commercial Nuclear Power Plants in the United States,” issued September 1993, the range of 

5 reactor power from criticality to 5 percent.

6 Mastery test: A term synonymous with “criterion-referenced test” (i.e., one that evaluates the

7 expected behavior following instruction).

8 Mean: An indication of “central tendency.” Mean usually refers to the arithmetic mean, which

9 is computed by summing all the scores of a group and dividing that sum by the number of

10 scores in the group.

11 Median: A measure of “central tendency”; the point on a scale of scores that splits the scores 

12 in half, with 50 percent of the scores below this point and 50 percent of the scores above this

13 point.

14 Mode: The least reliable of the common measure of “central tendency”; the “mode” is the most

15 frequently occurring score in a distribution of scores.

16 Multiple-choice item: A test item that is composed of an item stem and several alternatives 

17 from which the trainee must select the best answer.

18 Normal distribution: A theoretical frequency distribution represented by a symmetrical 

19 bell-shaped curve; sometimes referred to as the bell curve.

20 Norm referenced: A score interpretation based on the comparison of an individual’s score 

21 with a comparable reference group.

22 Nuclear power plant experience: As defined in Section 2 of American National Standards 

23 Institute (ANSI)/American Nuclear Society (ANS) 3.1-1993, “American National Standard for 

24 Selection, Qualification, and Training of Personnel for Nuclear Power Plants,” applicable work 

25 performed in a nuclear-fueled electric power production plant during preoperational, startup 

26 testing, or operational activities. Observation of others performing work does not constitute 

27 experience.

28 Objective test: A test that can be scored without subjective judgment in the scoring.

29 On-the-job training: Participation in nuclear power plant startup, operation, maintenance, or

30 technical services as a trainee under the direction of experienced personnel.

31 Operating test: That portion of the operator licensing examination based on direct interaction 

32 between an examiner and an applicant. The operating test assesses applicant knowledge of 

33 the design and operation of the reactor and its associated plant systems, both inside and 

34 outside the control room. It is administered in a plant walkthrough and a simulation facility.

35 Operational validity: A test item that (1) relates to the operations of the job and appears 

36 reasonable to ask and (2) is expressed in an operational context that requires the candidate to

37 perform mentally or physically through understanding or analysis.

38 Performance deficiency: An observed action or inaction (including operational tasks, 

39 procedure/process implementation, communications, and administrative functions), or a 
ES-8, Page 5 of 7

1 statement of understanding or intent, that demonstrates a lack of ability or understanding as 

2 outlined by an established standard for operator performance (e.g., facility procedure, policy, 

3 learning objective, regulatory requirement).

4 Performance test: Any test that requires the trainee to demonstrate either mental 

5 performance through knowledge testing or skill by actual operation or manipulation of tools and 

6 equipment. Typically, performance tests involve skill testing.

7 Plant-referenced simulator: As defined in 10 CFR 55.4, “Definitions,” a simulator that models

8 the systems of the reference plant with which the operator interfaces in the control room, 

9 including operating consoles, and that permits use of the reference plant’s procedures. A

10 plant-referenced simulator used to administer operating tests (under 10 CFR 55.45(b)) or to

11 meet experience requirements (under 10 CFR 55.31(a)(5)) must be designed and implemented 

12 in accordance with 10 CFR 55.46, “Simulation facilities.”

13 Power plant experience: As defined in Section 2 of ANSI/ANS 3.1-1993, applicable work 

14 performed in a fossil-fueled or nuclear-fueled electric power production plant during 

15 preoperational, startup testing, or operational activities. Observation of others performing work 

16 does not constitute experience.

17 Predictive validity evidence: The ability of a test to forecast future performance on a

18 subsequent measure.

19 Psychomotor: The domain of human performance that relates to physical performance based

20 on mental activity.

21 Range: The smallest interval on a scale of scores that will include all scores; mathematically

22 defined as the largest score minus the smallest score plus one.

23 Raw score: The numerical score first assigned when scoring a test before conversion to a

24 derived score.

25 Reactor operator applicant: An unlicensed individual who is applying for an RO license.

26 Reference plant: As defined in 10 CFR 55.4, the specific nuclear power plant from which a

27 simulation facility’s control room configuration, system control arrangement, and design data are

28 derived.

29 Related technical training: Formal training beyond the high-school level in technical subjects 

30 associated with the position in question, such as that acquired in training schools or programs 

31 conducted by the military, industry, utilities, universities, vocational schools, or others. Such

32 training programs shall be of a scheduled and planned length and include textual material and

33 lectures.

34 Reliability: The consistency or repeatability of any measure as an indicator of confidence in

35 that measure.

36 Scenario: An integrated group of events that simulates a set of plant malfunctions and

37 evolutions at a simulation facility.
ES-8, Page 6 of 7

1 Scenario set: For a given operating test, all scenarios that are developed, reviewed, and 

2 approved for examination administration. The scenario set does not include the spare scenario 

3 unless it is subsequently administered.

4 Score: A numerical indication of the performance an individual displays on a test.

5 Senior reactor operator-instant applicant: An unlicensed individual who is applying for an 

6 SRO license.

7 Senior reactor operator-upgrade applicant: A licensed RO who is applying for an SRO 

8 license on the same unit(s).

9 Significant control manipulations: An operation (excluding those required for fuel handling) 

10 of an apparatus or mechanism that directly affects the reactivity or power level of a critical 

11 reactor by an amount of sufficient magnitude to allow for the observation of clear effects on the 

12 plant by the operator. The requirements in 10 CFR 55.31(a)(5) define this term for operations 

13 conducted in a simulator.

14 Significant performance deficiency: Used in the context of simulator scenarios, a 

15 performance deficiency that is more severe than a regular applicant performance deficiency but 

16 does not meet the criteria for a critical performance deficiency. The identification of a 

17 significant performance deficiency only occurs after completion of the scenario because of an 

18 error made during the scenario by one or more applicants. (ES-3.5, “Administering Operating 

19 Tests,” contains the criteria for a significant performance deficiency.) 

20 Simulation facility: As defined in 10 CFR 55.4, one or more of the following components, 

21 alone or in combination, used for the partial conduct of operating tests for operators, senior

22 operators, and applicants (under 10 CFR 55.45(b)) or to establish on-the-job training and

23 experience prerequisites for operator license eligibility (under 10 CFR 55.31(a)(5)):

24 • a plant-referenced simulator

25 • a Commission-approved simulator under 10 CFR 55.46(b)

26 • another simulation device, including part-task and limited-scope simulation devices, 

27 approved under 10 CFR 55.46(b)

28 Standard deviation: A measure of variability of a set of scores around the group mean. The

29 standard deviation is mathematically defined as the square root of the mean of the squared

30 deviations of the scores from the mean of the distribution.

31 Standard error of measurement: An estimate of the standard deviation of the errors of

32 measurement associated with the scores in a given test.

33 Standardized test: A test that has the directions, time limits, and conditions of administration 

34 made consistent for all offerings of the test; this test is usually norm-referenced.

35 Statistic: A numerical value computed on a sample of data.

36 Technical specifications: A document that identifies the plant-specific safety limits, system

37 operability and surveillance testing requirements, and administrative controls. Whether stated 
ES-8, Page 7 of 7

1 or not, references to the technical specifications in this NUREG include those administrative 

2 controls that have been moved to other technical requirements documents.

3 Test: A measurement instrument; examination.

4 True score: The ideal or correct score for an individual. Its value cannot be known, but it can

5 be estimated when assumptions about error of measurement are made.

6 Validity: The degree to which a test measures what it purports to measure.

7 Verifiable action: A term used in the context of evaluating an examinee’s competence during 

8 portions of the operating test. A verifiable action is an examinee action who provides insight for 

9 the examiner to be able to determine whether the examinee can operate the

10 equipment/equipment controls and control the system response. The purpose of a verifiable 

11 action is to allow the examiner to observe the examinee perform an action or, in the case of a 

12 job performance measure in the plant, describe exactly what it takes to perform an action.

13 Video recording: An electronic medium used for recording, copying, and playback of moving 

14 visual images and associated audio components. Storage media include digital file, digital

15 video disk, video cassette tape, and compact disk.

16 Waive: To forgo or relinquish a legal requirement that the NRC is legally entitled to enforce. 

17 Forgoing such a requirement is documented as a “waiver.” The NRC specifies its ability to 

18 waive examination and test requirements in 10 CFR 55.47, “Waiver of examination and test 

19 requirements.”

A-1

1 APPENDIX A

2 OVERVIEW OF GENERIC EXAMINATION CONCEPTS

3 This appendix explains the rationale for providing instructions and guidance (through NUREG4 1021) for the preparation, review, and approval of U.S. Nuclear Regulatory Commission (NRC) 

5 operator licensing and requalification examinations. It provides an overview of two fundamental 

6 examination concepts—validity and reliability—as they apply to the development of NRC 

7 examinations. It also includes background information about the written examination for 

8 licensing purposes. Specifically, this appendix discusses the following topics:

9 • the various aspects of validity and how the NRC establishes the validity of its 

10 examinations (Section B) 

11 • the concept of reliability and how it is maintained on NRC examinations (Section C) 

12 • the importance of the written examination (Section D) 

13 • the difference between objective and subjective test items (Section E) 

14 A. Background

15 The NRC’s operator licensing examinations are prepared and administered by many different 

16 individuals working in various locations; therefore, it is imperative to establish and follow a 

17 defined set of administrative structures and protocols to ensure that the examinations are 

18 administered successfully and consistently. Consistent external attributes, such as the number 

19 and types of items, the length of the examination, security procedures, proctoring instructions, 

20 and other administrative details, are essential to the orderly conduct of an examination. These 

21 factors have a significant effect on the validity and reliability of an examination—the examination 

22 cornerstones that allow the NRC to make confident licensing decisions.

23 The internal attributes of the examination, such as the level of knowledge it requires, level of 

24 difficulty, and use of item banks, also impact the operational and discriminatory validity of the 

25 examination, which, in turn, can affect its validity and reliability. If the internal and external 

26 attributes of examinations are allowed to vary significantly, the uniform conditions that are 

27 required by Section 107 of the Atomic Energy Act of 1954, as amended, and the basis upon 

28 which the NRC’s licensing decisions rest are challenged. The NRC must reasonably control 

29 and structure the examination processes to ensure the integrity of the licenses it issues.

30 Acceptable levels of examination consistency, uniformity, and fairness would be impossible to 

31 achieve without quantitative and qualitative acceptance criteria. The examination standards 

32 identify many of the quantitative criteria necessary for a well-balanced and consistent 

33 examination. Although NUREG-1122, “Knowledge and Abilities Catalog for Nuclear Power 

34 Plant Operators: Pressurized Water Reactors”; NUREG-2103, “Knowledge and Abilities 

35 Catalog for Nuclear Power Plant Operators: Pressurized Water Reactors Westinghouse 

36 AP1000”; NUREG-1123, “Knowledge and Abilities Catalog for Nuclear Power Plant Operators: 

37 Boiling Water Reactors”; and NUREG-2104, “Knowledge and Abilities Catalog for Nuclear 

38 Power Plant Operators: Advanced Boiling Water Reactors,” for pressurized-water reactors and 

39 boiling-water reactors have brought a degree of consistency to the qualitative issue of safety 

40 significance, there is no comparable mechanism to aid in determining an examination’s required 

41 level of knowledge or difficulty before it is administered. In the end, the validity and consistency 

42 of the NRC’s examinations depend largely on the individual and collective judgments of the 
A-2

1 people who write and review the examinations. The discussions herein clarify the intent of the 

2 NRC’s examination criteria, thereby decreasing the likelihood that inconsistencies among 

3 examinations, particularly with regard to the level of knowledge and difficulty, will jeopardize the 

4 integrity of the NRC’s licensing decisions.

5 B. Validity

6 For a test to be considered valid, it must be shown to measure that which it is intended to 

7 measure. The NRC examinations are intended to measure the examinee’s knowledge and 

8 ability (K/A) such that those who pass will be able to perform the duties of a reactor operator 

9 (RO) or senior reactor operator (SRO) to ensure the safe operation of the plant. The following 

10 subsections outline the three principal facets of test validity and the techniques that are used to 

11 establish the validity of NRC examinations.

12 1. Content Validity

13 a. Establish a Link to Job Duties

14 In order to develop valid examinations, the K/As selected for testing must be linked to and 

15 based upon a description of the most important job duties. This is accomplished by conducting 

16 a job task analysis (JTA) focusing on the delineation of essential K/As.

17 The testing industry endorsed this approach to the development of content-valid licensing 

18 examinations in the 1985 revision of the “Standards for Educational and Psychological Testing” 

19 published by the American Educational Research Association, the American Psychological 

20 Association, and the National Council on Measurement in Education. Those standards treat 

21 licensing examinations in a separate section in recognition of their importance and uniqueness. 

22 Accordingly, those seeking additional technical guidance are encouraged to consult Chapter 11 

23 of those standards for further clarification.

24 To ensure content validity in the NRC’s examinations, the JTA performed on the licensed 

25 operator and senior operator positions by the Institute of Nuclear Power Operations (INPO) 

26 served as the initial source of information. The INPO JTA identified more than 28,000 K/As and 

27 nearly 800 tasks. The extensive number of tasks and K/A statements is attributable, in part, to 

28 the specific purpose of the analysis, which was to provide an information base to be used in 

29 developing training programs that would be applicable to all pressurized- and boiling-water 

30 reactor facilities. Accordingly, many of the individual statements were too specific or too 

31 elementary for use as the basis for development of the NRC’s examinations. The job content of 

32 special interest to the NRC is that subset of K/As that are required for the safe operation of the 

33 nuclear plant. Although safe performance and efficient performance may have considerable 

34 overlap, any K/A that contributes to efficiency but not to safety is an inappropriate focus for the 

35 NRC’s examinations.

36 The applicable K/A catalogs provide the basis for the development of content-valid 

37 examinations for ROs and SROs, consistent with the testing industry standards described 

38 above.

39 The fact that the K/As from which test items are developed are drawn or sampled from the same 

40 universe regardless of who develops the examination helps ensure that the examinations are 

41 consistently content valid. Furthermore, developing the examinations using the appropriate K/A 

42 catalog in conjunction with the applicable examination standards and related appendices helps 
A-3

1 ensure that the examinations cover a representative sample of the topics listed under Title 10 of 

2 the Code of Federal Regulations (10 CFR) Part 55, “Operators’ licenses.”

3 The NRC developed K/A catalogs NUREG-1122 and NUREG-1123 based on the INPO JTA. 

4 The catalogs were reviewed by licensed ROs and SROs, as well as the NRC’s own license 

5 examiners. The NRC developed K/A catalogs NUREG-2103 and NUREG-2104 based on the 

6 design center JTA. These catalogs were reviewed by system experts and personnel 

7 knowledgeable of the design, many of whom were legacy-plant ROs and SROs, as well as the 

8 NRC’s own license examiners. These experts reviewed the K/A statements for accuracy and 

9 completeness and then rated each statement with respect to its importance to safe plant 

10 operation. Chapter 1 of each catalog further explains the content of the K/A catalogs.

11 In addition to the NRC’s K/A catalogs, learning objectives from the facility licensee’s training 

12 program often provide a supportive reference for test items to include in the NRC’s examination. 

13 Since facility learning objectives are specific to the job requirements at a given site, they should 

14 provide an excellent basis for test item development. However, because they are not always 

15 stated at the comprehension- or analysis-level of knowledge (the preferred focus for NRC 

16 examinations), they should be referenced only to the extent that they support a test item that is 

17 being developed.

18 b. Use a Sample Plan

19 Once the essential K/As have been identified through the JTA, test specifications must be 

20 developed. The test specifications consist of a content outline or sample plan indicating what 

21 proportion of items or questions shall deal with each K/A. Because a single test cannot 

22 measure every K/A required to be a licensed operator, it must sample the required knowledge 

23 or performance in a manner that allows inferences to be made about the examinees’ 

24 performance on the broader population of knowledge, even though the full body of knowledge

25 was not tested. The sample must be evenly distributed and soundly based so that the NRC can 

26 confidently assume that the untested knowledge is proportionately known or not known in 

27 relation to the score on the sample. In other words, by testing performance on the sample, it is 

28 possible to make inferences concerning the broader area of knowledge not tested. This is 

29 referred to as a “validity inference.”

30 The sample plan is at the heart of making a validity inference. Research indicates that when 

31 samples are not chosen systematically and according to the sample plan, the sample is biased, 

32 and, therefore, its validity is reduced. When the sample is biased or skewed in a particular 

33 direction, it introduces some degree of sampling error, which makes it impractical to infer or 

34 generalize that the examinees have mastered the larger population of untested knowledge from 

35 which the sample was drawn.

36 Test items selected for inclusion in an NRC examination should be based on K/As contained in 

37 the appropriate K/A catalog. Testing outside the documented K/As can jeopardize the content 

38 validity of the examination. Content validity can also be reduced if the examination excludes 

39 important K/As. Therefore, the sample of K/As that are tested should cover all of the K/A 

40 categories in the catalog in a fashion that is consistent with their contribution to the public 

41 protection function of the examination. Not all categories are equal in this regard. This 

42 conclusion is based on the analysis of ratings on importance and testing emphasis collected 

43 from licensed SROs and NRC license examiners. The specific examination standards provide 

44 additional guidance on how to develop test outlines that will ensure adequate content coverage.
A-4

1 It is important to note that the testing demands for an initial examination are different than those 

2 for a requalification examination. The requalification examination is based on the plant’s 

3 systems approach to training during the requalification cycle and will more closely parallel the 

4 training received in the requalification program. Consequently, the instructional and testing 

5 processes are more closely linked. The initial examination, on the other hand, covers all 

6 instruction related to safety-significant K/As that either were or should have been taught during 

7 the training program. The examination standards ensure that the K/As are sampled in a 

8 relatively uniform process that would likely include content and instruction that occurred from the 

9 beginning to the end of the program and would not focus on any particular segment of 

10 instruction.

11 2. Operational Validity

12 The second facet of validity is operational validity. To the extent possible, test items should 

13 address an actual or conceivable mental or psychomotor activity performed on the job. In this 

14 regard, the more operationally oriented a test item is, the more valid the test item will be. 

15 Because operationally valid items involve skills central to job performance (i.e., analysis, 

16 prediction of events or system responses, or solving problems), the items should be written at 

17 the comprehension or analysis level rather than at the level of simple fundamental knowledge. 

18 The theoretical knowledge classification system upon which the NRC bases its operational 

19 validity estimates is Bloom’s taxonomy.

20 Bloom’s taxonomy suggests that testing knowledge at a higher cognitive level (HCL) 

21 (i.e., comprehension and analysis) is more efficient and operationally valid because those 

22 higher levels include the fundamental knowledge required, in part, to answer the higher level 

23 question. Furthermore, the higher the level tested in the test item, generally the more 

24 operationally valid that test item will be, since it is at the higher levels that questions invoke 

25 problem-solving, diagnosis, prediction, and analysis of conditions, events, and responses.

26 Designing test items that test the application of knowledge in different content situations 

27 (i.e., process testing) is at the heart of designing good, discriminatory test items. Just as a math 

28 teacher would not design a test to ask multiplication questions that were identical to practice 

29 questions, so too should the examination author minimize asking questions that are identical to 

30 those previously rehearsed or tested. Test items should attempt to assess similar knowledge 

31 applications in different contexts, thereby assessing the examinee’s problem-solving skills in 

32 new and different applications. These applications should be item substitutions of comparable 

33 difficulty, neither harder nor easier than those practiced. This practice provides assurance that 

34 the examination is valid and discriminatory, since the process (rather than the specific content) 

35 is primarily measured.

36 The NRC cannot make confident and consistent validity inferences (i.e., licensing decisions) if 

37 one examination assesses knowledge at lower cognitive levels and another assesses 

38 knowledge at higher levels (greater depth). Although each examination may meet sample plan 

39 coverage guidelines, they test different levels of knowledge; consequently, they are different 

40 and inconsistent measuring instruments. Therefore, they yield different validity inferences of

41 minimally safe operator performance. Section D of this appendix provides a more detailed 

42 discussion of consistency and reliability, and Appendix B gives a more detailed discussion of the 

43 various levels of knowledge as they relate to the development of written examination questions.
A-5

1 3. Discrimination Validity

2 The third facet of validity concerns the examination’s ability to discriminate or make some 

3 distinction along a continuum of examinee performance. In that regard, the primary objective of 

4 the NRC’s examinations is to determine whether the examinees have sufficiently “mastered” the 

5 knowledge, skills, abilities, and other attributes to perform the job of an RO or SRO at a specific 

6 plant. The NRC’s examinations are not intended to distinguish among levels of competency or 

7 to identify the most qualified individuals but to make reliable and valid distinctions at the 

8 minimum level of competency that the agency has selected in the interest of public protection.

9 a. Criterion-Referenced Testing

10 The NRC’s initial and requalification examinations, like most licensing examinations, are 

11 criterion- rather than norm-referenced tests. This means that the examinee must achieve a 

12 pass/fail or minimal cut score or grade to demonstrate sufficient K/As to safely operate the 

13 power plant. If the examination does not intend to discriminate at an agreed-upon minimal 

14 measure of knowledge or performance, there is little reason to administer the examination. For 

15 a criterion-referenced test to be effective, both the individual test items and the overall 

16 examination must discriminate between applicants who have and have not mastered the 

17 required K/As and skills.

18 b. Cut Scores

19 For NRC examinations, the overall cut scores (on the written examination and walkthrough 

20 portion of the operating test) are fixed at 80 percent (although lower cut scores apply to 

21 subparts of the examination); it is the content of the examination that varies from occasion to 

22 occasion because of the plant-specific character of the test material. As discussed below, the 

23 cut score must be fixed for several reasons, including the uniqueness of each examination, 

24 consistency, and public confidence.

25 In the writing, reviewing, setting of scoring standards, and grading of any particular NRC 

26 examination, both the examination author and the reviewer are well aware of the 

27 NRC-established passing score of 80 percent. They may also have knowledge of how prior 

28 examinees have performed on questions similar to those used on the examination being 

29 prepared and expectations as to how a qualified or unqualified applicant should perform on the 

30 examination. They must use this knowledge to control the nature and difficulty of the 

31 examination such that an examinee who is deemed to be qualified scores above the passing 

32 grade, while an examinee who is deemed to be unqualified scores below that grade.

33 The traditional cut score on the examination should not be viewed as arbitrary. Rather, it 

34 reflects a point on the test at which author and reviewer judgment separates the qualified from 

35 the unqualified. Nonetheless, the judgment is probably similar to other methodologies for 

36 determining passing test scores. For example, rather than explicitly judging the probability that 

37 a minimally qualified applicant will pass an item, the author is implicitly being asked to write an 

38 examination on which, in the author’s judgment, the minimally qualified applicant will obtain a 

39 score of at least 80 percent. Achieving this objective requires the author and reviewer to 

40 integrate their content and process skills.

41 c. Level of Knowledge Versus Level of Difficulty

42 As further discussed in Appendix B, the NRC uses Bloom’s taxonomy as the basis for 

43 classifying the level of knowledge of its test items (i.e., written examination questions, job 
A-6

1 performance measures (JPMs), and simulator events). Simply stated, level of knowledge 

2 represents the range of mental demands required to answer a question or perform a task. In 

3 other words, level of knowledge is a continuum of mental rigor that ranges from retrieving 

4 fundamental knowledge (low level) to retrieving that knowledge and understanding, analyzing, 

5 and synthesizing that knowledge with other knowledge (high level).

6 The accurate classification of knowledge as either low or high level requires the application of 

7 objective criteria. Although different reviewers can arrive at different conclusions about the 

8 knowledge level needed for individual test items, a common set of criteria can make the 

9 classification an informed process, thereby minimizing the differences among reviewers. 

10 Consistency among reviewers is important because this NUREG establishes specific criteria 

11 relative to the number of HCL test items on the site-specific written licensing examination. Keep 

12 in mind that classifying a test item’s required level of knowledge is not equivalent to determining 

13 its level of difficulty, which is discussed as a separate issue below.

14 When evaluating level of knowledge, two key elements must be considered:

15 1. the number and type of mental steps necessary to process the given data and arrive at 

16 the correct answer 

17 2. the training and experience level of the target test group

18 Generally, an HCL test item will require at least two mental steps—one requires the recall of 

19 acquired knowledge, and the other requires associating two or more pieces of data. The 

20 number and types of mental steps that must be considered are those necessary to rule out

21 plausible incorrect distractors, as well as the steps needed to identify the correct answer. 

22 Distractors can contain knowledge that the applicant might need to manipulate with other 

23 information contained in the question in order to answer the question, and this, in turn, may 

24 raise the level of knowledge needed for the question. However, it is largely the stem of the 

25 question that drives the mental thought required to answer the question.

26 An HCL test item will have at least two data points that must be associated. The test item may 

27 provide these data points, or the examinee may have to recall them from memory. For 

28 example, the examinee may be given one plant operating parameter in the stem of a question 

29 and have to recall a setpoint to evaluate whether a particular action should have occurred. This 

30 is considered HCL because it requires the examinee to (1) recall a setpoint beyond the 

31 information given in the stem and (2) compare the setpoint to a given data point. Because more 

32 than one mental step was necessary to answer this question and two data points had to be 

33 associated or compared, it should be classified as HCL.

34 Similarly, if a test item elicits a mental demand that requires a “why” or “how” response such 

35 that the examinee must derive the correct explanation, prediction, or action, the item is testing at 

36 the comprehension or application level. Comprehension/application-level test items require the 

37 examinee to recall stored knowledge and understand the relationship between two or more

38 pieces of data (such as events or conditions) given in the stem of the test item. In sum, HCL 

39 test items require multiple mental processing steps, which usually involve the recall and 

40 integration of two or more pieces of data. Good HCL test items are operational in nature and 

41 require demonstration of understanding and problem-solving.

42 Test items that simply ask examinees to provide a single answer that requires a “who, what, 

43 when, or where” response are typically fundamental knowledge (low-level) questions because 

44 they involve recalling or recognizing a single answer or block of information. The examinee is 
A-7

1 not required to understand cause-effect relationships or system responses. Therefore, if a test 

2 item simply asks for a reactor trip setpoint and does not require a comparison with an operating 

3 parameter value, it would be considered a lower cognitive-level question because only one 

4 mental step, with no data association, is necessary to arrive at the answer.

5 As previously noted, the training and experience of the target test group also must be 

6 considered when evaluating level of knowledge. A reviewer can approach the classification 

7 from the perspective of an “expert,” with a predetermined belief about the mental processes 

8 required to answer a given question, and incorrectly assume that the novice applicant will use 

9 the same processes. This is a form of perceptual bias that can affect classifications of the level 

10 of knowledge, as well as the level of difficulty. When examining new license applicants, the 

11 NRC expects that the typical applicant will need to mentally analyze, or figure out, the answers 

12 to HCL questions. Whereas the expert is able to answer a test item quickly and easily, the 

13 novice may have to eliminate plausible distractors to arrive at the correct answer, an indication 

14 of an HCL question. Therefore, when making the level of knowledge determination, 

15 examination writers and reviewers should place themselves in the context of the “novice 

16 applicant” and assess the components of the test item that the novice must manipulate to 

17 answer the test item.

18 Keep in mind that many test takers may easily arrive at the answer; however, ease of answering 

19 a question is a relative concept and should be clearly separated from the mental processes, or 

20 level of knowledge, required to answer the test item.

21 For the operating test items (i.e., JPMs and simulator events), the regulations at 

22 10 CFR 55.45(a) and 10 CFR 55.59(a)(2)(ii) specifically require an assessment of the 

23 examinees’ understanding of and ability to perform the actions specified in the regulation. 

24 Alternate-path JPMs are used to assess such understanding during the walkthrough portion of 

25 the operating test because they require examinees to evaluate unplanned conditions or events 

26 while executing procedures and to implement acceptable, alternative methods of accomplishing 

27 the assigned tasks.

28 In summary, the following concepts apply:

29 • Level of knowledge is a taxonomy to determine the mental processes used to answer a 

30 question. Those processes are classified as either lower or higher cognitive level and 

31 should not be confused with level of difficulty.

32 • An HCL test item requires at least two mental steps—one involving the recall of acquired 

33 knowledge and the other requiring the association of two or more pieces of data. The 

34 number and type of mental steps that must be considered include those necessary to 

35 rule out plausible incorrect distractors, as well as the steps needed to identify the correct 

36 answer. If there is doubt concerning the number of associations, an item should be 

37 classified as HCL. As a tip, the author should attempt to answer the question in an 

38 unaided recall manner (i.e., if the question were in the completion or short answer 

39 format, by covering the distractors and attempting to complete the answer). Then, the 

40 author should analyze the mental process needed to answer the question using the 

41 “who, what, when, or where” (fundamental) or the “how or why” 

42 (comprehension/analysis) criteria discussed above.

43 • When assessing level of knowledge, the examination writers/reviewers must use the 

44 perspective of the test taker in the target group (i.e., novice versus expert) to avoid 
A-8

1 perceptual bias. The reviewer has seen the item, knows the answer, and may not 

2 appreciate the mental processes that an examinee may use to answer the question.

3 Level of difficulty is a separate concept but is often influenced by the level of knowledge needed 

4 to answer a test item. Although HCL test items are generally more difficult, this may not always 

5 be true. A fundamental knowledge question may be easy (e.g., how many inches are in a foot) 

6 or difficult (e.g., in what year was the printing press invented).

7 The NRC evaluates the level of difficulty to answer a test item to ensure that the item can help 

8 discriminate between safe and unsafe operators. The examination’s overall level of difficulty, as 

9 well as that of its individual test items, should center around the 80-percent cut score, as 

10 described further in the next subsection. 

11 Assigning a level of difficulty rating to an individual test item is a somewhat subjective process. 

12 As when assessing the level of knowledge needed to answer an item, examination authors and 

13 reviewers must “detach themselves” as subject matter experts, place themselves in the position 

14 of the novice applicant, and apply what they know about previous applicants’ performance on 

15 similar test items. For example, if 10 to 20 percent of past license applicants missed a particular 

16 item, the item would be considered moderately discriminating, with a difficulty rating of 3 on a 

17 5-point scale. It would be reasonable to expect that a comparable test group would perform in 

18 the same way with a similar item. Conversely, if 95 percent or more of license applicants 

19 typically answer a particular test item correctly, future use of a comparable item will likely yield a 

20 similar result; therefore, a difficulty rating of 1 would be justified.

21 d. Cut Scores and the Level of Difficulty

22 For the cut score of 80 percent to be meaningful, individual test items must be written “near” that 

23 level. A target level of difficulty range of 70 to 90 percent is recommended for individual test 

24 items. Test items that are so difficult that few (if any) of the examinees are expected to answer 

25 correctly do not discriminate and should not be used on an NRC examination. Similarly, test 

26 items that are so easy or fundamental that even those examinees who are known to have 

27 performance problems will be able to answer them correctly should be used with discretion. 

28 The NRC expects that every examination will contain some test items that all or most of the 

29 examinees will answer correctly or incorrectly. This does not necessarily mean that the test 

30 items or the examination are invalid.

31 It should be stressed that the intent is not for all examinees to achieve a score of 80 percent. In 

32 fact, historically about 90 percent of examinees score 80 percent or above on the NRC 

33 examinations. A score of 80 percent is the minimal pass score that the author and reviewer 

34 must keep in mind as a functional level of discrimination for setting item difficulty. To achieve 

35 this, the test author must keep in mind and integrate the following concepts:

36 • the level of knowledge required of examinees taking the examination

37 • the operational validity of the questions (i.e., are they expressed as a conceivable job 

38 behavior)

39 • the ability of the distractors to distract the examinees

40 • the examinees’ past performance on items of similar difficulty
A-9

1 e. Use of Item Banks

2 Test item banks are a valuable resource for learning and represent one fundamental basis for 

3 training and testing. However, it would be inappropriate to copy all or a significant portion of the 

4 items for an examination directly from the bank if the same items were previously used for 

5 testing or training. Test item banks must be used properly to maintain the validity, reliability, 

6 and consistency of the examinations. Previously administered test items reduce examination 

7 integrity because examination discrimination is reduced.

8 Discrimination is reduced because the cognitive level at which the examinees are tested could 

9 decrease to the simple recognition level if the item bank is small and available for the 

10 examinees to study. The comprehension and analysis levels of knowledge may not be 

11 assessable because mental thought has been reduced to a recognition level, and 

12 decisionmaking is absent because test items, JPMs, or scenario events have been rehearsed 

13 and are anticipated. In short, challenge and mental analysis are lost, and the examinees are 

14 tested at a rote-rehearsal level. An examination cannot assess higher cognitive and analytical 

15 abilities if examinees have already seen a significant portion of the items within the examination.

16 Furthermore, when the bank of items from which the examination is drawn is known to the 

17 examinees before the examination, the examination is said to be highly predictable. Predictable 

18 examinations tend not to discriminate because what is being tested is simple recognition of the 

19 answer. Although studying past examinations can have a positive learning value, total 

20 predictability of examination coverage through overreliance upon examination banks reduces 

21 examination integrity. When the examinees know the precise and limited pool from which test 

22 items will be drawn, they will tend only to study from that pool (i.e., studying to the test) and may 

23 likely exclude from study the larger domain of job knowledge. When this occurs, it decreases 

24 the confidence in the validity inferences that are made from performance on the test to the 

25 larger realm of knowledge or skill to be mastered.

26 Therefore, the NRC has placed limits on the use of facility item banks or other such available 

27 banks or resources that have been published, reviewed, or used as the basis for training; the 

28 examination standards discuss the specific limits. The NRC appreciates the amount of 

29 resources required to develop new test items that are appropriate for use on an NRC 

30 examination, and it realizes that existing test items are a valuable resource that should not be 

31 wasted. Therefore, the NRC has elected to strike a balance in setting limits on the mix of 

32 previously used bank items, modified bank items, and newly developed (i.e., not previously 

33 seen) items. The agency has placed additional limits on the repetition of test items from prior 

34 quizzes and examinations given at the facility.

35 C. Reliability

36 Reliability is the second fundamental testing concept that has played a decisive role in the 

37 development of the NRC’s initial and requalification examination programs. Whereas the notion 

38 of validity emphasizes the appropriateness of the content of the NRC examinations, reliability 

39 stresses consistency, repeatability, and the degree of confidence that the examination process 

40 will result in valid pass/fail decisions. The reliability of an examination is as important as its 

41 validity; if an examination is not reliable, it cannot be valid.

42 The importance of examination consistency (reliability) cannot be overstated. In fact, test 

43 reliability represents the consistency among examinations that, in turn, gives the NRC the 

44 confidence that all examinations are valid measures from which to make confident and valid

45 licensing decisions. The combined effects of item bank use, the level of knowledge tested in the 
A-10

1 individual test items, and the expected discriminatory (difficulty) level of the items play an 

2 important role in determining the reliability of the examination.

3 The higher the reliability of a test, the fewer errors will be made in determining whether the 

4 examinees have mastered the job requirements. Examinations should differ only in the specific 

5 content covered, not in their developmental processes, manner of sampling, item construction 

6 criteria, level of item bank use, or levels of knowledge required and difficulty. The 

7 standardization of the process creates consistency of measurement. Ideally, any two 

8 examinations that are developed using these procedures and guidelines and administered to 

9 the same group of examinees should produce comparable results; likewise, the results of any 

10 examination administered to different but similarly trained and qualified examinees should also 

11 yield comparable results.

12 The standardized examination development, administration, and grading procedures described 

13 in this NUREG have evolved over a period of years in an effort to enhance the reliability and, 

14 hence, the validity of the NRC’s licensing decisions. The importance of having these 

15 procedures and complying with their intent has grown in proportion with the number of 

16 individuals and organizations that have become involved in the examination process.

17 Section 107 of the Atomic Energy Act of 1954, as amended, requires the Commission to 

18 prescribe uniform licensing conditions for operators. Therefore, the NRC expects facility 

19 licensees to develop and submit their proposed examinations based on the guidelines and 

20 instructions contained here. The NRC discourages facility licensees from using testing 

21 methodologies that do not conform to the policies, procedures, and practices defined in this 

22 NUREG. Nonetheless, facility licensees may propose alternatives to specific guidance in 

23 NUREG-1021, and the NRC will review and rule on the acceptability of those alternatives.

24 D. The Importance of the Written Examination

25 Society has institutionalized written examinations as an accepted and important facet of 

26 performance testing, and these examinations are used routinely as an integral factor in 

27 measuring human performance in nearly every field of study. Educational institutions from 

28 elementary through graduate schools use written examinations as all or part of an assessment

29 to measure intended competencies. Moreover, many fields of business, including the legal, 

30 medical, education, and accounting professions, use written examinations for licensing and 

31 credentialing activities.

32 The importance of knowledge testing should not be underestimated, because knowledge is the 

33 underpinning of professional performance. The objectives of knowledge testing are varied; they 

34 may include assessing fundamental understanding, as well as testing more advanced levels of 

35 expertise. The most effective tests of knowledge include questions and test items that measure 

36 the application of knowledge that directly relates to an individual’s job. In the case of operator 

37 licensing, the NRC’s written examination yields a key measure that allows the agency to make a 

38 confident decision about the safety-significant performance of the individual seeking a license.

39 Deemphasizing or sidestepping knowledge testing through careless or simplistic testing 

40 processes or treating it secondarily to other portions of the examination that are more 

41 operationally oriented could affect subsequent job performance. Failing to focus on testing the 

42 individual operator’s cognitive abilities (i.e., comprehension, problem-solving, and 

43 decisionmaking) or paying insufficient attention to the operator’s fundamental understanding of 

44 job content (e.g., systems, components, and procedures) may ultimately place job performance 

45 at risk of gradual degradation. When the demand for disciplined learning and study declines or 
A-11

1 the level of knowledge (depth of application) required for the job is reduced, it could lead to less 

2 time spent in training preparation, less mental review and practice, more forgetting of factual 

3 details, less reinforcement and application of job concepts, and a gradual decline in 

4 performance.

5 Moreover, without a solid fundamental knowledge base, operators may not perform acceptably 

6 in situations that procedures do not specifically address. Since every performance has an 

7 underlying knowledge component, that knowledge and its depth need development and 

8 assessment to ensure the operators’ competence on the job. Studies assessing mental 

9 performance in cognitively demanding emergencies point out that higher level cognitive thought 

10 (such as event diagnosis and response planning) are important in responding to safety-related 

11 events.

12 E. Objective Versus Subjective Test Items

13 Traditionally, questions that require the examinee to supply an answer (e.g., short answer and 

14 essay) have been considered “subjective,” while questions requiring the examinee to select an 

15 answer (e.g., multiple choice) have been considered “objective.” These terms arose from the 

16 scoring of the items. If graders need to have subject matter expertise to interpret the answers, 

17 the question is considered subjective. By contrast, if graders can score the examination by 

18 verifying a single letter or number, the question is considered objective.

19 Multiple choice items are the most common and most popular of the select type items. For 

20 reasons of consistency and reliability, they are currently the only type of items acceptable for 

21 use on NRC initial licensing examinations. Although multiple choice items are not as easy to 

22 construct as other question forms, they are very versatile, can be used to test for all levels and 

23 types of knowledge, and minimize the likelihood that the examinee will obtain the correct answer 

24 by guessing. Scoring multiple choice examinations is also considerably more reliable and less 

25 time consuming than scoring open-ended response items. Furthermore, since each item 

26 requires less time to answer, more items can be used to test a larger sample of K/As. This 

27 provides better content coverage and increases test reliability.

28 For the purposes of NRC requalification examinations and initial operating tests, the definition of 

29 “objective” differs from the traditional definition discussed above. In this case, an objective test 

30 item is one for which (1) there is only one correct answer, and (2) all qualified graders would 

31 agree on the amount of credit allowed for any answer.

32 Therefore, all questions on NRC examinations shall be objectively gradable, regardless of the 

33 item format. Questions with no single correct answer or for which the credit given can vary, 

34 depending on who graded it or when it was graded, have no place on an NRC examination.

B-1

1 APPENDIX B

2 EXAMPLES OF WRITTEN EXAMINATION QUESTIONS 

3 Sections A – G of this appendix contain examples of written examination questions that illustrate 

4 the psychometric principles. Section H of this appendix contains examples of satisfactory and 

5 unsatisfactory Senior Reactor Operator (SRO)- only questions. 

6 

7 A. Levels of Knowledge

8 The first three examples in this category illustrate how the level of knowledge tested can vary 

9 among a series of questions that focus on the same pair of knowledge and ability (K/A)

10 statements. Even though the K/A statements below use verbs (i.e., identify, define) that elicit a 

11 fundamental or simple memory level of knowledge, the item author can increase their 

12 operational validity by testing at a higher cognitive level:

13 • 19104K101 (pressurized-water reactor (PWR)) or 291004K101 (boiling-water reactor

14 (BWR)): identification, symptoms, and consequences of cavitation

15

16 • 193006K111 (PWR) or 293006K109 (BWR): definition or explanation of cavitation

17 1. Fundamental Knowledge/Simple Memory

18 Which one of the following describes pump cavitation?

19 a. Vapor bubbles form when the enthalpy difference between pump discharge and

20 pump suction exceeds the latent heat of vaporization.

21 b. Vapor bubbles form in the eye of the pump and collapse as they enter higher

22 pressure regions of the pump.

23 c. Vapor bubbles are produced when the localized pressure exceeds the vapor

24 pressure at the existing temperature.

25 d. Vapor bubbles are discharged from the pump, where they impinge on downstream

26 piping and cause a water hammer.

27 This question simply asks for a description of cavitation and, as such, is a “low cognitive 

28 order” question that does not require any understanding, analysis, or problem-solving. The 

29 examinee merely needs to recognize the correct description (b); the other options appear 

30 plausible but are, nonetheless, incorrect.

31 2. Comprehension

32 Cavitation in an operating pump may be caused by:

33 a. lowering the pump suction temperature

34 b. throttling the pump suction valve

35 c. increasing the pump back-pressure

36 d. increasing the pump suction pressure

37 This example requires the examinee to determine causation, which requires an 

38 understanding of the correct answer and recognition that the incorrect answers are, indeed, 
B-2

1 incorrect. As with any item, the quality of this item is determined by the ability of the 

2 incorrect options to distract from the correct answer.

3 3. Analysis

4 While on surveillance rounds, an operator notices that a centrifugal pump is making a great 

5 deal of noise (like marbles rattling inside the pump casing), and the discharge pressure is 

6 fluctuating. This set of conditions indicates pump:

7 a. runout

8 b. cavitation

9 c. bearing deterioration

10 d. packing deterioration

11 This example requires the candidate to analyze multiple abnormal indications (multiple 

12 effects) for an operating centrifugal pump and determine the cause (complex cause-effect). 

13 All the distractors are initially plausible in that they have face validity (i.e., they have 

14 reasonable connections to centrifugal pump operation).

15 4. Low Level of Knowledge Examples 

16 The following four examples illustrate questions requiring a low level of knowledge, which 

17 should be used judiciously on U.S. Nuclear Regulatory Commission (NRC) examinations.

18 Which one of the following is powered from 4,160 VAC bus 1A?

19 a. residual heat removal (RHR) pump A

20 b. RHR pump B

21 c. RHR pump C

22 d. RHR pump D

23 Select the full core display indication of a drifting control rod:

24 a. red light

25 b. white light

26 c. blue light

27 d. amber light

28 Although the above items have high K/A values, they are written at a low level of knowledge 

29 and have low operational validity and low discriminatory value. The following question tests 

30 at a low level of knowledge because it does not test the examinee’s ability to recognize the 

31 class of fire or select the correct extinguisher.

32 Concerning the use of water as a fire extinguishing agent, select the correct statement from 

33 the following:

34 a. It is the primary agent for extinguishing Class A fires and is also effective on Class B 

35 and C fires.

36 b. It is the primary agent for extinguishing Class B fires and is also effective on Class A 

37 and C fires.

38 c. It is the primary agent for extinguishing Class A and B fires but is not effective on 
B-3

1 Class C fires.

2 d. It is the primary agent for extinguishing Class B and C fires but is not effective on 

3 Class A fires.

4 The next question might be considered a fundamental knowledge-level question that errs in 

5 the opposite direction (i.e., it could be too difficult unless the operators are expected to 

6 memorize the correct time requirement to prevent damage to equipment). Moreover, this 

7 item may also have low discriminatory validity unless at least 80 percent of the examinees 

8 are expected to know the answer from memory.

9 RCP 2A tripped after running for 50 minutes. The RCP was restarted but tripped again 

10 within 15 seconds. Which ONE of the following is the minimum required interval before the 

11 next attempt to start RCP 2A?

12 a. 15 minutes

13 b. 30 minutes

14 c. 45 minutes

15 d. 60 minutes

16 B. Low Operational Validity

17 The next three questions illustrate another common psychometric deficiency, known as low 

18 operational validity, which should be avoided on NRC examinations.

19 1. Under which one of the following conditions should the shift supervisor inform the 

20 shop steward? 

21 a. initiation of a directed overtime request

22 b. disciplinary action against a supervisory employee

23 c. medical injury of a contractor employee

24 d. personnel error by a bargaining unit member

25 While this question may be related to a shift supervisor’s job, it has nothing to do with 

26 nuclear safety and should not be included on an NRC examination.

27 2. Which one of the following main steam line components is designed to limit the 

28 differential pressure across the steam dryer assembly?

29 a. main steam line flow elbows

30 b. main steam isolation valves

31 c. main steam shutoff valves

32 d. main steam line flow restrictors

33 Knowing the purpose of a flow restrictor is not a good indicator of the operator’s ability to 

34 operate the plant. Thus, knowing the answer to this question is not clearly job related.

35 3. Given that all components controlled by the “Locked Valve, Breaker, and Component 

36 Control” administrative procedure must be properly sealed and tagged, which one of 

37 the following is the correct location for the “XXXX-XXXX” tag for an electrical 

38 breaker?

39 a. wired to the breaker handle
B-4

1 b. glued to the breaker cubicle

2 c. attached to the breaker cubicle with a magnetic clip

3 d. wired to the breaker cabinet door

4 This question is likely unrelated to the reactor operator’s (RO’s) job function and, therefore, 

5 would be unacceptable.

6 C. Low Discriminatory Validity

7 The next three questions illustrate another common psychometric deficiency, known as low 

8 discriminatory validity, which should be avoided on NRC examinations.

9 1. Which one of the following reactor water levels will initiate the RHR pumps?

10 a. level 1 only

11 b. levels 1 and 2 only

12 c. levels 1 and 2 and 3 only

13 d. level 6 only

14 The information in this question should be known by all operators at all times. Therefore, 

15 the question has low discriminatory value and also tests at a low level of knowledge.

16 2. The plant is recovering from a scram that resulted from a spurious Group I isolation. 

17 The cause of the isolation has been repaired, and preparations are being made to 

18 reopen the main steam isolation valves (MSIVs). Reactor pressure is currently 825 

19 psig and the main steam lines are being pressurized.

20 WHICH ONE of the following is the LOWEST main steam line pressure that will allow 

21 the MSIVs to be opened in accordance with the procedure?

22 a. 625 psig

23 b. 675 psig

24 c. 725 psig

25 d. 775 psig

26 Similar to the previous example, this question also has low discriminatory validity.

27 3. SG (corrected) = SG (uncorrected) + (T - 77 degrees F)(0.001) + (Level Mark)(0.003)

28 3

29 Based on the above information, the specific gravity (SG) is ? , which ? 

30 meet the technical specification (TS) Category A limit. Note: This question requires 

31 the use of TS 3.8.2.3.

32 a. 1.198, does NOT

33 b. 1.195, does NOT

34 c. 1.207, does

35 d. 1.201, does

36 This question might appear to test the examinees’ ability to understand and apply battery 

37 parameters to the determination of TS operability. However, the question really only tests 

38 their ability to substitute certain parameters into a given equation and perform an arithmetic 
B-5

1 calculation. Reference to the TS noted in the question is not required based on the three 

2 different values of SG (corrected) supplied as distractors. Therefore, the question has a low 

3 discriminatory value because any individual possessing adequate arithmetic knowledge will 

4 arrive at the correct answer.

5 D. Implausible Distractors

6 The next two questions illustrate the concept of implausible distractors, which is another 

7 common psychometric deficiency that should be avoided on NRC examinations.

8 1. Which of the following will cause the RHR pumps to start during a design-basis 

9 loss-of-coolant accident (LOCA)?

10 a. low drywell pressure

11 b. high reactor water level

12 c. high drywell pressure

13 d. MSIVs in the NOT OPEN position

14 Distractors “a,” “b,” and “d” are implausible, considering minimal knowledge of the plant 

15 response to a LOCA.

16 2. Which ONE of the following conditions will NOT result in a shutdown of the standby 

17 gas treatment system?

18 a. manual shutdown

19 b. high-temperature (107 degrees C, 225 degrees F) charcoal bed

20 c. high-temperature (82 degrees C, 180 degrees F) heater inlet

21 d. overloads in the local control panel

22 Distractor “a” is very implausible, and distractor “d” is subjective. The question is also 

23 written from a negative perspective.

24 E. Confusing Language

25 The following questions illustrate how confusing language and inappropriate negatives in the 

26 stem of the question can mislead examinees. NRC examinations should avoid such questions.

27 1. Which of the following parameters will start the high-pressure coolant injection, 

28 reactor core isolation cooling, or standby gas treatment systems?

29 a. low reactor water level

30 b. high primary containment pressure

31 c. high reactor building exhaust radiation

32 d. low reactor building differential pressure

33 This question could result in four correct answers, since the question could be interpreted 

34 individually or collectively.

35 2. Which ONE of the following most accurately describes the response to a static 

36 inverter failing?

37 a. The power supply will automatically transfer to the alternate 600-V Bus 2C/Vital AC 

38 Transformer 2A.
B-6

1 b. The 125-V DC battery will maintain power to the Vital AC Cabinet for up to 5 hours.

2 c. The power supply can be manually transferred to the alternate 

3 600-V Bus 2C/Alternate Static Inverter by pressing a transfer pushbutton.

4 d. The power supply can be manually transferred to the alternate 600-V Bus 2C/Vital 

5 AC Transformer 2A by positioning the transfer switch to ALTERNATE.

6 This question implies an automatic response, but the listed correct answer and one 

7 distractor are operator actions, not responses to the loss of the static inverter.

8 3. With regard to temporary plant alterations (TPAs), technical reviews are NOT 

9 required for the following:

10 a. a TPA NOT installed using an approved procedure

11 b. TPAs installed on balance-of-plant systems, BUT they ARE required for 

12 safety-related systems

13 c. a TPA that has NOT been directed by the shift supervisor to be an emergency TPA

14 d. all TPAs directed by the shift supervisor

15 This question contains two main problems. First, although negative questions can be used, 

16 they should be used for good reason; there appears to be no good basis for asking this 

17 question negatively. Second, two of the distractors (“a” and “c”) also contain a negative, 

18 thus creating a double negative with readability confusion, which violates good item writing 

19 practice. The question should more appropriately ask the conditions under which technical 

20 reviews are required, thereby eliminating the negative in the stem.

21 F. Collection of True/False Statements

22 Collections of true/false statements typically only test simple rote memory; the examinee simply 

23 needs to recall a definition or condition. The questions elicit no comprehension or 

24 problem-solving; hence, they lack operational validity. This type of question allows an examinee 

25 to answer the question without referring to the stem of the question and should be avoided on 

26 NRC examinations.

27 1. Which ONE of the following is true?

28 a. High drywell pressure will automatically start the emergency diesel generators.

29 b. Low reactor water level will trip the main turbine.

30 c. High reactor pressure will initiate reactor core isolation cooling.

31 d. High reactor power with the mode switch in startup will NOT close the MSIVs.

32 2. Which one of the following describes pump cavitation? 

33 a. Vapor bubbles form when the enthalpy difference between pump discharge and 

34 pump suction exceeds the latent heat of vaporization.

35 b. Vapor bubbles form in the eye of the pump and collapse as they enter higher 

36 pressure regions of the pump.
B-7

1 c. Vapor bubbles are produced when the localized pressure exceeds the vapor 

2 pressure at the existing temperature.

3 d. Vapor bubbles are discharged from the pump where they impinge on downstream 

4 piping and cause a water hammer.

5 G. Backward Logic

6 Backward logic questions ask the examinee for information that is normally received while 

7 providing the examinee with information that he or she normally has to supply. In an operational 

8 setting, operators are faced with conditions and are required to know which procedure(s) to use. 

9 These questions ask them to do just the opposite and, therefore, should be avoided on NRC 

10 examinations.

11 1. Which of the following parameters will simultaneously start high-pressure coolant 

12 injection, reactor core isolation cooling, and the standby gas treatment system?

13 a. high reactor pressure vessel water level

14 b. high drywell pressure

15 c. low reactor pressure vessel water level

16 d. low drywell pressure

17 It would be better to select a parameter and then request the expected system response 

18 because that is more operationally relevant.

19 2. If it takes 0.354 cubic meters (12.5 cubic feet) of concrete to build a square loading 

20 pad that is 6 inches thick, what is the length of one side of the pad?

21 This question gives the examinees information they should be asked to calculate, while it 

22 requires them to provide information they would be given in an actual work situation.

23 H. Senior Reactor Operator-Only Questions

24 1. Examples of Satisfactory Senior Reactor Operator-Only Questions

25 a. Westinghouse: E07 Saturated Core Cooling

26 EA2.2: Ability to determine and interpret the following as they apply to the saturated core 

27 cooling: Adherence to appropriate procedures and operation within the limitations in the 

28 facility’s license and amendments. (CFR: 43.5 / 45.13): 3.3/3.9

29 A steam generator tube rupture has occurred, and the crew is performing actions contained 

30 in Emergency Operating Procedure (EOP) 3.1, “SGTR with Loss of Reactor Coolant—

31 Subcooled Recovery.” The following plant conditions currently exist:

32 • All critical safety function (CSF) status trees are GREEN except:

33 − core cooling—YELLOW due to reactor vessel level instrumentation system 

34 (RVLIS) level

35 − inventory—YELLOW due to RVLIS level

36 • The crew has determined that the RHR sump level (based on refueling water storage 
B-8

1 tank drawdown) is LESS than expected.

2 Which ONE of the following identifies the correct implementation of procedures for this 

3 event? 

4 A. Transition to EOP 3.2, “SGTR with Loss of Reactor Coolant—Saturated Recovery.” 

5 Implementation of the CSF Yellow Path procedures is not allowed while in EOP 3.2.

6 B. Remain in EOP 3.1.

7 Implementation of the CSF Yellow Path procedures is not allowed while in EOP 3.1.

8 C. Transition to EOP 3.2.

9 The actions of both Yellow Path procedures may be performed if desired.

10 D. Remain in EOP 3.1.

11 The actions of both Yellow Path procedures may be performed if desired.

12 Justification: The question requires the applicant to assess plant conditions and to know the 

13 content of procedures in order to select a required course of action. Linked to 

14 10 CFR 55.43(b)(5).

15 b. EPE: 295028 High Drywell Temperature

16 EA2.01: Ability to determine or interpret (or both) the following as they apply to HIGH 

17 DRYWELL TEMPERATURE: Drywell temperature. (CFR: 41.10 / 43.5 / 45.13): 4.0*/4.1*

18 Following a small-break LOCA on Unit 2, the following conditions exist: 

19 • drywell temperature 270 degrees F

20 • drywell pressure 5.0 psig

21 • torus pressure 2.5 psig

22 • torus level +5 inches

23 • reactor pressure 395 psig

24 Containment H2O2 Monitors CAC-AT-4409 and -4410 are not available at this time. The 

25 Chemistry staff has been notified but has not yet sampled the drywell.

26 Which ONE of the following procedures provides the required actions that mitigate these 

27 plant conditions?

28 A. SEP-05, “Primary Containment Purging”

29 B. SEP-10, Section 4, “Defeating Drywell Cooler LOCA Lockout”

30 C. SEP-03, “Suppression Pool Spray Procedure”

31 D. SEP-02, “Drywell Spray Procedure”

32 Justification: The question requires the applicant to assess plant conditions and to know the 

33 content of procedures in order to select a required course of action. These procedures are 

34 not major EOPs (i.e., they are supplementary emergency procedures directed from within 

35 the major EOP). Linked to 10 CFR 55.43(b)(5).

36 c. Generic APE: 027 Pressurizer Pressure Control System (PZR PCS) Malfunction

37 AA2.15: Ability to determine and interpret the following as they apply to the pressurizer 
B-9

1 pressure control malfunctions: actions to be taken if the PZR pressure instrument fails high. 

2 (CFR: 43.5 / 45.13): 3.7/4.0

3 Unit 1 initial conditions:

4 • time = 10:00

5 • reactor power = 100%

6 • 1-RC-PORV-1455C (PZR Pressure PORV) indicates open

7 • both PZR spray valves indicate open

8 • RCS pressure = 2,200 psig decreasing

9 • 1-AP-31.00 (Increasing or Decreasing RCS Pressure) initiated

10 Current conditions:

11 • time = 10:01

12 • reactor power = 97%

13 • RCS pressure = 2,100 psig increasing

14 • spray valve in MANUAL and closed

15 • 1-RC-PORV-1455C in MANUAL and closed

16 Based on these conditions, which ONE of the following identifies (1) the PZR pressure 

17 control component that failed high and (2) the status of 1-RC-PORV-1455C operability in 

18 accordance with TS?

19 A. 1-RC-PT-1444

20 Power-operated relief valve (PORV) is OPERABLE.

21 B. 1-RC-PT-1444

22 PORV is INOPERABLE.

23 C. 1-RC-PT-1445

24 PORV is OPERABLE.

25 D. 1-RC-PT-1445

26 PORV is INOPERABLE.

27 Justification: The first part of the question can be answered using RO knowledge of 

28 systems. The second part of the question can only be answered by a senior reactor 

29 operator (SRO) applicant if he/she knows the information in the TS bases. No reference 

30 was provided. This question is linked to 10 CFR 55.43(b)(5).

31 d. Generic K/A G2.2.6: Knowledge of the process for making changes to procedures.

32 (CFR: 41.10 / 43.3 / 45.13): 3.0/3.6

33 The plant has developed a new surveillance test procedure with the following attributes:

34 • The test procedure involves a process that was NOT previously described in the final 

35 safety analysis report.

36 • The test procedure does NOT constitute an unreviewed safety question.

37 • The test procedure will require a change to TS.

38 Which ONE of the following identifies whether a license amendment is required and whether 
B-10

1 the surveillance test procedure can be implemented without NRC approval in accordance 

2 with 00056-C, 10 CFR 50.59 screening and evaluation?

3 A. License amendment is NOT required; NRC approval is NOT required.

4 B. License amendment is NOT required; NRC approval is required.

5 C. License amendment is required; NRC approval is NOT required.

6 D. License amendment is required; NRC approval is required.

7 Justification: The question is linked to one of the duties reserved for the SRO licensed 

8 individual (i.e., 10 CFR 55.43(b)(5) (procedures used to obtain authority for design and 

9 operating changes to the facility)).

10 2. Examples of Unsatisfactory Senior Reactor Operator-Only Questions

11 a. APE: 008 Pressurizer (PZR) Vapor Space Accident (Relief Valve Stuck Open) 

12 AA2.22: Ability to determine and interpret the following as they apply to the pressurizer 

13 vapor space accident: consequences of loss of pressure in the RCS; methods for 

14 evaluating pressure loss. (CFR: 43.5 / 45.13): 3.8/4.2

15 • A pressurizer steam space LOCA has caused PPLS and SIAS actuation.

16 • CETs are stable at 550 degrees F.

17 • RCS pressure is stable at 1,300 psia.

18 • Pressurizer level is 20 percent and rising.

19 • High-pressure safety injection (HPSI) flow is 390 gpm.

20 With no operator action and assuming temperatures remain constant, how will pressurizer 

21 level, pressurizer pressure, and HPSI flow respond?

22 A. Pressurizer level will stabilize slightly above 20 percent, pressure will lower, and 

23 HPSI flow will increase.

24 B. Pressurizer level will rise to 100 percent, pressure and HPSI flow will remain 

25 constant.

26 C. Pressurizer level will rise to 100 percent, pressure will rise and HPSI flow will 

27 decrease.

28 D. Pressurizer level will stabilize slightly above 20 percent, pressure will rise, and HPSI 

29 flow will decrease.

30 Justification: The question stem does not link to one of the seven 10 CFR 55.43(b) 

31 statements even though the K/A is linked to 10 CFR 55.43(b)(5). The question only tests 

32 the assessment of plant conditions. An RO is expected to understand integrated system 

33 response.

34 b. Generic K/A G2.1.7: Ability to evaluate plant performance and make operational 

35 judgments based on operating characteristics, reactor behavior, and instrument 

36 interpretation. (CFR: 41.5 / 43.5 / 45.12 / 45.13): 4.4/4.7

37 Reactor power is 29 percent during a reactor startup when the reactor operator trips the 

38 main turbine due to high vibration. Which ONE of the following identifies the required 
B-11

1 procedures?

2 The SRO should now anticipate implementing procedures that will do the following:

3 A. Maintain reactor power less than 29 percent since power will increase after the main 

4 turbine trip.

5 B. Recover from the reactor scram caused by the turbine trip.

6 C. Recover vessel level using the feed and condensate system.

7 D. Scram the reactor.

8 Justification: The question asks for plant response and what to do about it, NOT selection 

9 or application of a procedure. An applicant can answer the question using integrated plant 

10 and system knowledge (i.e., knowledge that is not unique to the SRO). 

11 c. APE: 065 Loss of Instrument Air

12 AA2.06: Ability to determine and interpret the following as they apply to the loss of 

13 instrument air: When to trip the reactor if instrument air pressure is decreasing. (CFR: 43.5 

14 / 45.13): 3.6*/4.2

15 Unit 1 is currently at 82 percent power. A down power is in progress to remove the 1A main 

16 feedwater pump from service. Which ONE of the following plant conditions would require 

17 you to direct an IMMEDIATE manual trip of the reactor?

18 A. Instrument air pressure is currently 59 psig and lowering.

19 B. 1A and 1B SG levels are 75 percent and increasing.

20 C. BOTH heater drain pumps trip.

21 D. 4.16-kV bus 1B3 deenergizes due to an electrical fault on the bus.

22 Justification: The SRO is responsible for directing the action to trip the reactor; however, the 

23 RO is still required to know immediate reactor trip criteria listed in the abnormal event 

24 procedure. Just because the SRO directs the action does not mean that the knowledge is 

25 unique to the SRO position.

26 d. K/A 007: Pressurizer Relief Tank/Quench Tank System

27 G2.2.44: Ability to interpret control room indications to verify the status and operation of a 

28 system and understand how operator actions and directives affect plant and system 

29 conditions. (CFR: 41.5 / 43.5 / 45.12): 4.2/4.4

30 Given the following—

31 • Unit 1 is at 100-percent power; RCS pressure indicates 2,225 psig and stable.

32 • 1B-F1, PRZ RELIEF TK HI PRESS alarm is received.

33 • Pressurizer relief tank (PRT) pressure indicates 14 psig and rising slowly.

34 • PRT temperature is 92 degrees F and stable.

35 • PRT level is 70 percent and stable.

36 Which ONE of the following describes the appropriate operator response?
B-12

1 A. Ensure PZR PORVs are closed and PG and N2 to PRT are isolated. 

2 Go to 1-AP-16, “Increasing Primary Plant Leakage.”

3 B. Ensure PZR PORVs are closed and PG and N2 to PRT are isolated.

4 Vent the PRT in accordance with 1-OP-5.7, “Operation of the Pressurizer Relief 

5 Tank.”

6 C. Submit a WR. Verify PG water alignment and cool the PRT by draining and refilling 

7 in accordance with 1-OP-5.7.

8 D. Submit a WR. Cool the PRT by draining and refilling in accordance with 1-OP-5.7. 

9 Refer to 1-AP-16.

10 Justification: Each choice required the selection of procedures. However, this is not 

11 SRO-only knowledge because the choices also include responsive actions, which an RO 

12 can deduce using systems knowledge; therefore, procedure selection is not actually 

13 required to answer the question.

14
B-13

1 I. Bibliography

2 Bloom, B., Taxonomy of Educational Objectives: The Classification of Educational Goals, 

3 Handbook I: Cognitive Domain, David McKay Company, New York, NY, 1956.

4 Isaac, S., Handbook in Research and Evaluation: A Collection of Principles, Methods, and 

5 Strategies Useful in the Planning, Design, and Evaluation of Studies in Education and the 

6 Behavioral Sciences, 3rd edition, Edits Publishers, San Diego, CA, 1995.

7 Novac, J., and Gowin, D., Learning How to Learn, Cambridge University Press, New York, NY, 

2002. 8 

9 Ehrhardt, F., “SRO Written Examination Questions,” U.S. Nuclear Regulatory Commission

10 Region II Examiner’s Workshop, July 19–20, 2006 (Agencywide Documents Access and 

11 Management System (ADAMS) Accession No. ML062060498). 

12 “Operator Licensing Program Feedback,” March 24, 2016 (ADAMS Accession 

13 No. ML16084A735 (questions 401.29, 401.30, 401.35, 401.36, and 401.37)).

14




NUREG-1021, Rev. 12

Draft

Operator Licensing Examination Standards for Power Reactors