UNITED STATES

NUCLEAR REGULATORY COMMISSION 

WASHINGTON, D.C. 20555-0001 

October 6, 2016 

MEMORANDUM TO: Nancy Salgado, Chief

Operator Licensing and Training Branch 

Division of Inspection and Regional Support 

Office of Nuclear Reactor Regulation 

FROM: David Muller, Reactor Engineer /RA/ 

 Operator Licensing and Training Branch 

 Division of Inspection and Regional Support 

 Office of Nuclear Reactor Regulation 

SUBJECT: SUMMARY OF THE AUGUST 30, 2016, PUBLIC MEETING WITH 

THE NUCLEAR ENERGY INSTITUTE’S OPERATOR LICENSING 

FOCUS GROUP 

On August 30, 2016, the U.S. Nuclear Regulatory Commission (NRC) staff held a public 

meeting with the Nuclear Energy Institute’s Operator Licensing Focus Group. The purpose of 

this meeting was for the NRC to provide a status update for Revision 11 to NUREG-1021, 

“Operator Licensing Examination Standards for Power Reactors.” The NRC did not solicit 

additional comments on NUREG-1021 during this meeting, since the 60-day public comment 

period on Draft Revision 11 to NUREG-1021 closed on May 6, 2016. 

This meeting was the latest in a series of meetings intended to promote efficiency, 

effectiveness, and open communications with regard to initial operator licensing. 

Representatives of the NRC and the industry agreed that this meeting was useful for the 

exchange of information and agreed to continue the periodic meetings. 

Enclosures: 

1. List of Attendees 

2. Agenda 

3. Discussion Summary 

CONTACT: David Muller, NRR/DIRS/IOLB 

 (301) 415-1412 

 
 

MEMORANDUM TO: Nancy Salgado, Chief

Operator Licensing and Training Branch 

Division of Inspection and Regional Support 

Office of Nuclear Reactor Regulation 

FROM: David Muller, Reactor Engineer 

 Operator Licensing and Training Branch 

 Division of Inspection and Regional Support 

 Office of Nuclear Reactor Regulation 

SUBJECT: SUMMARY OF THE AUGUST 30, 2016, PUBLIC MEETING WITH 

THE NUCLEAR ENERGY INSTITUTE’S OPERATOR LICENSING 

FOCUS GROUP 

On August 30, 2016, the U.S. Nuclear Regulatory Commission (NRC) staff held a public 

meeting with the Nuclear Energy Institute’s Operator Licensing Focus Group. The purpose of 

this meeting was for the NRC to provide a status update for Revision 11 to NUREG-1021, 

“Operator Licensing Examination Standards for Power Reactors.” The NRC did not solicit 

additional comments on NUREG-1021 during this meeting, since the 60-day public comment 

period on Draft Revision 11 to NUREG-1021 closed on May 6, 2016. 

 

This meeting was the latest in a series of meetings intended to promote efficiency, 

effectiveness, and open communications with regard to initial operator licensing. 

Representatives of the NRC and the industry agreed that this meeting was useful for the 

exchange of information and agreed to continue the periodic meetings. 

Enclosures: 

1. List of Attendees 

2. Agenda 

3. Discussion Summary 

CONTACT: David Muller, NRR/DIRS/IOLB 

 (301) 415-1412 

DISTRIBUTION: 

CMiller, NRR NSalgado, NRR MScheetz, NRR DJackson, RI GMcCoy, RII 

GGuthrie, RII ROrlikowski, RIII VGaddy, RIV 

ADAMS Accession No.: ML16273A274 

OFFICE IOLB IOLB IOLB/BC 

NAME DMuller MScheetz NSalgado 

DATE 09/30/2016 9 /30/2016 10/6 /2016 

 OFFICIAL RECORD COPY
 

 Enclosure 1 

List of Attendees – Nuclear Regulatory Commission (NRC) / Nuclear Energy Institute’s 

Operator Licensing Focus Group (NEI OLFG) 

August 30, 2016 

Name Organization 

Chris Miller NRC 

Maurin Scheetz NRC 

Don Jackson NRC 

Tim Kolb NRC 

John Munro NRC 

David Muller NRC 

Gregg Ludlam NEI OLFG 

Chuck Sizemore NEI OLFG 

Russel Joplin NEI OLFG 

Scott Bauer NEI 

Roger Jones Entergy 

Steve Dennis PSE&G 

Michael Petersen Xcel Energy 

Mark Otten AmerGen 

Chuck Millard Exelon 

Matthew Henson Southern Nuclear Company 

Kostas Dovas Exelon 

Rich Meyers Energy Northwest 

Jana Bergman Curtiss-Wright Corporation 

Mike Schoppman Certrec 
2

List of Attendees via Teleconference – Nuclear Regulatory Commission (NRC) / 

Nuclear Energy Institute’s Operator Licensing Focus Group (NEI OLFG) 

August 30, 2016 

Name Organization 

Gerald McCoy NRC 

Gene Guthrie NRC 

Bob Orlikowski NRC 

Vince Gaddy NRC 

Carole Revelle NRC 

Ken Jenkins Vogtle Electrical Generating Plant 

Ed Jones and Hatch Staff Edwin I. Hatch Nuclear Plant 

Robert Meyer Public/Professional Reactor Operator 

Society 
 

 Enclosure 2 

AGENDA FOR THE CATEGORY 2 PUBLIC MEETING WITH THE NUCLEAR ENERGY 

INSTITUTE’S (NEI) OPERATOR LICENSING FOCUS GROUP (OLFG) 

Tuesday, August 30, 2016, 9:00 AM to 12:00 PM EST 

NRC Two White Flint North, Room T2B3 

11545 Rockville Pike 

Rockville, MD 20852 

Time Topic Speaker 

 

9:00 a.m. Introductions and Openings NRC/NEI 

9:20 a.m. Status of Major Changes to NUREG-1021 NRC 

11:00 a.m. Schedule for Implementation of NUREG-1021, Rev. 11 NRC 

11:30 a.m. Public Comments Public 

12:00 p.m. Adjourn 
Enclosure 3

DISCUSSION SUMMARY 

The NRC staff presented to the meeting attendees a status update for Revision 11 to 

NUREG-1021, “Operator Licensing Examination Standards for Power Reactors” (Rev. 11). 

During the NRC’s presentation, the staff discussed the resolution of public comments, 

publication schedules, and some items of interest contained in Rev. 11. Prior to the meeting, 

the attendees were informed via the public meeting notice of two related publically available 

documents, located in the Agencywide Documents Access and Management System (ADAMS): 

ADAMS No. ML16237A275, “NUREG-1021, Rev. 11, Status of Changes (August 2016)” 

ADAMS No. ML16237A279, “ES-201-1 Rev. 11” 

Introductions and Openings 

The NRC staff opened the meeting by stating that the overall goals of the meeting were to 

provide a current status of Rev. 11. Included in the meeting goals, the NRC stated that it would 

present resolutions of public comments received on some of the more significant changes 

contained in Rev. 11 (items of interest), and that the NRC would not be collecting comments on 

Rev. 11 at this meeting. 

Included in the NRC’s opening remarks was a brief discussion of the two driving forces for the 

changes contained in Rev. 11: lessons learned from a 2014 operator licensing decision made 

by the NRC’s Atomic Safety and Licensing Board (ASLB) (available at ADAMS No. 

ML14077A573), and NRC’s Project Aim (an initiative to improve the effectiveness, efficiency, 

and agility of the NRC). 

Status of Major Changes to NUREG-1021 

The NRC staff briefed the attendees on the current status of Rev. 11. Since the last meeting on 

this topic, held on March 31, 2016, (meeting minutes available at ADAMS No. ML16112A349) 

the staff has: 

• Reviewed and incorporated internal NRC comments on draft Rev. 11. 

• Reviewed and dispositioned over 300 public comments received on draft Rev. 11. 

• Incorporated many of the public comments received into changes to draft Rev. 11. 

• Determined that a small number of the public comments received were outside the 

intended scope of Rev. 11. 

 

The staff explained that significant changes made to draft Rev.11, in part to address public 

comments, were reviewed and dispositioned with input from all four NRC regions’ operations 

branches. With some administrative tasks still remaining, the staff stated the target date for 

publication of the final version of Rev. 11 was October 31, 2016. 

Following this presentation of the overall status of Rev. 11, specific items of interest associated 

with Rev. 11, including the staff’s resolution of comments on these items, were presented and 

discussed. 
2

Items of Interest: Scenario Critical Tasks, Revised Examination Preparation Timeline, and 

Revisions to the Knowledge and Abilities (K/A) Catalogs 

In response to public comments, the staff presented two changes to draft Rev. 11: a change 

from a lower limit to a target value for the metric of two critical tasks per simulator scenario, and 

the expansion of the timeline for examination preparation (Form ES-201-1) by two months. The 

staff stated that the expanded examination timeline will allow approximately 1.4 extra months for 

the initial preparation of an exam, and an additional 0.6 months for exam review and making 

exam changes (see ADAMS No. ML16237A279, “ES-201-1 Rev. 11”). The staff also presented 

that revised K/A catalogs (NUREGs-1122 and 1123) should be issued for public comments in 

approximately 30 days, with a target date of January 1, 2017, for final publication of the 

catalogs. 

The attendees acknowledged the information presented. 

Item of Interest: Frequency of Administration of the Generic Fundamentals Examination (GFE) 

The staff presented to the attendees that a change to draft Rev.11 was being planned to reduce 

the frequency of administration of the GFE from four times per year to twice per year beginning 

in 2017. The staff stated that a reduction in GFE administration frequency results in a budget 

savings to the NRC, and was an action proposed and approved as a part of NRC’s Project Aim. 

The staff further stated that they understood that the reduced GFE frequency would impact the 

scheduling of NRC site-specific examinations and nuclear facilities’ licensed operator training 

programs. 

Attendees and members of the Nuclear Energy Institute’s Operator Licensing Focus Group 

(OLFG) took issue with the reduction in GFE frequency, noting negative impacts in the 

scheduling of site-specific NRC licensing examinations and scheduling of licensed operator 

training. It was further stated that these impacts would result in additional monetary costs to the 

nuclear industry in preparing individuals for licensing. The OLFG questioned if passed GFE 

results could be considered valid in excess of the current two year shelf life, suggested 

incorporating GFE topics into NRC site-specific examinations, and suggested that the nuclear 

industry could develop and administer the GFE. 

The NRC staff noted these concerns, and stated that the two-year shelf life of the GFE is 

dictated by Title 10 of the Code of Federal Regulations (10 CFR 55.47), that adding GFE topics 

to NRC site-specific examinations was not desirable, and that other alternatives regarding the 

GFE were being considered in the near future. 

Item of Interest: Revised Simulator Scenario Grading Scheme for Individuals Taking NRC 

Licensing Examinations 

The NRC staff stated that no changes were planned to Rev. 11 regarding the revised simulator 

scenario grading scheme for individuals taking NRC licensing examinations. The staff then 

stated the major features of the revised grading scheme, which the NRC plans to implement 

with the issuance of the final version of Rev. 11: 

 

• A 0-3 point scale will be used for grading of rating factors in place of a 1-3 point scale. 

• A critical task error will result in a rating factor score of zero, but will not, on its own, 

result in examination failure. 

• A single performance deficiency can impact no more than two rating factors. 
3

• The practice of giving a point back for correct performance will be eliminated. 

• A passing score of 1.8 will be set for each competency (except for competency 4, 

“Communications”). 

The staff also reported on an NRC effort which applied the draft Rev. 11 revised grading 

scheme to NRC examinations administered in 2015 under Revision 10 to NUREG-1021. The 

following results were noted: 

• Per Revision 10 grading there were zero individuals who failed the simulator scenario 

portion of their 2015 NRC examination (out of 128 individuals who took the reactor 

operator (RO) examination and 138 individuals who took the senior reactor operator 

(SRO) examination). 

• Applying the draft Rev. 11 grading scheme, six individuals (2 ROs, 4 SROs) would have 

failed the simulator scenario portion of their NRC examination. 

• This indicated an increase in simulator scenario failure rates from 0% using Revision 10 

grading to 1.6% for ROs and 2.9% for SROs using draft Rev. 11 grading. 

The meeting attendees and OLFG acknowledged the information presented, but stated several 

concerns with the proposed Rev. 11 revised simulator scenario grading scheme, including: 

• A single individual failure of an NRC licensing examination represents a lost investment 

of hundreds of thousands of dollars for the nuclear industry. 

• The revised grading scheme contained in draft Rev. 11 does not address the problems 

identified by the ASLB in their 2014 operator licensing decision. 

• Changing to a 0-3 point scale from a 1-3 point scale for grading rating factors does not 

decrease subjectivity, nor does eliminating the point back practice. 

• Maintaining the same competency passing score of 1.8 with a grading scale for rating 

factors expanded downwards to zero effectively raises the standard for what is 

considered a competent operator. 

• The nuclear industry was not represented during the formulation of the revised simulator 

scenario grading scheme. 

 

The OLFG also questioned whether the NRC considered alternatives to the grading scheme 

contained in draft Rev. 11, such as only eliminating the point back practice or shifting to a 0-4 

grading scale. 

 

The NRC staff noted these concerns and stated that the revised grading scheme, including the 

elimination of the point back practice, was devised to reduce grading subjectivity, increase 

regional and examiner consistency, and address lessons learned from the 2014 ASLB decision. 

The staff further stated that at no time were failure rates of failing more individuals considered in 

the development of the revised simulator scenario grading scheme. 

Item of Interest: Simulator Scenario Overlap and Re-Use of Events 

The NRC staff informed the attendees of a change being made to draft Rev. 11 regarding the 

limits on simulator scenario overlap and event re-use. The staff stated that instead of specifying 

a 50% overlap limit of scenario events used on the past two NRC licensing examinations, the 

limit will be relaxed to require that each simulator scenario contain at least two events not used 

on the past two NRC licensing examinations. The staff stated that this relaxation of the limit for 

overlap and re-use of simulator scenario events should ease the development of simulator 
4

scenarios for future NRC licensing examinations. The staff further stated that these limits were 

being put in place to prevent NRC simulator examinations at a given nuclear power plant from 

being too similar from one examination to the next examination. 

 

The meeting attendees and OLFG acknowledged the information presented, but stated 

concerns with: 

• The added burden necessary to account for and track the usage of simulator scenario 

events. 

• The subjectivity involved in what is classified as a previously used versus unused 

event. 

• The possibility of not having enough different malfunctions to meet the event re-use 

limit. 

• The potential for examinees to have an advantage on NRC simulator exams, by being 

able to eliminate previously used scenario events. 

The NRC staff acknowledged these concerns. 

Item of Interest: Elimination of Informal NRC Staff Reviews (Appeals) of Examination Failures 

The NRC staff informed the attendees that a change being made to draft Rev. 11 was the 

elimination of appeal reviews for individuals who fail an NRC licensing examination and request 

an informal review. With this elimination, the staff would no longer accept appeals six months 

after the publication of final Rev. 11 to NUREG-1021. The staff stated that the elimination of 

appeals was an action approved as a part of NRC’s Project Aim, that post-examination 

comments from individual applicants and facilities should identify problematic test items, and 

that the post-examination comment period was being extended from 5 days to 20 days. The 

staff further stated that if an applicant fails an NRC licensing examination, the applicant can still 

request a formal hearing. 

The meeting attendees and OLFG acknowledged the information presented, but stated the 

following concerns regarding the elimination of appeals: 

• The results of simulator examinations will not be known within 20 days, so submitting 

post-examination comments on simulator exams will be problematic. 

• The appeal process has resulted in overturned license denials, which has saved the 

nuclear industry money and resources when compared to having individuals re-take an 

NRC examination. 

• The appeal process has served as an effective means for having a second look at 

examination failures and promoting NRC regional consistency. 

The OFG also question what will be the involvement of NRC headquarters in reviewing postexamination comments. 

The NRC staff noted these concerns and provided the following comments: 

• The appeal process is not required by NRC regulations. 

• Facilities and individuals should be able to identify problematic test items within the 

expanded 20-day post-examination comment period. 

• NRC headquarters may be involved in the review of post-examination comments and 
5

examination failures prior to issuing a license or sending a denial letter; however, the 

NRC regions will maintain the licensing authority. 

• The revised process will be faster than the current appeal process. 

• Individuals who passed with little margin will no longer have their licenses held while 

awaiting the results of an appeal. 

Item of Interest: Applicant Eligibility Waivers 

The NRC staff informed the meeting attendees that efforts to improve the processing applicant 

eligibility waiver requests was still a work in-progress. The staff noted that it is proving difficult 

to develop a process for waiver requests such that the NRC can issue a decision on a waiver 

prior to the receipt of a formal application (NRC Form 398) for a RO or SRO license. Members 

of the OLFG acknowledged this information, and questioned whether changes to NRC Form 

398 and the processing of this form should be considered. 

Item of Interest: SRO Administrative Job Performance Measures (JPMs) 

The NRC staff stated that it was proposing no changes to the draft Rev. 11 requirement that all 

five SRO administrative JPMs be at the SRO-level. Unlike Revision 10 to NUREG-1021, 

Rev. 11 will require all five SRO administrative JPMs to test at the higher license level, which 

means SRO administrative JPMs cannot exactly match administrative JPMs used to test ROs. 

The staff stated that this would add consistency to the content of NRC licensing examinations. 

The OLFG agreed that requiring a specific number of administrative JPMs at the SRO-level 

would add consistency, but questioned whether three out of five or four out of five JPMs at the 

SRO-level was a more appropriate requirement. The OLFG further stated that with the draft 

Rev. 11 requirement in-place, additional resources will be required to develop extra 

administrative JPMs that test at the SRO-level. 

Closing Remarks 

The NRC staff re-stated that the target date for publication of final Rev. 11 to NUREG-1021 was 

October 31, 2016, and that it would become fully effective 6 months after the publication date. 

The OLFG then provided some closing comments, including: 

• Their appreciation for the NRC in conducting this meeting. 

• The NRC has made very few changes to draft Rev. 11 although over 300 public 

comments were received. 

• The more significant changes contained in draft Rev. 11 will cost the nuclear industry 

millions of dollars, especially since a single individual who fails an NRC licensing 

examination represents a lost investment of up to half a million dollars. 

• Making significant changes to a sound and mature operator licensing program does not 

appear justified based upon a single ASLB decision. 

• The changes proposed in draft Rev. 11 have no impact or benefit to public safety. 

The OLFG also questioned whether the changes contained in Rev. 11 should be considered a 

backfit. 

Finally, comments from the public were provided, and included the following statements: 

• Every few years some kind of event identifies problems with operator licensing. 
6

• How the NRC and the nuclear industry responds to these events is important. 

• Every solution seems to lead to additional new problems. 

• Instead of reducing the frequency of the GFE, the GFE should be computer-based and 

on-demand, or let the nuclear industry develop the GFE. 

• A 2012 study on operator licensing conducted by the Institute for Nuclear Power 

Operations identified no changes necessary, yet based upon a single ASLB decision, a 

multitude of significant changes are being proposed. 

• The cumulative effect of multiple changes to operator licensing needs to be considered.