January 6, 2012 

MEMORANDUM TO: Frederick D. Brown, Director 

Division of Inspection and Regional Support 

Office of Nuclear Reactor Regulation 

FROM: John J. McHale, Chief /RA/ 

Operator Licensing and Training Branch 

Division of Inspection and Regional Support 

Office of Nuclear Reactor Regulation 

SUBJECT: SUMMARY OF DECEMBER 8, 2011, MEETING WITH INDUSTRY 

FOCUS GROUP ON OPERATOR LICENSING ISSUES 

On December 8, 2011, the U.S. Nuclear Regulatory Commission (NRC) staff held a public 

meeting with the Industry Focus Group (FG) on operator licensing to discuss a number of 

operator licensing issues. Enclosure 1 lists the attendees at the meeting. 

This meeting was the latest in a series of meetings intended to promote efficiency, effectiveness, 

and open communications with regard to initial operator licensing and licensed operator 

requalification. The discussions addressed a variety of issues, including an Industry FG initiative 

to examine possible changes to the knowledge and abilities (K/A) catalogs for licensed 

operators; potential changes to NUREG-1021, “Operator Licensing Examination Standards for 

Power Reactors;” and changes associated with the NRC’s recent revision to its procedure for 

inspecting licensed operator requalification programs (Inspection Procedure 71111.11). 

Enclosure 2 is the agenda for the meeting, and the discussion topics are summarized in 

Enclosure 3. As a part of the discussion associated with possible changes to the K/A catalogs, 

the Industry FG prepared a slide presentation, which is available in the NRC’s Agency 

Documents and Management System (ADAMS) at accession no. ML113500224. 

Representatives of the NRC and the industry agreed that this meeting was useful for the 

exchange of information and agreed to continue the periodic meetings. 

Enclosures: As stated 

CONTACT: David Muller, NRR/DIRS/IOLB 

 (301) 415-1412 
 

MEMORANDUM TO: Frederick D. Brown, Director 

Division of Inspection and Regional Support 

Office of Nuclear Reactor Regulation 

FROM: John J. McHale, Chief /RA/ 

Operator Licensing and Training Branch 

Division of Inspection and Regional Support 

Office of Nuclear Reactor Regulation 

SUBJECT: SUMMARY OF DECEMBER 8, 2011, MEETING WITH INDUSTRY 

FOCUS GROUP ON OPERATOR LICENSING ISSUES 

On December 8, 2011, the U.S. Nuclear Regulatory Commission (NRC) staff held a public 

meeting with the Industry Focus Group (FG) on operator licensing to discuss a number of 

operator licensing issues. Enclosure 1 lists the attendees at the meeting. 

This meeting was the latest in a series of meetings intended to promote efficiency, effectiveness, 

and open communications with regard to initial operator licensing and licensed operator 

requalification. The discussions addressed a variety of issues, including an Industry FG initiative 

to examine possible changes to the knowledge and abilities (K/A) catalogs for licensed 

operators; potential changes to NUREG-1021, “Operator Licensing Examination Standards for 

Power Reactors;” and changes associated with the NRC’s recent revision to its procedure for 

inspecting licensed operator requalification programs (Inspection Procedure 71111.11). 

Enclosure 2 is the agenda for the meeting, and the discussion topics are summarized in 

Enclosure 3. As a part of the discussion associated with possible changes to the K/A catalogs, 

the Industry FG prepared a slide presentation, which is available in the NRC’s Agency 

Documents and Management System (ADAMS) at accession no. ML113500224. 

Representatives of the NRC and the industry agreed that this meeting was useful for the 

exchange of information and agreed to continue the periodic meetings. 

Enclosures: As stated 

CONTACT: David Muller, NRR/DIRS/IOLB 

 (301) 415-1412 

DISTRIBUTION: PUBLIC BBoger JLubinski FBrown JMcHale 

RPelton DJackson MWidmann BCaballero MJunge MFranke 

HPeterson MHaire 

ADAMS Accession No. ML120040069 

OFFICE IOLB IOLB/BC 

NAME 

DMuller JMcHale 

DATE 1/6 /2012 1/6 /2012 

 OFFICIAL RECORD COPY
 ENCLOSURE 1 

List of Attendees - NRC / Industry Focus Group Meeting on Operator Licensing 

December 8, 2011 

Name Organization 

John (Jack) McHale NRC / HQ 

Mark Lintz NRC / HQ 

John Munro NRC / HQ 

Chris Cowdrey NRC / HQ 

David Muller NRC / HQ 

Tim Kolb NRC / HQ 

Rick Pelton NRC / HQ 

Rick DeVercelly NRC / TTC 

Don Jackson NRC / RI 

Bruno Caballero NRC / RII 

Hironori Peterson NRC / RIII 

Chuck Sizemore NextEra Energy 

Scott Bauer Nuclear Energy Institute (NEI) 

Charles Sawyer Duke Energy 

J.R. Steely Duke Energy 

Mike Llewellyn INPO 

Michael Petersen XCEL Energy 

Patrick Berry Entergy 

Roger Jones Entergy 

Mark VanSicklen Progress Energy 

Jerry Gauding (via telephone) PSE&G – Salem Nuclear Generating Station 

Mark Haire (via telephone) NRC / RIV 

Gregg Ludlam (via telephone) Exelon 

 
ENCLOSURE 2

AGENDA FOR THE CATEGORY 2 PUBLIC MEETING WITH THE NEI OPERATOR 

LICENSING FOCUS GROUP REGARDING TRAINING, EXAMINATION, AND INSPECTION 

PROCESSES 

Thursday, December 8, 2011 

1:00 p.m. to 5:00 p.m. Eastern Standard Time 

Nuclear Energy Institute 

1776 I St. N.W., Suite 400 

Washington, DC 20006 

TOPIC LEAD

Introductions and Opening Remarks NRC/NEI panel 

Public Comments Public 

Integrated examination proposals NEI 

K/A Catalog update NEI 

Potential NUREG-1021 changes NRC 

National Workshop for industry examiners NEI 

IP 71111.11 (Licensed Operator Requalification Program 

Inspection Procedure) changes NRC 

Public Comments Public 

Summary/Conclusions/Action Items NRC/NEI 
 ENCLOSURE 3 

DISCUSSION SUMMARY

K/A Catalog Update and Integrated Examination Proposals

Members of the Industry FG presented to the attendees some preliminary recommendations for 

changes to the current knowledge and abilities (K/A) catalogs (NUREGs-1122 and 1123) and 

changes to current initial licensing examinations. The Industry FG’s presentation included a 

slide show, available in the NRC’s Agency Documents and Management System (ADAMS) at 

accession no. ML113500224. In summary, the Industry FG recommended: 

• Identifying generic K/As more appropriately evaluated on the operating exam, and 

moving these K/As to a new generic section. 

• Removing 10 CFR references from K/A statements. 

• Standardizing common systems between the PWR and BWR catalogs and adding 

systems that may be missing. 

• Making the vendor specific emergency plant evolutions and abnormal plant evolutions 

more specific. 

• Re-rating all K/A importance ratings (IRs), and replacing reactor operator/senior reactor 

operator IRs with a single IR, except for the A2, generic, and fuel handling categories. 

• Removing K/As not applicable to licensed operators. 

• Removing the list of tasks provided for each plant system. 

• Revising the reactor operator written exam sampling methodology. 

• Identifying clear and specific criteria for rejecting K/As from the written exam. 

• Modifying the written exam topic selection criteria to more appropriately emphasize topics 

with higher safety significance. 

• Providing clear and specific guidance for allowing the use of references on initial 

licensing written exams. 

During the presentation, the NRC staff had several questions and comments, including: 

• If some generic K/As were moved to a new section to be more appropriately tested on 

the operating test, would these K/As not be sampled or tested on written examinations? 

The Industry FG did not appear to have a definitive answer to this question, in that 

flexibility may be desirable when testing these K/As, but perhaps these K/As should not 

be sampled or tested on the written exam. 

• If a K/A was sampled for the written exam, but then rejected because it was better tested 

on an operating test, will it be required then to test this K/A on that exam’s operating 

test? Or could the K/A not be tested at all for that exam? The Industry FG stated that it 

would depend upon the circumstances, and that a decision could be made not to test a 

sampled and rejected K/A at all. 

• The NRC stated that adequate guidance already exists in NUREG-1021 for rejecting 

K/As on the written exam, but the NRC also understands that perhaps additional 

clarification to that guidance may be warranted. 

• The NRC cautioned the Industry FG about re-rating all the K/As, in that knowledge and 

abilities important to a licensed operator (e.g., operating the main turbine) may not 

always be associated with preventing or mitigating core damage (i.e., safety 

significance). In re-rating K/As, safety significance should not be the only consideration. 

The NRC also cautioned the Industry FG about over-emphasizing safety significance in
2 

revising the sampling selection process for K/As. The Industry FG acknowledged the 

NRC concerns. 

• The NRC stated that there were no hard limits on the use of open-reference questions, 

and that the current guidance provides only “targets” for the number of open reference 

questions. The NRC further stated that open reference questions have their own unique 

problems, such as being direct look-ups or not meeting the “application” portion of a K/A 

statement. The Industry FG acknowledged this information, but stated that examiners 

are using the “targets” as a perceived limit on the use of open reference questions, and 

that some types of questions simply cannot be asked without providing a reference (e.g., 

an examinee is not required to memorize graphs). 

Potential NUREG-1021 Changes

The NRC staff presented to the attendees a list of potential changes to NUREG-1021, “Operator 

Licensing Examination Standards for Power Reactors,” including: 

• Updating section ES-202 of NUREG-1021 to reflect the eligibility requirements contained 

in National Academy for Nuclear Training (ACAD) 10-001. 

• Adding clarifying guidance regarding examination waivers for the generic fundamentals 

examination and for site-specific re-take examinations. 

• Potential revisions stemming from the Fukushima accident. 

In response to questions from the attendees, additional possible changes to NUREG-1021 were 

discussed, including: 

• Updating the ES-600 sections to conform with the new revision to Inspection Procedure 

(IP) 71111.11. 

• Incorporating answers from the NRC’s operator licensing website for frequently asked 

questions. 

• Changes associated with new reactors. 

National Workshop for Industry Examiners

The Industry FG proposed conducting a two-day national workshop for industry examiners in 

September 2012, with both industry and NRC participation. A question was asked by the NRC 

staff as to what would be the focus of such a workshop, given that regional workshops are 

routinely conducted with NRC and industry participation. The attendees agreed that the topics 

of the national workshop should be carefully considered. 

IP 71111.11 (Licensed Operator Requalification Program Inspection Procedure) Changes

The NRC staff presented to the attendees an overview of the revisions to IP 71111.11, 

“Licensed Operator Requalification Program,” and its associated Significance Determination 

Process (SDP). In summary: 

• The revisions to the IP and SDP were issued December 6, 2011, and become effective 

January 1, 2012. 

• The SDP was rewritten to contain only Green and White findings, and aggregation of 

findings was eliminated, consistent with current NRC practice. 
3 

• The revised SDP contains White findings in three areas at a consistent threshold of 

greater than 40% of the sample size – exam failure rates, flawed written exam questions, 

and flawed operating test items. Green findings occur in these areas at a threshold of 

greater than 20%. 

• One additional White finding is possible for simulator performance, if incorrect simulator 

performance provided negative training which impacted operator performance during a 

reportable plant event. 

• The revised SDP contains no Yellow or Red findings, unlike the previous SDP. 

• The revised IP has, for the most part, the same content as the previous IP, with three 

notable differences: (1) the operating history section has been replaced with a problem 

identification and resolution section, (2) the training feedback inspection section has 

been eliminated, and (3) 4 hours per calendar quarter has been added for the Resident 

Inspectors to observe licensed operator performance in the actual plant/main control 

room. 

• The revised IP maintains a core element - the collection of exam pass/fail rates as an 

indicator of licensed operator performance. However, exam pass fail rates are only 

meaningful if the exams are of appropriate quality, properly administered, and if exam 

security is maintained. 

The NRC staff then described in detail how each area in the new IP would be inspected, noting 

that the assessment tools in most areas have been greatly improved by clarifying the important 

assessment items and by eliminating some items that were difficult to assess (e.g., going from a 

11 item checklist for written exam questions to a concise 6 item checklist). The NRC staff then 

described issues that would likely result in traditional enforcement consistent with current NRC 

practices (e.g., licensed operator medical issues, actual exam compromise). 

The attendees had a few questions for the NRC staff, which included: 

• Are findings for exam failure rates possible for any requalification examination given by a 

facility? The NRC responded that findings for exam failure rates only apply to 

requalification examinations required by 10 CFR 55.59, (i.e., an annual operating test 

and the biennial written exam), and that these exams are the only exams that the NRC 

will collect exam pass/fail rates. All other exams and evaluations performed by a facility 

are not subject to exam failure rate findings. 

• The new IP is not consistent in some cases with the ES-600 sections of NUREG-1021. 

Will NUREG-1021 be updated? The NRC responded that changes to the ES-600 series 

are within the scope of the next revision to NUREG-1021. 

• The new IP contains a new requirement for scenario level of difficulty. How should 

facilities and the NRC determine scenario level of difficulty? The NRC responded that 

scenario level of difficulty is a subjective measurement, and that the subject matter 

expertise of industry exam writers and NRC examiners will be relied upon. 

• How are the checklists for observing licensed operator performance in the actual 

plant/main control room to be used by NRC inspectors? Will answering an item “no” on 

these checklists result in documentation in a report? The NRC stated that the checklists 

for operator performance did not contain any regulatory requirements and were provided 

as guidance only. If deficiencies in operator performance were observed, it would be up 

to the inspector to determine the negative consequences of that deficiency, and if the 

consequence were of such a magnitude as to warrant documentation. 

• What is meant by providing “industry operating experience” to NRC inspectors? In many 
4 

cases, industry operating experience is considered proprietary information and cannot be 

provided to the NRC. The NRC acknowledged this information, stated that the industry 

should provide this information if possible, but consistent with their rules for proprietary 

information, and stated that perhaps a correction to the IP was warranted. 

Summary/Conclusions/Action Items

The Industry FG and the NRC agreed that the meeting was effective and that it was important to 

continue such interactions and communications of operator licensing issues. Moving forward, 

two specific action items were discussed: 

• Coordinating a National Examiner’s Conference, and 

• Inviting members of the Industry FG to the NRC’s Examiner Conference (May 1-3, 2012, 

near the Region II office in Atlanta, Georgia).