December 3, 2015 

MEMORDANUM TO: Scott C. Sloan, Chief 

 Operator Licensing and Training Branch 

 Division of Inspection and Regional Support 

 Office of Nuclear Reactor Regulations 

FROM: Jacob W. Dolecki, Reactor Engineer /RA/ 

 Operator Licensing and Training Branch 

 Division of Inspection and Regional Support 

 Office of Nuclear Reactor Regulations 

SUBJECT: SUMMARY OF THE NOVEMBER 10, 2015, PUBLIC MEETING 

WITH NEI LICENSED OPERATOR FOCUS GROUP 

On November 10, 2015, the U.S. Nuclear Regulatory Commission (NRC) staff held a public 

meeting with the Nuclear Energy Institute (NEI) Licensed Operator Focus Group to discuss the 

NRC requalification written examination quality grading process as it related to aspects of direct 

look-up (DLU) questions and level of difficulty (LOD). 

The Category 2 public meeting was noticed on October 29, 2015. The notice is available 

electronically at the NRC’s Public Meeting Schedule website at 

http://meetings.nrc.gov/pmns/mtg?do=details&Code=20151624 or in the NRC’s Agencywide 

Document Access and Management System (ADAMS) under Accession No. ML15302A510. 

Meeting participants included NRC staff and contractors, members of the power reactor 

licensee community, and representatives from the NEI. 

This meeting was held with the intention to promote efficiency, effectiveness, and open 

communications with regard to licensed operator requalification written examination quality 

standards. The discussions addressed a variety of issues, including the definition of DLU and 

how that definition potentially affects requalification examination inspections; the ambiguity 

associated with determining LOD in examination questions; a Vogtle finding as a result of a 

NRC Inspection Procedure 71111.11, “Licensed Operator Requalification and Licensed 

Operator Performance,”; and potential changes to guidelines within the Operator Licensing 

program. Representatives of the NRC and the industry agreed that this meeting was useful for 

the exchange of information and agreed that some clarification was made. 

Enclosures: 

1. List of Attendees 

2. Agenda 

3. Discussion Summary 

CONTACT: Jacob Dolecki, NRR/DIRS/IOLB 

(301) 415-2947 
MEMORDANUM TO: Scott C. Sloan, Chief 

 Operator Licensing and Training Branch 

 Division of Inspection and Regional Support 

 Office of Nuclear Reactor Regulations 

FROM: Jacob W. Dolecki, Reactor Engineer 

 Operator Licensing and Training Branch 

 Division of Inspection and Regional Support 

 Office of Nuclear Reactor Regulations 

SUBJECT: SUMMARY OF NOVEMBER 10, 2015, PUBLIC MEETING WITH 

NEI LICENSED OPERATOR FOCUS GROUP 

On November 10, 2015, the U.S. Nuclear Regulatory Commission (NRC) staff held a public 

meeting with the Nuclear Energy Institute (NEI) Licensed Operator Focus Group to discuss the 

NRC requalification written examination quality grading process as it related to aspects of direct 

look-up (DLU) questions and level of difficulty (LOD). 

The Category 2 public meeting was noticed on October 29, 2015. The notice is available 

electronically at the NRC’s Public Meeting Schedule website at 

http://meetings.nrc.gov/pmns/mtg?do=details&Code=20151624 or in the NRC’s Agencywide 

Document Access and Management System (ADAMS) under Accession No. ML15302A510. 

Meeting participants included NRC staff and contractors, members of the power reactor 

licensee community, and representatives from the NEI. 

This meeting was held with the intention to promote efficiency, effectiveness, and open 

communications with regard to licensed operator requalification written examination quality 

standards. The discussions addressed a variety of issues, including the definition of DLU and 

how that definition potentially affects requalification examination inspections; the ambiguity 

associated with determining LOD in examination questions; a Vogtle finding as a result of a 

NRC Inspection Procedure 71111.11, “Licensed Operator Requalification and Licensed 

Operator Performance,”; and potential changes to guidelines within the Operator Licensing 

program. Representatives of the NRC and the industry agreed that this meeting was useful for 

the exchange of information and agreed that some clarification was made. 

Enclosures: 

1. List of Attendees 

2. Agenda 

3. Discussion Summary 

CONTACT: Jacob Dolecki, NRR/DIRS/IOLB 

(301) 415-2947 

ADAMS Accession No.: ML15317A185 

OFFICE NRR/DIRS/IOLB NRR/DIRS/IOLB NRR/DIRS/IOLB 

NAME JDolecki SSloan JDolecki 

DATE 11/16/15 12/3/15 12/3/15 

 OFFICIAL RECORD COPY
Enclosure 1 

List of Attendees – Nuclear Regulatory Commission / Nuclear Energy Institute Licensed 

Operator Focus Group / Power Reactor Licensee Community 

November 10, 2015 

Scott Sloan NRC 

Tim Kolb NRC 

Theresa Buchanan NRC 

John Munro NRC 

David Muller NRC 

Jacob Dolecki NRC 

Eric Cushing NRC 

Hironori (Pete) Peterson NRC 

Don Jackson NRC 

Gene Guthrie NRC 

Bruno Caballero NRC 

Mark Bates NRC 

Matt Emrich NRC 

Vince Gaddy NRC 

Bruce Montgomery NEI 

Gregg Ludlam Exelon Corp. 

Fred Bruns Exelon Corp. 

Bruce Hennigan Exelon Corp. 

Shaun Batts Exelon Corp. 

Chuck Sizemore NextEra Corp 

Travis Ouret NextEra Corp 

Randall Amundson NextEra Corp. 

Sean Wylie NextEra Corp. 

Mark Otten Union Electric Co. 

Russell Joplin TVA Corp. 

Roger Jones Entergy Corp. 

James Knapp Wolf Creek Nuclear Operating Corp. 

Corry Craine Southern Nuclear 

Trevor O. McCraw Southern Nuclear 

Andy Perkins Southern Nuclear 

Ron Kelly Southern Nuclear 

Gary Ohmstede Southern Nuclear 

Tony Roberts DTE Electric Co. 

Jana Bergman Curtis-Wright 
Enclosure 2 

PUBLIC MEETING AGENDA FOR THE CATEGORY 2 MEETING WITH THE 

NUCLEAR ENERGY INSTITUTE LICENSED OPERATOR FOCUS GROUP 

Tuesday, November 10, 2015 

01:30 PM to 02:30 PM, Eastern Time 

Teleconference

Time Topic Speaker

1:30 p.m. Introductions and Opening Remarks NRC/NEI 

 

1:45 p.m. Discussion related to determining requalification written 

examination quality 

NRC/NEI 

2:15 p.m. Public comments Public 

2:30 p.m. Adjourn 

 
Enclosure 3 

DISCUSSION SUMMARY

Requalification Examination Quality Standards

The Nuclear Energy Institute (NEI) Licensed Operator Focus Group (LOFG) discussed with the 

U.S. Nuclear Regulatory Commission (NRC) staff their concerns regarding use of direct look-up 

(DLU) questions in open-reference written examinations for individuals taking 

facility-administered requalification examinations (as discussed in NUREG-1021, “Operator 

Licensing Examination Standards for Power Reactors,” Section ES-602). In particular, the NEI 

LOFG stated that there appears to be some ambiguity associated with what constitutes a 

satisfactory DLU question and stated the definition affects requalification examination quality 

and the content of the requalification exam bank. In addition, NEI LOFG stated there is also 

ambiguity associated with determining the level of difficulty (LOD) of a particular examination 

question. NEI LOFG asked NRC staff to clarify both items. 

NRC staff acknowledged the NEI LOFG’s concerns but, at this time, although particular 

avenues were discussed, the staff had no specific recommendations for improvements related 

to the existing NUREG-1021 and ACAD 07-001 guidance for either DLU or LOD questions. In 

addition, the staff stated that no interim, new, or revised guidance has been provided on how to 

assess DLU questions. NRC staff stated that it is unwise to revise the existing Inspection 

Procedure (IP) 71111.11, “Licensed Operator Requalification Program and Licensed Operator 

Performance,” or NUREG-1021 guidance solely based on the recent requalification examination 

inspection findings at one facility. 

The staff and the NEI LOFG were in agreement that questions should be evaluated on a 

case-by-case basis to determine whether a question is in whole or in part a DLU question or the 

question’s LOD was too high or low. As such, the nuclear industry should continue to follow all 

related guidance currently found in NUREG-1021 and ACAD 07-001. However, the staff further 

informed the NEI LOFG that the NRC will continue to look into these issues and determine 

whether any changes to NUREG-1021 or IP 71111.11 are deemed necessary. 

The NRC staff and NEI LOFG discussed these topics being a potential topic at next year’s 

(2016) NEI/NRC National Operator Licensing Workshop.