**NRC INSPECTION MANUAL**IOLB

	INSPECTION MANUAL CHAPTER 0609 APPENDIX I

	LICENSED OPERATOR REQUALIFICATION

	SIGNIFICANCE DETERMINATION PROCESS

	Effective Date:  01/01/2019 

0609I-01	PURPOSE

The Licensed Operator Requalification Significance Determination Process (SDP) is used for determining the risk significance of findings identified during the inspection of licensed operator requalification activities and licensed operator performance.  

0609I-02	BACKGROUND

This SDP was designed to assess the risk significance of findings associated with Inspection Procedure 71111, Attachment 11 (IP 71111.11), “Licensed Operator Requalification Program and Licensed Operator Performance” in the following areas:  (1) requalification examination results, (2) biennial requalification written examinations, (3) annual requalification operating tests, (4) administration of an annual requalification operating test, (5) requalification examination security, (6) remedial training and re-examinations, and (7) the control room simulator.  

With regard to conformance with operator license conditions, such as the medical fitness of licensed operators and compliance with the regulations contained in Title 10 of the *Code of Federal Regulations* (10 CFR) Section 55.53, it may be appropriate to use traditional enforcement to disposition violations.  Inspectors should refer to guidance in Inspection Manual Chapter 0612, “Issue Screening,” and the Enforcement Manual.

0609I-03	GUIDANCE

Figure I.1, a flowchart contained on the following pages, presents a series of yes/no decision blocks for assessing licensed operator requalification and licensed operator performance findings.  Following the flowchart, a description of each flowchart block is presented.

Figure I.1 – Licensed Operator Requalification SDP Flowchart

1

Licensed Operator

Requalification Finding

4

Related to 

Biennial Requal.

Written Exam Quality? 

2

Related to Requalification Exam Results?

NO

NO

Go to A on page 3

YES

YES

5

Were greater than 40% of the reviewed written examination questions flawed? 

3

Failure rate greater than 40%?

NO

NO

Green Finding

Green Finding

YES

YES

White Finding

White Finding

Figure I.1 – Licensed Operator Requalification SDP Flowchart (continued)

A from page 2

9

Related to Licensee Admin. of an Annual Requal. Operating Test?

6

Related to Annual Requal. Operating Test Quality? 

Go to B on page 4

NO

NO

YES

YES

Green Finding

7

Were greater than 40% of the reviewed JPMs flawed?

White Finding

YES

NO

8

Were greater than 40% of the reviewed simulator scenario events flawed?

YES

White Finding

Finding

NO

Green Finding

Figure I.1 – Licensed Operator Requalification SDP Flowchart (continued)

B from page 3

10

Related to Requalification Exam Security? 

12

Related to Licensee Remedial Training and Re-exams? 

Go to C on page 5

NO

NO

YES

YES

11

Was there an actual effect on the equitable and consistent administration of any examination required by 10 CFR 55.59?

Green Finding

NO

Green Finding

YES

Evaluate using traditional enforcement  against 10 CFR 55.49.   and evaluate using IMC 0609 Appendix M.

Figure I.1 – Licensed Operator Requalification SDP Flowchart (continued)

C from page 4

13

Related to Simulator Performance, Testing, Maintenance, or Modification? 

NO

16

Re-evaluate the finding by entering the SDP at block 1.

YES

14

Was a simulator performance, modeling, or fidelity deficiency identified?    

Green Finding

NO

(Simulator testing, maintenance, or modification deficiency)  

YES

15

Did deficient simulator performance, modeling, or fidelity negatively impact operator performance in an actual plant event where the risk increase due to the operator performance was >10E-6 delta CDF or >10E-7 delta LERF?

Green Finding

NO

YES

Reference appropriate SDP to determine significance of operator performance issues

Flowchart Block Descriptions:

#1 – The SDP starts after a single licensed operator requalification finding is identified from IP 71111.11 and screened through Manual Chapter 0612, Appendix B.  Each specific finding must be evaluated separately.

#2 – This is the top-level entry block associated with licensed operator performance as measured by the results of the requalification examinations required by 10 CFR 55.59(a)(2).  This block is answered “yes” or “no” based upon completing the specific guidance contained in Section 03.03 of IP 71111.11 and upon completing the screening of inspection issues in accordance with IMC 0612.

#3 – Based upon the requalification examination results collected at the end of the testing cycle, was the failure rate greater than 40%?  This block will be answered “yes” if either:

(a) The individual examination failure rate is greater than 40% (IP 71111.11, Line 4 of Table 03.03-1), or

(b) The crew simulator scenario failure rate is greater than 40% (IP 71111.11, Line 7 of Table 03.03-1).

#4 – This is the top-level entry block associated with the quality of biennial requalification written examinations that are required by 10 CFR 55.59(a)(2).  This block is answered “yes” or “no” based upon completing the specific guidance contained in Section 03.04.a and Appendix B of IP 71111.11.

#5 – Were greater than 40% of the reviewed written examination questions flawed?  In answering this question, the inspector will need to review the results from section 03.04.a and Appendix B of IP 71111.11.  If the answer to this block is “yes,” then a White finding results, based upon a higher percentage of flawed written examination questions used on a requalification examination required by 10 CFR 55.59(a)(2).  If the answer to this block is “no,” then a Green finding results, based upon a lower percentage of flawed questions or other written examination deficiency.   

#6 – This is the top-level entry block associated with the quality of annual requalification operating tests that are required by 10 CFR 55.59(a)(2).  This block is answered “yes” or “no” based upon completing the specific guidance contained in Section 03.04.b and Appendix C of IP 71111.11.  

#7 – Were greater than 40% of the reviewed job performance measures (JPMs) flawed?  In answering this question, the inspector will need to review the results from Section 03.04.b and Appendix C of IP 71111.11.  If the answer to this block is “yes,” then a White finding results, based upon a higher percentage of flawed JPMs used on a requalification examination required by 10 CFR 55.59(a)(2).

#8 – Were greater than 40% of the reviewed simulator scenario events flawed?  In answering this question, the inspector will need to review the results from Section 03.04.b and Appendix C of IP 71111.11.  If the answer to this block is “yes,” then a White finding results, based upon a higher percentage of flawed simulator scenario events used on a requalification examination required by 10 CFR 55.59(a)(2).  If the answer to this block is “no,” then a Green finding results, based upon a lower percentage of flawed simulator scenario events and JPMs (checked in block 7 above), or based upon some other operating test deficiency.   

#9 – This is the top-level entry block associated with the licensee’s administration of annual requalification operating tests that are required by 10 CFR 55.59(a)(2).  This block is answered “yes” or “no” based upon completing the specific guidance contained in Section 03.04.c and Appendix D of IP 71111.11.  

#10 – This is the top-level entry block associated with requalification examination security.  This block is answered “yes” or “no” based upon completing the specific guidance contained in Section 03.04.d and Appendix E of IP 71111.11.

#11 – Was there an actual effect on the equitable and consistent administration of any examination required by 10 CFR 55.59?  In these instances, a licensed operator has gained an unfair advantage on an examination required by 10 CFR 55.59, and this condition was not corrected prior to being authorized to resume licensed duties.  These occurrences can be willful or intentional (“cheating”) or unintentional.  Examples of gaining an unfair advantage on an examination  include:  (1) a licensed operator obtains unauthorized assistance during an examination, such as by receiving assistance on a test item during an examination from an unauthorized individual or by copying answers from another examinee; (2) a licensed operator obtains specific knowledge of or is exposed to requalification examination content prior to taking the requalification examination; (3) a licensed operator is used to validate requalification examination test items during exam development, and is then subsequently administered a requalification examination with any test items duplicated from those that the operator previously validated.  IMC 0609, Appendix M should be used to evaluate the significance of these types of inspection findings.  Note that the traditional enforcement process may also be used for violations of 10 CFR 55.59 (e.g., in cases where the violation involves willfulness or impacts the NRC’s ability to perform its regulatory function).  Refer to IMC 0612 and the Enforcement Manual for guidance on dispositioning traditional enforcement violations.    

#12 – This is the top-level entry block associated with remedial training and re-examinations, which occurs whenever a licensed operator fails any portion of a requalification examination required by 10 CFR 55.59(a)(2).  This block is answered “yes” or “no” based upon completing the specific guidance contained in Section 03.04.e and Appendix F of IP 71111.11.  

#13 – This is the top-level entry block associated with control room simulator performance, maintenance, and testing, as specified in 10 CFR 55.46.  This block is answered “yes” or “no” based upon completing the specific guidance contained in Section 03.04.g and Appendix G of IP 71111.11.  

#14 – Was a simulator performance, modeling, or fidelity deficiency identified?  This block is used to differentiate between deficiencies associated with simulator performance (including deficiencies with modeling or fidelity) and deficiencies associated with simulator testing, maintenance, and modification.  These issues are treated slightly differently in the SDP, due to the potential for unrealistic operator training due to deficient simulator performance.  If this block is answered “no,” the deficiency is associated with simulator testing, maintenance, or modification (as verified in the next block), and results in a Green finding.  If this block is answered “yes,” proceed to block 15.

#15 – Did deficient simulator performance, modeling, or fidelity negatively impact operator performance in the actual plant during a plant event?  The concern with this block is that the simulator provided un-realistic or negative training to licensed operators (due to deficiencies in simulator performance, modeling, or fidelity), and that this un-realistic simulator training was the primary cause of negatively impacted operator performance during an event.  Reference appropriate SDP guidance (At-Power, Shutdown, or others) to determine if the negative operator performance resulted in a risk increase of greater than 10E-6 delta CDF or greater than 10E-7 delta LERF.  Qualitative SDP results may also be used to determine if the risk increase is greater-than-Green.  If the answer to this block is “yes,” then this results in a finding with significance commensurate with the risk increase due to the negative operator performance, based upon the appropriate SDP guidance.  If the answer to this block is “no,” then this results in a Green finding, since deficient simulator performance was still identified.

#16 – Re-evaluate the finding by entering the SDP at block 1.  The SDP is arranged as a *series* of top-level entry blocks, and block 16 should not occur unless all the entry blocks have been answered “no.”  If this is the case, re-evaluate the finding and enter the SDP at block 1, or consult with the program office for guidance.

0609I-04	REFERENCES

1. IP 71111, Attachment 11, “Licensed Operator Requalification Program and Licensed Operator Performance”

2. IMC 0612, Appendix B, “Issue Screening”

3. NRC Enforcement Manual

4. IMC 0609, Appendix M, “Significance Determination Process Using Qualitative Criteria”

Issue Date:  01/10/19	1	0609 Appendix I

ATTACHMENT 1 - Revision History – IMC 0609, Appendix I

Issue Date:  01/10/19	Att1-1	0609 Appendix I

# image1.emf

# image10.emf