Issue Date: 07/28/05 1 0308, Att 3, App I

APPENDIX I

TECHNICAL BASIS FOR OPERATOR REQUALIFICATION HUMAN PERFORMANCE

SIGNIFICANCE DETERMINATION PROCESS

This SDP covers those issues related to the operator requalification program. It is

expected that performance errors made by a licensed operator, leading to or during an |

actual operational event, are an integral part of the overall outcome of the event and would

be reflected in the event risk determination or ultimately, in a performance indicator. |

This SDP applies to the programmatic aspects (e.g., exam grading, exam quality, exam |

security) of operator requalification and to the performance of licensed operators during |

the written exam or the annual operating test. This SDP is applicable to requalification

issues related to all licensed operators, including both shift and staff crews, with either

active or inactive licenses. The process is applicable to all license holders since a staff crew

member with an active license could, at any time, be asked to go on-shift and because a |

staff crew member with an inactive license needs only to spend the required time on-shift |

to activate a license. A crew is defined as any group of individuals evaluated as a single

entity by the licensee on the basis of its performance on the dynamic simulator.

This SDP includes only those aspects of the requalification program considered to be risk |

important. For example, the student feedback system in-and-of itself has little risk |

importance, but its review might lead to issues that are risk important. |

The SDP first looks at overall requalification examination results by asking if less than 75% |

of the operators passed all portions of the exam (NUREG-1021, Rev. 9, ES 601). If this |

criteria is met, it may be indicative of the need for a “for cause” inspection. |

|

The SDP then looks at overall crew performance on the simulator and compares the crew |

pass rate to the Simulator Operational Evaluation Matrix. The Simulator Operational |

Evaluation Matrix provides the perceived risk associated with the number of crews failing

the annual operating test relative to the number of crews taking the test. The "Number |

of Crews that took the Annual Operating Test" axis includes multiple units to accommodate |

those instances where operators hold dual unit licenses. If a multiple unit site has separate

unit licenses, use the matrix to assess the results of each unit separately. Less than 20% |

failure rate is considered satisfactory and does not constitute a finding. A failure rate of

20% to 34% is considered a Green finding to be turned over to the licensee for corrective

action. An operating test failure rate greater than 34% but less than 50% is considered |

to be a White finding. It is indicative of a serious programmatic weakness and a Yellow |

finding if more than half the crews fail the operating test. Requalification operating test |

failure rate alone is never considered a Red finding unless over half the crews failed and

one or more of the failed crews return to shift without remediation. |

The SDP examines the licensee’s grading of the exam to ensure that failed candidates or |

crews are properly identified and not passed inappropriately. The risk importance is not |

that the licensee’s grading process was inadequate or flawed, but that inadequately trained |

operators may be allowed to return to shift. |

The next part of the SDP process is related to the written and walkthrough portions of the |

requalification examination and address findings of exam quality, exam security and |

individual performance. The risk determination assumes that a single individual failure in |

requalification does not rise to the risk significance of a Green finding. A failure rate greater |

than 20% has been selected as the threshold for unacceptable risk significance. This is |

consistent with the guidance in NUREG-1021, Rev 9. Thus, more than 20% unacceptable |

written test items is the quality threshold; more than 20% of the operators failing the written
0308, Att 3, App I 2 Issue Date: 07/28/05

| portion of the exam is the performance threshold; more than 20% of the operators failing

the operating test walkthrough is the walkthrough performance threshold, etc.

| The simulator portion of the SDP evaluates individual operating test quality, simulator

| fidelity, test and scenario integrity, and licensed operator license conditions. An individual

| failing the operating test portion of the exam does not rise to the risk significance of a

| Green finding. The risk significance of crew performance is dependent upon the

| percentage of crews that failed, whether the crews were remediated prior to returning to

| shift, and whether the facility failure rate was Green or higher (as determined by the SDP

Simulator Operational Evaluation Matrix) in the previous annual operating test. The risk

assessment of operator performance on the simulator should include all crews tested

| based on test records, even if the inspectors only witnessed testing of some crews.