**APPENDIX BTECHNICAL BASIS FOR**

**EMERGENCY PREPAREDNESS SIGNIFICANCE DETERMINATION PROCESS**

**1.0	OVERVIEW OF THE EP SIGNFICANCE DETERMINATION PROCESS**

The framework of the Emergency Preparedness (EP) Cornerstone is described in SECY-99-007, dated January 8, 1999, and SECY-99-007a, dated March 22, 1999.  The Cornerstone Objective and Performance Expectation are the bases for the related inspection program and performance indicators:

**EP Cornerstone Objective**

Ensure that the licensee is capable of implementing adequate measures to protect public health and safety in the event of a radiological emergency.

**EP Performance Expectation**

Demonstrate that reasonable assurance exists that the licensee can effectively implement its emergency plan to protect public health and safety adequately in the event of a radiological emergency.

To meet the cornerstone objective and performance expectation, the staff of the U.S. Nuclear Regulatory Commission (NRC) assesses licensee performance in this cornerstone by considering performance indicators (PIs) with regard to thresholds and the significance of inspection findings.  The significance determination process (SDP) provides a method to place inspection findings in context for risk-significance in a manner that allows them to be considered in conjunction with the results from the PIs to assess overall licensee performance in the cornerstone.  This information is then used to determine the level of NRC engagement in accordance with the Reactor Oversight Process (ROP) Action Matrix.[footnoteRef:1]   [1:  	Found in Inspection Manual Chapter 0305, “Operating Reactor Assessment Program”] 

The NRC policy statement “Safety Goals for the Operation of Nuclear Power Plants,” (51 FR 30032) states, “. . . emergency response capabilities are mandated to provide additional defense-in-depth protection to the surrounding population.”  EP and many other elements of reactor safety (e.g., remote siting and containment) are implemented as a matter of prudence rather than in response to a quantitative analysis of accident probabilities.  Accordingly, the EP SDP is risk-informed, rather than risk-based, and does not involve numerical estimates of risk metrics such as core damage frequency (CDF) or large early release fraction (LERF).  The planning basis for EP is not predicated on a single accident sequence or even a limited number of sequences, but rather on a spectrum of accidents that differ in release characteristics, timing, and potential consequences.  A low probability sequence may have the higher consequences; while a higher probability sequence could have lower consequences.  Consequently, the probability of a reactor accident requiring implementation of the licensee’s emergency plan has no relevance in determining the significance of EP findings.  Colloquially, if the emergency plan is being activated in response to a radiological emergency, the event has occurred (i.e., probability = 1.0)

**2.0	EMERGENCY PREPAREDNESS REGULATORY BASIS**

An applicant is required by various provisions in 10 CFR Parts 50 and 52, as applicable, to submit, as part of its application, plans for coping with emergencies including the items specified in 10 CFR Part 50, Appendix E.  A nuclear power reactor applicant is further required by 10 CFR 50.47(b) to meet the 16 PLANNING STANDARDS (PS)[footnoteRef:2] established in 10 CFR 50.47(b)(1) through (16).   [2: 	Capitalized words and phrases used in this appendix are defined in Section 2.0 of the EP SDP (IMC 0609, App. B). Also, acronyms of these phrases include the singular and plural, as applicable.] 

Pursuant to 10 CFR 50.47(a), no early site permit,[footnoteRef:3] initial operating license, or initial combined operating license, will be issued unless a finding is made by the NRC that there is reasonable assurance that protective measures can and will be taken in the event of a radiological emergency. [3:  	If the early site permit contained a complete and integrated emergency plans.] 

Once an operating license is issued under Part 50 or a combined license under Part 52,[footnoteRef:4] a nuclear power reactor licensee is required by 10 CFR 50.54(q)(2) to follow and maintain the effectiveness of an emergency plan that meets the requirements of 10 CFR 50.47(b) and the supporting requirements in Appendix E.  The EP SDP is largely based upon the PS because the requirements in Appendix E are generally expansions on the broadly worded standards.[footnoteRef:5]  These regulations, the licensee’s approved emergency plan, along with relevant license conditions, Commission orders, and other commitments, comprise the REGULATORY REQUIREMENTS that the licensee’s EP program must meet. [4:  	After the Commission makes the finding under 10 CFR 52.103(g).]  [5:  	The PS are applicable to both onsite and offsite plans whereas the requirements in Appendix E are directed to the licensee.] 

Regulatory Guide (RG) 1.101, “Emergency Planning and Preparedness for Nuclear Power Reactors,” Revision 5 [ML050730286], states that the criteria and recommendations contained in Revision 1 of NUREG-0654/FEMA-REP-1 are considered to be acceptable methods for complying with the PS in 10 CFR 50.47(b) that must be met in onsite and offsite emergency response plans.  The regulatory guide also endorses certain industry emergency action level scheme guidance.

NUREG-0654/FEMA-REP-1, “Criteria for Preparation and Evaluation of Radiological Emergency Response Plans and Preparedness in Support of Nuclear Power Plants” [ML040420012] provides guidance to State and local governments and licensees in the development of E-plans.  Implementation guidance is provided in the form of evaluation criteria for each PS.  The methods and capabilities described in the emergency plan are evaluated against these criteria in determining whether the plan complies with the PS.  As with all NRC guidance documents, the licensee may propose alternative approaches of demonstrating compliance with the PS.  The licensee’s arrangements for meeting these evaluation criteria, as modified or supplemented by commitments made in the licensee’s approved emergency plan, comprise the PROGRAM ELEMENTS (PE) that form the basis for evaluating the EP Cornerstone.  

In December 2011, an amendment to the EP regulations at 10 CFR 50.47(b), 10 CFR 50.54(q), and Appendix E became effective.  Supporting guidance was issued:

· Regulatory Guide 1.219, “Guidance on Making Changes to Emergency Response Plans for Nuclear Power Reactors” [ML102510626]

· NUREG-0654/FEMA-REP-1, Supplement 3, “Guidance for Protective Action Strategies” [ML113010596]

· NUREG/CR-7002, “Criteria for Development of Evacuation Time Estimate Studies” [ML113030515]

· NSIR/DPR-ISG-01, “Interim Staff Guidance: Emergency Planning for Nuclear Power Plants” [ML113010523]

The latter document provides interim guidance that modifies previous guidance found in NUREG-0654/FEMA-REP-1, and other previously issued guidance, until the earlier guidance is updated.  

**3.0	DEVELOPMENT OF THE EP SDP**

During the development of the EP SDP, the most risk-significant PS (i.e., RISK-SIGNIFICANT PLANNING STANDARDS (RSPS)) were identified as being distinct from the other PS.  These development efforts were performed by a group of EP subject matter experts, including NRC staff and industry stakeholders, with input from members of the public.  Timely and accurate classification of events (§ 50.47(b)(4)), notifications to OFFSITE RESPONSE ORGAN-IZATIONS (OROs) (§ 50.47(b)(5)), assessments of radioactivity releases (§ 50.47(b)(9)), and development and recommendation of protective measures (§ 50.47(b)(10)), are essential if adequate measures are to be taken to minimize the risk to the public should a radiological emergency occur at the facility.  If these functions are not performed adequately during an actual event, the public may be placed at greater risk.  Similarly, if a PE in the emergency plan is non-compliant, the function(s) may not be adequately implemented should an actual emergency occur.

This is not to say that findings related to the other 12 PS may not warrant enforcement action; but, that they are not as significant as RSPS[footnoteRef:6] findings. The non-RSPS often support the RSPS.  For example, findings in EMERGENCY RESPONSE ORGANIZATION (ERO) staffing under PS 10 CFR 50.47(b)(2), could impact the performance of the RSPS. [6:  	RSPS are a subset of the PS.  References to “planning standard” or “PS” include the RSPS, but references to “RSPS” do not include the PS.  ] 

Because the PS are broadly worded to be applicable to both onsite and offsite emergency plans and they describe what is required to be in an emergency plan, rather than what is to be done by the licensee during an emergency, one or more functions for each PS were developed.  These PS FUNCTIONS are paraphrases of the PS in terms of the significant functions that need to be accomplished, or the capabilities that need to be in place, to maintain the effectiveness of the emergency plan.  The PS FUNCTIONS are used in assessing significance, not compliance, and are identified in the EP SDP.

**4.0	EP SIGNIFICANCE PROCESS**

The EP SDP addresses three categories of findings, those findings that:

· are associated with the licensee’s failure to follow its emergency plan—an emergency response**issue—during an actual radiological emergency, referred to as a FAILURE TO IMPLEMENT (FTI);

· are associated with the licensee’s failure to maintain its emergency plan—an emergency preparedness issue—typically identified through baseline and supplemental inspections, referred to as a FAILURE TO COMPLY (FTC); 

· are associated with the licensee’s failure to identify a WEAKNESS in a drill or exercise, or to correct that WEAKNESS.  These findings are addressed as FTC.  WEAKNESSES that are identified by the licensee in a drill or exercise, and entered into a corrective action program are not considered findings in the EP SDP and are not assigned significance; or;

The EP SDP is not applied to offsite deficiencies identified by the Federal Emergency Management Agency (FEMA).[footnoteRef:7]  However, if the licensee has assumed responsibility (i.e., self-imposed standard) for alert and notification system (ANS) testing and maintenance commitments made in the FEMA-approved ANS final design report, the EP SDP will be applied to findings related to these commitments. [7: 	Including deficiencies identified by FEMA in offsite plans developed by the licensee pursuant to 10 CFR 50.47(c)).] 

The significance of non-compliances with REGULATORY REQUIREMENTS that are not associated with a PS, such as non-compliance with 10 CFR 50.54(q)(4), (5), and (6); 50.54(t), 50.72,[footnoteRef:8] or certain requirements of Appendix E (e.g., § VI), are assigned Green significance by the EP SDP because of the very low safety significance of these findings. [8:  	In accordance with the enforcement policy Section 2.4.C.2.c.1, a failure to make NRC notification of a condition is assigned the significance of the unreported condition.] 

**4.1	Failure to Implement (FTI)**

4.1.1	Description

A FTI occurs when performance deficiencies are observed in a licensee’s response to an actual radiological emergency in which the failure precluded effective implementation of the licensee’s PE.  A FTI denotes that a PE was not effectively implemented by the licensee’s ERO during an actual radiological emergency such that protection of the public may have been impacted.  Such a finding may be identified by reviewing the licensee’s ERO performance during (or after) a radiological emergency for compliance with REGULATORY REQUIREMENTS and would generally be associated with failure of the licensee to follow its emergency plan as required by 10 CFR 50.54(q)(2).  The EP SDP incorporates the following considerations:

· A performance deficiency that occurs during an actual radiological emergency might not rise to the level of a FTI, particularly if the deficiency is self-identified by the ERO (e.g., peer review) and corrected in a timely manner such that the PS FUNCTIONS are successfully accomplished.  

· The failure of the ERO to implement a single PE does not necessarily mean that any of the associated PS FUNCTIONS were not accomplished.  

· A FTI may uncover performance deficiencies in the licensee’s emergency program (e.g., the emergency plan is less than adequate, the EP program design is not fully adequate, or, ERO personnel are not capable of implementing the plan).  The EP SDP requires that such findings be evaluated as a FTI and as a FTC with the higher significance being assigned to the finding.

· There are specific requirements for the licensee to maintain a capability to make classifications, notifications, and PARs, within certain time criteria.  However, there may be unanticipated circumstances during an emergency that cause the licensee’s performance to be delayed.  Such delays do not necessarily represent a FTI.[footnoteRef:9]   [9: 	This is consistent with the definition of a finding in that only those issues under the control of the licensee to foresee or prevent can be considered as findings.] 

4.1.2	Significance Determination

The significance of a FTI is based on the emergency classification level that was, or should have been, declared during the event and whether a RSPS was involved.  Generally, findings associated with a FTI are assigned greater significance than those associated with a FTC because findings that occur during actual events may have a greater impact on public health and safety.  The minimum significance level of a FTI is Green;[footnoteRef:10] the maximum is Red.[footnoteRef:11]  In comparison, the maximum significance level for a FTC associated with a lost RSPS FUNCTION is Yellow.  This elevated significance is consistent with the increased risk to the public of the non-compliance during an actual radiological emergency. [10: 	Significance is determined by the EP SDP for more than minor findings only; the minimum significance is Green. ]  [11: 	Such FTI may have actual safety consequences and the associated finding may be treated under traditional enforcement.] 

Because the significance of a FTI is based in part on the emergency classification applicable to the event, the EP SDP requires the inspector to base the significance on what the licensee should have declared.  Further, the failure to declare the appropriate emergency classification is, in itself, a finding and is to be evaluated separately from the other finding, with the finding having the greater significance cited.  

Because an over-classification by the licensee could result in unnecessary protective actions, the EP SDP provides for significance determination based on whether the OROs initiated protective actions for the public.  The EP SDP assigns a Yellow significance if the licensee’s classification causes public officials to implement a public evacuation; a White significance for a protective action other than evacuation (e.g., school or park closures, sheltering, etc.); and Green significance otherwise.  This protocol is consistent with the increased risk of unnecessary public evacuations, and does not apply if the ORO action was clearly inappropriate (e.g., ordering a public evacuation at an Alert).

**4.2	Failure to Comply (FTC)**

4.2.1	Description

A FTC occurs when a licensee’s EP program is noncompliant with a REGULATORY REQUIREMENT where the cause was reasonably within the licensee’s ability to foresee and correct, and which should have been prevented.  Such findings are associated with preparedness issues rather than response issues (i.e., FTI).  

A FTC denotes that a PE is not adequate, not compliant with the PS, or otherwise not functional to such an extent that the PS is not available for emergency response.  It may be that certain emergency plan commitments are not met; the emergency plan is less than adequate; implementing procedures are not effective; personnel are not capable of implementing the PE; or the EP program design is not fully adequate.  If the deficient PE was implemented, or could not be implemented by the ERO, the PS FUNCTION would not be met.  Such findings are generally identified during normal program inspection activities.  The EP SDP provides the following considerations:

· A single noncompliant PE does not necessarily mean that the associated PS FUNCTION(S) could not be accomplished.  

· A FTI may uncover performance deficiencies in the licensee’s emergency program (e.g., the emergency plan is less than adequate, the EP program design is not fully adequate, or, ERO personnel are not capable of implementing the plan).  The EP SDP requires that such findings be evaluated as a FTC and as a FTI with the finding having the higher significance cited.

· A single noncompliant PE may affect more than one PS FUNCTION.  For example, inadequate ERO staffing (non RSPS 10 CFR 50.47(b)(2)) may affect one or more RSPS. 

· There are specific requirements for the licensee to maintain a capability to make classifications, notifications, and PARs, within certain time criteria, as a matter of preparedness.  The licensee is expected to demonstrate these capabilities in exercise and program inspections.  Generally, if the licensee’s procedures, staffing, equipment, etc., do not provide the requisite capabilities, a performance deficiency exists because the licensee had the ability to foresee and correct the condition.

4.2.2	General Significance Determination

Generally, a finding associated with a FTC is assigned lesser significance than that associated with a FTI, because a FTC finding identified during routine oversight activities has only a prospective impact on public health and safety.  The minimum significance level of a FTC is Green; the maximum is Yellow.  

The significance of a FTC is based on whether a RSPS was involved and whether the FTC constituted a loss of the licensee’s ability to implement a PS FUNCTION if an emergency had occurred or was to occur in the future, or a degradation in that ability. Four resulting conditions and the associated significance levels are:

· LOST RSPS FUNCTION	Yellow

· DEGRADED RSPS FUNCTION	White

· LOST PS FUNCTION	White

· DEGRADED PS FUNCTION	Green

The EP SDP provides that a LOSS OF PS FUNCTION exists when PE are not adequate, are noncompliant with the PS, or otherwise not functional to the extent that the PS FUNCTION would not be accomplished if a radiological emergency were to occur.  A LOSS OF PS FUNCTION would be assigned White significance.  A LOSS OF RSPS STANDARD would be assigned Yellow significance consistent with its increased potential impact on public health and safety.

There are instances in which the PS FUNCTION is degraded, but not lost.  The finding category of DEGRADED PS FUNCTION was incorporated into the EP SDP to address these situations and allow an intermediate level of significance to be assigned.  A licensee may be noncompliant with a PE, but there could be reasonable assurance that the associated PS FUNCTION would be accomplished if a radiological emergency were to occur.  The EP SDP provides guidance for the inspector to consider diverse and/or redundant PE, MITIGATING FACTORS, and COMPENSATORY MEASURES, taken in determining if the affected PS FUNCTION would still be accomplished, albeit in a degraded manner, thereby justifying a lesser significance.  

An EP finding not associated with a PS is generally assigned Green significance.  A logic chart in the EP SDP illustrates this significance determination and the EP SDP provides a series of significance examples for each PS.

A FTC is assessed significance based upon a qualitative predictive evaluation of the potential impact of the noncompliant PE on the licensee’s capability to implement its emergency plan and to take measures to protect the public health and safety if a radiological emergency occurred, or was to occur in the future.  Re-paraphrased, the question becomes, “What would have been the likely impact of this noncompliant PE if a radiological emergency had occurred, or was to occur in the future and the finding had not been corrected?”  This assessment does not consider the circumstances that existed at the time of discovery because the adverse condition could occur under different circumstances when the impact of the noncompliant PEs would be greater.  For example, the fact that a plant is shut down when an at-power emergency action level (EAL) was determined to be inadequate has no bearing on the significance of the ineffective EAL.

The following sections explain some exceptions to the above significance determination methodology.

**4.3	Significance of Emergency Action Level Findings**

The EP SDP provides separate guidance for significance determination of findings associated with EALs.  Such findings may involve an EAL that has been rendered ineffective such that it no longer results in a timely and accurate declaration, or is associated with a deficient EAL process that results in over-classification of an emergency condition.  Such findings are cited against 10 CFR 50.54(q)(2) and 10 CFR 50.47(b)(4) and treated under ROP.  However, if these conditions were the result of emergency plan changes made by the licensee without prior NRC approval pursuant to 10 CFR 50.54(q)(3), the issue will be treated under traditional enforcement as a violation of 10 CFR 50.54(q)(3).  The significance determination is made in the same manner in either case.

4.3.1	Ineffective EALs 

PS 10 CFR 50.47(b)(4) requires that a standard emergency classification and action level scheme be in use by the nuclear facility operator.  Emergency classification schemes typically have a series of initiating conditions (IC), which represent a particular classification level and, for each IC, one or more EALs that identify particular indications or conditions that correspond to the IC.  An EAL may be rendered ineffective when, for whatever reason, the EAL no longer results in a timely and accurate declaration for the IC.  EALs may be rendered ineffective by unavailability or mis-calibration of instruments relied upon by the EAL, errors in calculation of the EAL threshold, and by deficiencies in classification procedures, ERO staffing or training, or any other capability necessary to complete the classification or declaration.  A particular EAL may include a list of redundant instrument channels; however, it is treated as a single EAL for significance purposes.  

The significance of findings related to ineffective EALs is based on: (1) the emergency classification level the EAL is associated with; and, (2) one of the following characteristics:

· The emergency would not be declared for a particular off-normal event (i.e., LOSS OF RSPS FUNCTION),

· The emergency would not be declared for a particular off-normal event, but because of other EALs, an appropriate declaration could be made in a degraded manner (i.e., DEGRADED RSPS FUNCTION), or,

· The emergency would not be declared for a particular off-normal event, but because of other EALs, an appropriate declaration could be made in an accurate and timely manner (i.e., Green).

The EP SDP generally requires that the alternative EAL being credited be one of those specified in the EAL scheme for the same IC.  The staff included this condition because each IC addresses a particular abnormal condition that gives rise to an emergency (e.g., uncontrolled radioactivity release, loss of one or more fission product barriers, natural phenomena, etc.).  An ineffective EAL affects the licensee’s ability to classify that particular abnormal condition.  For example, an IC may be “a release of radioactivity that results in doses at the site boundary exceeding X dose.”  While one may argue that X dose could not occur without one or more EAL thresholds in the fission product barrier matrix being exceeded, a release that may result in X dose may be due to a condition not encompassed by the fission product barrier matrix (e.g., spent fuel pool accident, distinguishing between Site Area or General Emergency for a steam generator tube rupture with steam release to the environment).

The consideration of the emergency classification does not factor into most FTC significance assessments, but is appropriate here because an ineffective EAL has an impact only if that emergency classification occurs; whereas, a finding related to a notification could affect all emergencies classifications.  Accordingly, an ineffective General Emergency EAL has greater potential significance than would an ineffective EAL associated with a Notice of Unusual Event (NOUE).  

4.3.2	Deficient EAL Processes that Could Cause Over-classifications 

The EP SDP also establishes significance for emergency action level scheme findings that could lead to over-classifications and unwarranted declarations:

· A finding associated with a deficient emergency classification process that would cause over classification and would result in OROs implementing unnecessary protective actions for the public would have White significance.  The White significance is consistent with the emergency classification function being degraded rather than lost (e.g., other EALs in scheme are effective) and the fact that the deficiency was identified during normal operations, rather than an emergency.  The EP SDP applies this criterion only in cases in which the offsite response would be explicitly driven by ORO response procedures triggered by the declared classification (e.g., “...when the plant reports this, do this...”) or the licensee makes an unnecessary PAR to the OROs because of an over-classification.

· A finding associated with a deficient emergency classification process that would cause over-classification and would result in an unnecessary emergency declaration would have Green significance.  

**4.4	Significance of Findings Related to Hardware Issues**

The EP SDP addresses findings related to hardware issues (e.g., instrumentation or communication system unavailability), in which the significance may be reduced if the licensee implements COMPENSATORY MEASURES within a certain period following discovery.  This protocol recognizes that not all hardware failures are under the control of the licensee and instead assesses significance on the timeliness and adequacy of short-term COMPENSATORY ACTIONS, and/or, the scope of the outage.  

**4.5	Significance of Findings Related to Alert and Notification System**

Alert and notification systems (ANS) are used by OROs to alert and to provide instructions to the public (i.e., notify) during an actual radiological emergency.  Because many licensees have assumed responsibility for the testing and maintenance of ANS on behalf of the ORO officials (i.e., self-imposed standard), significance examples related to the ANS have been included in the EP SDP. [footnoteRef:12]   The EP Cornerstone does not evaluate the ability of the ANS to alert and notify the public as the technical adequacy of the ANS and its testing and maintenance is under the purview of FEMA.  Rather, the EP cornerstone of the ROP evaluates the licensee’s performance with regard to maintaining the ANS in accordance with the testing and maintenance commitments as outlined in the FEMA-approved final design report.   [12: 	Appendix E to 10 CFR Part 50, Section IV.D.3, requires that the licensee demonstrate that administrative and physical means have been established for alerting and providing prompt instructions to the public within the plume exposure pathway emergency planning zone.  The regulation does not require the licensee to provide the capability.  The licensee demonstrates the capability exists via the FEMA-approved ANS Final Design Report.] 

Within the EP Cornerstone, there is a performance indicator (ANS PI) based on the reliability of the ANS system as demonstrated in scheduled testing.  Although the EP SDP contains significance examples under Section 5.5 of the EP SDP that appear similar in purpose, the EP SDP examples are based on the availability, rather than the reliability, of the ANS.  The EP SDP does provide that if the ANS PI has fallen below the Green band, or fallen below the White band, during the period under consideration an additional finding is not necessary, as the appropriate regulatory response will already be taken.

**4.6	Significance of Findings Related to Exercise CRITIQUES**

The EP Cornerstone of the NRC Reactor Oversight Process is designed to foster drill, exercise, and training programs that develop and maintain ERO skills.  Licensees are required under Appendix E, §IV.F.2.g to provide for formal CRITIQUES for all exercises, drills, and training that provide performance opportunities to develop, maintain, and demonstrate key skills, and to correct all weaknesses identified in those CRITIQUES.  The licensee’s failure to identify WEAKNESSES is a FTC with 10 CFR 50.47(b)(14).  

It is the nature of an exercise and drill program that WEAKNESSES in ERO performance will occur and that equipment, facility and procedure problems will be identified.  The identification and correction of these WEAKNESSES is a positive and vital aspect of the program that enhances and maintains key ERO skills.  A WEAKNESS observed during an exercise or drill has little or no direct safety-significance if the WEAKNESS is identified and corrected as this will ultimately enhance the ERO performance during an actual radiological emergency.  If NRC oversight were to penalize the identification of WEAKNESSES, this enhancement might not occur and ERO performance could degrade.  For these reasons, the EP SDP does not treat ERO performance WEAKNESSES as performance deficiencies and instead places focus on the licensee’s ability to identify a WEAKNESS and on the timeliness and adequacy of the corrective actions taken.  A licensee’s ability to observe, evaluate, and CRITIQUE a weakness associated with a RSPS is critical.  Although all drill or exercise WEAKNESSES are required to be identified and corrected, the EP SDP puts the highest priority to WEAKNESSES associated with a RSPS.

This treatment of exercise and drill WEAKNESSES is consistent with the licensee response band green threshold of 90 percent for the Drill and Exercise Performance (DEP) performance indicator (PI).  The DEP PI tracks ERO performance in drills and exercises in three RSPS: emergency classification; protective action recommendation (PAR); and notification.  ERO performance in a fourth RSPS, dose assessment, would most likely be reflected in the PAR performance.  If the DEP PI falls below 90 percent, signifying WEAKNESSES in greater than 10 percent of classification, PAR, and notification opportunities, the increased regulatory response band is entered.  Although the DEP PI only tracks performance in the RSPS, the same oversight philosophy is appropriate when considering ERO performance in areas not addressed by the DEP PI.

Identification of exercise and drill WEAKNESSES is critical to maintaining the validity of the EP Cornerstone licensee response band, which is predicated in part on accurate PI data from drill and exercise CRITIQUES.  If the licensee does not appropriately assess DEP PI opportunities through the CRITIQUE process, the validity of the DEP PI and the licensee response band is brought into question.  Accordingly, findings related to the identification of WEAKNESSES are assigned significance as follows:

· The failure of a licensee to identify a WEAKNESS that constitutes a DEP PI failed opportunity involving a RSPS in a CRITIQUE for a FULL-SCALE EXERCISE is considered a LOST PS FUNCTION and is assigned a White finding.  FULL-SCALE EXERCISES provide more opportunity for ERO members to exercise key skills and response functions and to be observed by a number of observers/evaluators.  Accordingly, the failure of the licensee to adequately identify WEAKNESSES under these circumstances is deemed to have greater significance.  This level of significance is also based on the NRC’s need to ensure the efficacy of the licensee’s CRITIQUE program and, hence, the licensee response band.  

· Licensee failures to identify WEAKNESSES in limited scope drills, including those that constitute a DEP PI successful[footnoteRef:13] opportunity involving a RSPS are assigned Green significance.  [13: 	For example, a correct emergency classification, consistent with scenario expectations, may have been made based on misinformation, lack of information, or invalid indications. ] 

A WEAKNESS in ERO performance may uncover inadequate PE.  For example, an incorrect emergency classification by the ERO may have been caused by an incorrect procedure.  In this case, the ERO performance may have been in accordance with the procedure, but the procedure was wrong.  These inadequate PE are performance deficiencies that the EP SDP treats as FTC, rather than WEAKNESSES, even if the licensee identifies the performance deficiency in its CRITIQUE.

**4.7	Significance of Findings Related to Failure to Correct Weaknesses**

The EP Cornerstone of the ROP is based on the licensee response band established by the PI program and the licensee’s problem identification and resolution (PI&R) program.  As it relates to emergency preparedness, PI&R encompasses the drill and exercise CRITIQUE program, CRITIQUES of actual events and other assessment activities (such as QA audits and reviews performed in accordance with 10 CFR 50.54(t)), as well as the corrective action program.  The EP baseline inspection program provides oversight of a licensee’s efforts to CRITIQUE drills and exercises and correct WEAKNESSES.  NRC regulations in 10 CFR 50.47(b)(14) and Section IV.F.2.g of Appendix E to 10 CFR Part 50 require licensees to formally CRITQUE drills and exercises to identify and correct any WEAKNESSES.

The EP SDP provides that determination of a failure to correct a drill or exercise WEAKNESS requires a detailed review of the WEAKNESS and the associated corrective actions.  It is not intended that a single repetition of a WEAKNESS (e.g., in a drill) should automatically be deemed to be a failure of the corrective action system.  Conversely, success in a drill or exercise (e.g., by one well-drilled team) should not necessarily be considered a success of the corrective action system.  When an apparent failure to resolve a WEAKNESS is observed, specific corrective actions, as well as similar occurrences in response to actual events, drills, exercises and training evolutions, are reviewed.  The status of relevant performance indicators and the corrective action, self-assessment and inspection records for an entire inspection cycle with emphasis on similar issues are also considered.  In addition, completion of corrective actions is verified.  The intent of these actions is to discover a pattern of recurring performance issues in similar activities in order to identify ineffective corrective actions.

The EP SDP assigns significance for failure to correct a WEAKNESS as follows:  

· For a WEAKNESS associated with the RSPS 10 CFR 50.47(b)(4), -(5), and -(10), a failure to correct is assigned White significance, a high standard based on the reliance that NRC places on timely corrective actions to maintain the integrity of the licensee response band.  

· For the RSPS incorporated in the DEP PI, if the DEP PI has fallen below the license response band, there is no need for an additional finding as the regulatory response band (or higher band) would have been entered because of the PI.

· For RSPS 10 CFR 50.47(b)(9), White significance will be assigned if the original WEAKNESS is observed in more than 10 percent of the performance opportunities; otherwise, no finding is assessed.  This treatment of uncorrected WEAKNESSES is consistent with the licensee response band threshold of 90 percent for the DEP PI.  This includes all observed WEAKNESSES having a common uncorrected root cause.

· Similarly, Green significance will be assigned to non-RSPS WEAKNESSES if observed in more than 10 percent of the performance opportunities; otherwise, no finding is assessed.  

Issue Date:  12/19/12  		0308, Att 3, App B

0308, Att 3, App C		Issue Date: xx/xx/xx

Issue Date:  12/19/12		0308, Att 3, App B

Revision History for IMC 0308, Att 3, App B

Issue Date:  12/19/12	Att1-1	0308, Att 3, App B