**NRC INSPECTION MANUAL**	NSIR/DPR

INSPECTION PROCEDURE 71114 ATTACHMENT 06

DRILL EVALUATION

Effective Date:  10/01/2016

PROGRAM APPLICABILITY:  2515 A

71114.06-01	INSPECTION OBJECTIVE

To evaluate the adequacy and capability of the licensee’s assessment of performance, via a formal critique process, to identify emergency preparedness (EP) weaknesses in selected drills and training evolutions and use of its corrective action program (CAP) to correct EP identified weaknesses. 

The intent of this inspection procedure is to evaluate the licensee’s ability to conduct and critique their own drills and or training evolutions between U.S. Nuclear Regulatory Commission (NRC) graded EP exercises hence performance of this inspection sample is to be completed independent of Inspection Procedure (IP) 71114.01 Exercise Evaluation or 71114.07 Exercise Evaluation - Hostile Action Event.

Note:  If the last Hostile Action Base (HAB) evaluated exercise was greater than eight years ago, request the licensee to demonstrate the emergency preparedness support during the next Force-on-Force drill or exercise

71114.06-02	INSPECTION REQUIREMENTS

02.01	Select a sample of EP drills and training evolutions identified as contributing to the Drill and Exercise Performance (DEP) and Emergency Response Organization (ERO) performance indicators (PI) to observe.  The inspection sample should include, to the extent possible, all types of scenarios (e.g. 10 CFR 50.54(hh)(1) & (2), radioactive release, no release, etc.).  Plan to inspect a sample of three drills and/or training evolutions (see note above concerning last HAB evaluated exercise), such that a minimum of one EP drill and some combination of two additional drills and/or training evolutions are observed.  Drills or exercises performed by the licensee for the purpose of outside organization observation or audit, such as Institute of Nuclear Power Operations or peer audit team, should not be selected as part of this inspection procedure’s sample. 

02.02	Observe the licensee’s performance during the EP drill/training evolution.  Emphasis should be on the risk-significant activities of classification, notification, dose assessment and protective action recommendation (PAR) development as well as other inspection areas based on resource availability.  Review DEP PI individual inputs for degraded performance and/or adverse trends to ensure areas of weakness are observed.

Issue Date: 10/25/06		71114.06, Att 1

Att 1-

Issue Date:  07/21/16	1 	71114.06

02.03	Verify any EP weakness observed by the inspector are appropriately identified in the licensee’s formal critique and entered into the corrective action program.

02.04	Identify recurring weaknesses in similar activities from previous drill and/or training opportunities in order to evaluate the effectiveness of corrective actions.

02.05	Identify any weaknesses that may reveal a failure to comply with a regulatory requirement.

02.06	Determine if the licensee assessment of DEP PI opportunities are accurate.

71114.06-03	INSPECTION GUIDANCE

The primary focus of this inspection is to verify the licensee critique of the risk-significant areas from EP drills and training evolutions identified as DEP and ERO Drill PI opportunities.  Select other areas for inspection based on resource availability, past history, efforts to correct weaknesses and/or logistical limitations.  Consider the prioritization guidance in Attachment 1, “Prioritization of Additional Areas for Inspection” to develop a plan to deploy inspection resources to observe other activities as practical.  

03.01	Select Emergency Preparedness Drills and/or Training for Observation.

a. Obtain the licensee’s schedule of EP drills and training evolutions identified as DEP and ERO Drill PI opportunities.  

b. Inspect as a minimum one EP drill, and some combination of two EP drills and/or simulator-based licensed operator requalification training (LORT) evolutions involving shift operating crews (total of three samples).  The inspection sample should include, to the extent possible, all types of scenarios (e.g. 10 CFR 50.54(hh)(1) & (2), radioactive release, no release, etc.).  

c. Observation of drill and training evolutions may be performed unannounced.

Note:	The licensee is not required to include simulator-based LORT training evolutions in DEP PI opportunities.  Additionally, the licensee may wish to collect “as found” operator proficiency information.  There is no intent to disrupt ongoing operator qualification programs.

d. Review the selected drill/training evolution scenario to identify the timing and location of classification, notification, dose assessment and PAR development activities and licensee expectations of a successful response.  If the scenario contains a potential aircraft threat the inspector should review the requirements of 10 CFR 50.54 (hh)(1).  If the scenario contains a hostile action event and/or loss of large areas of the plant due to explosions or fire, the inspector should review the applicable requirements of 

10 CFR 50 Appendix E and 10CFR 50.54 (hh)(2).    

Note:	Refer to NEI 99-02, “Regulatory Assessment Performance Indicator Guideline,”

Section 2.4, Emergency Preparedness Cornerstone, ERO Drill Participation, Clarifying Notes, to understand how drills and training evolutions qualify as performance enhancing experiences. 

Determine whether the drill/training evolution qualifies to be included in the DEP (and ERO) PI statistics.

e. Review the Emergency Plan Implementing Procedures (EPIPs) that provide instructions for classification, notification, dose assessment, and PAR development activities, to develop an understanding of the criteria for timely and accurate completion of these activities based on the EPIPs, the scenario, and NEI 99-02.  If the licensee’s DEP PI indicator is in the licensee response band, the inspector should review EPIPs governing other ERO program areas to develop an understanding of the activity’s success criteria for inspection.

03.02	Drill/Training Evolution Observation.

a. Review checklists and forms used for classification, notification, dose assessment and PAR development activities. 

Note:	The licensee has demonstrated the capability to make a notification in 15 minutes if offsite response organizations identified in the Emergency Plan (E-plan) receive notification of, at a minimum, the declared emergency classification level within 15 minutes of declaration.  The licensee’s critique should identify any delay that occurred in making the notification to one or more offsite response organizations (OROs) (e.g., an ORO cannot be reached).  Any notification delay under the control of the licensee to foresee and or prevent (e.g., telephone call lists not kept up-to-date) should be evaluated as a failure to comply and assessed for significance. 

b. Identify any weaknesses in licensee performance of classification, notification, dose assessment PAR development and other observed EP areas and activities.  If the licensee’s DEP PI indicator is in the licensee response band, the inspector should sample DEP activities and consider other ERO program areas for inspection (See Attachment 1 “Prioritization of Additional Areas for Inspection”).  Inspector identified weaknesses must be held confidential until after the licensee’s formal critique.

Note:	Prompting of drill participants is not a finding under the assessment process because it represents no risk significance in itself.  However, prompting related to a DEP PI opportunity is basis for failing a. DEP PI opportunity and should be documented when observed.

03.03	Licensee’s Emergency Preparedness Drill/Training Evolution Formal Critique Observation.

a. Determine if the licensee critique identified all weaknesses observed by the inspector.   

b. Licensee critique failures are to be documented and assessed for significance.  

c. Determine if the licensee properly identifies failures in classification, notification, dose assessment and PAR development activities. 

d. Verify that licensee identified weaknesses are entered into the CAP in a manner to allow NRC review of the resolution in the future.

Note:	Poor performance in a drill is not a regulatory issue if the licensee corrects the performance issue or enters it into its CAP.  The DEP PI captures performance failures and unless the PI crosses the “green band” threshold then increased NRC involvement is not warranted.

Note:  PI opportunities to be counted in the DEP PI quarterly report are at the discretion of the licensee and must be identified in advance and cannot be removed from the count due to poor performance.

03.04	Identify Recurring Weaknesses.

a. Use previous drill and training critiques to determine if previously identified weaknesses represent a trend or a repeat (i.e., recurring, with the same or similar cause) weakness.  

b. Review specific corrective actions identified for the previous weaknesses, as well as similar occurrences in response to actual events, drills, exercises and training evolutions. 

c. Verify completion of associated corrective actions. 

d. Determine if there is a trend or repeat of a weakness.  Verify the licensee identified the trend or repeat weakness and entered it into the corrective action system.  

e. Consider the status of the DEP PI as well as the status of the relevant risk significant planning standard components of the DEP PI.

03.05	Identify Failures to Comply with Regulatory Requirements.

a. Evaluate any noted program element issues related to the effectiveness and adequacy of the E-Plan, or it’s implementing procedures[footnoteRef:1], observed during the drill and/or training as an apparent failure to comply with the associated 10 CFR 50.47(b) planning standards and Appendix E requirements.  Assess significance in accordance with the EP Significance Determination Program.   [1:  The E-plan contains the licensee’s commitments to NRC regulations.  The implementing procedures are the licensee’s methods of implementing those commitments and may be used to judge effective, timely, and accurate implementation.  ] 

For example, an ERO field monitoring team is unable to perform a survey because of ineffective/inadequate survey procedures or equipment.  Such issues, whether identified by the licensee or inspector, are not treated as weaknesses, which are defined as deficiencies in ERO performance.  The inspector should:

1. Review the history of identified issue(s) to obtain relevant information.  

2. Determine, immediately, if possible, if the program no longer meets the applicable planning standard.  If this cannot be accomplished immediately, confer with regional management for direction.  

03.06	Assessment of Licensee Performance Indicator Accuracy.

a. Determine if the licensee’s assessment of PI opportunity performance is accurate.  

b. Determine if the licensee is properly dispositioning failures in classification, notification, dose assessment and PAR development activities with regard to PI statistics.  

c. Refer any discrepancies to the regional EP inspector, regional management and headquarters’ Office of Nuclear Security and Incident Response/Division of Preparedness and Response (NSIR/DPR).

71114.06-04	RESOURCE ESTIMATE

The direct inspection effort is estimated to average, between 9 and 20 hours per year, regardless of the number of reactor units at a site.

71114.06-05	PROCEDURE COMPLETION

The minimum inspection sample size will constitute completion of this procedure in the Reactor Programs System.  Routine reviews of problem identification and resolution activities performed in this attachment should equate to approximately 10 to 15 percent of the resource estimate range described above.  The minimum sample size is 3 samples representing observation of one EP drill and two additional drills and/or training evolutions.  Participation in the performance of IP 71114.01, “Exercise Evaluation or 71114.07 Exercise Evaluation - Hostile Action (HA) Event,” may not be credited as one of the three samples required for this inspection procedure.

71114.06-06	REFERENCES

NSIR/DPR-ISG-01, “Emergency Planning for Nuclear Power Plants” (ML1130105230)

NEI 06-04 Rev 1, "Conducting a Hostile Action-based Emergency Response Drill" (ML073100460)

NEI 06-04, Rev. 2, Appendix A, “Drill and Exercise Objectives” (ML112091915)

IN 07-12, Tactical Communications Interoperability between Nuclear Power Reactor Licensees and First Responders (ML070710233)

IN 09-19, Hostile Action-Based Emergency Preparedness Drills (ML092250360)

RIS 06-12, Endorsement of NEI Guidance “Enhancements to Emergency Preparedness Programs for Hostile Action” (ML061530290)

RIS 2008-08, Endorsement of Revision 1 to NEI Guidance Document NEI 06-04, “Conducting a Hostile Action-Based Emergency Response Drill”

END

ATTACHMENT 1

PRIORITIZATION OF ADDITIONAL AREAS FOR INSPECTION

General

In general, NRC oversight in emergency preparedness is focused on adherence to the emergency plan with an emphasis on these most risk-significant areas, and inspection resources should be deployed in a manner to cover these areas.  However, within the constraint of resources, a broad range of response areas should be inspected.  

Corrective action system data is used to identify response areas of concern and deploy inspection resources accordingly.  Areas, (e.g., operational support center, field monitor teams) that have had few critique findings or more than average as compared to the technical support center (TSC) or emergency operation facility (EOF) findings should be selected for observation.  Inspection resources usually deployed in the TSC, EOF, or control room may be used to observe other areas.  

If a licensee’s performance in previous baseline inspections in the risk-significant areas of classification, notification, dose assessment and PAR development in conjunction with its performance under the Drill and Exercise Performance Indicators indicates reliable acceptable performance within the licensee response band, inspectors should reduce the inspection sampling in those areas and instead use a portion of available inspection resources to sample a selection of less risk significant areas as described below. 

In order to facilitate review of critique related corrective actions, the inspector should request a corrective action system listing sorted for drill and exercise critique findings of the for the previous 2-3 years.  If possible, the findings should be sorted by emergency response facility.

The inspector should remain alert to the impact that the licensee’s performance in less risk-significant areas (e.g., staffing and training) may have on the licensee’s performance in the risk-significant areas.  

Prioritization of Additional Areas for Inspection

Guidance for deployment of inspection resources beyond the most risk-significant areas is provided below.  These areas may generally be considered in order of importance.  Selection for deployment of inspection resources should be based on knowledge of the program, previous problems and logistics.

a. Adequacy of worker protection including accountability, evacuation, exposure authorization and thyroid protection, including actions during a hostile action 

[10 CFR 50.47(b)(10) & (11) and Sections IV.E and IV.I of Appendix E to 10 CFR 50].

b. Adequacy of interface with offsite authorities (e.g., in the area of PAR communication and technical support).  [10 CFR 50.47(b)(6) and Sections IV.A.7, IV.E.9, and IV.D of Appendix E to 10 CFR 50].

Issue Date:  07/21/16 	Att1-1	71114.06

c. Adequacy of arrangements for offsite resources responding to an emergency, including hostile actions, at the licensee’s site [10 CFR 50.47(b)(6) and Section IV.A.7 of Appendix E to 10 CFR 50.]

d. Ability to formulate mitigating actions.

e. Ability to prioritize mitigation and assessment efforts to protect the public health and safety.

f. Ability to implement mitigating actions (e.g., damage control teams) under accident conditions.

g. Effectiveness of command and control [10 CFR 50.47(b)(1)].

h. Ability to diagnose plant accident conditions, other than offsite consequences addressed in the risk-significant area discussion.

i. Adequacy of communications between licensee facilities [10 CFR 50.47(b)(6) and Section IV.E.9 of Appendix E to 10 CFR 50].

j. Accuracy and completeness of licensee-approved press releases [10 CFR 50.47(b)(7)].

Issue Date:  07/21/16 	Att1-2	71114.06

ATTACHMENT 2 - Revision History For IP 71114.06

Issue Date:  07/21/16	Att2-1	71114.06 

Issue Date:  07/21/16	Att2-2	71114.06 

Issue Date:  07/21/16	Att2-3	71114.06 

Issue Date:  07/21/16	Att2-4	71114.06