**NRC INSPECTION MANUAL**	IRAB

INSPECTION MANUAL CHAPTER 0307 APPENDIX A

# REACTOR OVERSIGHT PROCESS SELF-ASSESSMENT METRICS AND DATA TRENDING

Effective Date: 07/01/2023

This appendix contains a description of each of the Reactor Oversight Process (ROP) performance metrics and data trending as described in sections 06.01(a) and (b) of Inspection Manual Chapter (IMC) 0307, “Reactor Oversight Process SelfAssessment Program.” The objectives, applicability, and requirements in IMC 0307 apply to this appendix.

The objective performance metrics and data trending are organized by the Principles of Good Regulation as described in section 05.02 of IMC 0307, which include independence, openness, efficiency, clarity, and reliability. Additional detail related to the specifics and basis of the metrics can be found in the reference documents noted in the basis section of each metric. The ROP goals and ROP intended outcomes (see sections 05.01 and 05.03, respectively, of IMC 0307) related to each metric are also provided. Of note, data trending focus areas do not reference bases documents, and do not have performance thresholds, as they are designed to allow flexibility in data monitoring and analysis. The ROP performance metrics are tracked and reported on a calendar year basis. ROP performance metrics reporting requirements are outlined in IMC 0307, section 07.01.

In general, the ROP objective performance metrics are defined and measured at an agencywide level, though many of the metrics also analyze the data by region and/or office for comparison purposes and to ensure uniform and effective program implementation. Regional and office goals for a given metric are provided in the notes beneath agencywide criteria, when applicable. If the results of an ROP performance metric indicate that an individual evaluation of a given region or office is warranted, the Division of Reactor Oversight (DRO) in the Office of Nuclear Reactor Regulation (NRR) will typically include that evaluation in the annual ROP performance metrics report, which is approved by DRO management. Typically, these evaluations consist of a few paragraphs of independent analysis performed by the ROP selfassessment program lead based on discussions with and data provided by the region or office being evaluated.

To ensure consistency in collecting and reporting of metric data, NRR/DRO will specify the data elements needed to calculate all of the ROP performance metrics and will perform quality assurance to verify data accuracy and consistency.

The ROP data trending focus areas provide for routine monitoring of associated ROP program execution data for each focus area, looking for significant positive or negative trends (as compared to historical averages or expected trends). While the ROP objective performance metrics are generally measured at an agencywide level, the ROP data trending focus areas are purposefully widescope, so that flexibility exists for the data to be monitored and analyzed at the appropriate level.

# 0307A-01	INDEPENDENCE PERFORMANCE METRICS (I)

## 01.01	I-1 Completion of Baseline Inspection Program

## 01.02	I-2 Resident Inspector Objectivity through Diverse Experience

## 01.03	I-3 Inspector Objectivity and Performance Reviews

## 01.04	I-4 Fully Qualified Inspectors, Examiners, and Senior Reactor Analysts

## 01.05	I-5 Analysis of Resident Inspector Site Staffing

# 0307A-02	OPENNESS PERFORMANCE METRICS (O)

## 02.01	O-1 Issuance of Inspection Reports

## 02.02	O-2 Issuance of Assessment Letters

## 02.03	O-3 Conduct of Annual Assessment Meetings or Other Engagement Activities

## 02.04	O-4 Issuance of ROP Public Meeting Notices

# 0307A-03	EFFICIENCY PERFORMANCE METRICS (E)

## 03.01	E-1 Completion of Supplemental Inspections

## 03.02	E-2 Completion of Temporary Instructions

## 03.03****E-3 SDP Completion Timeliness for Potentially Greater-than-Green Findings

# 0307A-04	CLARITY PERFORMANCE METRICS (C)

## 04.01	C-1 Maintenance of ROP Web Pages

## 04.02	C-2 Maintenance of ROP Governance Documents

# 0307A-05	RELIABILITY PERFORMANCE METRICS (R)

## 05.01	R-1 Predictability and Repeatability of Significance Determination Results

## 05.02	R-2 Predictability of Agency Actions and Response

## 05.03	R-3 Supportability of Inspection Findings

# 0307A-06	ROP DATA TRENDING FOCUS AREAS

## 06.01	Overview and Basis for ROP Data Trending

To leverage ROP program execution data to monitor ROP program health and conduct ROP selfassessment activities, the staff established ROP data trending focus areas that are intended to be complementary to the formal ROP performance metrics. The staff will trend and analyze ROP program execution data (as compared to historical averages or expected trends) in these focus areas throughout the year, with the objective of identifying any significant trends (positive, negative, stable) or other insights into ROP program performance in these areas.

To the extent possible, the data processing and data visualization for each data trending focus area will use existing, robust data sources, and be as automated and uptodate as possible. As appropriate, internal ROP stakeholders will be offered opportunities to provide feedback on possible improvements to the selected data sources, the displayed trends, and the available visualization and filtering options for the focus areas as displayed on the ROP data trending dashboard. The dashboard should be flexible enough to meet the overall needs of ROP stakeholders, including the ability to observe any shortterm or longterm trends.

The ROP selfassessment data trending dashboard will be monitored on at least a monthly basis by ROP selfassessment staff. Should any significant trends or insights be identified (as compared to historical averages or expected trends), the ROP selfassessment lead will provide the data to the appropriate program area lead for further analysis and action, including input to the annual ROP metrics report and the ROP self-assessment Commission paper. Any identified significant trends or insights from the ROP data trending efforts shall also be discussed as part of the ROP selfassessment briefing at the Agency Action Review Meeting and the associated Commission briefing. Additionally, insights from the ROP data trending efforts will inform future ROP selfassessment activities, including topics for effectiveness reviews, focused assessments, and ROP implementation audits.

## 06.02	Data Trending Focus Area Selection Criteria

The standard set of data trending focus areas will adhere to the following criteria: cover aspects of at least two of the four ROP program areas (inspection, assessment, performance indicators, and SDP); include enough data to allow for meaningful trending and analysis; and be informed by recent ROP program or data changes, data needs or interests of ROP internal and external stakeholders, and recommendations from other ROP selfassessment activities. Importantly, when considering the addition of any new data trending focus areas, consider whether the data are available from authoritative data sources through the NRC data warehouse, such as inspectionrelated data from RPS-Inspections, timecharging data from the Cost Activity Code System (CACS), or other authoritative data systems.

## 06.03	Standard Data Trending Focus Areas

The data trending focus areas will include:

In approximately February of each calendar year, after the regional Division Director counterpart meeting, if there are any additional data trending focus areas that are desired and for which data are readily available, the DRO Division Director will promulgate any additional focus areas to the ROP self-assessment lead. The additional data trending focus areas shall be aligned with applicable Principles of Good Regulation and the ROP program areas and will be added to the ROP selfassessment data trending dashboard and this appendix.

# 0307A-07	REFERENCES

IMC 0040, “Preparation, Revision, Issuance, and Ongoing Oversight of NRC Inspection Manual Documents”

IMC 0102, “Oversight and Objectivity of Inspectors and Examiners at Reactor Facilities”

IMC 0305, “Operating Reactor Assessment Program”

IMC 0609, “Significance Determination Process”

IMC 0611, “Power Reactor Inspection Reports”

IMC 1245, “Qualification Program for New and Operating Reactor Programs”

IMC 2515, “Light-Water Reactor Inspection Program -- Operations Phase”

MD 3.14, “U.S. Nuclear Regulatory Commission Public Web Site”

MD 3.5, “Attendance at NRC Staff-Sponsored Meetings”

MD 8.14, “Agency Action Review Meeting”

OIG16A21, Office of the Inspector General, “Audit of NRC’s Significance Determination Process for Reactor Safety,” dated September 26, 2016 (ML16270A359)

Staff report, “Davis-Besse Reactor Vessel Head Degradation Lessons-Learned Task Force Report,” dated September 30, 2002 (ML022760172)

Issue Date: 06/14/23	9	0307 App A

ATTACHMENT 1

Revision History for IMC 0307, Appendix A

Issue Date: 06/14/23	Att1-1	0307 App A

Issue Date: 06/14/23	Att1-2	0307 App A

Issue Date: 06/14/23	Att1-3	0307 App A