May 7, 2015 

MEMORANDUM TO: Chairman Burns 

Commissioner Svinicki 

Commissioner Ostendorff 

Commissioner Baran 

FROM: Mark A. Satorius /RA Michael R. Johnson Acting for/ 

Executive Director for Operations 

SUBJECT: PROPOSED ELIMINATION OF ANNUAL REPORTING 

REQUIREMENTS FOR SPECIFIC EVALUATIONS WITHIN THE 

REACTOR OVERSIGHT PROCESS SELF-ASSESSMENT PROCESS 

The purpose of this memorandum is to request Commission approval to eliminate the annual 

reporting requirements for several evaluations that are included in the existing Reactor 

Oversight Process (ROP) self-assessment process. As part of its ongoing evaluation of 

program improvements, the staff has identified a potential resource savings in the existing 

self-assessment program. Specifically, the staff recommends eliminating three evaluations that 

are enclosures to the current annual ROP self-assessment: the regulatory impact summary, the 

resident inspector demographic analysis, and the ROP resource expenditure analysis. These 

detailed evaluations have been shown to offer limited insights, are redundant to other 

processes, and do not appear to return the level of value as they did when they were initiated by 

Commission direction. This recommendation aligns with recent Commission direction in its staff 

requirements memorandum (SRM) to SECY-15-0005, “Recommendation to Sunset the 

Decommissioning Trust Fund Spot-Check Program” (Agencywide Documents Access and 

Management System (ADAMS) Accession No. ML15058A111), dated February 26, 2015, which 

states that the staff should continue to identify regulatory activities that do not return value to our 

regulatory program in the way that was forecast when they were initiated. 

In its SRM to COMSECY-14-0030, “Proposed Suspension of the Reactor Oversight Process 

Self-Assessment for Calendar Year 2014” (ADAMS Accession No. ML14168A532), dated 

August 5, 2014, the Commission approved the staff’s recommendation to suspend the annual 

self-assessment of the ROP for calendar year (CY) 2014. As noted in the COMSECY and 

SRM, the temporary suspension would result in resource savings that the staff would use to: 

(1) develop a more effective self-assessment process with more meaningful metrics to use in 

CY 2015 and beyond, and (2) address ROP improvement recommendations from the multiple 

independent and focused ROP-related assessments performed in CY 2013 and CY 2014. The 

staff also noted that in lieu of the typical self-assessment discussions at the Agency Action 

CONTACT: Ronald K. Frahm, NRR/DIRS 

 301-415-2986
The Commissioners - 2 - 

Review Meeting (AARM) and subsequent Commission meeting in CY 2015, senior 

U.S. Nuclear Regulatory Commission (NRC) management discussions would focus on the 

revised self-assessment process and the ongoing activities addressing the recent ROP 

program evaluations and recommendations. 

The staff is currently developing a revised ROP self-assessment process that will be formalized 

over the coming months and implemented in the CY 2015 ROP self-assessment and beyond. 

The revised approach is designed to assess the effectiveness of a mature program, focusing on 

the efficacy of recent changes to the program and performing in-depth reviews of specific areas 

of interest. The staff is exploring a three-part approach that would: (1) measure the 

effectiveness of and adherence to the current program using objective, measureable metrics 

organized around the principles of good regulation; (2) assess recent changes to the program 

and monitor the status of ROP revisions and enhancements; and (3) perform a focused, indepth assessment of specific program area(s). This approach also would include consideration 

of revising the ROP self-assessment process to better solicit and assess the breadth and depth 

of program performance insights and feedback similar to that achieved from the Commissiondirected independent program evaluation, “Reactor Oversight Process Independent 

Assessment 2013” (ADAMS Accession No. ML14035A571), dated February 11, 2014, as 

discussed in COMSECY-14-0030. The revised process will be reflected in revisions to 

Inspection Manual Chapter (IMC) 0307, “Reactor Oversight Process Self-Assessment 

Program,” and its appendices, which will be forwarded to the Commission for information by 

September 30, 2015. 

Since 1992, the regulatory impact summary has been submitted to the Commission annually as 

a result of stakeholder concerns that the NRC was encumbering the industry with unnecessary 

regulatory burden. Under this process, regional management solicited feedback from licensees 

during routine visits to reactor sites, and the staff then evaluated the issues raised and took any 

necessary corrective actions. At the end of each year, the staff compiled and summarized this 

feedback, evaluated the potential unnecessary impact, and recommended improvement actions, 

as warranted. 

Since 1998, the resident inspector demographic analysis has been submitted to the 

Commission annually based on the concern that resident inspector experience was diminishing 

and the regions were challenged to attract and retain highly qualified resident inspection staff. 

The analysis included (1) resident inspector experience, (2) resident inspector turnover and 

losses, and (3) resident inspector site staffing. The staff compiled and analyzed the data, and, 

as warranted, recommended improvement actions necessary to ensure that all sites continued 

to be staffed with knowledgeable and experienced resident inspectors. 

The ROP resource expenditure analysis has been submitted to the Commission as an 

enclosure to the annual ROP self-assessment paper since its first report in 2001. The purpose 

of the analysis was to baseline the expected expenditures necessary to implement the revised 

program, since significant changes had been made. The staff summarized and analyzed the 

staff resources expended to implement the ROP compared with previous years, and explained 

any anomalies, as warranted. 

Soon after implementation of the ROP, the regulatory impact summary and resident 

demographic analysis were combined with the annual ROP self-assessment and have since 

been included as enclosures. The most recent evaluations can be found as enclosures to 
The Commissioners - 3 - 

SECY-14-0047, “Reactor Oversight Self-Assessment for Calendar Year 2013” (ADAMS 

Accession No. ML14066A365), dated April 18, 2014. A more detailed history on when and why 

these evaluations were developed and how they were reported to the Commission is included 

as an enclosure to this paper. 

As noted above, the staff is developing objective performance metrics that will be collected and 

reported on an annual basis as the first part of the proposed self-assessment process. Certain 

objective aspects of these three evaluations will be incorporated into those metrics. With 

respect to regulatory impact, the primary purpose of NRC Form 649, “Site Visit Observation,” 

which provides the data to inform the regulatory impact analysis, is to ensure and record NRC 

management’s evaluation of inspector objectivity in accordance with IMC 0102, “Oversight and 

Objectivity of Inspectors and Examiners at Reactor Facilities” (ADAMS Accession No. 

ML12012A053). NRC management will continue to record their site visits, and this data will be 

captured as one or more performance metrics. Numerous other communications will continue 

to provide additional means to assess regulatory impact and unnecessary burden, including 

ongoing efforts to evaluate the cumulative effects of regulation and frequent interactions with 

industry during Regional Utility Group meetings, ROP Working Group public meetings, 

management drop-ins, and other venues. 

Similarly, with respect to resident inspector demographics, the permanent site-staffing metric 

currently included as part of the resident inspector demographic analysis will be maintained as a 

performance metric. The staff recognizes the importance of the resident inspectors as our front 

line of regulatory oversight, and focuses on staffing the sites with experienced individuals. As 

such, regional management will continue to monitor and evaluate resident staffing and 

demographics to ensure that sites are appropriately and sufficiently staffed with qualified 

inspectors. The data will continue to be reviewed annually by the program office and the 

regions. While the staff is recommending the removal of the annual Commission reporting 

requirement as part of the ROP self-assessment process, the staff will explore ways to better 

measure meaningful data that can provide insights into inspector staffing and demographics. 

This revised demographic analysis would occur on a less frequent (triennial) basis and would be 

forwarded to the Commission for information. The staff will also coordinate any new approach 

under the broader umbrella of human capital management. 

With respect to ROP resources, the staff plans to continue conducting its periodic review and 

realignment of the ROP baseline inspection procedures in accordance with IMC 0307, 

Appendix B, “Reactor Oversight Realignment” (ADAMS Accession No. ML112990461), to 

ensure the most effective overall application of inspection resources. The ROP realignment 

process is being revised based on lessons learned from the staff’s ROP enhancement efforts. 

Given the lack of meaningful, actionable insights resulting from these processes and that some 

of the previously reported information will be captured as performance metrics (or evaluated 

using other management processes), NRC staff recommends eliminating the annual 

Commission reporting requirements for regulatory impact, resident inspector demographics, and 

ROP resource expenditures. With completion of ongoing efforts to develop the revised selfassessment process and its implementation for CY 2015 ROP self-assessment, the staff will 

evaluate the changes and obtain stakeholder feedback as experience is gained using the new 

process. The staff will keep the Commission apprised of any lessons learned and resulting 

modifications made to address them as part of its annual self-assessment going forward. 
The Commissioners - 4 - 

There are no significant resource implications. The resource savings from eliminating the 

subject evaluations would be used to implement the new additions to the revised selfassessment process, including the evaluation of recent changes to the program and focused 

evaluations of particular areas of concern. 

The Office of the Chief Financial Officer has reviewed this paper for resource implications and 

has no objections. The Office of the General Counsel has reviewed this memorandum and has 

no legal objection. 

SECY, please track. 

Enclosure: 

As stated 

cc: SECY 

 OGC 

 OCA 

 OPA 

CF
Enclosure 

HISTORY AND BASIS FOR SUBJECT EVALUATIONS 

As part of the staff’s efforts to improve the effectiveness of the Reactor Oversight Process 

(ROP) self-assessment process, the staff seeks to discontinue the reporting requirements for 

three evaluations that have been shown to offer limited benefit and insights for program 

improvements and overlap other processes. These evaluations historically have been included 

as enclosures to the annual ROP self-assessment SECY paper. The most recent evaluations 

can be found as enclosures to SECY-14-0047, “Reactor Oversight Self-Assessment for 

Calendar Year 2013” (Agencywide Documents Access and Management System (ADAMS) 

Accession No. ML14066A365), dated April 18, 2014. 

 

• the regulatory impact summary (Enclosure 3 to SECY-14-0047) 

• the ROP resource expenditures analysis (Enclosure 4 to SECY-14-0047) 

• the resident inspector demographic analysis (Enclosure 5 to SECY-14-0047) 

The history and basis for each of these evaluations is discussed below. 

Regulatory Impact Summary

On December 20, 1991, the Commission issued a staff requirements memorandum (SRM) 

directing the staff to develop a process for obtaining continual feedback from licensees and to 

report it to the Commission each year. The staff described the continual feedback process in 

SECY-92-286, “Staff’s Progress on Implementing Activities Described in SECY-91-172, 

‘Regulatory Impact Survey Report—Final’” (ADAMS Accession No. ML12256B345), dated 

January 1, 1992. Under this process, NRC management solicits informal feedback from its 

licensees during routine visits to reactor sites, documents the feedback on U.S. Nuclear 

Regulatory Commission (NRC) Form 649, “Site Visit Observation,” and forwards the completed 

forms to affected internal stakeholders. Responsible staff then evaluates any concerns 

identified and coordinates with management to take any necessary corrective actions. 

Annually, the staff compiles this feedback and creates a comprehensive regulatory impact 

summary, which includes evaluating the potential unnecessary impact, and recommends 

improvement actions, as warranted. 

Originally a stand-alone annual SECY, the regulatory impact summary was consolidated as an 

enclosure to the annual ROP self-assessment SECY in SECY-05-0070, “Reactor Oversight 

Self-Assessment for Calendar Year 2004” (ADAMS Accession No. ML050630297), dated April 

25, 2005, and has been an enclosure ever since. The last stand-alone SECY for the regulatory 

impact analysis was SECY-03-0221, “Annual Report on Feedback from Licensees Regarding 

the Impact of NRC’s Activities on Licensees’ Operations” (ADAMS Accession No. 

ML033430115), dated December 22, 2003. Additionally, a trend-only metric was added to 

Appendix A, “Reactor Oversight Process Self-Assessment Metrics,” to Inspection Manual 

Chapter (IMC) 0307, "Reactor Oversight Process Self-Assessment Program," in 2006 to 

formally capture this effort as a part of the ROP self-assessment process. 

Since ROP implementation in 2000, the staff has concluded in each of the self-assessments 

that the favorable percentage of comments/feedback has remained high, with the past several 

averaging well over 90 percent. The distribution of comments was similar to previous years, 

and feedback was primarily in the areas of inspector performance and formal communication
- 2 - 

with licensees. The few unfavorable comments were isolated in nature or were focused in 

discreet areas that the NRC had previously identified and was addressing, and no additional 

programmatic adjustments have been made because of the analysis. In addition, the site visit 

observation forms had been forwarded to affected organizations for their consideration and 

potential action when they were originally submitted soon after the site visits. Specific 

management action on feedback was taken well before the annual self-assessment. 

Although the staff recommends discontinuing the detailed analysis in the enclosure to the 

annual SECY paper, aspects of the regulatory impact evaluation will remain in the updated selfassessment program. Specifically, the primary purpose of the site visit observation form is to 

ensure and record NRC management’s evaluation of inspector objectivity in accordance with 

IMC 0102, “Oversight and Objectivity of Inspectors and Examiners at Reactor Facilities.” NRC 

management will continue to record their site visits and ensure appropriate actions to address 

negative feedback, and this aspect will be captured as one or more performance metrics. In 

addition, numerous other forums will continue to provide redundant means to assess regulatory 

impact and unnecessary burden, including ongoing efforts to evaluate the cumulative effects of 

regulation and frequent interactions with industry during Regional Utility Group meetings, ROP 

Working Group public meetings, management drop-ins, and other venues. 

Resident Inspector Demographic Analysis

The ROP resident inspector annual demographic update was originally requested by the 

Commission in its non-publicly available SRM for COMGJD-98-001/COMEXM-98-002, 

“Discussion of Resident Inspector Demographics and the Balance between Expertise and 

Objectivity,” dated April 8, 1998 (ADAMS Accession No. ML003753515). The SRM requested, 

in part, that “the staff provide annual updates of the resident inspector demographics data and 

any recommendations warranted by the updated data until there is evidence that the agency’s 

actions have resulted in a stable or improving resident program.” Shortly after that SRM, the 

staff recommended and the Commission approved enhancements to the resident inspector 

staffing and compensation policies as noted in SECY-98-152, “Summary of Issues and 

Recommended Improvements to the Resident Inspector Program” (ADAMS Accession No. 

ML992880053), dated June 28, 1998, and SECY-98-281, “Resident Inspector Compensation 

Policy” (ADAMS Accession No. ML992870008), dated December 2, 1998. The enhancements 

included modifying the resident inspector pay schedule to include locality pay and revising the 

eligibility requirements for the saved pay provisions when transferring out of the program, while 

other policies such as relocation bonus provisions were maintained. Annually, the staff 

compiles and analyzes (1) resident inspector experience, including NRC time, total resident 

time, current site time, and relevant non-NRC experience, (2) resident inspector turnover and 

losses, and (3) resident inspector site staffing. The data is collected, analyzed, and presented 

in numerous graphs and charts to support the analysis. The analysis also develops and 

provides recommendations, if appropriate, to ensure that all sites continue to be staffed with 

knowledgeable and experienced resident inspectors. 

Originally a stand-alone annual SECY, the resident inspector demographic analysis was 

consolidated as an enclosure to the annual ROP self-assessment SECY in SECY-02-0062, 

“Calendar Year 2001 Reactor Oversight Self-Assessment” (ADAMS Accession 

No. ML020710781), dated April 3, 2002, and has been an enclosure ever since. The last 
- 3 - 

standalone SECY for the resident demographic analysis was SECY-01-0135, “Annual Update 

and Analysis of Demographic Data on Resident Inspectors” (ADAMS Accession No. 

ML011620424), dated July 24, 2001. Additionally, several trend-only metrics were added to 

Appendix A to IMC 0307 in 2002 (ADAMS Accession No. ML023650446), to formally capture 

this effort as a part of the ROP self-assessment process. In 2004, in response to a 

recommendation by the Davis-Besse Lessons Learned Task Force, the staff added a site 

staffing metric to evaluate the agency’s ability to provide continuity of onsite regulatory 

oversight. 

Although inspector experience and turnover have fluctuated from year to year and has created 

staffing challenges, the staff has concluded in each of the self-assessments that sites continue 

to be staffed with knowledgeable and experienced resident inspectors and senior resident 

inspectors. Furthermore, the staff believes that the Agency’s actions have resulted in a stable 

or improving resident program and thus meets the intent of the original SRM cited above. 

Although the regions use the data they gather to inform their hiring and staffing decisions and to 

ensure a qualified and experienced resident inspection staff, no programmatic adjustments have 

been recommended or made because of the analysis. Separately, an independent task force 

examined challenges in relocating and retaining employees in resident inspector positions as 

noted in the non-publicly available SECY-09-0050, “Actions to Enhance Relocation and 

Retention for Employees” (ADAMS Accession No. ML090570713), dated March 30, 2009. The 

task force determined that existing authorities and flexibilities should be more fully used or 

further developed, when appropriate, to enhance the Agency’s relocation and retention 

processes. As reported in the non-publicly available SECY-11-0180, “Effectiveness Review of 

Actions to Enhance Relocation and Retention of Employees,” (ADAMS Accession No. 

ML11347A263), dated December 21, 2011, the staff reviewed the effectiveness of the 

recommendations from 2009 and noted that improvements were realized and no changes to 

relocation or retention policies were recommended. 

The effort spent by the program office to collect, analyze, process, and communicate the data 

has not been justified by the outcomes gained. While the staff is recommending the removal of 

the Commission reporting requirement as part of the ROP self-assessment process, the staff 

will also explore ways to realign the program to measure meaningful data that can provide 

insights into inspector staffing and demographics. This revised demographic analysis would 

occur on a less frequent (triennial) basis, every three years. The staff will also coordinate any 

new approach under the broader umbrella of human capital management. 

Certain aspects of the resident inspector demographics evaluation will remain in the updated 

self-assessment program. Specifically, the permanent site staffing metric currently included as 

part of the resident inspector demographic analysis will be maintained as a performance metric. 

Regional management also will continue to monitor resident staffing and demographics to 

ensure that sites are staffed with qualified and experienced inspectors, and the staff will notify 

the Commission if challenges or concerns arise that impair the resident inspector program. 
- 4 - 

ROP Resource Expenditure Analysis

Before initial implementation of the ROP, the staff stated in SECY-00-0049, “Results of the 

Revised Reactor Oversight Process Pilot Program,” dated February 24, 2000 (ADAMS 

Accession No. ML003683227), that there would be no resource adjustments to the inspection 

and assessment programs until adequate experience was gained with the new process. A full 

year of implementation beyond the pilot program was planned to obtain reliable data on which to 

estimate the resources needed to accomplish individual inspections as well as to execute the 

overall ROP. In the first annual self-assessment after implementation of the ROP as 

documented in SECY-01-0114, “Results of the Revised Reactor Oversight Process Pilot 

Program,” dated June 25, 2001 (ADAMS Accession No. ML011410551), the staff included a 

resource expenditure analysis and concluded that the resources budgeted for initial 

implementation of the new oversight process were appropriate. The staff noted its intent to 

continue to refine the ROP regional resource model to look for other efficiencies, and to report 

to the Commission on an annual basis the results of its self-assessment and any significant 

changes to the ROP. Annually, the staff summarizes and analyzes the staff resources 

expended to implement the ROP during the past three calendar years, and explains any 

anomalies, as warranted. A trend-only metric was added to Appendix A to IMC 0307 in 2006 to 

formally capture this effort as a part of the ROP self-assessment process. 

The staff implemented coping measures and adjusted the inspection budget during the first few 

years of ROP implementation. Since that time, the staff has concluded in each of the selfassessments that although the resource expenditures have increased or decreased in certain 

areas, the overall expenditures have remained relatively stable. In addition, no significant 

programmatic adjustments have been recommended or made because of the analysis. 

Although the staff recommends discontinuing the detailed analysis and enclosure to the annual 

SECY paper, aspects of the ROP resource expenditures evaluation will remain in the 

self-assessment program. For example, the staff plans to continue conducting its periodic 

review and realignment of the ROP baseline inspection procedures in accordance with IMC 

0307, Appendix B, “Reactor Oversight Realignment,” to ensure the most effective overall 

application of inspection resources. The ROP realignment process is being revised based on 

lessons learned from the staff’s ROP enhancement efforts.