March 28, 2008 

MEMORANDUM TO: Marc L. Dapas, Deputy Regional Administrator, RI 

 Charles Casto, Acting Deputy Regional Administrator, RII 

 Mark Satorius, Deputy Regional Administrator, RIII 

 Arthur Howell, Acting Deputy Regional Administrator, RIV 

FROM: Frederick D. Brown, Director /RA/ 

 Division of Inspection and Regional Support 

 Office of Nuclear Reactor Regulation 

SUBJECT: REACTOR OVERSIGHT PROCESS ANNUAL SELF-ASSESSMENT 

 PERFORMANCE METRICS 

The Reactor Oversight Process (ROP) self-assessment process uses objective measures and 

predetermined criteria to monitor the performance of the ROP as described in Inspection 

Manual Chapter (IMC) 0307, “Reactor Oversight Process Self-Assessment Program.” The 

calendar year (CY) 2007 ROP self-assessment was performed in accordance with IMC 0307 

which was recently revised to consolidate and clarify several of the performance metrics. These 

metrics rely on information from various sources, including the Reactor Program System, the 

inspection program, periodic independent audits, stakeholder surveys, and public comments. 

The staff collects data quarterly and uses pre-established success criteria to analyze the data. 

The Nuclear Regulatory Commission (NRC) solicited comments on the eighth year of ROP 

implementation from external stakeholders in a Federal Register notice in October 2007. Of the 

seven respondents, four were from the utilities and/or their representatives, one was from a 

State agency, and two were from public interest groups or public citizens. There was no internal 

survey conducted in CY 2007 consistent with its biennial frequency as defined by IMC 0307. 

The next survey to internal stakeholders will be conducted during the fourth quarter of CY 2008.

Based on the NRC staff’s review, most of the metrics met the established criteria. Two metrics 

in the assessment (AS) area, two metrics in the performance indicator (PI) area, one metric in 

the inspection (IP) area and one metric in the significance determination process (SDP) area did 

not meet the established criteria. All Overall (O) ROP metrics met the established criteria. The 

NRC staff’s corrective actions to address these issues are discussed in the following 

paragraphs, in the enclosed metric analyses, and in the CY 2007 ROP self-assessment 

Commission paper. 

CONTACT: Ronald Frahm, NRR 

 301-415-2986 

 Donna Wright, NRR 

 301-415-1864 
M. Dapas, et al - 2 - 

PI Program Results

Two of the eight PI metrics did not meet the established criteria. PI-3, “Timely Indication of 

Declining Safety Performance,” which tracks PIs that cross multiple thresholds, did not meet 

established criteria because there were four occurrences at 3 sites, an increase from last year. 

PI-4, “PI Program Provides Insights to Help Ensure Plant Safety,” did not meet its criteria 

because public and state respondents gave feedback that the PIs do not provide an adequate 

indication of declining safety performance. The NRC staff recognizes the need to improve the 

PI Program and is reviewing the program to provide more timely and meaningful indications of 

declining plant performance to ensure plant safety. 

Inspection Program Results 

One of the nine inspection program metrics did not meet established criteria. IP-5, “Temporary 

Instructions (TI) Are Completed Timely,” failed to meet the established criteria of completing all 

TIs within the TI requirements. Only one site did not meet the criteria. The NRC staff is 

considering revising the metric criteria from “all” to 95 percent, consistent with other metrics, in 

the next revision of IMC 0307. 

SDP Results 

One of the seven SDP metrics, SDP-4, “The SDP Provides an Objective and Understandable 

Regulatory Response to Performance Issues,” did not meet its criteria. This is based on a 

stable negative perception by external stakeholders over the past seven years of ROP 

implementation. Findings are continuously under review by the NRC to determine the need for 

adjustments to the SDPs in this area and several recent changes have been made. SDP-3, 

“Inspection Staff Is Proficient and Find Value in Using the SDP,” was not analyzed in the CY 

2007 metric report because the internal survey was not conducted in CY 2007, consistent with 

its biennial frequency as defined by IMC 0307. 

Assessment Program Results

Two of the eight assessment program metrics did not meet established criteria. AS-4, “The 

NRC’s Response to Performance Issues Is Timely,” counts the elapsed time between issuance 

of an assessment letter and the respective supplemental inspection exit meeting date. The 

average elapsed time increased from last year. AS-7, “Degradations in Plant Performance are 

Gradual and Allow Adequate Agency Engagement of the Licensees,” tracks the number of 

instances plants moved more than one column to the right in the Action Matrix. There were five 

occurrences at 4 sites, an increase from last year. The NRC staff plans to address these issues 

during CY 2008 and continue to monitor them closely. AS-8, “Perceived Effectiveness of Safety 

Culture Enhancements to ROP,” was not applicable this assessment period. A more 

meaningful assessment of the AS-8 metric criterion will be achieved in follow-on years as the 

safety culture changes have been implemented for a longer period. 
M. Dapas, et al - 3 - 

Overall ROP Results

All 16 metrics in the Overall ROP area met the established criteria. 

Conclusions and Next Steps

The performance metrics provide the staff with valuable insights and lessons learned that lead 

to continued improvements in ROP effectiveness. This report provides a significant input into 

the annual ROP self-assessment and the resulting Commission paper. Aspects of this report, 

particularly the six metrics not met, will be discussed in the self-assessment paper under the 

respective program areas. The NRC staff will prepare and distribute a consolidated response to 

stakeholder comments from the CY 2007 external survey. 

Enclosure: As stated 
M. Dapas, et al. - 3 - 

Overall ROP Results

All 16 metrics in the Overall ROP area met the established criteria. 

Conclusions and Next Steps

The performance metrics provide the staff with valuable insights and lessons learned that lead 

to continued improvements in ROP effectiveness. This report provides a significant input into 

the annual ROP self-assessment and the resulting Commission paper. Aspects of this report, 

particularly the six metrics not met, will be discussed in the self-assessment paper under the 

respective program areas. The NRC staff will prepare and distribute a consolidated response to 

stakeholder comments from the CY 2007 external survey. 

Enclosure: As stated 

DISTRIBUTION: 

DRP/DRS 

ADAMS ACCESSION NUMBER: ML080350368 

OFFICE NRR/DIRS NRR/DIRS NRR/DIRS NRR/DIRS NRR/DIRS 

NAME DWright RFrahm JAndersen TKobetz FBrown 

DATE 03/12/2008 03/12/2008 03/13/2008 03/13/2008 03/28/2008 

OFFICIAL RECORD COPY 
ENCLOSURE 

Calendar Year 2007 Analysis of the 

Reactor Oversight Process Self-Assessment Metrics 

(In Accordance with IMC 0307, Appendix A) 
- 2 - 

PI-1 Consistent Results Given Same Guidance 

Definition: Independently verify PIs using Inspection Procedure (IP) 71151, “PI Verification.” 

Count all performance indicators (PI) that either (a) result in a crossed threshold 

based on a data correction by the licensee (as noted in the resultant inspection 

report), or (b) have been determined to be discrepant by the staff in accordance 

with IP 71150, “Discrepant or Unreported Performance Indicator Data.” 

Criteria: Expect few occurrences, with a stable or declining trend. 

Goals Supported: Objective, Predictable, Ensure Safety 

0

1

2

3

4

5

1Q/042Q/043Q/044Q/041Q/052Q/053Q/054Q/051Q/062Q/063Q/064Q/061Q/072Q/073Q/074Q/07

Calendar Quarter

Number of Occurrences

Comments: The graph represents the number of significant deficiencies or discrepant PIs 

reported for each quarter. Significant discrepancies are issues identified by the 

NRC during a PI verification inspection that caused the PI to cross a threshold. 

Analysis: During this assessment period there were two PIs that resulted in a crossed 

threshold based on a data correction by the licensee and no PIs were identified 

as discrepant. 

 

Ginna crossed the threshold from Green to White for the Emergency Response 

Organization (ERO) Drill Participation PI when the data for 4Q06 was re-reported 

with the 1Q07 data submittal. Ginna moved to Yellow for 1Q07 as a result of the 

changed data. 

Vogtle Unit 2 crossed the threshold from Green to White for the Mitigating 

System Performance Index - Cooling Water Systems PI when the data for 4Q06 

was re-reported with the 1Q07 data submittal. 

Metric Criterion Met: Yes. 
- 3 - 

PI-2 Questions Regarding Interpretation of PI Guidance

Definition: Quarterly, count the number of frequently asked questions (FAQs). 

Criteria: Expect low numbers, with a stable or declining trend. 

Goals Supported: Understandable, Risk-Informed, Predictable

0

5

10

15

20

1Q/042Q/043Q/044Q/041Q/052Q/053Q/054Q/051Q/062Q/063Q/064Q/061Q/072Q/073Q/074Q/07

Calendar Quarter

Number of FAQs

Comments: Each quarter represents the total number of new FAQs introduced during the 

ROP NRC/Industry Working Group meetings held during the respective quarter. 

Analysis: There is currently a stable long term trend. The number of FAQs introduced in 

CY 2007 has been comparable to previous years. 

Metric Criterion Met: Yes. 
- 4 - 

PI-3 Timely Indication of Declining Safety Performance

Definition: Quarterly, track PIs that cross multiple thresholds (e.g., green to yellow or white 

to red). Evaluate and characterize these results to allow timely indication of 

declining performance. 

Criteria: Expect few occurrences, with a stable or declining trend. 

Goals Supported: Ensure Safety, Risk-Informed, Ensure Effectiveness 

0

1

2

3

4

5

1Q/042Q/043Q/044Q/041Q/052Q/053Q/054Q/051Q/062Q/063Q/064Q/061Q/072Q/073Q/074Q/07

Calendar Quarter

Number of Occurrences

Analysis: There were four occurrences of PIs crossing multiple thresholds during this 

assessment period: 

1) D.C. Cook Units 1 and 2 moved from Green to Yellow for the Alert and 

Notification System PI with the 2Q07 data submittal. 

2) Salem 1 moved from Green to Yellow for the Mitigating System Performance 

Index - Emergency AC Power PI with the 4Q07 data submittal. 

3) Browns Ferry 1 moved from Green to Yellow for the Unplanned Scrams with 

Complications PI with the 4Q07 data submittal. 

This metric did not meet its criteria based on the fact that three sites (four units) 

crossed multiple thresholds in CY 2007. This is an increasing trend from 

previous years. Also, each site crossed multiple thresholds due to a different PI, 

indicating that this is not PI dependent or an isolated occurrence. The 

assessment period data indicates an adverse trend that requires additional 

attention and close monitoring from the staff. 

Metric Criterion Met: No. 
- 5 - 

PI-4 PI Program Provides Insights to Help Ensure Plant Safety

Definition: Survey external and internal stakeholders asking whether the PI Program 

provides useful insights to help ensure plant safety. 

Criteria: Expect a low number of negative comments, with a stable or declining trend. 

Goals Supported: Ensure Safety, Ensure Effectiveness, Risk-Informed 

NOTE: There was no internal survey in CY 2007 consistent with its biennial frequency as 

defined by IMC 0307. The next survey to internal stakeholders will be conducted 

during the fourth quarter of CY 2008. 

Analysis: Responses from the public questioned the PI Program due to the declining 

number of greater-than-green PIs but not a proportional decrease in the number 

of reactors receiving heightened NRC attention. A public response also stated 

that PIs only provide some insight to plant safety, referring to the Davis Besse 

plant; and that it was difficult to show that some PIs are actually related to plant 

safety. 

 The State respondent agreed that PIs are based on operating safety but 

questioned whether the indicators in the PI program are periodically “re-set” to 

reflect the differences in observed occurrences and the current expectations. 

Industry responses were generally in agreement that the PI program does 

provide useful insights to ensure plant safety but noted that efforts should 

continue to better risk-inform the PIs and improve the level of insight they 

provide. Industry responses also stated that initial indications of the 

enhancements (e.g., Scrams with Complications and Mitigating Systems 

Performance Index PIs) made, improved the program but need more time to fully 

assess the effectiveness of the changes. 

The staff recognizes the need to improve the PI Program to provide more 

meaningful indications of declining plant performance. This metric does not meet 

its established criteria because public and State respondents gave overall 

feedback that the PIs do not provide an adequate indication of declining safety 

performance. Also, the internal survey conducted last year noted adverse trends 

regarding the NRC staff’s view of the public’s confidence in the PI program. 

Many comments noted that threshold for the PIs is too high to identify declining 

performance and, because the PIs are almost always green, they do not 

enhance public confidence. 

Metric Criterion Met: No. 
- 6 - 

PI-5 Timely PI Data Reporting and Dissemination

Definition: Within 5 weeks of the end of each calendar quarter, track (count) late PI postings 

on the NRC’s external Web site. Also note the number of late submittals from 

licensees that did not meet the 21-day timeliness goal. 

Criteria: Expect few occurrences, with a stable or declining trend. 

Goals Supported: Ensure Effectiveness, Ensure Openness, Predictable 

Analysis: There were no late postings on the NRC’s external Web site. 

 

 One power plant provided a PI submittal that was late; however, the submittal 

was made within a few days after the date required by NEI 99-02, “Regulatory 

Assessment Performance Indicator Guideline.” The late PI data submittal from 

the licensee did not impact the NRC’s ability to post the results on the web page 

in a timely manner. 

 The criteria for this metric have been met because there have been no late PI 

data postings on the NRC's external web site since the inception of the ROP. 

Metric Criterion Met: Yes. 

0

1

2

3

4

5

1Q/042Q/043Q/044Q/041Q/052Q/053Q/054Q/051Q/062Q/063Q/064Q/061Q/072Q/073Q/074Q/07

C a len dar Q uarter

Number of Untimely Actions
- 7 - 

PI-6 Stakeholders Perceive Appropriate Overlap Between the PI Program and 

Inspection Program 

Definition: Survey external and internal stakeholders asking if appropriate overlap exists 

between the PI program and the inspection program. 

Criteria: Expect a low number of negative comments, with a stable or declining trend. 

Goals Supported: Ensure Effectiveness, Ensure Safety, Ensure Openness 

NOTE: There was no internal survey in CY 2007 consistent with its biennial frequency as 

defined by IMC 0307. The next survey to internal stakeholders will be conducted 

during the fourth quarter of CY 2008.

Analysis: Public response varied regarding proper overlap between the PI and inspection 

programs. One public response again noted the ineffectiveness of the PI 

program. Another public response agreed that appropriate overlap exists but 

gave an example of not enough overlap to catch some safety issues. 

The State response indicated that there appears to be appropriate overlap but 

was not knowledgeable enough to ascertain the information. 

Industry comments noted appropriate overlap overall, but some stated concerns 

with possible double counting of inspection findings and performance indicator 

results that arise from the same event. 

There was a decline in the number of negative comments from CY 2006; 

therefore this metric has been met. 

Metric Criterion Met: Yes. 
- 8 - 

PI-7 Clarity of Performance Indicator Guidance

Definition: Survey external and internal stakeholders asking if NEI 99-02, “Regulatory 

Assessment Performance Indicator Guideline,” provides clear guidance 

regarding performance indicators. 

Criteria: Expect a low number of negative comments or examples of interpretation issues, 

with a stable or declining trend in the number of negative comments received. 

Goals Supported: Understandable, Ensure Openness, Objective 

NOTE: There was no internal survey in CY 2007 consistent with its biennial frequency as 

defined by IMC 0307. The next survey to internal stakeholders will be conducted 

during the fourth quarter of CY 2008.

Analysis: Public responses stated that combined with the FAQ process, the guidance was 

clear. 

State stakeholders generally felt that the guidance was clear but that it would be 

more appropriate for the licensees to provide comments on effectiveness of the 

PI guidance. 

Industry respondents commented that the PI guidance is clear and that the FAQ 

process and the appeal process have proven to be responsive and effective in 

addressing questions and resolving issues. 

Based on comments, MSPI guidance is effective but needs improvement, which 

will be an on-going process. There was a decline in the number of negative 

comments from CY 2006; therefore this metric has been met. 

Metric Criterion Met: Yes. 
- 9 - 

PI-8 PI Program Identifies Performance Outliers In an Objective and Predictable 

Manner

Definition: Survey external and internal stakeholders asking if the PI program can effectively 

identify performance outliers based on risk-informed, objective, and predictable 

indicators. 

Criteria: Expect a low number of negative comments, with a stable or declining trend. 

Goals Supported: Risk-Informed, Objective, Predictable 

NOTE: There was no internal survey in CY 2007 consistent with its biennial frequency as 

defined by IMC 0307. The next survey to internal stakeholders will be conducted 

during the fourth quarter of CY 2008.

Analysis: Public comments state that the PI program could and does identify outliers when 

not manipulated by the industry and is also dependent on the type of 

performance and indicator. 

State stakeholders noted that the PI program provides the ability to track and 

trend, but as noted earlier, the thresholds may need periodic re-examination. 

Industry comments were favorable about the PI program, particularly regarding 

MSPI because it is risk based and incorporates unavailability and unreliability. 

Industry responses also encouraged continued development of risk informed 

elements for future and existing PIs. 

The staff is continually reviewing and revising PIs to provide meaningful 

indications of plant performance and to better identify performance outliers. 

There was a decline in the number of negative comments from CY 2006; 

therefore this metric has been met. 

Metric Criterion Met: Yes. 
- 10 - 

IP-1 Inspection Findings Documented In Accordance With Requirements

Definition: Audit inspection reports in relation to program requirements (IMC 0612, “Power 

Reactor Inspection Reports”) for documenting green findings, greater-than-green 

findings, and violations. Report the percentage of findings that meet the program 

requirements. 

Criteria: Expect a stable or improving trend in the percentage of findings documented in 

accordance with program requirements. 

Goals Supported: Objective, Risk-Informed, Predictable 

70%

75%

80%

85%

90%

95%

100%

2004 2005 2006 2007

Calendar Year

% Findings that Meet All Requirements

Analysis: The staff audited integrated inspection reports from each branch and a number of 

team inspection reports from each region. Of the 618 inspection reports issued 

in CY 2007, 43 were included in this audit. The percentage of findings 

documented in accordance with IMC 0612 requirements for the sample audited 

was 96 percent. Overall the data confirms that a stable trend has been 

maintained since 2004. 

Metric Criterion Met: Yes. 
- 11 - 

IP-2 Number of Feedback Forms per Document

Definition: Count the number of feedback forms received for each program document each 

quarter. Use a histogram to chart the number of documents for which feedback 

forms were received. Analyze the trend in number of feedback forms received 

for individual program documents, taking into account major changes in the 

documents and feedback forms unrelated to the document itself. 

Criteria: Expect a declining trend in the number of feedback forms received for program 

documents. 

Goals Supported: Understandable, Predictable, Objective 

Analysis: The staff received 123 feedback forms in CY 2007. Approximately 66 percent of 

all feedback forms received during this period relate to issues in the areas of: 

(1) IMC 0612, Power Reactor Inspection Reports - (22 Forms), 17%; 

(2) IMC 0305, ROP Assessment - (10 Forms), 8%; 

(3) IMC 0609, Significance Determination Process (SDP) including all 

Appendices - (9 Forms), 7%; 

(4) IMC 1245, Qualification Program for the Office of Nuclear Reactor Regulation 

Programs – (19 forms), 15% 

(5) IMC 2515, Appendix D, Plant Status – (8 forms), 7% 

(6) IP 71151, Performance Indicator Verification - (5 Forms), 4%. 

(7) IP 95001, Inspection for One or Two White Inputs in a Strategic Performance 

Area – (8 forms), 7% 

Of the 123 feedback forms received this calendar year, the staff resolved 101 

forms (82%). Overall, the staff has resolved 191 feedback forms from a total of 

223 feedback forms (86%). The total number of feedback forms includes 100 

feedback forms brought forward from CY 2006 of which, the staff closed 90. 

There were 32 feedback forms open at the end of CY 2007. This number 

included the remaining 22 forms received this year and 10 forms carried over 

from CY 2006. 

Changes to the ROP Feedback Process implemented at the beginning of FY 

2007 had mixed results. The use of the NRR Work Planning and Control Center 

for resource usage and timeliness tracking proved cumbersome. It was of no 

benefit for feedback resolution within the NRR inspection branches or that 

required coordination outside of the Office of Nuclear Reactor Regulation, and 

only limited benefit within the other NRR branches. The process will be refined 

to allow flexibility in the use of the work planning process where appropriate. 

Significant progress in staff performance and responsiveness to feedback 

occurred as expected in the past year, with a reduction in the backlog of 

unresolved feedback from 57 forms in 2006 to 32 forms this year. In the coming 

year, the staff will concentrate on implementation of agreed upon resolutions of 

feedback. There is a current backlog of 106 resolved feedback forms that 
- 12 - 

identified changes to be implemented to ROP documentation. Of these 106 

pending changes, 22 forms exceed a year in age. 

Feedback submission demonstrates a downward trend for IMC 0612, IMC 0609 

and its appendices, and IMC 1245 which meets the metric criterion. The 

submission rate for IMC 0305 was slightly positive. The trend for IMC 0305 

submissions is driven in part by a high number of submissions in early 2006 and 

as the result of safety culture enhancements. A review of the submissions 

showed that they included 3 duplicates that were administratively cancelled and 

the majority were the result of solicitation of feedback on the assessment process 

following the end of cycle assessment meetings for the prior calendar year. This 

trend is not statistically significant, particularly when corrected for duplicate 

submissions. A slight upward trend is noted for feedback linked to IP 71151. 

Feedback for this procedure focused primarily on the interpretation of 

performance indicators or their guidelines, NEI 99-02, rather than on the 

procedure itself. Discounting these submissions, a downward trend in feedback 

is present. A slight upward trend was noted in feedback received for IMC 2515 

Appendix D. This trend is statistically insignificant, in part due to the low number 

of feedback forms received. The feedback rate for IP 95001 showed an upward 

trend, driven by the submission of seven forms in the first quarter of CY 2007. 

Field implementation of IP 95001 at the Palisades plant had resulted in the lead 

inspector submitting almost all of the IP 95001 feedback forms. The feedback 

provided value added recommendations to further enhance IP 95001 and related 

inspection guidance documents. The upward trend does not appear to be 

statistically significant due to the otherwise low number of submissions. 

With the few exceptions detailed above, the feedback received on the ROP 

exhibited downward trends. For those exceptions, drivers not impacting the 

understandability, predictability, and objectivity of the documentation were 

present.

Metric Criterion Met: Yes. 
- 13 - 

IP-3 Completion of Baseline Inspection Program 

Definition: Annual completion of baseline inspection program. 

Criteria: Defined as per IMC 2515, “Light-Water Reactor Inspection Program - Operations 

Phase.” 

Goals Supported: Ensure Safety, Predictable, Ensure Effectiveness 

Analysis: All four regions completed their baseline inspections in CY 2007. Each region 

documented completion of the program in a memorandum to the Division of 

Inspection and Regional Support in NRR. These memoranda can be found in 

ADAMS under ML080430029 (Region I), ML080770153 (Region II), ML080450429

(Region III), and ML080730456 (Region IV). As in the 2006 inspection cycle, all 

regions completed their baseline inspections in 2007 with the allocated regional 

resources.

In addition, NSIR documented completion of the security baseline inspection 

program (ML080390446), but this document is not publicly available.

Metric Criterion Met: Yes. 
- 14 - 

IP-4 Inspection Reports Are Timely

Definition: Obtain RPS data on the total number of reports issued and the number issued 

within timeliness goals as stipulated in IMC 0612, “Power Reactor Inspection 

Reports.” 

Criteria: Expect 90 percent of inspection reports to be issued within program's timeliness 

goals. 

NOTE: For inspections not conducted by a resident inspector, inspection completion is 

normally defined as the day of the exit meeting. For resident inspector and 

integrated inspection reports, inspection completion is normally defined as the 

last day covered by the inspection report. 

Goals Supported: Ensure Effectiveness, Ensure Openness, Predictable 

85%

90%

95%

100%

Q1/05 Q2/05 Q3/05 Q4/05 Q1/06 Q2/06 Q3/06 Q4/06 Q1/07 Q2/07 Q3/07 Q4/07

Calendar Quarter

% of Timely Reports

RI RII RIII RIV

Analysis: A total of 618 inspection reports were issued during CY 2007. Regions met or 

exceeded the inspection report timeliness goal of 90 percent in each quarter 

throughout the year. In CY 2007, only one inspection report out of the 618 failed 

to meet the timeliness requirement per IMC 0612. 

Metric Criterion Met: Yes. 
- 15 - 

IP-5 Temporary Instructions (TIs) Are Completed Timely

Definition: Audit the time to complete TIs by region. Compare the completion status in RPS 

to TI requirements. Report by region the percentage of TIs closed within goals. 

Criteria: Expect all TIs to be completed within TI requirements. 

Goals Supported: Ensure Effectiveness, Ensure Safety, Predictable 

0%

20%

40%

60%

80%

100%

120%

2005 2006 2007

Year

Percent complete on time 

R I R II R III R IV

Analysis: Only one TI had an expiration date in 2007 (2515/154, “Spent Fuel Material 

Control and Accounting at Nuclear Power Plants,” expired on August 31, 2007). 

Although the TI was completed at 98 percent of the sites within timeliness goals, 

one site completed the TI approximately 3 weeks past the expiration date. The 

delay was necessary after the licensee identified issues affecting their readiness 

for the inspection, which in turn delayed NRC inspection efforts. The staff is 

considering changing the criteria from 100 percent complete to 95 percent in the 

next revision of IMC 0307 to allow for conditions beyond the staff’s control. 

 

Metric Criterion Met: No. 
- 16 - 

IP-6 Public Communication Is Accurate

Definition: Each calendar quarter, sample information on the NRC's external (public) Web 

site and count the number of times and reasons for regions changing PIMs or 

inspection reports (i.e., inaccuracy, new information). 

Criteria: Expect few inaccuracies, with a stable or declining trend. 

Goals Supported: Ensure Openness, Ensure Effectiveness, Understandable 

0

1

2

3

4

5

6

7

8

9

10

Q1/05 Q2/05 Q3/05 Q4/05 Q1/06 Q2/06 Q3/06 Q4/06 Q1/07 Q2/07 Q3/07 Q4/07

Calendar Quarter

Number of Inaccuracies

Analysis: There were few inaccurate postings of PIM entries or inspection reports on the 

web identified during CY 2007. No region had more than two inaccurate 

postings per quarter. The total number of inaccurate postings has been steadily 

declining since CY 2005. This metric met program expectations. 

Metric Criterion Met: Yes. 
- 17 - 

IP-7 Inspection Reports Are Relevant, Useful, and Written in Plain Language

Definition: Survey external and internal stakeholders asking whether the information 

contained in inspection reports is relevant, useful, and written in plain English. 

Criteria: Trend average level of agreement. 

Goals Supported: Ensure Effectiveness, Understandable, Ensure Openness 

NOTE: There was no internal survey in CY 2007 consistent with its biennial frequency as 

defined by IMC 0307. The next survey to internal stakeholders will be conducted 

during the fourth quarter of CY 2008. 

Analysis: External stakeholders generally agreed that the inspection reports were clearly 

written and useful. However, the following comments for improvement were 

provided: 

• More effort is spent documenting the details of an issue then in providing 

insights as to why the issue is a concern. The true nature, magnitude, and 

risk significance of the issue is often not addressed. When cross-cutting 

aspects are assigned to findings and/or violations, the inspection report 

documentation sometimes lacks a clear explanation as to how the crosscutting aspect is a significant contributor to the cause of the issue. 

• Current inspection reports are so concise that it is often difficult to obtain 

useful information. It is even more difficult to read last year’s report and 

figure out exactly what inspectors observed. 

The NRC staff is evaluating these comments and plans to address them in the 

consolidated response. 

Metric Criterion Met: Yes. 
- 18 - 

IP-8 Inspection Program Effectiveness and Adequacy in Covering Areas 

Important to Safety

Definition: Survey external and internal stakeholders asking whether the inspection program 

adequately covers areas that are important to safety and is effective in identifying 

and ensuring the prompt correction of performance deficiencies. 

Criteria: Trend average level of agreement. 

Goals Supported: Ensure Safety, Ensure Effectiveness, Risk-Informed 

NOTE: There was no internal survey in CY 2007 consistent with its biennial frequency as 

defined by IMC 0307. The next survey to internal stakeholders will be conducted 

during the fourth quarter of CY 2008. 

Analysis: The responses were positive with the following comments for improvement: 

• The ROP realignment is only one method for monitoring the inspection 

program and may not be the most efficient. Obtaining, reviewing, and 

acting on continuous feedback from licensees and inspection personnel 

can be more timely and efficient. Maintaining a self-critical, questioning 

position regarding the value of inspection activities is a key to keeping the 

inspection program vital and robust. 

• Enhancements could be made to the inspection program to make better 

use of the NRC generic communications program (such as technical 

questions identified during inspections that involve development of new 

regulatory positions). Two examples of inspection issues with generic 

implications are 1) manual actions for response to fires and 2) 

assessment of post-fire safe shut down equipment. A process is needed 

to ensure early stakeholder involvement in the identification and 

resolution of inspection issues that potentially have generic implications. 

• Performance Identification and Resolution (PI&R) inspections fail to 

effectively identify and mitigate performance deficiencies in a timely 

manner. It would be beneficial if NRC inspectors would follow-up on 

findings from the NRC Inspection Program as opposed to examining a 

large sample of corrective action reports. By using NRC inspection 

findings to define the sample, the audit automatically begins with 

evidence of corrective action program problems. 

• Inspectors need more freedom to inspect rather than follow prescribed 

quotas of items. The program does not ensure corrective action but only 

that an item is entered into the corrective action program. 

The NRC staff is evaluating these comments and plans to address them in the 

consolidated response. 

Metric Criterion Met: Yes. 
- 19 - 

IP-9 Analysis of Baseline Inspection Procedures

Definition: Annually, review each baseline inspection procedure to determine its 

effectiveness and contribution to the overall effectiveness of the baseline 

inspection program. The objectives of the review are: (1) to determine if changes 

in scope, frequency, or level of effort are needed based on recent experience, (2) 

to determine if a change to the estimated hours for completion is needed, (3) to 

define or change what constitutes minimum completion of each inspectable area, 

if needed, and (4) to critically evaluate all of the inspectable areas together along 

with the PI program to ensure that the inspectable areas are adequately 

monitored for safety performance. In addition, a more detailed review and 

realignment of inspection resources will be performed at least biennially in 

accordance with Appendix B to this Chapter. The focus of this effort is to adjust 

existing inspection resources to improve the effectiveness of the inspection 

program in identifying significant licensee performance deficiencies. 

 

Criteria: None; trend only. Summarize and evaluate the individual inspection procedure 

reviews and propose program adjustments as necessary to address noted 

inefficiencies. Provide basis for any meaningful increase or decrease in 

procedure scope, frequency, or level of effort as a result of the review. 

Goals Supported: Ensure Effectiveness, Ensure Safety 

Analysis: The inspection program staff conducted the biennial review of the ROP 

inspection procedures to ensure most effective application of inspection 

resources. This biennial review process is part of the ROP self-assessment 

program and it is described in Appendix B to Inspection Manual Chapter 0307, 

“ROP Alignment Process.” As part of this review, the NRR staff established a 

working group to gauge the effectiveness of each of the inspection procedures. 

The working group examined the inspection resources used for each procedure 

in the Initiating Events, Mitigating Systems, Barrier Integrity, Occupational 

Radiation Safety, and Public Radiation Safety cornerstones to determine whether 

appropriate inspection resources were being applied. Inspection procedures in 

the EP and Security cornerstones were not reviewed as part of the ROP 

realignment effort in CY 2007. The review considered inspection results over a 

3-year period (CY 2004 through CY 2006). 

The working group consisted of NRR staff and each of the four regions. The 

results of the ROP working group were shared with senior NRR management 

from Division of Inspection and Regional Support (DIRS), NRR, and with regional 

Division Directors from Division of Reactor Projects and Reactor Safety. Majority 

positions were reached by senior regional managers for those inspection 

procedures which were determined to require a revision to the inspection 

program element of the ROP. The staff made changes affecting inspection 

scope and frequency to 12 baseline inspection procedures and implemented the 

revised baseline inspection program beginning in CY 2008. The results of the 

2007 ROP realignment process can be obtained from the Agencywide 

Documents Access and Management System (ADAMS ML073020593). The 

staff plans to perform the next ROP realignment in CY 2009, and the baseline 
- 20 - 

inspection program will reflect any changes resulting from that effort starting in 

CY 2010. 

In addition to the detailed ROP realignment process, the staff performed its 

annual evaluation of the inspection procedures in fiscal year (FY) 2007 to 

determine whether any additional improvements to the baseline inspections were 

warranted based on inspection findings over the most recent FY. The staff also 

performed a best practices review of the problem identification and resolution 

inspection procedure (IP 71152, “Identification and Resolution of Problems”). 

The purpose of the review was to help ensure consistent implementation of the 

procedure and to identify potential effectiveness and efficiency improvements. 

The staff made recommendations and identified some potential changes as a 

result of these reviews that will be evaluated in CY 2008.

Metric Criterion Met: Yes. 
- 21 - 

SDP-1 The SDP Results Are Predictable and Repeatable and Focus Stakeholder 

Attention on Significant Safety Issues

Definition: Annually, audit a representative sample (up to four per region) of inspection 

findings against the standard criteria set forth in IMC 0609, ASignificance 

Determination Process,@ and its appendices. To the extent available, samples 

should include potentially greater-than-green findings that were presented to the 

Significance Determination Process/Enforcement Review Panel (SERP). 

Findings should contain adequate detail to enable an independent auditor to 

trace through the available documentation and reach the same significance color 

characterization. 

Criteria: The target goal is at least 90% are determined to be predictable and repeatable. 

Any SDP outcomes determined to be non-conservative will be evaluated and 

appropriate programmatic changes will be implemented. 

Goals Supported: Ensure Safety, Risk-Informed, Predictable 

Analysis: An independent review of 32 inspection findings from the 2006-2007 inspections 

was performed by the Division of Risk Assessment in NRR. The representative 

sample was doubled to ensure accuracy of the results given the metric was 

revised and this review is its first use. Each sample represents a finding 

assessed using the risk-informed process detailed in IMC 0609 Appendix A, 

“Determining the Significance of Reactor Inspection Findings for At-Power 

Situations.” All samples were determined to include adequate detail to be 

predictable and repeatable and the documentation of the final determination and 

evaluation of the green findings was clear, and is consistent with past reviews. 

Performance during this assessment period meets program expectations. 

Metric Criterion Met: Yes. 
- 22 - 

SDP-2 SDP Outcomes Are Risk-Informed and Accepted by Stakeholders

Definition: Track the total number of appeals of final SDP results. 

Criteria: Expect zero appeals of SDP significance that result in a final determination being 

overturned across all regions. All successful appeals will be assessed to 

determine causal factors and to recommend process improvements. 

Goals Supported: Risk-Informed, Objective, Predictable 

0

1

2

3

4

5

6

7

1Q/042Q/043Q/044Q/041Q/052Q/053Q/054Q/051Q/062Q/063Q/064Q/061Q/072Q/073Q/074Q/07

Calendar Quarter

Number of Appeals

SDP Appeals Successful Appeals

Analysis: There were two appeals by licensees of SDP significance outcomes. Both of the 

appeals were aimed at findings of low to moderate safety significance (i.e., 

WHITE findings). One appeal was for an emergency preparedness issue and 

the other for a reactor mitigating system. In each case, the final outcome was 

unchanged. 

 Performance during this assessment period meets program expectations based 

on no successful appeals of SDP significance determinations that result in a final 

greater-than-green being overturned. 

Metric Criterion Met: Yes. 
- 23 - 

SDP-3 Inspection Staff Is Proficient and Find Value in Using the SDP

Definition: Survey internal stakeholders using specific quantitative survey questions that 

focus on training, effectiveness, and efficiency. 

Criteria: Expect either a stable or an increasingly positive perception of the SDP process 

over time. 

Goals Supported: Ensure Effectiveness, Understandable, Risk-Informed 

NOTE: There was no internal survey in CY 2007 consistent with its biennial frequency as 

defined by IMC 0307. The next survey to internal stakeholders will be conducted 

during the fourth quarter of CY 2008. 
- 24 - 

SDP-4 SDP Provides an Objective and Understandable Regulatory Response to 

Performance Issues

Definition: Survey external and internal stakeholders asking if the SDP results in an 

objective and understandable regulatory response to performance issues. 

Criteria: Expect stable or increasingly positive perception of the SDP over time. 

Goals Supported: Understandable, Objective, Predictable 

NOTE: There was no internal survey in CY 2007 consistent with its biennial frequency as 

defined by IMC 0307. The next survey to internal stakeholders will be conducted 

during the fourth quarter of CY 2008. 

Analysis: Generally, the respondents believe the SDP is a useful tool to quickly determine 

a plant’s status in specific oversight areas and that it is generally risk-informed; 

however, it remains complex requiring one to be an expert on the SDP process. 

More than half of the industry respondents stated that the Radiation Protection, 

Security, and Emergency Preparedness SDPs are subjective and deterministic, 

not based on risk or actual effect thresholds and may result in exaggeration of 

actual risk. 

 For reactor SDPs supported by SPAR modeling, the industry representatives 

believe the methodology minimizes realistic factors such as operator actions and 

tends to utilize hypothetical worst case assumptions. While the steps in an SDP 

decision are well outlined, they feel that the bases and rationale for these steps 

are often neither objective nor predictable and can be inconsistent with industry 

norms for probabilistic risk assessment (PRA) decisions. They further believe 

that the NRC should abandon the use of SPAR and use the Licensee’s 

Regulatory Guide 1.200 compliant PRA models to support the SDP process. 

 In the first half of 2007, the NRC met with representatives from industry and other 

stakeholders, in a series of public meetings, to discuss the industry proposal to 

use industry PRA analyses in lieu of the NRC risk assessment tools. The NRC 

reviewed the industry proposal and concluded that the reactor oversight process 

required the NRC to maintain independence by evaluating the significance of 

findings and not just review the results of licensee’s assessment. At present, the 

industry has not uniformly implemented a standardized approach to performing 

risk analysis that would ensure uniform application across the spectrum of 

industry PRA models. In this regard, the NRC’s use of the SPAR models, 

together with the ongoing development of guidance on conducting Phase 3 risk 

assessments, commonly referred to as the risk assessment standardization 

project (RASP), ensures greater uniformity in the agency’s regulatory 

assessments. 

 This metric does not meet its criteria based on a stable negative perception over 

the past seven years of ROP implementation. 

Metric Criterion Met: No. 
- 25 - 

SDP-5 The Resources (Direct Charges and Support Activities) Expended Are 

Appropriate

Definition: Track the percentage of total resource expenditures attributed to SDP activities 

to determine the effort expended by the regions in completing SDP evaluations 

as a percentage of the total regional direct inspection effort. 

Criteria: Total SDP expenditures should not exceed 10 percent of the total regional direct 

inspection effort (DIE) with a stable or declining trend. 

Goals Supported: Ensure Effectiveness, Predictable 

0

1

2

3

4

5

6

7

8

9

10

1Q/042Q/043Q/044Q/041Q/052Q/053Q/054Q/051Q/062Q/063Q/064Q/061Q/072Q/073Q/074Q/07

Calendar Quarter

Percentage of DIE for SDP Efforts (%)

R1 R2 R3 R4 National

Analysis: Regional expenditures associated with SDP evaluations remain stable and below 

the target goal. Performance during this assessment period meets program 

expectations. 

Metric Criterion Met: Yes. 
- 26 - 

SDP-6a Final Significance Determinations Are Timely

Definition: Conduct a quarterly audit of RPS data to identify the total number of inspection 

items finalized as greater than green that were under review for more than 90 

days since: 

(1) the date of initial licensee notification of the preliminary 

significance in an inspection report, or 

(2) the item was otherwise documented in an inspection report as an 

apparent violation pending completion of a significance 

determination. 

Criteria: At least 90% of all SDP results that are counted per the criteria above should be 

finalized within 90 days. All issues greater than 90 days will be assessed to 

determine causal factors and to recommend process improvements. 

Goals Supported: Ensure Effectiveness, Ensure Openness, Predictable 

0%

20%

40%

60%

80%

100%

1Q/042Q/043Q/044Q/041Q/052Q/053Q/054Q/051Q/062Q/063Q/064Q/061Q/072Q/073Q/074Q/07

Calendar Quarter

Actions Completed within 90 days

R1 R2 R3 R4 Total

Analysis: Timeliness of final significance determinations increased from 96% for FY 2006 

to 100% for FY 2007. Performance during this assessment period met and 

exceeded program expectations. 

Metric Criterion Met: Yes. 
- 27 - 

SDP-6b Final Significance Determinations Are Timely

Definition: Conduct a quarterly audit of issues that were assessed by the Significance 

Determination Process/Enforcement Review Panel (SERP) to identify the total 

number of inspection items finalized as green or greater-than-green that were 

under review for more than 90 days since: 

(1) the date of initial licensee notification of the preliminary significance in 

an inspection report or otherwise documented in an inspection report as 

an AAV@ pending completion of a significance determination, or 

(2) the date the item was presented to the SERP for review. 

Criteria: At least 90% of all SDP results that are counted per the criteria above should be 

finalized within 90 days on average and 100% in 180 days. All issues greater 

than 180 days will be assessed to determine causal factors and to recommend 

process improvements. 

NOTE: This metric is being piloted as a potential replacement for the existing 

SDP timeliness metric. 

Goals Supported: Ensure Effectiveness, Ensure Openness, Predictable 

Analysis: No issues assessed by the SERP exceeded 90 days under review. The average 

age of all the SDP results that were presented to the SERP during FY 2007 

issues was 62 days. 

Metric Criterion Met: Yes. 
- 28 - 

SDP-7 SDP Results Are Communicated Accurately to the Public

Definition: Each calendar quarter, track the number of inspection findings that are 

inaccurately communicated to the public (color of findings is inaccurately 

reported) by auditing the inspection findings summary information available on 

the NRC Web. The detailed review will include item type, significance 

characterization, enforcement action status, and text descriptions of greater-thangreen inspection findings prior to release to external stakeholders. 

Criteria: The target goal is zero inaccuracies, with a stable or declining trend. All 

inaccuracies must be addressed. 

Goals Supported: Ensure Openness, Understandable, Ensure Effectiveness 

0

0.5

1

1.5

2

2.5

3

3.5

2004 2005 2006 2007

Calendar Quarter

Number of Inaccuracies

Analysis: During the current assessment period no inaccuracies were identified. 

Performance during this assessment period meets program expectations. 

Metric Criterion Met: Yes. 
- 29 - 

AS-1 Actions Are Determined by Quantifiable Assessment Inputs (i.e., PIs and 

SDP Results) and are Commensurate With the Risk of the Issue and Overall 

Plant Risk. 

Definition: Audit all assessment-related letters and count the number of deviations from the 

Action Matrix. Evaluate the causes for these deviations and identify changes to 

the ROP, if any, to improve the guidance documents. 

Criteria: Expect few deviations, with a stable or declining trend. 

Goals Supported: Objective, Predictable, Ensure Openness 

Analysis: There have been a total of fifteen (15) deviations from the Action Matrix since the 

beginning of the ROP in CY 2000. Three of these deviations occurred in CY 

2007. This metric meets its criteria based on the unique circumstances 

addressed by the deviations, and the approval of only one new deviation. The 

staff’s evaluation of the 2007 Action Matrix Deviations concluded the following: 

(1) On October 28, 2005, and renewed on December 11, 2006, and December 

19, 2007, the Executive Director for Operations (EDO) approved deviation 

memorandums to provide heightened NRC oversight at the Indian Point Energy 

Center. The staff intends to continue to closely monitor the licensee’s actions in 

CY 2008 to address issues associated with onsite ground-water contamination 

characterization and mitigation and with the ANS, including implementation and 

testing of the replacement ANS that Entergy is installing in response to the 

Energy Policy Act of 2005. The actions for the Indian Point Energy Center 

represent a customized approach that considers factors beyond each unit’s 

Action Matrix categorization. This approach is consistent with underlying 

concepts of IMC 0305. 

(2) On May 16, 2005, and renewed in July 2006 and August 2007, the EDO 

approved deviation memorandums to provide heightened NRC oversight at 

Davis-Besse. The staff intends to continue monitoring the licensee’s efforts to 

sustain improved plant performance following resolution of the long-standing 

underlying problems that culminated in a red finding associated with the severe 

wastage that was discovered on the reactor vessel head. As noted in last year’s 

self-assessment, the staff revised IMC 0305 to allow the regional offices to use 

additional followup actions for plants that are exiting the IMC 0350 process. The 

programmatic changes made as a result of this deviation could prevent the need 

for similar deviations in the future. 

. 

(3) The NRC issued a deviation memorandum in November 2007 to address 

security-related concerns at the Peach Bottom site. The security-related finding 

also had a documented cross-cutting aspect in the area of safety conscious work 

environment (SCWE). The NRC has taken several actions in evaluating the 

licensee’s scope of effort and progress in addressing the SCWE cross-cutting 

aspect and inattentiveness issues. The NRC actions included augmented 

inspection teams and a confirmatory action letter. These NRC actions provide 

the regulatory framework to monitor the company’s progress in addressing 
- 30 - 

security-related and SCWE issues at Peach Bottom until the next performance 

assessment. The deviation memorandum documents the company’s agreement 

to take certain actions in response to inattentiveness on the part of some security 

officers. The company’s actions include detailed briefings to security force 

personnel on acceptable behavior; round-the-clock supervisory oversight of 

security activities, and keeping the NRC informed of the status of the Peach 

Bottom transition from a contractor security force to one that is run by Exelon. 

The commitments in the letter will remain in effect until the NRC has reviewed 

Exelon’s root cause analysis of the security program issues, the company’s 

corrective actions and implementation schedule, and the company’s method for 

assessing the effectiveness of the corrective actions. As a result of these issues, 

a temporary instruction has been developed to inspect the transition of contract 

security force to proprietary security force. The staff continues to evaluate this 

deviation for impact on the ROP and will consider program improvements based 

on the lessons learned from the ongoing evaluation. 

Metric Criterion Met: Yes. 
- 31 - 

AS-2 The Number and Scope of Additional Actions Recommended as a Result of 

the Agency Action Review Meeting (AARM) Beyond Those Actions Already 

Taken Are Limited

Definition: Review the results of the Agency Action Review Meeting (AARM). 

Criteria: Few additional actions, with a stable or declining trend. 

Goals Supported: Understandable, Predictable, Objective 

Analysis: The AARM was held on April 18, 2007, in Bethesda, Maryland. The participants 

confirmed the appropriateness of agency actions for Point Beach 1 and 2, Perry, 

and Palo Verde. The participants did not recommend any additional actions 

beyond those already taken or planned. The next Agency Action Review 

Meeting is scheduled for early May 2008. 

Metric Criterion Met: Yes. 
- 32 - 

AS-3 Assessment Program Results (Assessment Reviews, Assessment Letters 

and Public Meetings) Are Completed in a Timely Manner

Definition: Track the number of instances in which timeliness goals stipulated in IMC 0305, 

“Operating Reactor Assessment Program,” were not met for: (1) the conduct of 

quarterly, mid-cycle, and end-of-cycle reviews; (2) the issuance of assessment 

letters; and (3) the conduct of public meetings. 

Criteria: Expect few instances in which timeliness goals were not met, with a stable or 

declining trend. 

Goals Supported: Ensure Effectiveness, Ensure Openness, Predictable 

0

1

2

3

Q1/2004Q2/2004Q3/2004Q4/2004Q1/2005Q2/2005Q3/2005Q4/2005Q1/2006Q2/2006Q3/2006Q4/2006Q1/2007Q2/2007Q3/2007Q4/2007

Calendar Quarter

Number of Untimely Actions

Analysis: Timeliness goals for the following activities are as follows: 

(1) quarterly reviews - within 5 weeks of the end of quarter 

(2) mid-cycle reviews - within 7 weeks of the end of the 2nd quarter 

(3) end-of-cycle reviews - within 7 weeks of the last quarter 

(4) issuance of assessment letters - within 2 weeks of the quarterly 

review, within 9 weeks of the mid-cycle review, and within 9 weeks of 

the end-of-cycle review 

(5) conduct of public meetings - within 16 weeks of the end of the 

assessment period. 

4Q/2007: All quarterly reviews and assessment follow-up letters were completed 

within timeliness goals. 
- 33 - 

3Q/2007: All mid-cycle review meetings, mid-cycle letters and quarterly 

assessment reviews were conducted within timeliness goals. Additionally, all 

public meetings were completed within timeliness goals. 

2Q/2007: All quarterly assessment reviews were completed within timeliness 

goals. One assessment follow-up letter was not completed within timeliness 

goals. All public meetings were completed within timeliness goals. 

1Q/2007: All end-of-cycle reviews, assessment letters, quarterly assessment 

reviews and assessment follow-up letters were completed within timeliness 

goals. Additionally, all public meetings were completed within timeliness goals. 

Metric Criterion Met: Yes. 
- 34 - 

AS-4 The NRC's Response to Performance Issues Is Timely

Definition: Count the number of days between issuance of an assessment letter discussing 

an issue of more than very low safety significance and completion of the 

supplemental inspection (by exit meeting date, not issuance of the inspection 

report). 

Criteria: Expect a stable or declining trend. 

Goals Supported: Ensure Safety, Ensure Effectiveness 

0

30

60

90

120

150

180

210

1Q/042Q/043Q/044Q/041Q/052Q/053Q/054Q/051Q/062Q/063Q/064Q/061Q/072Q/073Q/074Q/07

Calendar Quarter

Average Number of 

Elapsed Days 

Comments: The data represents an average timeliness for the supplemental inspections 

completed in each region in any given quarter. 

Analysis: Data collected to date indicates an increasing trend regarding the elapsed time 

between the issuance of an assessment letter and the completion of the 

corresponding supplemental inspection over previous years. The staff will 

continue to monitor the adverse trend for this metric. However, the delays in 

performing the follow-up inspections were often due to the licensee not being 

ready for the inspection. As a result, the staff is considering revising this metric 

to better account for licensee readiness, replacing it with something more 

meaningful, or deleting it altogether. 

Metric Criterion Met: No. 
- 35 - 

AS-5 NRC Takes Appropriate Actions to Address Performance Issues

Definition: Survey external and internal stakeholders asking whether the NRC takes 

appropriate actions to address performance issues for those plants outside the 

Licensee Response Column of the Action Matrix. 

Criteria: Expect stable or improved perception. 

Goals Supported: Ensure Safety, Ensure Effectiveness, Understandable 

NOTE: There was no internal survey in CY 2007 consistent with its biennial frequency as 

defined by IMC 0307. The next survey to internal stakeholders will be conducted 

during the fourth quarter of CY 2008.

Analysis: Public response agreed that the NRC takes appropriate actions to address 

performance deficiencies, safety culture problems, and cross-cutting aspects, but 

suggested that the NRC actions with enforcement actions involving civil penalties 

were not appropriate (i.e., too lenient). 

The industry and States agreed that actions taken by the NRC for plants outside 

of the licensee response column have been appropriate. There are industry 

concerns that the NRC action in accordance with the Action Matrix is consistent 

for single White findings, but appears less consistent for more complex issues. 

Industry responses also stated that there appears to be a degree of 

inconsistency between regions with the assignment of cross-cutting aspects. 

The overall level of external stakeholder satisfaction in this area was favorable 

and similar to previous years. 

Metric Criterion Met: Yes. 
- 36 - 

AS-6 Assessment Reports Are Relevant, Useful, and Written in Plain Language

Definition: Survey external and internal stakeholders asking whether the information 

contained in assessment reports is relevant, useful, and written in plain English. 

Criteria: Expect stable or improved perception of the relevance, usefulness, and 

understandability of assessment reports. 

Goals Supported: Understandable, Ensure Effectiveness, Ensure Openness 

NOTE: There was no internal survey in CY 2007 consistent with its biennial frequency as 

defined by IMC 0307. The next survey to internal stakeholders will be conducted 

during the fourth quarter of CY 2008.

Analysis: One public interest group stated that the assessment reports contain too much 

boilerplate information preventing substantive insights about performance at 

individual sites. This has been a similar comment made in previous years but 

has been refuted by the NRC staff in previous consolidated responses to 

stakeholder comments. 

The industry representatives and the State respondent agreed that the 

information contained in assessment reports is relevant, useful, and written in 

plain English. Industry responses noted concern with cross-cutting issues. 

The overall level of external stakeholder satisfaction in this area was favorable 

and similar to previous years. 

Metric Criterion Met: Yes. 
- 37 - 

AS-7 Degradations in Plant Performance Are Gradual and Allow Adequate 

Agency Engagement of the Licensees

Definition: Track the number of instances each quarter in which plants move more than one 

column to the right in the Action Matrix (as indicated on the Action Matrix 

Summary). 

Criteria: Expect few instances in which plant performance causes a plant to move more 

than one column to the right in the Action Matrix. Provide a qualitative 

explanation of each instance in which this occurs. Expect a stable or declining 

trend from the first-year benchmark. 

Goals Supported: Risk-informed, Ensure Safety, Predictable 

0

1

2

3

4

5

Q1/2004Q2/2004Q3/2004Q4/2004Q1/2005Q2/2005Q3/2005Q4/2005Q1/2006Q2/2006Q3/2006Q4/2006Q1/2007Q2/2007Q3/2007Q4/2007

Calendar Quarter

Number of Occurrences

 

Analysis: D.C. Cook Units 1 and 2, moved from the licensee response column to the 

degraded cornerstone column in 2Q/2007 due to a yellow PI in the EP 

cornerstone for the Alert and Notification System. 

Farley Unit 1, moved from the licensee response column to the degraded 

cornerstone column in 3Q/2007 due to a White PI in the Mitigating System 

Cornerstone for Cooling Water System issues and a parallel White Performance 

Indicator finding in the Mitigating System cornerstone regarding breaker failures. 

Salem Unit 1 moved from the licensee response column to the degraded 

cornerstone column in 4Q/2007 due to a yellow PI in the Mitigating System 

Cornerstone for Emergency AC Power System. 

Browns Ferry Unit 1 moved from the licensee response column to the degraded 

cornerstone column in 4Q/2007 due to a yellow PI for Unplanned Scrams in the 

Initiating Events Cornerstone. 
- 38 - 

This metric did not meet its criteria because 4 distinct sites (5 units) moved two 

or more columns to the right in the Action Matrix in CY 2007, an increasing trend 

from previous years. Also, each site moved more than one column to the right 

due to a different PI or inspection finding. The assessment period data indicates 

a short-term adverse trend as only one site had moved two or more columns in 

the Action Matrix since the fourth quarter of 2004. The staff will assess the data 

and engage with the industry to better understand the root causes to determine if 

this is actual degradation in licensee performance or something else. 

Metric Criterion Met: No.
- 39 - 

AS- 8 Perceived Effectiveness of Safety Culture Enhancements to ROP

Definition: Survey external and internal stakeholders asking whether the ROP safety culture 

enhancements help in identifying licensee safety culture weaknesses and 

focusing licensee and NRC attention appropriately. 

Criteria: Expect stable or improved perception over time. Trend average level of 

agreement. 

Goals Supported: Ensure Effectiveness, Ensure Safety 

NOTE: There was no internal survey in CY 2007 consistent with its biennial frequency as 

defined by IMC 0307. The next survey to internal stakeholders will be conducted 

during the fourth quarter of CY 2008.

Analysis: Several questions were included in the external survey related to the ROP safety 

culture enhancements. Public comments stated that the CY 2007 survey 

questions related to safety culture were not written in plain English and difficult to 

understand. The majority of the industry responses stated that it was too soon to 

judge whether the ROP inspection and assessment safety culture enhancements 

have helped to focus licensee and NRC attention on performance issues 

associated with safety culture. 

Several respondents provided input on IP 95003, “Supplemental Inspection for 

Repetitive Degraded Cornerstones, Multiple Degraded Cornerstones, Multiple 

Yellow Inputs, or One Red Input.” Industry commented that a focused lessons 

learned review should be performed on IP 95003 following its recent 

implementation at Palo Verde. The industry commented that the IP overemphasizes the potential contribution of safety culture and that it raises 

questions about industry use of safety culture surveys. 

Four industry responders stated that the cross-cutting aspects need to be reconsidered as only a few are being used and that some of the cross-cutting 

aspects are too broadly defined. Two industry responses stated the minimum 

number of inspection findings that are necessary to develop a theme and a 

substantive cross-cutting issue should be increased from the current four 

findings. Comments were also received about inspection report documentation 

of cross-cutting aspects, for example two industry responses stated that 

inspection report documentation should be updated if the assigned cross-cutting 

aspect changes. 

 Based on the relatively short period of time that the enhanced ROP has been in 

effect, a trend has not been established. A more meaningful assessment of the 

metric criterion will be achieved in follow-on years as the safety culture changes 

have been implemented for a longer period. The safety culture performance 

criterion is considered not applicable at this time. As part of the ROP safety 

culture lessons learned evaluation described in Enclosure 2 to this SECY, the 

staff is considering comments from the external survey. The NRC staff will 
- 40 - 

repeat the sub questions related to safety culture in future external surveys, 

using plain English, in an effort to clarify them. 

Metric Criterion Met: Not applicable at this time. 
- 41 - 

O-1 Stakeholders Perceive the ROP to Be Predictable and Objective

Definition: Survey external and internal stakeholders asking if ROP oversight activities are 

predictable (i.e., controlled by the process) and reasonably objective (i.e., based 

on supported facts, rather than relying on subjective judgment). 

Criteria: Expect a stable or increasing positive perception over time. 

Goals Supported: Objective, Predictable, Ensure Effectiveness 

NOTE: There was no internal survey in CY 2007 consistent with its biennial frequency as 

defined by IMC 0307. The next survey to internal stakeholders will be conducted 

during the fourth quarter of CY 2008.

Analysis: The State respondent noted that ROP oversight activities did provide consistency 

and objectivity. 

 Public responses were somewhat contrary. One public respondent stated that 

the objectivity of the ROP was one of the biggest problems and that subjectivity 

is beneficial. However, another public response stated that due to the lack of 

public documentation of the basis for decisions, they perceived the ROP 

activities and outcomes to be arbitrary. 

The responses from licensees were generally in favor of the predictability and 

objectivity aspects in the ROP. Some utilities and their representatives indicated 

that the substantive cross-cutting issues, cornerstones not closely tied to risk 

analysis, double counting, and characterizing minor issues are the most 

unpredictable and subjective areas that need improvement. It was also 

suggested that Manual Chapter 0612 Appendix E, be a “living” document (i.e., 

continually add and update examples). 

Comments received were similar to previous years. As a result of the positive 

perception, this metric has been met. 

Metric Criterion Met: Yes. 
- 42 - 

O-2 Stakeholders Perceive the ROP to Be Risk-informed 

Definition: Survey external and internal stakeholders asking if the ROP is risk-informed, in 

that actions and outcomes are appropriately graduated on the basis of increased 

significance. 

Criteria: Expect stable or increasingly positive perception over time. 

Goals Supported: Risk-Informed, Ensure Effectiveness, Ensure Openness 

NOTE: There was no internal survey in CY 2007 consistent with its biennial frequency as 

defined by IMC 0307. The next survey to internal stakeholders will be conducted 

during the fourth quarter of CY 2008.

Analysis: The State and public responses agreed that the ROP is risk-informed. However, 

one public response stated the ROP being risk-informed was a program 

weakness, mentioning it was overly risk-informed. The public respondent also 

expressed reservations over the direction of the risk informed process. 

 Industry representatives agreed that some of the cornerstones were riskinformed, but for other cornerstones, the risk basis for the thresholds were more 

subjective and less predictable (as opposed to graduated on the basis of 

increased significance). Industry responses were favorable but suggested areas 

for improvement. 

 

 Overall, respondents believe the ROP is generally risk informed. 

Metric Criterion Met: Yes. 
- 43 - 

O-3 Stakeholders Perceive the ROP to Be Understandable 

Definition: Survey external and internal stakeholders asking if the ROP is understandable 

and if the processes, procedures, and products are clear and written in plain 

English. 

Criteria: Expect stable or increasingly positive perception over time. 

Goals Supported: Understandable, Ensure Effectiveness, Ensure Openness 

NOTE: There was no internal survey in CY 2007 consistent with its biennial frequency as 

defined by IMC 0307. The next survey to internal stakeholders will be conducted 

during the fourth quarter of CY 2008.

Analysis: In general, industry, State and public respondents stated that the ROP is 

understandable and that products are written in clear and plain English. One 

response from the public disagreed, citing the MSPI as a specific example. 

Licensees expressed difficulty in following some of the complex documents 

without the appropriate technical background. Similar to previous years, the SDP 

is recognized as the most complex portion of the ROP and the NRC staff is 

continuously looking to improve this aspect of the program. 

Metric Criterion Met: Yes. 
- 44 - 

O-4 Stakeholders Perceive That the ROP Provides Adequate Regulatory 

Assurance That Plants Are Operated and Maintained Safely

Definition: Survey external and internal stakeholders asking if the ROP provides adequate 

regulatory assurance, when combined with other NRC regulatory processes, that 

plants are being operated and maintained safely. 

Criteria: Expect stable or increasingly positive perception over time. 

Goals Supported: Ensure Safety, Ensure Effectiveness, Ensure Openness 

NOTE: There was no internal survey in CY 2007 consistent with its biennial frequency as 

defined by IMC 0307. The next survey to internal stakeholders will be conducted 

during the fourth quarter of CY 2008. 

Analysis: The two public respondents did not believe the ROP (combined with other 

regulatory processes) assured safe plant operations. 

 

 Industry and State stakeholders believe the ROP, combined with other regulatory 

processes assures plants are operated and maintained safely. The State 

response was more positive than others in previous years, noting that the ROP 

provides a substantial framework for ensuring that safety remains a primary 

focus for the licensees. 

 There is an increasingly positive industry and State perception of the ROP 

maintaining safety. Public comments were similar to previous years’ surveys.

Metric Criterion Met: Yes. 
- 45 - 

O-5 Stakeholders Perceive the ROP to Be Effective, Efficient, Realistic, and 

Timely 

Definition: Survey external and internal stakeholders asking whether the ROP is effective, 

efficient, realistic, and timely. 

Criteria: Expect stable or increasingly positive perception over time. 

Goals Supported: Ensure Effectiveness, Ensure Openness 

NOTE: There was no internal survey in CY 2007 consistent with its biennial frequency as 

defined by IMC 0307. The next survey to internal stakeholders will be conducted 

during the fourth quarter of CY 2008.

Analysis: The State respondent did not comment citing lack of experience. 

 The two public responses ranged from “no” to all portions of the question, to 

perhaps effective, yes efficient, likely realistic and not timely. Responses were 

comparable to previous years. 

 In general, utility respondents believe that the ROP is effective, efficient, realistic 

and timely in comparison to previous programs. The utility stakeholder 

suggested areas for improvement such as the SDP, the appeal process, and for 

plants coming out of extended outages. Some industry responses noted that the 

NRC should consider reevaluating the frequency of certain inspections and the 

mid-cycle assessment for plants in the licensee or regulatory response column. 

 This metric met its criteria with a stable perception over time. 

Metric Criterion Met: Yes. 
- 46 - 

O-6 Stakeholders Perceive That the ROP Ensures Openness

Definition: Survey external and internal stakeholders asking if the ROP ensures openness in 

the regulatory process. 

Criteria: Expect stable or increasingly positive perception over time. 

Goals Supported: Ensure Openness, Ensure Effectiveness 

NOTE: There was no internal survey in CY 2007 consistent with its biennial frequency as 

defined by IMC 0307. The next survey to internal stakeholders will be conducted 

during the fourth quarter of CY 2008.

Analysis: External stakeholders generally acknowledged that the ROP ensures openness 

in the regulatory process, but both public and utility stakeholders expressed 

some concerns and noted that further improvements could be made. 

For example, one public response stated that there is not enough openness in 

the ROP because the bases for decisions are not made publicly available. 

Utility stakeholders reiterated that the ROP is generally a very open process, but 

pointed out the security process, the TIA process and the SDP activities as 

specific areas that should be more open and allow for greater stakeholder input. 

The State respondent indicated that the ROP promotes openness but cannot 

ensure openness because the process can be intimidating for members of the 

public. 

This metric met its criteria with a stable perception over time. 

Metric Criterion Met: Yes. 
- 47 - 

O-7 Opportunities for Public Participation in the Process

Definition: Survey external and internal stakeholders asking if there are sufficient 

opportunities for the public to participate in the process. 

Criteria: Expect positive responses or an improving trend. 

Goals Supported: Ensure Openness, Ensure Effectiveness 

NOTE: There was no internal survey in CY 2007 consistent with its biennial frequency as 

defined by IMC 0307. The next survey to internal stakeholders will be conducted 

during the fourth quarter of CY 2008.

Analysis: Public respondents do not believe there is sufficient opportunity for public 

participation. One response noted the Category 2 designation of the meetings, 

stating members of the public can only make comments during inopportune 

times. The public respondent also noted that the agenda for the monthly public 

meetings is not detailed enough. However, based on prior feedback from 

external stakeholders, the staff added significant detail to the agenda in the 

meeting notices so that potentially interested stakeholders could determine prior 

to the meeting whether topics of interest to them were planned topics of 

discussion. The State respondent did not comment citing lack of experience. 

Industry stakeholder responses were very positive. They acknowledged the 

ample opportunities for public participation such as monthly public meetings at 

NRC headquarters, annual public meetings conducted in the reactor 

communities, annual assessment meetings at each site, and annual ROP 

surveys. 

The NRC is always striving to ensure public participation and will continually 

provide sufficient and timely opportunities for members of the public to do so. 

Metric Criterion Met: Yes. 
- 48 - 

O-8 Stakeholders Perceive the NRC to Be Responsive to its Inputs and 

Comments

Definition: Survey external and internal stakeholders asking if the NRC is responsive to the 

public's inputs and comments on the ROP. 

Criteria: Expect positive responses or an improving trend. 

Goals Supported: Ensure Openness, Ensure Effectiveness 

NOTE: There was no internal survey in CY 2007 consistent with its biennial frequency as 

defined by IMC 0307. The next survey to internal stakeholders will be conducted 

during the fourth quarter of CY 2008.

Analysis: Public stakeholders feel that the NRC is responsive to input and feedback but 

sometimes with only select public interest groups. A public response stated that 

there is a perception on the part of the public that the NRC does not value their 

input. The State respondent did not comment citing lack of experience. Industry 

stakeholders believe that the NRC is responsive to inputs and comments, noting 

the published response to the 2006 ROP survey and encouraged continued 

responses for future surveys. 

Overall, stakeholder satisfaction as reported in the survey responses for the ROP 

was generally favorable and consistent. The NRC will continue to publish its 

consolidated response to the external surveys and encourage public input and 

feedback. 

Metric Criterion Met: Yes. 
- 49 - 

O-9 Stakeholders Perceive That the ROP Is Implemented as Defined 

Definition: Survey external and internal stakeholders asking if the ROP has been 

implemented as defined by program documents. 

Criteria: Expect stable or increasingly positive perception over time. 

Goals Supported: Predictable, Understandable, Ensure Openness 

NOTE: There was no internal survey in CY 2007 consistent with its biennial frequency as 

defined by IMC 0307. The next survey to internal stakeholders will be conducted 

during the fourth quarter of CY 2008.

Analysis: In general, most external stakeholders (including utilities, State, and public 

respondents) believe the ROP is being implemented as defined. 

 One public stakeholder did not agree that the NRC implemented the ROP as 

defined citing post-renewal inspections as an area for improvement. 

 Utility stakeholders only concern in this area was the potential lack of consistency 

when informal communications and decisions are made between inspectors and 

NRR staff. 

 This metric met its criteria with mostly positive comments and a stable positive 

perception over time. 

Metric Criterion Met: Yes. 
- 50 - 

O-10 Stakeholders Perceive That the ROP Does Not Result in Unintended 

Consequences 

Definition: Survey external and internal stakeholders asking if the ROP results in unintended 

consequences. 

Criteria: Expect stable or increasingly positive perception over time. 

Goals Supported: Ensure Safety, Ensure Effectiveness 

NOTE: There was no internal survey in CY 2007 consistent with its biennial frequency as 

defined by IMC 0307. The next survey to internal stakeholders will be conducted 

during the fourth quarter of CY 2008.

Analysis: Responses from Public stakeholders were contrary. One public response stated 

the ROP does not result in unintended consequences. Another public response 

stated yes, it does result in unintended consequences because licensees 

emphasize some PIs at the suffering of others. 

 The State stakeholder did not comment citing lack of experience. 

 Utility responses stated that generally the ROP does not result in unintended 

consequences but one industry respondent cited a specific example of when the 

ROP did result in unintended consequences, therefore additional guidance 

should be issued for clarification. 

 This metric met its criteria with a stable positive perception over time. 

Metric Criterion Met: Yes. 
- 51 - 

O-11 Analysis of NRC=s Responses to Significant Events 

Definition: Review reports from incident investigation teams (IITs) and augmented 

inspection teams (AITs) to collect lessons learned regarding ROP programmatic 

deficiencies (i.e., did the baseline inspection program inspect this area? Did the 

SDP accurately characterize resultant findings?). IITs already have the provision 

to determine NRC program deficiencies. AITs will be reviewed by NRR/DIRS to 

identify any weaknesses. 

Criteria: Expect no major programmatic voids. 

Goals Supported: Ensure Safety, Ensure Effectiveness 

Analysis: No IITs were conducted during the 2007 ROP cycle. One AIT was conducted in 

CY 2007. Staff review of the AIT did not identify any program weaknesses or 

voids. Two feedback forms were received for IP 93800 regarding resource 

allocation for AIT follow-up inspections. The feedback forms resulted in a 

revision to the IP. 

Metric Criterion Met: Yes. 
- 52 - 

O-12 Analysis of Inspection Hours and Resource Expenditures

Definition: Annually, collect and analyze resource data (e.g., direct inspection effort, 

preparation/documentation, plant status hours) for Baseline, Supplemental/PlantSpecific, and Safety Issues Inspections, and other ROP activities. 

Criteria: (1) Significant deviations are not expected on an annual basis. Explore 

reasons for any deviations that may be evident. 

(2) Track and trend resource usage for the baseline inspection program and 

supplemental/plant-specific inspections. Analyze causes for any 

significant departure from established trend. 

(3) Track and trend resource usage for preparation, documentation, and 

other ROP activities, and assess the effects on budgeted resources. 

 NOTE: This metric is intended primarily for tracking and trending resource usage 

for the ROP. The results are used to improve the efficiency and effectiveness of 

the ROP and to make management and budget decisions. A detailed ROP 

resource analysis is included in the annual ROP self-assessment Commission 

paper. 

Goals Supported: Ensure Effectiveness, Predictable 

Analysis: Overall staff effort for ROP related activities in Fiscal year (FY) 2007 increased 

2.3% compared with FY 2006. 

Baseline inspection hours increased in 2007 primarily due to increased direct 

inspection effort with a corresponding increase in baseline inspection preparation 

and documentation. The inspection procedures (IP) that account for the bulk of 

the increase are: IP 7111121, Component Design Bases Inspection; IP 71152, 

Identification and Resolution of Problems; and IP 71153, Follow-up of Events and 

Notices of Enforcement Discretion. The direct inspection effort for the baseline 

inspections funded by the Office of Nuclear Security and Incident Response 

(NSIR) in FY 2007 remained essentially unchanged from the FY 2006 levels. 

There was a noticeable overall decrease in plant-specific inspections in FY 2007 

compared with FY 2006. The decrease was evident in all the components of 

plant-specific inspections. However, only resource data for the period 

September 24, 2006, through September 22, 2007, is included and several 

significant inspections took place after this time period. Those expenditures will 

be captured in the FY 2008 results and the NRC staff expects a significant 

increase in resources spent. 

An increase in effort related to Generic Safety Inspections (GSIs) reflects the 

growing activity in this area. The GSIs are typically one time inspections of 

specific safety issues with significant variability in effort possible from year to 

year. 

Metric Criterion Met: Yes. 
- 53 - 

O-13 Analysis of Resident Inspector Demographics and Experience 

Definition: Annually, collect and analyze data in order to determine the relevant inspection 

experience of the resident inspector (RI) and senior resident inspector (SRI) 

population. The following four parameters will be measured and analyzed for 

both RIs and SRIs to ensure that the NRC maintains a highly qualified resident 

inspection staff: 

 (1) NRC time - the total time the individual has accumulated as an NRC 

employee. 

 (2) Total resident time - the total time the individual has accumulated as an 

RI or SRI. 

 (3) Current site time - the total time the individual has spent as an RI or SRI 

at the current site. 

 (4) Relevant non-NRC experience - the total time the individual has gained 

relevant nuclear power experience outside of the NRC. Examples of 

relevant non-NRC experience are operation, engineering, maintenance, 

or construction experience with commercial nuclear power plants, naval 

shipyards, U.S Department of Energy facilities, or the U.S. Navy nuclear 

power program. 

Criteria: None; trend only. Provide reasons for any meaningful increase or decrease in 

these resident demographic metrics. 

 NOTE: This metric is intended primarily for tracking and trending resident 

inspection experience. The results are used to make any necessary 

modifications to the RI and/or SRI programs in order to attract and retain highly 

qualified inspectors to the respective programs. A detailed resident demographic 

and staffing analysis, including additional graphs, data, and analysis for these 

resident demographic metrics, is included in the annual ROP self-assessment 

Commission paper. 

Goals Supported: Ensure Safety, Ensure Effectiveness 

Analysis: Analysis of 2007 RI Group - RI demographic data for 2007 (see Tables 1 and 3 

and Figure 1) indicate that the RI turnover rate increased resulting in a decrease 

of both total resident time and current site time. Although the turnover rate has 

increased and current site time and total resident time numbers are down, the 

RIs continue to maintain a high level of experience. 

 During 2007, 33 of 72 RIs left an RI position (46 percent). Of the 33 RIs who left, 

13 were promoted to SRI positions, 13 were either promoted or laterally 

reassigned to a region or headquarters, 3 retired, and 4 resigned from the NRC. 

Table 1 tracks the RI turnover from 2003 to 2007. 
- 54 - 

 Table 1 

 Resident Inspector Turnover 

2003 2004 2005 2006 2007 

Promoted 

to SRI 14 3 10 11 13 

Promoted / 

reassigned 12 3 9 2 13 

Retired 1 0 2 1 3 

Resigned 0 0 2 0 4 

Total 27 6 23 14 33 

Turnover 

Rate 38% 8% 32% 20% 46% 

The RI turnover rate for 2007 increased by 26 percent from 2006. Region II had 

13 RI vacancies, and Regions I and IV both had 8 RI vacancies. Even though a 

significant portion of the RIs were promoted to an SRI position (40 percent), an 

equal share of the RIs were either promoted or reassigned outside the RI 

program. 

Nationally both the total resident time and current site time are less than two 

years. The national median value (NMV) for total resident time decreased 

20 percent from 2006 to 2007. This decrease offset the gradual 20 percent 

increase in the NMV for total resident time from 2003 to 2006. 

The national data from 2003 to 2007 shows that the RIs have maintained an 

average of 10 years relevant non-NRC experience and 4 years of NRC time. 

This demonstrates that the RIs continue to maintain a high level of experience 

despite the high turnover rates in recent years. There were 18 new RIs in 2007 

and they had an average of 10 years of relevant non-NRC experience and 3 

years of NRC time. This shows that the new RIs that are filling open positions 

have a substantial amount of nuclear experience. The staff is considering 

combining the NRC time and relevant non-NRC experience data to reflect overall 

nuclear experience in the next revision to IMC 0307. 

NOTE: The RI demographics data in 2008 will reflect the addition of five new RIs 

from November to December 2007. 

The staff was directed in the staff requirements memorandum dated June 14, 

2007 to evaluate the recruitment, training, and development of the RI program to 

confirm that the human resources are adequate to meet changing needs. The 

staff collected the following information: 
- 55 - 

RI Recruitment -The regions recruit inspectors externally from universities, 

service academy career conferences, job fairs, the Nuclear Safety Professional 

Development Program, U.S. Navy and shipyards, the nuclear power industry, 

and corporate engineering firms. The regions also recruit internally by posting RI 

vacancies within the region or nationwide. Due to the high turnover rate in 2007, 

the regions have had difficulty filling RI vacancies and have implemented various 

recruitment strategies to fill the open positions. The staff will continue to evaluate 

the RI recruitment strategies in 2008. 

RI Training - In 2007, the regions qualified 23 individuals in accordance with 

Inspection Manual Chapter (IMC) 1245, “Inspector Qualification Program.” 

Currently, 41 individuals are in the IMC 1245 qualification program, and 38 are 

projected to receive their inspector qualification in 2008. Overall, the inspector 

training program in the regions is well established and continues to produce 

qualified inspectors. 

RI Development - RIs and SRIs continue to develop professionally by filling 

rotational assignments and participating in team inspections, training 

opportunities, inspector seminars, and knowledge transfer sessions. 

Analysis of 2007 SRI Group - SRI demographic data for 2007 (see Tables 2 and 

4 and Figure 2) indicate that the SRI turnover rate was high resulting in a national 

decrease in current site time from 2006 to 2007. Although there was a national 

drop in current site time, the NMV for total resident time and relevant non-NRC 

time has increased annually since 2003. 

In 2007, 17 of 66 SRIs left the program (26 percent). Of those 17, 7 were 

promoted, 7 were laterally reassigned to headquarters or a region, 1 retired, and 

2 resigned from the NRC. Table 2 tracks the SRI turnover from 2003 to 2007.

Table 2 

Senior Resident Inspector Turnover 

2003 2004 2005 2006 2007 

Promoted 7 0 5 7 7 

Reassigned 6 3 4 7 7 

Retired 1 2 1 1 1 

Resigned 0 0 0 1 2 

Total 14 5 10 16 17 

Turnover 

Rate 21% 8% 15% 24% 26% 
- 56 - 

The SRI turnover rate in 2007 (26 percent) is about the same as in 2006; 

however, it has been increasing since 2004. The high national SRI turnover rate 

directly affected the NMV for current site time. In 2007, all four regions’ current 

site time was low. This decrease offset the gradual national current site time 

increase from 2003 to 2006. Although the national current site time numbers 

have decreased, the NMV for total resident time and relevant non-NRC 

experience has been gradually increasing since 2003. 

The staff was directed in the staff requirements memorandum dated June 14, 

2007, to consider ways to enable SRIs to be promoted and still remain within the 

RI program. A task force of staff from the Office of the Executive Director for 

Operations and the Deputy Regional Administrators is currently assessing RI 

program retention issues and will provide the Deputy Executive Director for 

Regulatory Programs with recommendations and potential solutions. 

Conclusions - The staff concluded the following: 

• The combined RI and SRI turnover rate increased significantly in 2007 

creating a complex regional human resource allocation issue which directly 

affected the site staffing metric (O-14). 

• The NMVs for RI total resident time and for SRI current site time decreased 

from 2006 to 2007 as a result of the high turnover rates. 

• The overall experience levels (NRC time and relevant non-NRC time) of RIs 

and SRIs have remained high as indicated by the national data since 2003 

(see Figures 1 and 2). 

Metric Criterion Met: Yes. 
- 57 - 

Table 3 

Resident Inspectors 

2003 2004 2005 2006 2007 

NRC Time 4.13 3.42 3.36 4.04 4.25 

Total Resident Time 1.99 2.00 2.31 2.39 1.87 

Current Site Time 1.00 1.85 2.25 2.23 1.85 

Relevant non- NRC 

Experience 10.00 10.00 10.63 10.75 10.38 

Figure 1 

Resident Inspectors - Figure 1

0

2

4

6

8

10

12

Nov-03 Nov-04 Nov-05 Nov-06 Nov-07

Year

Years (median values)

NRC time Total resident time

Current site time Relevant non-NRC experience
- 58 - 

Table 4 

Senior Resident Inspectors 

2003 2004 2005 2006 2007 

NRC Time 11.00 8.80 8.84 9.28 10.11 

Total Resident Time 6.48 7.32 7.54 7.77 7.93 

Current Site Time 1.76 2.31 2.63 3.21 2.52 

Relevant non- NRC 

Experience 6.42 6.55 7.96 9.08 10.04 

Figure 2 

Senior Resident Inspectors - Figure 2

0

2

4

6

8

10

12

Nov-03 Nov-04 Nov-05 Nov-06 Nov-07

Year

Years (median values)

NRC time Total resident time

Current site time Relevant non-NRC experience
- 59 - 

O-14 Analysis of Site Staffing 

Definition: Annually, collect and analyze data in order to measure the permanent inspector 

staffing levels at each of the reactor sites for both RIs and SRIs in order to 

evaluate the agency’s ability to provide continuity of regulatory oversight. 

Criteria: The criterion is set at 90% program-wide. Any single site that falls below 90% 

will be individually evaluated. Provide reasons for any meaningful increase or 

decrease in the inspector staffing level at reactors sites. 

NOTE: Inspectors assigned to the site permanently or through a rotation with a 

minimum duration of 6 weeks shall be counted. Inspectors on 6 week or longer 

rotational assignments will be identified as such. Inspectors assigned to the site 

for less than six weeks will not be counted, but should be indicated as such. 

Additionally, the regions shall indicate sites where permanently assigned resident 

or senior resident inspectors are away from the site for an extended period of 

time (one continuous time period which is greater than 6 weeks). Only 

inspectors who have attained at least a basic inspector certification status, as 

defined by Appendix A to Inspection Manual Chapter 1245, shall be counted. 

Data will indicate number of days a qualified resident and senior resident 

inspector are permanently assigned to the site during the year divided by the 

number of days in the year. Number of days spent on training; meetings away 

from the site; participation in team inspections; leave; or other temporary duties 

(e.g. acting for branch chiefs in his/her absence) will not be counted against the 

metric unless the absence exceed 6 continuous weeks. 

Goals Supported: Ensure Safety, Ensure Effectiveness 

Analysis: The criterion for the metric is 90 percent program-wide. In 2007 the average site 

staffing for all the regions was 96 percent, with each region exceeding 

90 percent. However, nine sites were below the 90 percent mark. Of these nine 

sites, eight were between 84 and 89 percent, and one site was 74 percent. The 

site that had 74 percent site staffing had an RI that retired from the NRC in June 

of 2007. Qualified inspectors filled the vacancy temporarily for periods of less 

than 6 weeks for the duration of calendar year. In 2005 three sites did not meet 

the criterion of 90 percent and in 2006 only one site was below 90 percent. 

As a result of the high turnover rate in 2007, the regions were presented a 

significant challenge in providing continuity of regulatory oversight at the affected 

sites. Two regions were able to meet the 90 percent site staffing goal at all of the 

sites; however, this was a significant burden on the regional staff and 

management. The RI and SRI vacancies were filled by inspectors on extended 

rotations, resulting in complex regional human resource allocation issues. 

Additionally, inspection resources were provided by other regions and 

headquarters enabling the region to fill the openings for 6 weeks or longer and 

meet the metric. The other two regions experienced difficulty in staffing the 

vacant RI and SRI positions for extended periods resulting in 9 sites falling below 

the 90 percent site staffing goal. In 2007, the sites that fell below the 90 percent 
- 60 - 

mark were dealing with an RI or SRI who left the site permanently (either by 

retirement, resignation, or transfer to a regional or headquarters position). To 

support the site inspection efforts, the regions provided qualified inspectors to the 

sites where inspectors were needed for periods of less than 6 weeks. Because 

these periods were less than 6 weeks, the site was recorded as not continuously 

staffed during this timeframe. However, at no time did these sites remain without 

qualified inspectors to support the required inspection efforts. 

Metric Criterion Met: Yes. 
- 61 - 

O-15 Analysis of ROP Training and Qualifications

Definition: Annually, evaluate the implementation of IMC 1245, “Qualification Program for 

the Office of Nuclear Reactor Regulation Programs,” particularly as it pertains to 

ROP implementation. 

 

Criteria: None; trend only. Summarize and evaluate the training accomplished over the 

previous year and propose program improvements as necessary to address 

noted concerns. 

 NOTE: This metric is intended primarily for tracking and trending the 

effectiveness of the ROP training and qualifications programs. A discussion of 

training effectiveness is included in the annual ROP self-assessment 

Commission paper. 

Goals Supported: Ensure Effectiveness, Ensure Safety 

Analysis: The staff continued to improve the initial and continuing inspector training 

programs in order to produce and maintain well-qualified, competent inspectors. 

Improvement actions identified by the staff were reviewed in accordance with the 

ROP feedback process and the improvements incorporated into inspection 

standards, as appropriate. The staff updated safety culture training for 

inspectors and performed further training at the regional counterpart and security 

inspector counterpart meetings. The staff also developed and implemented 

computer-based training on the new Unplanned Scrams with Complications 

performance indicator. Additional discussion of training effectiveness is included 

in the inspection program evaluation in Enclosure 1 of the CY 2007 ROP SelfAssessment Commission paper. 

Metric Criterion Met: Yes. 
- 62 - 

O-16 Analysis of Regulatory Impact

Definition: Annually, collect and analyze licensee feedback and develop a summary of 

regulatory impact forms that are critical of the ROP. 

Criteria: None; trend only. Summarize and evaluate the feedback received and propose 

program improvements as necessary to address common concerns. 

 NOTE: This metric is intended primarily for tracking and trending regulatory 

impact. A detailed regulatory impact summary is included in the annual ROP 

self-assessment Commission paper. 

Goals Supported: Ensure Effectiveness, Ensure Safety 

Analysis: The staff receives and evaluates feedback from licensees on an annual basis as 

part of the regulatory impact process. The regulatory impact process was 

established in 1991 based on Commission direction to develop a process for 

obtaining feedback from licensees and reporting the feedback to the 

Commission. Over the past year, the staff received feedback from 68 reactor 

licensees on 139 issues, which was a decrease over 2006. The comments fell 

into two main categories, formal communication with licensees and inspector 

performance. Of the comments received, 86 percent were favorable and 14 

percent were unfavorable. A summary of the feedback received, the staff's 

evaluation, and the proposed improvement actions are provided in Enclosure 4 of 

the CY 2007 ROP Self-Assessment Commission paper. 

Metric Criterion Met: Yes.