CONSOLIDATED RESPONSE TO THE 

2005 REACTOR OVERSIGHT PROCESS EXTERNAL SURVEY

This document contains the consolidated results of the Federal Register notice (FRN) that

solicited external stakeholder comment and feedback on the Reactor Oversight Process (ROP). 

The FRN, entitled “Solicitation of Public Comments on the Implementation of the Reactor

Oversight Process,” was published on October 13, 2005 (Agencywide Documents Access and

Management System (ADAMS) Accession Number ML052860373). This notice was part of an

ongoing effort by the staff to obtain external stakeholder input regarding the efficacy of the

ROP. The comment period expired on December 1, 2005. 

In an effort to actively solicit feedback on the implementation of the ROP, the Nuclear

Regulatory Commission (NRC) staff: (1) mailed approximately 700 surveys directly to

stakeholders, (2) placed a direct link to the survey information on both the ROP Web page and

the “Documents for Comment” page of the NRC’s external Web site, and (3) issued a press

release and posted it on the NRC’s external Web site.

Following the positive feedback and success of the 2004 consolidated response, which was

developed to address the continued concerns that the NRC has been unresponsive to

stakeholder feedback, the staff has developed a response to the comments received during the

2005 survey. This includes consolidating the comments by survey question and providing a

response categorized by survey question. Again, the respondent comments for each question

are listed in chronological order as received and the responses are provided in no particular

order.

The questions used in the questionnaire were developed for the staff to gain specific feedback

regarding the ROP’s performance metrics as described in Inspection Manual Chapter (IMC)

0307, “Reactor Oversight Process Self-Assessment Program.” This allows that staff to assess

whether the metrics are meeting the required criteria. A table showing the relation of each

survey question to its specified performance metric can be found in section 2 on pages 8 and 9. 

In addition, this year’s response includes an acronym listing to aid in understanding the

comments received.

For those respondents who made general comments that were not directed to a specific

question, the comments were listed as a response to question #19 (additional information or

comments on other program areas related to the ROP). In order to provide a more concise

response to comments in this category, those comments were grouped and addressed by the

appropriate ROP program areas of performance indicators, inspection, significance

determination process, and assessment. Comments that were outside of these areas are

addressed under the category of other.

The staff attempted to represent stakeholder comments in this document exactly as they were

received; therefore, no corrections were made for spelling or grammatical errors on the FRN

responses (but corrections have been included in parenthesis for clarification purposes as

needed). The Accession numbers from ADAMS after each respondent has also been provided

for access to the official record copy of the specific FRN response.
-2-

9

7

1 4

Industry

Public

State

Anonymous

Comments were received from the following respondents (listed in chronological order as

received). 

AMEC Earth & Environmental, Inc. (Public) (ML052990251)

Senior Nuclear Industry Consultant (Public) (ML053040030)

Pannell Consulting (Public) (ML053040032)

T. Gurdziel (Public) (ML053040034 &

ML053040036 Duplicate)

Georgia Environmental Protection Division (State) (ML053040070)

Nuclear Management Company (Industry) (ML053040072)

Exelon, Kennett Square (Industry) (ML053040062)

First Selectman Connecticut (State) (ML053220250)

Alabama Emergency Management Agency (State) (ML053360410)

Greenpeace (Public) (ML053360474)

Region IV Utility Group (Industry) (ML053430120)

Nuclear Management Company, LLC (Industry) (ML053430121)

Exelon, Byron (Industry) (ML053190067)

Union of Concerned Scientists (Public) (ML053430122)

Nuclear Energy Institute (Industry) (ML053430124)

Anonymous (ML053430123)

Strategic Teaming and Resource Sharing (Industry) (ML053430125)

AmerGen & Exelon (Industry) (ML053500119)

Region 5/6 Emergency Management (State) (ML053630061)

Southern California Edison (Industry) (ML053640300)

EFMR Monitoring Group (Public) (ML060250245)

Nine of the 21 responses came from the Nuclear

Energy Institute (NEI) or utilities endorsing the NEI

response, while 4 responses came from State or

local agencies, and 7 responses came from public

interest groups or members of the public. Also, one

of the 21 responses was left anonymously as shown

in the chart. The number of responses was the

same as last year’s, however, this year saw a

decrease in industry, State, and local agency

responses, and an increase in responses from

members of the public. 

Each question number includes all comments received followed by the NRC’s response to

those comments. The Table of Contents on the following page (pages 4-6) can be used to find

the comments and responses to specific survey questions. To the extent practicable, we

reference the relevant portions of the annual staff paper to the Commission

and the annual ROP metric report to demonstrate how the staff addressed the comments. 

These documents are available on the external ROP Web page and can also be obtained in the

NRC document management system (ADAMS). The document accession numbers are

ML060590188 and ML060590135 respectively.
-3-

In some cases the staff plans to consider the specific comments and suggested improvements

in future revisions to program guidance. Accordingly, some issues will be entered into the ROP

issue tracking system as feedback forms in accordance with IMC 0801, “Reactor Oversight

Process Feedback Program,” to ensure that these issues are considered and tracked to

resolution.

This consolidated response, along with the Commission paper and the annual ROP

performance metric report, will be posted to the ROP Web page and sent to each respondent to

the survey.
-4-

Table of Contents

Section 1: List of Acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 7

Section 2: Survey Questions and Performance Metric Relationships .............. Page 8

Section 3: Survey Questions and Responses ............................... Page 10

 

1. Does the Performance Indicator Program provide useful insights to help ensure

plant safety?

Respondent Comments: ........................ Page 10

NRC Response: ............................... Page 12

 

2. Does appropriate overlap exist between the Performance Indicator Program and

the Inspection Program?

Respondent Comments: ........................ Page 13

NRC Response: ............................... Page 15

 

3. Does NEI 99-02, “Regulatory Assessment Performance Indicator Guideline”

provide clear guidance regarding Performance Indicators?

Respondent Comments: ........................ Page 16

NRC Response: ............................... Page 17

 

4. Does the Inspection Program adequately cover areas important to safety and is

it effective in identifying and ensuring the prompt correction of performance

deficiencies?

Respondent Comments: ........................ Page 17

NRC Response: .............................. Page 20

 

5. Is the information contained in inspection reports relevant, useful, and written in

plain English?

Respondent Comments: ........................ Page 21

NRC Response: .............................. Page 23

 

6. Does the Significance Determination Process yield an appropriate and consistent

regulatory response across all ROP cornerstones?

Respondent Comments: ........................ Page 24

NRC Response: .............................. Page 26

 

7. Does the NRC take appropriate actions to address performance issues for those

plants outside of the License Response Column of the Action Matrix?

Respondent Comments: ........................ Page 27

NRC Response: ............................... Page 29

 

8. Is the information contained in assessment reports relevant, useful and written in

plain English?

Respondent Comments: ........................ Page 31

NRC Response: ............................... Page 32

 
-5-

9. Are the ROP oversight activities predictable (i.e., controlled by the process) and

reasonably objective (i.e., based on supported facts, rather than relying on

subjective judgement)?

Respondent Comments: ........................ Page 32

NRC Response: ............................... Page 35

 

10. Is the ROP risk-informed, in that the NRC’s actions are graduated on the basis

of increased significance?

Respondent Comments: ........................ Page 36

NRC Response: ............................... Page 38

 

11. Is the ROP understandable and are the processes, procedures and products

clear and written in plain English?

Respondent Comments: ........................ Page 39

NRC Response: ............................... Page 40

 

12. Does the ROP provide adequate regulatory assurance when combined with

other NRC regulatory processes that plants are being operated and maintained

safely?

Respondent Comments: ........................ Page 40

NRC Response: ............................... Page 41

 

13. Is the ROP effective, efficient, realistic, and timely?

Respondent Comments: ........................ Page 42

NRC Response: ............................... Page 43

 

14. Does the ROP ensure openness in the regulatory process?

Respondent Comments: ........................ Page 44

NRC Response: ............................... Page 45

 

15. Has the public been afforded adequate opportunity to participate in the ROP and

to provide inputs and comments?

Respondent Comments: ........................ Page 46

NRC Response: ............................... Page 47

 

16. Has the NRC been responsive to public inputs and comments on the ROP?

Respondent Comments: ........................ Page 48

NRC Response: ............................... Page 49

 

17. Has the NRC implemented the ROP as defined by program documents?

Respondent Comments: ........................ Page 49

NRC Response: ............................... Page 50

 

18. Does the ROP result in unintended consequences?

Respondent Comments: ........................ Page 50

NRC Response: ............................... Page 52
-6-

 

19. Please provide any additional information related to the Reactor Oversight

Process

Performance Indicator Section

Respondent Comments: ................... Page 52

NRC Response: ......................... Page 53

Inspection Section

Respondent Comments: ................... Page 54

NRC Response: ......................... Page 56

Significance Determination Process Section

Respondent Comments: ................... Page 58

NRC Response: ......................... Page 62

Assessment Section

Respondent Comments: ................... Page 64

NRC Response: ......................... Page 67

Other Section

Respondent Comments: ................... Page 68

NRC Response: ......................... Page 71
-7-

Section 1: List of Acronyms

ACRS - Advisory Committee on Reactor Safeguards

AIT - Augmented Inspection Team

ALARA - As Low As Reasonably Achievable

CAP - Corrective Action Program

CDF - Core Damage Frequency

CFR - Code of Federal Regulations

CY - Calendar Year

DOJ - Department of Justice

ECCS - Emergency Core Cooling System

EP - Emergency Preparedness

EPRI - Electric Power Research Institute

FAQ - Frequently Asked Questions 

FRN - Federal Register Notice

FEMA - Federal Emergency Management Agency

FENOC - FirstEnergy Nuclear Operating Company

FOIA - Freedom of Information Act

FP - Fire Protection

FPSDP - Fire Protection Significance Determination 

 Process

FTE - Full-time Equivalent 

FY - Fiscal Year

HP - Health Physics

IMC - Inspection Manual Chapter

LCO - Limiting Condition for Operation

LER - Licensee Event Report 

LERF - Large Early Release Frequency

LOCA - Loss of Coolant Accident

MC - NRC Manual Chapter

MRP - Materials Reliability Program

MSPI - Mitigating System Performance Index

MW - Megawatts

NCV - Non-Cited Violation

NDE - Nondestructive Examination

NEI - Nuclear Energy Institute

NRC - Nuclear Regulatory Commission

NRR - Office of Nuclear Reactor Regulation

NSIR - Office of Nuclear Security and Incident Response

OGC - Office of the General Counsel 

OI - Office of Investigations

PD - Performance Deficiency 

PI - Performance Indicator

PIP - Performance Indicator Program

PI&R - Problem Identification and Resolution

PPSDP - Physical Protection Significance Determination 

 Process 

PRA - Probabilistic Risk Assessment

PURTA - Public Utility Realty Tax Assessment

RCS - Reactor Coolant System 

RG - Regulatory Guide

RG&E - Rochester Gas and Electric

ROP - Reactor Oversight Process 

RUG IV - Region IV Utility Group

RV - Reactor Vessel

SALP - Systematic Assessment of Licensee Performance

SCE - Southern California Edison 

SCWE - Safety-Conscious Work Environment

SDP - Significance Determination Process

SFRP - Security Finding Review Panel

SSFF - Safety System Functional Failure

SSU - Safety System Unavailability 

STARS - Strategic Teaming and Resource Sharing 

SwLONHR - Scram with Loss of Normal Heat Removal

TI - Temporary Instruction

TMIA - Three Mile Island Alert

TMI - Three Mile Island

VHP - Vessel Head Penetration
-8-

Section 2: Survey Questions and Performance Metric Relationships

2005 External Survey Question ROP 

Performance Metric

ROP Metric Description

1. Does the Performance Indicator Program provide usefulinsights to help ensure plant safety?

PI-4 PI Program Provides Insights to Help 

2. Does appropriate overlap exist between the PerformanceIndicator Program and the Inspection Program?

PI-6 Stakeholders Perceive Appropriate O

PI Program and Inspection Program

3. Does NEI 99-02, “Regulatory Assessment PerformanceIndicator Guideline” provide clear guidance regardingPerformance Indicators?

PI-7 Clarity of Performance Indicator Gui

4. Does the Inspection Program adequately cover areasimportant to safety and is it effective in identifying andensuring the prompt correction of performance deficiencies?

IP-10 Inspection Program Effectiveness an

Covering Areas Important to Safety

5. Is the information contained in inspection reports relevant,useful, and written in plain English?

IP-9 Inspection Reports Are Relevant, Us

Plain Language

6. Does the Significance Determination Process yield anappropriate and consistent regulatory response across allROP cornerstones?

SDP-5 Results of the Same Color Are Perce

Warrant the Same Level of RegulatoCornerstones

7. Does the NRC take appropriate actions to addressperformance issues for those plants outside of the LicenseResponse Column of the Action Matrix?

AS-9

NRC Takes Appropriate Actions to AIssues

8. Is the information contained in assessment reportsrelevant, useful and written in plain English?

AS-10 Assessment Reports are Relevant, U

Plain Language

9. Are the ROP oversight activities predictable (i.e.,controlled by the process) and reasonably objective (i.e.,based on supported facts, rather than relying on subjectivejudgement)?

O-1 Stakeholders Perceive the ROP to b

Objective
-9-

2005 External Survey Question ROP 

Performance Metric

ROP Metric Description

10. Is the ROP risk-informed, in that the NRC’s actions aregraduated on the basis of increased significance?

O-2 Stakeholders Perceive the ROP to b

11. Is the ROP understandable and are the processes,procedures and products clear and written in plain English?

O-3 Stakeholders Perceive the ROP to b

12. Does the ROP provide adequate regulatory assurancewhen combined with other NRC regulatory processes thatplants are being operated and maintained safely?

O-4 Stakeholders Perceive that the ROP 

Regulatory Assurance That Plants aMaintained Safely

13. Is the ROP effective, efficient, realistic, and timely? O-5 Stakeholders Perceive the ROP to b

Realistic, and Timely

14. Does the ROP ensure openness in the regulatoryprocess?

O-6 Stakeholders Perceive that the ROP 

15. Has the public been afforded adequate opportunity toparticipate in the ROP and to provide inputs and comments?

O-7 Opportunities for Public Participation 

16. Has the NRC been responsive to public inputs andcomments on the ROP?

O-8 Stakeholders Perceive the NRC to b

Inputs and Comments

17. Has the NRC implemented the ROP as defined byprogram documents?

O-9 Stakeholders Perceive that the ROP 

Defined 

18. Does the ROP result in unintended consequences? O-10 Stakeholders Perceive that the ROP 

Unintended Consequences
-10-

Section 3: Survey Questions and Responses

1. Does the Performance Indicator Program provide useful insights to help ensure plant

safety?

Respondent Comments:

Senior Nuclear Industry Consultant

The answer is No. The SALP process provided a much more meaningful assessment. It is too easy for

the facilities to achieve a "Green" score and avoid scrutiny. All of the Green on the PI summaries

provide a false sense of Safety. The bar for Green needs to be raised.

Pannell Consulting

Depends on the rigor and candor used when developing and reporting performance, people still game

the system. Process did not catch Davis Besse problem!

Greenpeace

The performance indicator program has been so manipulated by NEI that it tells the NRC and the public

very little. ACRS has repeatedly told the NRC that its thresholds for many indicators are absurd. Either

the ACRS has been ignored by a staff or the pace of change is so glacial at the NRC that little or no

substantive change can be perceived by the public. Instead the industry, in an attempt to rid itself of an

performance indicator it could not manipulate downward, has wasted NRC's time with MSPI which is so

convoluted as to be unintelliable.

But the staff already knows this. Their own critique of the ROP identifed those indicators that have never

been anything but green. If NEI would allow the staff to follow through on the problems identified in the

2004 & 2005 Secy papers perhaps NRC would actually catch declining performance before it becomes

self evident. But the staff should change the thresholds so they are maningful [meaningful] not scrap the

indicators.

Region IV Utility Group

YES

Performance Indicators have resulted in plants improving performance (industry overall). Mitigating

Systems Performance Index (MSPI) implementation is expected to improve the insights to plant safety.

RUG IV fully supports NRC and industry efforts to enhance the current indicators such as Safety System

Unavailability (SSU), Scrams with Loss of Normal Heat Removal (SwLONHR) and Reactor Coolant

System Leakage (RCS).

Two areas have the potential to cause Licensees to take actions that can adversely impact plant safety.

One is in the current Mitigating Systems Cornerstone, specifically with the Safety System Unavailability

(SSU) PIs. The Safety System Unavailability (SSU) PIs are not risk informed and are not consistent with

Maintenance Rule Program goals in most cases. Planned maintenance schedules are managed in order

to maintain plant performance in the “GREEN band”. The second area is in the Scrams with Loss of

Normal Heat Removal metric. This metric could possibly cause an operator to delay or eliminate actions

they may have taken to make a transient easier to control simply due to some perceived performance

standard being communicated by the metric.
-11-

A significant effort has been made by the industry and the NRC to aggressively address the problems

associated with the Safety System Unavailability PIs by better risk-informing them. The Mitigating

Systems Performance Index (MSPI) has been piloted and accepted as a replacement for the Safety

System Unavailability (SSU) PIs. We continue to support the implementation of the Mitigating Systems

Performance Index (MSPI) and are supportive of the scheduled implementation in April of 2006. The

Performance Indicator (PI) relative to Scrams with Loss of Normal Heat Removal has been modified by

the Reactor Oversight Process Task Group and is being piloted for implementation.

Union of Concerned Scientists

The Performance Indicators (PIs) originally did indeed provide useful insights. But their value has eroded

over time via the frequently asked question (FAQ) process and other means to the point where the PIs

are essentially all GREEN all the time. The least useless of the PIs had been the Safety System

Unavailability one, but the industry got rid of it by pressuring the Commission to force the NRC staff to

change its position on the Mitigating System Performance Index (MSPI). The MSPI is quite simply the

easiest indicator to "game." If one is about to cross the GREEN to WHITE threshold on MSPI, all one

needs to do to avoid that outcome is reschedule some tests from next quarter to this quarter and chalk

up several successful surveillance tests to boost the reliability numbers. For variety, one could "games"

MSPI by rescheduling some surveillance tests from this quarter to next quarter to boost the availability

numbers. MSPI is a ludicrous exercise that has no rightful place in a regulatory oversight package.

Nuclear Energy Institute

The objective of the ROP is to arrive at an objective assessment of licensee safety performance using

Performance Indicators and Inspection findings. (IMC 305 Operating Reactor Assessment Program) The

evaluations by NRC inspectors, the communication of results to licensees, and quarterly reported

performance indicators generally appear to be focused on the overall safety performance of operating

nuclear reactors. 

The Performance Indicator Program in particular provides useful insights to help ensure plant safety.

Since the inception of the ROP, site programs have improved as result of the performance indicators,

notably evident by the positive trends in Unplanned Power Changes, Safety System Functional Failures,

Emergency Preparedness Drill Participation, Alert and Notification System Reliability, Occupational

Radiation Safety, and Physical Protection Equipment Performance. Some indicators have shown

consistently high industry performance (i.e., high in the green band) since the initial implementation of

the Reactor Oversight Process. Areas such as these serve to maintain a balanced focus on safety

across all cornerstones, while also promoting public confidence. 

Two areas of the Performance Indicator program that have been identified as not having the proper

safety focus are the Safety System Unavailability (SSU) PIs and the Scrams with Loss of Normal Heat

Removal (SwLONHR) PI. The SSU PIs are not risk informed and are inconsistent with Maintenance Rule

Program goals in most cases. In certain cases, planned maintenance schedules would have to be

inappropriately shortened in order to maintain plant performance in the “GREEN band” for the SSU PIs.

The SwLONHR PI has the potential to lead plant operations personnel to delay or eliminate actions they

would otherwise take to make a transient easier to control simply to avoid a “hit” with this PI.

A significant industry and NRC effort to develop the Mitigating System Performance Index (MSPI) as a

replacement for the SSU PIs is close to completion. The MSPI indicators will be implemented as a

replacement for the SSU PIs beginning on April 1, 2006.
-12-

A replacement indicator for the SwLONHR PI has been developed and efforts are currently underway to

pilot this indicator and ready it for implementation. 

Strategic Teaming and Resource Sharing

The Performance Indicator (PI) Program does provide useful insights to help ensure plant safety. Over

the past six years, site programs have improved as a result of the PI Program, notably evident by the

positive trends in Unplanned Power Changes, Safety System Functional Failures, Emergency

Preparedness Drill Participation, Alert and Notification System Reliability, Occupational Radiation Safety,

and Physical Protection Equipment Performance. Some indicators have been consistently high in the

green band since the initial implementation of the Reactor Oversight Process. While these may not

demonstrate a notable improvement from an indicator perspective, they continue to promote public

confidence that barrier integrity is being maintained and radioactive releases are being tightly controlled.

STARS supports current ongoing efforts by the NRC and Industry to better risk inform the Safety System

Unavailability, Scrams with loss of normal heat removal, and RCS Leakage PIs.

NRC Response:

The responses regarding the performance indicators (PIs) from public stakeholders showed a

lack of confidence in the capability of the current PIs to contribute to the identification of

declining performance (most notably at Davis-Besse). They noted that, while the PIs may have

provided useful insights at one time, they have been undermined by the Frequently Asked

Question (FAQ) process so that they are now susceptible to manipulation or “gaming” and are

almost always all green. However, industry stakeholders stated that the PIs have provided

useful insights to help ensure plant safety which has resulted in improved performance of the

industry as a whole. Industry responders agreed, however, that the Safety System

Unavailability (SSU) and Scrams with Loss of Normal Heat Removal (SwLONHR) PIs are not

risk-informed and could cause operators to delay or eliminate actions that would make a

transient easier to control.

As discussed in Enclosure 1 to SECY-06-0074, the percentage of greater-than-green PIs

continue to decline. Although the PI program continues to provide the NRC with objective

indicators regarding plant performance, and in some areas has focused licensee attention and

contributed to improved performance, the staff and some public stakeholders remain concerned

with the capability of the current PIs to contribute to the identification of declining performance. 

As a result, the staff is in the process of revising several PIs. The staff plans to continue to

work with the industry to revise and/or introduce other PIs to improve the program’s

effectiveness in contributing to the identification of declining performance.

Comments were also received regarding the development and implementation of the Mitigating

Systems Performance Index (MSPI), which is replacing the existing SSU PIs. As discussed in

Enclosure 1 to SECY-06-0074, the staff and industry developed the MSPI to address known

problems with the SSU PIs. MSPI is a complex, risk-based index that combines component

reliability and availability with plant-specific probabilistic risk assessment (PRA) information to

arrive at a single performance index for the monitored system. Specific individual plant design

characteristics influence MSPI significantly. Since conclusion of the MSPI pilot in 2003, the

staff and industry have continued to work to finalize the technical guidance needed for

implementing MSPI, to define and address a minimum level of PRA quality needed for MSPI, to
-13-

develop the databases and software necessary for each licensee to be able to implement

MSPI, and to resolve issues identified throughout the development and review processes. 

The staff reviewed each licensee’s MSPI basis document during the fall of 2005 and both the

staff and industry concluded that a significant number of licensees were not ready to implement

MSPI by January 2006. Therefore, the ROP working group decided to delay MSPI

implementation until April 2006. Additionally, specific comments were brought up related to the

SwLONHR PI. As mentioned in Enclosure 1 of SECY-06-0074, the NRC/industry task group

reached agreement on the definition for a proposed replacement PI for monitoring complicated

scrams. Data collection to establish thresholds is underway. The staff believes that this new PI

has the potential to be a leading indicator of declining performance in that a plant that has a

history of complicated scrams may be more likely to have a risk-significant scram.

With the MSPI implementation and a draft proposal for a replacement for the SwLONHR

currently under review, there has been less focus on other staff concerns, particularly with

regard to Safety System Functional Failures (SSFFs) and the need to update several other PIs. 

The staff will continue to work with industry to review PI definitions and supporting information

to make NEI 99-02 more readable, more concise, and better organized. In addition, the staff

has taken the following actions:

< Implemented the “Discrepant or Unreported Performance Indicator Data” process once

in 2004 and again in 2005. This has contributed to the resolution of the two PI issues

involved.

< Began using the process agreed upon by the NRC and industry to resolve PI issues that

could not be resolved during the monthly ROP public meetings. After several meetings

with no agreement, the issues were presented to the responsible NRC Division Director

who made the final determination. This has reduced the backlog of frequently asked

questions.

< Met with industry to improve the Reactor Coolant System Leakage PI. The proposed

changes include counting unidentified leakage rather than identified leakage. Vendor

users groups are also involved in the same activity and are collecting data that may be

useful in establishing thresholds. These data are expected to become available to the

staff in the fall of 2006.

< Identified other PIs for potential improvement, including Unplanned Scrams, Unplanned

Power Changes, and SSFFs. There may also be a need to revise some of the

Emergency Preparedness PIs.

2. Does appropriate overlap exist between the Performance Indicator Program and the

Inspection Program?

Respondent Comments:

Senior Nuclear Industry Consultant

Green are too easy to achieve and do not reflect problems and issues identified in the Inspection

Reports.
-14-

 

Pannell Consulting

Pendulum has once again swung too far away from deterministic evaluations to where everything can be

pencil whipped away under the heading of Risk Insights as opposed to a better balance between good

engineering and operations practices and risk assessment.

Georgia Environmental Protection Division

There is some confusion between the color codes for the Performance Indicator Program (PIP) and

inspection reports ... i.e. green is GOOD for the PIP matrix, but a "green finding" in an inspection report

is one of very low safety significance. In my opinion, a "green" inspection report should be one with NO

findings ... and then mirror the PIP color scale ... white for low-to-moderate safety significance, yellow for

moderate to high safety significance, red for high safety significance.

Greenpeace

Its hard to say whether the inability of NRC to catch and reverse declining performace is due to a gap in

the Inspection and PI programs or is merely due to the large fall off in inspection hours over the past

decade.

Region IV Utility Group

YES

In most cases appropriate overlap exists between the Performance Indicator Program and the Inspection

Program. There are, however, exceptions to this. For example, NRC performs an SDP for single

equipment failures even though the Safety System Unavailability (SSU) PI (soon to be MSPI) and the

Safety System Functional Failure (SSFF) PI monitors this. Another example would be the Occupational

Radiation Safety PI, “Occupational Exposure Control Effectiveness”. This PI monitors the number of

technical specification high radiation area occurrences, very high radiation area occurrences, and

unintended exposure occurrences. Even though this PI effectively monitors these regulatory

requirements, NRC continues to document inspection findings (NCVs) related to the same issues. These

examples seem to be inconsistent with some of the underlying precepts of the ROP when

implemented—if it can be monitored adequately with a PI, then no inspection activities are required other

than PI verification.

Union of Concerned Scientists

Because the Performance Indicators are essentially as useless as an appendix on a mannequin, there

can be no appropriate overlap. The Inspection Program, and the event followup process, are left to

provide all of the useful insights into licensee performance and plant safety levels.

Nuclear Energy Institute

In most cases appropriate overlap exists between the Performance Indicator Program and the Inspection

Program. A noted exception is the performance of significance determinations for single equipment

failures even though the safety system unavailability PI and Safety System Functional Failure PI monitor

failures in key systems. This overlap will continue upon implementation of the MSPI, an indicator that

provides a clear measure of the risk significance of equipment failures. Another example would be the

Occupational Radiation Safety PI, “Occupational Exposure Control Effectiveness”. This PI monitors the

number of technical specification high radiation area occurrences, very high radiation area occurrences,

and unintended exposure occurrences. Even though this PI effectively monitors these regulatory

requirements, NRC continues to document inspection findings (NCVs) related to the same issues. These
-15-

examples undermine a defining premise of the ROP that overlap between the PI and inspection activities

should be minimized.

Strategic Teaming and Resource Sharing

In some cases, the PI and Inspection Programs have excessive overlap. For example, Occupational

Radiation Safety PI, Occupational Exposure Control Effectiveness," monitors the number of technical

specification high radiation area occurrences, very high radiation area occurrences, and unintended

exposure occurrences. Even though this PI effectively monitors these regulatory requirements, NRC

continues to document inspection findings (NCVs) related to the same issues. If it can be monitored

adequately with a PI, then the inspection activities should only need to focus on licensee's compliance

with PI data reporting requirements.

NRC Response:

Comments were received specifically relating to the overlap between the Performance Indicator

Program and the Inspection Program in the areas of single equipment failures and occupational

radiation safety. Some stakeholders believe there is little overlap between PIs and inspection

findings. Public stakeholders noted that, while a green PI represents good performance, a

green inspection finding has very low safety significance. They also commented that green

inspection findings can be achieved by analysis and therefore mask weak licensee

performance. Industry stakeholders believe there is appropriate overlap between PIs and

inspection findings in most cases, but view there is excessive overlap in some areas, such as

SSU (which has been replaced by MSPI), SSFFs, and Occupational Exposure Control

Effectiveness.

As noted in Enclosure 1 of SECY-06-0074 regarding the SSFFs, this PI counts all events or

conditions that could have prevented the fulfillment of the safety function of structures or

systems needed to shut down the reactor, remove residual heat, control the release of

radioactive material, or mitigate the consequences of an accident. A number of events have

occurred that are not explicitly covered in the guidance document, leaving room for

interpretation. However, the staff believes that the NEI 99-02 guidance is clear that the PI

includes a wide variety of events or conditions, ranging from actual failures on demand to

potential failures attributable to various causes, including environmental qualification, seismic

qualification, human error, design or installation errors, etc. Many SSFFs do not involve actual

failures of equipment. Because the contribution to risk of the structures and systems included

in the SSFF varies considerably, and because potential as well as actual failures are included, it

is not possible to assign a risk significance to this indicator. This differs somewhat from

NUREG-1022 guidance, but it is these potential failures that were reported more frequently

prior to ROP implementation but have not been reported as consistently since ROP

implementation. The staff plans to review the data in this area further and continue to discuss

this PI with the industry.

In addressing the comments regarding the occupational radiation safety overlaps, the staff

considers licensee performance in the area of radiation protection to be a significant element of

our regulatory oversight responsibility. In setting thresholds, however, it is difficult to connect

performance in these areas to measures such as core damage frequency or large early release

frequency. In establishing the PI program, the staff worked with industry to define thresholds

that would indicate that additional regulatory attention was warranted. The staff believes the PI

has been effective in focusing licensee attention on the occupational radiation safety area. As
-16-

previously discussed above, the staff is working to risk inform safety system PIs by the

implementation of MSPI. The staff meets monthly with industry representatives and is open to

discussion on how to further improve the PI program. As noted previously, the staff intends to

discuss with industry possible changes to increase the effectiveness of the program. 

As mentioned in Enclosure 1 to SECY-06-0074, the staff continues to evaluate several PIs, with

inputs from internal and external stakeholders, in an effort to improve their effectiveness at

identifying poor performance. However, the staff does not believe that there is excessive

overlap between the PIs and inspection findings because they are measuring different aspects

of performance. The PI program monitors those events and/or conditions that can provide

meaningful performance data through counts of events or conditions. The PIs, then, identify

declining performance based on the number of counts in a given area. The inspection

program, however, looks at the safety significance of each individual performance deficiency,

including licensee actions. The two programs work together to monitor the safety of the plant.

3. Does NEI 99-02, “Regulatory Assessment Performance Indicator Guideline” provide clear

guidance regarding Performance Indicators?

Respondent Comments:

Senior Nuclear Industry Consultant

It is better than the NRC's guidance but still needs improvement.

Pannell Consulting

Only as good as the user's implementation!

Greenpeace

Why not have NEI ask that question of the industry and then report to NRC?

Region IV Utility Group

YES

NEI 99-02 provides sufficient guidance on the implementation and administration of the PI process. We

particularly applaud NRC and Industry efforts to clarify ambiguities; especially as related to the new

Frequently Asked Question Process (FAQ). The revisions made to the Frequently Asked Question (FAQ)

template now being used by industry improves the communication of specific nuances and assures a

more common understanding by the NRC and the ROP Task Group. 

Currently the NRC, Industry and NEI are working to improve the Scrams with Loss of Normal Heat

Removal PI to remove subjectivity and help ensure consistency of reporting. NEI and Industry are

compiling data from reviews of historical licensee event reports (LERs) to further implementation efforts.

Union of Concerned Scientists

No opinion.

Nuclear Energy Institute

NEI 99-02 provides clear guidance regarding the performance indicators with the exception of IE02,

“Scrams with loss of normal heat removal.” Efforts are underway to address this problem area through
-17-

development and piloting of a replacement indicator. Other efforts taken to date by the industry and NRC

to clarify the guidance in NEI 99-02 have been very successful especially with the recent revision to the

Frequently Asked Question (FAQ) process. The incorporation of the dispositioned FAQs and the new

FAQ process has reduced the number of questions submitted by the industry and the NRC seeking NEI

99-02 guidance clarification.

Strategic Teaming and Resource Sharing

NEI 99-02 provides clear guidance regarding the PIs with the exception of IE02, Scrams with loss of

normal heat removal." STARS understands and continues to support the ongoing efforts by the Industry

and NRC to simplify the IE02 PI. Other efforts taken to date by the Industry and NRC to clarify the

guidance in NEI 99-02 have been very successful, especially with the recent revision to the Frequently

Asked Question (FAQ) process. Incorporation of dispositioned FAQs and the new FAQ process have

reduced the number of questions submitted by the Industry and the NRC seeking NEI 99-02 guidance

clarification.

NRC Response: 

The responses regarding the guidance for PIs from the public stakeholders either believed it

needed improvement or believed it was only as good as the user’s implementation. While, the

industry stakeholders expressed the view that the guidance is clear with the exception of the

SwLONHR PI.

The staff’s view is that the guidance document is perhaps the most critical part of the PI

program. The number of FAQs that need resolution in the staff’s monthly public meetings with

industry is a sign that the guidance document is not as clear as it could be. The resolution of

an FAQ, in some instances, depends upon the way a particular phrase or sentence is

interpreted.

To address some of the concerns described above, as noted in Enclosure 1 to SECY-06-0074, 

the staff plans to work with the industry to review each PI definition and supporting information

in the NEI guidance document. This effort is not intended to change definitions, but rather to

make the document more readable, more concise, and not subject to as much interpretation. 

This effort, however, will not totally address all of the staff concerns previously mentioned

regarding efficiency and effectiveness. To address these, the staff will continue to work with

the industry and advocate change in PI definitions and guidance as discussed previously.

Comments received regarding the SwLONHR PI are addressed in Enclosure 1 to SECY-06-

0074 and in the response to questions #1 and #2.

4. Does the Inspection Program adequately cover areas important to safety and is it effective

in identifying and ensuring the prompt correction of performance deficiencies?

Respondent Comments:

Pannell Consulting

It has become too burdensome for inspectors because of the complexity of forms and analysis. Need a

better balance between deterministic evaluations and use risk assessment perspective.
-18-

Senior Nuclear Industry Consultant

The Inspection Program has an adequate focus. However, the NRC is weak in driving the prompt

corrective actions necessary to correct performance deficiencies. Over the past 5 to 7 years, the NRC

has weakened its stance on enforcement to the point where it is like the dog with no teeth -- Bark with no

Bite.

T. Gurdziel

For example, at Davis-Besse, nobody noticed the buildup of boric acid residue on the ventilation plenum

or, if they did, nobody considered it an important sign of (unidentified) reactor coolant leakage, even

though it took 15 - 5 gallon containers to eventually remove it. (The actual hole through the plenum

metal apparently wasn't a sign, either.) After the 0350 committee got this place straightened out, a

Lessons Learned item remained: it was that the ASME requirements for accepable pressure vessel

leakage was too lenient. Today, to the best of my knowledge, this item still remains completely

unaddressed. 

Greenpeace

While it may cover the appropriate areas, it is thoroughly ineffective at ensuring prompt corrective

actions. Even after the NRC places a reactor in the regulatory response column, performance often

continues to decline into degraded cornerstones and multiple degraded cornerstones. Examples include: 

Calvert Cliffs 2001 - 2003, Cooper 2000 - 2004, Cook 2 2002 -2004, Oconee 1, 2 & 3 2000 - 2004, Point

beach [Beach] 1 & 2 2000-2004, Perry 2003-2005. The only reason Indian Point and Davis besse [DavisBesse] are not included, is that their performace [performance] could not decline any further on NRC's

scale; absent a meltdown. 

Region IV Utility Group

YES

The current inspection program can be improved by performing only one ALARA inspection per cycle

and combining the ALARA and Access Control to Radiologically Significant Areas Inspections. One

ALARA Inspection per fuel cycle would be sufficient to provide adequate oversight of this program.

Additional consideration should also be given to combining the ALARA Inspection with the Access

Control to Radiological Significant Areas Inspection during a licensee’s outage. This is where the

success of the ALARA and radiation protection programs can be measured most effectively (during

implementation). The current practice of performing Radiation Protection Team Inspections as

implemented in Region IV has been highly effective. 

If the recommended approach regarding ALARA and Access Control to Radiologically Significant Areas

Inspections is implemented, a site could expect two team inspections each cycle with any additional

inspection effort being covered by resident baseline and the action matrix as appropriate. We believe

that given the improved performance in the area of radiation protection a reduction in inspection hours is

warranted and the combination of these inspections may be appropriate and would result in a resource

savings as well. 

A second area worthy of comment is engineering inspections. The new effort to replace the “Safety

System Design and Performance Capability Inspection” by focusing on low margin systems/components

appears to be an improvement with regard to safety focus. Given the size of the team and duration of the

activity it may prove adequate to extend the frequency beyond two years for this inspection procedure.

However, it is recognized that additional inspections must be conducted to develop a better

understanding in this regard.
-19-

The NRC’s baseline inspection process and “Problem Identification and Resolution Inspection” have

been providing adequate focus on actions to correct performance deficiencies. Additionally, the NRC’s

Action Matrix provides additional oversight when warranted to ensure prompt correction of performance

deficiencies.

Union of Concerned Scientists

With the unfortunate demise of Performance Indicators as oversight tools, the Inspection Program has,

by default, become the only "eyes and ears" within the reactor oversight process. Perhaps following the

old adage that losing one sense heightens the remaining senses, the Inspection Program is better today

than when the ROP hit the streets in April 2000. Of particular note is the area of cross-cutting issues. I

attended a session at the 2005 Regulatory Information Conference during which industry representatives

protested that the NRC staff's use of cross-cutting issues was subjective and out of control. My

subsequent look into these charges led me to conclude that, while some improvements could be (and

have been) made to clarify identification of cross-cutting issues and exit from them, the NRC staff had

not made mountains out of mole-hills. Also noteworthy was NRC Region IV's inspection efforts at Palo

Verde relative to the ECCS piping issue. At the top of the "To Do" list for the Inspection Program is the

Problem Identification and Resolution (PI&R) module. This module is quite simply inadequate and needs

extensive overhaul. Its two major flaws are: (1) poor criteria for selection of sample size and (2) poor

criteria for placing findings in context. The selection criteria are almost exclusively linked to risk

significance of systems/components, with some insights from recent inspection findings. What is lacking

from the selection criteria is an explicit, formal attempt to probe the breadth of the corrective action

process. For example, the PI&R inspection, either individually or spanning a series of PI&R efforts,

should evaluate how the corrective action process is implemented by major plant organizations (e.g.,

Operations, I&C Maintenance, Electrical Maintenance, Mechanical Maintenance, Radiation Protection,

System Engineering, Civil Engineering, Fuels Engineering, etc.). 

Nuclear Energy Institute

The inspection program is effective in accomplishing its goals in covering areas important to safety and

ensuring that performance deficiencies are identified and promptly corrected. Some of the inspected

areas with demonstrated good and improving performance, such as Radiation Protection, may be over

inspected. Some thought should be given to adjusting the inspection schedule to more effectively

distribute the inspection resources.

The current inspection program can be improved by performing only one ALARA inspection per cycle

and combing the ALARA and Access to Radiologically Significant Areas inspections. One ALARA

inspection per fuel cycle would be sufficient to provide adequate oversight of this program. Additional

consideration should also be given to combining the ALARA inspection with the “Access to Radiological

Significant Areas Inspection” during a licensee’s outage. This is where the success of the ALARA and

radiation protection programs can be measured directly—implementation. The current practice of

performing Radiation Protection Team Inspections as implemented in Region IV has been highly

effective and is a step in the correct direction. If the recommended approach regarding ALARA and

Access to Radiologically Significant Areas inspection were implemented, a site could expect two team

inspections each cycle with any additional inspection effort being covered by resident baseline and the

action matrix as appropriate. 

A second area worthy of comment is engineering inspections. The new effort to replace the “Safety

System Design and Performance Inspection” by focusing on low margin systems/components appears to

be an improvement with regard to safety focus. Given the size of the team and duration of the activity it

may prove adequate to extend the frequency beyond two years for this inspection procedure. However, it
-20-

is recognized that additional inspections must be conducted to develop a better understanding in this

regard.

The NRC’s baseline inspection process and “Problem Identification and Resolution Inspection” have

been providing adequate focus on actions to correct performance deficiencies. Additionally, the NRC’s

Action Matrix provides additional oversight when warranted to ensure prompt correction of performance

deficiencies. 

Strategic Teaming and Resource Sharing

The inspection program is effective in accomplishing its goals in covering areas important to safety and

ensuring that performance deficiencies are identified and promptly corrected. Some of the inspected

areas with demonstrated good and improving performance, such as Radiation Protection, may be over

inspected. Some thought should be given to adjusting the inspection schedule to more effectively

distribute the inspection resources.

NRC Response: 

A comment was received about the NRC’s perceived weakness in driving prompt corrective

actions necessary to correct performance deficiencies. For greater than minor findings found

or reviewed during inspections that also include a violation of regulatory requirements,

inspectors are required by our inspection program (see section 06.03.d of IMC 0612) to

document these findings in accordance with the NRC Enforcement Policy (NUREG-1600) and

section 3.12 of the NRC Enforcement Manual. The majority of these types of findings are

determined to be of very low risk significance (Green) and are dispositioned as non-cited

violations (NCVs). A NCV is the term used to describe a method for dispositioning a Severity

Level IV violation or a violation associated with a finding that the Significance Determination

Process (SDP) evaluates as having very low safety significance (i.e., Green). These issues are

documented as violations in inspection reports (or inspection records for some materials

licensees) to establish public records of the violations, but are not cited in Notices of Violation

which normally require written responses from licensees. Dispositioning violations in this

manner does not eliminate the NRC’s emphasis on compliance with requirements nor the

importance of maintaining safety. Licensees are still responsible for maintaining safety and

restoring compliance to the regulations by taking steps to address corrective actions for these

violations. Specifically, the staff ensures that Green findings are placed in the licensee’s

corrective action program, and licensee corrective actions to address greater than Green

findings are reviewed to ensure that the performance deficiencies were corrected and that their

action addresses the potential for recurrence. Additionally, the NRC performs a major review of

each licensee’s corrective action program every two years. This review provides a level of

assurance that all issues, whether NRC or licensee identified, are being appropriately

addressed. If repeated problems are identified with licensee corrective actions, the NRC may

assess this to be a substantive cross-cutting issue and address this with the licensee via the

performance assessment process.

Comments were received pertaining to engineering inspections and suggested that given the

size of the team and duration of the activity, the frequency of the new “Safety System Design

and Performance Capability Inspection” might be extended beyond the current two-year

inspection frequency. As noted in Enclosure 2 to SECY-06-0074, the staff committed in the

CY 2004 ROP self-assessment to assess the results of the pilot engineering design inspections

and develop recommendations for Commission consideration in FY 2005. The staff also
-21-

developed Temporary Instruction (TI) 2515/158, “Functional Review of Low Margin/Risk

Significant Components and Human Actions,” and implemented it at one pilot site in each

region. SECY-05-0118, "Results of the Pilot Program to Improve the Effectiveness of NRC

Inspections of Engineering and Design Issues,” issued July 1, 2005, assessed the results of the

pilot inspections. As a result of the assessment, the staff changed the title of Inspection

Procedure (IP) 71111.21 from “Safety System Design and Performance Capability Inspection”

to “Component Design Bases Inspection.” The staff also revised the inspection details to be

consistent with the TI and focus on components rather than systems. The revised IP will be

conducted at all plant sites over the CY 2006 and CY 2007 biennial period. The staff plans to

develop additional guidance for engineering design inspections after CY 2007 based on lessons

learned during initial implementation of IP 71111.21.

Also, the staff received a comment that the American Society of Mechanical Engineers (ASME)

code requirement for acceptable pressure vessel leakage was too lenient and that the NRC

should take actions to address this weakness. As noted in Enclosure 2 to SECY-06-0074, the

agency has an action item to encourage ASME Code requirement changes for bare metal

inspections of nickel-based alloy nozzles for which the code does not require the removal of

insulation for inspections. Also, the NRC staff supports a change to the ASME Code

requirement to require conduct of non-visual nondestructive examination (NDE) inspections of

vessel head penetration (VHP) nozzles. If changes to the ASME Code requirements are not

possible, the NRC staff may pursue a revision to 10 CFR 50.55a to address these areas. The

staff plans to review the Electric Power Research Institute/ Materials Reliability Program (EPRI/

MRP) guidelines, which are expected to form the basis for ASME code changes, when issued. 

Issuance has been delayed several times. NRC management has communicated with industry

to encourage timely issuance of proposed guidelines. The staff also participates in ASME Code

committees. Once the ASME Code requirements are updated, the staff will evaluate them for

inclusion by reference into 10 CFR 50.55a.

The staff received a comment that the current baseline inspection program can be improved by

performing only one as low as reasonably achievable (ALARA) inspection per cycle and

combining the ALARA and Access Control to Radiologically Significant Areas Inspections. The

staff plans to review this recommendation during the 2006 calendar year. Any recommended

changes to the baseline inspection program will be implemented at the start of a calendar year.

Also, the staff received comments about the Problem Identification and Resolution (PI&R)

inspection module regarding: (1) poor criteria for selection of sample size and (2) poor criteria

for placing findings in context. The staff plans to review these comments on the PI&R

procedure during the 2006 calendar year. Any recommended changes to the baseline

inspection program will be implemented at the beginning of a calendar year.

5. Is the information contained in inspection reports relevant, useful, and written in plain

English?

Respondent Comments:

Senior Nuclear Industry Consultant

Inspection Reports are generally well written. However, the NRC should be more explicit in what they

expect the utility to do about the problems and issues identified.
-22-

Pannell Consulting

Always too "PC", ask yourself why Davis[-]Besse could happen at this point in time in the industry??

T. Gurdziel

I don't like having information on a single problem spread out over 2 or 3 sections in the report, though. 

Also, it is impossible to determine what section you are reading from.

Region IV Utility Group

YES

Information in inspection reports is generally useful. The organization of the reports and the ties to

cornerstones help to provide better definition and focus in problem areas. The listing in the reports of

inspection scope is duplicative of the Inspection Procedures and could be eliminated. 

Recent definition changes to IMC 0612 to allow more credit for licensee identified findings (NCVs) and

improvements to Appendix ‘E’ (adding additional examples of cross-cutting aspects) are applauded. We

encourage working with NEI and the Industry to develop and implement a process to apply thresholds to

cross-cutting aspects as discussed in the September 2005 ROP Meeting. 

One area of concern to licensees is the recent change to IMC 0612 and its expansion of what constitutes

a performance deficiency. The current guidance provided imposes standards beyond those specified or

committed to by a licensee in their licensing basis. This process circumvents the backfit process.

Additionally, the use of findings has greatly increased since its inception. Inspection reports should

contain regulatory based findings for the docket. Observations with insights now being developed in the

findings are very useful to the licensees; however, when documented as a “finding” in an inspection

report and on the PIM they take on the same weight as a violation (NCV). This practice seems

inappropriate and may result in inappropriate resources being applied to the issue at the expense of

potentially more significant issues.

Exelon, Byron

The formatting and outlining approach used though is very difficult to follow and non-sensical.

Union of Concerned Scientists

Overall, the inspection reports today are far more relevant and better written than prior to the ROP. One

area that has improved but not as consistently as desired is the incorporation of simplified

drawings/schematics. Sometimes, an inspection report will contain an electrical one-line drawing to

complement a narrative about an electrical distribution event. But many inspection reports contain the

narrative sans visual aids. My suggestion would be to formally consider the need for graphics when the

text write-up for a single event or finding exceeds 3/4 page in length. It may not be necessary to illustrate

each and every 3/4-plus long narrative, but it would likely be worthwhile to visit the option each time. 

Nuclear Energy Institute

Documentation required by Manual Chapter (MC) 0612 meets the needs of licensees. The recent

changes to MC 0612 did a good job clarifying and distinguishing between the terms “Licensee-Identified,”

“NRC-Identified,” and “Self-Revealing.” Other recent improvements include the addition of examples of

minor issues and cross-cutting aspects to MC 0612 Appendix E.
-23-

Information in inspection reports is generally useful. The organization of the reports and the ties to

cornerstones help to provide better definition and focus in problem areas. The listing in the reports of

inspection scope is duplicative of the Inspection Procedures and should be eliminated.

Recent definition changes to capture licensee identified findings (NCVs) and improvements to Appendix

‘E’ (adding additional examples of cross-cutting aspects) are applauded. We encourage working with NEI

and the Industry to develop and implement a process to apply thresholds to cross-cutting aspects as

discussed in the September 2005 ROP Meeting.

One area of concern to licensees is the recent change to IMC 0612 and its expansion of what constitutes

a performance deficiency. The current guidance provided imposes standards beyond those specified or

committed to by a licensee in their licensing basis. This process circumvents the backfit process.

Additionally, the use of findings has greatly increased since its inception. Inspection reports should

contain regulatory based findings for the docket. Observations with insights now being developed in the

findings are very useful to the licensees; however, when documented as a “finding” in an inspection

report they take on the same weight as a violation (NCV) and this seems inappropriate and may result in

inappropriate resources being applied to the issue at the expense of potentially more significant issues.

Strategic Teaming and Resource Sharing

Documentation required by Manual Chapter (MC) 0612 meets the needs of licensees. The recent

changes to MC 0612 were effective in clarifying and distinguishing between the terms "LicenseeIdentified," "NRC-Identified," and “Self-Revealing." Other recent improvements include the addition of

examples of minor issues and cross-cutting aspects to MC 0612 Appendix E.

NRC Response: 

Feedback was received regarding the utility of the information in the inspection reports,

specifically, the duplication of information in the inspection scope portion of the reports and the

inspection procedures. The NRC requires this information to be provided in the inspection

reports, as noted in IMC 0612, “Power Reactor Inspection Reports” because the inspection

scope provides detailed information regarding how the inspection was conducted and what was

inspected. Also, this duplication of information occurs because the information in the inspection

scope can be used to verify that the inspection was completed satisfactorily.

Also, comments were received regarding revisions to IMC 0612, in particular the recent

definition changes. The staff is continuing to develop these to potentially include credit for

licensee-identified findings, the clarification to Appendix E (adding examples of cross-cutting

aspects), and the expansion of the definition of a performance deficiency. As noted in SECY06-0074 Enclosure 4, based upon further evaluation and discussions with regional

management, the staff revised Appendix E to IMC 0612 to include examples of cross-cutting

aspects associated with sample findings. Additionally, the staff is reviewing the current

definition of a performance deficiency which was changed in September of 2005. 

As part of its ongoing efforts to improve the effectiveness of the ROP, the staff intends to

evaluate whether licensees should receive credit for certain self-assessments. However, the

staff has deferred this work until completion of the revised engineering design inspections. 

After completing the revised design/engineering inspections and assessing the results, the staff

will evaluate the proposed policy of granting licensee credit for self-assessment activities. The
-24-

staff is in the process of revising inspection program documents to reflect the recommended

changes to the inspection program from the safety culture working group.

In addition, comments were received that the practice of documenting observations in

inspection reports appears inappropriate when they are documented as a “finding.” 

Documentation of observations is controlled by IMC 0612. Observations are a factual detail

noted during an inspection. Observations are not generally documented in inspection reports

but may be documented in conjunction with and to support a finding. Additionally, observations

may be documented to support the requirements of temporary instructions and IP 71152,

“Identification and Resolution of Problems.” Observations which result from IP 71152

inspection activities are entered into the Plant Issues Matrix (PIM) to assist NRC managers in

assessing the effectiveness of the licensee’s corrective action program during their periodic

review. However, unlike NCVs where the licensees are required to take actions to correct

deficient conditions, these types of observations are more assessment in nature and do not

require actions by the licensee.

To address the comments regarding the inclusion of visual aid representation in inspection

reports, the inspection procedure working group plans to meet in CY 2006 to address these

recommendations as well as other recommended changes to the inspection report structure

and format.

6. Does the Significance Determination Process yield an appropriate and consistent

regulatory response across all ROP cornerstones?

Respondent Comments:

Senior Nuclear Industry Consultant

The answer is No. Davis-Besse is the perfect example here. The SDP showed that the Reactor Vessel

Head Degradation (RVHD) event was still Green (until consideration for continued operations was

factored into later SDP analysis).

Pannell Consulting

Great question, I don't think so! What are we really looking for and how soon can you tell?

Nuclear Management Company

The SDP process at time overestimates the risk of issues especially if they are historical design issues. 

This can result in undue burden to the licensee.

Greenpeace

Never has never will. The insights I gained from review of the Davis[-]Besse FOIA on the SDP and other

documents released to Congress make it evident that the NRC's SPD [SDP] is broken beyond repair. 

NRC staff spent much of its time attemtping [attempting] to get the SDP to conform with their view of the

signigficance of the problem at Davis[-]Besse. They contunually [continually] had to manipulate the

process to get the response they thought was appropriate. That is not how the SDP is supposed to work. 

NRC should scrap this ex post facto attmept [attempt] to down play the significance of event s [events]

and findings it does no one any good and undermines public confidence in NRC as a strong regulator.
-25-

Region IV Utility Group

NO

The Significance Determination Process (SDP) does not apply the same risk significance to issues

across the seven cornerstones. Some SDPs are still deterministic in nature – especially in the areas of

Emergency Preparedness and to a lesser degree, Occupational and Public Radiation Safety.

Deterministic thresholds have the effect of aggregating lesser items of minor risk significance to create

findings with a final significance out of proportion to the risk presented by any credible situation. 

We recognize that both the industry and the NRC have been working to better risk-inform the Emergency

Preparedness and the Radiation Safety SDPs. Additional effort is warranted in both areas. 

A generic area of concern to us is the current guidance provided by IMC 609-01 for the conduct of

Significance and Enforcement Panels (SERP). The guidance states: “Use of licensee provided

information and use of the plant specific Standardized Plant Analysis Risk (SPAR) model is at the

discretion of the SRA or other analyst and their management but these review efforts must take into

account the SDP timeliness goal.” Our specific concern is that in the interest of timeliness alone a SRA

could at his discretion discount licensee information relevant to accurate risk determination. We believe

that all relevant information available should be considered to reach the most accurate risk

determination.

Union of Concerned Scientists

No comment.

Nuclear Energy Institute

For results with elevated colors (worse than green) after Phase 2 assessment, the SDP process

frequently degenerates to a costly and resource-intensive risk analysis exercise that has little safety

value. Engineering analysis, testing, and plant modeling are conducted to understand conditions beyond

the level necessary to characterize the risk and to take appropriate corrective actions. Alternatively, the

color should be based on a combination of 1) a rule-based risk assessment that discourages taking

credit for mistakes and equipment failures, 2) the quality and effectiveness of the corrective actions, and

3) the risk management capabilities that were in place at the time of the event. EPRI has investigated

such an approach that would lead to a more safety-effective and cost-effective SDP process.

Reconciliation of the NRC PRA and the Licensee PRA is essential to producing equivalent results. These

efforts should continue until reconciliation is complete.

The Significant Determination Process (SDP) does not yield equivalent results for issues of similar

significance across all ROP cornerstones. Specifically, issues and events such as Emergency

Preparedness, Security, Radiation Protection, and Fire Protection are evaluated using processes that are

more deterministic in nature. These deterministic SDPs tend to exaggerate the actual risk. The reactor

safety SDP is the most risk informed and should be used whenever possible. "Specialized" SDPs should

be minimized. The Significance Determination Process (SDP) does not apply the same risk significance

to issues across the seven cornerstones. Some SDPs are still deterministic in nature – especially in the

areas of emergency preparedness and to a lesser degree, occupational and public radiation safety. 

Deterministic thresholds have the effect of aggregating lesser items of minor risk significance to create

findings with a final significance out of proportion to the risk presented by any credible situation.
-26-

We recognize that both the industry and the NRC have been working to better risk-inform the Emergency

Preparedness and the Radiation Safety SDPs. Additional effort is warranted in both areas.

Strategic Teaming and Resource Sharing

The Significant Determination Process (SDP) does not yield equivalent results for issues of similar

significance across all ROP cornerstones. Specifically, issues and events such as Emergency

Preparedness, Security, Radiation Protection, and Fire Protection are evaluated using processes that are

more deterministic in nature. These deterministic SDPs tend to exaggerate the actual risk. The reactor

safety SDP is the most risk informed and should be used whenever possible. "Specialized" SDPs should

be minimized. This same comment was submitted in 2004.

NRC Response: 

To respond to comments received regarding the timeliness of the final significance

determinations, the staff has continued to evaluate a new metric for SDP timeliness and has

proposed a replacement metric of average time combined with a “backstop” approach of no

more than a certain time, such as 180 days, to complete all SDP evaluations. The staff revised

IMC 0307 accordingly to test the proposed average time and backstop metric in 2006 along

with the existing timeliness metric. According to its existing timeliness goal, the NRC will

complete the final significance determinations that are of greater-than-green significance within

90 days after the issuance of the first written correspondence to the licensee describing the

finding. 

As noted in SECY-06-0074, in 2005 the staff completed other initiatives to improve timeliness. 

The staff introduced the Planning Significance and Enforcement Review Panel (SERP) aimed

at engaging decisionmakers early in the SDP evaluation. Early management involvement will

ensure a more effective process by addressing the scope, the resources, and the schedule to

complete the evaluation within the specified time period. The Planning SERP will allow the

regions to identify (1) findings of such technical complexity that existing SDP evaluation tools

are not readily adaptable to the issue, and/or that the regions do not have the expertise or

resources to perform the evaluation, and (2) findings that have potentially high safety

significance (i.e, yellow or red) that may require additional inspection and assessment efforts.

These changes were incorporated into the November 2005 revision of IMC 0609.

The staff made other enhancements to the program. One is to encourage the use of the best

available information when assessing the significance of inspection findings. Best available

information is the data assembled through the inspection process that is acceptable for use as

the basis for risk-informing a finding within the targeted time period. Other specific changes

included developing guidance on the use of a simplified versus detailed SDP Phase 3

evaluation. This eliminates the option of creating new risk tools or performing extensive

evaluations. Additionally, the new process reduces the amount of time available for the

licensees to complete their review of the preliminary SDP decisions and to schedule the

subsequent regulatory conference. Finally, the licensees’ option to provide new information

once a final SDP decision is made has been limited.

As a result of these changes, the staff anticipates improvement in SDP timeliness. However,

challenges remain. The staff is developing a process which will aid management in reaching a

decision on the significance of findings in areas not covered by SDPs or when significant

uncertainty exists.
-27-

Several comments addressed the difference in the SDP methods that were used to determine

significance across the cornerstones where some SDPs are risk-informed, while others, such

as Emergency Preparedness (EP) and Radiation Protection (RP) SDPs are deterministic. In

establishing SDPs in these areas, it is difficult to relate deficient performance in these areas to

quantitative risk measures such as core damage frequency or large early release frequency. 

The staff has worked to define SDPs in the EP and RP areas that result in the agency response

that is considered appropriate for a range of performance problems. Because of the inherent

differences between the various cornerstones, the staff does not presently envision being able

to define a single risk-informed SDP approach that would address all cornerstones.

7. Does the NRC take appropriate actions to address performance issues for those plants

outside of the License Response Column of the Action Matrix?

Respondent Comments:

Senior Nuclear Industry Consultant

NRC needs to push utility actions harder.

Pannell Consulting

No!!!

T. Gurdziel

Example: the use of OI and DOJ at Davis-Besse to prosecute criminal charges has been a multi-year

exercise in futility. At Perry, a mainly unsuccessful plant performance improvement initiative simply

results in the promise of improvement under "Phase 2", all the while running the plant at full power. 

Same thing at Hope Creek: why shut down for 6 weeks to replace a generally acknowledged cracked

reactor recirculation pump shaft when you have already been allowed to run it 50,000 hours BEYOND

the recommended inspection time (of 80,000 hours)?

Nuclear Management Company

At times the actions are overly punitive with no definitive design for exit from a higher column in the

action matrix.

Greenpeace

No, if it did then performance would not continue to decline. One of the most aggregious [egregious]

exaplmnes [examples] of this is Cooper's original greaterthangreen [greater than green] finding where

NRC determined that the incident could have resulted ina [in a] meltdown but that was ok because

licensee said that they could evecauate [evacuate] the surrounding population. 

"the NRC determined that there was no change in LERF because, during the dominant accident

sequences, effective evacuation of the close-in population could be achieved prior to the

release." (US NRC EA-04-131, March 31, 2005, p. 5.)

PATHETIC! This is an example of why the NRC is viewed as a lap dog rather than a watch dog.

If this is how NRC is going to play the game, then the definition of LERF needs to exlude [exclude] the

possibility of evacuation. At least the NRC didnt [didn’t] try this line of reasoning at Indian Point where

the claim of possible evacuation has undermined public confidence in both NRC & FEMA. 
-28-

Region IV Utility Group

YES

The NRC follows the Action Matrix and takes appropriate actions to address performance issues. 

We continue to believe an improvement that should be considered is limiting the length of time a finding

is reflected against licensee performance. A graduated approach should be considered correlating the

length of time a finding remains visible (or effective in the action matrix) to the severity of the finding

(e.g., a green finding stays for one quarter, a white finding stays for 2 quarters, etc.) rather than retaining

all findings for four quarters, this approach results in retaining the finding for a period of time

commensurate with its significance. We do not think that two whites are equivalent to a yellow given that

the numerical risk thresholds in the reactor SDP for yellow are nominally ten times that of the threshold

for white. This is another area where NRC could further risk inform the process. Of course, findings

should continue to be retained until the NRC is satisfied that the performance issue has been

satisfactorily resolved.

Union of Concerned Scientists

No Comment.

Nuclear Energy Institute

Actions taken by the NRC to address performance issues for licensees outside the Licensee Response

Column conform to the current ROP program. While the program is being followed as written,

improvements should be considered. Specifically, two white inputs into the action matrix is not

necessarily equivalent to a yellow input, especially in cornerstones that provide more opportunities for

input, such as Mitigating Systems. A more appropriate approach would be to consider 3 white inputs or

some combination more equivalent to a yellow input as the entry threshold for a degraded cornerstone. 

We continue to believe an improvement that should be considered is limiting the length of time a finding

is reflected against licensee performance. A graduated approach should be considered correlating the

length of time a finding remains visible (or effective in the action matrix) to the severity of the finding

(e.g., a green finding stays for one quarter, a white finding stays for 2 quarters, etc.) rather than retaining

all findings for four quarters, this approach results in retaining the finding for a period of time

commensurate with its significance. We do not think that two whites are equivalent to a yellow. This is

another area where NRC could further risk inform the process. Of course, findings should continue to be

retained until the NRC is satisfied that the performance issue has been satisfactorily resolved.

Strategic Teaming and Resource Sharing

Actions taken by the NRC to address performance issues for licensees outside the Licensee Response

Column conform to the current ROP program. While the program is being followed as written,

improvements should be considered. Specifically, two white inputs into the action matrix is not

necessarily equivalent to a yellow input, especially in cornerstones that provide more opportunities for

input, such as Mitigating Systems. A more appropriate approach would be to consider 3 white inputs or

some combination more equivalent to a yellow input as the entry threshold for a degraded cornerstone.
-29-

NRC Response:

As noted in last year’s response and outlined in the assessment evaluation in Attachment 1 to

SECY-04-0053 (the CY 2003 ROP self-assessment), the staff performed a detailed analysis of

the industry’s recommendation to increase the threshold for a degraded cornerstone from two

to three white PIs or inspection findings, as directed by the Commission Staff Requirement

Memorandum (SRM) dated June 10, 2003 (ML031900342). As documented in a memorandum

to the Commission, dated August 29, 2003, the staff did not support changing the existing

threshold of two white inputs to three white inputs, in part, for the following reasons:

< The staff reviewed the plants that have entered the degraded cornerstone column or

multiple/repetitive degraded cornerstone column of the Action Matrix during the 3-year

period from April 1, 2000, through March 31, 2003. That review revealed that 4 of the

11 plants that entered the degraded cornerstone column would not have entered that

column if the entry threshold had been three white inputs, rather than the current

threshold of two white inputs. As a result, those plants would have received a lessintensive IP 95001 inspection instead of an IP 95002 supplemental inspection. After

further review of the IP 95002 inspections that were performed, the staff concluded that

in these four cases, IP 95002 was the appropriate inspection for the issues at the plants,

and that the degraded cornerstone column of the Action Matrix was the appropriate

action level.

< The SDP Task Group concluded that the current threshold of two white inputs in the

same cornerstone as the criterion for a degraded cornerstone was reasonable and there

was no information to suggest that it was inappropriate.

Again as noted in last year’s response, the industry also recommended a graded approach for

removing inspection findings from consideration in the assessment program. This

recommendation involved applying a graded approach based on safety significance, such that

white findings would remain in the assessment program for two quarters, yellow findings for

three quarters, and red findings for four quarters. As discussed in the assessment evaluation in

Attachment 1 to SECY-04-0053 (the CY 2003 ROP self-assessment), the staff disagreed with

this approach because the range of actions across the Action Matrix is graded, such that

increased regulatory actions occur with the accumulation of “greater than green” assessment

inputs. One concern with the industry’s recommendation is that inspection findings would not

remain in the assessment program long enough to allow increased NRC action with degrading

performance, as envisioned during the development of the ROP. This would be inconsistent

with the PI program, in which the indicators reflect performance over the past year or more

based on specific algorithms. 

Additionally, experience since the inception of the ROP indicates that, in many cases, the

licensee’s corrective actions were not completed and were not deemed adequate within the four

quarters for consideration in the Action Matrix in accordance with the existing program. The

staff does not currently plan to change this policy or expend additional resources to further

evaluate this industry recommendation. However, the staff will continue to review the Action

Matrix annually as part of the self-assessment and the Agency Action Review Meeting (AARM),

and to assess the appropriateness of the criteria for determining a licensee’s placement in the

Action Matrix.
1 LERF is defined in Regulatory Guide 1.174, “An Approach for using Probabilistic Risk Assessment in

Risk-Informed Decisions on Plant-Specific Changes to the Licensing Basis,” in part as, “...LERF is being

used as a surrogate for the early fatality Quantitative Health Objective. It is defined as the frequency of

those accidents leading to significant, unmitigated releases from containment in a time frame prior to

effective evacuation of the close-in population such that there is a potential for early health effects.” 

Regulatory Guide 1.174 further states that this definition is consistent with accident analyses used in the

safety goal screening criteria discussed in the Commission’s regulatory analysis guidelines.

-30-

Additionally, a comment was received concerning the staff’s significance determination

evaluation of an inspection finding for the Cooper Plant. This was a finding involving the

misalignment of a service water valve which identified a question with the use of the Emergency

Plan in the SDP. The preliminary significance determination for the finding was Green based

on core damage frequency (CDF) and White based on large early release frequency (LERF1

). 

During the regulatory conference, the licensee argued that the calculated radiological release

(LERF) associated with the finding would occur after the close-in population around the plant

would have been evacuated based on the relatively slow progression of the accident scenario

and Emergency Plan implementation estimates. The staff assessed the licensee’s evacuation

estimates and found that when effective evacuation of the close-in population was considered,

the significance of the finding could be reduced from White to Green. In this case, the staff

assessed the inspection finding as Green. In developing the SDP, the staff intended for LERF

to represent containment performance separate from site Emergency Plan performance. 

However, LERF is defined as the frequency of those accident sequences leading to significant,

unmitigated releases from containment in a time frame prior to effective evacuation of the

close-in population such that there is a potential for early health effects. Therefore, by

definition, evacuation is considered in the LERF calculation. Since the licensee’s data

eliminated the LERF concern, the finding was assessed based on CDF, which considers latent

health effects (i.e., cancer). 

Also, comments were received regarding performance improvement initiatives, specifically the

Phase 2 SDP notebooks. Currently, the staff is actively engaged in implementing the SDP

Improvement Plan. As noted in Enclosure 3 to SECY-06-0074, the Phase 2 SDP notebooks

provide the guidance on risk-informing reactor inspection findings for at-power situations. Initial

versions of the documents failed to adequately capture the individual plant characteristics, and

the process was not coordinated with licensee Probabilistic Risk Assessments (PRAs). This

often resulted in under or over estimation of the change in risk associated with inspection

findings. Subsequently, the staff revised all 71 notebooks. The NRC issued the notebooks as

Revision 1 in 2003 and posted them to the NRR internal Web page for staff use. In retrospect,

it became important to further standardize all benchmarked notebooks to match the quality of

the last notebooks benchmarked. The staff completed this standardization effort and issued

the amended notebooks as Revision 2 in 2005.

Each Revision 2 notebook now includes SDP Phase 2 pre-solved tables. These tables identify

the value of each sequence when a particular component or human action is degraded for a

certain exposure time. Each spreadsheet contains approximately 40 to 50 plant-specific key

components and/or operator actions. The spreadsheet and corresponding pre-solved tables

represent the solution and answer key to these items. In addition, the staff incorporated LERF

risk aspects in both the notebooks and the associated spreadsheets. In December 2005, the

NRC posted the Revision 2 SDP inspection notebooks and the new pre-solved spreadsheets on

the Office of Nuclear Reactor Regulation (NRR) internal Web page. The notebooks are
-31-

classified as sensitive information and are therefore not available to the public. Since training

on the use of the spreadsheets has not yet been provided, users are directed to an instruction

file prior to applying the process and are advised to use the spreadsheets only as a verification

of the outcome of the existing SDP products. The staff will complete additional guidance on

use of the pre-solved spreadsheets and associated training in 2006.

8. Is the information contained in assessment reports relevant, useful and written in plain

English?

Respondent Comments:

Senior Nuclear Industry Consultant

The NRC should be more explicit in what they expect the utility to do about the problems and issues

identified.

Pannell consulting

Always too "PC"

Region IV Utility Group

YES

Significant improvement has been noted in the NRC’s efforts to address the basis and closure

process for Substantive Cross-cutting Issues identified in the assessment letters. 

Additional effort is needed in the use of cross-cutting aspects in the assessment process— especially the

threshold for identification. However, the current process guidance is much better than last year and if

implemented consistently will be a step in the correct direction.

Union of Concerned Scientists

I seldom read the assessment reports because they follow boilerplate language so closely and

completely as to preclude substantive insights about performance at individual sites. Reading one

assessment report is like reading all assessment reports because noen [not one] really says anything..

Nuclear Energy Insitute

The information contained in assessment reports is relevant, useful, and written in plain English. Some

improvement could be made in the way “Safety-Conscious Work Environment” issues are characterized

and what actions would be considered as successfully addressing a “Safety-Conscious Work

Environment” issue. 

Significant improvement has been noted in the NRC’s efforts to address the basis and closure process

for Substantive Cross-cutting Issues identified in the assessment letters.

Additional effort is needed in the use of cross-cutting aspects in the assessment process—especially the

threshold for identification. However, the current process guidance is much better than last year and if

implemented consistently will be a step in the correct direction.
-32-

Strategic Teaming and Resource Sharing

Yes, the information contained in assessment reports is relevant, useful, and written in plain English.

Some improvement could be made in the way 'Safety-Conscious Work Environment" (SCWE) issues are

characterized and what actions would be considered appropriate for successfully addressing a SCWE

issue.

NRC Response:

As noted in Enclosure 4 of SECY-06-0074, the staff has worked to improve the language and

consistency of assessment determinations of plant performance through continuous feedback

from interested stakeholders. Further enhancements are underway to incorporate safety

culture into the inspection and assessment processes. Specifically, the staff has been working

to develop an approach with involvement of internal and external stakeholders to enhance the

treatment of cross-cutting areas in the ROP and in supplemental procedures to more fully

address safety culture. The planned approach is within the ROP framework and is consistent

with ROP basic regulatory principles. The staff expects to accomplish the safety culture

enhancements to the cross-cutting areas and selected inspection procedures and manual

chapters, conduct training for regional inspectors and management, and fully implement the

enhancements by July 2006. 

Comments regarding the language used in the assessment reports have been previously

addressed. As noted in SECY-05-0070, program guidance directs that the level of detail in the

assessment letters increases as the plant performance decreases as detailed in IMC 0305,

“Operating Reactor Assessment Program.” For example, IMC 0305 requires additional detail

for those plants outside of the licensee response column of the Action Matrix and/or those

plants that have substantive cross-cutting issues. 

9. Are the ROP oversight activities predictable (i.e., controlled by the process) and

reasonably objective (i.e., based on supported facts, rather than relying on subjective

judgement)?

Respondent Comments:

Senior Nuclear Industry Consultant

The ROP is predictable in assessing every area as Green unless there has been a significant issue or

event. The bar needs to be raised.

Pannell Consulting

Hard to tell!

Nuclear Management Company

Clear information as to the reasoning for placing a licensee in a higher column of the action matrix is not

always based upon objective evidence in my view. Politics does appear to enter into the equation at

times.
2

Reverse SDP means predetermining significance of an issue based on subjective judgment then

developing the supporting arguments.

3

IMC 0612 Definition: Performance Deficiency: An issue that is the result of a licensee not meeting a

requirement or standard where the cause was reasonably within the licensee’s ability to foresee and

correct, and that should have been prevented. The licensee does not have to be committed to a

standard in order to determine whether there is a performance deficiency (PD). For example, a PD is

determined to exist if the licensee fails to adhere to a widely accepted industry standard.

-33-

Greenpeace

NRC's ROP activities are "predictable." They regulate the agency rather than the industry but are

predicable [predictable]. Its also predicable [predictable] that the agency will give the industry every

opportunity to talk itself out of a greater than green finding. 

Region IV Utility Group

YES

For the majority of the normal baseline inspections, the ROP oversight activities are predictable and

objective as reported in the end product (i.e. the inspection report). During the course of the actual

inspection activities this is not always the case. Most inspectors follow the guidance but a few still appear

to use aggregation and “reverse SDP” techniques. 2

The subjective nature of some of the SDP screening questions reduces the predictability of the ROP

oversight activities. NRC has substantially improved the inspection process guidance— especially IMC

0612, Appendix ‘E’. However, some inspectors continue to default to “the issue I found is not in the

Appendix”. This approach allows them to move into the more subjective process of minor questions. The

expectation should be that most types of issues are included within the scope of examples within IMC

0612 Appendix ‘E’ and that few exceptions would be found. 

The use of cross-cutting aspects in the inspection process has been improved by the recent changes to

the inspection process. The process now provides specific “buckets” in which to categorize issues for

consistency. However, much subjectivity remains when deciding whether or not to “flag” a specific finding

as having a cross-cutting aspect. As the process is written, essentially all findings will be flagged as

having a cross-cutting aspect. The current guidance provided would not identify a “substantive” crosscutting issue unless there is an NRC concern with the licensee’s scope of efforts or progress in

addressing the cross-cutting area performance deficiency. We believe that this is appropriate and

properly measured. 

Characterization of a performance deficiency not associated with any regulatory requirement as a finding

is of concern to us when docketed in an inspection report. This is especially true in light of the new

definition of performance deficiency that would allow NRC to document on the docket a finding resulting

from a licensee not implementing a standard that the licensee had not committed to it in its license

basis.3

 This activity circumvents the backfit process when implemented and does not represent past

NRC practice. Additionally, the documentation of findings in inspection reports that are not related to

specific regulatory requirements is not in keeping with the enforcement practices of the NRC.

Specifically, what response is required?
-34-

It is recommended that the NRC reconsider the documentation of non-regulatory based findings on the

docket of a licensee. Observations may be a better method to communicate these issues to the licensee.

Union of Concerned Scientists

No Comment.

Nuclear Energy Institute

ROP oversight activities are very predictable. These range from the type of inspection a licensee can

expect based on location in the Response Matrix to the determination of significance of inspection

findings through use of the Significance Determination Process.

ROP oversight activities are predictable and objective when a plant is in the Green Band of the Action

Matrix with no events with the exception of the application of cross-cutting aspects to findings. Recent

revisions to Manual Chapters 0305 and 0612 provide some guidance and examples of cross-cutting

aspects, but the application of cross-cutting aspects to date has been too subjective.

One recent change to MC 0612 is of concern. The verbiage associated with term “performance

deficiency” is very subjective and permits actions to be taken against a licensee for non[-]compliance

with a standard which the licensee may not be committed to. While there are no specific docketed

examples where performance deficiencies were issued for non compliance with a standard to which the

licensee was not committed, the process as described has the potential to circumvent the backfit rule.

Other areas in the ROP oversight activities that are very unpredictable are Security and Fire Protection –

Security because of the constantly changing requirements driven “secretly” and solely by a organization

within the NRC that is outside the Inspection Branch and Fire Protection because of the constant

attempts to try and hold licensees accountable to standards they are not committed to in their licensing

basis.

For the majority of the normal baseline inspections, the ROP oversight activities are predictable and

objective as reported in the end product (i.e. the inspection report). During the course of the actual

inspection activities this is not always the case. Most inspectors follow the guidance but a few still appear

to use aggregation and “reverse SDP” techniques. (Reverse SDP means predetermining significance of

an issue based on subjective judgment then developing the supporting arguments.)

The subjective nature of some of the SDP screening questions reduces the predictability of the ROP

oversight activities. NRC has substantially improved the inspection process guidance—especially IMC

0612, Appendix ‘E’. However, some inspectors continue to default to “the issue I found is not in the

Appendix”. This approach allows them to move into the more subjective process of minor questions. The

expectation it seems is that most types of issues are included within the scope of examples within IMC

0612 Appendix ‘E’ and that few exceptions would be found.

The use of cross-cutting aspects in the inspection process has been improved by the recent changes to

the inspection process. The process now provides specific “buckets” to capture issues in. However,

much subjectivity remains when deciding whether or not to “flag” a specific finding as having a crosscutting aspect or not. As the process is written, essentially all findings will be flagged as having a

crosscutting aspect. This practice imparts the evaluation of importance to the mid-cycle and end-of-cycle

performance assessments performed by NRC. The current guidance provided would not identify a

“substantive” cross-cutting issue unless there is an NRC concern with the licensee’s scope of efforts or
4

IMC 0612 Definition: Performance Deficiency: An issue that is the result of a licensee not meeting a

requirement or standard where the cause was reasonably within the licensee’s ability to foresee and

correct and that should have been prevented. The licensee does not have to be committed to a standard

in order to determine whether there is a performance deficiency (PD).

-35-

progress in addressing the cross- cutting area performance deficiency. We believe that this is

appropriate and measured.

An area of concern is the documentation of performance deficiencies not associated with regulatory

requirements as findings. The new definition of performance deficiency would allow NRC to document on

the docket a finding resulting from a licensee not implementing a standard that the licensee had not

committed to it in its license basis. 4

 This activity circumvents the backfit process when implemented and

does not represent past NRC practice. Additionally, the documentation of findings in inspection reports

that are not related to specific regulatory requirements is not in keeping with the enforcement practices

of the NRC. Specifically, what response is required?

It is recommended that the NRC reconsider the documentation of non-regulatory based findings on the

docket of a licensee. Observations may be a better method to communicate these issues to the licensee.

If one reviews the history of ROP, findings such as those being written today were not generally

documented in an inspection report and appears to be a digression to pre-ROP methods.

Strategic Teaming and Resource Sharing

ROP oversight activities are predictable and objective when a plant is in the Green Band of the Action

Matrix with no events with the exception of cross-cutting aspects applied to findings. Recent revisions to

Manual Chapters 0305 and 0612 provide some guidance and examples of cross-cutting aspects, but the

application of cross-cutting aspects to date has been overly subjective.

One recent change to MC 0612 is of concern to STARS. The verbiage associated with the term

“performance deficiency” is very subjective and permits actions to be taken against a licensee for non

compliance with a standard to which the licensee may not be committed. STARS is not aware of any

specific docketed examples where performance deficiencies were identified for non compliance with a

standard which the licensee was not be committed to. However, the process as described, appears to

circumvent the back fit rule.

Other unpredictable areas in the ROP are Security and Fire Protection; Security because of the

constantly changing requirements driven "secretly" and sometimes solely by an NRC organization

outside the Inspection Branch, and Fire Protection because of the continuing efforts to hold licensees

accountable to industry standards outside their licensing basis.

NRC Response:

To address specifically the comments regarding the fire protection significance Determination

process (FPSDP), as annotated in Enclosure 3 to SECY-06-0074, the staff significantly revised

the FPSDP in 2004 and expected improvements in the timeliness of finalizing fire protection

issues. In 2005, the staff conducted an interim evaluation and determined that timely

completion of FPSDP determinations continued to lag behind other areas. Currently, the staff

does not have enough experience implementing the revised guidance to determine whether

enhancing the FPSDP will significantly improve the timeliness of fire protection issues.
-36-

The staff believes that the FPSDP is a logical and probabilistic process, given the inherent

complexities associated with fire-related issues. The evaluation process continues to encounter

delays generally associated with initial understanding and implementation and, to some extent,

inspection practices. However, the staff believes that the revised FPSDP is a much improved

risk-informed approach.

Towards the end of 2005, licensees notified the staff that more than 30 plants will adopt

National Fire Protection Association (NFPA) Standard 805, “Performance-Based Standard for

Fire Protection for Light Water Reactor Electric Generating Plants,” as it is endorsed by Title

10, Section 50.48c, of the Code of Federal Regulations (10 CFR 50.48c). Plants that have

committed to NFPA 805 are covered by an enforcement and assessment discretion during the

transition period. In the long term, NFPA 805 has the potential to help focus licensee and NRC

attention on those issues of greatest safety significance. 

The majority of the comments received discussed greater than green findings, as well as crosscutting aspects of findings, the new performance deficiency definition and its associated

implementation, and the subjectivity of evaluations. The staff has addressed comments

regarding the balance between deterministic evaluations and risk assessment perspective in

the response to question #2 and it is further discussed in the Enclosures to SECY-06-0074. 

The response to comments regarding cross-cutting issues can be found in the response to

question #8. With respect to the new definition of performance deficiency, the staff is currently

reevaluating this definition and expects to issue revised guidance in the near future to address

this concern.

Also, the staff has previously addressed comments regarding IMC 0612, as discussed in the

response to question #5, and revised it to clarify definitions for the terms “NRC-identified,” “selfrevealing,” and “licensee-identified” findings; provide additional guidance on documenting

cross-cutting issues; improve guidance on closure of licensee event reports; clarify the

definition of a performance deficiency; and provide additional examples of cross-cutting aspects

of a finding. And comments relating to security portions of the ROP can be found in the

responses to questions #10, #13, and #14.

10. Is the ROP risk-informed, in that the NRC’s actions are graduated on the basis of

increased significance?

Respondent Comments:

Senior Nuclear Industry Consultant

Improvement in risk-informing the ROP is needed as well as increasing the knowledge base of the

utilities in this area.

Pannell Consulting

Too much gaming of the system and shade tree lawyering! System tends to promote cleverness in

responding as opposed to rigorous assessment of situation!

Nuclear Management Company

Please note that the NRC actions do appear punitive for what appear to be low significance items[.]
-37-

Greenpeace

Take a good hard look at the Cooper greater than green and ask youselves [yourselves] whether NRC

actions were appropriate? Trying [Try] telling the public at large that the NRC failed take significant

enfocement [enforcement] action because if NPPD had melted down Cooper FEMA would have

evacuated them. Right! Not even Chairman Diaz could not utter that pabulum with a straight face!

Region IV Utility Group

YES

The majority of the ROP is risk-informed due to actions taken over the past years of implementation to

further risk-inform the process. Actions that result from findings that are classified using the Reactor

Safety SDP, IMC 0609 App A, are the most risk-informed and are the ones most graduated on the basis

of an actual increased significance. Actions resulting from findings that are classified based on SDPs

that are still deterministic in nature are not as likely to be graduated consistent with actual significance.

For example, the number of occurrences does not equate readily to the "significance" of an issue. It

would seem that the "significance" of each occurrence would have to be the overriding consideration,

rather than the aggregation of a few "minor" items or the sheer number of insignificant occurrences

(Radiation Safety, Physical Security, etc.). We believe that a degraded cornerstone should result from

three, rather than two, white outcomes (inspection findings and PIs), and the period of time findings

remain in the action matrix should be graduated based on safety significance. Implementation of the

Mitigating Systems Performance Index (MSPI) will be an improvement in risk informing the ROP.

Exelon, Byron

Except where NC discretion or judgement can be utilized to determine a resolution or in instances where

inspectors utilize revised versions manual chapters not yet available to the station being inspected. 

Union of Concerned Scientists

No Comment.

Nuclear Energy Institute

Many areas of the ROP are risk-informed such that NRC actions are on the basis of increased

significance but there are some areas that are still deterministic in nature such as Emergency

Preparedness, Security, and Operator Training. The areas covered by Manual Chapter 0609, Appendix

A are the most risk informed. 

The majority of the ROP is risk-informed due to actions taken over the past years of implementation to

further risk-inform the process. Actions that result from findings that are classified using the Reactor

Safety SDP, IMC 0609 App A, are the most risk-informed and are the ones most graduated on the basis

of an actual increased significance. Actions resulting from findings that are classified based on SDPs

that are still deterministic in nature are not as likely to be graduated consistent with actual significance. 

For example, the number of occurrences does not equate readily to the "significance" of an issue. It

would seem that the "significance" of each occurrence would have to be the overriding consideration,

rather than the aggregation of a few "minor" items or the sheer number of insignificant occurrences

(Radiation Safety, Physical Security, etc.). We believe that a degraded cornerstone should result from

three, rather than two, white outcomes (inspection findings and PIs), and the period of time findings

remain in the action matrix should be graduated based on safety significance.

Implementation of MSPI will be an improvement in risk informing ROP.
-38-

Strategic Teaming and Resource Sharing

Many areas of the ROP are risk informed such that NRC actions are on the basis of increased

significance but there are some areas that are still deterministic in nature such as Emergency

Preparedness, Security, and Operator Training. The areas covered by Manual Chapter 0609, Appendix

A are the most risk informed.

Efforts are currently ongoing to better risk inform the “Safety System Unavailability” and “Scrams with

loss of normal heat removal” PIs. STARS supports the implementation of the “Mitigating Systems

Performance Index” and the “Scrams with Complications” PIs as replacements for these indicators.

NRC Response:

Comments concerning the deterministic nature of EP and operator re-qualification SDPs are

addressed in the response to question #6. The staff considers licensee performance in the

areas of EP and operator re-qualification to be significant elements of our regulatory oversight

responsibility, and that increased regulatory oversight is needed when weaker performance is

indicated. In establishing SDPs, correlating performance in these areas with changes in core

damage frequency was not practical. Therefore, the staff used an expert panel and

subsequent lessons learned through implementation, to define the NRC’s response level in the

EP and operator re-qualification areas.

The security cornerstone of the ROP is another area where performance assessment does not

correlate with core damage frequency. Although it is difficult to connect performance in security

to measures such as core damage frequency or large early release frequency, the security

cornerstone of the ROP (formerly called the physical protection cornerstone) has improved its

risk-informed aspects with the issuance of a new significance determination process in

July 2005; the physical protection significance determination process (PPSDP). The PPSDP no

longer uses the number of similar findings as a measure of significance. It uses instead the

depth and scope of the potential effect of deficiencies and a categorization of program

elements as its measures of significance. Although specifics can not be revealed here (the

PPSDP is official use only), its structure increases significance of individual findings based on

the impact on the protective function across key attributes and program elements, and the

deficiencies in the security program that resulted in the finding. The force-on-force portion of

the PPSDP rates significance based on performance failures during drills, increasing

significance with increasing failures in drill performance.

Many of the comments received for this section had been previously addressed under other

question responses. Specifically, the comments related to MSPI and SWLONHR are

addressed in the response to question #1 or in Enclosure 1 to SECY-06-007, and the comment

on Cooper’s LERF is responded to in question #7. 
-39-

11. Is the ROP understandable and are the processes, procedures and products clear and

written in plain English?

Respondent Comments:

Senior Nuclear Industry Consultant

In general the ROP is adequate. However, its implementation needs much improvement and the bar

needs to be raised. What good are PIs if the threshold for good performance is too low?

Pannell Consulting

No, too complex!

Nuclear Management Company

There is no clear exit strategy set down by the NRC when a plant is place in Column 2, 3 or 4.

Region IV Utility Group

YES

In general the ROP is understandable and the processes, procedures, and products are clear and written

in plain English. Good examples are the recent improvements to IMC 0612 and IMC 0305 relative to

cross-cutting issues and their use in the assessment process. 

Some of the newer SDPs do require a technical background to understand. The Fire Protection and

Steam Generator SDPs are particularly difficult to follow.

Union of Concerned Scientists

The ROP may be understandable and the processes, procedures, and products may very well be written

in plain English - I just don't know. The ROP information on the NRC website is so maddeningly difficult

to find that I seldom - very very seldom - am able to find the document that answers my questions. In the

past year, several questions were raised as I looked at ROP information on the NRC website. For

example, I'd want to know why the "x-to-y" threshold was established at "n" or want to know the precise

definition for a PI. I suspected that the answers to these questions were hidden somewhere on the

NRC's website, but I almost always gave up looking at half-dozen or more futile attempts to find the

answers. The organization of ROP materials on the NRC website is illogical, poorly cross-linked, and

basically unsuable [unusable].The presentation of ROP materials on the NRC website should be

completely overhauled. It is currently unusable.

Nuclear Energy Institute

In general the ROP is understandable and the processes, procedures, and products are clear and written

in plain English. Good examples are the recent improvements to IMC 0612 and IMC 0305 relative to

cross-cutting issues and their use in the assessment process.

Some of the newer SDPs do require a technical background to understand. The Fire Protection and

Steam Generator SDPs are particularly difficult to follow.

Strategic Teaming and Resource Sharing

Generally the ROP is understandable and the process, procedures, and products are clear and written in

plain English. STARS recognizes the efforts made by the NRC to provide necessary clarifications such
-40-

as the definitions of “Licensee-Identified,” “NRC-Identified” and “Self-Revealing” in MC 0612 and “crosscutting aspects” and “substantive cross-cutting issue” in MC 0305. STARS believes these definitions will

improve consistency across the Regions.

NRC Response:

As noted in SECY-06-0074, the staff will consider any specific recommendations to further

improve the presentation and organization of the ROP-related information on the Web. The

staff is currently modifying the ROP Web pages to make them more user friendly and intuitive. 

In order to facilitate the feedback process and potentially implement improvements to the ROP

portion of the NRC’s Website, the staff is planning to contact those stakeholders that have

expressed concerns with the usability of the ROP Website to gather specific ideas regarding

organization of ROP information on the Website. 

The staff recognizes that the ROP is complex, as are the design and operation of the nuclear

reactors that the NRC regulates. The staff strives to make the ROP as understandable as

possible as it balances this goal with the other ROP goals of being objective, risk-informed, and

predictable. The Office of Public Affairs published a plain language description of the ROP in

NUREG-1649, “Reactor Oversight Process.” The ROP portion of the NRC’s website is also

designed to facilitate the understandability of and access to ROP-related information; these

Web pages are being modified to make them more user friendly and intuitive as noted above.

As described in IMC 0305, the NRC determines its regulatory response in accordance with an

Action Matrix that provides for a range of actions commensurate with the significance of the PI

and inspection results. For a plant that has all of its PIs and inspection findings characterized

as green, the NRC will only implement its baseline inspection program. For plants that do not

have all green PIs and inspection findings, the NRC will perform additional inspections and

initiate other actions, including supplemental inspection and pertinent regulatory actions ranging

from management meetings up to and including orders for plant shutdown, commensurate with

the safety significance of the issues. A safety-significant inspection finding is carried forward

for four calendar quarters or until appropriate licensee corrective actions have been completed,

whichever is greater. Since performance is assessed every quarter based on the significance

of PIs and inspection findings, no plant-specific exit strategy is necessary or warranted. 

As previously discussed, the staff is enhancing the guidance on cross-cutting issues as part of

revisions being made to the ROP to more fully address safety culture, as noted in the response

to question #8. Also, comments regarding the PI thresholds are addressed in the response to

question #2.

12. Does the ROP provide adequate regulatory assurance when combined with other NRC

regulatory processes that plants are being operated and maintained safely?

Respondent Comments:

Senior Nuclear Industry Consultant

All of the Green on the PIs give a false sense of Safety.
-41-

Pannell Consulting

Not necessarily!

Greenpeace

Absolutely not. Davis Besse was all green and then shutdown of an extended period of time due to

significant problems that were not identified by the NRC nor by the licnesee [licensee]. Reactors continue

to appear on NRC's radar only after significant degradations in safety which result in them being placed

directly into the degraded cornerstone or multiple degraded cornerstone response column. If the ROP

provided adeqaute [adequate] regualtory [regulatory] assurance it would identify reactors before they got

to that point. 

Region IV Utility Group

YES

The ROP provides adequate assurance that plants are being operated and maintained safely

as indicated by the continuously improving industry trends.

Union of Concerned Scientists

I would honestly like to answer this question with a "1" or "2" but I can point to no evidence to support

such an answer.

Nuclear Energy Institute

The ROP does provide adequate regulatory assurance that plants are being operated and maintained

safely. 

Strategic Teaming and Resource Sharing

The ROP does provide adequate regulatory assurance that plants are being operated and maintained

safely.

NRC Response:

As discussed in the response to question #1, the staff plans to work with external stakeholders

to improve the PI program’s effectiveness in contributing to the identification of declining

performance.

The staff agrees that a goal of the ROP is to detect declining performance, however, we

disagree that reactors “appear on the NRC’s radar screen” only after significant degradations in

safety which result in them being placed directly into the degraded cornerstone or multiple

degraded cornerstone response column. A recent example of this is the Perry plant, which

entered the multiple/repetitive degraded cornerstone column of the Action Matrix due to an

accumulation of white findings. This has resulted in a substantial increase in NRC oversight of

the Perry plant, although Perry has not experienced a recent, relatively high, safety-significant

event. There are several other instances of plants progressing across the Action Matrix based

on multiple white findings and/or PIs without significant degradations in safety.

In addition, as noted in last year’s consolidated response (ML052090158) under question #16,

the staff disagrees with the assertion that the ROP is wholly reactive. The staff considers some

of the PIs to be potential indicators of future performance. Substantial cross-cutting issues are

also an indicator of possible future plant performance problems. One reason that each plant
-42-

receives the baseline level of inspection is to look for indications of declining performance prior

to poor performance becoming self-revealing. Given the generally good performance of most

licensees, the staff considers the present allocation of inspection resources to be appropriate. 

Nonetheless we remain open to specific suggestions for objective indicators of future licensee

performance.

The ROP was not designed to predict and/or prevent all failures, but was more realistically

designed to detect declining performance and focus NRC and licensee attention and resources

on the most significant issues. The staff believes that the ROP has been successful in assuring

that plants are being operated and maintained safely. As previously discussed under question

#4, the staff agrees that there are lessons to be learned from the Davis-Besse event. There

have been a number of program changes made to the ROP as a result of the Davis-Besse

Lessons-Learned Task Force (DBLLTF). A summary of agency actions and their status is

located on the agency website at http://www.nrc.gov/reactors/operating/ops-experience/

vessel-head-degradation/lessons-learned/lessons-learned-files/lltf-web-page-status022205.pdf. 

However, all of the recommendations associated with the ROP have been acted on, with three

items still requiring effectiveness reviews. These reviews are scheduled for CY 2006, once the

changes have been implemented for a sufficient amount of time to evaluate their effectiveness. 

13. Is the ROP effective, efficient, realistic, and timely?

Respondent Comments:

Senior Nuclear Industry Consultant

The ROP need more forward looking performance indicators to prevent issues and problems from

becoming more significant.

Pannell Consulting

No! Cumbersome for inspectors, promotes cleverness on the part of the utility in answering versus fixing

plant problems. The pendulum has swung too far!!!

Nuclear Management Company

Again, exit strategy is not clear when a plant is placed in column 2, 3, or 4 of the action matrix.

Greenpeace

See all comment above.

Region IV Utility Group

YES

The ROP improves the efficiency, effectiveness, and realism of the regulatory process over the old SALP

process. However, in some cases the efficiency and effectiveness are decreasing in the existing ROP

process. The scope and resources needed for the baseline Radiation Protection Cornerstone inspections

seem excessive relative to overall industry performance (See item #4 above). The NRC should consider

reevaluating the frequency of these inspections. The development of many SDPs are complicating the

ROP process and causing significant training issues for the NRC inspection staff as well as licensees. 
-43-

The Physical Security process is vague, not well communicated and is developed with little stakeholder

input. Security inspections continue to judge compliance subject to interpretations by individual

inspectors. This area needs improvements in transparency and communication with stakeholders.

Additionally, security needs to get back to using the regulatory process and get out of the “order” mode.

Union of Concerned Scientists

SDP timeliness for greater than green inspection findings is, and has always been, unacceptably slow. 

Nuclear Energy Institute

Since the Significance Determination Process has provided a more realistic approach to the evaluation of

safety significant issues, the effectiveness of the regulatory process has been improved.

The ROP is an improvement over the old SALP process. Inspections are more focused, findings are

evaluated using a more structured tool, and performance assessment is more objective. Efficiency could

be gained by combining related inspection activity and evaluating the need for some inspection activity

that may be excessive such as in the area of occupational radiation protection which has been a very

good performing area. Timeliness of the SDP process has improved. The ROP improves the efficiency,

effectiveness, and realism of the regulatory process over the old SALP process. However, in some cases

the efficiency and effectiveness are decreasing in the existing ROP process. The scope and resources

needed for the baseline Radiation Protection Cornerstone inspections seem excessive relative to overall

industry performance (See item #4 above). The NRC should consider reevaluating the frequency of

these inspections. The development of many SDPs are complicating the ROP process and causing

significant training issues for the NRC inspection staff as well as licensees.

The Physical Security process is vague, not well communicated and is developed with little stakeholder

input. Security inspections continue to judge compliance subject to interpretations by individual

inspectors. This area needs improvements in transparency and communication with stakeholders.

Additionally, security needs to get back to using the regulatory process and get out of the “order” mode. 

Strategic Teaming and Resource Sharing

The ROP is an improvement over the old SALP process. Inspections are more focused, findings are

evaluated using a more structured tool, and performance assessment is more objective. Efficiency could

be gained by combining related inspection activities. Evaluating a reduction in some inspection areas

such as Occupational Radiation Safety should be considered due to sustained safety performance.

Timeliness of the SDP process has improved.

NRC Response: 

There were numerous comments regarding lack of efficiency of the SDP process. The staff

agrees that the SDP process can be more timely and more efficient. However, the NRC has no

control over the level of resources that licensees invest in the process. Program guidance

encourages staff to communicate and share information with the licensee as the process

progresses. The program requirement is that the staff’s preliminary and final assessment be

formally shared in writing with the licensee offering the licensee opportunities to respond and

appeal the outcome. It is then the NRC staff’s responsibility to consider the best available

information, and to reach a final significance determination. However, the decision-making

aspect of the SDP is not a shared responsibility with the licensee. The staff considers the

process appropriately open to the licensee.
-44-

Another common concern among most respondents was the efficiency of the SDP in regards to

timeliness, the staff agrees and, as previously discussed in response to question #6 and in the

SECY-06-0074 paper, is working towards this end. 

With respect to the comment that the ROP is cumbersome for inspectors, the staff assumes

that the respondent is referring to the complexity of the SDP, because the burden in

documenting and tracking inspection issues has been reduced significantly. With regard to the

SDP, as noted under question #7, the staff has recently issued Revision 2 of the risk-informed

inspection notebooks and the accompanying pre-solved tables, which were specifically

designed to simplify the SDP Phase 2 process and reduce burden on inspectors.

Specifically to address the security comments, the NRC (since 2002) has been using a multioffice, management-level panel to review security inspection findings to ensure that the findings

are valid and that interpretations of security requirements are consistent across the regions. 

Between 2002 and early 2005, the panel reviewed only inspection findings related to the

requirements imposed by order. Starting in April 2005, the security finding review panel (SFRP)

reviews all security inspection findings for validity and consistency. When appropriate, the NRC

has engaged the nuclear power industry, through NEI, to gain stakeholder input on changes to

the NRC’s oversight program for security. Although the processes initially had limited

stakeholder input, the staff has recently met with the industry to discuss and resolve comments.

Regarding regulating by order, the staff is on an aggressive schedule to incorporate order

requirements into Title 10 of the Code of Federal Regulations (10 CFR) through the normal

rulemaking process. This process includes opportunities for stakeholder participation and

comment.

All other comments have been previously addressed in responses to comments for questions 1

through 12.

14. Does the ROP ensure openness in the regulatory process?

Respondent Comments:

Senior Nuclear Industry Consultant

The answer is No. The public perception is that the NRC change from the SALP process to the ROP is

hand-holding with the industry to provide elevated assessments of plant safety performance.

Pannell Consulting

No!

Region IV Utility Group

YES (REACTOR) NO (SECURITY)

The ROP provides an objective, repeatable process for assessing plant performance. PIs and inspection

results are readily available for public review and scrutiny (with the exception of Physical Security).

Regular public meetings are held with licensees to discuss annual performance assessment results.

The Security process needs to return to the rulemaking process and improve stakeholder involvement
-45-

and transparency. NRC needs to consider a Security specific website much like INPO uses with

assigned passwords for licensees to use to share information and operating experience.

Union of Concerned Scientists

No comment.

Nuclear Energy Institute

From its inception, the ROP has solicited input from all stakeholders, including the public, which has

resulted in a very open process.

For most areas of the ROP, the regulatory process is open. Regular public meetings are held with

licensees to discuss annual performance assessment results. However, Security has been very closed

and disassociated with the rest of the ROP. Another area of the ROP that remains closed is the

Significance Determination Process and Enforcement Review Panel (SERP)

The Security process needs to return to the rulemaking process and improve stakeholder involvement

and transparency. NRC needs to consider a Security specific website much like INPO uses with

assigned passwords for licensees to use to share information and operating experience.

Strategic Teaming and Resource Sharing

For most areas of the ROP, the regulatory process is open. However, Security has been closed and

disassociated with the rest of the ROP. If this question applied to Security alone, STARS would rate it at

a 4 or 5. STARS believes more openness in Security could be gained by minimizing the use of “Orders”

or “Advisories” and returning to the rule making process. Another area of the ROP that remains closed is

the Significance Determination Process and Enforcement Review Panel (SERP).

Region 5/6 Emergency Management

My participation is such an example of the openness.

NRC Response:

There were numerous comments regarding the security portion of the ROP and its lack of

openness (i.e, very closed and disassociated with the rest of the ROP). The security

cornerstone remains a part of the ROP. In March 2004, the Commission directed the staff to

develop a separate method for assessing appropriate NRC actions in response to security

performance, and to keep all sensitive security-related information out of the public domain. 

The Office of Nuclear Security and Incident Response (NSIR) works closely with NRR to ensure

program consistency. In addition, NSIR holds periodic, public meetings on its security oversight

programs and related issues. Nonetheless, the staff recognizes that fundamental differences

between cornerstones and the nonpublic nature of security information result in unique

differences between the security cornerstone and the other six ROP cornerstones.

With regard to the openness of the SERP, it provides a management review of pre-decisional

information in the process of the preliminary determination of significance associated with a

finding. Since the information reviewed by the SERP is pre-decisional, the process is not open

to the public and licensees. However, details of how the process was applied to each finding is

incorporated into the publicly available inspection report. Details of the SERP process are

described in the publicly available document IMC 0609.01, “Significance and Enforcement

Review Panel Process.”
-46-

The NRC has reconsidered certain aspects of the closed nature of its security oversight

program for nuclear power plants. For example, later this year the NRC will allow public access

to the cover letters for security inspection reports, although the details of any related findings

will not be included in the letters. As the staff continues to strive for the right balance between

public scrutiny and ensuring common defense, it will consider during its deliberations the

recommendation for a secure website devoted to the security cornerstone of the ROP. The

staff is on an aggressive schedule to incorporate order requirements into 10 CFR through the

normal rulemaking process. This process includes opportunities for stakeholder participation

and comment.

15. Has the public been afforded adequate opportunity to participate in the ROP and to

provide inputs and comments?

Respondent Comments:

Senior Nuclear Industry Consultant

In addition to truly public input, an effort should be made to seek comments from nuclear professionals

and to give that input extra weighting.

Greenpeace

The public and for that matter the ACRS's comments have so often been ignored that I a can hardly

rationalize wasting my time to fill out this questionaire that never gets to the heart of the issue. 

Region IV Utility Group

YES (REACTOR) NO (SECURITY)

The public is afforded adequate opportunity to participate and provide input and comment. Regular

public meetings are held with licensees to discuss annual performance assessment results. Additionally,

public representatives attend the monthly ROP Task Force meeting. 

Security is not visible to the stakeholders and public. Additionally, licensee and stakeholder input are

somewhat stifled by the continued operation outside the normal rulemaking process.

Union of Concerned Scientists

As evidenced by (a) the monthly public meetings on the ROP conducted at NRC headquarters, (b) the

annual public meetings for individual plant assessments conducted in the reactor communities, (c) the

annual solicitation of public comments on the ROP, (d) the annual webcast Commission briefing on the

ROP, and (e) the August 2005 NRC staff documented response to public comments from the 2004

comment period, the public has ample opportunity to participate in the ROP and provide comments. The

only thing that prevents assigning a "1" rating to this question is the remaining need to improve the NRC

feedback to the public to their inputs and comments. As noted above, the August 2005 NRC staff

documented response to comments submitted by the public in 2004 is very good. If that feedback were

matched by comparable feedback provided to public inputs and comments received during the annual

assessment meetings, the rating would be "1." 
-47-

Nuclear Energy Institute

The public is afforded adequate opportunity to participate in the ROP process. On a monthly basis, the

NRC has held public ROP meetings to discuss improvements in the ROP process and answer

Frequently Asked Questions. The monthly ROP meetings have been effective in maintaining open lines

of communication between the NRC, industry, and other stakeholders. The one exception is in the area

of Security. While there may be opportunities to participate in the area of security for a select few, these

opportunities do not exist for the public.

Strategic Teaming and Resource Sharing

The public is afforded adequate opportunity to participate in the ROP process. On a monthly basis, the

NRC has held public ROP meetings to discuss improvements in the ROP process and answer

Frequently Asked Questions. The monthly ROP meetings have been effective in maintaining open lines

of communication between the NRC, industry, and other stakeholders. The one exception is in the area

of Security. If this question applied to Security alone, STARS would rate it at a 4 or 5. While there may

be opportunities to participate in the area of security for a select few, these opportunities do not exist for

the public.

Region 5/6 Emergency Management

Yes, I am an example

NRC Response:

Regarding the notion that comments from nuclear professionals should be sought and given

extra weighting, the NRC does seek input from nuclear professionals and gives it appropriate

consideration as with all other feedback received regarding the ROP.

The staff disagrees with the notion that the public’s comments have been ignored. As noted in

SECY-06-0074 and each of the previous ROP self-assessments, staff analysis of the survey

responses were provided in the applicable portions of the program area evaluations as well as in

the annual ROP performance metric report. In addition, to address a concern that the staff has

been unresponsive to survey comments, the staff consolidated the comments by question and

provided a comprehensive response to each question in the CY 2004 survey (reference

ML052090158). The staff received positive feedback on the consolidated response from several

stakeholders in this year’s survey, and therefore decided to provide a similar detailed response

to the CY 2005 survey.

The staff also disagrees that the Advisory Committee on Reactor Safeguards’ (ACRS)

comments have been ignored. The staff has been involved in numerous briefings and

correspondence with the ACRS and has addressed a number of their recommendations. Most

recently, as noted in Enclosure 1 to SECY-06-0074, the staff has proposed to the NRC/industry

task group to change the red threshold for the unplanned scrams PI to a deterministic value,

down from the current risk-based threshold of 25, as recommended by the ACRS. Additional

changes, as noted in previous responses, will also be discussed further with stakeholders during

the monthly ROP meetings.

As previously addressed in the response to question #14, due to the sensitive nature of

information in the security cornerstone, the Commission has directed the staff to keep all

sensitive security-related information out of the public domain. However, the NRC staff is open

to public input on a plant specific basis, either orally or in writing. We routinely receive
-48-

correspondence, e-mails, and telephone calls from members of the public, either to express

opinions, seek agency action, or to seek information and we do our best to respond in an

effective and timely manner to such inquiries.

16. Has the NRC been responsive to public inputs and comments on the ROP?

Respondent Comments:

Region IV Utility Group

YES (REACTOR) NO (SECURITY)

The NRC makes special efforts to recognize the public representatives at the monthly public ROP

meetings and allows the public to have an opportunity to voice their opinion on the issues discussed.

Additionally, annual performance review meetings are held with each licensee and the public is

encouraged to participate. Public comments are received, evaluated, and dispositioned in a professional

manner. 

In the area of Security see item #15 above.

Exelon, Byron

Unknown

Union of Concerned Scientists

See comments to Question No. 15 above.

Nuclear Energy Institute

The NRC has for the most part been responsive to public input and comments on the ROP. Following

the 2004 solicitation for public comment on the ROP, the NRC published a response to comments

submitted which was welcomed by the industry. Also, the action item list used to track and status issues

discussed at the monthly ROP meetings has been useful in ensuring issues and actions are assigned

and tracked to closure. The NRC makes special efforts to recognize the public representatives at the

monthly public ROP meetings and allows the public to have an opportunity to voice their opinion on the

issues discussed. Additionally, annual performance review meetings are held with each licensee and the

public is encouraged to participate. Public comments are received, evaluated, and dispositioned in a

professional manner.

Strategic Teaming and Resource Sharing

In the aggregate, the NRC has been responsive to public input and comments on the ROP. Following the

2004 solicitation for public comment on the ROP, the NRC published a response to comments submitted

which was welcomed by the industry. Also, the action item list used to track and status issues discussed

at the monthly ROP meetings has been useful in ensuring issues and actions are assigned and tracked

to closure. This comment does not apply to the area of Security in which the NRC remains relatively

unresponsive to public input. If this question applied to Security alone, STARS would rate it at a 4 or 5.

Region 5/6 Emergency Management

First hand experience
-49-

NRC Response:

As part of the NRC’s mission, the staff continues to put emphasis on maintaining and improving

its open communication with the public and stakeholders. As in previous years, the staff will

acknowledge receipt of each FRN response by correspondence indicating that the staff has

considered and generally addressed the comments in SECY-06-0074, as appropriate. In

addition, the staff has consolidated the comments by question and provided a comprehensive

response to each question. The SECY-06-0074 paper, the annual ROP performance metric

report, and this consolidated response will be posted to the ROP Web page and sent along with

the acknowledgment letters to each survey respondent.

Again, as previously noted in questions #14 and #15, due to the sensitive nature of security

related topics, to a large extent these are to remain out of the public domain.

17. Has the NRC implemented the ROP as defined by program documents?

Respondent Comments:

Nuclear Management Company

Shifting form [from] the ROP to the old regulatory oversight process for various issues is not as clearly

defined as it should be.

Region IV Utility Group

YES (REACTOR) NO (SECURITY)

The NRC as a whole has implemented the ROP as defined by the program documents. NRC has made

significant improvements in the program guidance relative to cross-cutting issue identification,

characterization and evaluation. Additionally, IMC 0612, Appendix ‘E’ has been improved to provide more

specific focus on cross-cutting aspects that may exist relative to NRC findings.

See items #15 and 16 above for Security.

Union of Concerned Scientists

See comments to Question No. 11 above. The ROP program documents are so dog-gone well hidden on

the NRC website that I am unable to find them, let alone determine if the NRC is abiding by them whilst

impliementing [implementing] the ROP.

Nuclear Energy Institute

At times NRC has regulated inconsistent with the program documents. These issues are usually resolved

through the FAQ process and resolved through revision to the program documents.

The NRC has implemented the ROP as defined by program documents with the exception of the

Security area which is neither scrutable nor predictable. 

The NRC as a whole has implemented the ROP as defined by the program documents. NRC has made

significant improvements in the program guidance relative to cross-cutting issue identification,

characterization and evaluation. Additionally, IMC 0612, Appendix ‘E’ has been improved to provide more

specific focus on cross-cutting aspects that may exist relative to NRC findings.
-50-

Strategic Teaming and Resource Sharing

Overall, the NRC has implemented the ROP as defined by program documents with the exception of the

Security area which is neither scrutable nor predictable. If this question applied to Security alone, STARS

would rate it at a 4 or 5.

NRC Response:

As discussed in the response to question #11, the staff is aware of the difficulties that can be

encountered when accessing the NRC Website and is continually working to make the site

more user friendly. As discussed under question #8, the staff is revising the guidance for

cross-cutting issues as part of ROP enhancements in the safety culture area. Also, the staff

agrees that improvements can be made with the PI program, as discussed under questions #1

and #2.

18. Does the ROP result in unintended consequences?

Respondent Comments:

Senior Nuclear Industry Consultant

Again, the SDP performed on the Davis-Besse RVHD event demonstrates that the process is flawed.

Pannell Consulting

Yes, too much analysis too little good engineering and operations.

Nuclear Management Company

This is clearly in the eye of the beholder. From the licensee standpoint there does appear to be an undo

burden for what appear to be relatively low significance issues. Despite the heavy reliance on a

corrective action process (CAP) at each plant there are no clear NRC guidelines as to what is an

adequate CAP[.] The NRC appears to be continually changing its view with each PI&R inspection.

Exelon, Kennett Square

This question is poorly worded since positive responses are on other side of scale[.] Less than needed

does not make sense.

Greenpeace

Yes, I doubt the NRC expected the industry to fight all the greater than green findings and waste the

NRC regualtory [regulatory] time and effort in SDP processes that last forever and give questionable

results. 

Region IV Utility Group

NO (REACTOR) YES (SECURITY)

The ROP has the potential to cause unintended consequences. Two Performance Indicators

could cause unintended consequences as discussed earlier in question # 1 above. 

A potential unintended consequence may result from the recent practice of identifying crosscutting

aspects for essentially all NRC findings. This practice naturally results in additional regulatory focus
-51-

during mid-cycle and end-of-cycle performance meetings by NRC staff. However, the impact should only

be the additional time expended by NRC during the assessment process. The potential exists to equate

numbers with significance and this was never understood to be the intent of the ROP at its inception—no

aggregation of minor or green findings was intended.

Security: The practice of notifying the state and local officials per the action matrix and implementing a

“no comment policy” could have the unintended consequence of undermining public trust and

confidence.

Union of Concerned Scientists

The ROP continues to result in the unintended consequences of wasting time and money on nonsense -

but that waste is not the fault of the NRC staff. It's the product of terribly poor judgment by the NRC's

licensees. The quintessential example this past year was the licensee for Palo Verde wasting untold time

and money with its silly plexiglass scale model of the ECCS piping in a futile attempt to convince the

NRC staff that its having deliberately drained water from the piping was no big deal. Likewise, the equally

misguided attempt by the Cooper license to convince the NRC staff that a 10-plus fire at its site was

actually a 9-minute fire followed shortly thereafter in the same vicinty by a 4-plus minute fire (all in a futile

attempt to avoid having violating the requirement to notify the NRC each time a fire lasts longer than 10

minutes) was another wasteful exercise. But the NRC staff and the ROP is not to blame when commonsense-deprived licensees opt to supplement poor performance with poor judgment.

Nuclear Energy Institute

The ROP does minimize unintended consequences. A good example of avoiding an unintended

consequence is the recent revision to MC 0612 which bettered defined “Licensee-Identified.” The

definition in the previous revision did not encourage a licensee to openly identify problems. Caution

should be used when implementing change without thorough review of the ROP working group. One

area susceptible to unintended consequences is Security.

A potential unintended consequence may result from the recent practice of identifying cross-cutting

aspects for essentially all NRC findings. This practice naturally results in additional regulatory focus

during mid-cycle and end-of-cycle performance meetings by NRC staff. However, the impact should only

be the additional time expended by NRC during the assessment process. The potential

exists to equate numbers with significance and this was never understood to be the intent of the ROP at

its inception—no aggregation of minor or green findings was intended.

Security: The practice of notifying the state and local officials per the action matrix and implementing a

“no comment policy” can have the unintended consequence of undermining public trust and confidence.

Strategic Teaming and Resource Sharing

The ROP does minimize unintended consequences. A good example of avoiding an unintended

consequence is the recent revision to MC 0612, which clarified the definition of “Licensee-Identified.” The

definition in the previous revision did not encourage a licensee to openly identify problems. Caution

should be used when implementing change without thorough review of the ROP working group. One

area susceptible to unintended consequences is Security.
-52-

NRC Response:

The NRC staff continues to work to improve PI and overall ROP effectiveness to minimize

potential unintended consequences, however we acknowledge that any open regulatory

process which assesses performance in a meaningful way will likely result in some level of

unintended consequences. The staff’s priority is to ensure that the unintended consequences

do not have a negative impact on plant safety. It is also the responsibility of licensee

management to ensure that operation of their facility is driven by safety, and not by a desire to

meet performance monitoring criteria.

Regarding the statement that this question was poorly worded and that a positive response is

inconsistent with the other survey questions, the staff acknowledged this inconsistency in the

annual ROP metric report (ML060590135) under metric O-10. The staff noted that this

inconsistency made it difficult to determine the external stakeholders’ relative perception of

whether the ROP results in unintended consequences. The staff will reword the question in

future surveys to read as follows; “does the ROP minimize unintended consequences?” which

will provide a more consistent response.

The aspects of the comments related to the security cornerstone are addressed in the

responses to question #13 and #14. Comments regarding the additional regulatory focus

during mid-year and end-of-cycle when identifying cross-cutting issues have been addressed in

the response to question #8, as noted in Enclosure 4 to SECY-06-0074. 

19. Please provide any additional information related to the Reactor Oversight Process*

*Note to better address the comments: The staff evaluated these comments according to the ROP

program areas (performance indicators, inspection, significance determination process, and

assessment) that they were related to. The responses were then accordingly grouped, assessed, and

addressed by the appropriate ROP program area leads within their respective areas of expertise. 

Comments received that were outside of the ROP program areas are addressed under the category of

other.

**Due to the magnitude and complexity of EFMR Monitoring Group’s comment, the text has not been

included in this section but can be viewed in ADAMS (ML060250245) at the NRC public website and is

responded to as part of the Other section.

Performance Indicator Section

Respondent Comments:

Strategic Teaming and Resource Sharing

STARS views the Reactor Oversight Process as an evolving process and, as such, appreciates the

ability to participate in the ROP task force. The ROP task force is an excellent collaborative effort

allowing stakeholders to continuously review and improve the process. Some of the more notable

improvements made in the past year include progress with MSPI implementation, forward movement

with a replacement indicator for the "Scrams with loss of normal heat removal" PI, implementation of a
-53-

revised FAQ process that includes tools to deal with questionable FAQs and provides an appeal process,

and revisions to MC 0612 and 0305. 

Southern California Edison

There are several areas, however, that we believe require continuing attention:

! As in all things, Performance Indicators (PIs) and other aspects of the Reactor Oversight Process

(e.g., Significance Determination Process (SDP), etc.) can create unintended consequences. There is

a continuing need for a robust and ongoing process to identify and address such situations as they

arise.

! SCE remains concerned with various recent proposals made by the NRC staff to revise upward some

of the Performance Indicator thresholds. Changing the PI thresholds would impose a de facto "rising

standard." SCE supports the original NRC position that the thresholds were set with the expectation

that, while licensee performance would be expected to improve, performance at the current thresholds

represent "acceptable licensee performance."

! The Mitigating Systems Performance Index (MSPI) effort has been a lengthy and difficult process and

the resultant MSPI has evolved well beyond what was originally piloted. SCE believes, therefore, that

the NRC should conduct a "lessons learned" evaluation of the MSPI effort before further changes are

made.

NRC Response:

With regards to the development of a lessons learned report for MSPI, industry has volunteered

to take the lead on developing a lessons learned report for MSPI. The staff will continue to

work with industry over the remainder of 2006 to compile and finalize any lessons learned. 

With regard to PIs, one utility and one utility group noted the progress made in the past year

with regard to the MSPI and SwLONHR but, were concerned about what they called “recent

proposals made by the NRC staff to revise upward some of the Performance Indicator

thresholds.” This is a mis-characterization of the staff’s proposal regarding the Unplanned

Scrams per 7,000 Critical Hours and the Unplanned Power Changes per 7,000 Critical Hours. 

In 1998, when the PI program was developed, the industry proposed to count unplanned

scrams per year. The staff, however, was aware that more than 20 units had critical hours

significantly below the industry average. For those plants, a lower threshold corresponding to

the fewer hours would be appropriate. 

To implement such a plan, the staff proposed and industry agreed to use the method the staff

and the Institute for Nuclear Power Operations have used for many years - to count unplanned

scrams per 7,000 critical hours. The 7,000 critical hours was the industry average value for all

plants. This method provided margin to the threshold for plants with more than 7,000 critical

hours per year and captured plants with fewer critical hours per year. However, the industry

average number of critical hours per year is now up to about 8,000 and the staff proposes to

use that value in place of the original 7,000 critical hours. If this change is not made, the PI

becomes scrams per 10.5 months rather then scrams per 12 months.
-54-

Inspection Section

Respondent Comments:

AMEC Earth & Environmental, Inc. 

What needs to happen is that trending of GREEN NCV findings has to happen - number, frequency, and

functional area - based on a "chips fall where they may" philosophy. Every finding from every inspection,

no matter how trivial, goes into the NRC trending report. This trending information can then be used as

part of the significance determination process - by providing objective evidence of whether things are

getting better or they are getting worse. This has the effect of having licensees to start worrying about

number, type and extent of any problems and the need for them to find them and fix them before NRC

finds them. It also takes the blinders off the inspectors - since all inspection findings will now matter in

some shape or form. The trending results can also be used to adjust the GREEN-WHITE threshold

down to something more appropriate. Further, don't let licensees argue about GREEN violations - this is

what the NRC inspectors found - period. Individually it is not significant, but the aggregate is being

trended. This could then change the focus of P&IR inspections to determining trends licensees finding

and fixing their own problems compared to how many and what types of problems are being found by

NRC. This will have three beneficial effects - (1) licensees won't argue with inspectors and waste their

time; (2) even "marginal" GREEN-WHITE that are decided in the licensees favor will still count in the

trending, and: (3) it will reduce licensees ability to "manage" the PI and thereby avoid ever having to look

at the big picture until it is too late.

Part of the trending should also levelize the playing field - how many inspection hours were expended to

determine how many findings? It has always seemed to me to be patently unfair that a plant that is

perceived as being a "bad" performer gets more inspection hours; and SURPRISE, there are more

findings! Thus, they waste a lot of time and resources on problems that would be perceived as trivial if

they happened at another plant that is perceived as a "good" performer. The playing field needs to be

leveled. 

If you believe that the ROP and significance determination process is good, then how about an objective

validation of the premise. One way that could be done is that NRC increases inspection hours on a

"good" plant or two annually on a random basis to see what happens to the number and severity of the

findings. That is one way to guard against complacency. It is particularly important to start this NOW

since in the near-term, we can expect new construction. Where do you think all of the management

talent will go to - overseeing existing plants (which requires a level of attention to detail that most

managers simply do not possess) or building the new plants where the corporate financial risk and hence

management career rewards lie?

Region IV Utility Group

! Inspection Manual changes very positive and an overall improvement to process especially the selfidentification of findings and guidance regarding substantive crosscutting issues.

! NRC should institute a quality review on some periodicity with regard to the Website as there are

disconnects at times between inspection report data and the PIM.

! Enforcement manual guidance needs to be improved to clarify expected actions relative to NCVs

within the ROP.

Since the inception of ROP, most licensee’s have dispositioned NCVs as determined by

their corrective action process relative to the significance of the particular issue. Subsequently, NRC has

evaluated this disposition during the Baseline and Problem Identification and Resolution Inspections. The
-55-

NRC evaluations have appropriately focused on the licensee’s corrective action program and its

adequacy in response to the issue. It has come to our attention that some individuals within NRC and

licensee organizations are questioning this practice and the wording of the Enforcement Guidance and

interpreting the guidance to mean that all NCVs require root causes and actions to prevent recurrence. 

It is our understanding that NRC expectations are that inspectors will address the adequacy of corrective

actions to prevent recurrence for a NCV within the specific inspection report that discusses the issue

based upon the information available at that time. This evaluation would be based upon a

“reasonableness” determination by the inspector at that time. Additionally, NCVs do not necessarily

require a root cause determination. The important aspects are correct the condition and take reasonable

actions to address recurrence. 

Presently, no specific guidance is available to inspectors reflecting how to evaluate this area and this

lack of guidance could result in unintended consequences. We suggest that the enforcement guidance,

inspection guidance, and performance assessment guidance be revised to clarify expectations.

Nuclear Management Company, LLC

NMC wholly shares the views expressed by the Nuclear Energy Institute (NEI) in their comments dated

November 30, 2005. In particular, NMC is concerned about the recent revision to the definition of

Performance Deficiency in Manual Chapter 0612, as discussed by NEI’s comments in response to

Question 5 and 9, and the potential for this new definition to circumvent the process for imposing plantspecific backfits pursuant to 10 CFR 50.109. 

Nuclear Energy Institute

! Inspection Manual changes very positive and an overall improvement to process—especially the selfidentification of findings and guidance regarding substantive cross-cutting issues.

! NRC should institute a quality review on some periodicity with regard to the Website as there are

disconnects at times between inspection report data and the PIM.

! Enforcement manual guidance needs to be improved to clarify expected actions relative to NCVs

within the ROP.

Southern California Edison

There are several areas, however, that we believe require continuing attention:

! The ROP Task Force public meetings have ensured good communication vehicle between the ROP

stakeholders. We note, however, that the recent revision to MC 0612 was not afforded the appropriate

opportunity to be commented on by stakeholders prior to its' final release.

Greenpeace Attachment

Below in no specific order are some areas in which the ROP could be improved:

6. If the ROP and the decision to move away from N+1 resident inspectors have proven effective, why is

it that at the end of each of the last couple of years, inspector-qualified personnel from NRC

headquarters have been called on to complete the base[-]line (minimum) inspection programs at

numerous sites? If the answer is that significant resources were needed to be expended at 

Davis[-]Besse, why isn’t the contingency for such a “problem plant” built into the assumptions, as there

always seems to be at least one such plant?

10. Even though not specified in the definition of a performance deficiency, as a matter of practice, the

licensee performance at issue in a particular finding has to be current licensee performance (and the
-56-

definition of exactly what that - how far back you can go and why - is undefined). From one perspective,

evaluated “current performance” makes sense, as the action matrix is designed to reflect the NRC

assessment of current performance. However, if past licensee performance results in unintended risk, it

is too old to be considered a “performance deficiency”, and does not meet the old design issue criteria,

how is that risk evaluated/accounted for? Shouldn’t be? Example: What if it was identified by an NRC

inspector that many years ago a licensee employee used an inadequate procedure to perform an

evolution and the evolution resulted in a risk significant situation? Would it be a performance deficiency

despite its age? What if the same employee still worked for the licensee and currently would perform

similar duties? What if upon review, the management and employees were different but the same

inadequate procedure was in place today? Would it then be considered a “performance deficiency” or

only if the procedure was used more currently?

11. MC 0612, 5.04.c allows branch chiefs to disposition as green findings certain issues that do not fit

into existing SDPs. How often is this provision used? Are there any reviews done to assure consistency

of the types of issues that branch chiefs determine can’t be put through the SDPs and their bases for

determining such issues are green?

NRC Response:

To address the comments regarding the Enforcement Manual guidance, the staff will review

whether additional program guidance to clarify program expectations with respect to whether

root cause determinations and licensee’s actions to prevent recurrence of the NCVs are

needed. 

Comments were received regarding the focus of IP 71152, “Identification and Resolution of

Problems” and its ability to identify trends. Inspection requirement 02.01.d from IP 71152

requires inspectors to perform a semiannual review of identified deficiencies for the purpose of

identifying trends (either NRC or licensee identified) that might indicate the existence of a more

significant safety issue. Included within the scope of this review are repetitive or closely related

issues that may have been documented by the licensee outside the normal corrective action

program, such as in trend reports or performance indicators, major equipment problem lists,

repetitive and/or rework maintenance lists, departmental problem/challenges lists, system

health reports, quality assurance audit/surveillance reports, self assessment reports,

maintenance rule assessments, or corrective action backlog lists. This review is performed by

summarizing the results of the licensee’s reviews and comparing those results to those

identified by the NRC through the baseline or supplemental inspection program, including

issues identified as a result of the daily review of corrective action program items listed above. 

Additionally, there is a requirement that if a biennial PI&R inspection is scheduled within six

months of the semiannual review, the senior resident inspector could forward any concerns to

the PI&R team. This information should be incorporated into the scope of the team inspection.

The results of this review are documented in an NRC inspection report.

With regards to the definition of performance deficiency, it is defined in IMC-0612 and it states: 

“Performance Deficiency: An issue that is the result of a licensee not meeting a

requirement or standard where the cause was reasonably within the licensee’s

ability to foresee and correct, and that should have been prevented. The licensee

does not have to be committed to a standard in order to determine whether there

is a performance deficiency (PD). For example, a PD is determined to exist if the
-57-

licensee fails to adhere to a widely accepted industry standard. If the

performance deficiency has related cross-cutting aspects, the cross-cutting

aspects are generally considered an underlying cause of the performance

deficiency rather than an independent issue. Issues of problem identification and

resolution, human performance, or establishment of a safety-conscious work

environment, in and of themselves, do not provide the basis for a performance

deficiency.”

Failure of plant hardware (e.g., plant system) to meet a regulatory requirement needs to be

corrected by the licensee, regardless of when it occurred. Similar to plant hardware,

deficiencies in plant procedures which can result in human performance issues need to be

addressed by the licensee, regardless of how long the procedure may have been in use.

In response to the comment suggesting a change to the inspection hours, the ROP baseline

inspection program uses the same sampling process for all licensees. Inspection effort is

increased at a plant when the need is so indicated by the assessment process Action Matrix,

based on findings identified by the NRC or the licensee. To keep the process objective and

fair, inspection resources cannot be increased at random at any one plant. However, the NRC

staff is continuously monitoring findings for cross-cutting issues, which may be indicative of

deteriorating licensee performance. 

A comment was raised regarding the NRC policy change in the resident inspector staffing and

budget. Specifically, there was no reduction in the total inspector staffing positions as a result

of the change from the NRC's "N+1" resident inspector policy. The resident inspector

resources saved as a result of changing the "N+1" resident inspector policy were transferred to

the regions. However, due to a number of events during recent inspection cycles, the ability of

the NRC staff to complete the required baseline inspections was challenged. In anticipation of

the potential impacts of these challenges, the staff took preemptive action in order to avoid any

adverse consequences which included NRR and other staff contributing significant resources to

assist two regions in successfully completing the baseline inspection program in 2002 and

2003. No such assistance has been required for the 2004 and 2005 inspection cycles. In order

to address potential budget shortfalls and avoid inspection resource challenges in future years,

the staff re-evaluated the inspection resource needs in each of the four regions. As a result,

the annual regional budget for operating reactor inspection activities for fiscal years FY 2004-

2006 was increased by approximately 15 full-time equivalent (FTE) positions over the FY 2003

budget, in part to provide additional resources for oversight of an IMC 0350 plant and to assist

in post-supplemental inspection activities to verify licensees' improvement plans. 

A comment was received suggesting a periodic review of the ROP website, specifically to

ensure there are no disconnects between the inspection report data and the PIM. The NRC

staff reviews the ROP website on a regular basis and strives to keep the information provided

up to date. If there are any inconsistencies between the inspection report data and the PIM,

please notify our staff with detailed information on the issue so we may quickly correct the

problem. Send issues to http://www.nrc.gov/site-help/feedback.html. 

Finally, comments received regarding the disposition of findings has been addressed in

response to question #4. Similarly, the use of management to review green findings that fall

outside of the existing SDPs has been addressed in response to question #6.
-58-

Significance Determination Process Section

Respondent Comments:

Pannell Consulting

My comments are based on 37 years of experience as a leader in the Nuclear Industry, we need to get

the balance right for the next generation of plants and personnel. Remember the initial learning curve,

here it comes again. Why didn't the process prevent the Davis Besse situation? If it had gone to the

next step (LOCA in the RV head) the industry would never have recovered, how did we get that close

with all this great risk analysis stuff? It is still the people and their training that make the difference not a

bunch of forms that tend to allow for clever analysis as opposed to rigorous conservative engineering

and operations assessment and corrective actions. Make the process as objective and simple as

posssible [possible] with Nuclear Safety in the front of everyone's mind!!!

AMEC Earth & Environmental, Inc. 

Too much of what NRC does (and hence most of the licensees) is reactive and not pro-active. This

leads to too much subjectivity in the significance determination process, i.e., the need to "force-fit" the

"objective" results to a general impression of what is going on at a given facility. It also continues the

politicizing of the NRC oversight role. If a lot of plants are not meeting the mark then Congress will get

on NRC for not doing its job. Conversely, if all plants come out OK, then Congress will also get on NRC

for not doing its job. The ideal political solution therefore, is to have some plants that are "bad" requiring

extra NRC oversight, and which plants they are needs to change periodically so that no one is forced to

lose their license (and the fees that are paid to NRC). Unfortunately, even this political compromise is

being undermined by the NRC reliance on "the usual suspects." 

Region IV Utility Group

! With the implementation of Regulatory Guide (RG) 1.200 PRAs, industry is concerned regarding how

the significance determination process (SDP) will deal with licensees that have RG 1.200 compliant

PRAs. This concern is founded on the basis that the NRC SPAR Model has not been made to comply

with RG 1.200.

! NRC continues to pursue the development of limited scope SDPs that are not as risk informed as 

would be expected at this stage of ROP (should rely upon Reactor Safety SDP whenever possible). 

This complicates the ROP and should be minimized.

Nuclear Energy Institute

! NRC continues to pursue the development of limited scope SDPs that are not as risk informed as

would be expected at this stage of ROP (should rely upon Reactor Safety SDP whenever possible).

This complicates the ROP and should be minimized.

Strategic Teaming and Resource Sharing

Areas for continued improvement are Security and Fire Protection which need to be more scrutable and

predictable. STARS believes a greater degree of oversight in the Security and Fire Protection areas by

the ROP task force would further the improvement process and the NRC goals of scrutability and

predictability in the ROP.

Southern California Edison

There are several areas, however, that we believe require continuing attention:

! While some conservative "false positives" are acceptable from any such processes (i.e., Performance

Indicators, SDPs), it is also necessary that the Reactor Oversight Process identifies and resolves
-59-

potential opportunities for "false negatives." "False negatives" have the potential to significantly

undermine the credibility of the entire Reactor Oversight Process.

! Difficulties continue to be experienced with the development, precision, and robustness of the

Significance Determination Processes. Several SDPs are not as robust as they should be, and do not

produce consistent and/or accurate results.

! The NRC has initiated effort to improve Significance Determination Process timeliness and we support

this effort. The Commission is establishing goals for Significance Determination Process timeliness at

90 days. There may, however, occasionally remain complex engineering judgment issues that may

take longer than 90 days to resolve. SCE believes the NRC should give a licensee sufficient time to

determine the actual facts and circumstances of an event, even if that should require waiving the 90

day goal for such exceptional cases.

Greenpeace Attachment

Below in no specific order are some areas in which the ROP could be improved:

1. There appears to be a need for some type of validation on the thresholds for Green, White, Yellow,

and Red findings, both within a given cornerstone and across cornerstones. 

A. Within a cornerstone- Very few actions year to year rise to the level of even White

significance (low to moderate). That could be because there are few actually significant

issues or it could be that some portion of issues found now to be green are in fact more

significant. More significant meaning that the risk determination fails/cannot quantify

important information or that all the risk is correctly taken into accounted but the threshold

needs to be lower for it to be a valid indicator. The fact that the thresholds for green,

white, yellow and red for the numerically based SDPs are the same as they have been

since the initial implementation of the program implies that the thresholds were initially

and are still appropriate. How do you know they are correct? Said another way, what

benchmarks(s) are greater than green findings, as now defined and accounted for

(multiple whites being a degraded cornerstone etc.), measured against to ensure they are

an adequate (early enough) indicator of degraded or declining performance? Can events

that have occurred be used to improve our indicators? Might another indicators such as,

lowering the white threshold, giving more significance to findings in certain cornerstones,

single greater than green findings in multiple cornerstones, or total numbers of findings

(including green findings) be equally or more valid predictors of performance? Note:

certain SDPs count/have counted issues per time and in effect acknowledge some added

significance to otherwise green findings that occur with some frequency. 

B. Across cornerstones- A relatively large portion of the greater than green findings issued to

date are in areas such as EP and HP. Given that each greater than green finding,

regardless of area, affects the action matrix equally it would seem there are needs to be

proper normalization across areas to ensure that a white in one area is equally reflective

of licensees performance as the same color of finding in another area. As each SDP was

developed by subject matter experts in that area, was there any effort to normalize similar

colored findings across SDPs? More importantly, is such normalization periodically

performed or re-performed when, for instance, a given SDP is significantly revised? The

enforcement policy specifically cautions “Comparison of significance between activity area

are inappropriate” because of the difficulty there would be in comparing across disciplines.

Given that the SDPs focus on outcomes ad not causes such as poor oversight or

procedural adherence, which could be compared across disciplines, how are findings

across areas judged comparable? Are areas such as EP and HP alone valid indicators of
-60-

overall performance? Are such areas over-represented in the mix of greater than green

findings, are plant operational areas under-represented or maybe both? How do you

know? Why is it that a plant like Cooper, with a long history of operational performance

issues including an extended shutdown to fix hundreds of environmental qualification

findings, is only enhanced oversight because of EP issues?

2. The bands for what constitutes a finding of a particular color appear to overstate the degree of

accuracy that is reachable using the tool available. a) The uncertainty/error bands associated with come

very thorough but complex determinations may span two or eve three colors. In fact, on a given issue,

multiple one decade answers may be “right”. Are the bands in use too narrow or rigidly defined? b) PRA

quality is important and will realize some improvements however it remains to be seen if a standard of

quality of PRA “commensurate with its application” is going to bring about quality that is actually good

enough to define risk into the narrow bands of the ROP. Further, without standardized PRAs,

comparison of results between plants and maybe even between systems in a given plant is problematic.

Results at the E-4 through E-7 levels are being compared and slight differences in HRA assumptions,

generic vs plant specific equipment data, and assumed outage times will have significant impacts.

Further, given that each plant starts with a unique baseline CDF, how are changes to CDF within given

fixed one decade bands the proper measurements? Shouldn’t there be some considerations of change

relative to the baseline? c) MC 0609A 2.5 recognizes that the external events risk contribution may

significantly change the color of the SDP determination, yet the quality and rigor of the licensee IPEEEs

is more variable than IPEs. How is this additional variation in quality and rigor accounted for in the

existing narrow color bands? Doesn’t the need for the NRC staff to do “best estimate” calculations when

it comes to many issues involving external events acknowledge that many IPEEEs are inadequate? More

generally, why docketed, subject to uniform quality standards, and tied to a formal change process has

got to be a mystery to many NRC stakeholders. 

4. The current MC guidance says that the risk increase of a given performance deficiency is evaluated

discretely and not considered in an integrated fashion without other performance deficiencies that affect

the same equipment. The approach was criticized externally after it was used for various deficiencies at

Davis Besse and the defenses of that approach given by various NRC managers have been in the press.

It would seem that an integrated evaluation of all the degraded conditions that affect a system that are

known: a) provides a fuller disclosure of the risk and b) avoids the perception that the NRC looks at

licensee performance in merely in an event by event fashion when in fact the goal is to have an

integrated assessment of performance. That view aside, if the NRC is going to use this deficiency

approach, it should do it consistently. When the NRC evaluated a degraded fire door condition at ANO,

initially it was classified a potential white finding. Subsequently, the licensee argued that a fire watch

unrelated and un aware of the deficiency should be credited because that fire watch just happened to

transit the area in question during the time period in question. The NRC accepted that argument and

because of that, the deficiency received a final green disposition. When are/should surrounding but not

directly related circumstances be included in the NRC assessment of a particular issue?

7. Timeliness of the assessment process continues to be an issue despite timeliness goals. It would

appear to be because timeliness is not tied to discover/identification but rather report issuance which

may occur long after an issue is first identified. As stated in the ROP implementing documents, a finding

is not placed in the action matrix until it is a final determination. Many preliminary SDP assessments are

not sent to the licensees for many months after the item occurs/is first identified. Similarly, the final

findings are not completed in some cases for many months after the issuance of the preliminary finding,

resulting in the actual posting of the final finding lagging the actual occurrence/identification of the event
-61-

or issue by many months. Is the correct balance of accuracy of the determination and timeliness of

issuance being achieved if the action matrix is trying to evaluate current licensee performance?

9. 0609A states that the SDP is to estimate in CDF due to conditions which contribute “unintended risk

caused by licensee performance”. It goes on to say that other events that contribute to overall risk fall

within “acceptable plant normal operating risk.” In other places this normal operating risk is referred to as

baseline risk. a) Occasionally when risk assessments are performed certain risk contributors are

excluded because the assertion is made they are part of the normal operating/baseline risk. What

contributors can be excluded in such a manner? Besides being all risk not evaluated through SDP, what

exactly is this normal operating or baseline risk? It is an average value of risk contributions from

maintenance, random failures, and equipment outages? Does it include the presence of a certain risk

contribution attributed to longstanding but undiscovered design flaws or QA violations? It is a limiting

value arrived at by considering the risk contribution of having the maximum amount of TS required

equipment out at a given time? Does the definition differ from PRA to PRA? If so, how are comparisons

of risk increases from approaches that define this term differently valid? Example: If a motor driven AFW

pump was inoperable due to a performance deficiency would the risk associated with the other motor

driven AFW pump being out for maintenance fall into normal operating risk? What about the

unavailability of one steam supply to the stream driven AFW pump? Both? What about emergency

power to a power operated dump valve? What about all three? b) Given the above guidance on what

risk is considered in an SDP, if a licensee performed a 50.65 (a)(4) assessment, determined that the

group of normal maintenance activities not due to be performed for some time would increase CDF by E5, and despite those factor took the equipment out of service to perform the maintenance, this would fall

into normal operating risk because it was not “unintended risk” and therefore would not be evaluated

using SDP. There also would be no regulatory violation provided all LCOs were met and the licensee not

only assessed the risk but “managed” it as well. However, wouldn’t the NRC want to capture the decision

by the licensee to increase CDF by so much when it was not actually required? But, neither the SDP or

the enforcement process would appear to provide a methods for that to be accomplished. So, is the

standards of “unintended risk caused by licensee performance” the correct one if minimizing all

undue/unnecessary risk is goal of NRC oversight?

12. How the term “performance deficiency” is implemented can have a large effect on the number of

greater than green findings. Example: A plant that has three separate and largely independent

procedural problems in the maintenance area. Each instance, after evaluation, has a risk contribution of

1E-6. If the performance deficiency is defined as a failure to adequately implement maintenance

procedures, you end up with a single white issue with an associated risk increase of approximately 3E-6.

However, if in each case the performance deficiency is narrowly defined (failure to adequately implement

electrical, mechanical and instrumentation maintenance) the result is three separate white findings and a

degraded cornerstone. The ROP guidance is silent on which is the correct approach and is similarly

silent on how to deal with related issue of additional examples the licensee may discover when correcting

the identified issue(s). Are such additional examples assessed separately? Is the risk contribution of

those additional examples combined with that of the initial final finding? Is so, what if the additional risk

would cause a color change? Can the definition of the performance deficiency simply be expanded to

include the examples and no additional finding assessed? If to, does the licensee get an opportunity to

review/comment? Alternatively, should findings that the licensee discovers through its corrective action

efforts be assessed and considered in the action matrix? Doesn’t assessing and considering such issues

create a disincentive to finding additional issues?
-62-

NRC Response:

A comment was received regarding RG 1.200 compliant PRAs and their correlation to SPAR

models. A regulatory guide provides one way for a licensee to comply with the NRC

regulations. Specifically, RG 1.200 provides techniques used by the NRC staff in evaluating

specific problems or postulated accidents and the data needed by the staff in its review of

applications for permits or licenses. RG 1.200 is not a requirement but the PRAs that satisfy

the RG 1.200 guidance will provide an improved basis for risk-informing findings. SPAR

models, on the other hand, are a confirmatory tool used by the NRC staff in support of the

assessment of risk significance for findings. 

To address the comments regarding the thresholds for findings within and across cornerstones, 

SDPs are tools used to approximate the risk of licensee performance deficiencies within a

conservative range up to one order of magnitude. Hence, white findings often end up as green

when challenged by licensees, who have invested significant resources to quantify the

conservatism built into their robust plant design, thus demonstrating system capability within

plant safety design requirements under the identified degraded conditions. It is important to

remember that all conditions adverse to quality, that is all findings minor, green, white, or

greater, must be corrected. Evaluating the threshold is a continuous process. Several

adjustments since the program’s inception have been made to the threshold for the

deterministic risk-informed SDPs. An expert panel (SERP) reviews every finding with a

potential greater than green outcome. Normalization, and assuring equivalence, is part of the

panel’s responsibility. Also, the NRC holds an annual Agency Action Review Meeting (AARM)

to review the agency actions resulting from the performance of nuclear reactor licensees for

those plants with significant performance problems as determined by the Action Matrix and to

review industry performance trends. The AARM also considers whether the ROP is

accomplishing the proper level of regulatory oversight. 

Specifically, with regard to the number of green findings, the NRC baseline inspection program

routinely reviews corrective action associated with Green findings. Green findings are generally

violations of regulatory requirements, or licensee commitments that are part of the plant’s

licensing basis. Green findings prompt licensee actions to correct associated performance

deficiencies. Once a year the NRC staff reviews a sample of Green findings to determine if any

should have been designated as White. The sample size provides 80% confidence that there

were no non-conservative risk assessments during the period.

In response to the comment regarding the significance assessment of a licensee’s performance

deficiency, this is generally based on the actual occurrence, not the potential failure that has not

happened. The example of the fire door and the unrelated fire watch demonstrates this. The

fire door left open inadvertently came out as a white finding; however, the area fire watch, not

related to the door, reduced the notification time had a fire actually occurred, thus reducing the

actual impact of the open door.

In response to the comment regarding the level of risk uncertainty, the staff agrees that

periodically the level of uncertainty associated with a finding will cause the outcome of the SDP

to cover a band of two or even three orders of magnitude. In such situations the NRC staff

makes an effort to reduce the uncertainty using the various available tools including additional

inspection, licensee PRA data, etc. However, extra effort always results in expending additional

resources and may prolong the process beyond the 90 day timeliness target. The staff seeks
-63-

to achieve a balance in this regard. Additionally, to address some issues, the staff is

considering an enhanced risk-informed management decision making process.

Another comment questioned the SDP timeliness target and its appropriateness when

evaluating complex issues. The NRC staff agrees that in some instances of highly complex

issues, the 90 days may not be sufficient time to come to a final risk determination. The

process allows for such cases, however, the failure to meet the timeliness goal will count

towards the appropriate metric. The timeliness requirement prompts the NRC staff to not to

over engineer, over analyze, or initiate costly research to arrive at a risk conclusion not readily

supported by available information. 

There was a comment received regarding the definition of performance deficiency and its

implementation with the example of multiple performance deficiencies involving maintenance

procedures being assessed independently or together. The guidance is provided in IMC 0609

Appendix A, “Determining the Significance of Reactor Inspection Findings for At-Power

Situations,” Section IV titled, “Treatment of Concurrent Multiple Equipment Functional

Degradation.” If the multiple failures are linked to a common cause, e.g. maintenance

procedure, then a single finding reflecting the most severe impact on safety will be identified. If

and when the licensee identifies additional examples of the same issue as part of the extended

condition requirement, then appropriate corrective actions are to be implemented. If the

concurrent performance deficiencies are not linked with a common cause, then each deficiency

is assessed and independent findings are issued. Licensee identified performance deficiencies

that are assessed as greater then green during the extended condition review are also issued

as findings. Additionally, the accident sequence precursor (ASP) program is a tool used by the

NRC staff to assess the risk significance of issues and events based on the plant specific

conditions at that time. 

In response to the comment regarding the ROP’s inability to prevent the Davis Besse event, the

Commission chartered a Lessons Learned Task Force (LLTF) to carry out an extensive review

of the event to identify recommendations for staff action. In accordance with agency

procedures, Davis-Besse was placed under the IMC 0350 process and was subjected to a

detailed review prior to restart of the plant. The Commission also directed the staff to

incorporate safety culture aspects into the ROP, and directed the staff to enhance our

inspection of engineering practices related inservice inspections. A summary of NRC actions

and their status is available on the NRC’s Website at

http://www.nrc.gov/reactors/operating/ops-experience/vessel-head-degradation/lessons-learned

/lessons-learned-files/lltf-web-page-status083105.pdf. In 2005, the staff completed the

necessary enhancements to the ROP based on the implementation of DBLLTF action items. 

These changes have enhanced the NRC’s regulatory oversight rule.

In response to the comment about the NRC process being reactive, each plant receives the

baseline level of inspection with the goal of identifying plant problems prior to poor performance

resulting in more serious plant conditions. This process is enhanced by PIs in conjunction with

substantial cross-cutting issues. These are incorporated into the NRC Action Matrix which is

used objectively and fairly to assess licensee performance. Also, the NRC staff has a process,

described in MD 8.3 entitled, “NRC Incident Investigation Program,” specifically designed to

respond to events. Nonetheless, we remain open to specific suggestions for objective

indicators of future licensee performance. 
-64-

To address the comment regarding the subjectivity of the SDP, the staff believes that the ROP

is an appropriate but evolving process. As stated in ROP program guidance, it is the

responsibility of the NRC to objectively consider the available information and to reach a final

significance determination. Furthermore, the ROP is significantly more objective than the

previous inspection and oversight process. It is also much more performance based, in

contrast to the relatively higher level of programmatic focus in the previous process.

All other comments received in this section have been addressed in Enclosure 3 to SECY in 06-

0074 and in response to previous questions. Specific comments regarding the security and fire

protection portions of the SDP are addressed in response to question #10.

Assessment Section

Respondent Comments:

Senior Nuclear Industry Consultant

I am a Senior Nuclear Licensing Consultant and an Ex-NRC NRR Project Manager. Since leaving the

NRC and supporting commercial nuclear utilities, I have observed the slow deterioration of the NRC's

regulatory strength and rigor in enforcement activities. 

Regarding the ROP: Like the SALP before it, PUCs and PSCs use the ROP to make decisions on, and

challenges to, the financial business of nuclear utilities. The "Sea of Green" on the ROP PIs help the

nuclear utilities in their financial battles with the PUCs and PSCs (and stockholders for that matter), but

they mask developing problems at the plants. All of those Greens provide a false mesasure [measure]

of plant safety -- the performance bar needs to be raised. By the way, it is the public perception that the

NRC changed the performance grading system to intentionally and artificially inflate performance of the

plants (the same was also said when the number of SALP categories were reduced). Davis-Besse

happened on the ROP program's watch. The poor implementation of the ROP allowed FENOC's

management to distroy [destroy] the nuclear safety culture, SCWE, and corrective action program

effectiveneess [effectiveness] at Davis-Bess and almost resulted in an un[-]isolable LOCA that would

have been the death nell to any resurgence of the commercial nuclear power industry. Part of that

responsibility would have been on the NRC's shoulders.

On the Enforcement side: Having been involved in the recovery and restart efforts for Millstone, D.C.

Cook, and Davis-Besse, it is evident that the ROP implementation has resulted in a weaker NRC

enforcement program. Anyone involved in these restart efforts call tell you the differences between the

NRC's efforts at Millstone and the progressive weaking of NRC enforcement applied to D.C. Cook and

then on to what is almost pathetic enforcement applied to FENOC and Davis-Besse. I personnally fear

that the NRC's weaknesses in enforcement will be an accomplice to a major safety event in the near

future if not corrected. 

The ROP needs to be founded on real measures of safety performance, the cross-cutting cornerstones,

and forward-looking measures. The performance levels should be should be baselined to show only 50

percent of the plants as achieving overall Green performance. When truly Green performance has been

achieved for some designated period of time, the bar needs to be raised to drive further improvements.
-65-

AMEC Earth & Environmental, Inc. 

My general comment is that the ROP is of limited use. This is tied to what I regard as a very high

threshold to reach "White" status. Or, to put it another way, the range between "good" GREEN and

"bad" GREEN is at least as much as that between the GREEN-WHITE threshold and RED.

This results in the following effects:

1. Too much is kept strictly between the NRC regions and the individual licensees insofar as to

non-conformance with regulations. This leads to

2. Hampered ability of NRC to objectively trend licensee performance BEFORE it reaches a point of

general breakdown. 

Nuclear Management Company

In general the program has been a good one. However, my main issue with the process is the lack of a

clear exit stratgey once a plant has been placed in a column other than Column 1 of the action matrix.

Region IV Utility Group

! A change to the Action Matrix should be considered such that three white findings are required to

transition to the Degraded Cornerstone column. The additional effort required to prepare for and

implement a 95002 inspection is rarely warranted for two low safety significance issues. A threshold

of three white findings to change from the Regulatory Response column to the Degraded

Cornerstone column in the Action Matrix would be a better use of available resources. 

Nuclear Energy Institute

! A change to the Action Matrix should be considered such that three white findings are required to

transition to the Degraded Cornerstone column. The additional effort required to prepare for and

implement a 95002 inspection is rarely warranted for two low safety significance issues. A threshold

of three white findings to change from the Regulatory Response column to the Degraded

Cornerstone column in the Action Matrix would be a better use of available resources.

Southern California Edison

There are several areas, however, that we believe require continuing attention:

! While much improvement has been realized, there is a continuing need to improve the public's

understanding of all the elements of the Reactor Oversight Process. It appears that much of the

public continues to perceive the new Reactor Oversight Process as solely the "Performance

Indicators", and is less aware of the revised Inspection Process, SDPs, Action Matrix, and

Enforcement Policy. 

! The Commission staff has initiated an comprehensive effort to appropriately incorporate Safety

Culture into the Reactor Oversight Process. The issues identified to date are significant and will

require considerable deliberation and stakeholder involvement to ensure that a predictable and

"scruitable" process results. As this effort proceeds, it should include participation of the Reactor

Oversight Process stakeholders, and be subject to the same checks and balances as any other

Reactor Oversight Program changes.

Greenpeace Attachment

As a staff member I have heard repeatedly the ROP is a “better” process. I would agree that an action

matrix approach is an improvement over the SALP and is problem plant list. However, how has the

system that produces inputs to the action matrix been shown to be “better”? Yes, the reactor systems

related SDPs are “objective” because they rely on numbers but are they and the other SDPs producing

the right results? While the review at Davis[-]Besse is showing us that hard to quantify things like “safety
-66-

culture” are at the root of many of the issues there, we as an agency are putting more and more trust in

a small group of risk trained specialists using tools that don’t do well with such unquantifiable issues. Our

justification is that problems will manifest themselves as poor performance that we can quantify and we

will act on it before a real big problem occurs. Do the action matrix and the SDPs as presently

constructed do that? What metric (s) show that to be the case?

Below in no specific order are some areas in which the ROP could be improved:

3. Objective 02.03 of Manual Chapter 0305 states that the ROP assists management in making timely

decisions regarding appropriate agency actions to oversee, inspect and assist licensee performance. Is

the ROP as now implemented achieving the objective? Specifically, if the outputs of the ROP are in fact

suppose to determine agency response to events, how does the agency response to the Indian Point

tube failure and the head cracking at Davis[-]Besse, to only mention two events, demonstrated that

approach? In those events the NRC properly authorized and expended the resources necessary to

address the immediate response to the event. However, the NRC also committed to and took many

longer term follow up actions such as follow up team inspections long before either PI or SDP outputs

were completed which justified such expenditures. Is assessment driving response or vice versus? Does

the process as now constructed and/or implemented is too unwieldy or complex to meet the stated

objective?

5. The action matrix is an improvement over SALP. However, when a plant reaches the far right of the

action matrix or has a significant event the NRC still does not have any guidance for making a shutdown

decision, despite a longstanding recommendation from the GAO in that area. Recognizing that each

situation is unique, there can be no exact formula. However, (a) Finding #4 of OIG report 02-03S argues

for a better defined process. Despite having concurrence on an order from all necessary offices and

OGC, the Director NRR thought he lacked the regulatory authority to issue the order to Davis[-]Besse. It

would seem that an acceptable process would not leave a manager in such a situation but would lay out

what the concurrences meant and what methods were available to resolve the question of regulatory

authority and (b) without any guidance, the often criticized “show of hands” for Davis[-]Besse would be as

valid as any shutdown decision making process in existence. Does the NRC consider the decisionmaking process used to allow Davis[-]Besse continue to operate with head cracks “a good and

appropriate model for future actions” as advocated by Mr. Holahan in his1/14/03 note to the

Commission? If so, how does the NRC answer the critique of the process, in particular the validity of the

staff’s risk argument, Mr. Long offers in his 2/20/03 note to the Commission? If not in agreement with Mr.

Holahan’s assessment, what process will the NRC use when confronted with a similar situation in the

future?

8. Manual Chapter 0305 Section 0305-1 “Purpose” states that the ROP “integrates the NRC’s

inspection, assessment and enforcement programs.” Given that actions that receive only severity levels

under the Enforcement Policy do not appear in/affect the action matrix, how does the ROP in fact fully

integrate the enforcement program? Looking forward, if another plant has information accuracy problems

like Davis Besse and such problems are detected and dispositioned using enforcement policy, how will

inspection resource allocations, which are made through the ROP, be adjusted for these non ROP

concerns? Would the staff wait to increase the inspection resources until hardware issues manifest the

significance of the information accuracy problem?
-67-

NRC Response:

To address the comment regarding the exit strategy for plants in certain columns of the Action

Matrix, the ROP assessment process, as defined by IMC 0305, has prescriptive criteria for

determining when licensee performance warrants placing a plant in a column other than in the

licensee response column. Since performance is assessed every quarter based on the

significance of PIs and inspection findings, no NRC defined exit strategy is necessary or

warranted. It is the individual licensee's responsibility to adequately address performance

deficiencies including identifying root causes, extent of condition, and corrective actions. For

plants with significant performance problems who are in column 4 or 5 of the Action Matrix, the

ROP allows NRC some discretion and latitude when asasessing whether the licensee has

corrected the underlying conditions that led to the performance deficiencies and whether

actions and programs are in place to prevent recurrence. 

A comment was received about the performance levels within the ROP and the ability to

acheive "green performance." As discussed in Enclosure 1 to SECY 06-0074, when the PI

program was developed, the green-white thresholds were set using industry performance data

from 1995 to 1997 such that about 5 percent of the plants were expected to exceed the greenwhite threshold—that is, be designated either white, yellow, or red—for each PI in the mitigating

systems and initiating events cornerstones. Data from 1995 to 1997 were used to establish the

thresholds because they were the most recent data available and the NRC considered industry

performance in that period acceptable for the purposes of establishing ROP thresholds. The

comment stated dissatisfaction with most plants acheving "green performance" and suggested

that "the bar needs to be raised." A fundamental principle of the ROP is that it be objective. 

Therefore, licensee performance is generally judged against objective criteria. Adjusting

performance levels so that only 50 percent of plants acheive green performance would be

inconsistent with this principle.

Another comment questioned the large number of green PIs and the ability of PIs to

appropriately measure performance. An objective of the Industry Trends Program (ITP) is to

assess whether the nuclear industry is maintaining the safety performance of operating

reactors. To accomplish this goal, the ITP monitors trends in indicators of industry performance

as a means to confirm that the safety of operating power plants is being maintained. Should

any long term indicators show a statistically significant adverse trend, the NRC will evaluate

them and take appropriate regulatory action using its existing processes for resolving generic

issues and issuing generic communications. The NRC formally reviews these indicators as part

of the AARM each year, and any adverse trends are reported to Congress in the NRC's

Performance and Accountability Report. No statistically significant adverse trends have been

identified to date, based on level or declining long term trends in the indicators. The ITP

indicates that overall industry perfromance has significantly improved over the past 20 years.

A comment was received regarding variance in the enforcement actions involved with the

restart efforts for Millstone, D.C. Cook, and Davis-Besse. The comment referred to the "almost

pathetic enforcement applied to FENOC and Davis-Besse." In the past, a number of facilities

have been shutdown for prolonged periods to address regulatory concerns. If the NRC

considers a plant unsafe to operate, the NRC will take prompt action to ensure the plant does

not operate until the safety issues are addressed as directed by MD 8.3 entitled "NRC Incident

Investigation Program." For example, the NRC has taken significant enforcement action and

imposed civil penalties against FirstEnergy for the incomplete and inaccurate information
-68-

provided to the NRC in the fall of 2001 about the condition of the reactor vessel head at DavisBesse (reference EA-05-071). On April 21, 2005, a Notice of Violation and Proposed

Imposition of Civil Penalties in the amount of $5,450,000 was issued for multiple violations

(some willful) related to the significant degradation of the reactor pressure vessel head

identified in February and March 2002. The significant violations included: (1) operation with

reactor coolant system pressure boundary leakage (associated with a Red SDP finding,

$5,000,000), (2) failure to provide complete and accurate information (Severity Level I,

$110,000), (3) failure to promptly identify and correct a significant condition adverse to quality

(Severity Level II, $110,000), (4) failure to implement procedures (Severity Level II, $110,000),

(5) failure to provide complete and accurate information (Severity Level I, $120,000), (6) failure

to promptly identify and correct a significant condition adverse to quality (associated with a Red

SDP finding), (7) failure to implement procedures (associated with a Red SDP finding), and (8)

failure to provide complete and accurate information (Severity Level III).

Another comment was received about the integration of the enforcement program into the ROP. 

The assessment program currently does not directly use traditional enforcement items as an

input into the Action Matrix. The NRC staff uses traditional enforcement items during mid-cycle

and end-of-cycle assessment meetings as inputs into the determination of future inspection

plans and focus, as well as what actions the staff will take within an Action Matrix column. The

NRC staff is currently reviewing this issue to determine whether a change in the process is

warranted. 

A comment was received regarding the need to improve the public's understanding of the ROP. 

The NRC is aware of this concern and is contiually looking to improve its communication with

the public. The NRC staff is currently revising the ROP webpage to help improve the public's 

understanding of the ROP as well as revising reference documents about the program.

Other comments have been addressed in previous responses. Specifically, comments related

to the green/white threshold are addressed in response to question #7 and comments

regarding public understanding of the ROP are addressed in response to question #11. 

Other Section

Respondent Comments:

Greenpeace

This questionaire does to get to the heart of the issue. The NRC survey of the staff asked better, more

probing questions but I doubt the NRC actually wants to get the real answers. All you need to know is

that since the new ROP was implemented, the industry experienced the most significant accident

precursor since they melted down Three Mile Island! Was that NRC's intended result when it revised the

ROP? 

When the NRC first instituted the revised reactor oversight process, the staff was surveyed. The results,

as reported in Inside NRC, should have given the Commission cause for concern:

! 70% of those surveyed believed that the new process would not catch declining performance “before

a significant reduction in safety margins.”

! 70% of NRC’s resident inspectors believed that the new process “may not identify and halt degrading

performance.”
-69-

! 79% of NRC staff either had no opinion or believed that the new performance indicators did not

provide an adequate indication of declining performance.

! 75% of the NRC staff thought that the nuclear industry and NEI had too much influence and input into

the new process. 

Guess what? The NRC staff was right! 

The ROP is fundamentally flawed. It handcuffs the NRC staff and regulates them rather than regulating

those splitting atoms. It fails to identify and halt decling performace before it results in a significant

degradation of safety. The NRC may have reduced the regulatory burden on the nuclear industry but the

agency has failed to maintain safety and has further undermined the public’s confidence in the NRC as

an independent and unbiased regulator of the nuclear industry. 

 

Additionally, NRC should do some soul searching and ask itself why a member of its own staff 

had to submit ROP comments anonymously last year. I belive this fact alone speaks volumes about the

Commission and its senior management. It is ironic that the NRC is working so hard on defining

"adequate safety culture" for the industry when its own staff does not feel free to speak its mind. 

According to the Staffer:

"Whiile [While] I am an NRC staff member I am providing these comments: 1), anonymously, because

management statements to the effect that staffers who have criticisms of the ROP just need more risk

training sends a message, at least to me, that dissent is unwelcome, 2) publically - because the ROP

feedback process, while an adequate mechanism for fixing discrete procedural problems, has not been

demonstrated to be an appropriate forum with sufficient management involvement for discussion of more

programmatic issues and 3),now - because after four years, the process has been in place a sufficient

time to draw conclusions on its performance. Further, while there have been past efforts to address

issues with the ROP, they have not taken on/addressed some of the more substantial issues.

These comments represent the views of only one staffmember however, many of the issues

discussed below are concerns of other stafftmembers as well."

I have resubmitted the anonymous comments with this survey. Unfortunately they are still legitimate and

are still falling upon deaf ears at the NRC.

Greenpeace Attachment

These comments are provided for consideration as part of the NRC’s annual review of the Reactor

Oversight Process.

While I am an NRC staff member I am providing these comments: 1) anonymously, because

management statements to the effect that staffers who have criticisms of the ROP just need more risk

training sends a message, at least to me, that dissent is unwelcome, 2) publically - because the ROP

feedback process, while an adequate mechanism for fixing discrete procedural problems, has not been

demonstrated to be an appropriate forum with sufficient management involvement for discussion of more

programmatic issues and 3) now - because after fours years, the process has bee in place a sufficient

time to draw conclusions on it’s performance. Further, while there have been past efforts to address

issues with the ROP, they have not taken on/addressed some of the more substantial issues. 

These comments represent the views of only one staff member however, many of the issues discussed

below are concerns of other staff members as well. 
5

STARS is an alliance of six plants (eleven nuclear units) operated by TXU Power, AmerenUE, Wolf Creek

Nuclear Operating Corporation, Pacific Gas and Electric Company, STP Nuclear Operating Company and Arizona

Public Service Company.

-70-

It is understandable that the nuclear industry looks somewhat favorably on this relatively new process

because the action matrix make NRC actions more predictable, as presently implemented there are

fewer “significant” actions issued, for those that are issued there are extended opportunities to offer any

manner of outside the design basis risk arguments to downgrade the findings, and finally there are very

few civil penalties issued to power reactor licensees any more. For the public, the benefits of the new

process are less clear. The ROP does provide a predictable response but it has yet to demonstrate

measurable improvement over the old SALP system in detecting declining performance. It has also

introduced a whole new level of complexity that must be understood to have any confidence in what the

NRC is doing. Previously, a member of the public had to understand the technology of the plant and its

associated language and acronyms. Now on top of that, a member of the public has to acquire a working

knowledge of risk and its associated language/acronyms including when/how issues/events are

evaluated using Core Damage Frequency, Large Early Release Frequency, or Conditional Core Damage

Probability. 

Region IV Utility Group

! NRC has been open to comments and change when needed (exception Security).

Union of Concerned Scientists

No additional comments.

Nuclear Energy Institute

! NRC has been open to comments and change when needed (exception Security).

Strategic Teaming and Resource Sharing

Attached are comments from the Strategic Teaming and Resource Sharing (STARS)5 nuclear power

plants on the implementation of the Reactor Oversight Process (ROP). The STARS plants appreciate

this opportunity to provide comments on the ROP. The STARS plants have been working with NEI and

RUG IV in the development of industry comments. STARS endorses the comments submitted by NEI

and RUG IV.

Since implementation in April 2000, the ROP has exhibited marked improvement over the former

inspection and enforcement process. Subjecting the ROP to continuous improvement by way of the

routine ROP public meetings and the periodic solicitation of public feedback has assisted the ROP in

effectively meeting the intended objectives, i.e., to maintain reactor safety; to enhance public confidence;

to improve the effectiveness, efficiency, and realism of the oversight process; and to reduce

unnecessary regulatory burden. Reassessment of performance indicators and adopting more effective

indicators (e.g., Mitigating Systems Performance Index and Scrams with complications) is applauded.

STARS supports and looks forward to assisting in the continuing efforts to further develop and improve

the ROP.
-71-

Southern California Edison

SCE endorses the comments, provided separately, by the Nuclear Energy Institute (NEI). The following

SCE comments are provided to augment those of NEI, and include programmatic issues/comments we

have identified previously. 

There are several areas, however, that we believe require continuing attention:

! The opportunity to provide comments on the NRC's revised Reactor Oversight Process Program is

appropriate and appreciated. We recommend that the NRC staff provide formal, timely, and public

feedback on comments received from the external stakeholders.

SCE appreciates the opportunity to provide these comments to the Nuclear Regulatory Commission.

AmerGen & Exelon

EGC/AmerGen consider that the ROP meets its established performance goals. Furthermore, we fully

support the comments submitted by the Nuclear Energy Institute (NEI) on behalf of its member

companies regarding the implementation of the ROP.

**Due to the magnitude and complexity of this comment, it has been not been included in this section but

can be viewed in ADAMS (ML060250245) at the NRC public website. It is responded to as part of this

section.

NRC Response:

As noted in the staff’s response to question #12, the staff disagrees that it typically fails to

identify and halt declining performance before it results in a significant degradation of safety. 

However, the Davis-Besse event was a unique exception and resulted in a number of program

improvements designed to prevent recurrence. The potential “events” that actually have been

prevented by implementing the ROP cannot be directly measured by the simple fact that they

do not occur. However, one could argue that many of the NRC and licensee-identified findings

since the implementation of the ROP have possibly prevented significant degradation of safety.

Regarding the concern that an anonymous NRC staff member submitted comments in 2004,

the staff acknowledged the comments and noted its concern with this fact in SECY-04-0053

and considered those comments in performing its self-assessment. Nonetheless, a

consolidated response was not issued following that annual self-assessment so the anonymous

comments are being specifically addressed in the applicable portions of this response. For

example, the anonymous staff member’s comments regarding the complexity of the ROP are

addressed under question #12. 

The staff disagrees with the notion that external stakeholder feedback and the resultant NRC

analysis is biased towards industry representatives and State and local government agencies. 

As noted in SECY-06-0074, 9 of the 21 responses to the 2005 external survey were from

utilities, while 4 were from State agencies, 7 were from the public, and 1 was from an

anonymous respondent. Each of these responses was considered in performing the CY 2005

ROP self-assessment and are addressed in this consolidated response. 

Regarding the comments that the Commission routinely ignores state and local input when it

comes to taxation and staffing, these particular areas are outside of the NRC’s regulatory
-72-

purview and are not evaluated as part of the annual ROP self-assessment process. The staff

also disagrees with the statement that staffing levels and property values can not be ignored by

an agency which is charged with protecting the well being of reactor communities (assuming

that the respondent is referring to the NRC). The NRC’s primary mission is to license and

regulate the Nation's civilian use of byproduct, source, and special nuclear materials to ensure

adequate protection of public health and safety, promote the common defense and security,

and protect the environment. Additional information on the NRC’s mission and strategic goals

can be found in the FY 2004-2009 Strategic Plan (NUREG-1614, Vol. 3).

As recommended by one respondent, the NRC does perform a quality review of the ROP web

information prior to the external postings, via both an automated and manual process. 

However, there may still be infrequent instances where disconnects occur between inspection

report data and the PIM. If and when these are detected they are corrected in a timely manner. 

We encourage anyone who identifies broken links or other website errors to send an email to

NRRwebservices@nrc.gov with a description of your concern.

The NRC’s responses to questions #13 and #14 address the security aspect of the

respondents’ comments.

END