CONSOLIDATED RESPONSE TO THE

2006 REACTOR OVERSIGHT PROCESS EXTERNAL SURVEY

This document contains the consolidated results of the Federal Register notice (FRN) that

solicited external stakeholder comment and feedback on the Reactor Oversight Process (ROP).

The FRN, entitled “Solicitation of Public Comments on the Implementation of the Reactor

Oversight Process,” was published on October 10, 2006 (Agencywide Documents Access and

Management System (ADAMS) Accession Number ML052860373). This notice was part of an

ongoing effort by the staff to obtain external stakeholder input regarding the efficacy of the

ROP. The comment period expired on December 1, 2006.

In an effort to actively solicit feedback on the implementation of the ROP, the Nuclear

Regulatory Commission (NRC) staff: (1) mailed approximately 700 surveys directly to

stakeholders, (2) placed a direct link to the survey information on both the ROP Web page and

the “Documents for Comment” page of the NRC’s external Web site, and (3) issued a press

release and posted it on the NRC’s external Web site.

Following the positive feedback and success of the 2004 and 2005 consolidated responses, the

staff has developed a response to the comments received during the 2006 survey. This

includes consolidating the comments by survey question and providing a response categorized

by survey question. The respondent comments for each question are listed in chronological

order as received and the responses are provided in no particular order.

The questions used in the questionnaire were developed for the staff to gain specific feedback

regarding the ROP’s performance metrics as described in Inspection Manual Chapter (IMC)

0307, “Reactor Oversight Process Self-Assessment Program.” This allows the staff to assess

whether the metrics are meeting the required criteria.

For those respondents who made general comments that were not directed to a specific

question, the comments were listed as a response to question #22 (additional information or

comments on other program areas related to the ROP). In order to provide a more concise

response to comments in this category, those comments were grouped and addressed by the

appropriate ROP program areas of performance indicators, inspection, significance

determination process, and assessment. Comments that were outside of these areas are

addressed under the category of other.

The staff attempted to represent stakeholder comments in this document exactly as they were

received; therefore, no corrections were made for spelling or grammatical errors on the FRN

responses (but corrections have been included in parenthesis for clarification purposes as

needed). The Accession numbers from ADAMS after each respondent has also been provided

for access to the official record copy of the specific FRN response.
-2-

Comments were received from the following respondents (listed in chronological order as

received).

• Stanley Gamble, Exelon Limerick Generating Station (ML063000055)

• Raymond Tierney, Private Citizen (ML063070324)

• T. Gurdziel, Private Citizen (ML063130355)

• George Vargo, Private Citizen (ML063200031)

• James Sniezek, Utility Consultant (ML063240279)

• Union of Concerned Scientists (ML063340400)

• Pennsylvania Department of Environmental Protection (ML063260281)

• South Texas Nuclear Operating Company (ML063340361)

• Farouk Baxter, Private Citizen (ML063340365)

• New Jersey Division of Environmental Safety and Health (ML063400352)

• Nuclear Energy Institute (ML063390650)

• Region IV Utility Group (ML063400360)

• Florida Power and Light (ML063400368)

• Southern California Edison (ML063470342)

• Slovenian Nuclear Safety Administration (ML063560068)

• Strategic Teaming and Resource Sharing (ML063560024)

The level of participation (16 responses) was down from previous years (21 responses in each

of the previous two years). Approximately twelve of the respondents answered the survey

questions, while four of those that responded provided only comments. Half of the 16

responses came from the Nuclear Energy Institute (NEI) or utility representatives, two came

from State agencies, five came from public interest groups or members of the public, and one

came from an international regulator.

Each question number includes all comments received followed by the NRC’s response to

those comments. The Table of Contents on the following page (pages 4-6) can be used to find

the comments and responses to specific survey questions. To the extent practicable, we

reference the relevant portions of the annual staff paper to the Commission (SECY-06-0074)

and the annual ROP metric report to demonstrate how the staff addressed the comments. 

These documents are available on the external ROP Web page and can also be obtained in the

NRC document management system (ADAMS). The document accession numbers are

ML070730517 and ML070720085 respectively.

In some cases the staff plans to consider the specific comments and suggested improvements

in future revisions to program guidance. Accordingly, some issues will be entered into the ROP

issue tracking system as feedback forms in accordance with IMC 0801, “Reactor Oversight

Process Feedback Program,” to ensure that these issues are considered and tracked to

resolution.

This consolidated response, along with the Commission paper and the annual ROP

performance metric report, will be posted to the ROP Web page and sent to each respondent to

the survey.
-3-

Table of Contents

1. Does the Performance Indicator Program provide useful insights to help ensure plant

safety?

Respondent Comments: Page 6

NRC Response: Page 7

2. Does appropriate overlap exist between the Performance Indicator Program and the

Inspection Program?

Respondent Comments: Page 9

NRC Response: Page 9

3. Does NEI 99-02, “Regulatory Assessment Performance Indicator Guideline” provide

clear guidance regarding Performance Indicators?

Respondent Comments: Page 11

NRC Response: Page 11

4. Does the Performance Indicator Program, including the Mitigating Systems Performance

Index, effectively identify performance outliers based on risk-informed, objective, and

predictable indicators?

Respondent Comments: Page 12

NRC Response: Page 13

5. Does the Inspection Program adequately cover areas important to safety, and is it

effective in identifying and ensuring the prompt correction of performance deficiencies?

Respondent Comments: Page 14

NRC Response: Page 16

6. Is the information contained in inspection reports relevant, useful, and written in plain

English?

Respondent Comments: Page 19

NRC Response: Page 20

7. Does the Significance Determination Process yield an appropriate and consistent

regulatory response across all ROP cornerstones?

Respondent Comments: Page 22

NRC Response: Page 23

8. Does the NRC take appropriate actions to address performance issues for those plants

outside of the License Response Column of the Action Matrix?

Respondent Comments: Page 26

NRC Response: Page 27

9. Is the information contained in assessment reports relevant, useful and written in plain

English?

Respondent Comments: Page 28

NRC Response: Page 29
-4-

10. Are the ROP oversight activities predictable (i.e., controlled by the process) and

reasonably objective (i.e., based on supported facts, rather than relying on subjective

judgement)?

Respondent Comments: Page 30

NRC Response: Page 31

11. Is the ROP risk-informed, in that the NRC’s actions are graduated on the basis of

increased significance?

Respondent Comments: Page 33

NRC Response: Page 33

12. Is the ROP understandable and are the processes, procedures and products clear and

written in plain English?

Respondent Comments: Page 34

NRC Response: Page 35

13. Does the ROP provide adequate regulatory assurance, when combined with other NRC

regulatory processes, that plants are being operated and maintained safely?

Respondent Comments: Page 36

NRC Response: Page 37

14. Do the ROP safety culture enhancements help identify licensee safety culture

weaknesses and focus licensee and NRC attention appropriately?

Respondent Comments: Page 38

NRC Response: Page 39

15. Is the ROP effective, efficient, realistic, and timely?

Respondent Comments: Page 41

NRC Response: Page 42

16. Does the ROP ensure openness in the regulatory process?

Respondent Comments: Page 43

NRC Response: Page 44

17. Has the public been afforded adequate opportunity to participate in the ROP and to

provide inputs and comments?

Respondent Comments: Page 45

NRC Response: Page 45

18. Has the NRC been responsive to public inputs and comments on the ROP?

Respondent Comments: Page 46

NRC Response: Page 47

19. Has the NRC implemented the ROP as defined by program documents?

Respondent Comments: Page 47

NRC Response: Page 48

20. Does the ROP minimize unintended consequences?

Respondent Comments: Page 48

NRC Response: Page 49
-5-

21. Would you support a change in frequency of the ROP external survey from annually to

every other year, consistent with the internal survey, as proposed in SECY-06-0074?

Respondent Comments: Page 49

NRC Response: Page 50

22. Please provide any additional information related to the Reactor Oversight Process

Performance Indicator Section

Respondent Comments: Page 51

NRC Response: Page 51

Inspection Section

Respondent Comments: Page 52

NRC Response: Page 54

Significance Determination Process Section

Respondent Comments: Page 55

NRC Response: Page 57

Assessment Section

Respondent Comments: Page 59

NRC Response: Page 60

Other Section

Respondent Comments: Page 60

NRC Response: Page 64
-6-

1. Does the Performance Indicator Program provide useful insights to help ensure

plant safety?

Respondent Comments:

Union of Concerned Scientists

The Performance Indicator Program initially provided useful insights, but the NRC caved in to

industry pressure and made ROP program changes that have rendered the Performance

Indicators as useful as an appendix on a mannequin. There are countless examples of the

utter uselessness of the current Performance Indicators, including but not limited to:

a. Alert and Notification System - The purpose of this Performance Indicator, according to

the NRC website, is to monitor the "Availability of notification system for area residents."

By anyone's measure, the Indian Point nuclear plant has had the worst performing

sirens in the nation. Yet the Performance Indicators for Indian Point Units 2 and 3 are

Green, as are the Alert and Notification System Performance Indicators for every

reactor in the country (except for the unique situation at Hatch). Both the Indian Point

licensee and the NRC have expended considerable resources on the siren problems at

Indian Point - suggesting a siren performance indicative of a White level rather than a

Green level. As with the majority of the other Performance Indicators, the Alert and

Notification System Performance Indicator looks at bogus data that bears little or no

relationship with underlying performance levels. From the get go, UCS has maintained

that the Alert and Notification System tracked meaningless data. Indian Point, and

others (e.g., Davis-Besse) have proven us correct. The Alert and Notification System

Performance Indicator should be revised into something meaningful, or the Green color

redefined to stand for what it currently represents, "Goofy."

b. GAO's September 2006 report titled "NUCLEAR REGULATORY COMMISSION:

Oversight of Nuclear Power Plant Safety Has Improved, but Refinements Are Needed"

stated: "In the performance indicator program, almost all of the indicator data were

reported to be within the acceptable levels of performance (green). Only 156, or less

than 1 percent, of over 30,000 indicator reports from 2001 to 2005, did not meet the

acceptable performance threshold." Thus, the performance indicators that initially had

green/white thresholds established based on 95/5 performance results, have been

watered down and diluted to the point where they provide meaningless insights nearly

100 percent of the time.

Pennsylvania Department of Environmental Protection

It is not clear as to whether the existing PIs and their associated thresholds can provide useful

insights to help ensure "plant safety". The basis for setting the existing thresholds are

inconsistent; some are based on PRAs and others are based on regulatory requirements or

technical specification limits. Therefore, some PIs and their associated thresholds do not

directly correlate with risk. Additionally, the PI Program might not be working as originally

envisioned because the great majority of PIs have remained consistently green and have not

changed color (have not crossed green/white threshold). It is possible that the thresholds for

certain PIs are not set at the proper level. NRC should review the effectiveness of the PI

Program and should also evaluate the need for revisions to the existing thresholds.
-7-

South Texas Project Nuclear Operating Company

The performance indicators have developed into performance standards that the industry

strives to meet. Since the performance indicators are based on NRC defined acceptable limits

they reinforce industry and licensee safety performance.

New Jersey Division of Environmental Safety and Health

• On one hand the performance indicators set minimum thresholds for important areas of

nuclear operation but on the other hand having all the performance indicators green

raises doubt about the thresholds.

• The emergency preparedness performance indicators do not provide a comprehensive

indicator which would result in public safety.

• When performance indicator thresholds are exceeded, licensees are motivated to

perform well to remove the color change. The licensee argues that the thresholds may

have been exceeded but in many cases “extenuating circumstances” changes the color

back to green.

• We recommend that the NRC take a stronger position when any performance indicator

threshold is crossed.

Nuclear Energy Institute

The ROP strives to provide an objective assessment of licensee safety performance using, in

part, the Performance Indicator Program. The Performance Indicator Program provides useful

insights, identifies areas requiring increased focus and provides an objective assessment of

licensee safety performance. The Performance Indicator Program, in conjunction with the

Inspection Program, helps ensure plant safety.

Region IV Utility Group

The performance indicators have developed into performance standards that the industry

strives to meet. Since the performance indicators are based on NRC defined acceptable limits,

they reinforce industry and licensee safety performance. Implementation of MSPI is considered

an enhancement by adding a more risk-informed performance indicator to the PI program.

Strategic Teaming and Resource Sharing

The performance indicators have developed into performance standards that the industry

strives to meet. Since the performance indicators are based on NRC defined acceptable limits,

they reinforce industry and licensee safety performance.

NRC Response:

As noted in the ROP self-assessment and metric report for PI-4, “PI Program Provides Insights

to Help Ensure Plant Safety,” responses to the external survey question show a wide

divergence of opinion. Responses from the public ranged from strongly agree to strongly

disagree whereas the feedback from the states was neutral. Industry responses were generally

in agreement that the PI program does provide useful insights to ensure plant safety. As a

result, this metric did not meet its criteria because internal, public, and state respondents gave

significant feedback that the PIs do not provide an adequate indication of declining safety

performance and do not enhance public confidence. As noted in last year’s self assessment,

the staff recognizes the need to improve the PI Program to provide more meaningful indications

of declining plant performance.

As noted in Enclosure 1 to SECY-07-0069, the staff acknowledged that many stakeholders

believed that the emergency preparedness PIs, and the ANS PI in particular, provide
-8-

inadequate indication of declining or deficient performance and noted that the staff plans to

discuss these PI concerns with the industry during ROP monthly meetings in CY 2007.

As discussed in Enclosure 1 to SECY-06-0074 and reemphasized in Enclosure 1 to

SECY-07-0069, the staff recognizes that the percentage of greater-than-green PIs has declined

over the years. Although the PI program continues to provide the NRC with objective indicators

regarding plant performance, and in some areas has focused licensee attention and contributed

to improved performance, the staff and some public stakeholders remain concerned with the

capability of the current PIs to contribute to the identification of declining performance. As a

result, the staff is in the process of revising several PIs. In fact, Enclosure 1 to SECY-07-0069

noted that tabulation of industry MSPI data over the first three quarters of implementation

revealed an increase in the number of white PIs reported with MSPI when compared to its

predecessor, the Safety System Unavailability (SSU) PI. The staff plans to continue to work

with the industry to revise and/or introduce other PIs to improve the program’s effectiveness in

contributing to the identification of declining performance.

Regarding the inconsistency in basis for setting the PI thresholds, the staff acknowledges that

some are more deterministic than others and are not directly correlated with risk. The areas

outside of the reactor safety cornerstones (i.e., emergency preparedness, radiation safety, and

security) are not conducive to formal risk analysis and are not directly incorporated into licensee

PRAs. We attempt to incorporate risk insights to the extent practicable along with the other

ROP goals of being objective, understandable, and predictable as well as the three applicable

performance goals listed in the NRC’s Strategic Plan (ensuring safety, openness, and

effectiveness). Greater detail on the basis for the PIs and their thresholds can be found in

IMC 0308, Attachment 1, “Technical Basis for Performance Indicators.”

Regarding the comment that the NRC should take a stronger position when any PI threshold is

crossed, the NRC’s response to crossed PI thresholds is objectively determined by the relative

significance, and in concert with greater-than-green inspection findings, as noted in the NRC’s

Action Matrix per IMC 0305, “Operating reactor Assessment Program.” A greater-than-green PI

is equally weighted and treated the same as a greater-than-green inspection finding in the

NRC’s Action Matrix. Since the inception of the ROP, it has been a basic tenet to not double

count events/findings in the Action Matrix, when there are both a PI threshold crossing and a

corresponding inspection finding. However, the greater significance of the two inputs would be

the input used for assessment in the Action Matrix. Deviations from this policy are expected to

be rare, as seven years of ROP implementation have shown. Additionally, with the

development and implementation of the relatively new MSPI, given that this is the first set of PIs

that are risk-informed, there will be instances where MSPI inputs and inspection findings on the

same system will both be counted in the Action Matrix, because the two processes are

fundamentally different in concept, thus have different meanings and each should stand on their

own merit.

Regarding the comment that in many cases “extenuating circumstances” change the PI color

back to green, the NRC has built into the process a means for ensuring that corrective actions

have been adequately addressed to prevent recurrence even if the PI quickly changes back to

green. In those cases where a supplemental inspection is performed for a greater-than-green

PI and the NRC is not satisfied with the licensee’s corrective actions, the NRC may open a

parallel inspection finding to track the issue until they are satisfied that the performance

deficiency has been adequately addressed. More guidance and background on this process

can be found in IMC 0305, paragraphs 04.12 and 06.06.d.
-9-

2. Does appropriate overlap exist between the Performance Indicator Program and

the Inspection Program?

Respondent Comments:

Union of Concerned Scientists

The lack of overlap is good in this case. The Performance Indicator Program is a useless

waste of space and time. The Inspection Program, fortunately, has some redeeming value.

UCS hopes that the NRC will not allow the nuclear industry to eviscerate the inspection

program as it has the Performance Indicator Program.

Pennsylvania Department of Environmental Protection

Overall, there is appropriate overlap between the PI Program and the Inspection Program.

However, it is recommended that the NRC periodically reexamine the Baseline Inspection

Program and the PI Program to ensure proper focus and effectiveness. NRC should also

consider developing supplemental inspections for those areas that PIs are not very effective.

South Texas Project Nuclear Operating Company

Performance Indicators look at the areas where clear performance thresholds can be

developed. This allows the inspection program to spend more time looking at those areas that

require evaluation and investigation. The process is well integrated and, while overlap exists,

the overlap seems appropriate.

Nuclear Energy Institute

Performance Indicators look at areas where clear performance thresholds can be developed.

This allows the inspection program resources to be more appropriately allocated in areas that

are best addressed through evaluation and inspection. While overlap exists, the overlap

generally seems appropriate.

One area of note and possible attention is instances where inspection findings are documented

for issues/items that are currently being addressed through performance indicators.

Region IV Utility Group

Performance indicators look at the areas where clear performance thresholds have been

developed. This allows the inspection program to spend more time looking at those areas that

require evaluation and investigation. The process is well integrated and, while overlap exists,

the overlap seems appropriate. There appears to be more overlap in the security area between

performance indicators and inspections than in the other areas.

Strategic Teaming and Resource Sharing

Performance Indicators look at the areas where clear performance thresholds can be

developed. This allows the inspection program to spend more time looking at those areas that

require evaluation and investigation. The process is well integrated and, while overlap exists,

the overlap seems appropriate.

NRC Response:

As noted in the ROP self-assessment and metric report for PI-6, “Stakeholders Perceive

Appropriate Overlap Between the PI Program and Inspection Program,” public response varied

from agreement to disagreement regarding proper overlap between the PI and inspection
-10-

programs. However public response again noted that the PIs are not effective. State response

indicated that there is appropriate overlap but the NRC should periodically re-evaluate the PI

and inspection programs to ensure proper focus and effectiveness, and that where PIs are not

effective, inspection should be performed. Industry comments noted appropriate overlap

overall, but added that improvement can be made in the security area and that the NRC should

evaluate inspection findings that overlap PIs.

As noted in Attachment 1 to SECY-07-0069, the staff’s annual evaluation of the inspection

program indicated that the inspection program verified that plants were operated safely,

appropriately identified performance issues, and ensured the adequacy of licensee corrective

actions to address the noted performance issues. The staff also refined and formalized the

process to realign inspection resources in CY 2006. In CY 2007, inspection program staff and

regional representatives completed their review of all inspection procedures in the baseline

inspection program to evaluate the effectiveness of each inspection procedure in identification

of performance deficiencies and whether the inspection resources required to complete these

inspection activities were accurate. This effort, referred to as ROP realignment, is an attempt

to improve the inspectors’ ability to identify risk significant licensee performance deficiencies by

ensuring that appropriate inspection resources are focused on areas of significance. Inspection

program staff with help from regional representatives have reviewed the past three years of

inspection findings associated with each inspection procedure as well as other inspectionrelated materials (e.g., inspection findings resulting from Special Inspections, relevant operating

experience information, and feedback forms received from the regions) to determine whether

appropriate inspection resources were applied in each of the inspectible areas. Based on these

reviews, NRR and regional staff have made recommendations to improve the baseline

inspection program to both regional and NRR management. Inspection program staff plans to

obtain regional and NRR management approval on these recommendations and to revise and

issue the affected inspection procedures before the beginning of CY 2008. Many of these

recommendations to the inspection procedures (IPs) addressed issues such as accuracy of

inspection effort required to complete the IPs and strategies to improve identification of

inspection findings using some of our IPs.

Regarding the comment that there appears to be more overlap in the security area between

performance indicators and inspections than in the other areas, the staff recognizes this

concern and is considering options for reevaluating and enhancing the security PIs, including

working with the industry to develop more effective PIs. In addition, the Office of Nuclear

Security and Incident Response (NSIR) plans to perform a similar type of realignment review as

the one conducted by NRR during CY 2007. This review will evaluate the effectiveness of the

security baseline inspection procedures and help determine the appropriate scope and level of

effort of security inspections.

As mentioned in Enclosure 1 to SECY-07-0069, the staff continues to evaluate several PIs, with

inputs from internal and external stakeholders, in an effort to improve their effectiveness at

identifying poor performance. However, the staff generally does not believe that there is

excessive overlap between the PIs and inspection findings because they are measuring

different aspects of performance. The PI program monitors those events and/or conditions that

can provide meaningful performance data through counts of events or conditions. The PIs,

then, identify declining performance based on the number of counts in a given area. The

inspection program, however, looks at the safety significance of each individual performance

deficiency, including licensee actions. The two programs work together to monitor the safety of

the plant.
-11-

3. Does NEI 99-02, “Regulatory Assessment Performance Indicator Guideline”

provide clear guidance regarding Performance Indicators?

Respondent Comments:

Union of Concerned Scientists

It is abundantly clear from NEI 99-02 how the industry whittled what had been meaningful

Performance Indicators into their present mockery of regulatory processes.

Pennsylvania Department of Environmental Protection

The existing guidance document appears to be helpful in defining the PIs. However, it would be

more appropriate for the industry to comment on the effectiveness of this document.

South Texas Project Nuclear Operating Company

While questions on the guidance do arise, the FAQ process is very responsive to those

questions and the guidance is enhanced periodically based on the FAQs to improve the

guidance in an ongoing process.

Nuclear Energy Institute

Although NEI 99-02 questions do arise, the FAQ process is responsive in addressing those

questions. In addition, the guidance is periodically updated to incorporate the FAQ answers.

Efforts to incorporate FAQs into NEI 99-02 in a timely manner should continue.

Region IV Utility Group

While questions on the guidance do arise, the FAQ process is responsive to those questions.

The guidance is then updated periodically based on the FAQs to enhance the guidance in an

ongoing process.

Strategic Teaming and Resource Sharing

While questions on the guidance do arise, the FAQ process is responsive to those questions.

Periodic updates based on the FAQs are incorporated to enhance the guidance in an ongoing

process.

NRC Response:

As noted in the ROP self-assessment and metric report for PI-7, “Clarity of Performance

Indicator Guidance,” public response varied widely and comments again noted the

ineffectiveness of the PI program which resulted in the PI-4 metric not being met. State

stakeholders generally felt that the guidance was clear but that it would be more appropriate for

the licensees to provide comments on effectiveness of the PI guidance. Utility group

respondents commented that the PI guidance is clear and that the FAQ process is responsive

in addressing questions.

As discussed in Enclosure 1 to SECY-07-0069 and previously noted in SECY-06-0074, the staff

continues to believe that NEI 99-02, the PI guidance document, can be improved. This

document has been revised four times since ROP implementation, mostly to incorporate new

guidance in response to more than 425 frequently asked questions (FAQs) from licensees. In

some cases, the lack of clear, concise guidance has contributed to timeliness and efficiency

problems. To address this concern, the staff will continue to work with industry to clarify the

guidance in NEI 99-02.
-12-

4. Does the Performance Indicator Program, including the Mitigating Systems

Performance Index, effectively identify performance outliers based on riskinformed, objective, and predictable indicators?

Respondent Comments:

Union of Concerned Scientists

The Performance Indicator Program can't identify squat. The Mitigating Systems Performance

Index (MSPI) is a joke, a fraud perpetuated on the American public by a misguided regulator.

The only redeeming value from MSPI is that it fills a gap -we now have a tool to measure PRA

quality. Any licensee who cannot manipulate the data to have MSPI spit out green findings all

the time doesn't understand PRA very well. Thus, non-green MSPIs don't say anything about

system performance levels, but they do identify PRA-challenged licensees.

See the comments for Question (12) below for additional discussion of the MSPI evils.

Pennsylvania Department of Environmental Protection

See the previous comments regarding the effectiveness of the PI Program. The new MSPI is a

positive development because it is risk-based and it is the integration of both unavailability and

unreliability of systems. However, MSPI is complex and relies heavily on plant specific PRAs,

which vary in quality.

South Texas Project Nuclear Operating Company

The MSPI is the first risk-based indicator and does identify conditions based on risk

implications. While the other indicators have some limited risk insights they do not all identify

conditions that are risk significant. The industry and NRC staff should endeavor to develop

more risk based indicators.

New Jersey Division of Environmental Safety and Health

There are uncertainty bounds with performance indicator thresholds. It is not clear that all

outliers are identified unless the uncertainty is quantified.

Nuclear Energy Institute

The MSPI does identify conditions based on risk implications. While the other indicators have

some limited risk insights they do not all identify conditions that are risk significant. The

industry and NRC staff should endeavor to develop more risk-based indicators. A concern

exists that the value of the MSPI could be undermined if the NRC continues to provide

additional levels of review on equipment failures and/or unavailability even though performance

is within the licensee response band.

Region IV Utility Group

Although still newly implemented, the MSPI appears to be a good risk-based/informed indicator

and does identify conditions based on risk implications for the systems monitored. Other

indicators may falsely indicate conditions as risk significant when they are not, because of the

limited risk insights in the PI. The industry and NRC staff should endeavor to develop more risk

informed elements for other existing indicators.

Strategic Teaming and Resource Sharing

The MSPI is the first risk-based indicator and does identify conditions based on risk

implications. Because the other indicators have limited risk insights, they may inaccurately
-13-

identify risk significant conditions. The industry and NRC staff should continue to risk inform

the remaining indicators.

NRC Response:

As noted in the ROP self-assessment and metric report for PI-8, “PI Program Identifies

Performance Outliers In an Objective and Predictable Manner,” comments from the public

clearly state that the PI program cannot identify outliers, especially the MSPI indicator because

it allegedly can be manipulated and is difficult to understand. State stakeholders noted that the

PI program is always green and, as noted earlier, the thresholds may need reexamination. 

They noted, however, that the MSPI was a positive step being risk based and incorporating

unavailability and unreliability, but is dependent on the quality of the licensee’s PRA. Industry

comments were favorable about the PI program and in particular regarding MSPI because it is

risk based. As a result, this metric did not meet its criteria because a significant number of the

internal survey respondents along with several public and state stakeholders noted that the PI

program does not adequately identify outliers. The staff is in the process of reviewing and

revising several PIs to provide more meaningful indications of plant performance and to better

identify performance outliers, including unplanned scrams with loss of normal heat removal,

reactor coolant system leakage, safety system functional failures, and others in the emergency

preparedness cornerstone.

Regarding the MSPI specifically, as noted in Enclosure 1 to SECY-07-0069, a preliminary

assessment of the effectiveness of MSPI has confirmed that the definition of component

failures and the use of probabilistic risk assessment (PRA) are critical elements of MSPI. A

preliminary review of four quarters of submitted quarterly MSPI data indicates that MSPI has

been as effective as predicted during the pilot study, but the staff continues to be challenged in

resolving a number of complex and controversial FAQs, that may require several significant

changes to the guidance. The staff plans to monitor MSPI over the course of CY 2007 and

2008, will continue to engage the industry through the monthly ROP public meetings, and make

any necessary changes to the MSPI based on lessons learned.

The staff does not currently plan to develop additional risk-based PIs, but would be willing to

discuss and consider any specific proposals to further risk-inform the PI program.

Regarding uncertainty bounds with performance indicators, PIs are numerical counts of events

or conditions in the plant. There are no uncertainty bounds; for each PI a plant either exceeds

the threshold or it does not. The MSPI doesn't have uncertainty issues with its risk-informed

threshold, other than the PRA accuracy issues associated with the MSPI algorithm which are

being evaluated. 

Regarding the comment that the NRC applies additional levels of review on equipment failures

and/or unavailability even though performance is within the licensee response band, a feature

of the baseline inspection program is to review equipment failures and/or unavailability even

when the licensee is operating within the licensee response band. The baseline program has

four parts: (1) inspection in inspectable areas in which PIs are not identified and/or in which PIs

do not fully cover the inspectable area; (2) ongoing verification of the information provided in

PIs; (3) comprehensive review of licensee effectiveness in identifying and resolving problems,

and (4) initial follow up to plant events and degraded conditions to determine their safety

significance.
-14-

5. Does the Inspection Program adequately cover areas important to safety, and is it

effective in identifying and ensuring the prompt correction of performance

deficiencies?

Respondent Comments:

Union of Concerned Scientists

UCS's recent report, "Walking a Nuclear Tightrope: Unlearned Lessons of Year plus Reactor

Outages," concluded that the root cause of the majority of year-plus reactor outages was a

failure to comply with the quality assurance regulations embodied in Appendix B to 10 CFR

Part 50 coupled with the untimely detection of violations and/or ineffective enforcement of the

regulations. Since the first such year-plus outage in 1973, the NRC's oversight program has

undergone numerous revisions, but regulatory oversight of the quality assurance regulations

remains ineffective. The Problem Identification and Resolution (PI&R) inspection (IP 71152)

and the Appendix B - oriented portions of numerous other inspection procedures is simply

broken. Each NRC inspection finding is prima facie evidence that the corrective action program

at the site failed. Instead of focusing on whether licensees fixed broken widgets identified by

NRC inspectors, IP 71152 should delve into the licensee's testing and inspection scope and

frequency to ascertain why they failed to figure out the widgets were broken before the NRC

inspectors came by. In other words, IP 71152 has to - repeat, has to - place far greater

emphasis on the first have [half] of the PI&R equation; problem identification. IP 71152 has to

figure out why the licensee failed to identify problems found by NRC inspectors.

Pennsylvania Department of Environmental Protection

The NRC Inspection Program is intended to cover areas that are important to safety, but there

are opportunities for further improvements.

1. There is extensive use of resources on the part of the NRC staff and the licensees to

assess the significance of inspection findings, specifically for greater than “Green”

findings. This also applies to those findings that are not of safety significance.

2. The number of findings in the cross-cutting areas (human performance, safety culture

and problem Identification and resolution) is relatively high. Additionally, there are some

plants that are experiencing relatively high number of "substantive" cross-cutting issues.

The effectiveness of the ROP Inspection Program as it relates to the identification and

resolution of cross-cutting issues is subject to further review.

3. The role of the NRC in situations that do not involve regulations, but might involve a

performance deficiency should be examined and better defined.

4. The NRC should consider more frequent inspection or verification of the licensees'

Corrective Action Program (CAP). This is important considering that the ROP relies

heavily on the CAP for timely resolution of issues or problems. Additionally, it is clear

that the effectiveness of the CAP varies significantly within the industry and some

utilities are not very effective in this area.

T. Gurdziel

Without a strong Enforcement Program, (which does not presently exist, in my opinion), the

Inspection Program cannot ensure correction.
-15-

South Texas Project Nuclear Operating Company

The inspection program does adequately cover areas important to safety and does identify the

issues that require prompt correction. However the process also identifies performance

deficiencies that do not require prompt correction since they are not risk significant, and the

inspectors are identifying and documenting issues that are not risk significant. The NRC should

be watchful about regulating excellence instead of compliance.

New Jersey Division of Environmental Safety and Health

• The ROP inspection program is based on plant probabilistic risk assessments (PRA),

which have never been reviewed and approved by the NRC. The NRC seeks to assure

PRA adequacy in its PRA quality program but the only way to insure that PRAs are

accurate, reliable, and current is to review them from the ground up and require that

they be updated every 5 years. Many examples exist that continue to question the

inspection program. These include the Davis-Besse event, Safety Conscious Work

Environment concerns, emergency preparedness issues, and security concerns.

• From the time an NRC finding is identified until the time it is corrected and verified by

the NRC could take between one and two years – this is not prompt.

• An NRC inspection finding, when identified, becomes so narrow in scope that the final

assessment can become an argument over semantics between the NRC and the

licensee.

• Inspectors have much less latitude to explore safety issues and much less authority

under this inspection program. Again, unless the inspection finding is greater than

green, the NRC cannot influence trends in the licensee’s performance.

Nuclear Energy Institute

While the inspection program adequately covers areas important to safety and identifies the

issues that require prompt correction, the process also identifies issues that are not risk

significant. Periodic reviews of the overall effectiveness of certain inspection modules should

be performed. The reviews should consider the resources spent on the inspection against the

numbers, and level of significance of, findings from the previous performances of the

inspections covered under certain inspection modules. These reviews could be used to

reallocate inspection resources to areas of greater risk significance.

Region IV Utility Group

In particular, resident inspectors help ensure areas important to safety are appropriately

addressed. The NRC should consider enhancing the use of generic communications for

inspection trends. A process is needed to ensure early stakeholder involvement in the

identification and resolution of inspection issues that potentially have generic implications

(a process similar to the ROP PI FAQ process would seem appropriate). Examples include

manual actions for response to fires, assessment of post-fire safe shut down equipment, and

technical questions identified during inspections that involve development of new regulatory

positions. Enhanced use of generic communications would also promote consistency between

the NRC regions.

Strategic Teaming and Resource Sharing

In particular, the resident inspectors ensure areas important to safety are appropriately

addressed. The NRC should consider enhancing the use of generic communications for

inspection trends. Examples include manual actions for response to fires, assessment of

post-fire safe shut down equipment, and technical questions identified during inspections that

involve development of new regulatory positions.
-16-

Enhanced use of generic communications would also promote consistency between the NRC

regions.

NRC Response:

As noted in the ROP self-assessment and metric report for IP-9, “Inspection Program

Effectiveness and Adequacy in Covering Areas Important to Safety,” the responses to whether

the inspection program adequately covers areas that are important to safety and is effective in

identifying and ensuring the prompt correction of performance deficiencies were generally

positive with comments for making improvements.

Regarding the comment that periodic reviews of the overall effectiveness of certain inspection

modules should be performed, as noted in Enclosure 1 to SECY-07-0069, the staff refined and

formalized the process to realign inspection resources in CY 2006 to include consideration of

industry performance. As previously noted in the response to question #2, in CY 2007,

inspection program staff and regional representatives completed their review of all inspection

procedures in the baseline inspection program to evaluate the effectiveness of each inspection

procedure to identify performance deficiencies and whether the inspection resources required

to complete these inspection activities were accurate. This effort, referred to as ROP

realignment, is an attempt to improve the inspectors’ ability to identify risk significant licensee

performance deficiencies by ensuring that appropriate inspection resources are focused on

areas of significance. Inspection program staff with help from regional representatives have

reviewed the past three years of inspection findings associated with each inspection procedure

as well as other inspection-related materials (e.g., inspection findings resulting from Special

Inspections, relevant operating experience information, and feedback forms received from the

regions) to determine whether appropriate inspection resources were applied in each of the

inspectible areas. Based on these reviews, NRR and regional staff have made

recommendations to improve the baseline inspection program to both regional and NRR

management. Inspection program staff plans to obtain regional and NRR management

approval on these recommendations and to revise and issue the affected inspection procedures

before the beginning of CY 2008. Many of these recommendations to the inspection

procedures (IPs) addressed issues such as accuracy of inspection effort required to complete

the IPs and strategies to improve identification of inspection findings using some of our IPs.

Regarding the comments that IP 71152 needs a better focus on problem identification, the staff

periodically reviews inspection procedures to ensure that they are efficient and effective. The

staff is currently conducting an ROP realignment effort as discussed above. For IP 71152, this

will include an assessment of regional implementation practices, and IP 71152 inspection

requirement and guidance content, to develop recommendations to enhance the IP to be more

effective in identifying licensee problems. In addition, the staff considers the applicability of

cross-cutting aspects in the problem identification and resolution cross-cutting area for greater

than minor inspection findings which are then used in the assessment process to identify

whether a substantive cross-cutting issue exists. In this way, the results of findings from any

inspection procedure can make a contribution to the identification of a substantive cross-cutting

issue in the area of problem identification and resolution. The staff will generate an ROP

feedback form to further evaluate whether IP 71152 should focus more on whether the licensee

is effectively identifying problems.

Regarding the comment that there is extensive use of resources to assess the significance of

inspection findings, the NRC established a criterion for limiting the number of hours spent on

the significance determination of inspection findings to less than ten percent of the total time
-17-

spent on baseline inspections. Since the implementation of the ROP, the staff has always met

this criterion. As a matter of fact, for the 2006 assessment period the hours spent on the

Significance Determination Process (SDP) were approximately four percent of the hours spent

on the baseline inspections.

Regarding the comment that there are a high number of findings with cross-cutting aspects and

plants with substantive cross-cutting issues, the staff has been monitoring the effects of the

safety culture enhancements. With respect to the number of findings with assigned crosscutting aspects the staff has noticed that the percentage of findings with cross-cutting aspects

has increased to slightly over 70 percent from the historical average of approximately 66

percent of the findings that were in a cross-cutting area prior to the safety culture

enhancements. This is not viewed by the staff as a negative development as the cross-cutting

aspects are used to identify potential themes during the assessment process. With respect to

the number of plants with substantive cross-cutting issues, there were 12 plants with an SCCI

following the most recent end-of cycle assessments. This is comparable with the number of

plants with SCCIs prior to the SC enhancements. At this time the staff is of the opinion that the

SCCI threshold is appropriate.

Regarding the comment that the NRC should consider more frequent inspection and verification

of licensee corrective action programs, as discussed above the staff is currently evaluating

IP 71152 as part of the ROP realignment effort to develop recommendations to enhance the IP

to be more effective in identifying licensee problems. IP 71152 already includes an ongoing

component where licensee corrective action programs are reviewed to ensure that adverse

conditions are properly identified, evaluated, and corrected. There is also a portion of the

procedure that is performed on an annual basis and biennial basis at each plant. It should be

noted that in recognition that one of the underpinnings of the ROP is that licensees have

effective corrective action programs, the staff expends considerable resources on IP 71152

inspections. In CY 2006, approximately 22,600 hours were expended on 71152 inspections

which was a significant fraction of the ROP baseline inspection effort.

Regarding the comment that the role of the NRC in situations that do not involve regulations but

might involve a performance deficiency should be examined and better defined, the NRC made

a change to the definition of a performance deficiency in IMC 0612 in November 2006. This

change revised the definition of a performance deficiency to bring it in alignment with the basis

for a performance deficiency as described in the ROP basis document, IMC-0308

Attachment 3, “Significance Determination Process Basis Document.” The current definition of

a performance deficiency requires that an issue must be a “performance deficiency” before it

can be considered a finding and that the performance deficiency satisfy the following criteria:

• Did the licensee fail to meet a requirement or a standard, where the cause was

reasonably within the licensee’s ability to foresee and correct and which should have

been prevented?

• A performance deficiency can exist if a licensee fails to meet a self-imposed standard or

a standard required by regulation.

Regarding the comment that without a strong enforcement program, the inspection program

cannot ensure correction, the ROP was designed to provide increasing levels of oversight

consistent with increasing levels of safety significance. The ROP requires that all licensee’s

corrective actions taken to address inspection findings which have been determined to be

greater than very low risk significance (>Green) be verified for adequacy by the NRC. For

those inspection findings which are determined to be of very low risk significance (Green), we
-18-

expect that the licensee will take the necessary actions in a timely manner to correct these

deficiencies. Additionally, for those very low risk significant findings, we verify that the

licensee’s corrective actions were adequate in both scope and timeliness by selecting a sample

of deficiencies which were entered into the licensee’s corrective action program. These

verification inspections are completed semiannually, annually and biennially as described in our

Problem Identification and Resolution (PI&R) inspection, IP 71152.

Regarding the comment that NRC inspectors are identifying and documenting issues that are

not risk significant, IMC 0612, “Inspection Reports,” provides guidance to the inspectors on

what types of findings can be documented in inspection reports and requires that only

performance deficiencies which are greater than minor and also have an effect on the

objectives of the cornerstones can be documented. Therefore, the inspection program by

design is intended to focus inspectors on more risk significant findings. However, inspectors

are able to identify and document very low risk significant findings (Green) as long as they were

determined to be performance deficiencies and were greater than minor findings. Additionally,

inspectors must document all violations of NRC regulatory requirements which are determined

to be greater than minor findings.

Regarding the comment that the inspection program is based on plant PRAs that have never

been approved, although the plant PRA is not a regulatory requirement, licensees have

developed PRA models for their facilities, in part, to independently perform analysis of and to

better understand NRC’s risk determination of inspection findings identified at their facilities. 

Therefore, although it is accurate to state that the licensee’s PRAs have never been approved

by the NRC, its results are reviewed by the NRC during the evaluation of the finding’s

significance. Licensee PRA quality continues to be enhanced through the benchmarking of the

NRC’s independent plant specific risk-informed inspection notebooks, by the improvements of

the NRC’s independent Standardized Plant Analysis Risk (SPAR) models, and by the

availability of additional guidance (e.g., Regulatory Guide 1.200, “An Approach for Determining

the Technical Adequacy of Probabilistic Risk Assessment Results for Risk-Informed Activities;” 

NUREG CR-6823, “The Handbook of Parameter Estimation for Probabilistic Risk Assessment;”

and NUREG-1792, “HRA Good Practices”).

Regarding the comment that many examples exist that continue to question the inspection

program (including the Davis-Besse event, Safety Conscious Work Environment concerns,

emergency preparedness issues, and security concerns), the NRC is currently addressing all of

these issues. NRC developed and implemented comprehensive improvements to the

inspection program based on the Davis-Besse lessons learned task force recommendations. 

Additionally, NRC has made changes to the inspection program to measure safety culture (in

which safety conscious work environment is a part), and is continuing to verify various security

improvements being implemented at the facilities.

Regarding the comment that the time between when an NRC finding is identified until the time it

is corrected and verified by the NRC is not timely, it is the expectation of the inspection program

that conditions adverse to quality be promptly corrected. Corrective actions are typically

implemented promptly by licensees, independent of the significance determination, though the

correction of the root cause may take longer. Corrective actions taken to address inspection

findings found to be greater than very low risk significance (greater than Green) are verified

within 180 days of being notified by the licensees that the corrective actions have been

completed. However, most greater than Green findings are corrected long before the SDP is

complete. Corrective actions for very low risk significant inspection findings are verified on a

sampling basis on either a semiannual, annual or biennial frequency in accordance with the
-19-

requirements from the Problem Identification and Resolution (PI&R) inspection. Because of the

very low risk significance of the Green findings, NRC believes that their timeliness of review of

selected licensees’ corrective actions is sufficient.

Regarding the comment that an NRC inspection finding, when identified, becomes so narrow in

scope that the final assessment can become an argument over semantics between the NRC

and the licensee, for those more risk significant inspection findings (Greater than Green) there

is a tendency for more discussion on the proper characterization of the inspection finding to

allow better understanding of the issue by both the NRC and the licensee. Although at times,

this may appear to be a discussion over semantics, it is an important part of the open regulatory

process where those regulated can openly discuss the inspection finding with the NRC so that

they have a better understanding of our regulations and requirements.

Regarding the comment that inspectors have much less latitude to explore safety issues and

much less authority under this inspection program, the ROP requires that all baseline inspection

inspections be conducted at all facilities. The purpose of the baseline inspection requirement is

to focus inspectors on more risk significant areas, so that the baseline inspection program is

more effective at all facilities. In the past, we relied more heavily on individual inspectors to

make the determination on which areas the inspectors needed to focus on at each of the

facilities. Although this type of inspection approach allowed more latitude by inspectors to

explore different areas, it also led to inspection inconsistencies between licensees and also was

difficult to measure and determine the effectiveness of the inspection program as a whole. The

ROP baseline inspection program requires all inspections to be completed at all sites

regardless of their performance. This type of inspection program allows increased consistency

and a way to measure overall inspection effectiveness.

The comment that the NRC should consider enhancing the use of generic communications for

inspection trends (similar to the ROP PI FAQ process) will be entered into the ROP feedback

process for future consideration.

6. Is the information contained in inspection reports relevant, useful, and written in

plain English?

Respondent Comments:

Union of Concerned Scientists

This aspect of the ROP used to be one of marked improvement over the inspection reports

issued pre-ROP. The current inspection reports remain better written than pre-ROP inspection

reports, but significant margin has been lost over the past year. The authors of the ROP

inspection reports are not to blame - the program relies far too heavily on boilerplate templates

where authors can only fill in the plant's mailing address and change a few words of text

(e.g., “is / is not doing a swell job”). The result of such overblown, mis-applied standardization

is that the inspection report for a plant doing a stellar job is scarcely discernible for an

inspection report for a plant dodging meltdowns. And even when outside readers ferret out the

"not" usages, the boilerplate templates provide little ability to explain why the "not" labels were

warranted. Today's ROP inspection reports do a fine job of chronicling what was examined.

They do less well explaining what the NRC concluded from the examinations. And they are

almost totally inadequate in explaining the safety significance of the unclear findings. The

inspection reports should clearly articulate what was examined, what was found, and what
-20-

significance the NRC assigned to the findings. Current inspection reports [fall] far short of that

objective.

Pennsylvania Department of Environmental Protection

The information contained in the inspection reports is relevant and useful.

New Jersey Division of Environmental Safety and Health

• If the NRC inspection findings are greater then green, then the inspection report

provides more assessment content that is relevant and useful. Otherwise, by the time

they are issued the information is outdated.

• The referenced document section in the back of the inspection report is irrelevant.

These are documents that are not public so they provide no real value in the report.

Nuclear Energy Institute

Inspection report information is generally useful and the organization helps to provide focus in

problem areas. However, there is some disagreement about whether the reports are written in

“plain English.”

It is also noted that the link between an inspection finding and a cross-cutting aspect is not

always clearly articulated in the inspection reports. Often, the inspection report language only

states that the finding is related to the crosscutting aspect with no explanation of how the

cross-cutting aspect significantly contributed to the cause of the finding. The link between the

cross-cutting aspect and the cause of the finding should be clear to the reader of the inspection

report.

Region IV Utility Group

Generally, the reports are relevant, useful and written in plain English. We note, however, that

the reports are growing in size and detail with minimal added value. Preliminary experience

with the NRC's Safety Culture initiative, indicates cross cutting aspects associated with

inspection findings are appropriately documented.

Strategic Teaming and Resource Sharing

Generally, the reports are relevant, useful and well written. Preliminary experience with the

NRC's Safety Culture initiative, indicates cross cutting aspects associated with inspection

findings are appropriately documented.

NRC Response:

As noted in the ROP self-assessment and metric report for IP-8, "Inspection Reports Are

Relevant, Useful, and Written in Plain Language," the majority of those who provided feedback

to the question on whether the information in the inspection reports was relevant, useful, and

written in plain language responded that the inspection reports were clearly written and useful. 

There were, however, comments for improvement in this area as discussed below.

Regarding the comments that inspection reports use mis-applied standardization and that the

inspection reports do not differentiate levels of licensee performance, the purpose of an

inspection report is to document the scope of inspectors’ review of areas inspected and also to

document greater-than-minor inspection findings which were identified during their review. 

NRC program document, IMC 0612 (Inspection Reports) requires that inspection reports

document what was examined, what was found, and what significance the NRC assigned to

these inspection findings. At plants where more safety significant (greater than Green)
-21-

inspection findings are identified, IMC 0612 also requires that such inspection findings be

highlighted in the inspection report cover letter. Inspection reports document the variety of

inspections conducted under the ROP and are a vehicle to document both the scope of review

and the different inspection findings identified during the period of inspection. Inspection

reports are not meant to communicate NRC’s view of overall licensee performance based on

the inspection findings discussed in the inspection reports. Assessment of licensee’s

performance are conducted using IMC 0305, “Operating Reactor Assessment Program,” and

are communicated through the mid-cycle and end-of-cycle letters.

Regarding the comment that information is outdated by the time an inspection report is issued,

most inspection reports document inspection activities which occur over a three-month time

period. At the end of the three-month period, all inspection inputs are collected and compiled

into one inspection report that is issued within 45 days of the end of the calendar quarter. This

could potentially result in situations where an NRC’s review of an event or a condition which is

identified at the beginning of the inspection period is made public up to 4 1/2 months later. 

However, issuing inspection reports on a quarterly periodicity is a more efficient process since

the inspection staff spends less time on inspection report preparation and documentation and

more time conducting baseline inspections. Inspection report timeliness goals are accelerated

as necessary for those inspection reports covering potential escalated enforcement actions and

as specified in Management Directive 8.3 for reactive inspections (e.g., AITs, and IITs). 

Additionally, it is recommended that whenever an inspection reveals findings of significance

(i.e., White or higher) or other significant or immediate public health and safety concerns, an

expedited inspection report that is limited in scope to the specific findings should be considered.

The comment regarding the fact that the referenced document section in the back of the

inspection report is irrelevant since the documents are not publicly available has been entered

into the ROP feedback process for future consideration in revising IMC 0612.

Regarding the comment that the link between an inspection finding and a cross-cutting aspect

is not always clearly articulated in the inspection reports, the staff has continued to reinforce the

expectations for inspectors to include discussion as to why the cross-cutting aspect is a

significant contributor to the finding. The new requirement to describe the cross-cutting aspects

of the inspection finding in the analysis paragraph was added to IMC 0612, “Inspection

Reports,” in June of 2006. Since June of 2006, the inspection program staff has conducted

training sessions in all regions on how to document cross-cutting aspects of inspection findings

in inspection reports. There should be an observable improvement in NRC inspection reports

on the relationship between the cross-cutting aspect and the cause of the finding in CY 2007. 

In addition, as part of the initial implementation monitoring of the safety culture enhancement,

the staff intends to review a sample of inspection reports to assess the cross-cutting aspect

documentation. Based on the results of this review, the staff will recommend changes to IMC

0612 as needed.

Regarding the comment that the reports are growing in size and detail with minimal added

value, the increase in the length of the inspection report is caused by issuing one quarterly

inspection report which includes all inspections conducted by all region-based inspectors during

a 3 month period. In the past, NRC issued shorter but more numerous inspection reports to

document each of the inspections conducted by different region based inspection groups. The

inspection reports are constructed so that more safety significant (greater-than-Green) NRC

inspection findings are documented in the inspection report cover letters and also a synopsis of

all inspection findings are documented in the summary of findings section. The cover letter and

summary of findings section of a typical NRC quarterly inspection report is about four pages in

length.
-22-

7. Does the Significance Determination Process yield an appropriate and consistent

regulatory response across all ROP cornerstones?

Respondent Comments:

Union of Concerned Scientists

The SDP is broken. It currently only works to differentiate between Green and Greater

than-Green findings. The leisurely pace at which SDP seeks to determine which of the three

Greater-than-Green colors to ultimately dole out makes SPD [SDP] a consistently ineffective

means of dictating regulatory response. Since its inception, the ROP has promised the public

timely SDP determinations. Since its inception, the ROP consistently let the public down in this

area. It has stopped being funny. If NRC senior managers were serious about the ROP, they

would fix SDP timeliness NOW. No more excuses, no more lies, no more empty promises.

Final SDP colors must be determined within 90 days or someone at NRC should lose his/her

job.

Pennsylvania Department of Environmental Protection

Overall, SDP has contributed to improved communications between NRC and licensees

regarding the issues that are risk-significant. However, it is a resource-intensive and

complicated process. Also, it may not always yield equivalent results for issues of similar

significance in all ROP cornerstones. Additionally, licensees continue to challenge the SDP

findings.

The NRC should continue to collect data, trend the timeliness and the resource intensiveness of

the process, and make improvements as necessary.

South Texas Project Nuclear Operating Company

There are too many SDPs that are not based on risk or actual effect thresholds. The Radiation

Protection, Security, and Emergency Preparedness SDPs are very subjective and deterministic.

They do not reach consistent results and are very dependent on subjective views of the

individual exercising the SDP, especially in the case of the security SDP. The Industry and

NRC staff should attempt to develop risk based platforms to perform these evaluations that are

founded on actual data and limit the subjectiveness in the process.

New Jersey Division of Environmental Safety and Health

• Not enough greater than green findings are identified by the NRC to adequately

evaluate the SDP, even after all this time.

• Every time a greater then green finding has been identified, our experience has been

that the greater than green finding is usually lowered via licensee argument, in spite of

the SDP. The SDP is then usually changed in the less conservative direction.

• The SDP is very easy to manipulate.

• If the SDP was appropriate and consistent then why does the NRC continue to revise

them, create new ones, and meet frequently with the licensee to discuss them?

Nuclear Energy Institute

The Significant Determination Process (SDP) does not yield equivalent results for issues of

similar significance across all ROP cornerstones due to the limitations of current risk analysis

methodologies. Specifically, issues and events such as Security, Fire Protection, Emergency 
-23-

Preparedness and Public Radiation Safety are evaluated using processes that are more

subjective or qualitative in nature and may result in exaggeration of actual risk. However, when

the SDP does utilize a more detailed risk analysis, an environment of cooperation and mutual

learning typically prevails between the utility and the NRC. This promotes the consistency and

quality of the evaluation and produces appropriate risk oversight.

The SDP should have a graded approach to timeliness in that additional time should be

provided to work with and respond to the NRC for findings with higher significance rather than

the blanket 30 days.

For SDP results with elevated colors (particularly white), using Phase 2 notebooks to issue

choice letters results in inaccurate determinations solely for the purpose of meeting NRC

timeliness goals and results in a costly and resource-intensive risk analysis exercise that has

little safety value for the licensee. In addition, the amount of credit the NRC staff permits for

operator actions and engineering evaluations is inconsistent and arbitrary.

For SDP evaluations conducted for conditions for which specialized PRA results are needed

and are not available in the Regions (e.g., Shutdown Conditions), communications between

sites and Headquarters personnel (facilitated by Regional SRAs) has been inconsistent and not

as constructive as similar communications with the Regions.

Region IV Utility Group

There are too many SDPs that are not based on risk or actual effect thresholds. The Radiation

Protection, Security, and Emergency Preparedness, and other deterministically based SDPs,

are very subjective (and commonly aggregate multiple non-significant findings into a single

significant finding). As a result, a white finding under these SDPs is not the same significance

as a white finding in the more risk informed SDPs - (covered by IMC 0609 Appendix A). These

deterministic SDPs do not always produce consistent results because of the dependence on

the subjective views of the individuals applying the SDP guidance. The industry and NRC staff

should strive to improve these SDPs by including more risk-based elements thus helping to limit

the subjectiveness in the process.

Strategic Teaming and Resource Sharing

There are too many SDPs that are not based on risk or actual effect thresholds. The Radiation

Protection, Security, and Emergency Preparedness SDPs are subjective and deterministic.

They do not produce consistent results because of the dependence on the subjective views of

the individuals applying the SDP guidance, especially in the case of the Security SDP. The

industry and NRC staff should strive to improve these SDPs by including more risk-based

elements, thus limiting the subjectivity and promoting more consistent significance

determinations between cornerstone areas.

NRC Response:

As noted in the ROP self-assessment and metric report for SDP-4, "Results of the Same Color

Are Perceived by the Public to Warrant the Same Level of Regulatory Attention for All

Cornerstones," the external survey respondents remain negative on the SDP yielding consistent

regulatory response across all ROP cornerstones. Regarding the comments that the Radiation

Protection, Security, and Emergency Preparedness SDPs are very subjective and deterministic,

the staff continues to believe that relative parity has been achieved among the cornerstones,

based on the potential impact on public health and safety and the designated NRC response to

specific findings. The comments and our response have been essentially the same for this
-24-

question for the last several external surveys with no new insights or planned actions. As a

result, the staff plans to delete this metric from the ROP self-assessment program as described

in IMC 0307 and remove the related question from future external surveys. Findings are

continuously under review by the staff to determine the need for adjustments to the SDPs in

this area. For example, based on a finding identified during this assessment period, the

outcome of the Public Radiation Safety SDP is being evaluated by the staff to assure

equivalence between the various SDP outcomes.

To respond to comments received regarding the timeliness of the final significance

determinations, as noted in Enclosure 1 to SECY-07-0069, process improvements in the SDP

resulted in efficiency gains in determining the safety significance of identified performance

issues. Since implementation in April 2000, the SDP has gone through several significant

changes based on feedback from internal and external stakeholders and the recommendations

of two independent audits. As a result, SDP timeliness has improved significantly, meeting its

goal for the first time since the implementation of the ROP. Enhancements to the process

continue, such as the current implementation of the SDP Phase 2 pre-solved tables to

complement the plant specific risk-informed inspection notebooks. The staff has addressed

several significant issues during CY 2006, including implementing the pre-solved tables; issuing

Appendix M to IMC 0609, “Significance Determination Process Using Qualitative Attributes;”

and addressing the need for risk-informing findings that do not fit a previously developed SDP.

The staff streamlined the process and simplified the procedure for inspectors to implement the

Phase 2 tools by benchmarking and updating the notebooks and developing the associated

pre-solved tables. The Tables were added to the SDP Phase 2 process for Appendix A to

IMC 0609, “Determining the Significance of Inspection Findings for At-Power Situations.” 

Training of inspectors in the use of the revised Notebooks and Tables is being accomplished in

two steps. The staff completed training of the Senior Reactor Analysts (SRAs) at the two SRA

counterpart meetings, and the SRAs are in the process of training the inspectors at the

inspector counterpart meetings. Once this training is completed, the Phase 2 tools with the

increased use of the Tables will reduce the burden on inspectors, allowing for additional

improvement in the timely assessment of SDP findings.

Regarding the comments that: (1) not enough greater than green findings are identified by the

NRC to adequately evaluate the SDP, (2) every time a greater then green finding has been

identified, it is usually lowered via licensee argument and the SDP is then usually changed in

the less conservative direction, (3) the SDP is very easy to manipulate, and (4) if the SDP was

appropriate and consistent then why does the NRC continue to revise them, create new ones,

and meet frequently with the licensee to discuss them, the SDP is rigorous process outlined

and defined in public documents. Following the SDP process, a certain number of findings

during the assessment period were assessed to be white or greater in the final assessment

determination. Licensee appeals did not reduce the level of these findings during the

assessment period. Nonetheless, the SDP process is designed to consider licensee inputs,

and when appropraite to incorporate the input into its analysis. SDPs get revised to improve

quality, make the process more user friendly, and to incorporate lessons learned from the

implementation of the process. For example, during the assessment period, the phase 2

risk-informed inspection notebooks were updated to better reflect licensee PRAs and

pre-solved tables were added to simplify implementation. No other changes to SDPs were

made during the period.

Regarding the comment that the SDP should have a graded approach to timeliness in that

additional time should be provided to work with and respond to the NRC for findings with higher 
-25-

significance, the need for additional time to complete SDPs assessing complex issues has been

recognized and addressed in IMC 0609 “Significance Determination Process” and IMC 0609.01

“Significance and Enforcement Review Panel.” Both documents are available on the NRC

public website.

Regarding the comment that using Phase 2 notebooks to issue choice letters results in

inaccurate determinations and results in a costly and resource-intensive risk analysis exercise

that has little safety value for the licensee, choice letters provide the licensees the options to

accept the results of the preliminary assessment or provide the staff additional information

which may affect the outcome of the initial significance determination associated with a licensee

performance deficiency. The extent and cost of the licensee resources to develop such

information that may or may not change the significance of the finding is entirely up to the

individual licensee. The SDP is designed to risk-inform findings and thereby provide licensees,

the public and the NRC staff information on licensee performance and expected NRC response. 

Regarding the comment that the amount of credit the NRC staff permits for operator actions

and engineering evaluations is inconsistent and arbitrary, the information contained in the

Phase 2 plant-specific notebook is based on the licensee’s Individual Plant Examination (IPE)

submittal, the updated PRA, system information obtained from the licensee during site visits as

part of the review of earlier versions of the notebook, and subsequent interactions. Approaches

used to maintain consistency within the SDP, i.e, consistent modeling considerations and

applicable assumptions, specifically within similar plant types, resulted in sacrificing some plantspecific modeling approaches and details. The approaches used are documented in the

Technical Basis Document for At-Power Significance Determination Process (SDP) Notebooks

(IMC 0308). Plant-specific comments and insights were also considered. Specific changes

made addressing plant-specific features and insights were summarized. A benchmarking of the

notebook was conducted comparing and analyzing the risk significance of the inspection

findings obtained using this notebook and the plant-specific PRA. Following benchmarking, the

notebook was updated considering the licensee’s updated PRA, and any changes in plant

design and operational practices. The notebook provides a table comparing the order of

magnitude results, i.e., the colors for an inspection finding, using the notebook and the plant

PRA. Conservative and non-conservative results by the notebook, compared to the plant PRA,

are noted in the table with a brief discussion of the reasons for the differences.

Regarding the comment that communications between sites and NRC Headquarters personnel

(facilitated by Regional SRAs) has been inconsistent and not as constructive for SDP

evaluations for specialized conditions, as previously stated the conduct of SDPs, including peerreviews and the request of involvement by specialists outside the regional office, is a highly

structured process that cannot be trumped or routinely altered by SRAs, risk practitioners and

inspectors. Consistent implementation amongst the regions is monitored through scheduled

workshops and telephone conferences at inspector, SRA and senior management levels. The

ongoing implementation of recommendations resulting from the “Regional Best Practices for

Managing Significance Determination Process Timeliness” task group is also enhancing the

uniform implementation of the SDP.

Several comments addressed the difference in the SDP methods that were used to determine

significance across the cornerstones where some SDPs are risk-informed, while others, such

as Emergency Preparedness (EP) and Radiation Protection (RP) SDPs are deterministic, and

that the industry and NRC staff should strive to improve the deterministic SDPs by including

more risk-based elements (thus helping to limit the subjectivity in the process). In establishing

SDPs in these areas, it is difficult to relate deficient performance in these areas to quantitative
-26-

risk measures such as core damage frequency or large early release frequency. The staff has

worked to define SDPs in the EP and RP areas that result in the agency response that is

considered appropriate for a range of performance problems. Because of the inherent

differences between the various cornerstones, the staff does not presently envision being able

to define a single risk-informed SDP approach that would address all cornerstones.

8. Does the NRC take appropriate actions to address performance issues for those

plants outside of the License Response Column of the Action Matrix?

Respondent Comments:

T. Gurdziel

If this statement were true, Indian Point would have reliable emergency sirens and would be

pumping contaminated ground water or wouldn't be running two plants at 100 percent.

Union of Concerned Scientists

GAO's September 2006 report titled "NUCLEAR REGULATORY COMMISSION: Oversight of

Nuclear Power Plant Safety Has Improved, but Refinements Are Needed" clearly and

undeniably shows the NRC isn't getting this job done. If the NRC took appropriate action to

address performance issues for plants outside of the Licensee Response Column, declining

performance at a plant would trigger heightened response by the NRC that induced the

changes necessary to turnaround the decline and restore any lost margins. Table 8, "Highest

GAO Oversight Level Applied during at Least Some Portion of the Year, 2001 Through 2005,"

in GAO's report shows that the NRC is failing. Peach Bottom Unit 2 is on the list for 2001,

2002, 2003, 2004, and 2005 - showing that NRC's response was not effective. Fermi Unit 2 is

on the list for 2001, 2002, 2003, 2004, and 2005 - showing that NRC's response was

ineffective. All three reactors at Oconee are on the list for 2001, 2002, 2003, 2004, and 2005

(except Unit 2 absent from the list in 2002) - showing that NRC's response was not effective.

Indian Point Unit 2 is on the list for 2001, 2002, 2003, 2004, and 2005 - showing that NRC's

response was ineffective. Both reactors at Point Beach at [are] on the list for 2001, 2002, 2003,

2004, and 2005 - showing that NRC's response was not effective. This evidence harkens back

to the futility of the former oversight process - the Systematic Assessment of Licensee

Performance (SALP). The NRC's SALP process populated semi-annual Watch Lists, but

reactors like Dresden occupied positions on the Watch List for nearly entire decades. SALP

failed because the NRC flagged, rather than fixed, poor performance at reactors. The ROP

seems to be SALP reincarnated. The NRC's oversight process simply cannot allow reactors

with known safety performance problems to operate year in, year out seemingly immune to

NRC's efforts.

Pennsylvania Department of Environmental Protection

Based on our experience with the ROP implementation at the PA power plants, the actions

taken by the NRC to address performance issues for licensees outside the Licensee Response

Column conform to current ROP program.

South Texas Project Nuclear Operating Company

The NRC action in accordance with the Action Matrix is clear and consistent.

New Jersey Division of Environmental Safety and Health

The NRC actions are usually too narrow in scope to ensure that the larger issues are corrected.
-27-

Nuclear Energy Institute

Actions taken by the NRC to address performance issues for licensees in accordance with the

Action Matrix is clear and consistent.

Region IV Utility Group

The NRC action in accordance with the Action Matrix is clear and consistent for single White

findings, but appears less consistent for more complex issues. There is evidence that process

is not always followed (or may be deviated from) when circumstances should result in moving a

licensee to a lower action state. Once a deviation from the process has occurred, it becomes

unclear how to exit from the overall process.

Strategic Teaming and Resource Sharing

The NRC action in accordance with the Action Matrix is clear and consistent for single White

findings, but appears less consistent for more complex issues.

NRC Response:

As noted in the ROP self-assessment and metric report for AS-8, "NRC Takes Appropriate

Actions to Address Performance Issues," the industry and states generally agreed that actions

taken by the NRC for plants outside of the licensee response column have been appropriate,

while Public interest groups were generally critical of NRC actions. The overall level of external

stakeholder satisfaction in this area was generally favorable and similar to previous years.

Regarding the comment that the NRC's oversight process allows reactors with known safety

performance problems to operate, declining performance at a plant triggers a heightened

response from the NRC in accordance with IMC 0305 and the ROP Action Matrix. If the NRC

lacks reasonable assurance that the licensee can or will conduct its activities to ensure

protection of public health and safety, continued plant operation will not be permitted. Even

though licensees operate plants in various columns other than the licensee response column

(other than the unacceptable response column), these plants are safe to operate. However,

one concern that is being addressed is that some plants may languish in these lower

performance columns for far too long. As a result, the Commission has directed the staff to

enhance the assessment process to increase senior management and the Commission

involvement at earlier intervals, prior to further degradation of performance. The staff plans to

revise the ROP to reflect these changes in the near future.

Regarding the comment that NRC actions are usually too narrow in scope to ensure that the

larger issues are corrected, the NRC performs supplemental inspections beyond the baseline

inspection program to address known performance issues. As noted in Appendix B to IMC

2515, the breadth and depth of the supplemental inspections increase in proportion to the

relative risk significance of the identified performance issues and will be based upon the

guidance provided in the NRC’s "Action Matrix” and the Supplemental Inspection Selection

Table, summarized below:

• For plants in Column 2 of the Action Matrix, Inspection Procedure (IP) 95001 is

performed consisting of review of the licensee’s evaluation of root cause and extent of

condition plus review of proposed corrective actions. Follow-up NRC inspection is

conducted to determine effectiveness of corrective actions. Inspection is limited to

specific issue(s) or performance area of concern.

• For plants in Column 3 of the Action Matrix, IP 95002 is performed consisting of review

of the licensee’s evaluation of root cause and extent of condition, plus review of
-28-

proposed corrective actions for both individual and collective issues. An independent

NRC inspection is conducted to assess validity of licensee’s extent of condition, follwed

by a follow-up NRC inspection to evaluate the adequacy of licensee corrective actions.

• For plants in Column 4 of Action Matrix, IP 95003 is performed by a large multidisciplined NRC inspection team comprised of regional inspectors and safety culture

assessors. The inspection is focused on all key attributes associated with affected

strategic performance areas. The NRC expects the licensee to perform a third-party

assessment of their safety culture. As part of the IP 95003 inspection the staff will

evaluate the licensee's safety culture assessment, and perform an independent

assessment of the licensee's safety culture.

A commenter noted that the Action Matrix appears less consistent for more complex issues,

that process is not always followed (or may be deviated from) when circumstances should result

in moving a licensee to a lower action state, and that once a deviation from the process has

occurred, it becomes unclear how to exit from the overall process. As the commentor has

noted, the staff has on rare occasions deviated from the Action Matrix when the situation was

warranted. This was not unexpected at the outset of the ROP, since the ROP can not be so

predictive to incorporate all situations. The staff is also aware of the comments that focus on

the lack of column exit criteria. However, the assessment process, as described in IMC 0305,

is predicated on completely re-assessing plant perfomance every quarter, or in other words,

once a plant is in a particular column, the findings and performance indicators are re-assessed

against the column entry criteria for the Action Matrix quarterly. If any of these inputs change,

then licensee performance would be re-assessed for the corresponding column of the Action

Matrix, and the licensee's performance is matched with the correpsonding column that reflects

the revised Action Matrix inputs. In rare situations that justify a deviation, the licensee may

remain in a particular column until such time that the deviation is no longer warranted.

9. Is the information contained in assessment reports relevant, useful and written in

plain English?

Respondent Comments:

Union of Concerned Scientists

The mid-cycle and annual assessment letters are worse than inspection reports with regard to

the level of standardization that renders them useless. It doesn't matter which NRC region you

choose, but if you take the mid-cycle or annual assessment letter for the "worst" performing

reactor in that region (i.e., the reactor furthest to the right in the Action Matrix) and the mid-cycle

or annual assessment letter for the "best" performing reactor in that region (i.e., the reactor in

the left-most column of the Action Matrix least adorned with footnotes and substantive

cross-cutting issues markers) and read them, you are likely to acquire a sense of which reactor

the NRC likes most. But you will hardly be able to explain to any one else why the NRC viewed

one reactor as naughty and the other reactor as nice. The overly standardized format of the

mid-cycle and annual assessment letters reduces NRC's commentaries to mere hints and

innuendoes. The NRC must step out from behind this template nonsense and clearly explain

what it thinks (and why) about reactor performance.

Pennsylvania Department of Environmental Protection

The assessment reports are generally relevant and contain useful information.
-29-

South Texas Project Nuclear Operating Company

The recent work by the NRC staff to clarify the exit process for a Substantive Cross Cutting

Issue was very effective.

New Jersey Division of Environmental Safety and Health

• Assessment reports are limited. It is usually less then two pages. One or two lines in

the assessment report actually provides a written assessment. The rest of the report

refers to the action matrix, which most people don’t understand.

• The NRC has two opportunities each year to tell the public and the licensee how the

plant is doing – so why doesn’t the NRC provide more assessment language for the

public so they can understand how the plant is performing. Currently the public must

rely on the action, which doesn’t clarify true performance.

Nuclear Energy Institute

The information contained in assessment reports is relevant and useful. However, there is

some concern that the reports are not written in “plain English.” Recent work by the NRC staff

to clarify the exit process for a Substantive Cross Cutting Issue has been very effective.

Region IV Utility Group and Strategic Teaming and Resource Sharing

The recent work by the NRC staff to clarify the exit process for a Substantive Cross Cutting

Issue was very effective. The documented analysis of cross cutting aspect inputs to the

assessment process could be improved. The current assessment guidance permits the cross

cutting aspect to be changed if additional insights are available following publication of the

associated inspection report. Given the regulatory principles that guided the development of

the ROP (that overall assessments of licensee performance remain transparent,

understandable, objective, predictable, risk-informed, and performance-based), any change in

the assigned aspect should be readily available to the licensee as well as other stakeholders.

NRC Response:

As noted in the ROP self-assessment and metric report for AS-9, “Assessment Reports Are

Relevant, Useful, and Written in Plain Language,” the industry and states generally agreed that

the information contained in assessment reports is relevant, useful, and written in plain English. 

One public interest group stated that the assessment reports are limited and that only one or

two lines in the assessment report actually provide a written assessment. The overall level of

external stakeholder satisfaction in this area was generally favorable and similar to previous

years.

Comments regarding the level of standardization used in the assessment reports have been

previously addressed. As noted in SECY-05-0070, program guidance IMC 0305, “Operating

Reactor Assessment Program,” directs that the level of detail in the assessment letters

increases as the plant performance decreases. For example, IMC 0305 requires additional

detail for those plants outside of the licensee response column of the Action Matrix and/or those

plants that have substantive cross-cutting issues. A review by the staff of the most recent midcycle and end-of-cycle assessment letters confirmed that the scope and depth of the letters

were significantly more for plants with performance issues (i.e., outside the Licensee Response

Column of the Action Matrix) than for those plants which had no performance indicators and

inspection findings that were greater than green.

Regarding the comment that the NRC should provide more assessment language in the

assessment letters so the public can understand how the plant is performing, the staff believes
-30-

that the ROP was designed to prevent lengthy, subjective discussion of plant performance. 

This tenet was followed in the streamlining of the format of inspection reports, as well as the

changes made to the assessment process with the development of the Action Matrix. The

assessment letters were designed to communicate the Action Matrix column of performance,

any identified substantive cross-cutting issues (and more recently, whether two of the three

criteria were met), and what if any NRC oversight actions were pending. This language

becomes more standard if licensee performance is governed by the licensee response column,

and more descriptive for licensees in the multiple/repetitive degraded cornerstone column. 

Understandably, any performance information relative to security findings has been removed

from the public assessment letter version.

Regarding the comment that the current assessment guidance permits the cross cutting aspect

to be changed if additional insights are available following publication of the associated

inspection report, IMC 0305 states "In order to support the evaluation of findings with

cross-cutting aspects, the inspectors should provide sufficient detail in the PIM and provide

periodic updates as new information becomes available in accordance with IMC 0306 and IMC

0612." The intent is to keep the process transparent so that changes in the cross-cutting

aspect characterization are readily available to the licensee and other stakeholders.

10. Are the ROP oversight activities predictable (i.e., controlled by the process) and

reasonably objective (i.e., based on supported facts, rather than relying on

subjective judgement)?

Respondent Comments:

Union of Concerned Scientists

The assignment of substantive cross-cutting issues and the dispatching of special inspection

teams are competing for the most arbitrary NRC action. There seems little rhyme or reason to

which reactors get substantive cross-cutting issues and special inspectors [inspections]. The

ROP was supposed to be transparent so that internal and external stakeholders would

understand why action B resulted from condition A. One can follow the ROP trail when a

substantive crosscutting issue is levied or when a special inspection team is dispatched. But

there are plenty of times when conditions at a reactor seem to mirror those of other reactors

that earned substantive cross-cutting issues or special inspection team visits. One cannot

follow the ROP trail and figure out why the NRC failed to respond with similar actions for

apparently identical circumstances. As a minimum, the ROP fails to adequately communicate

NRC's regulatory decisions when those decisions are "no-go's." The "go" decisions

(e.g., substantive cross-cutting issues and sending a special inspection team out) are

documented and publicly communicated. But the times when the NRC makes "no-go"

decisions are not made public. Thus, external stakeholders never learns What differences the

NRC saw between two apparently identical situations that allowed the agency to take action in

one case and justify inaction in the other. Absent that public record, external stakeholders view

NRC's actions/inactions as evidence of regulatory inconsistency. The NRC must document and

make public all ROP decisions, not just those decisions that trigger some NRC action.

Pennsylvania Department of Environmental Protection

The ROP is more objective and predictable than the previous process and the Inspection

Program is better structured. However, there is a concern about the relatively high number of
-31-

issues and findings in the cross-cutting areas and whether the ROP could reasonably and

effectively predict the potential implications of these findings.

New Jersey Division of Environmental Safety and Health

The NRC ROP is designed to maintain an adequate margin of safety. We continue to believe

that the NRC ROP should move to ensure that a continuous improvement process is instilled at

the nuclear power plants because, over time, maintaining adequacy will lead to a plant accident.

Nuclear Energy Institute

ROP oversight activities are generally predictable regarding the type of inspection a licensee

can expect based on location in the Action Matrix and the determination of significance of

inspection findings through the SDP. Subjectivity remains, however, in the FAQ process and in

the recently implemented safety culture process.

The Triennial Fire Protection inspection is deemed very unpredictable. The inspectors attempt

to hold licensees accountable to standards they are not committed to in their licensing basis

and continue to write up issues that are being worked out between the industry and the NRC.

Region IV Utility Group and Strategic Teaming and Resource Sharing

The most recent revision to the Performance Deficiency definition is an improvement, however,

additional improvements could be realized (e.g., defining the scope of "self imposed"). We

recommend that this definition include a condition that the self imposed standard must have

been incorporated into plant procedures prior to being considered for a performance deficiency.

A number of potential findings default to "affects the cornerstone objective" as the reason for

the issue being greater than minor. Additional examples in IMC 0612 Appendix E are needed

to improve the objectivity in this area.

NRC Response:

As noted in the ROP self-assessment and metric report for O-1, “Stakeholders Perceive the

ROP To Be Predictable and Objective,” a majority of the respondents (including utilities, state

agencies and public interest groups) agreed that the ROP is predictable and objective in

comparison to the previous process. Some concerns were noted as discussed below.

The comments about the inconsistency and high number of issues and findings in the

cross-cutting areas was addressed in the response to question #5.

Regarding the comments that the dispatching of special inspection teams appears arbitrary,

NRC has deterministic and probabilistic criteria which determine whether there should be an

NRC reactive inspection for a significant operational event or degraded condition, and whether

that should be an Incident Investigation Team, Augmented Inspection Team, or Special

Inspection. The criteria are in Management Directive 8.3, “NRC Incident Investigation

Program,” and Manual Chapter 0309, “Reactive Inspection Decision Basis for Reactors.”

Regarding the comment that the ROP fails to adequately communicate NRC's regulatory

decisions when those decisions are "no-go's" (e.g., substantive cross-cutting issues and

sending a special inspection team out), the staff recognizes this concern and will continue to

consider enhancements to improve objectivity and public confidence. The decision making

process during the internal mid-cycle and end-of-cycle assessment meeting processes involve

inspectors, regional staff, and management coming together to discuss licensee performance. 
-32-

The mid and end-of-year assessment processes are designed under the ROP to facilitate frank

discussion among the inspectors, management, and other staff and are the best way to come

to a consensus given the facts and issues that have been identified during the past six months. 

However, one problem that has increased in its impact is the increased attention and

significance given by some stakeholders to how the staff decides if a licensee meets the three

acceptance criteria for determining whether there is a substantive cross-cutting issue, which is

not an input into the Action Matrix, but has been increasingly considered important. Recently,

the staff instituted a change in practice to document in the licensee assessment letters when

two out of the three SCCI criteria are met. This will provide some additional insights on those

situations when the NRC has concluded that the licensee is taking appropriate steps to address

the cross-cutting theme.

Regarding the comment that the ROP should move to ensure that a continuous improvement

process is instilled at the nuclear power plants, the NRC Commission has decided that this

approach is not warranted. The NRC’s mission, as stipulated in its Strategic Plan

(NUREG-1614), is to license and regulate the Nation’s civilian use of byproduct, source, and

special nuclear materials to ensure adequate protection of public health and safety, promote

the common defense and security, and protect the environment.

The staff recognizes that there is some degree of subjectivity in the FAQ process because

some plant events do not fit neatly into the PI definitions. Events or conditions occasionally

occur that had not been foreseen or predicted when the guidance was written. Normally the

staff can rather quickly determine how it believes such situations should be resolved, but in

some instances industry disagrees. For those instances where the NRC and industry working

group can’t agree, the issue is taken to the NRC’s Director for the Division of Inspection and

Regional Support in NRR who makes the final decision.

Regarding the comment about the fire protection inspection being unpredictable, plans are in

development to address longstanding fire protection issues including manual actions and circuit

issues. Once finalized later this year, the proposed resolutions will become publically available.

Regarding the recommendation that the definition of performance deficiency should be revised

to include a condition that the self imposed standard must have been incorporated into plant

procedures prior to being considered for a performance deficiency, the definition was recently

revised/clarified in IMC 0612, “Inspection Reports,” and is consistent with the ROP basis

document. Overdefining performance deficiency could result in other problems such as

misinterpretation.

Regarding the comment that a number of potential findings default to "affects the cornerstone

objective" as the reason for the issue being greater than minor, in accordance with IMC 0612

the inspectors need to state why the inspection finding is greater than minor and how the

inspection finding affects the cornerstone objective.
-33-

11. Is the ROP risk informed, in that the NRC's actions and outcomes are

appropriately graduated on the basis of increased significance?

Respondent Comments:

Union of Concerned Scientists

In the beginning, the ROP achieved a remarkable balance between performance-based

regulation and risk-based regulation. That is no longer true.

Pennsylvania Department of Environmental Protection

The ROP is more risk-informed than the previous process and overall, the NRC actions are

generally graduated on the basis of increased significance.

South Texas Project Nuclear Operating Company

While true for the Initiating Events, Mitigating Systems, and Barrier Integrity cornerstones, it is

not true for the other cornerstones since the outcomes are not risk informed. The outcomes of

issues in these other cornerstone areas are more subjective and less risk informed as a result.

New Jersey Division of Environmental Safety and Health

• The ROP is designed to be risk informed but the underlying risk assumptions have still

not been reviewed and approved by the NRC.

• In light of the limited inspection findings greater than green, the ROP should be

re-designed to make sure that more risk significant items are included in the risk basis.

Nuclear Energy Institute

The Initiating Events, Mitigating Systems and Barrier Integrity ROP cornerstones are

risk-informed. However, other cornerstones are not risk-informed. This results in more

subjective outcomes in these cornerstones.

Region IV Utility Group and Strategic Teaming and Resource Sharing

We agree that the Action Matrix is graduated based on increased significance.

While also true for findings in the Initiating Events, Mitigating Systems, and Barrier Integrity

cornerstones, it is not true for findings in the other cornerstones since the outcomes are not risk

informed. For example, findings in the Radiation Protection cornerstone that should be

considered minor are often conservatively treated as green due to the limited number of

applicable examples of minor violations available in IMC 0612, Appendix E.

When an applicable example is not found, the application of the screening questions is not

consistent with the principles used in developing the examples.

See also responses to items 7 and 10 as applicable to this item.

NRC Response:

As noted in the ROP self-assessment and metric report for O-2, “Stakeholders Perceive the

ROP To Be Risk-informed,” a majority of the respondents believe the ROP is more risk

informed than the previous process. Some comments from the public expressed reservations

over the direction of the risk informed process and requested that more items that are risk

significant be included in the risk basis.
-34-

The comments regarding some of the cornerstones not being risk-informed have been

addressed in our response to question #7.

Regarding the comments that the ROP is designed to be risk informed but the underlying risk

assumptions have still not been reviewed and approved by the NRC and that the ROP should

be redesigned to make sure that more risk significant items are included in the risk basis, it

would not be practical or a wise use of resources to review and approve every underlying risk

assumption used in a PRA. No matter how detailed probabilistic risk models become, they will

not penetrate to any absolute “true” risk value, due to uncertainties and incompleteness. In

fact, the word “model” itself is used to convey the fact that the interactive physical realities of a

nuclear plant’s operation and responses (e.g., failure mechanisms, timing of events, human

errors of commission) cannot be specified in any absolute sense. Therefore, such complexities

must be treated at a higher level that “models” the physical realities. The nature of risk models

is to use probability distributions to represent some of the known uncertainties, and the use of

numerical probabilities (and probability distributions) is then a means to relatively “weight”

various elements of a probabilistic risk model. The results of risk model calculations cannot be

fairly equated to those of engineering models (e.g., thermal-hydraulics models) that can be

benchmarked against actual physical experiments. Thus, it is crucial that all SDP tools using

probabilistic risk methods be represented as “thinking frameworks” that are designed to allow

technically knowledgeable persons intellectual access to manipulate the variables within this

framework, and explicitly to either accept or challenge the built-in assumptions.

The comment that findings in the radiation protection cornerstone that should be considered

minor are often conservatively treated as green due to the limited number of applicable

examples of minor violations available in IMC 0612, Appendix E, has been entered into the

ROP feedback process for further consideration.

12. Is the ROP understandable and are the processes, procedures and products clear

and written in plain English?

Respondent Comments:

Union of Concerned Scientists

I realize the NRC's ROP webpages were extensively revised this past year and information is

somewhat easier to locate, but it is still an extremely daunting task to find ROP information on

the web. In preparing these comments, I tried hard to access the web information when

providing references for comments, etc. But the lengthy, frustrating searches for information I

knew to be available but very cleverly hidden proved too much. So I reverted to the system that

works best for me - downloading relevant ROP documents from ADAMS to my hard drive and

then using a real search engine to quickly find the information in need. Even in its "enhanced"

form, the ROP webpages are so illogically arranged and cyberally cluttered that I cannot use

them, nor can I in good conscious direct any one else to the website. The NRC's ROP

webpages really need to be overhauled. The guiding concept should be changed from

"let's put it all on one page, they are bound to find something they'll like" to something more

along the lines of "let's present information in a well-constructed, tiered approach that meets the

needs of rookie and veteran ROP web-users."

The Mitigating Systems Performance Index is an unmitigated disaster. The MSPI is as far from

plain English as can be. This ratio of ratios is not readily understood. It is so artificial and so
-35-

contrived that no one can predict what it'll show based on publicly available plant events. Prior

to the MSPI fiasco, one could trace back a White PI finding to publicly available plant

information such as a pump failure (or failures) or extended maintenance time on an

emergency diesel generator. With the secret database upon which the MSPI math is based,

the MSPI outcomes are not understandable.

Pennsylvania Department of Environmental Protection

In general, the ROP is an understandable process, however certain aspects of the ROP are

complex and are not transparent to the public stakeholders (i.e., MSPI, SDP, etc.).

New Jersey Division of Environmental Safety and Health

Has the NRC ever conducted a focus group to determine if “ordinary citizens” can understand

the ROP?

Nuclear Energy Institute

In general the ROP is understandable and the processes, procedures and products are clear.

However, there is some disagreement about being written in “plain English.” As an example, it

was noted that some SDP documents can be difficult to follow without the appropriate technical

background.

Region IV Utility Group

The ROP products are generally clear and understandable. However, inconsistent and/or

changing interpretations of implementation guidance can impact the stakeholder's confidence in

the final product.

A process for addressing generic implementation issues would be helpful. Please see our

discussion in response to Item 5.

Strategic Teaming and Resource Sharing

The ROP procedures and products are generally clear and understandable. The ROP process

is complex and does require significant licensee resources to maintain a working level

understanding.

NRC Response:

As noted in the ROP self-assessment and metric report for O-3, “Stakeholders Perceive the

ROP To Be Understandable,” most of the stakeholders stated that the ROP is understandable

and that products are written in clear and plain English. Some stakeholders noted concerns,

however, as addressed below.

The staff recognizes that the ROP is complex, as are the design and operation of the nuclear

reactors that the NRC regulates. The staff strives to make the ROP as understandable as

possible as it balances this goal with the other ROP goals of being objective, risk-informed, and

predictable. The Office of Public Affairs published a plain language description of the ROP in

NUREG-1649, “Reactor Oversight Process,” which was revised in December 2006. The ROP

portion of the NRC’s website is also designed to facilitate the understandability of and access to

ROP-related information; these Web pages are being modified to make them more userfriendly and intuitive as noted above.

Regarding the arrangement of information on the ROP Web pages, the staff had noted in

SECY-06-0074 that they would consider any specific recommendations to further improve the
-36-

presentation and organization of the ROP-related information on the Web. In order to facilitate

the feedback process and potentially implement improvements to the ROP portion of the NRC’s

Website, the staff contacted those stakeholders that had expressed concerns with the usability

of the ROP Website in 2006 to gather specific ideas regarding organization of ROP information

on the Website. Several specific concerns were addressed and the Web pages were improved

as a result. Additional recommended improvements are welcomed and can be sent directly to

nrrwebservices@nrc.gov. This address should also be used to report any broken links or

problems in accessing ROP-related information.

Regarding the understandability of the MSPI, the staff agrees that the indicator is a complex

algorithm that is not made public due to the security nature of its reliance on licensee PRA

information. The staff regrets that much of the MSPI information has been withheld, and the

output cannot be traced based on the limited information on the website, but this is offset by the

fact that MSPI is a true risk-informed PI and is the first PI to accurately indicate an integrated

performance summed from unavailability and reliability. Additionally, after four quarters of

reported data, the results are generally positive.

Regarding the comment about conducting a focus group to determine if “ordinary citizens” can

understand the ROP, the NRC has reached out to its stakeholders to obtain feedback on the

ROP and the ROP web site. The staff will continue to do this in an effort to improve the

information made available to the public.

Regarding comments about the understandability of the SDP, it is true that some SDP

appendices are complex and, at times, difficult for a layman to fully comprehend. The SDP was

developed to provide a clear framework to facilitate communication of each significance

determination and its basis among technically knowledgeable stakeholders (both internal and

external). The objective of such communication is to achieve a common understanding, to the

extent desired by any interested stakeholder, of SDP decision bases. This allows for broad and

independent verification of the staff’s objectivity and most directly enhances NRC public

credibility. When a quantitative risk model is used, the greatest challenge to achieving this

attribute is to allow stakeholders a means to independently assess SDP result sensitivity to the

most influential assumptions, to understand the basis of the assumptions, and to reveal the

limitations and uncertainties of the risk model and how these were considered by the staff in

arriving at a final result. Given the lack of regulatory prescription or approval of the plantspecific assumptions used in licensee probabilistic risk models, it is essential that this

scrutability attribute is effective for those stakeholders best positioned to identify logical or

factual errors that may alter the final results (e.g., NRC inspectors).

13. Does the ROP provide adequate regulatory assurance, when combined with other

NRC regulatory processes, that plants are being operated and maintained safely?

Respondent Comments:

J. Sniezek, Utility Consultant

The ROP should provide additional focus on precursor issues.

Union of Concerned Scientists

Kudos - no reactors melted down this past year.
-37-

Pennsylvania Department of Environmental Protection

There are no signs of declining plant safety or performance at any of the nine operating plants

in Pennsylvania since the implementation of the ROP. However, as mentioned previously,

there are concerns about the relatively high number of performance deficiencies and findings in

the cross-cutting areas. It should be mentioned that the event at Davis Besse plant has eroded

public confidence in the effectiveness of the ROP to detect and correct problems or

weaknesses in a timely manner.

South Texas Project Nuclear Operating Company

While some actions taken by the NRC staff and licensees are not necessary to assure safety,

those actions that are necessary are assuredly addressed by the regulatory process.

New Jersey Division of Environmental Safety and Health

Until a continuous improvement process is in place, the ROP will only maintain mediocrity.

Nuclear Energy Institute

The ROP does provide adequate regulatory assurance that plants are being operated and

maintained safely.

NRC Response:

As noted in the ROP self-assessment and metric report for O-4, “Stakeholders Perceive That

the ROP Provides Adequate Regulatory Assurance That Plants Are Operated and Maintained

Safely,” a majority of utility stakeholders believe the ROP maintains safety while some of the

non-utility stakeholders (state agencies and public interest groups) feel it does not.

The ROP was generally not designed to predict and/or prevent all failures, but was more

realistically designed to detect declining performance and focus NRC and licensee attention

and resources on the most significant issues. This design recognizes the defense-in-depth

approach to NRC regulation. The staff believes that the ROP has been successful in assuring

that plants are being operated and maintained safely. As previously noted in last year’s

consolidated response, the staff agrees that there are lessons to be learned from the

Davis-Besse event. There have been a number of program changes made to the ROP as a

result of the Davis-Besse Lessons-Learned Task Force (DBLLTF). A summary of NRC actions

is located on the agency website at http://www.nrc.gov/reactors/operating/ops-experience/

vessel-head-degradation/lessons-learned/lessons-learned-files/lltf-web-page-status022205.pdf.

Regarding the comment about providing additional focus on precursor issues, the NRC is

currently attempting to identify licensees which may have declining regulatory performance but

have not yet developed a risk significant performance deficiency. The purpose of identifying

cross-cutting aspects associated with NRC inspection findings is to identify safety culture

problems earlier in the process and to help prevent further licensee performance degradation.

The comment about the relatively high number of performance deficiencies and findings in the

cross-cutting areas was addressed in our response to question # 10.

The comment about implementing a continuous improvement process was addressed in our

response to question #10.
-38-

14. Do the ROP safety culture enhancements help identify licensee safety culture

weaknesses and focus licensee and NRC attention appropriately?

Respondent Comments:

Union of Concerned Scientists

The NRC staff did a remarkable job in this area. Their approach to safety culture is sound,

practical, and realistic. In addition, the revision to the ROP webpage to add indications of

reactors with substantive cross-cutting issues, including any in the safety culture area, greatly

adds to the public's understanding of who is having safety culture problems.

Pennsylvania Department of Environmental Protection

It is premature at this time to make any conclusions regarding the effectiveness of the ROP

safety culture enhancements. Safety culture is an important indicator of licensee performance.

The ROP's processes for detecting safety culture problems have not been very well defined,

although the NRC and the industry are making progress in this area. It would be a challenge

for the NRC to effectively incorporate this relatively subjective area into the ROP, which is

designed to be an objective process.

The NRC must routinely assess safety culture as part of the ROP Inspection Program and

develop the capability to determine, in a timely manner, whether safety culture problems are

contributing to performance problems. An important component of an effective regulatory

oversight of safety culture is to provide adequate training to the NRC inspectors, including

lessons learned from the event at Davis Besse plant.

South Texas Project Nuclear Operating Company

It is too early to determine this, but the NRC staff and licensee staff are spending a large

amount of time in this effort. Under the current safety culture program direction the NRC staff

identifies isolated, specific performance deficiencies and "bins" these to develop performance

trends. The current program does not consider and evaluate all the licensee actions or lack of

actions in response to the overall event. Ignoring the overall picture in favor of a small part of

the issue is not representative of actual plant safety culture and can give skewed and erroneous

conclusions.

The cross-cutting aspects in MC 305 are somewhat ambiguous. Certain aspects are too broad

to identify generic trends such as "Work Practices - procedure compliance (4b)" and

"PI&R - problem evaluation (1.c)."

New Jersey Division of Environmental Safety and Health

The NRC will never identify licensee safety culture weaknesses. These will come from workers

who voice safety culture concerns and raise them through the licensee organization and then

the NRC, if needed.

Nuclear Energy Institute

The ROP safety culture enhancements have only been recently implemented and it is too early

to determine if they identify safety culture weaknesses and appropriately focus licensee and

NRC attention. Under the current safety culture program direction, the NRC staff identifies

isolated, specific performance deficiencies and “bins” these to develop performance trends. 

The current program does not consider and evaluate all the licensee actions or lack thereof in

response to the overall event. Ignoring the overall picture in favor of a small part of the issue is
-39-

not representative of actual plant safety culture and can give skewed and erroneous

conclusions. In addition, it is important that the NRC Staff ensure consistency across regions

and from plant to plant within regions.

The cross-cutting aspects in MC 305 are somewhat ambiguous. Certain aspects are too broad

to identify generic trends such as “work practices – procedure compliance (4b)” and

“PI&R – problem evaluation (1c).”

Region IV Utility Group

It is too soon to make a conclusion. Early impressions are the NRC staff and licensees are

spending an inappropriate amount of time in this effort when compared to direct inspection of

plant activities. Continued monitoring and oversight by NRC management and licensees is

required to ensure the intended enhancements are realized. Consistent application of the cross

cutting aspects is critical. We appreciate NRC's support of the RUG IV and other industry

workshops as an important measure to help licensees understand the Safety Culture elements

and to help achieve consistent implementation of those elements.

Strategic Teaming and Resource Sharing

It is too soon to make a conclusion. Early impressions are the NRC staff and licensees are

spending an inappropriate amount of time in this effort when compared to direct inspection of

plant activities. Continued monitoring and oversight by NRC management and licensees is

required to ensure the intended enhancements are realized. Consistent application of the cross

cutting aspects is critical. We do appreciate the NRC's continued support of industry

communication forums to promote understanding and successful implementation of the safety

culture enhancements.

NRC Response:

As noted in the ROP self-assessment and metric report for AS-11, “Perceived Effectiveness of

Safety Culture Enhancements to ROP,” this was the first time that the external survey has

included a safety culture question. Several stakeholders commented that it was too early in the

implementation phase of the related program documents to answer whether the changes are

helping to identify safety culture weaknesses and focusing licensee and NRC resources

appropriately, while others complemented the NRC’s efforts and/or noted concerns as

discussed below.

As noted in Attachment 1 to SECY-07-0069, the staff's efforts to enhance the ROP to more fully

address safety culture are described in SECY-06-0122, "Safety Culture Initiative Activities to

Enhance the Reactor Oversight Process and Outcomes of the Initiatives," and RIS 2006-13,

"Information on the Changes Made to the Reactor Oversight Process to More Fully Address

Safety Culture." The staff implemented the enhanced inspection procedures and inspection

manual chapters on July 1, 2006, and implemented the revised guidance during the CY 2006

mid-cycle plant assessments. The final supplemental inspection procedure (IP) that was

enhanced as part of the safety culture initiative, IP 95003, "Supplemental Inspection for

Repetitive Degraded Cornerstones, Multiple Degraded Cornerstones, Multiple Yellow Inputs, or

One Red Input," was issued on October 26, 2006. The staff will compile lessons learned during

the initial 18-month implementation phase of the enhanced ROP.

Regarding the comment that the ROP process for identifying safety culture problems has not

been well defined, and that it is a challenge to effectively incorporate this subjective subject into

the ROP, the staff modified several inspection manual chapters (e.g., 0305 and 0612) to define
-40-

the process for addressing aspects of safety culture. For example, the NRC inspection staff

identifies those aspects that are important to safety culture in the baseline inspection program

within the three cross-cutting areas. The three cross-cutting areas remained the same. 

Although the “bins” within each of the cross-cutting areas were enhanced to better address

attributes of safety culture, the process for assessing them related to an inspection finding

remained the same. As part of the CY 2007 ROP self-assessment process, the staff will gather

insights from the 18-month initial implementation period (7/1/06 - 12/31/07) and will identify

follow-on ROP program enhancements related to the safety culture enhancements, as needed, 

in the associated Commission paper.

Regarding the comment that the NRC must routinely assess safety culture as part of the ROP

and develop the capability to determine, in a timely manner, whether safety culture problems

are contributing to performance problems, the staff believes the routine approach applied in the

baseline inspection program of identifying applicable cross-cutting aspects for inspection

findings and identifying substantive cross-cutting issues during the assessment process

provides a framework to identify issues related to aspects of safety culture in a timely manner.

Regarding the comment that an important component of safety culture oversight is a robust

training environment for inspectors, the staff provided training for inspectors that included

discussion of safety culture in general, safety culture contributors to the Davis Besse reactor

vessel head degradation, the Chernobyl accident, and the Columbia space shuttle accident. All

inspectors also received specific training on the safety culture enhancements to the ROP

including the use of case studies.

In response to the comment that the NRC staff identifies isolated, specific performance

deficiencies and "bins" these to develop performance trends without evaluating all the licensee

actions in response to the overall issue, that staff believes that the assessment process through

the implementation of the substantive cross-cutting issue criteria does in fact address the

adequacy of the broad scope of the licensee actions. The third SCCI criterion is specifically to

address whether the Agency has a concern with the licensee’s scope of efforts or progress in

addressing the cross-cutting theme. Application of this criterion ensures that broad licensee

actions are in fact considered.

Regarding the concern that the cross-cutting aspects are somewhat ambiguous and some are 

too broad to identify generic trends, the staff reviewed a wide spectrum of information from both

US and international sources to initially identify the safety culture components and their crosscutting aspects. The staff then refined the descriptions with the involvement of external

stakeholders. The staff plans to evaluate lessons learned from the 18-month initial

implementation period including the cross-cutting aspect descriptions.

With regard to the comment that the NRC will never identify safety culture weaknesses and that

they will normally manifest themselves when workers raise allegations, while it is true that

safety culture issues may become apparent during the evaluation of worker allegations, the

staff believes that the enhanced ROP where inspection findings are routinely evaluated to

determine whether a cross-cutting aspect was a significant contributor to the finding also serves

as a mechanism to identify performance problems related to aspects of safety culture.

Regarding the comment that continued monitoring and oversight is required and that consistent

application of the cross-cutting aspects is critical, the staff has implemented several monitoring

functions for the enhanced ROP including: the Safety Culture Focus Team which has overall

responsibility to promote consistent implementation of the process and has provided additional
-41-

guidance to the regions since the enhanced ROP was implemented; the IMC 0612 Working

Group which will perform an audit of issued inspection reports to assess the adequacy of 

cross-cutting aspect characterizations; and a RIV led effort to examine regional consistency

from the issuance of findings through assignment of cross-cutting aspects and cross-cutting

issues. The staff has also met monthly with the industry to engage in a dialogue about

implementation issues.

15. Is the ROP effective, efficient, realistic, and timely?

Respondent Comments:

J. Sniezek, Utility Consultant

It is efficient but not effective or realistic in identifying and reporting precursor issues. It

appears that the inspectors hands are tied. A good example is Davis-Besse event.

Union of Concerned Scientists

None. (I suspect that NRC staff assigned to read public comments are as tired by question 15

as I have become in typing them.)

Pennsylvania Department of Environmental Protection

Overall, the ROP is more effective and more realistic than the previous process however, there

are areas of improvement.

1. ROP Efficiency: The NRC and the licensees continue to expend significant amount of time

and resources in addressing findings that are greater than "Green", although some of these

findings do not have safety implications.

2. ROP Efficiency and Effectiveness: The PI Program, including the NRC verification of

licensee PIs, is resource intensive. This may not be an efficient use of resources since it is not

very clear as to whether the existing PIs provide useful insights to help ensure plant safety.

3. ROP Timeliness and Efficiency of SDPs: The NRC timeliness goals are not always being

met, which also questions the efficiency of SDPs.

4. ROP Effectiveness: The effectiveness of the ROP in identifying problems in the

cross-cutting areas, and particularly inspections of safety culture, is unknown at this time.

New Jersey Division of Environmental Safety and Health

• The ROP benefits the licensee but the ROP itself for the long term remains questionable

for being the right program for ensuring plant safety.

• With limited plants outside column one of the NRC action matrix and corresponding

limited fines by the NRC, we question the effectiveness, efficiency, and realism of the

ROP.

• As the NRC shifts its focus to new plants, the current operating reactors will take a back

seat and be vulnerable in a regulatory environment that seeks to maintain adequacy.

Nuclear Energy Institute

The ROP process is generally effective, efficient, realistic and timely with some exceptions.

“Generic” inspection findings should be communicated to the industry by generic

communications in a timely manner. The timeliness of the SDP process has improved.
-42-

Region IV Utility Group and Strategic Teaming and Resource Sharing

Overall we are in general agreement. However, the current CDBI inspections are consuming

substantial licensee resources. There appears to be a significant opportunity to improve the

efficiency of this process by applying more discipline to maintaining the schedule. The number

and significance of the findings to date do not seem to support the level of resource the

inspection requires. We suggest that scope and periodicity of the CDBI be reevaluated based

on results of the first round of inspections. Occasionally, exits are significantly delayed in time

from close of inspection activities onsite, resulting in inefficiencies in the process.

NRC Response:

As noted in the ROP self-assessment and metric report for O-5, “Stakeholders Perceive the

ROP To Be Effective, Efficient, Realistic, and Timely,” the respondents believe that the ROP is

effective, efficient, realistic and timely in comparison to previous programs. Some specific

comments are addressed below.

The comment about providing additional focus on precursor issues was addressed in the

response to question #13.

Regarding the comment that the PI program, including the NRC verification of licensee PIs, is

resource intensive, IP 71151, “Performance Indicator Verification,” estimates the effort to

complete all annual PI verifications to be 40 to 55 hours per year for a one-unit site, 60 to 75

hours for a two-unit site, and 65 to 75 hours for a three-unit site. A total of 2,148 hours were

charged to IP 71151 during the CY 2006 inspection cycle out of 150,995 total direct inspection

hours (this equates to 1.4 percent of the total baseline inspection hours).

A staff initiative identified as “Regional Best Practices for Managing SDP Timeliness,” has been

completed with recommendations covering this objective. While the intent of this objective has

been met by all regions and regional efforts have been successful in meeting the timeliness

metric for 2006, the need for a common or uniform tracking system is under evaluation. 

Additional comment about SDP timeliness was addressed in the response to question #7.

Regarding the comment about limited plants being outside column one of the NRC action

matrix and corresponding limited fines by the NRC, during the formation of the ROP back in the

1998-2000 timeframe, the industry argued, and the Commission agreed, that acceptable plant

performance should be measured against the industry's 1995-1997 performance. Since that

time, industry performance has improved significantly, and therefore a large majority of plants

operate within the licensee response column of the ROP's Action Matrix. Additionally, the staff

and industry knew that the enforcement process of the old inspection program did not always

conform to the significance of the inspection finding. Consequently, these problems were

addressed by revamping the enforcement program to incorporate risk insights and reduce or

eliminate punitive or subjective enforcement. Risk-informing the Enforcement Program also

had the impact of a reduction in the number of fines. These aspects of the ROP do not in any

way impact the goals and objectives of the ROP, and in fact, should increase the efficacy,

efficiency, and the public confidence in the ROP.

As noted in SECY-07-0069, the NRC shares the concern that the increased focus on new

plants may have an adverse impact on the oversight of operating reactors and plans to closely

monitor resident demographics and site staffing in 2007 due to anticipated influences on the

program as a result of the projected expansion of the nuclear industry and our own growth to

support the formation of the Office of New Reactors. The staff performed a review to ensure
-43-

that the effective oversight of operating facilities is not adversely affected by the new reactor

construction inspection program. The results of this review are documented in a memorandum

dated May 16, 2007, and are publicly available (reference ADAMS accession number

ML070850158). The staff plans to perform this review annually to monitor potential adverse

impacts in accordance with the April 21, 2006, staff requirements memorandum for

SECY-06-0041, “Proposed Strategy to Support Implementation of the New Reactor

Construction Inspection Program.”

Regarding the comment that the scope and periodicity of the Component Design Basis

Inspection (CDBI) should be reevaluated based on results of the first round of inspections, as

noted in Enclosure 1 to SECY-07-0069, CDBIs were developed to improve the effectiveness of

NRC design/engineering inspections based on lessons learned from past inspections and

events. The intent of these inspections is to focus on risk-significant, low-margin components

and operator actions that could potentially affect risk-significant structures, systems, and

components. Thirty-seven CDBIs have been completed or are underway, and 29 remain for the

current ROP cycle. There have been 89 Green inspection findings to date. These CDBI

findings are being evaluated to identify generic issues and areas of emphasis for future ROP

inspections. In addition, the staff plans to evaluate potential revisions to the scope and

frequency of the CDBI during CY 2007.

Regarding the comment that occasionally exits are significantly delayed in time from close of

inspection activities onsite, clarification and/or specific examples would be needed to address

this comment. The staff is open to further discussion on this issue at a future ROP working

group meeting.

16. Does the ROP ensure openness in the regulatory process?

Respondent Comments:

Pennsylvania Department of Environmental Protection

Some aspects of the ROP are inconsistent with the NRC goal of openness in the regulatory

process. For example, the plant specific PRAs are not available for public review and scrutiny.

The SDP is a complex process and inspection findings that use this process are not generally

open to public scrutiny. Additionally and for obvious reasons, information regarding security

assessments are not being shared with the members of the public.

South Texas Project Nuclear Operating Company

The ROP process with its many public meeting and opportunities for involvement insures

openness not available in the previous process.

New Jersey Division of Environmental Safety and Health

The key decision making for the important plant findings are usually conducted without

stakeholder involvement.
-44-

Nuclear Energy Institute

There is significant dialogue between the industry and the NRC at the ROP meetings with good

discussion on both sides.

Region IV Utility Group

The ROP process with its many public meetings and opportunities for involvement ensures

openness not available in the previous process. The NRC website is useful and open to the

public.

Strategic Teaming and Resource Sharing

The ROP process with its many public meetings and opportunities for involvement ensures

openness not available in the previous process.

NRC Response:

As noted in the ROP self-assessment and metric report for O-6, “Stakeholders Perceive That

the ROP Ensures Openness,” external stakeholders generally acknowledged that the ROP

ensures openness in the regulatory process, but some noted that further improvements could

be made.

Regarding the comment that the plant specific PRAs are not available for public review and

scrutiny, as a result of the terrorist attacks on 9/11, public access to the risk-informed

inspection notebooks used for the SDP Phase 2 process for at power conditions and plant

specific PRAs has been restricted. As such, the ability of the public to engage in open

communications about plant specific probabilistic risk information has been reduced.

With regard to the openness of the SDP and decision making for important plant findings, the

Significance and Enforcement Review Panel (SERP) provides a management review of

pre-decisional information in the process of the preliminary determination of significance

associated with a finding. Since the information reviewed by the SERP is pre-decisional, the

process is not open to the public and licensees. However, details of how the process was

applied to each finding are incorporated into the publicly available inspection report. Details of

the SERP process are described in IMC 0609.01, “Significance and Enforcement Review Panel

Process.”

Regarding the comment about the openness of the security assessments, the NRC continues

to reevaluate certain aspects of the closed nature of its security oversight program for nuclear

power plants. For example, in 2006 the NRC began making the cover letters for security

inspection reports publicly available, although the details of any related findings were not

included in the letters. As the staff continues to strive for the right balance between public

scrutiny and ensuring common defense and security, it will consider opportunities to make

additional security-related information available to the public.
-45-

17. Has the public been afforded adequate opportunity to participate in the ROP and

to provide inputs and comments?

Respondent Comments:

Pennsylvania Department of Environmental Protection

The NRC has been seeking public input on the ROP but overall, the level of participation by the

public has been very low and public confidence in the process does not appear to be

increasing. Some of the contributing factors are the complexity of the ROP, particularly as it

relates to the use of PRAs and the SDPs, and the lack of confidence in the NRC's public

participation process. Additionally, the NRC assessment meetings and enforcement

conferences are very structured and lack the flexibility to effectively seek input from various

stakeholders, other than the industry.

It is recommended that NRC develop and implement an effective mechanism to receive public

input continuously and on a "plant specific basis". The NRC resident inspectors can and should

play a more active role in the agency's public involvement activities within the local

communities. The posting of plant specific information on the NRC Website can help improve

public confidence in the process and should continue. Unnecessary changes to the ROP may

reduce public confidence in the process and should be avoided.

South Texas Project Nuclear Operating Company

Members of the public and media are frequently present at the monthly ROP meeting.

New Jersey Division of Environmental Safety and Health

• Most of the hundreds of meetings that take place between the NRC and the licensee

never have public attendance.

• The use of more teleconferencing and webcasting will help this concern tremendously.

Nuclear Energy Institute

The public has been afforded adequate opportunity to participate in the ROP and to provide

inputs and comments. Monthly public ROP meetings are held to discuss improvements and

FAQs. Members of the public are frequently present at these meetings.

Region IV Utility Group

Members of the public and media are frequently present at the monthly ROP meeting. It is

important to closely follow process and ensure that all changes to ROP elements are properly

addressed in an open forum (e.g., change to IMC 0612 definition of performance deficiency).

Strategic Teaming and Resource Sharing

Members of the public and media are frequently present at the monthly ROP meeting.

NRC Response:

As noted in the ROP self-assessment and metric report for O-7, "Opportunities for Public

Participation in the Process," most respondents acknowledged the ample opportunities for

public participation (monthly public meetings at NRC headquarters, annual public meetings

conducted in the reactor communities, annual solicitation of public comments, annual ROP

Commission briefing, and the staff’s consolidated response to last year’s comments). The

noted comments are addressed below.
-46-

Regarding the comment that the NRC assessment meetings and enforcement conferences lack

the flexibility to effectively seek input from various stakeholders, the NRC staff needs to be able

to have frank and open discussion on all aspects of plant performance during the regional

internal mid-cycle and end-of-cycle (EOC) assessment meetings and enforcement conferences,

much of which is pre-decisional, and should not be discussed in a public setting for obvious

reasons. From a program and a procedural standpoint, the NRC staff is always open to

stakeholder feedback and we welcome comments through the external surveys and various

routine public meetings held, such as the monthly ROP public meeting. Additionally, the annual

public meeting after the EOCs are Category 1 meetings, meaning the public is invited to

observe the meeting between the NRC and the licensee and will have the opportunity to

communicate with the NRC staff before the end of the meeting.

Regarding the recommendation that the NRC develop and implement an effective mechanism

to receive public input continuously and on a "plant specific basis," and that the NRC resident

inspectors should play a more active role in the agency's public involvement activities within the

local communities, as noted in the 2005 consolidated response, the NRC staff is open to public

input on a plant specific basis, either verbally or in writing. We routinely receive

correspondence, e-mails, and telephone calls from members of the public, either to express

opinions, seek agency action, or to seek information. We agree that the NRC resident

inspectors play an important role as an interface with the local public, however we believe that

they should be focused on carrying out safety inspections at the plant.

Regarding the recommnedation that more teleconferencing and webcasting should be used for

public meetings, many of the NRC's public meetings can be accessed via teleconferencing

and/or webcasting. As noted in NUREG/BR-0297, "NRC Public Meetings," the NRC will make

a reasonable effort to provide teleconferecing access for meetings which are not easily

accessible to interested citizens. Information about teleconferencing is provided in individual

meeting notices. In addition, a listing of upcoming meetings are available on the NRC's Web

page at: http://www.nrc.gov/public-involve/public-meetings/index.cfm.

18. Has the NRC been responsive to public inputs and comments on the ROP?

Respondent Comments:

Pennsylvania Department of Environmental Protection

There is at least a perception on the part of the public that the NRC does not value their input.

One of the contributing factors is that the NRC has been generally slow to respond to public

inputs and comments. Since the inception of the new ROP, the public has expressed concerns

regarding the effectiveness of the PIs (and the PI thresholds), the timeliness of the SDP

findings, the lack of standardized risk analysis tools, the effectiveness of the ROP to identify

problems in the cross-cutting areas, the disproportionate reliance on the licensees' CAP to

resolve problems in a timely manner, etc. The NRC has taken measures to address some of

these issues or concerns however, the agency's response has been generally slow.

South Texas Project Nuclear Operating Company

The safety culture public meetings are a good example of this. The outside stakeholders

played a large part in developing the safety culture initiative process guidance.

New Jersey Division of Environmental Safety and Health

• The NRC responds to every public input because they are required to do so.
-47-

• The NRC rarely changes anything as a result of public input.

Region IV Utility Group and Strategic Teaming and Resource Sharing

The safety culture public meetings are a good example of this. The outside stakeholders

played a large part in developing the safety culture initiative program guidance. We also

appreciate the NRC staff's consideration of feedback provided in the 2005 ROP survey.

NRC Response:

As noted in the ROP self-assessment and metric report for O-8, "Stakeholders Perceive the

NRC To Be Responsive to its Inputs and Comments," the majority of stakeholders believe that

the NRC is responsive to inputs and comments; however, some stakeholders commented that

the NRC does not value public input and rarely changes anything as a result of their input. The

NRC disagrees with these perceptions. The NRC actively seeks feedback from the public

regarding the ROP through external surveys, public meetings, and other venues. Several

program improvements have been implemented as a result of public feedback. In fact, other

respondents to this very question noted that the outside stakeholders played a large part in

developing the safety culture initiative program guidance and that they appreciated the NRC's

consideration of feedback provided in previous surveys.

As noted in Enclosure 3 to SECY-07-0069, the staff reviewed all of the survey responses and

evaluated the stakeholder comments as part of the annual self-assessment. The staff also

prepared this consolidated response to the CY 2006 external survey as was done for CYs 2004

and 2005. In addition, to ensure continued openness and responsiveness to the public's inputs

and comments on the ROP, the staff plans to complete this task in response to future external

surveys, and revised IMC 0307 to institutionalize the process. As in previous years, the staff

acknowledged receipt of each FRN response by correspondence indicating that the staff has

considered and generally addressed the comments in this paper. In addition, the annual ROP

self-assessment paper (SECY-07-0069), the annual ROP performance metric report, and the

consolidated response will be posted to the ROP Web page and sent along with the

acknowledgment letters to each survey respondent.

Regarding the comments concerning the slow response to public concerns, the NRC maintains

that the ROP is a complex process and program changes take a significant amount of time to

implement effectively. The staff continues to make improvements to the ROP based on

feedback and lessons learned.

19. Has the NRC implemented the ROP as defined by program documents?

Respondent Comments:

Pennsylvania Department of Environmental Protection

Overall, it appears that the NRC has implemented the ROP as defined by the NRC program

documents.

South Texas Project Nuclear Operating Company

The NRC staff is diligent in following ROP Program documents.
-48-

New Jersey Division of Environmental Safety and Health

The licensee ensures that the NRC implements the ROP per procedure because the licensee

co-authored the ROP with the NRC.

Region IV Utility Group

See our comments on Items 5 and 12 above.

NRC Response:

As noted in the ROP self-assessment and metric report for O-9, "Stakeholders Perceive That

the ROP Is Implemented as Defined," most external stakeholders including utilities, state and

other agencies, and the public, believe the ROP is being implemented as defined.

20. Does the ROP minimize unintended consequences?

Respondent Comments:

Pennsylvania Department of Environmental Protection

It might be premature at this time to conclude that the ROP "minimizes" unintended

consequences. Based on our experience in PA and as it relates to plant safety, it can be

concluded that as of now, the ROP has not resulted in any unintended consequences.

South Texas Project Nuclear Operating Company

For the most part this is true, but the policy on press releases does cause unintended

consequences since follow-up press releases for special inspections and SDP results are not

routinely made. This policy can cause an incomplete, and at times an incorrect, public

perception of issues.

New Jersey Division of Environmental Safety and Health

The ROP minimizes NRC oversight resources.

Nuclear Energy Institute

The ROP does minimize unintended consequences. However, the policy on press releases

causes unintended consequences because follow-up press releases for special inspections and

SDP results are not routinely made. This policy can cause an incomplete and at times incorrect

public perception of the issues.

Region IV Utility Group and Strategic Teaming and Resource Sharing

Current licensee experience finds that on occasion, significant time is expended on minor

issues. Improved discipline in this area is needed to reduce the unnecessary regulatory burden

that can occur when efforts are not well focused.

Press releases are typically made in advance of NRC special inspections. However, there are

not always press releases or follow-ups made by NRC upon closure of the inspection informing

the public of the results. This has the potential unintended consequence of raising an issue to

public attention without resolving the issue in the same public arena.
-49-

NRC Response:

As noted in the ROP self-assessment and metric report for O-10, "Stakeholders Perceive That

the ROP Does Not Result in Unintended Consequences," in a change from previous external

surveys, the majority of stakeholders responding indicated that they believe the ROP does not

result in unintended consequences, with a few concerns noted as discussed below.

Regarding the comment that the policy on press releases does cause unintended

consequences since follow-up press releases for special inspections and SDP results are not

routinely made, this issue has been discussed at an ROP working group meeting and

discussions are ongoing between the NRC's and NEI's public affairs offices.

Regarding the comment that significant time is occasionally expended on minor issues and that

improved discipline is needed to reduce the unnecessary regulatory burden, clarification and/or

specific examples would be needed to help understand what is meant by minor. The word

"minor" has a special meaning in the inspection program, although others in the public forum

may use it more freely. The staff is open to further discussion on this issue at a future ROP

working group meeting.

21. Would you support a change in frequency of the ROP external survey from

annually to every other year, consistent with the internal survey, as proposed in

SECY-06-0074?

Respondent Comments:

Union of Concerned Scientists

UCS opposes any move by the NRC to distance itself from the public, whether in

space or in time. If anything, the frequency of the ROP external surveys should be changed to

quarterly from annually for consistency with the ROP update cycle. If the NRC does back off on

how often it solicits feedback from the public, the NRC should also back off on how often it

meets with industry to undermine the effectiveness of the ROP by the same factor. For

example, if the NRC doubles the interval between ROP surveys of the public from annually to

every other year, the NRC must likewise increase the internal between NRC/NEI meetings from

monthly to every other decade.

Pennsylvania Department of Environmental Protection

We support a change in the frequency of the ROP external survey from annually to biennially.

However, the NRC should continue to seek input from the stakeholders before it decides to

make any significant changes or modifications to the existing process. Also, see comments on

question # 22.

South Texas Project Nuclear Operating Company

The input on a yearly basis keeps stakeholders involved. Allowing issues to go for two years

without collecting concerns would hamper the process for improving the ROP.

New Jersey Division of Environmental Safety and Health

NRC changes as a result of the survey seem to be minimal, however a survey is the only

avenue for the public to voice its opinion. Every other year is an acceptable time frame if the

NRC carefully reviews all input and makes changes accordingly.
-50-

Region IV Utility Group

Significant changes in the ROP were introduced in 2006; specifically MSPI and safety culture

enhancements. There is a need for the continued stakeholder involvement this survey affords.

Allowing issues to go for two years without collecting feedback may not be consistent with the

continued success of the ROP driven by continuous improvement.

Nuclear Energy Institute

While we agree in concept with changing the ROP external survey from annually to every other

year we acknowledge that yearly does allow stakeholders to be involved in a timely manner.

Also, if there are significant changes in the ROP we believe it would be prudent to solicit

stakeholder responses in the year that they are made rather than waiting an extended period of

time.

Strategic Teaming and Resource Sharing

Significant changes in the ROP were introduced in 2006; specifically MSPI and safety culture

enhancements. There is a need for the continued stakeholder involvement this survey affords.

Allowing changes to go for two years without collecting feedback is not consistent with the

continued success of the ROP, fostered in part by this feedback.

NRC Response:

As noted in Enclosure 3 to the SECY-07-0069, the staff proposed a change in the frequency of

the external survey to every other year, consistent with the internal survey. This change was

suggested in order to gain further efficiencies, and because the comments and staff analysis

have tended to repeat the same themes from year to year. Half of the respondents indicated

that they agreed with the change in frequency, while half indicated that they disagreed,

including a few who expressed concerns with changing the frequency. As further noted in

SECY-07-0069, there was also a notable decline in the level of participation from previous

years’ surveys. As a result, the staff plans to conduct the external survey in CY 2007 and

revise IMC 0307 to change the frequency to every other year. As such, one year’s ROP

performance metrics and self-assessment would include survey inputs and analysis from

internal stakeholders, and the following year would include external survey inputs and analysis.

Regardless, internal and external feedback will be considered each year based on continuous

feedback during meetings, the feedback process, and other venues.

In addition, the staff will continue to solicit and consider stakeholder feedback for significant

ROP changes (e.g., safety culture, MSPI, etc.). As in previous years, the staff will acknowledge

receipt of each FRN response by correspondence indicating that the staff has considered and

generally addressed the comments in this paper. In addition, this paper, the annual ROP

performance metric report, and the consolidated response will be posted to the ROP Web page

and sent along with the acknowledgment letters to each survey respondent.
-51-

22. Please provide any additional information related to the Reactor Oversight

Process*

*Note to better address the comments: The staff evaluated these comments according to the

ROP program areas (performance indicators, inspection, significance determination process,

and assessment) that they were related to. The responses were then grouped, assessed, and

addressed by the appropriate ROP program area leads within their respective areas of

expertise. Comments received that were outside of the ROP program areas are addressed

under the category of other. This section also includes comments that were in the cover letter

or other portions of the survey response that were not directly related to a specific question.

Performance Indicator Section

Respondent Comments:

Union of Concerned Scientists

As subtly suggested by our comments, UCS believes the value of the ROP has significantly

diminished since its inception in 2000. The performance indicators are now virtually useless.

Data presented by the recent GAO report shows that the PIs are green over 99 percent of the

time (and the industry, with time, will green the remaining <1 percent).

Southern California Edison

• SCE remains concerned with continuing proposals to revise upward some of the PI

thresholds. Changing the PI thresholds in such a manner would impose a de facto

"rising standard." SCE supports the original NRC position that the thresholds were set

with the expectation that, while licensee performance would be expected to improve,

performance at the current thresholds represents "acceptable licensee performance.”

• The Mitigating Systems Performance Index (MSPI) effort has been a lengthy and

difficult process and the resultant MSPI has evolved well beyond what was originally

piloted. SCE believes, therefore, that the NRC should conduct a "lessons learned"

evaluation of the MSPI effort before further changes are made.

NRC Response:

The comment regarding PIs being green over 99 percent of the time has been addressed in our

response to question #1.

Regarding the comment that changing the thresholds would impose a rising standard, the

staff’s focus has been that the PI Program should contribute to the detection of declining

licensee performance. This can be by introducing new PIs or by revising existing PIs. Over the

last several years, a number of PIs have been modified or replaced, and the staff has focused

on setting the appropriate thresholds to detect declining licensee performance.

Regarding the comment that MSPI has evolved well beyond what was originally piloted and that

the NRC should conduct a "lessons learned" evaluation of the MSPI, the staff agrees that an 
-52-

implementation review is warranted, and will be conducting an extensive review of the MSPI

over the course of FY 2007 through the end of FY 2008. Recommendations resulting from the

review will be presented to the ROP Working Group for resolution and potential revision of the

MSPI.

Inspection Section

Respondent Comments:

J. Sniezek, Utility Consultant

The current process, including inspection reports do not provide the public with insight into

actual performance issues with the plants. The inspection reports are sterile and do not identify

the precursor type issues discussed between the inspectors and site personnel. They

(inspection reports) give the impression that the plants are stellar performers

(unless a significant issue is found). The inspection reports are virtually not worth reading by

people looking for performance insights. The foregoing said – I believe the plants are generally

operating with good attention to safe performance.

F. Baxter, Public

These comments are limited to my personal experiences and insights identified in the course of

conducting four CDBI Inspections in the 2005/2006 time frame as an NRC contractor.

A major weakness in the Region-based CDBI inspections (which are an integral part of the

ROP), has been the inability or unwillingness to bypass pre-conceived and preexisting

roadblocks, and mind-sets. Though these impediments vary from Region to Region, they have

been instrumental in preventing the identification and documentation of many potentially

significant issues at nuclear power plants, which in my view could be precursors to potential

accidents.

Some of these roadblocks and mind-sets are identified below:

1. High-risk is the cornerstone of ROP, and appears to be directing the Region based

inspection efforts without exception; if a finding does not meet the threshold criteria of high-risk

it is a likely to be dismissed. With this preconditioning, low-risk components do not have a

chance of being recognized in the ROP. In addition, the ROP uses risk-based parameters to

determine inspection review areas, as well as safety significance components. The underlying

concern here is, that these high-risk categories may not by themselves be sufficient to

determine accident risk, accident scenarios, and safety importance. Low-risk components,

scenarios, and sequences must also be factored into the ROP process because they are the

ones that will most likely contribute to the next accident because of their frequency of

occurrence and inattention by ROP.

An accumulation of low-risk parameters are capable of synergism and cause a serious incident

or accident. It is therefore not inconceivable for a nuclear accident to be initiated by a

sequence or combination of low-risk events that are individually considered low-to-no-risk, and

therefore inconsequential. Two examples are provided below where a number of low-risk

parameters came together to cause major catastrophes. It is interesting to note that in

retrospect, had these accumulations of low-risk inactions been collectively analyzed through

PRA methodology, they probably would have raised a flag - but this is unfortunately not part of

the ROP today.
-53-

a. The Columbia Space Shuttle accident was initiated by two events that had been categorized

as low-risk. The first, loss of foam, had been determined by NASA to be a maintenance related

item with low-to-no flight risk, and NASA had stated for years, with unwavering conviction, that

foam strikes to the space shuttle presented low-to-no flight risk. The second, loss of the

thermal protection system, also a routine flight occurrence had also been categorized as an

acceptable low risk event. However, the combination of these two low-risk events, where foam

pieces struck and removed the thermal protection tiles was the undoing of Columbia.

b. The recent Comair airline crash, where the pilots took off from the wrong runway, was

caused by a combination of relatively low-risk human errors. No individual occurrence by itself

could fall into the category of being high-risk. The low-risk human actions and inactions leading

up the Comair accident were:

• The pilots were not given clear alerts on recent changes to runway reconfiguration and

access requirements.

• Both pilots did not visually recognize that they were on the wrong runway.

• Both pilots failed to observe that their instruments indicated that they were aligned to the

wrong compass heading for take-off.

• Only one of two air traffic controllers was on duty.

• The one air traffic controller on duty was overworked, and thereby did not to observe

that the aircraft was on the wrong runway.

• And finally, both runways were poorly marked, and poorly lit.

The point should be clear - low risk components should not be summarily dismissed from

consideration as inconsequential; they may become significant factors leading to accidents

through synergism, and therefore should be made part of ROP.

2. A potential finding may be identified where the design and licensing basis had not been

implemented; however, the SER written by the NRC when the plant was licensed, stated

directly or indirectly, that the subject of contention was reviewed and found to be acceptable.

Both licensees and the NRC have used this argument on numerous occasions as valid grounds

for not pursuing and correcting an issue of potential safety significance.

The licensee has ultimate responsibility for implementing their design and licensing basis, and if

errors or omissions are subsequently identified in the implementation of the design basis, the

licensee has an obligation to correct them irrespective of what the SER may or may not say.

The ROP must acknowledge that the original SER may be have been flawed.

3. Findings of no color are frequently reported to the licensee but are not documented in the

inspection report. These findings are basically left to the licensee's discretion (and integrity) for

follow-up. The licensee usually writes a condition report, but is not obligated to take any

corrective action if the condition report does not call for it. Therefore, an issue identified by the

NRC may be disregarded entirely by the licensee based on inappropriate or unsound

conclusions. The fact that these findings of no color are not undocumented, nor tracked or

trended by the NRC, is a significant weakness in the ROP.

4. On identifying a finding, it is frequently noted by licensees that other nuclear plants have

identical or similar design shortcomings; on this basis the finding is identified as an

industry-wide generic issue, and is often not pursued further by the NRC.
-54-

5. An issue of concern that had been brought up numerous times during past inspections is

dropped because it is determined not to be cost-effective to pursue once again. Past reviews

do not necessarily make a problem go away, it remains a problem until corrected and closed. It

is possible that the past reviews were not rigorous, or perhaps they were not presented with the

required technical clarity to identify the safety significance.

6. Findings are sometimes dismissed from further consideration based on the conclusion that

by making the licensee correct the shortcoming, this would constitute a backfit.

Examples have arisen where a licensee failure to comply with their own design basis has been

termed a backfit issue, and dropped from further consideration.

Some of these pre-conceived and pre-existing roadblocks, and mind-sets, have existed for

many years and therefore may be difficult to overcome; however, the ROP process should

provide specific direction and guidance in these areas in order to make the process more

effective in accomplishing its objective and ensuring the safety of nuclear power plants.

Nuclear Energy Institute

• Several plants commented that the inspectors are straying outside guidance and

ignoring safety significance in raising issues.

Florida Power and Light

In addition to the above comments, FPL provides the following comments regarding two

specific definitions contained within NRC Inspection Manual Chapter 0612, Section 0612-03,

"Definitions":

"NRC-Identified" - The definition should be revised to read along the lines of "Added

value means that the inspector has identified previously unknown significant

weaknesses in the licensee's classification, evaluation, or corrective actions associated

with the licensee's correction of a finding after the licensee has completed the particular

activity in question." In this regard, the supporting comment for the definition should

address the situation in which an inspector asks questions or makes observations that

do not result in identification of unknown significant weakness in a finished product

(i.e., would not meet the definition of added value).

"Performance Deficiency" - Some clarification should be made in the last two sentences

of the first paragraph pointing out that self-imposed widely accepted industry standards

or practices are not necessarily regulatory requirements.

NRC Response:

The comment about providing additional focus on precursor issues was addressed in the

response to question #13.

Regarding the comment that the inspection reports are virtually not worth reading by people

looking for performance insights, inspection reports document identified performance

deficiencies at the facility. The inspection reports are not meant to communicate diagnostic

types of information about licensee performance.
-55-

With regard to the comments that low risk component failures can combine to cause a

significant event, the CDBI inspections, by design, focus on components and operator actions

of relatively high risk significance. These inspections are unique in the level of effort and the

use of contractors. The intent of the inspection, in part, is to assess the engineering work

behind the plant design. To effectively accomplish this, the focus of the CDBI should be on

areas of safety significance. However, other inspections within the ROP sample systems,

structures, and components (SSCs) of lower safety significance, therefore this category of

SSCs is included in the ROP.

With regard to the comment that errors in licensing issues should be corrected, the agency has

a process to correct such errors. Under the backfit process, the NRC can change the licensing

basis of a plant, however, by regulation, the NRC must demonstrate that the benefit justifies the

change.

With regard to findings of “no color,” when the ROP was established, one of the principle

building blocks of the ROP was that minor issues would not be documented, but would be

turned over to the licensee for review within their corrective action program. The reason for this

is to prevent NRC inspections from unduly directing licensee management time and attention to

issues of little or no significance. An inspection of the licensee’s corrective action program is an

element of the baseline inspection program. The staff considers this approach to be

appropriate.

Regarding issues that are considered generic, and therefore not pursued, if these issues

involve a performance deficiency, they can be documented as unresolved pending the NRC

reaching a decision on how best to address such issues. In this manner, the issues can be

further developed at a later date.

With regard to inspection issues being “dropped” because it is not cost-effective to pursue, it is

difficult to assess this issue without specific facts. However, as a general practice, decisions

must be made during inspections regarding which inspection issues to pursue. Unless

significant safety issues arise, the amount of time available for the inspection has a limit. On

this basis, it is appropriate to not pursue some issues in order that issues of more likely

significance can be fully developed.

Regarding the proposed revisions to IMC 0612 to clarify the definitions of “NRC-identified” and

“performance deficiency,” the NRC staff believes that the current definition of “NRC-identified”

is clear. However, we are developing an improved definition of what is meant by

“self-revealing” and how this definition is different from “NRC-identified.” The “performance

deficiency” definition was revised in late 2006 as noted in the response to question #5 and we

do not currently plan to make any changes to this definition.

Significance Determination Process Section

Respondent Comments:

Union of Concerned Scientists

As subtly suggested by our comments, UCS believes the value of the ROP has significantly

diminished since its inception in 2000. The inspection findings remain about the only useful

component of the ROP, but chronic lack of timeliness in figuring out which crayon to use for
-56-

greater-than-green findings (and there are only three choices - red, yellow, green -so it

shouldn't take months to a year to figure it out) lessens their value.

New Jersey Division of Environmental Safety and Health

Much of NRC ROP decision making is based on probabilistic risk assessments (PRA) under the

control of each nuclear power plant operator. These PRAs are vital for ensuring that the right

NRC resources are utilized in the most risk significant areas. Our concern continues to be the

quality of these PRAs. The NRC has yet to ensure PRAs are reliable, accurate and up-to-date.

Florida Power and Light

FPL supports the NRC's ROP-related initiatives to improve the Significance Determination

Process (SDP), as set forth in NRC Inspection Manual Chapter 0609 (IMC 0609), and

timeliness in issuing final SDP results. These initiatives include the incorporation in November

2005 of the Planning Significance and Enforcement Review Panel (SERP) process into

IMC 0609, the issuance of the Phase 2 SDP notebooks (Revision 2) in December 2005, and

the efforts to further enhance Standardized Plant Analysis Risk (SPAR) models to provide a

more enhanced Phase 3 SDP tool for evaluating at-power internal events consistent with the

licensee's probabilistic risk assessment (PRA).

One of the original objectives of the ROP was to improve the objectivity of the reactor oversight

process, i.e., so that subjective decisions and judgments are not central to the process

(see NRC Significance Determination Process - Task Group Report (TGR), December 13,

2002, page 4). The NRC routinely uses engineering judgment in formulating its assumptions

on whether a degraded structure, system or component would have performed its safety

function given a specific demand. Also, when performing Phase 3 SDP assessments of final

significance, the NRC often introduces hypothetical scenarios involving what could have

happened -- even though such hypothetical scenarios are a departure from the facts of what

occurred, the actions taken by the licensee in response to the actual event and the actual

consequences of the event (if any). These practices can introduce an outcome-determinative

subjective element in to the SDP risk significance determination.

FPL respectfully submits the SERP process should include, especially for more complex issues

involving Phase 3 SDP assessments, more specific guidelines to ensure that the final

significance characterization is consistent with the original premise of the ROP of providing an

objective process based upon supported facts and actual consequences (i.e., rather than upon

the possibility of what theoretically "could have happened" no matter how improbable the

postulated scenario). Such guidance would help to ensure that assumptions that are influential

to the significance result (i.e., will cause the color to vary) are both objective and realistic and

could include guidelines on determining when the results of a licensee's PRA are of sufficient

quality for use in the final significance determination. The guidelines could be expressly

incorporated, for example, in the SERP Worksheet for SDP-Related Findings

(Exhibit 4 of Attachment 1 to IMC 0609).

Southern California Edison

• Difficulties continue to be experienced with the development, precision, and robustness

of several of the Significance Determination Processes. Several SDPs are not as robust

as they should be, and do not produce consistent and/or meaningful results.

• The NRC has initiated efforts to improve SDP timeliness and we support this effort. The

Commission is establishing goals for SDP timeliness at 90 days. There may, however,

occasionally remain complex engineering judgment issues that may take longer than 90
-57-

days to resolve. SCE believes the NRC should give a licensee sufficient time to

determine the actual facts and circumstances of an event, even if that would require

waiving the 90-day goal for such exceptional cases.

• The NRC should make additional use of licensee-developed Probabilistic Risk

Assessment (PRA) models that meet the necessary quality standards. In those cases

where the SDP would require evaluation beyond the screening criteria, the NRC should

move directly to a Phase III evaluation using the licensee's plant specific PRA.

NRC Response:

As noted in Attachment 1 to SECY-07-0069, licensee PRA quality continues to be enhanced

through the benchmarking of the plant specific risk-informed inspection notebooks, by the

improvements of the NRC Standardized Plant Analysis Risk (SPAR) models, and by the

availability of additional guidance, including Regulatory Guide 1.200, “An Approach for

Determining the Technical Adequacy of Probabilistic Risk Assessment Results for

Risk-Informed Activities,” NUREG CR-6823, “The Handbook of Parameter Estimation for

Probabilistic Risk Assessment,” and NUREG-1792, “HRA Good Practices.” Implementation of

the risk-informed Mitigating Systems Performance Index in April 2006 has further focused

licensee and NRC resources on the importance of PRA quality.

More recently, the staff has developed additional guidance for SDP risk modeling, commonly

referred to as the RASP handbook [Risk Assessment Standardization Project]. The RASP

handbook provides guidance and standard approaches for risk modeling and for recognizing

when modeling uncertainties warrant a more in-depth analysis to characterize the significance

of an inspection finding or an operating event. The RASP handbook will be updated on a

continuing basis to include additional examples that address uncertainty analysis for both

modeling and parameter uncertainty associated with the Significance Determination Process,

and event assessment for Management Directive 8.3 decision-making. This document is under

NRC trial-use and is undergoing additional review by RES and NRR staff. Once finalized, the

RASP manual will be publically released.

The comments regarding SDP timeliness were addressed under our response to question # 7.

The comments regarding concerns with PRA quality were addressed under our response to

question #7.

Regarding the comments on the use of engineering judgement and hypothetical scenarios

introducing an outcome-determinative subjective element to the SDP risk significance

determination, the SDP is not a risk-based process but one that is risk-informed and

performance-based. The SDP is considered risk-informed by virtue of the expectation that SDP

result bases are sufficiently understood by those technically knowledgeable persons

(such as inspectors and technical staff) who are best positioned to critically examine the most

influential probabilistic and technical assumptions, as well as by the management

decision-makers who ultimately make the decisions. A “risk-informed” approach should

consider “other (unspecified) factors.” Historically such “other factors” included those listed in

Regulatory Guide 1.174 such as maintaining defense-in-depth, compliance with regulations,

engineered safety margins, and prevention of over-reliance on human operators for rapid

critical decisions. However, these factors are all already represented, in various ways, in a

probabilistic risk model. Other “factors,” such as NRC management assessment of the general

quality of licensee programs, have historically inserted significant subjectivity into reactor
-58-

oversight decision-making. Given the ROP goal to improve objectivity, the risk-informed

approach used within the ROP fundamentally views the use of a probabilistic framework as a

decision-framework which lends greater intellectual discipline and objectivity to the ROP

decision process and less reliance on subjectivity.

The comments concerning the SDPs not producing consistent or meaningful results were

addressed under our response to question #7.

Regarding the comment that the NRC should waive the 90-day SDP timeliness goal for

exceptional cases that involve complex engineering judgement issues, as noted in last year’s

consolidated response, the NRC staff agrees that in some instances of highly complex issues,

the 90 days may not be sufficient time to come to a final risk determination. The process allows

for such cases, however, the failure to meet the timeliness goal will count toward the

appropriate metric. An additional timeliness requirement limiting the maximum amount of time

to issue the final significance determination to 180 days was incorporated into the timeliness

metric starting with the 2008 assessment period. The timeliness requirement prompts the NRC

staff to not over engineer, over analyze, or initiate costly research to arrive at a risk conclusion

not readily supported by available information.

Regarding the comment that the NRC should make additional use of licensee-developed PRA

models (in the SDP process) that meet the necessary quality standards, few if any of the

licensee PRAs currently meet the acceptable quality standard which is Regulatory Guide

RG 1.200. This item has been under discussion with industry this year. Recently the staff

decided to continue to explore the potential to use licensee PRAs as the initial basis for riskinforming findings, however, many obstacles remain besides the lack of uniform quality of

PRAs. For example it is the regulator’s fundamental function to determine significance of

inspection findings. The staff has a concern from a public confidence viewpoint of allowing the

licensees to determine the significance of findings. The value of senior reactor analysts

performing risk assessments and engaging the industry is an important aspect of the

independent assessment of significance. The staff discussed the merits of addressing PRA

technical issues jointly with industry as a preferred approach. Finally, the staff noted that there

are fundamental differences with regard to risk-informed applications under RG 1.174 and

enforcement. The NRC staff will continue to deliberate the merits of industry’s proposal to

make better use of licensee PRAs in the ROP.

To address the comments regarding the thresholds for findings within and across cornerstones,

as noted in last year’s consolidated response, SDPs are tools used to approximate the risk of

licensee performance deficiencies within a conservative range up to one order of magnitude. 

Hence, white findings can end up as green when challenged by licensees, who have invested

significant resources to quantify the conservatism built into their robust plant design, thus

demonstrating system capability within plant safety design requirements under the identified

degraded conditions. It is important to remember that all conditions adverse to quality, that is

all findings minor, green, white, or greater, must be corrected. Evaluating the threshold is a

continuous process. Several adjustments since the program’s inception have been made to the

threshold for the deterministic risk-informed SDPs. An expert panel (SERP) reviews every

finding with a potential greater than green outcome. Normalization, and assuring equivalence,

is part of the panel’s responsibility. Also, the NRC holds an annual Agency Action Review

Meeting (AARM) to review the agency actions resulting from the performance of nuclear reactor

licensees for those plants with significant performance problems as determined by the Action

Matrix and to review industry performance trends. The AARM also considers whether the ROP

is accomplishing the proper level of regulatory oversight.
-59-

In response to the comment regarding the level of risk uncertainty, as noted in last year’s

consolidated response, the staff agrees that periodically the level of uncertainty associated with

a finding will cause the outcome of the SDP to cover a band of two or even three orders of

magnitude. In such situations the NRC staff makes an effort to reduce the uncertainty using

the various available tools including additional inspection, licensee PRA data, etc. However,

extra effort always results in expending additional resources and may prolong the process

beyond the 90-day timeliness target. The staff seeks to achieve a balance in this regard. 

Additionally, to address some issues, the staff implemented an enhanced risk-informed but

qualitative management decision making process.

To address the comment regarding the subjectivity of the SDP, as noted in last year’s

consolidated response, the staff believes that the ROP is an appropriate but evolving process. 

As stated in ROP program guidance, it is the responsibility of the NRC to objectively consider

the available information and to reach a final significance determination. Furthermore, the ROP

is significantly more objective than the previous inspection and oversight process. It is also

much more performance based, in contrast to the relatively higher level of programmatic focus

in the previous process.

Assessment Section

Respondent Comments:

Nuclear Energy Institute

As with any new program, the NRC’s oversight of safety culture will only be effective if the

agency executes it in a disciplined and consistent manner. In addition, there are two potential

impediments that could undermine the success of the NRC’s efforts.

The first area of concern is the low threshold for triggering a substantive cross-cutting

issue in the area of SCWE. Substantive cross cutting issues are not created equal. A

substantive cross cutting issue in the area of SCWE carries with it much greater

significance with our stakeholder community and the NRC oversight that follows is

anything but measured.

Given that the stakes are high, the NRC must be disciplined in their oversight of SCWE

because false positives in this area are unacceptable. Specifically, the NRC must

consistently adhere to their guidance which states that if the impact of the finding on a

plant’s SCWE is isolated or if the licensee’s response to the finding is appropriate and

timely, a substantive cross-cutting issue does not exist.

The other area of concern pertains to the discipline in which the NRC will link inspection

findings to cross-cutting aspects. If the NRC is undisciplined in this process and links

every Green or greater inspection finding to a cross-cutting aspect or if the NRC links

findings to multiple aspects, there could be an unwarranted escalation in plants with

substantive cross-cutting issues.

These false positives will undermine the credibility of the ROP, will lead the NRC to

dedicate resources to plants whose performance may not warrant greater oversight,

may divert NRC resources from plants that actually warrant greater oversight, and lead

licensees to dedicate resources to address issues that have no merit in actual risk
-60-

significance. Such a scenario is unacceptable and would erode the confidence of our

public, political and financial stakeholder communities.

Florida Power and Light

As a general comment pertaining to Question 14 of the subject Federal Register notice

(p. 59,450), while the NRC's recent ROP safety culture enhancements clarify NRC's

expectations and focus licensee attention, we are concerned with the potential unintended

result that ultimately licensees are more susceptible to having substantive cross-cutting issues

(based upon the sheer volume of cross-cutting aspects allocated to safety culture under the

ROP, as revised). This could impose unnecessary licensee burden.

Southern California Edison

The Commission has initiated a comprehensive effort to appropriately incorporate Safety

Culture into the ROP. The changes made to date are significant and will require considerable

management oversight and stakeholder involvement to ensure that a predictable and

"scrutable" process results. As this effort proceeds, it should include participation of the ROP

stakeholders and be subject to the same checks and balances as any other ROP changes.

NRC Response:

Regarding the comment that the NRC must be disciplined in their oversight of SCWE because

false positives in this area are unacceptable, the staff believes that a disciplined approach is

necessary when determining the existence of SCWE issues. The staff established a SCWE

Finding Review Group comprised of individuals from multiple program offices and the regions to

evaluate the appropriateness of potential SCWE cross-cutting aspects of the finding prior to a

regional decision on the potential SCWE cross-cutting aspect. The ROP assessment process

provides for layers of review prior to the identification of a SCWE substantive cross-cutting

issue. Since implementation of the enhanced ROP on July 1, 2006, there have been no

findings with SCWE cross-cutting aspects identified and no SCWE substantive cross-cutting

issues identified (however, two sites have met two out of the three criteria).

The comments that the NRC needs to remain disciplined when linking inspection findings to

cross-cutting aspects to avoid an unwarranted escalation in plants with substantive

cross-cutting issues have been previously addressed under the response to question #14.

Other Section

Respondent Comments:

R. Tierney, Private Citizen

In a general comment to that process I must inject that I am retired and was involved with the

nuclear energy activities for over forty years. There is today as it was back in 1955, the public

perception that nuclear power and nuclear weapons are one in the same. Unfortunately the

first use of this energy was a weapon. This perception is fostered by a lack of knowledge. Until

this countries education system promotes early year learning about radiation health effects and

the generic use of nuclear generation, this lack of knowledge will continue.
-61-

As you are well aware, the Europeans are far more advanced in utilizing this form of energy.

That is interesting in reference that they had the worst nuclear accident, Chernobyl,

(Unsafe design) in comparison to our Three Mile Island operator error incident. In addition, our

political structure is part of the problem.

I also must admit that your interfacing with the uneducated public is challenging with reference

of discussion in technical maters. Having attended several of these meetings I would advise

that your speakers be aware of this and use basic generic terms in there responses.

Pennsylvania Department of Environmental Protection

The NRC is encouraged to conduct periodic public meetings or workshops, as needed, to: 1)

discuss the results of the agency's most recent self-assessment of the ROP; 2) review and

discuss any changes or proposed changes to the ROP; and 3) seek feedback from external

stakeholders.

South Texas Project Nuclear Operating Company

The NRC staff and external stakeholders should develop a multi-year project plan to review the

ROP. This review should look at all areas and seek areas for improvement in resource

utilization. Some specific areas to review should be:

• An effectiveness review of the Component Design Basis Inspection.

• A review of overall inspection hour utilization. An effectiveness review of each

inspection area should be considered.

• A review of the deterministically based SDPs to make them more risk informed.

• A review of current Performance Indicators for effectiveness and possible improvement

or elimination.

• A review of crediting self assessments and external assessments instead of performing

direct inspection.

A new ROP process should be developed to address new plant construction.

Discussions need to be held and a process developed on how a licensee and the NRC staff will

interact when that licensee has both an operating plant and plants under licensing/construction

at the same site.

New Jersey Division of Environmental Safety and Health

I want to take this opportunity to provide some comments in response to your October 2, 2006

Federal Register Notice regarding the Reactor Oversight Process (ROP). We support the U.S.

Nuclear Regulatory Commission’s (NRC) annual assessment of the ROP which seeks to

improve its approach to inspecting and assessing the operation of commercial nuclear reactors.

The New Jersey Department of Environmental Protection (NJDEP) has been involved in the

NRC ROP since its inception in April 2000. We participated in the development of the NRC

ROP and have provided feedback on the program over the past five years.

Our interest in the NRC ROP is based on the four nuclear reactors which operate in the State of

New Jersey. Any improvement in your ROP program which leads to safer nuclear power plants

positively affects the citizens in our State.

Most safety improvements at Hope Creek, Salem 1, and Salem 2 were the result of the new

Exelon management team. The Safety Conscious Work Environment concerns were handled

outside the NRC ROP. Safety improvements at Oyster Creek were implemented as a result of

capital investment in anticipation of license renewal. None of these actions were taken as a
-62-

result of the NRC ROP. These examples demonstrate that the NRC ROP needs

improvements. If the NRC ROP was re-designed to promote continuous plant improvements

and included more enforcement authority, it would be more effective.

Nuclear Energy Institute

The NRC staff should develop a multi-year project plan to review the ROP with stakeholder

input. This review should look at all areas and seek areas for improvement in resource

utilization.

Some specific areas to review should be:

1. An effectiveness review of the Component Design Basis Inspection.

2. A review of overall inspection hour utilization. An effectiveness review of each inspection

area should be considered.

3. A review of the deterministically based SDPs to make them more risk-informed.

4. A review of current Performance Indicators for effectiveness and possible improvement or

elimination.

5. A review of crediting self assessments and external assessments instead of performing

direct inspection.

Region IV Utility Group

The NRC staff and external stakeholders should develop project plan to review the ROP over

the next few years. This review should look at all areas and seek areas for improvement in

resource utilization. Some specific areas to review include:

- An effectiveness review of the Component Design Basis Inspection.

- A review of overall inspection hour utilization. An effectiveness review of each inspection area

should be considered.

- A review of consistency between NRC regions should be performed that considers

1) the number of findings, 2) percent of findings with cross cutting aspects assigned,

3) inspection issues that appear to be confined to one region.

- Consider an improved process for more timely sharing of inspection issues with potential

generic implementation interest. (See comment on Item 5.)

RUGIV would be pleased to participate in a working group/task force with NRC staff and other

industry stakeholders to develop such a process.

- A review of the deterministically based SDPs to make them more risk-informed.

- A review of current performance indicators for effectiveness and possible improvement or

elimination.

- A review of crediting self assessments and external assessments instead of performing direct

inspection.

A new oversight process should be developed to address new plant construction, utilizing a

similar process for stakeholder involvement that was successfully used for the current ROP.

Discussions need to be held to define the interactions between the various NRC staff

organizations and the licensee when the licensee has both an operating plant and plants under

licensing/construction at the same site.
-63-

Southern California Edison

SCE believes the revised ROP has been successful in providing a more risk-informed

regulatory framework. There are several areas, however, that we believe require continuing

attention:

• As in all things, Performance Indicators (PIs) and other aspects of the ROP

(e.g., Significance Determination Process (SDP), etc.) can create unintended

consequences. There is a continuing need for a robust and ongoing process to identify

and address such situations as they arise.

• While some conservative "false positives" are acceptable from any such processes, it is

necessary that the ROP identifies and resolves potential opportunities for "false

negatives" (i.e., failure to identify a potential safety concern). "False negatives" have the

potential to significantly undermine the credibility of the entire ROP.

• While much improvement has been realized, there is a continuing need to improve the

public's understanding of all the elements of the ROP. It appears that much of the

public continues to perceive the new ROP as solely the self reported Performance

Indicators, and is less aware of the revised independent Inspection Process, SDPs,

Action Matrix, and Enforcement Policy.

• The opportunity to provide comments on the NRC's revised ROP is appropriate and

appreciated. We recommend that the NRC staff provide formal, timely, and public

feedback on comments received from the external stakeholders. We further

recommend that the ROP survey continue on an annual basis.

Slovenian Nuclear Safety Administration

We have been following the development of ROP since its beginning. We think of it very highly

and consider it a very useful tool. We are looking at ROP and US NRC risk based approach to

regulation as a source of inspiration for our own Slovenian program of regulatory oversight.

Nevertheless we do not have enough practical experiences with ROP to answer your

questionnaire.

Strategic Teaming and Resource Sharing

Since implementation in April 2000, the ROP has exhibited marked improvement over the

former inspection and enforcement process. The continued improvement by way of the routine 

ROP public meetings and the periodic solicitation of public feedback has assisted the ROP in

effectively meeting the intended objectives, i.e., to provide tools for inspecting and assessing

licensee performance in a manner that was more risk informed, objective, predictable, and

understandable than the previous oversight processes and provides for regulatory actions that

are open, effective, efficient, realistic, and timely.

Reassessment of performance indicators and adopting more effective indicators

(e.g., Mitigating Systems Performance Index and Unplanned Scrams with Complications) is

applauded. STARS supports and looks forward to assisting in the continuing efforts to further

develop and improve the ROP.

The NRC staff and external stakeholders should develop a multi-year project plan to review the

ROP. This review should look at all areas and seek areas for improvement in resource

utilization. Some specific areas to review include:
-64-

• An effectiveness review of the Component Design Basis Inspection.

• A review of overall inspection hour utilization. An effectiveness review of each

inspection area should be considered.

• A review of consistency between NRC regions should be performed that considers 1)

the number of findings, 2) the percent of findings with cross cutting aspects assigned, 3)

inspection issues that appear to be confined to one region.

• Consider an improved process for more timely sharing of inspection issues with potential

generic interest. Current process tends to develop the communication after several

licensees have been inspected. Recent use of the NRC Morning Report is an excellent

venue to communicate a potentially emerging issue.

• A review of the deterministically based SDPs to make them more risk-informed.

• A review of current performance indicators for effectiveness and possible improvement

or elimination.

• A review for crediting self assessments and external assessments as an alternative to

performance of direct inspection.

A new oversight process should be developed to address new plant construction, utilizing a

similar process for stakeholder involvement that was successfully used for the current ROP.

Discussions need to be held to define the interactions between the various NRC staff

organizations and the licensee when the licensee has both an operating plant and plants under

licensing/construction at the same site.

NRC Response:

Regarding the comment about the negative public perception of nuclear power and their lack of

understanding of the technical matters, the staff recognizes that the ROP is complex, as are the

design and operation of the nuclear reactors that the NRC regulates. The staff strives to make

the ROP as understandable as possible as it balances this goal with the other ROP goals of

being objective, risk-informed, and predictable. The Office of Public Affairs published a plain

language description of the ROP in NUREG-1649, “Reactor Oversight Process,” which was

recently updated in December 2006 to incorporate recent process improvements. The ROP

portion of the NRC’s website is also designed to facilitate the understandability of and access to

ROP-related information and is maintained on a regular basis to provide current ROP-related

information and results. NRC staff is also trained and instructed to use plain English to the

extent practicable when discussing matters at public meetings.

Regarding the comment that the country’s education system needs to improve to promote early

year learning about radiation health effects and the generic use of nuclear generation, the NRC

maintains a “Students’ Corner” Web page readily accessible from the NRC’s home page to

encourage students and other interested parties to learn more about the safe use of nuclear

technology. This Web page includes an interactive, easy-to-follow summary of nuclear power,

as well as teacher’s lesson plans, games, and a list of references to find more information

about nuclear power.

As noted in SECY-07-0069, the staff issues an ROP self-assessment Commission paper

annually and briefs the Commission on the self-assessment results, typically in May of each

year. These annual meetings are open to the public, can be viewed live through the NRC’s

Web page, are announced at least 10 days in advance, and are professionally transcribed. The

slides, transcripts, and other relevant information are readily available on the NRC’s Web page

under “Commission Documents.” In addition, the staff conducts monthly public working-level
-65-

meetings with the Nuclear Energy Institute (NEI), the industry, and interested stakeholders to

discuss ongoing refinements to the ROP. The staff conducted a number of public meetings

prior to implementing the safety culture enhancements. The staff also conducts public

meetings in the vicinity of each operating reactor to discuss the results of the NRC’s annual

assessment of the licensee’s performance. These meetings provide interested stakeholders an

opportunity to engage NRC on plant performance and the role of the agency in ensuring safe

plant operations. The staff also sponsors ROP-related breakout sessions at the Regulatory

Information Conference each Spring. The staff also issues its annual external survey through

the Federal Register each Fall to evaluate ROP effectiveness and gather stakeholder insights. 

The staff maintains the NRC’s Web pages to communicate current ROP-related information

and results. These outreach efforts have resulted in valuable feedback and ROP

improvements.

Regarding the comment that the NRC staff should develop a multi-year project plan to review

the ROP with stakeholder input, the staff believes that this concept is accomplished through

ongoing processes and the monthly ROP public meetings. The staff is open to further

discussions at upcoming ROP meetings regarding the need to develop such a plan. The

specific areas suggested by the comments to be included in the project plan have been

addressed in the responses to other specific questions of this survey.

Regarding the comment that a new oversight process should be developed to address new

plant construction, the NRC staff plans to involve stakeholders in developing the oversight

process for new plant construction, and is in the process of conducting several public meetings

regarding the development of a new plant construction oversight process.

Regarding the comment that discussions need to be held and a process developed on how a

licensee and the NRC staff will interact when that licensee has both an operating plant and

plants under licensing/construction at the same site, the two processes are independent of one

another. Oversight of the operating plant is in accordance with the ROP, while the plant under

construction will need to satisfy either Part 50 or Part 52 requirements. The staff agrees that

there could be interactions that need to be addressed. The staff is open to further discussion

on this issue at a future ROP working group meeting.

Regarding the comment that the ROP should identify and resolve potential opportunities for

"false negatives," the SDP process is constantly looking to identify “false negatives.” The peer

review of all greater than Green SDP evaluations and consensus of the SERP are checks and

balances for ensuring that false negatives and positives are identified and prevented.

All other comments have been addressed in previous responses.

END