POLICY ISSUE

INFORMATION

April 6, 2004 SECY-04-0052

FOR: The Commissioners

FROM: William D. Travers

Executive Director for Operations

SUBJECT: FY 2003 RESULTS OF THE INDUSTRY TRENDS PROGRAM FOR

OPERATING POWER REACTORS AND STATUS OF ONGOING

DEVELOPMENT

PURPOSE:

To inform the Commission of the results of the Nuclear Regulatory Commission’s (NRC)

Industry Trends Program (ITP) for FY 2003 and the status of ongoing development. 

SUMMARY:

The NRC staff implemented the ITP in 2001, and is continuing to develop the program as a

means to confirm that the nuclear industry is maintaining the safety of operating power plants

and to increase public confidence in the efficacy of the NRC’s processes. The NRC uses

industry-level indicators to identify adverse trends. Adverse trends are assessed for safety

significance and the NRC responds as necessary to any safety issues identified. One important

output of this program is to report to Congress each year on the performance goal measure of

“no statistically significant adverse industry trends in safety performance” as part of the NRC’s

Performance and Accountability Report. Based on the information currently available from the

industry-level indicators originally developed by the former Office for Analysis and Evaluation of

Operational Data (AEOD) and the Accident Sequence Precursor (ASP) Program implemented

by RES, no statistically significant adverse industry trends have been identified through

FY 2003. However, three of the industry trend indicators met or exceeded their prediction limits

and are discussed in more detail within this paper. 

CONTACT: David Wrona, NRR/DIPM

301-415-1924
The Commissioners -2-

The staff is continuing to use the AEOD and ASP indicators while it develops additional

industry-level indicators that are more risk-informed and better aligned with the cornerstones of

safety in the Reactor Oversight Process (ROP). These additional indicators will be developed

in phases and qualified for use in the ITP and the annual Performance and Accountability

Report to Congress. The results of this program, along with any actions taken or planned, are

reviewed annually during the Agency Action Review Meeting (AARM) and reported to the

Commission.

BACKGROUND:

This paper is the fourth annual report to the Commission on the ITP. The previous Commission

reports were SECY-01-0111, “Development of an Industry Trends Program for Operating

Power Reactors;” SECY-02-0058, “Results of the Industry Trends Program for Operating Power

Reactors and Status of Ongoing Development;” and SECY-03-0057, “FY 2002 Results of the

Industry Trends Program for Operating Power Reactors and Status of Ongoing Development.”

Additional information on the ITP and the process for identifying and addressing adverse trends

was provided in SECY-03-0057.

The ROP uses both plant-level performance indicators (PIs) and inspections to provide

plant-specific oversight of safety performance. The ITP provides a means to assess overall

industry performance using industry-level indicators. Issues that are identified from either the

ROP or the ITP are evaluated using information from agency databases, and those assessed

as having generic safety significance are addressed using existing NRC processes, including

generic safety inspections in the ROP, the generic communications process, and the generic

safety issue process.

The ITP is a joint effort between the Inspection Program Branch in the Office of Nuclear

Reactor Regulation (NRR) and the Operating Experience and Risk Analysis Branch in the

Office of Nuclear Regulatory Research (RES). The purposes of the ITP are to provide a means

to assess whether the nuclear industry is maintaining the safety performance of operating

reactors and, by clearly communicating that performance, to enhance stakeholder confidence in

the efficacy of the NRC’s processes. The specific objectives of the ITP are as follows:

(1) Collect and monitor industry-wide data that can be used to assess whether the nuclear

industry is maintaining the safety performance of operating plants and to provide

feedback on the ROP;

(2) Assess the safety significance and causes of any statistically significant adverse

industry trends, determine if the trends represent an actual degradation in overall

industry safety performance, and respond appropriately to any safety issues that may be

identified;

(3) Communicate industry-level information to Congress and other stakeholders in an

effective and timely manner; and 

(4) Support the NRC’s performance goals of maintaining safety and enhancing public

confidence in the agency’s regulatory processes.
The Commissioners -3-

The NRC currently uses the results of the ITP in the following ways:

(1) The NRC reports the industry indicators to Congress annually in the NRC’s

“Performance and Accountability Report, Fiscal Year 200X” (NUREG-1542 series) and

in the NRC’s “Budget Estimates and Performance Plan Fiscal Year 200X” (NUREG1100 series). The indicators demonstrate how the agency's ROP produced successful

results by meeting the measure of “no statistically significant adverse industry trends in

safety performance” for the performance goal of maintaining safety;

(2) The NRC communicates overall industry performance to stakeholders by publishing the

ITP indicators on the Nuclear Reactors portion of the agency’s public Web site at

http://www.nrc.gov/reactors/operating/oversight/industry-trends.html. The staff believes

that communication of the industry-level indicators, when added to the information on

individual plants from the ROP, enhances stakeholder confidence in the efficacy of the

NRC’s oversight of the nuclear industry;

(3) The results of the ITP are a key element of the review by senior NRC managers of the

agency’s oversight of operating facilities in the annual AARM;

(4) The staff informs the Commission of the results of the ITP in an annual report in the

same timeframe as the AARM;

(5) The Commission uses the ITP indicators when presenting the status of industry

performance to the NRC’s oversight committees and at major conferences with

stakeholders; and 

(6) NRC managers use the ITP indicators to provide an overview of industry performance at

various conferences with stakeholders, such as the NRC’s Regulatory Information

Conference.

DISCUSSION:

The ITP is intended to monitor trends in industry safety performance so that the staff can

identify and address adverse industry trends. The ITP does so using indicators of known

conditions and issues that are compiled from the best available data. The staff monitors a

comprehensive set of indicators; however, the staff recognizes that there are limits on what can

be tracked and trended by the ITP. Oversight of plant-specific conditions and events is

provided by the ROP. The staff recognizes that although the ITP attempts to identify

developing trends, new and unforseen events can occur, such as the significant erosion of the

reactor vessel head at Davis-Besse. The staff is mindful of the need to respond promptly to

these events, as well as the need to review its regulatory processes in light of the issues

revealed by these events.
The Commissioners -4-

RESULTS OF FY 2003 TREND ANALYSES

A key output of the ITP is that it provides the basis for agency monitoring and reporting to

Congress against the performance goal measure of “no statistically significant adverse industry

trends in safety performance,” as established by the NRC’s Strategic Plan. The current bases

for assessing performance against this measure are trends in the industry indicators developed

by the former AEOD (henceforth referred to as the “AEOD indicators”) and trends identified by

the ASP Program. Notably, these indicators were among those cited as demonstrating

improvements in industry safety performance that contributed to the agency’s decision to revise

the oversight process for operating power reactors.

Based on the AEOD indicators and the ASP Program results, no statistically significant adverse

trends in industry safety performance were identified through the end of FY 2003. Graphs of

the trends for each of these indicators are presented in Attachment 1.

In addition, as discussed in SECY-01-0111, the staff adopted a statistical approach using

“prediction limits” to provide a consistent method to identify potential short-term year-to-year

emergent issues before they manifest themselves as long-term trends. Three AEOD indicators,

automatic scrams while critical, safety system actuations (SSA), and equipment-forced outage

(EFO) rate reached or exceeded their prediction limit during FY 2003 and are discussed below. 

The staff followed the process for investigating adverse trends that was outlined in

Attachment 1 of SECY-03-0057, which included analyzing in detail the data supporting each

indicator to understand the causes.

 Automatic Scrams and Safety System Actuations

The automatic scrams while critical prediction limit was 0.680 per plant and the actual

FY 2003 value was 0.748 per plant, which equates to the threshold being exceeded by

7 scrams. The SSA prediction limit was 0.340 per plant and the actual FY 2003 value

was 0.398 per plant, which equates to 6 actuations over the prediction limit.

Automatic scrams and SSAs have a clear and established interdependency; events that

trigger scrams often trigger one or more SSAs. For example, and as part of the

explanation for the increases seen in FY 2003, the August 14, 2003, Northeast blackout

event resulted in 9 automatic scrams and 12 SSAs at the 10 nuclear plants affected 

(Davis-Besse was shut down at the time and thus did not experience a scram). This

single event alone did not account for all of the FY 2003 increase over the FY 2002

values for these two indicators. In fact, this event did not even account for the majority

of the increase over FY 2002 numbers in the case of automatic scrams. The same is

true if the FY 2003 numbers are compared with the average of the previous 3 years (FY

2000 through 2002).

Also of note is that of the nine plants that scrammed from the blackout, four plants

contributed four previous scrams to the FY 2003 scram count that were directly caused

by problems with the offsite power grid and/or the plant’s interface with the grid. 

Subsequent to the blackout, three other Northeast plants scrammed due to loss of

offsite power. A similar pattern was seen with SSAs.
The Commissioners -5-

A detailed analysis of the contributing factors and causes of the increased number of

automatic scrams and SSAs was performed and a summary of this analysis is provided

in Attachment 2. The data review was expanded to include a review of plant

comparison groups, activities in progress at the time of the event, and apparent causes. 

The results indicate that the number of automatic scrams increased primarily due to

grid-related problems and the increase in the number of automatic scrams resulted in a

related increase in SSAs. ASP analysis will provide some safety and risk significance

on the blackout event. Preliminary ASP analyses have been completed and forwarded

to the licensees for review. The NRC has previously developed an action plan for

reviewing U.S. nuclear power plant issues relating to recent electric power grid events

(see Memorandum from W. Travers to Chairman Diaz, “Action Plan for Resolving

Electrical Grid Concerns,” dated February 4, 2004). In the Memorandum, the staff

stated that it has not identified any safety issues warranting immediate regulatory action

and will evaluate whether generic communications or other regulatory processes are

needed to highlight and address grid issues. Since the increase in SSAs was due to an

increase in the number of scrams, no additional action beyond what will be done due to

the increase in the number of scrams is needed. 

Although not tracked as an industry trends indicator, the number of plants that exceeded

the green to white threshold of the unplanned scrams per 7,000 critical hours PI of the

ROP increased in FY 2003. Eight plants crossed the green to white threshold in FY

2003, while only two and three plants crossed the threshold in FY 2002 and 2001,

respectively. Supplemental inspections, as required by the ROP Action Matrix, were

performed or are planned at these plants that crossed the ROP PI threshold.

 Equipment-Forced Outage Rate

The FY 2003 equipment-forced outage rate was at the prediction limit of 0.157 per 1000

commercial critical hours.

In contrast to the automatic scrams and safety system actuation indicators discussed

above, the August 14 blackout event had very little effect on the FY 2003 EFO rate. 

None of the plant outages initiated by the blackout event were classified as

equipment-forced outages, and the associated outages were collectively of short

duration (thus having little effect on the denominator of the EFO calculation). A

breakdown of the EFO rate calculation shows that both the numerator and the

denominator were instrumental in driving the rate increase. The total number of industry

critical hours (EFO denominator) in FY 2003 was less than those in the 3 preceding

years (FYs 2000 through 2002). The extended Davis-Besse shutdown and the increase

in the number of scrams contributed to this decrease. The number of equipment-forced

outages (EFO numerator) was larger than those in the preceding four years (FYs 1999

through 2002). Several factors contributed to this increase. For example, one plant

reported a total of 11 individual equipment-forced outages in FY 2003, 9 of which were

consecutive delays in recovering from a single refueling outage.

Since the equipment-forced outage rate was influenced by one plant that was

specifically related to the extension of a refueling outage, NRR did not identify any

industry-wide safety issues associated with reaching the FY 2003 prediction limit for the
The Commissioners -6-

equipment-forced outage rate indicator, and any plant specific issues identified will be

addressed via the ROP.

In last year’s industry trend SECY paper, SECY-03-0057, the ASP indicator chart displayed

preliminary results for FY 2000 and FY 2001. Although the ASP indicator did not cross a

prediction limit, the staff noted an increasing number of potential precursors in 2000 and 2001

when compared to the relatively low number of precursors between 1997 and 1999 and

planned to analyze these short-term variations in the number of ASP events. Further, last

year’s SECY paper mentioned staff’s intentions to investigate the nature of the short-term

variations as part of the ITP. However, due to other priorities, staff was unable to meet prior

expectations, including the completion of the preliminary analysis of FY 2001 events and the

evaluation of the short-term variations.

ITP PROGRAM DEVELOPMENT

1. Incorporation of Additional Industry Operating Experience

The NRC’s Davis-Besse Lessons Learned Task Force (DBLLTF) provided a number of

recommendations to improve NRC utilization of operating experience information. An

Operating Experience Task Force was chartered to evaluate the agency’s reactor operating

experience program and to recommend specific program improvements. On November 26,

2003, the “Reactor Operating Experience Task Force Report” was issued. The staff is currently

assessing the recommendations of the task force to determine what changes or enhancements

to the ITP may be appropriate, and will highlight any necessary changes in the next ITP annual

report to the Commission.

The staff has updated the data and trends that were previously published in various

NUREG-series reports for system reliability studies, component reliability studies,

common-cause failure studies, and other special studies for which industry-wide trends were

reported. In addition, as a means of providing greater stakeholder access to this information,

the staff has posted the results of these updates on an internal RES Web page and is working

towards posting these results on the NRC public Web site. While these results are not formally

reported as indicators in the ITP, they are an integral part of any evaluation that will be

performed if any ITP indicator exceeds a prediction limit or shows an adverse trend.

2. Development of Additional, More Risk-Informed Indicators

The staff has continued to develop additional indicators that are more risk-informed and better

aligned with the cornerstones of safety in the ROP. For example, the staff has continued

development of industry-level indicators from the data that licensees submit for the plant-level

ROP PIs. However, since the ROP was only implemented in April 2000 and some of the PI

data submitted to support initial implementation were best estimates, there is still insufficient

data for reliable long-term trending of these indicators. Nonetheless, based on a review of the

indicator data submitted to date, no statistically significant adverse trends have emerged.

In SECY-03-0057, the staff reported on the development of an index for boiling water reactors

that monitors 9 risk-significant initiating events and a similar index for pressurized water

reactors that monitors 10 events (the additional category of events is steam generator tube
The Commissioners -7-

ruptures). Each initiating event is weighted in the index based on its relative contribution to

industry core damage frequency. This indicator is called the Baseline Risk Index for Initiating

Events (BRIIE) and is discussed in more detail in Attachment 3.

The staff has continued development work during FY 2003 and has given briefings on the

BRIIE concept during periodic ROP working group public meetings, at subcommittee meetings

of the Advisory Committee on Reactor Safeguards (ACRS), and during a public workshop. The

staff has received valuable feedback during these meetings, and is addressing comments that

have been received. The staff intends to increase its interactions with stakeholders on the

BRIIE during this fiscal year. NRR is currently evaluating BRIIE for inclusion into the ITP and

expects to make a decision on BRIIE by mid 2004. If the BRIIE concept is deemed acceptable

for use in the ITP, the staff plans to work towards a pilot program and possible implementation

within 1–2 years.

3. Risk-Informed Response Thresholds

As discussed in SECY-01-0111, the staff intends to develop risk-informed thresholds to the

extent practicable and use them to determine the appropriate agency response to trends in

indicator data. The Commission supported this position and in the staff requirements

memorandum (SRM) related to SECY-01-0111, dated August 2, 2001, directed the staff to

develop risk-informed thresholds for the industry-level indicators “as soon as practicable.”

However, as discussed in SECY-03-0057, the staff continues to develop more risk-informed

indicators for the initiating events cornerstone (e.g., the BRIIE). The staff continues to focus its

resources on developing those potential replacement indicators rather than on developing riskinformed thresholds for the current set of ITP indicators since the AEOD indicators are not

easily risk-informed.

4. Potential Single Integrated Indicator of Industry Performance

Attachment 4 shows an integrated indicator known as the Action Matrix Summary. It is a

histogram that shows the number of plants in each column of the NRC’s Action Matrix since

initial implementation of the revised ROP. This single indicator provides a representative

picture of industry performance because it effectively rolls up both performance indicators and

inspection findings from the ROP in all cornerstones of safety.

The staff has proposed to establish a new performance measure that will count the number of

plants with significant performance issues, using the Action Matrix Summary indicator for

reporting against the measure. The measure would be “number of operating reactors whose

integrated performance is in the multiple/repetitive degraded cornerstone column or above of

the ROP Action Matrix is less than or equal to five...” This measure has the advantage that the

staff is already taking actions to address safety issues for these plants in an objective and

predictable manner in accordance with the ROP Action Matrix.

This measure is in addition to the existing measure related to adverse industry trends in safety

performance. The staff is pursuing this measure for use in the NRC’s Budget Estimates and

Performance Plan for FY 2006 (also known as the Green Book).
The Commissioners -8-

5. Improved Data Collection and Reporting

Licensees report operating experience data to the NRC, including data for the ROP plant-level

indicators, licensee event reports (LERs), and monthly operating reports. Licensees also report

additional data to other organizations such as the Institute of Nuclear Power Operations (INPO). 

The staff uses all of these reports as data sources for the indicators used in the ITP, and the

databases are essential to investigating safety issues when trends in the ITP indicators are

identified.

The staff continues to work extensively with industry to develop a consistent set of data

elements, definitions, and reporting guidelines for reliability and unavailability data that would

meet the needs of all stakeholders. To reduce unnecessary regulatory burden, the industry has

proposed changes to the method of supplying monthly operating reports to the NRC. The staff

is evaluating the possibility of obtaining the data quarterly in an electronic format from an

industry database. The new process for submitting monthly operating report data to the NRC

could be in place for FY 2005.

6. Industry Trend Communication Mechanisms

Currently the staff prepares a SECY paper on the results of the ITP for operating power

reactors. Industry trend indicators are also published on the NRC external Web page quarterly

and reported to Congress annually in the NRC’s Performance and Accountability Report and in

the NRC’s Budget Estimates and Performance Plan. The staff plans to evaluate these

communication methods in calendar year 2004 to determine if they are the most effective and

efficient way to communicate the ITP results to the interested stakeholders.

RESOURCES:

For FY 2004, NRR has budgeted approximately 1.5 full-time equivalent (FTE) and $278K for

the continued development and implementation of the ITP. For FYs 2005 through 2007, NRR

estimates resource requirements of approximately 1.5 FTE per year, with estimated contract

assistance funding requirements of about $350K per year. NRR has included these

requirements in its budget request submittals. Research support for the industry trends

program utilizes operating experience data and models developed and budgeted under other

RES programs.
The Commissioners -9-

COORDINATION:

The Office of the Chief Financial Officer has reviewed this paper and concurs.

The Office of the General Counsel has reviewed this paper and has no legal objection.

/RA William F. Kane Acting For/

William D. Travers

Executive Director

 for Operations

Attachments: 1. FY 2003 Trend Results Based on AEOD and ASP Indicators

2. Scram and Safety System Actuation Review

3. Summary of Industry Trends Program Enhancement Effort for the Initiating

 Events Cornerstone of Safety

4. Action Matrix Summary
The Commissioners -9-

COORDINATION:

The Office of the Chief Financial Officer has reviewed this paper and concurs.

The Office of the General Counsel has reviewed this paper and has no legal objection.

/RA William F. Kane Acting For/

William D. Travers

Executive Director

 for Operations

Attachments: 1. FY 2003 Trend Results Based on AEOD and ASP Indicators

2. Scram and Safety System Actuation Review

3. Summary of Industry Trends Program Enhancement Effort for the Initiating

 Events Cornerstone of Safety

4. Action Matrix Summary

WITS 200100034

ADAMS Accession Number: ML040640629

*SEE PREVIOUS CONCURRENCE

OFFICE IIPB:DIPM IIPB:DIPM IIPB:DIPM Tech Editing IROB:DIPM

NAME DWrona* JAndersen* SRichards* PKleene* TReis*

DATE 3/22/04 3/11/04 3/11/04 2/24/04 3/15/04

OFFICE PPRB:PMAS OCFO DIPM:NRR OGC RES

NAME RCorreia* DDrucker (via email)* BBoger* NLO GMLongo for DD* AThadani*

DATE 3/17/04 3/23/04 3/15/04 3/12/04 3/23/04

OFFICE ADIP NRR EDO

NAME JCraig (BBoger for)* JDyer* WTravers

DATE 3/26/04 03/30/04 04/6/04

OFFICIAL RECORD COPY
FY 2003 Trend Results 

Based on AEOD and ASP Indicators

Indicators Originally Developed by the Former Office of AEOD and

Accident Sequence Precursor (ASP) Indicators

ATTACHMENT 1
-2-

Figure A1-1

Figure A1-2
-3-

Figure A1-3

Figure A1-4
-4-

Figure A1-5

Figure A1-6
-5-

Figure A1-7
-6-

Figure A1-8. Occurrence rate of all precursors, by fiscal year. No trend was identified

during the FY 1993–2001 period (on-going analysis of seven events in FY 2001 are

conservatively included in the trend). A trend line is not shown in the figure because the

slope is not statistically significant. Results are current as of 3/15/2004.

Accident Sequence Precursor Trends

Figure A1-8 below shows the occurrence rate per reactor-year for all Accident Sequence

Precursor (ASP) events by fiscal year. No statically significant adverse trend was observed in

the occurrence rate for all precursor (CCDP or CDP > 10-6) during the 1993–2001 period. 

The trend is based on the number of all precursors starting in FY 1993. ASP results prior to

FY 1993 use less rigorous methodology and are shown in the figure to provide historical

perspective.

No trend was identified in the occurrence rate of all precursors during the 1993–2001 period. 

The FY 1993–2000 data is final. Several analyses of FY 2001 events are ongoing. This

includes conditions involving the primary water stress corrosion cracking of control rod drive

mechanism (CRDM) nozzles which require the completion of the RES probabilistic analysis of

the time dependent failure frequencies of the CRDM nozzles. CRDM cracks were found at nine

plants in FY 2001 and FY 2002 and may result in nine precursors. The on-going analysis of

seven events in FY 2001 are conservatively included in the trending analysis.

Preliminary results for FY 2002 and FY 2003 are provided for information. The bar graph

includes final results and preliminary results that have undergone internal staff review, as well

as on-going analysis of potential precursors. Not all of these ongoing analyses will necessarily

meet the precursor threshold when completed. Typically, 30 to 50 events per year undergo

detailed analysis in the ASP Program of which more than half do not meet the precursor

threshold upon further analysis.
Scram and Safety System Actuation Review

INTRODUCTION

During the annual review of industry trend performance indicator (PI) data, Idaho National

Engineering and Environmental Laboratory (INEEL) reported that the 2003 automatic scram

and safety system actuation (SSA) PIs exceeded their early-warning and action thresholds (see

Table A4-1). These thresholds represent 95 and 99% prediction limits, respectively, developed

from a baseline period (Fiscal Year [FY] 1997-2002 for scrams and 1999-2002 for SSAs).

Table A4-1. FY 2003 Scram and SSA data.

Indicator FY 2003

Base Period

Average

Early-Warning

Threshold

Action

Threshold

Automatic scrams 0.748/plant

77 scrams total

0.534/plant

55 scrams total

0.680/plant

70 scrams total

0.738/plant 

76 scrams total

Safety system

actuations

0.398/plant

41 SSAs total

0.241/plant

25 SSAs total

0.340/plant

35 SSAs total

0.388/plant 

40 SSAs total

Equipment forced

outage rate

0.157 outages per

1000 critical hours

0.131 outages per

1000 critical hours

0.157 outages per

1000 critical hours

0.166 outages per

1000 critical hours

Exceeding a single early-warning threshold is not uncommon, because this threshold is based

on a 95 percent prediction limit. For each PI, the probability of exceeding warning-level and

action level thresholds are 1/20 (5%) and 1/100 (1%), respectively, from random fluctuation. 

However, the probability of a PI reaching its early-warning threshold, and two other PIs

exceeding their action thresholds from random fluctuation, without the influence of external

factors, is extremely small (0.043%).

In response to this unusual PI activity, NRC requested that the INEEL perform an in-depth

review of the scram and SSA data to identify any factors that may have caused the FY 2003

increase. This report contains the results of that review.

SUMMARY

Scrams

An increase in automatic scrams occurred in FY 2003 primarily due to grid-related problems.

SSAs

An increase in SSAs occurred in FY 2003. This increase resulted from the increase in total

scrams (primarily automatic scrams). This change was not isolated to FY 2003. Although SSA

numbers were decreasing before FY 2003, the number of SSAs associated with scrams was

not decreasing. This resulted in an almost continuous increase in the percentage of SSAs that

were associated with scrams. Thus, the increase in FY 2003 scrams resulted in a related

increase in SSAs.

ATTACHMENT 2
-2-

AUTOMATIC SCRAMS

INEEL conducted an analysis of the increase in the number of automatic scrams. The analysis

included a review of the information listed below (similar data was also reviewed during the

analysis of SSAs):

• scram causes (equipment, personnel error, or other)

• systems involved in equipment-related scrams (feedwater, turbine, generator, or other)

• activities in progress at the time the scram occurred (normal operation, maintenance,

testing, or power changes)

• scrams by reactor type (PWR or boiling water reactor [BWR])

• the number of automatic scrams caused by problems associated with offsite power grid

and/or the plants’ interface with the grid

• a histogram of the number of plants that experienced different levels of scrams in a year

FY 2003 automatic scrams were at the highest level since FY 1996 (greater than any year in

the FY 1997-2002 baseline period). The early-warning and action thresholds correspond to 70

and 76 scrams, respectively, for 103 plants (including Davis-Besse). In FY 2003, 77 automatic

scrams occurred, exceeding the thresholds by 7 and 1 scrams, respectively; and exceeding the

baseline period average (55) by 22 scrams (40%). It was noted that the August 14, 2003,

Northeast blackout event resulted in 9 automatic scrams. Had this event not occurred, neither

of the two thresholds would have been exceeded. However, the increase in FY 2003 automatic

scrams would still be notable, exceeding the baseline period average by 13 scrams (24%).

Grid-related scrams rose dramatically in FY 2003; 23 scrams in FY 2003 compared to only 26

for the entire 6-year baseline period. The fact that FY 2003 grid-related scrams were 19

greater than the baseline period average accounts for most of the increase in FY 2003

automatic scrams.

Equipment failure continued to be the leading cause of automatic scrams, accounting for 48

(62%) of the FY 2003 automatic scrams. FY 2003 scrams due to equipment failure were 9

scrams (23%) greater than the baseline period average (48 vs. 39). However, when viewed as

a percentage of all automatic scrams, the FY 2003 equipment failure scrams were 9% less than

the baseline period average (62% vs. 71%).

The next leading cause category is “Other”, accounting for 21 (27%) of FY 2003 automatic

scrams. This category includes causes such as Procedure, Natural Phenomena, Unknown,

and Other (including the grid related events that are outside of the affected plants’ control). 

This category accounted for 13 scrams (163%) greater than the baseline period average (21 

vs. 8). The Northeast blackout accounted for 9 of those 13 scrams.

During the baseline period, three systems resulted in more equipment scrams than others:

feedwater, main generator, and turbine. This trend continued in FY 2003. In addition to the

grid-related scrams discussed above, the main generator accounted for 5 scrams (71%) more

than the baseline period average (12 vs. 7). The other systems accounted for a level of scrams

similar to their baseline period averages.
-3-

Scrams during normal operation continued to dominate the other categories, accounting for

78% of FY 2003 automatic scrams. These figures indicate that the increase in the FY 2003

scrams occurred almost exclusively in the normal operation category.

Of the 103 operating commercial nuclear power plants, 67% are PWRs, and 33% are BWRs. 

Over the baseline period, PWRs experienced 63% of automatic scrams, and BWRs 37%. 

FY 2003 data were essentially identical, with PWRs experiencing 62% of auto scrams, and

BWRs 38%. This indicates that the increase in automatic scrams was not isolated to a reactor

type.

A histogram of the number of plants that experienced different levels of automatic scrams per

year was generated. For example, in FY 2003, 39 plants experienced only a single automatic

scram. The FY 2003 data exceeded the baseline period average in essentially every level, with

no plant exceeding 4 automatic scrams per year. This indicates that the increase in automatic

scrams was not isolated to a few plants.

Conclusion

This review identified that a single factor was primarily responsible for the increase in FY 2003

automatic scrams; problems associated with the offsite power grid and/or the plants’ interface

with the grid resulted in 23 FY 2003 automatic scrams. This included the Northeast blackout

event, which resulted in 9 automatic scrams. Without the blackout event, the early-warning and

action thresholds for automatic scrams would not have been exceeded, but the increase in FY

2003 automatic scrams (most of which was due to grid-related events) would still be notable. It

was also noted that the baseline period from which the thresholds were developed did not

include a singular event (such as the blackout) that resulted in multiple scrams. Additionally,

relatively few scrams from grid-related events occurred during the baseline period.

SAFETY SYSTEM ACTUATIONS

Safety System Actuations (SSAs) are comprised of challenges to two classes of systems,

emergency diesel generators (EDGs) and emergency core cooling systems (ECCS). The

criteria that determines an SSA does not align exactly with the criteria for engineered safety

feature (ESF) actuation reporting. Rather, SSAs are a subset of ESFs as determined by the

type of system and circumstances of the actuation. Certain criteria must be satisfied to classify

an actuation event as an SSA, and the classification is not solely restricted to actual safety

needs. Basically, an EDG SSA is an event in which the EDG did, or should have, re-powered

the associated de-energized emergency bus. An ECCS SSA is an event in which any ECCS

actuation signal occurs, valid or not, or any ECCS equipment actuates. In either case,

determination of an SSA may require some interpretation of the event report information. For

this reason, SSAs are not as strictly objective as scrams.

FY 2003 SSAs were at the highest level since FY 1996 (greater than any year in the FY 1999-

2002 baseline period). The early-warning and action thresholds correspond to 35 and 40

SSAs, respectively, for 103 plants (including Davis-Besse). In FY 2003, the industry

experienced 41 SSAs, exceeding the thresholds by 6 and 1, respectively, and exceeding the

baseline period average (25) by 16 (64%). Note that the August 14, 2003, Northeast blackout

event resulted in 12 SSAs. Had this event not occurred, neither of the two thresholds would
-4-

have been exceeded, but the increase in FY 2003 SSAs would still be notable, exceeding the

baseline period average by 4 SSAs (16%).

Due to the nature of SSAs, it is common for an event to cause both an SSA and a scram, or for

an SSA to occur during a scram transient. For example, an EDG SSA and scram can both

result from a loss of offsite power. Furthermore, an ECCS SSA can result from the level

transient following a scram. During the baseline period, 39% of SSAs occurred with scrams. 

However, during FY 2003, 68% of SSAs occurred with scrams, indicating that the increase in

FY 2003 SSAs was largely driven by the increase in scrams. For example, the events resulting

in the 23 automatic scrams caused by problems associated with offsite power grid and/or the

plants’ interface with the grid also resulted in 19 SSAs.

Both types of SSAs (EDG and ECCS) increased in FY 2003. The FY 2003 EDG SSAs were 7

(47%) greater than the baseline period average (22 vs. 15), and ECCS SSAs were 9 (90%)

greater than the baseline period average (19 vs. 10).

Of the 103 operating commercial nuclear power plants, 67% are PWRs and 33% are BWRs. 

Over the baseline period, PWRs experienced 60% of SSAs, and BWRs 40%. In FY 2003 data,

the data reversed, with PWRs experiencing 39% of SSAs, and BWRs 61%. In fact, while FY

2003 PWR SSAs were essentially equal to their baseline period average (16 vs. 15), BWR

SSAs were 15 (150%) greater than their baseline period average (25 vs. 10), in part due to

automatic scrams at BWRs. This indicates that the increase in FY 2003 SSAs occurred mostly

at BWRs.

A histogram of the number of plants that experienced different levels of annual SSAs was

generated. For example, in FY 2003, 17 plants experienced only a single SSA. The FY 2003

data exceeded the baseline period average in every category (because there are no counts in

the baseline 4/year category, compare the FY2003 4/year category with the baseline 3/year

category), with no plant exceeding 4 SSAs per year. This indicates that the increase in SSAs

was not isolated to a few plants.

Conclusions

The increase in FY 2003 SSAs was largely driven by the increase in scrams (primarily

automatic scrams). The Northeast blackout event resulted in 12 SSAs. Without this event,

neither of the two thresholds would have been exceeded. It is noted that the baseline period

from which the thresholds were developed did not include a singular event (such as the

blackout) that resulted in multiple SSAs.
Summary of Industry Trends Program Enhancement Effort

 for the Initiating Events Cornerstone of Safety

1 Reasons for Enhancing Current ITP Performance Indicators

Current ITP performance indicators have both strengths and weaknesses. Strengths include

availability of historical results, continuity and consistency in yearly evaluations, and broad

coverage of the Cornerstones of Safety. However, ITP performance indicator weaknesses in

the Initiating Events and Mitigating Systems Cornerstones of Safety include (1) overlapping

coverage for certain cornerstones, (2) limited risk coverage, and (3) difficulties in interpreting

the risk significance of significant adverse trends that are detected.

In terms of risk coverage, work documented in NUREG-1753, “Risk-Based Performance

Indicators: Results of Phase 1 Development,” indicates that the ROP indicators “unplanned

scrams” and “scrams with loss of normal heat removal” probably cover less than 20 percent of

the total internal event core damage risk for the Initiating Events Cornerstone of Safety. (The

other 80 percent of risk involves less frequent initiating events that cannot be monitored on a

plant-specific basis over the limited, 3-year period covered by the ROP indicators.) This limited

coverage of risk by the ROP performance indicators is supplemented by inspections.

Moreover, it can be difficult to determine whether an adverse trend is risk significant if one is

detected. This is because not all scrams are equally serious in terms of risk. For example, an

adverse trend in “automatic scrams” due to “general transients” might be offset by a favorable

(decreasing frequency) trend in “scrams with loss of normal heat removal.” The ITP has no

established method for determining risk significance of broad categories such as “unplanned

scrams,” nor a mechanism for aggregating and interpreting offsetting trends at the Cornerstone

of Safety level. 

As a first step in enhancing the ITP to remedy the weaknesses discussed above, the Initiating

Event Cornerstone of Safety was chosen as the area of focus. Work focused on development

of performance indicators that did not overlap in coverage, significantly increased the risk

coverage, and provided a mechanism for determining the risk significance of changes in

performance, at both the individual initiating event level and at the integrated Cornerstone of

Safety level. The process and results are documented in the following sections.

2 Enhancement Process for Initiating Events Cornerstone of Safety

To enhance the ITP coverage of the Initiating Events Cornerstone of Safety, a three-step

process was used. The first step was to identify appropriate classes of initiating events. Then

methods for trending and establishing performance-based prediction limits for these individual

event classes were developed (Tier 1, performance-based monitoring of classes of initiating

events). Finally, an integrated, risk-informed indicator was developed by combining the

individual initiating event class information (Tier 2, risk-informed monitoring at the Cornerstone

of Safety level). Figure A2-1 shows the ITP process with respect to the integrated initiating

event indicator.

ATTACHMENT 3
-2-

Collect Indicator Data

[BRIIE (2), IEs from LERs]

Monitor Performance

(Short term issues, Prediction Limits)

Estimate Trends

(Apply statistical methods to identify

short- and long term adverse trends)

Agency Response

(No action / continue to monitor,

Engage Industry,

Generic communications,

Generic safety inspection,

Evaluate Adverse Trends Generic safety issue)

[Few plants (no adverse trend),

Plant comparison groups,

Examine agency databases,

Attempt to establish causes,

NRR technical branch review,

Assess safety significance]

Feedback to NRC

Programs

Senior Management

Review

(Agency Action Review Meeting)

Communications

with Stakeholders

(NRC web page for ITP,

Annual report to Commission,

NRC Info Digest,

Industry conferences)

Report to Congress

(NRC Performance &

Accountability Report,

Green/Blue Books,

Oversight Committees)

Input for Report

to Congress

(BRIIE, Thresholds)

Figure A2-1 BRIIE and the ITP process

2.1 Identification of Risk-Significant Initiating Events

The initiating event study (NUREG/CR-5750, “Rates of Initiating Events at U.S. Nuclear Power

Plants”) provides data for a large number of initiating event types for calendar year (CY) 1987

through CY 1995. (NRC is continually updating these data, but NUREGs are no longer being

published. Instead, the results are posted on an internal NRC Web site with plans to post them

on the NRC public Web site.) Initiating event types are defined in that study as unplanned

reactor trips that occur while a plant is critical and at or above the point of adding heat. A

subset of these events has been identified as being risk significant (NUREG-1753). The list of

risk-significant initiating event types considered in the ITP consists of the following 10 initiating

events applicable to PWRs and 9 applicable to BWRs:

General Transients Loss of Heat Sink

Loss of Feedwater Loss of Offsite Power

Loss of Vital AC Bus Loss of Vital DC Bus

Stuck-Open SRV Loss of Instrument Air

Small/Very Small LOCA Steam Generator Tube Rupture (PWR only)

In general, these risk-significant initiating event types cover approximately 90% of the internal

event core damage risk (excluding internal flooding) from the 103 operating commercial nuclear

power plants in the U.S. Also, these initiating events do not overlap.
-3-

Fiscal Year

1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002

0

0.5

1

1.5

2

2.5

3

3.5

Yearly Rate

95% Prediction Limit

Fitted Trend

99% Prediciton Limit

Figure A2-2 Estimated annual frequency of PWR general transient

initiating event. The trend over the baseline period is not

statistically significant (p-value = 0.625).

2.2 Performance Monitoring of Risk-Significant Initiating Event Classes (Tier 1)

The proposed Tier 1 activity involves trending risk-significant initiating events and monitoring

yearly industry performance against prediction limits. To accomplish this, up-to-date baseline

frequencies were established for each of the risk-significant initiating events. Then, given these

baseline frequencies and estimated yearly industry total critical reactor years of operation, the

ITP enhancement determined performance-based prediction limits. Data for these initiating

events (numbers of event occurrences and corresponding reactor critical years) are already

being collected and analyzed by the NRC on a continual basis, so no additional data collection

is needed to support the Tier 1 activities.

The prediction limits are statistically determined and performance based. Both 95% and 99%

limits have been estimated for illustrative purposes. An expert elicitation approach is proposed

to select the appropriate limits for the Tier 1 activity.

As an example of the Tier 1 trending analysis, the historical performance of the PWR general

transients is shown in Figure A2-2. Over the period FY 1988 through approximately FY 1997,

industry performance improved considerably (the initiating event frequency dropped). However,

over the period FY 1998 through FY 2001 (the period used for determining an up-to-date

baseline frequency), the industry performance was essentially constant. Including the FY 2002

initiating event data in the Tier 1 trending analysis did not produce any increasing trends.
-4-

These ITP Tier 1 activities will help the NRC identify degrading industry performance as an

adjunct to the plant-specific performance assessment performed as part of the ROP. Potential

NRC responses if one or more of the prediction limits are reached or exceeded are outlined in

Section 4 of this attachment. Also, example scenarios are presented in Section 4 for illustrative

purposes. Tier 1 activities and results are not reported to the U.S. Congress, but they are used

by the NRC as a diagnostic tool. The Tier 1 results will be placed on the NRC Web site for

access by interested stakeholders. Therefore, the proposed Tier 1, performance-based

monitoring of risk-significant initiating event types supports many uses of ITP results discussed

in the main body of this paper.

2.3 Risk-Informed Monitoring of Initiating Events Cornerstone of Safety (Tier 2)

An integrated performance indicator is proposed for ITP Tier 2 coverage of the Initiating Events

Cornerstone of Safety. It involves evaluating the risk significance of changes in industry

initiating event class performance (the results of the Tier 1 activity). Risk significance is

evaluated in terms of a measure related to core damage frequency (CDF) or to changes in the

measure (related to CDF). The indicator combines operating experience for risk-significant

initiating event classes with associated internal event CDF-based importance information. The

indicator is able to appropriately combine frequent and infrequent initiating event class

frequencies with different risk measures (Birnbaum importances). This indicator is termed the

Baseline Risk Index for Initiating Events, or BRIIE. The main use of the BRIIE would be to

combine individual initiating event class performance changes into an integrated, risk-informed

indicator at the Initiating Events Cornerstone of Safety level. The BRIIE would solve several

deficiencies in the present ITP: no systematic and defined method for determining whether

individual initiating event class performance changes or adverse trends are risk significant, no

systematic and defined method for integrating individual initiating event class performance

changes into an overall risk result at the Cornerstone of Safety level, and untimely risk-informed

industry trend results. Results of the BRIIE would be reported to the U.S. Congress.

Several different quantification methods were considered for evaluating the BRIIE. One

method related to CDF is the following:

(1)

BRIIE B

B

i

i

m

ic

i

ic

=

=

=

=

∑

1

λ

λ

*

*

where

 industry - average Birnbaum for initiating event i

 common industry current frequency for initiating event i

Another formulation, related to changes in CDF ( CDF), is given by the following equation:
-5-

(2)

BRIIE B

B

i

i

m

ic ib

i

ic

ib

= −

=

=

=

=

∑

1

( ), *

*

λ λ

λ

λ

where

 industry - average Birnbaum for initiating event i

 common industry current frequency for initiating event i

 baseline frequency for initiating event i

BWRs and PWRs have different core damage frequencies, which depend to some extent on

different initiating event types. The risk weights for various initiating events are also different

for the two types of reactors. Therefore, BRIIE results are proposed for each reactor type. 

However, the two BRIIE results could be combined into a single index, if desired.

The BRIIE formulations in Equations 1 and 2 use industry-average (or PWR- or BWR-average)

Birnbaum importance measures and combine the industry-wide data to generate the “common

industry current frequency” for each initiating event type. Alternative formulations are possible

using plant-specific Birnbaum importances and plant-specific initiating event data and then

summing the individual plant results to obtain an industry result. Results using all of the various

calculation methods indicated that the proposed formulations in Equations 1 and 2 provide the

most BRIIE sensitivity.

BRIIE results, although representing industry-wide results, are presented as average results per

plant. This is done because NRC and stakeholders are more familiar with results per plant than

with integrated industry-wide impacts. If the PWR BRIIE result for a future year is calculated to

be 1.0E-6/year (per PWR), then the PWR-wide impact is actually (1.0E-6/year/PWR)(69

PWRs) = 6.9E-5/year. Similarly, if the BWR BRIIE is 1.0E-6/year (per BWR), then the BWRwide impact is actually (1.0E-6/year/BWR)(34 BWRs) = 3.4E-5/year. These industry-wide

impacts should be kept in mind when establishing reporting thresholds for the BRIIE.

Simulations of expected future performance of the PWR and BWR BRIIEs were performed to

understand the nature of the distribution and to obtain 95% and 99% prediction limits. 

Reporting thresholds for the two BRIIEs should be established considering the following

information:

(1) Uncertainty in the BRIIEs and the 95% and 99% prediction limits from simulations

(2) Distribution of the Birnbaum importance measures for each initiating event class and

understanding of the groups of plants that have large values for each class

(3) Major contributors (i.e., dominant initiating event classes) to the BRIIEs

(4) Sensitivity of BRIIEs to initiating event classes, especially those with lower frequencies

(5) Other factors, such as the NRC safety goal policy and Regulatory Guide 1.174
-6-

An expert panel would be established to propose BRIIE threshold values that satisfy policy and

operational needs and objectives.

3 BRIIE Historical Performance and Sensitivity

Historical performance of the BRIIE ( CDF basis) is shown in Figure A2-3 for FY 1997 through

FY 2002. For all 6 years, the PWR BRIIE dominates the BWR BRIIE in terms of variance from

a baseline CDF of 0.0. This shows why separate BRIIEs are proposed for PWRs and BWRs. 

For FY 1999, the PWR BRIIE peak of approximately 1.1E-5/year (per plant) is driven by two

occurrences of a loss of DC bus. This illustrates the sensitivity of the BRIIE to relatively

infrequent but risk-important initiating event types.

-2.00E-05

-1.50E-05

-1.00E-05

-5.00E-06

0.00E+00

5.00E-06

1.00E-05

1.50E-05

1997 1998 1999 2000 2001 2002

Fiscal Year

Change in CDF Index(per year)

PWR BWR

Figure A2-3 PWR and BWR BRIIE historical performance

To further investigate the sensitivity of the BRIIE to changes in frequency of individual initiating

event types, results were calculated assuming all initiating event types are occurring at baseline

frequency except for one, which is occurring at the Tier 1 prediction limit (95% or 99%). 

Sensitivity studies show the following. The PWR BRIIE is most sensitive to the small loss-ofcoolant accident (LOCA) and loss of DC bus initiator types. Loss of offsite power, steam

generator tube rupture, and stuck-open safety/relief valve initiator types also can affect the

PWR BRIIE. Sensitivity results for the BWR BRIIE are different, with the loss of offsite power

initiator class significantly affecting the BRIIE. To a lessor degree, loss of DC bus and

stuck-open safety/relief valve initiator classes can affect BWR BRIIE results.

4 Potential Responses to Tier 1 and Tier 2 Results

In this section we present two examples to show how the ITP might treat initiating event

performance changes. For the first example, suppose we observe four events in 1 year that

are classified as small/very small break LOCAs. Each event occurred in a separate plant. This

initiating event is very rare. The 95% prediction limit is three events, and the 99% prediction
-7-

limit is four events. We have exceeded the 95% prediction limit and hit the 99% prediction limit. 

Because the number of actual events exceeds the prediction limit, this initiating event is a

candidate for further investigation.

Because small LOCAs do not occur very often, NRC would probably look at each event in more

detail after it occurred. Thus, NRC would have inspectors and staff reviewing each event. The

ITP would look at these events to see if there were similarities among the events and to provide

any lessons learned from this evaluation. These lessons would be communicated to the

industry via some type of generic communication. Further regulatory action would probably not

be necessary since the NRC investigated each event in detail.

Although BRIIE calculations have not been performed for this hypothetical case, the sensitivity

study results (assuming the other initiating event types are at baseline performance) indicate

that the PWR BRIIE would be approximately 9E-5/year (per PWR). This is significantly above

the prediction limits of 2.2E-5/year (95%) and 3.5E-5/year (99%). However, an expert panel will

decide what the actual thresholds for reporting to Congress will be. If the threshold were

chosen to be the 99% prediction limit, then the report to Congress would identify the Initiating

Events Cornerstone of Safety as a risk-significant departure from expected baseline

performance and safety.

As another example, suppose that we see a marked increase in the number of general

transients at a certain reactor type (PWR or BWR). We observe 74 general transients for the

year. This exceeds both the 95% and 99% prediction limits, which are 61 and 67 respectively

at the Tier 1 level. However, no unit has exceeded the white/green ROP threshold for scrams. 

The ITP investigates this situation and finds that the majority of the scrams occurred at plants

of a specific reactor type. Further investigation reveals that a given device has been the cause

of the majority of the general transients. This information could be communicated to industry

via some type of generic communication.

In terms of reporting to Congress, at the Tier 2 level the BRIIE would probably indicate that this

increase in general transients is not risk significant and would not exceed the threshold.
82

79 79

82

77

74 73

70

73

78 76 78 77

81

75

16 16 16 15

22

25 25

28

21 19

23

19 21

18

22

2

5 5

333 4 4 5 4

2 2 1 02 11111111 3 1 1 3333 2222

0000 1111111

0

10

20

30

40

50

60

70

80

90

100

2Q2000 3Q2000 4Q2000 1Q2001 2Q2001 3Q2001 4Q2001 1Q2002 2Q2002 3Q2002 4Q2002 1Q2003 2Q2003 3Q2003 4Q2003

Number of Plants

Licensee Response Column

Regulatory Response Column

Degraded Cornerstone Column

Multiple/Repetive Degraded Cornerstone Column

MC 0350 Process

Figure A3-1

Action Matrix Summary

The NRC provided oversight of 103 operating power reactors using the Reactor Oversight

Process (ROP). On average, approximately 75% of the plants were listed in the Licensee

Response column of the ROP Action Matrix, which corresponds to the baseline level of NRC

oversight. The chart below shows trends in the numbers of plants that are listed in the

Regulatory Response, Degraded Cornerstone, Multiple/Repetitive Degraded Cornerstone, and

Unacceptable Performance columns of the Action Matrix, which correspond to increasing levels

of regulatory engagement with the licensee, plus the number of plants in the MC 0350 process.

ATTACHMENT 4