NUREG/CR-6932 

INL/EXT-06-11950 

Baseline Risk Index for 

Initiating Events 

(BRIIE) 

Idaho National Laboratory 

U.S. Nuclear Regulatory Commission 

Office of Nuclear Regulatory Research 

Washington, DC 20555-0001 
 NUREG/CR-6932 

 INL/EXT-06-11950 

Baseline Risk Index for 

 Initiating Events 

(BRIIE) 

Manuscript Completed: May 2007 

Date Published: June 2007 

Prepared by 

S.A. Eide1, T.E. Wierman1, C.D. Gentillon1

D.M. Rasmuson2

C.L. Atwood3 

1

Idaho National Laboratory 

Risk, Reliability and NRC Programs Department 

Idaho Falls, ID 83415 

3

Statwood Consulting 

2905 Covington Road 

Silver Spring, MD 20910 

D.M. Rasmuson, Technical Monitor 

Prepared for 

2

Division of Risk Assessment and Special Projects 

Office of Nuclear Regulatory Research 

U.S. Nuclear Regulatory Commission 

Washington, DC 20555-0001 

NRC Job Code Y6546 
iii

ABSTRACT 

This report summarizes an effort to enhance the U.S. Nuclear 

Regulatory Commission Industry Trends Program in the area of 

initiating events. The proposed enhancement is termed the Baseline Risk 

Index for Initiating Events (BRIIE). The BRIIE includes two tiers. Tier 

1 involves the monitoring of individual initiating events at the industry 

level against performance-based prediction limits, while Tier 2 involves 

an integrated, risk-informed indicator at the industry level that combines 

the risks from individual initiating events. Technical bases for decisions 

made in the development of the BRIIE are also presented, as well as 

historical results at both the Tier 1 and Tier 2 levels. 
iv
v

FOREWORD 

This report documents the bases for the Baseline Risk Index for Initiating Events (BRIIE) under 

the U.S. Nuclear Regulatory Commission’s (NRC’s) Industry Trends Program (ITP). BRIIE is an 

integrated industry-level initiating event performance indicator that is risk informed. Performance-based 

limits are also included for individual initiating events. The report documents the selection of initiating 

events for monitoring, identification of risk-informed weights for these initiating events (Birnbaum 

importance measures), determination of performance-based limits for individual initiating events (at the 

industry level), and selection of a risk threshold for reporting purposes. Reasons for developing the BRIIE 

and the development process are outlined below. 

NRC oversees plant safety performance for individual plants using both inspection findings and 

plant-specific performance indicators as part of its Reactor Oversight Process. NRC staff address 

individual issues that have generic safety significance using other processes, including the generic 

communications process and the generic issue process. As discussed in SECY-01-0111, “Development of 

an Industry Trends Program for Operating Power Reactors,” the NRC’s Office of Nuclear Reactor 

Regulation (NRR) initiated the ITP in 2001 to complement these processes. NRR uses the ITP to monitor 

and assess industry-level trends in safety performance. As indicated in the SECY, NRR recognized the 

need to investigate alternative performance indicators that are more risk informed and more integrated 

than those initially included in the ITP. Such enhancements allow for more accurate and transparent 

evaluations of risk implications of adverse trends. In addition, if lower-level results are integrated at a 

higher level, the enhancements lead to fewer indicators. This stated need led to user need requests to the 

NRC’s Office of Nuclear Regulatory Research, asking for support in this area. 

This report outlines a three-step process to enhance the ITP coverage of the Initiating Events 

Cornerstone of Safety. The Initiating Events Cornerstone of Safety was chosen for this development 

effort because of its importance to risk and the availability of industry-wide data for a variety of initiating 

events. The first step was to identify an appropriate set of risk-significant initiating events. The BRIIE 

covers nine initiating events applicable to boiling water reactors (BWRs) and ten applicable to pressurized 

water reactors (PWRs). The second step was to establish performance-based prediction limits for each 

initiating event. The third and final step was to develop an integrated, risk-informed indicator by 

combining the individual initiating event information. 

The Tier 1 activity under the BRIIE involves monitoring yearly industry performance of risksignificant initiating events against prediction limits. To accomplish this, the staff established up-to-date 

baseline frequencies for each of the risk-significant initiating events. The staff then determined 

performance-based prediction limits using these baseline frequencies and estimated yearly industry 

reactor critical years of operation. If a prediction limit (number of events within a year) is exceeded, it 

indicates that industry performance has degraded relative to baseline performance. Data for these 

initiating events (numbers of event occurrences and corresponding reactor critical years) are already being 

collected and analyzed by NRC on a continual basis, so no additional data collection is needed to support 

the Tier 1 activity. 

Yearly evaluation of the integrated performance indicator provides the BRIIE Tier 2 coverage of 

the Initiating Events Cornerstone of Safety. This Tier 2 activity evaluates the risk significance of changes 

in industry initiating event performance (the results of the Tier 1 activity). Risk significance is evaluated 

in terms of estimated change in core damage frequency (CDF). Birnbaum importance measures provide 

the appropriate risk-informed weights for the various initiating event performance changes. 

Using the BRIIE Tier 2 integrated indicator, NRC staff are able to combine changes in initiating 

event frequency performance from a number of initiators into a single risk measure at the cornerstone of 
vi

safety level. Results of this evaluation will help the ITP communicate industry-level information to 

Congress and other stakeholders in an effective and timely manner. 

As described in this report, BRIIE enhances the existing ITP coverage of initiating events as 

follows: 

• Expands initiating event coverage from several to nine BWR and ten PWR initiating events, 

which expands the CDF risk coverage of internal events from less than 20% to approximately 

60% 

• Eliminates the overlap of initiating event indicators 

• Provides performance-based prediction limits for each of the initiating events (BRIIE Tier 1) 

• Converts the current performance of the individual initiating events into an integrated risk 

measure (approximating changes in CDF) to assess the risk significance of changes in BWR and 

PWR initiating event performance (BRIIE Tier 2) 

• Provides a threshold of risk significance for the integrated risk measure. 

The BRIIE will be implemented following the process outlined in the NRC Inspection Manual, 

Manual Chapter 0313, “Industry Trends Program.” 

 _______________________________ 

 Brian W. Sheron, Director 

 Office of Nuclear Regulatory Research 

 U.S. Nuclear Regulatory Commission 
vii

CONTENTS 

Abstract........................................................................................................................................................iii

Foreword....................................................................................................................................................... v

Executive Summary ..................................................................................................................................... xi

Acknowledgments.....................................................................................................................................xvii

Abbreviations............................................................................................................................................. xix

1. Introduction........................................................................................................................................... 1

2. Industry Trends Program....................................................................................................................... 3

3. Reasons for Enhancing Current ITP Performance Indicators ............................................................... 7

4. BRIIE .................................................................................................................................................... 9

4.1 Identification of Risk-Significant and Monitorable Initiating Events..................................... 10

4.2 Performance Monitoring of Risk-Significant Initiating Events (Tier 1) ................................. 12

4.3 Risk-Informed Monitoring of Initiating Events Cornerstone of Safety (Tier 2) ..................... 12

5. Stakeholder and MSPI Interaction and Expert Panel .......................................................................... 17

6. BRIIE Historical Performance ............................................................................................................ 19

7. Responses to Tier 1 and Tier 2 Results............................................................................................... 25

8. Bases for the BRIIE ............................................................................................................................ 27

8.1 Selection of Initiating Events .................................................................................................. 27

8.2 Determination of Initiating Event Baselines ........................................................................... 27

8.3 Tier 1 Prediction Limits .......................................................................................................... 28

8.4 Tier 2 Birnbaum Importances.................................................................................................. 29

8.5 BRIIE Tier 2 Formulations...................................................................................................... 29

8.6 Tier 2 Threshold ...................................................................................................................... 31

8.7 Data Period.............................................................................................................................. 32

9. BRIIE Tier 2 Uncertainty and Sensitivity........................................................................................... 33

9.1 Baseline CDF .......................................................................................................................... 33

9.2 Predictive Distribution for BRIIE ........................................................................................... 36

9.3 Sensitivity of BRIIE to High Counts for Individual Initiators ................................................ 38

9.4 Sensitivity of BRIIE to Non-Baseline Conditions................................................................... 40
viii

10. Summary........................................................................................................................................ 43

11. References...................................................................................................................................... 45

Glossary ...................................................................................................................................................... 47

APPENDICES 

Appendix A— Initiating Event Data and Baselines .................................................................................A-1 

Appendix B— Initiating Event Prediction Limits .................................................................................... B-1 

Appendix C— Initiating Event Birnbaum Importance Measures............................................................. C-1 

Appendix D— BRIIE Tier 2 Formulations...............................................................................................D-1 

Appendix E— Summary of Expert Panel Meeting................................................................................... E-1 

Appendix F— Resolution of Comments....................................................................................................F-1 

FIGURES 

Figure ES-1. BRIIE ∆CDF historical results. ............................................................................................. xv

Figure 2-1. Regulatory framework for the Reactor Oversight Process......................................................... 3

Figure 4-1. BRIIE and the ITP process......................................................................................................... 9

Figure 6-1. Tier 1 historical performance for TRAN (PWR). .................................................................... 20

Figure 6-2. Tier 1 historical performance for LOOP. ................................................................................. 20

Figure 6-3. BRIIE Tier 2 (∆CDF) historical performance.......................................................................... 21

Figure 6-4. BWR CDF breakdown by FY. ................................................................................................. 22

Figure 6-5. PWR CDF breakdown by FY................................................................................................... 23

Figure 9-1. Simulated baseline BRIIE CDF distribution for an average BWR. ......................................... 35

Figure 9-2. Simulated baseline BRIIE CDF distribution for an average PWR........................................... 36
ix

Figure 9-3. Simulated predictive distribution of BRIIE for BWRs. ........................................................... 37

Figure 9-4. Simulated predictive distribution of BRIIE for PWRs............................................................. 38

Figure 9-5. BWR BRIIE ∆CDF sensitivity to individual initiating event 95% prediction limits............... 39

Figure 9-6. PWR BRIIE ∆CDF sensitivity to individual initiating event 95% prediction limits. .............. 40

Figure 9-7. Simulated cumulative distribution of BRIIE (∆CDF) for industry under three assumptions. . 41

TABLES 

Table ES-1. BRIIE initiating events. .........................................................................................................xiii

Table ES-2. BRIIE Tier 1 performance-based prediction limits................................................................ xiv

Table 2-1. Current ITP performance indicators grouped by cornerstone of safety....................................... 6

Table 4-1. Summary of the BRIIE development process. .......................................................................... 10

Table 4-2. BRIIE initiating events. ............................................................................................................. 11

Table 4-3. BRIIE initiating event baseline data and frequency distributions. ............................................ 13

Table 4-4. BRIIE Tier 1 performance-based prediction limits. .................................................................. 14

Table 4-5. Plant-group average Birnbaum values for initiating events....................................................... 14

Table 6-1. BWR initiating event counts by FY........................................................................................... 22

Table 6-2. PWR initiating event counts by FY........................................................................................... 23

Table 7-1. Relationship between the ROP, ITP and report to Congress..................................................... 25

Table 8-1. Plant-specific versus group approaches to quantifying the BRIIE Tier 2. ................................ 30

Table 8-2. Evaluation level and frequency calculation approaches to quantifying the BRIIE Tier 2......... 31

Table 9-1. Breakdown of BRIIE baseline CDF for BWRs......................................................................... 34

Table 9-2. Breakdown of BRIIE baseline CDF for PWRs. ........................................................................ 34
x
xi

EXECUTIVE SUMMARY 

The Baseline Risk Index for Initiating Events (BRIIE) described in this document is an 

enhancement to the Industry Trends Program (ITP) in the Initiating Events Cornerstone of Safety. The 

BRIIE enhances the ITP coverage of initiating events by the following: 

• Expanding initiating event coverage from several to nine boiling water reactor (BWR) and ten 

pressurized water reactor (PWR) initiating events, which expands the core damage frequency 

(CDF) risk coverage from internal events from less than 20% to approximately 60% 

• Eliminating overlapping of initiating event indicators 

• Providing performance-based prediction limits (yearly numbers of events) for each of the 

initiating events (BRIIE Tier 1) that, if reached or exceeded, indicate potential degradation of 

industry performance 

• Assembling the individual initiating event current performance along with Birnbaum importance 

measures into an integrated risk measure (approximating CDF) that can be used to assess the risk 

significance of changes in BWR and PWR initiating event performance (BRIIE Tier 2) 

• Providing a threshold of risk significance for the integrated risk measure. 

Existing U.S. Nuclear Regulatory Commission (NRC) programs provide the yearly initiating event 

data and the Birnbaum importance measures needed for the BRIIE, so no additional data collection is 

needed. At present the BRIIE addresses only internal event CDF, and does not include external events or 

large early release frequency. The following provides an introduction to the ITP and the development of 

the BRIIE. 

NRC provides oversight of plant safety performance on a plant-specific basis using both inspection 

findings and plant-level performance indicators as part of its Reactor Oversight Process (ROP). Public 

health and safety assurance is divided into three strategic performance areas: reactor safety, radiation 

safety, and safeguards. These areas are subdivided into seven cornerstones of safety: initiating events, 

mitigating systems, barrier integrity, emergency preparedness, public radiation safety, occupational 

radiation safety, and physical protection. Individual issues that are identified as having generic safety 

significance are addressed using other NRC processes, including the generic communications process and 

the generic issue process. 

As discussed in SECY-01-0111, “Development of an Industry Trends Program for Operating 

Power Reactors,” the NRC’s Office of Nuclear Reactor Regulation initiated the ITP in 2001 to 

complement these processes by monitoring and assessing industry-level trends in safety performance. The 

purposes of the ITP are to provide a means to confirm that the nuclear industry is maintaining the safety 

performance of operating reactors and, by clearly demonstrating that performance, to enhance stakeholder 

confidence in the efficacy of the NRC’s processes. Objectives of the ITP are the following: 

• Collect and monitor industry-wide data that can be used to assess whether the nuclear industry is 

maintaining the safety performance of operating plants and to provide feedback on the ROP. 

• Assess the safety significance and causes of any statistically significant adverse industry trends, 

determine if the trends represent an actual degradation in overall industry safety performance, and 

respond appropriately to any safety issues that may be identified. 

• Communicate industry-level information to Congress and other stakeholders in an effective and 

timely manner. 

• Support the NRC’s performance goal of ensuring safety while enhancing public confidence in the 

agency’s regulatory process. 
xii

From SECY-06-0076, current uses of the ITP results include the following: 

• Annual results are reported to Congress in the NRC’s Performance and Accountability Report 

Fiscal Year 200X (NUREG-1542 series) and Budget Estimates and Performance Plan Fiscal Year 

200X (NUREG-1100 series). These reports indicate whether NRC has met the performance goal 

of “no statistically significant adverse industry trends in safety performance.” 

• ITP results are posted on the NRC public website. 

• ITP results are a key element reviewed by senior NRC management in the Agency Action 

Review Meeting (AARM). 

• ITP results are reported to the Commission in an annual report coinciding with the AARM. 

• The Commission uses ITP results to present the status of industry performance to NRC’s 

oversight committees and at major conferences with industry. 

• NRC managers use ITP results in presentations to industry, such as the NRC’s Regulatory 

Information Conference. 

Additional information on the ITP and how enhancements to the program will be incorporated can be 

found in the NRC Inspection Manual, Manual Chapter 0313, “Industry Trends Program.” 

Current ITP performance indicators have both strengths and weaknesses. Strengths include 

availability of historical results, continuity and consistency in yearly evaluations, and broad coverage of 

the cornerstones of safety. However, weaknesses in the Initiating Events Cornerstone of Safety include 

(1) overlapping coverage for certain cornerstones, (2) limited risk coverage, and (3) difficulties in 

interpreting the risk significance of significant adverse trends. 

To enhance the ITP coverage of the Initiating Events Cornerstone of Safety, a three-step process 

was used. The first step identified appropriate initiating events included in the standardized plant analysis 

risk (SPAR) models. Table ES-1 lists the initiating events chosen. These are risk-significant and 

monitorable at the plant-type or industry level (i.e., they have occurred in the last 15 years). Also shown 

in Table ES-1 are the existing ITP performance indicators covering initiating events. 

The second step developed performance-based prediction limits for these individual initiating 

events (BRIIE Tier 1). The prediction limits, listed in Table ES-2, are performance based and include 

both aleatory uncertainty (the randomness of the event count in the future year) and epistemic uncertainty 

(lack of perfect knowledge of the value of the baseline frequency). They represent upper limits of 

initiating event counts in a year that, if reached or exceeded, indicate a potential degradation in plantgroup or industry performance. In recent years, these prediction limits were exceeded twice. In fiscal year 

(FY) 2003, the loss of offsite power (LOOP) event count was 12, which exceeded the prediction limit of 

eight listed in Table ES-2. However, eight of the 12 LOOPs that year were the result of a single gridrelated event. Also in FY 2003, the prediction limit for PWR general transients was exceeded. Multiple 

causes appeared to contribute to the large number of such events that year. 

The third step developed an integrated, risk-informed indicator at the industry level by combining 

the individual initiating event information with Birnbaum importance measures for each initiating event 

(BRIIE Tier 2, or BRIIE for short). NRC-developed standardized plant analysis risk (SPAR) models 

covering U.S. commercial nuclear power plants provided the Birnbaums. The BRIIE estimates the change 

in CDF (∆CDF) for an average plant resulting from changes in industry initiating event performance. 

Comprehensive uncertainty and sensitivity studies were conducted to characterize the BRIIE. Historical 

results for the BRIIE are presented in Figure ES-1. Using the ∆CDF format, the BWR, PWR, and 

industry baselines are 0.0. Therefore, bars extending above the zero horizontal line represent worse than 

baseline performance in terms of CDF, and bars extending below the zero horizontal line represent better 

than baseline performance. An expert panel proposed the threshold for the BRIIE for reporting to 
xiii

Congress, a ∆CDF of 1.0E–05 per reactor critical year. BRIIE results indicate that this threshold was 

exceeded in FY 1989, but recent results lie significantly below this threshold. 

Table ES-1. BRIIE initiating events. 

Initiating 

Event 

Identifier PWR BWR Industry Related ROP Initiating 

Event Performance 

Indicator 

Related Ex-AEOD 

Initiating Event 

Performance Indicator 

General 

Transient 

TRAN X X Unplanned Scrams Automatic Reactor Scrams 

while Critical (does not 

include unplanned manual 

scrams) 

Loss of 

Condenser 

Heat Sink 

LOCHS X X Scrams with Loss of 

Normal Heat Removal 

Counted under the 

Automatic Reactor Scrams 

while Critical indicator. 

However, the functional 

and risk impacts on the 

plant are not covered. 

Loss of Main 

Feedwater 

LOMFW X Scrams with Loss of 

Normal Heat Removal 

Same comment 

Loss of Offsite 

Power 

LOOP X Counted under the 

Unplanned Scrams 

indicator. However, the 

functional and risk 

impacts on the plant are 

not covered. Also, this 

event is too rare to 

monitor separately on a 

plant-specific basis. 

Same comment 

Loss of Vital 

AC Bus 

LOAC X Same comment Same comment 

Loss of Vital 

DC Bus 

LODC X Same comment Same comment 

Stuck Open 

SRV 

SORV X X Same comment Same comment 

Loss of 

Instrument Air 

LOIA X X Same comment Same comment 

Very Small 

LOCA 

VSLOCA X Same comment Same comment 

Steam 

Generator 

Tube Rupture 

SGTR X N/A Same comment Same comment 

Stakeholder interaction associated with three main activities benefited the BRIIE effort: a request 

for public review of a draft version of the BRIIE, a presentation to the Advisory Committee on Reactor 

Safeguards (ACRS), and a public workshop. These occurred in 2003. In addition to the stakeholder 

interaction, an expert panel was convened in July 2006 to provide suggestions concerning the draft BRIIE 

effort and to recommend thresholds for the BRIIE Tier 2. 

The BRIIE development effort also benefited from experience and lessons learned from the recent 
xiv

implementation of the Mitigating Systems Performance Index (MSPI) within the ROP. Implementation of 

the MSPI included significant effort in the area of probabilistic risk assessment (PRA) quality, including a 

comprehensive comparison of licensee risk models with the NRC’s SPAR models. That effort helped to 

improve both the licensee and NRC risk models. Also, the method used in the MSPI to estimate impacts 

on CDF of changes in mitigating systems performance is similar to the final approach used in the BRIIE. 

Table ES-2. BRIIE Tier 1 performance-based prediction limits. 

Initiating Event Identifier Mean 

(1/rcry) 

Estimated rcry 

Per Year for 

Plant Group 

(note a) 

Expected 

Number of 

Events Per Year 

(note b) 

95% 

Prediction 

Limit 

General Transient—PWR TRAN (PWR) 7.51E–01 62.1 46.6 59 

General Transient—BWR TRAN (BWR) 8.30E–01 30.6 25.4 35 

Loss of Condenser Heat 

Sink—PWR 

LOCHS (PWR) 8.11E–02 62.1 5.0 10 

Loss of Condenser Heat 

Sink—BWR 

LOCHS (BWR) 1.97E–01 30.6 6.0 11 

Loss of Main Feedwater LOMFW 9.59E–02 92.7 8.9 15 

Loss of Offsite Power LOOP 3.59E–02 92.7 3.3 8 

Loss of Vital AC Bus LOAC 8.80E–03 92.7 0.8 3 

Loss of Vital DC Bus LODC 1.17E–03 92.7 0.1 2 

Stuck Open SRV—PWR SORV (PWR) 2.88E–03 62.1 0.2 2 

Stuck Open SRV—BWR SORV (BWR) 2.23E–02 30.6 0.7 3 

Loss of Instrument Air 

—PWR 

LOIA (PWR) 9.81E–03 62.1 0.6 3 

Loss of Instrument Air 

—BWR 

LOIA (BWR) 1.02E–02 30.6 0.3 2 

Very Small LOCA VSLOCA 1.55E–03 92.7 0.1 2 

Steam Generator Tube 

Rupture—PWR 

SGTR (PWR) 3.54E–03 62.1 0.2 2 

Note a – There are 34 BWRs and 69 PWRs (total of 103) in the U.S. commercial nuclear power plant industry. The 

rcry estimates represent 90% critical operation during a calendar year. Rcry is reactor critical year. 

Note b – The expected number of events is the mean frequency multiplied by the plant group rcry. 
xv

-1.0E-05

-5.0E-06

0.0E+00

5.0E-06

1.0E-05

1.5E-05

2.0E-05

1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Fiscal Year

BRIIE Tier 2 Delta CDF (1/rcry) 

BWR PWR Industry

Figure ES-1. BRIIE ∆CDF historical results. 
xvi
xvii

ACKNOWLEDGMENTS 

The authors appreciate the comments and guidance provided by T. Boyce, 

P. Harris, D. Wrona, A. Masciantonio, C. Carpenter, S. Richards, M. Satorius, J. 

Andersen, P. Baranowsky, and D. Dube of the U.S. Nuclear Regulatory 

Commission. Also, R. Youngblood participated in the early phases of this 

project. Comments received through the formal review process (Appendix F) 

helped improve the final report. Finally, the participants in the expert panel 

included R. Barrett, N. Chokshi, T. Boyce, J. Vail, and R. Haag, all of the U.S. 

Nuclear Regulatory Commission. 
xviii
xix

ABBREVIATIONS 

AARM Agency Action Review Meeting 

ac alternating current 

ACRS Advisory Committee on Reactor Safeguards 

ADAMS Agencywide Documents Access and Management System 

AEOD Office for Analysis and Evaluation of Operational Data 

AIT augmented inspection team 

ASP Accident Sequence Precursor 

BRIIE Baseline Risk Index for Initiating Events 

BWR boiling water reactor 

CCDP conditional core damage probability 

CDF core damage frequency 

CNID constrained noninformative distribution 

CY calendar year 

dc direct current 

EB empirical Bayes 

ERO Emergency Response Organization 

FY fiscal year 

IEDB initiating events database 

IIT incident investigation team 

IPE Individual Plant Examination 

ITP Industry Trends Program 

LER licensee event report 

LOAC loss of vital ac bus 

LOCA loss-of-coolant accident 

LOCHS loss of condenser heat sink 

LODC loss of vital dc bus 

LOIA loss of instrument air 

LOMFW loss of main feedwater 

LOOP loss of offsite power 

MLE maximum likelihood estimate 

MSPI Mitigating Systems Performance Index 

NRC Nuclear Regulatory Commission 

NRR Office of Nuclear Reactor Regulation 

ODCM offsite dose calculation manual 

PDR Public Document Room 

PRA probabilistic risk assessment 

PWR pressurized water reactor 
xx

RADS Reliability and Availability Database System 

RETS radiological effluent technical specification 

rcry reactor critical year 

ROP Reactor Oversight Process 

SGTR steam generator tube rupture 

SI surveillance inspection 

SORV stuck open SRV 

SPAR standardized plant analysis risk 

SRV safety relief valve 

TRAN general transient 

VSLOCA very small LOCA 

∆CDF change in core damage frequency 
1

Baseline Risk Index for Initiating Events (BRIIE) 

1. INTRODUCTION 

This report documents the development process and bases for the Baseline Risk Index for Initiating 

Events (BRIIE). The BRIIE was developed to enhance the U.S. Nuclear Regulatory Commission’s 

(NRC’s) Industry Trends Program (ITP) (Ref. 1) monitoring of industry performance in the area of 

initiating events. The BRIIE enhances the ITP coverage of initiating events by the following: 

• Expanding initiating event coverage from several to nine boiling water reactor (BWR) and ten 

pressurized water reactor (PWR) initiating events, which expands the core damage frequency 

(CDF) risk coverage from internal events from less than 20% to approximately 60% 

• Eliminating overlapping of initiating event indicators 

• Providing performance-based prediction limits (yearly numbers of events) for each of the 

initiating events (BRIIE Tier 1) that, if reached or exceeded, indicate potential degradation of 

industry performance 

• Assembling the individual initiating event current performance along with Birnbaum importance 

measures into an integrated risk measure (approximating CDF) that can be used to assess the risk 

significance of changes in BWR and PWR initiating event performance (BRIIE Tier 2) 

• Providing a threshold of risk significance for the integrated risk measure. 

Existing NRC programs provide the yearly initiating event data and the Birnbaum importance measures 

needed for the BRIIE, so no additional data collection is needed. 

At present the BRIIE addresses only internal event CDF, and does not include external events or 

large early release frequency. In addition, the BRIIE includes yearly comparisons of initiating event 

performance against prediction limits and BRIIE Tier 2 result against a risk significant threshold. The 

BRIIE does not include trending, but the ITP may decide to include BRIIE results (individual initiating 

events and the BRIIE Tier 2) in its trending program. Finally, the BRIIE is designed to detect 

performance deviations at the industry, BWR, or PWR level. Performance deviations within smaller 

groups of plants may not be detected without additional analyses (not planned at this time). 

The ITP and the development of the BRIIE are discussed in the following sections. Section 2 of 

this report summarizes the current ITP, Section 3 explains why an enhancement was desired in the area of 

initiating events, Section 4 describes the BRIIE, and Section 5 describes the review and expert panel 

processes used to provide guidance in the BRIIE development. Historical results for the BRIIE are 

presented in Section 6 and potential NRC responses to such results are described in Section 7. Technical 

bases for various aspects of the BRIIE are presented in Section 8. Various characteristics of the BRIIE, 

including uncertainty and sensitivity, are presented in Section 9. Section 10 summarizes the overall results 

of the BRIIE development process and references are presented in Section 11. Finally, Appendices A 

through F provide detailed information on initiating event performance, initiating event prediction limits, 

Birnbaum importance measures, BRIIE formulations, expert panel, and resolution of comments. 
2
3

2. INDUSTRY TRENDS PROGRAM 

The NRC provides oversight of plant safety performance on a plant-specific basis using both 

inspection findings and plant-level performance indicators as part of its Reactor Oversight Process (ROP) 

(Ref. 2). The regulatory framework for the ROP is illustrated in Figure 2-1. Inputs to this regulatory 

framework are the seven cornerstones of safety: initiating events, mitigating systems, barrier integrity, 

emergency preparedness, public radiation safety, occupational radiation safety, and physical protection. 

These inputs can take two forms—performance indicators or inspection findings. Individual issues that 

are identified as having generic safety significance are addressed using other NRC processes, including 

the generic communications process and the generic issue process. 

Performance Indicators

Performance Indicators

Significance Evaluations

Management Conference

Monitor Licensee Actions

NRC Inspections

Additional Regulatory Actions

Assessment Process

(Action Matrix)

Enforcement

Significance Determination Process

Supplemental

Inspections

Agency Response Public Meetings

Communications

 Press Releases

 NRC Web Site

 PDR/ADAMS

Assessment Reports

Inspection Plans

Inspection Findings

Performance Indicators

Inspections

Significance Evaluations

Performance Results in all 7 Cornerstones of Safety

Cornerstones of Safety

Performance Indicator Thresholds

Event Response

(SI/AIT/IIT)

Risk Informed

Baseline Inspections

Generic Safety

Inspections

Figure 2-1. Regulatory framework for the Reactor Oversight Process. 

As discussed in SECY-01-0111, “Development of an Industry Trends Program [ITP] for Operating 

Power Reactors” (Ref. 3), the NRC’s Office of Nuclear Reactor Regulation (NRR) initiated the ITP in 

2001 to complement these existing processes by monitoring and assessing industry-level trends in safety 

performance. The purposes of the ITP are to provide a means to confirm that the nuclear industry is 

maintaining the safety performance of operating reactors and, by clearly demonstrating that performance, 

to enhance stakeholder confidence in the efficacy of the NRC’s processes. The objectives of the ITP are 

the following: 

• Collect and monitor industry-wide data that can be used to assess whether the nuclear industry is 

maintaining the safety performance of operating plants and to provide NRC feedback to its 

nuclear reactor safety inspection and licensing programs. 

• Assess the safety significance and causes of any statistically significant adverse industry trends, 
4

determine if the trends represent an actual degradation in overall industry safety performance, and 

respond appropriately to any safety issues that may be identified. 

• Communicate industry-level information to Congress and other stakeholders in an effective and 

timely manner. 

• Support the NRC’s performance goal of ensuring safety while enhancing public confidence in the 

agency’s regulatory process. 

Yearly summaries of ITP results and development efforts are presented in the following 

documents: 

• SECY-02-0058, “Results of the Industry Trends Program for Operating Power Reactors and 

Status of Ongoing Development” (Ref. 4) 

• SECY-03-0057, “FY 2002 Results of the Industry Trends Program for Operating Power Reactors 

and Status of Ongoing Development” (Ref. 5) 

• SECY-04-0052, “FY 2003 Results of the Industry Trends Program for Operating Power Reactors 

and Status of Ongoing Development” (Ref. 6) 

• SECY-05-0069, “FY 2004 Results of the Industry Trends Program for Operating Power Reactors 

and Status of Ongoing Development” (Ref. 7) 

• SECY-06-0076, “FY 2005 Results of the Industry Trends Program for Operating Power Reactors 

and Status of Ongoing Development” (Ref. 8) 

According to the ITP website (Ref. 1), the ITP is monitoring the performance of six industry 

indicators developed by the former NRC Office for Analysis and Evaluation of Operational Data 

(AEOD), 17 ROP performance indicators (the indicators for the Physical Protection Cornerstone of Safety 

are not publicly available), and two indicators from the Accident Sequence Precursor (ASP) Program. 

These indicators are listed in Table 2-1, organized by cornerstones of safety. The AEOD performance 

indicators, termed “ex-AEOD” performance indicators, were reported annually in the NUREG-1187 

series (Ref. 9) up through 1999. Although there are eight ex-AEOD indicators, only six are included in 

the present ITP. The indicators “Significant Events” and “Cause Codes” are excluded. The ROP 

performance indicators are evaluated quarterly and presented on the NRC website. The ASP results were 

reported annually in the NUREG/CR-4674 series up through 1998 (Ref. 10). The current status of the 

ASP program is summarized in SECY-06-0208 (Ref. 11). This current set of diverse performance 

indicators was chosen for initial inclusion in the ITP based mainly on historical precedent and availability. 

From SECY-06-0076, current uses of the ITP results include the following: 

• Annual results are reported to Congress in the NRC’s Performance and Accountability Report 

Fiscal Year 200X (NUREG-1542 series, Ref. 12) and Budget Estimates and Performance Plan 

Fiscal Year 200X (NUREG-1100 series, Ref. 13). These reports indicate whether NRC has met 

the performance goal of “no statistically significant adverse industry trends in safety 

performance.” 

• Results are posted on the NRC public website. 

• Results are a key element reviewed by senior NRC management in the Agency Action Review 

Meeting (AARM). 

• Results are reported to the Commission in an annual report coinciding with the AARM. 
5

• The Commission uses ITP results to present the status of industry performance to NRC’s 

oversight committees and at major conferences with industry. 

• NRC managers use ITP results in presentations to industry, such as the NRC’s Regulatory 

Information Conference. 

Additional information on the ITP and how enhancements to the program will be incorporated can be 

found in the NRC Inspection Manual, Manual Chapter 0313, “Industry Trends Program” (Ref. 14). 
6 

Table 2-1. Current ITP performance indicators grouped by cornerstone of safety. 

Program Initiating Events Mitigating Systems Barrier Integrity Emergency 

Preparedness 

Public Radiation Safety 

Ex-AEOD Automatic reactor 

scrams while critical 

Safety system actuations 

 Safety system failures  Forced outage rate  Equipment forced outage 

rate/1000 critical hours 

ROP Unplanned scrams Mitigating Systems 

Performance Index (MSPI) – emergency ac power systems 

Reactor coolant system activity 

Drill/exercise performance 

Radiological effluent technical specification (RETS)/offsite dose calculation manual (ODCM) radiological effluent 

 Unplanned scrams 

with loss of normal heat removal 

MSPI – high-pressure injection systems 

Reactor coolant system leakage 

Emergency Response Organization (ERO) drill participation 

 Unplanned power 

changes 

MSPI – heat removal systems 

 Alert and notification 

system 

 MSPI – residual heat 

removal systems 

 MSPI – cooling water 

systems 

 Safety system functional 

failures – BWR 

 Safety system functional 

failures – PWR 

ASP Precursor occurrence rate  Conditional core damage probability 
7

3. REASONS FOR ENHANCING CURRENT ITP PERFORMANCE 

INDICATORS 

Current ITP performance indicators have both strengths and weaknesses. Strengths include 

availability of historical results, continuity and consistency in yearly evaluations, and broad coverage of 

the cornerstones of safety. However, weaknesses in the Initiating Events Cornerstone of Safety include 

(1) overlapping coverage for certain cornerstones, (2) limited risk coverage, and (3) difficulties in 

interpreting the risk significance of significant adverse trends. The next paragraphs elaborate on these 

three points. 

As an example, the Initiating Events Cornerstone of Safety includes the ex-AEOD indicator 

“automatic reactor scrams while critical,” while the ROP includes “unplanned scrams” and “scrams with 

loss of normal heat removal.” The indicators “automatic reactor scrams while critical” and “unplanned 

scrams” are similar except that unplanned manual scrams are included in the ROP indicator. In addition, 

an initiating event that fits within the ROP “scrams with loss of normal heat removal” would generally be 

counted in all three performance indicators. 

In terms of risk coverage, work documented in NUREG-1753, Risk-Based Performance Indicators: 

Results of Phase 1 Development (Ref. 15) indicates that the ROP indicators “unplanned scrams” and 

“scrams with loss of normal heat removal” probably include less than 20% of the total internal event core 

damage risk for the Initiating Events Cornerstone of Safety. (The other 80% of risk involves less frequent 

initiating events that cannot be monitored on a plant-specific basis over the limited, three-year time period 

covered by the ROP indicators.) This limited coverage of risk by the ROP performance indicators is 

supplemented by inspections. 

Finally, if an adverse trend is detected, for example for the ROP “unplanned scrams,” there would 

be difficulty in determining whether the adverse trend is risk significant. This is because not all scrams 

are equally serious in terms of risk. Also, an adverse trend in “unplanned scrams” might be offset by a 

favorable (decreasing frequency) trend in “scrams with loss of normal heat removal.” The current ITP 

has established neither a method for determining risk significance of broad categories such as “unplanned 

scrams,” nor a mechanism for aggregating and interpreting offsetting trends at the cornerstone of safety 

level. 

As a first step in enhancing the ITP to remedy the weaknesses discussed above, the Initiating Event 

Cornerstone of Safety was chosen as the area of focus. Work focused on development of performance 

indicators that did not overlap in coverage, significantly increased the risk coverage, and provided a 

mechanism for determining the risk significance of changes in performance, at both the individual 

initiating event level and at the integrated, cornerstone of safety level. The process and results are 

documented in the following sections. 
8
9

4. BRIIE 

To enhance the ITP coverage of the Initiating Events Cornerstone of Safety, a three-step process 

was used, termed the BRIIE. The first step was to identify appropriate initiating event categories. [The 

term “initiating event categories” indicates that the initiating events referred to in this attachment may 

include more than one initiating event type as defined in NUREG/CR-5750 (Ref. 16). For the rest of this 

document, the term “initiating events” is used instead of “initiating event categories” to make the reading 

easier.] Then methods for trending and establishing performance-based prediction limits for these 

individual initiating events were developed (BRIIE Tier 1). Finally, an integrated, risk-informed indicator 

at the industry level was developed by combining the individual initiating event information (BRIIE 

Tier 2). Figure 4-1 shows the ITP process with respect to the BRIIE. The process steps are also listed in 

Table 4-1. Each of these steps is discussed in this section. 

Figure 4-1. BRIIE and the ITP process. 

Collect Indicator Data

(Initiating events from licensee 

event reports)

Monitor Performance

(Short term issues, Prediction 

limits)

Estimate Trends

(Apply statistical methods to identify

short- and long-term adverse

trends)

Evaluate Adverse Trends

[Few plants (no adverse trend),

Plant comparison groups,

Examine agency databases,

Attempt to establish causes,

NRR technical branch review,

Assess safety significance]

Feedback to NRC

Programs

Agency Response

(No action / continue to monitor,

Engage Industry,

Generic communications,

Generic safety inspection, and

Generic issue)

Analysis results

(BRIIE, Thresholds)

Report to Congress

(NRC Performance &

Accountability Report,

Green/Blue Books,

Oversight Committees)

Communications

with Stakeholders

(NRC web page for ITP,

Annual report to Commission,

NRC Info Digest,

Industry conferences)

Senior Management

Review

(Agency Action Review Meeting)

Tier 2

Tier 1
10

Table 4-1. Summary of the BRIIE development process. 

Tier Step Comment 

Identify subset of initiating events that are monitorable over a short 

period (one or several years) 

NUREG/CR-5750 starting 

point 

Determine baseline frequencies for monitorable initiating events: 

• Identify baseline period (ending in CY 2002) 

• Determine baseline frequency distribution 

NUREG/CR-6928 (Ref. 17) 

Common 

to both 

Collect industry data (events and reactor critical years [rcry] for BWRs 

and PWRs) by FY 

NRC initiating events 

database (IEDB) 

Determine prediction limits for monitorable initiating events: 

• # events in a year indicating potential degradation in performance 

Tier 1 Section 8.3 

Compare FY events for each initiator with prediction limits Section 6 

Develop BRIIE equations: 

• Risk implications of FY initiating event performance 

Section 8.5 

Determine PWR- and BWR-average Birnbaum importance measures: 

• Risk-based “weight” for each initiating event 

Section 8.4 

Calculate FY BWR and PWR BRIIE results (∆CDF) Section 6 

Tier 2 

Compare FY BRIIE results with threshold Section 6 

4.1 Identification of Risk-Significant and Monitorable Initiating 

Events 

NUREG/CR-5750 provides data for a large number of initiating events for the period calendar year 

(CY) 1987 through CY 1995. (NRC is continually updating these data, but updates to this NUREG are no 

longer being published. Instead, the results are posted on the NRC website, http://nrcoe.inel.gov/results/.) 

Initiating events are defined in that study to be unplanned reactor trips that occur while a plant is critical 

and at or above the point of adding heat. A subset of these events was identified as being risk significant 

in NUREG-1753. The list of risk-significant initiating events included in the BRIIE is presented in 

Table 4-2. 

The list in Table 4-2 includes ten initiating events applicable to PWRs and nine applicable to 

BWRs. Initiating events broken down into separate PWR and BWR plant groups were shown to have 

statistically significant differences in frequencies in the original initiating events study. For the other 

initiating events, PWR and BWR frequencies were not significantly different, and both types of reactors 

were combined to obtain frequencies. That historical breakdown of initiating events is preserved in the 

BRIIE. In general, these risk-significant initiating events account for approximately 60% of the current 

internal event CDF risk (excluding internal flooding) from the 103 operating commercial nuclear power 

plants in the U.S. (The types of initiating events not included in the BRIIE, contributing the other 40%, 

are listed in Section 8.1.) In addition, there is no overlap among these initiating events. 

The three BRIIE initiating events in Table 3 [general transient (TRAN), loss of condenser heat sink 

(LOCHS), and loss of main feedwater (LOMFW)] that roughly correspond with ROP performance 

indicators (Table 2-1) occur frequently and can be monitored on a plant-specific basis over a period of 1 

to 3 years. However, their coverage of internal events CDF is less than 20%. The BRIIE includes the 

other risk-significant initiating events listed in Table 4-2 for two reasons: including these other events 

increases the industry average risk coverage to approximately 60% and these events are frequent enough 

to monitor at the plant-group (BWR or PWR) or industry-wide level on a yearly basis. 
11

Table 4-2. BRIIE initiating events. 

Initiating 

Event 

Identifie

r 

PWR BWR Industry Related ROP Initiating Event 

Performance Indicator 

Related Ex-AEOD 

Initiating Event 

Performance Indicator 

General 

Transient 

TRAN X X Unplanned Scrams Automatic Reactor 

Scrams while Critical 

(does not include 

unplanned manual 

scrams) 

Loss of 

Condenser 

Heat Sink 

LOCHS X X Scrams with Loss of Normal 

Heat Removal 

Counted under Automatic 

Reactor Scrams while 

Critical. However, the 

functional and risk 

impacts on the plant are 

not covered. 

Loss of 

Main 

Feedwater 

LOMF

W 

 X Scrams with Loss of Normal 

Heat Removal 

Same comment 

Loss of 

Offsite 

Power 

LOOP X Counted under the 

Unplanned Scrams indicator. 

However, the functional and 

risk impacts on the plant are 

not covered. Also, this event 

is too rare to monitor 

separately on a plant-specific 

basis. 

Same comment 

Loss of 

Vital AC 

Bus 

LOAC X Same comment Same comment 

Loss of 

Vital DC 

Bus 

LODC X Same comment Same comment 

Stuck 

Open SRV 

SORV X X Same comment Same comment 

Loss of 

Instrument 

Air 

LOIA X X Same comment Same comment 

Very 

Small 

LOCA 

VSLOC

A 

 X Same comment Same comment 

Steam 

Generator 

Tube 

Rupture 

SGTR X N/A Same comment Same comment 
12

4.2 Performance Monitoring of Risk-Significant Initiating Events 

(Tier 1) 

The proposed Tier 1 activity involves monitoring yearly plant-group or industry (depending upon 

the initiating event) performance against prediction limits. To accomplish this, up-to-date baseline 

frequencies were obtained for each of the risk-significant initiating events from NUREG/CR-6928 

(Ref. 17). Those baseline frequency distributions are presented in Table 4-3. Methods for establishing the 

baseline periods and resulting distributions are described in Section 8.2 and Appendix B. 

The baseline frequency distributions cover industry performance up through CY 2002 (except for 

loss of offsite power, which extends through CY 2004). At the high end of the frequencies are TRANs for 

BWRs and PWRs. These two initiating event categories have mean frequencies of 8.3E–01 per reactor 

critical year (rcry) and 7.5E–01/rcry, respectively. Therefore, many TRAN events are expected each 

calendar year at the plant-group level (BWRs or PWRs). (Typical commercial nuclear power plant 

operation involves approximately 0.9 rcry per CY.) At the low end of the frequencies is loss of a vital dc 

bus (LODC), with a frequency of 1.17E–03/rcry. LODC events are rare and are not expected across the 

entire industry in a given calendar year. 

Given these baseline frequencies and estimated yearly industry, BWR, or PWR rcry of operation, 

performance-based prediction limits were established (Table 4-4). These prediction limits are explained 

further in Section 8.3 and Appendix B. The prediction limits in Table 4-4 are performance based, and 

include both aleatory uncertainty (the randomness of the event count in the future year) and epistemic 

uncertainty (lack of perfect knowledge of the value of the baseline frequency). They represent upper 

limits of initiating event counts in a year that, if reached or exceeded, indicate a potential degradation in 

plant-group or industry performance. 

Ten risk-significant initiating events are covered for PWRs, while nine are covered for BWRs. 

Data for these initiating events—numbers of events and corresponding rcry—are already being collected 

and analyzed by the NRC on a continual basis, so no additional data collection is needed to support the 

Tier 1 activities. 

4.3 Risk-Informed Monitoring of Initiating Events Cornerstone of 

Safety (Tier 2) 

The BRIIE uses an integrated performance indicator for the Tier 2 coverage of the Initiating Events 

Cornerstone of Safety. This integrated indicator involves evaluating the risk significance of changes in 

industry initiating event performance (the results of the Tier 1 activity). Risk significance is evaluated in 

terms of a measure related to CDF, or changes in this measure (∆CDF). The indicator combines operating 

experience for risk-significant initiating events with associated internal event CDF-based importance 

information. 

BRIIE is able to appropriately combine frequent and infrequent initiating events with different risk 

measures (Birnbaum importances). Table 4-5 lists the average initiating event Birnbaum importance 

measures used for the BRIIE. These importance measures were obtained from the standardized plant 

analysis risk (SPAR) models representing the U.S. commercial nuclear power plants. BRIIE solves two 

deficiencies in the present ITP: no systematic and defined method for determining whether individual 

initiating event performance changes or adverse trends are risk significant, and no systematic and defined 

method for integrating individual initiating event performance changes into an overall risk result at the 

cornerstone of safety level. 
13

The Birnbaum risk importance measure was selected to characterize ∆CDF due to the occurrence 

or not of a particular initiating event. To determine the Birnbaum importance, the plant CDF is calculated 

using 1.0/rcry and 0.0/rcry for the initiating event of interest and the difference is determined. Therefore, 

the Birnbaum is an approximation of the derivative of CDF with respect to the initiating event of interest. 

More information on the Birnbaums is presented in Section 8.4 and Appendix C. 

Table 4-3. BRIIE initiating event baseline data and frequency distributions. 

Baseline Data 

(note a) 

Gamma Frequency Distribution 

(note a) 

Initiating Event Identifier Baseline 

Period 

Events Exposure 

(rcry) 

Mean 

(1/rcry) 

α β 

(rcry) 

General Transient 

—PWR 

TRAN 

(PWR) 

1998–2002 228 304.0 7.51E–01 17.77 23.66 

General Transient 

—BWR 

TRAN 

(BWR) 

1997–2002 149 180.2 8.30E–01 149.5 180.2 

Loss of 

Condenser Heat 

Sink—PWR 

LOCHS 

(PWR) 

1995–2002 38 475.0 8.11E–02 38.50 475.0 

Loss of 

Condenser Heat 

Sink—BWR 

LOCHS 

(BWR) 

1996–2002 41 208.6 1.97E–01 11.08 56.38 

Loss of Main 

Feedwater 

LOMFW 1993–2002 84 881.9 9.59E–02 1.326 13.83 

Loss of Offsite 

Power 

LOOP 1997–2004 24 724.3 3.59E–02 1.580 44.02 

Loss of Vital AC 

Bus 

LOAC 1992–2002 8 965.8 8.80E–03 8.500 965.8 

Loss of Vital DC 

Bus 

LODC 1988–2002 1 1282.4 1.17E–03 0.500 427.5 

Stuck Open SRV 

—PWR 

SORV 

(PWR) 

1988–2002 2 866.6 2.88E–03 0.500 173.3 

Stuck Open SRV 

—BWR 

SORV 

(BWR) 

1993–2002 6 291.7 2.23E–02 6.500 291.7 

Loss of 

Instrument Air 

—PWR 

LOIA 

(PWR) 

1997–2002 3 356.9 9.81E–03 0.500 50.99 

Loss of 

Instrument Air 

—BWR 

LOIA 

(BWR) 

1991–2002 3 343.3 1.02E–02 3.500 343.3 

Very Small 

LOCA 

VSLOCA 1992–2002 1 965.8 1.55E–03 0.500 321.9 

Steam Generator 

Tube Rupture 

—PWR 

SGTR 

(PWR) 

1991–2002 2 706.4 3.54E–03 0.500 141.3 

Note a – Rcry is reactor critical year. 
14

Table 4-4. BRIIE Tier 1 performance-based prediction limits. 

Initiating Event Identifier Mean 

(1/rcry) 

Estimated rcry 

Per Year for 

Plant Group 

(note a) 

Expected 

Number of 

Events Per Year 

(note b) 

95% 

Prediction 

Limit 

General Transient—PWR TRAN (PWR) 7.51E–01 62.1 46.6 59 

General Transient—BWR TRAN (BWR) 8.30E–01 30.6 25.4 35 

Loss of Condenser Heat 

Sink—PWR 

LOCHS (PWR) 8.11E–02 62.1 5.0 10 

Loss of Condenser Heat 

Sink—BWR 

LOCHS (BWR) 1.97E–01 30.6 6.0 11 

Loss of Main Feedwater LOMFW 9.59E–02 92.7 8.9 15 

Loss of Offsite Power LOOP 3.59E–02 92.7 3.3 8 

Loss of Vital AC Bus LOAC 8.80E–03 92.7 0.8 3 

Loss of Vital DC Bus LODC 1.17E–03 92.7 0.1 2 

Stuck Open SRV—PWR SORV (PWR) 2.88E–03 62.1 0.2 2 

Stuck Open SRV—BWR SORV (BWR) 2.23E–02 30.6 0.7 3 

Loss of Instrument Air—

PWR 

LOIA (PWR) 9.81E–03 62.1 0.6 3 

Loss of Instrument Air—

BWR 

LOIA (BWR) 1.02E–02 30.6 0.3 2 

Very Small LOCA VSLOCA 1.55E–03 92.7 0.1 2 

Steam Generator Tube 

Rupture—PWR 

SGTR (PWR) 3.54E–03 62.1 0.2 2 

Note a – There are 34 BWRs and 69 PWRs (total of 103) in the U.S. commercial nuclear power plant industry. The 

rcry estimates represent 90% critical operation during a calendar year. Rcry is reactor critical year. 

Note b – The expected number of events is the mean frequency multiplied by the plant group rcry. 

Table 4-5. Plant-group average Birnbaum values for initiating events. 

Initiating Event Birnbaum 

Description Identifier BWR Average PWR Average 

General Transient TRAN 5.51E–07 7.97E–07 

Loss of Condenser Heat Sink LOCHS 4.31E–06 2.32E–05 

Loss of Main Feedwater LOMFW 2.32E–06 2.01E–06 

Loss of Offsite Power LOOP 6.15E–05 1.12E–04 

Loss of Vital AC Bus LOAC 6.85E–05 4.22E–05 

Loss of Vital DC Bus LODC 2.61E–04 9.39E–04 

Stuck Open SRV SORV 4.59E–06 7.14E–04 

Loss of Instrument Air LOIA 3.40E–05 6.57E–05 

Very Small LOCA VSLOCA (note a) 5.51E–07 7.97E–07 

Steam Generator Tube Rupture SGTR -- 3.89E–04 

Note a – The Birnbaum importance measures for VSLOCA are identical to that for TRAN. 

Two formats for presenting quantification results were considered for the BRIIE. One presentation 

format represents an absolute measure of CDF for the selected initiating events: 
15

∑=

=

m

i

BRIIE Bi ic

1

* λ , (4-1) 

where 

m = number of initiating events covered in the BRIIE 

Bi = plant-group average Birnbaum for initiating event i

* λic = plant-group average current frequency for initiating event i. 

Another formulation, related to changes in CDF (∆CDF), is given by the following equation: 

∑ ( ) =

= −

m

i

BRIIE Bi ic ib

1

* λ λ , (4-2) 

where 

λib = baseline frequency for initiating event i. 

Note that both equations predict plant-average results, rather than plant-group total results. Plant-average 

results were judged to be more comprehensible because most NRC programs deal with plant results rather 

than industry total results. 

In both equations, the plant-group average current frequency is calculated using a Bayesian update 

process, as indicated below: 

prior i i

prior i i

ic t

x

+

+ =

,

* ,

β

α

λ , (4-3) 

where 

α prior,i = 0.5 

β prior,i = 0.5/mean (using the mean from Table 4-3) 

xi = number of plant-group (BWR or PWR) events during fiscal year for initiating event i

ti = number of plant-group rcry during fiscal year. 

This approach to calculating the plant-group current frequency uses a constrained noninformative 

distribution (CNID) (Ref. 18) with mean values from Table 4-3. However, rather than using α’s from 

Table 4-3 (mostly larger than 0.5), the CNID uses α = 0.5 in all cases. The β parameter is then determined 

by dividing α by the mean from Table 4-3. This approach for calculating current frequencies was 

recommended in NUREG-1753 and is also the approach used in the MSPI (Ref. 19) for the ROP. The 

smaller α is, the weaker the prior distribution, allowing the plant-group data to more heavily influence the 

posterior distribution. 

BWRs and PWRs have different CDFs, which depend to some extent on different initiating events 

and plant response to different initiating events. The risk weights for various initiating events are also 

different for the two types of reactors. Therefore, BRIIE results are calculated for each reactor type (plant 

group) and then the industry result is just a weighted average of the BWR and PWR results: 
16

(34 69)

( *34 * 69)

+

+ = BWRs PWRs

industry

BRIIE BRIIE

BRIIE (4-4) 

The BRIIE formulations in Equations (4-1) and (4-2) use PWR- or BWR-average Birnbaums (see 

Table 4-5) and combine the plant-group data to generate the “plant-group average current frequency” for 

each initiating event. Alternative formulations are possible using plant-specific Birnbaums and plantspecific initiating event data and then averaging the individual plant results to obtain an industry-average 

result. Results using all of the various calculation methods indicated that the proposed formulations in 

Equations (4-1) and (4-2) provide an appropriate BRIIE sensitivity to group yearly performance. (See 

Section 8.5 and Appendix D for more details.) 

As formulated in Equations (4-2) and (4-3), the BRIIE is similar to the MSPI in the ROP. Both the 

BRIIE and the MSPI use Birnbaum importance measures as weighting factors. In addition, both 

determine current performance and compare that with baseline performance to evaluate changes in 

performance. However, the MSPI applies to the Mitigating Systems Cornerstone of Safety on a plantspecific basis, while the BRIIE applies to the Initiating Events Cornerstone of Safety on an industry-wide 

or plant-group (BWR or PWR) basis. In addition, the BRIIE integrates all of the initiating events into a 

single indicator at the cornerstone of safety level, while the MSPI does not integrate the results for the 

five individual mitigating system results into a single indicator. 

Tier 2 ∆CDF results will be calculated on a yearly basis, using plant-group data for the initiating 

events covered under the BRIIE. Results will be compared with a threshold value. The threshold value 

was determined using an expert panel process (described in Sections 5 and 8.6). If the threshold value is 

exceeded, responses are outlined in Section 7. 
17

5. STAKEHOLDER AND MSPI INTERACTION AND EXPERT PANEL 

The BRIIE development effort benefited from stakeholder interaction associated with three main 

activities: a request for public review of a draft version of the BRIIE, a presentation to the Advisory 

Committee on Reactor Safeguards (ACRS), and a public workshop. The request for public review is 

covered in Appendix F. Also included in that appendix are the resolutions of comments received through 

the public review process. A detailed presentation of the draft BRIIE methods and results was given to the 

ACRS on May 7, 2003. The meeting minutes (Ref. 20) describe additional comments and suggestions for 

final development of the BRIIE. Finally, a public workshop on the draft BRIIE methodology was held at 

NRC Headquarters on July 30, 2003. That interaction also resulted in stakeholder feedback concerning 

the draft BRIIE (Ref. 21). 

In addition, the BRIIE benefited from work leading up to implementation of the MSPI in 2006. 

Included was significant effort to address issues related to the quality of licensee and SPAR risk models. 

That work led to a comprehensive program to upgrade the SPAR models based on cut-set-level 

comparisons with licensee risk models. This helped to improve the Birnbaum importance measures used 

in the BRIIE and ensure that they are adequate for this application. Other MSPI issues and concerns (Ref. 

19) also helped to shape the final BRIIE. For example, the MSPI includes a performance limit for 

insensitive indicators, which is similar in concept to the prediction limits for the BRIIE Tier 1 monitoring. 

Also, the MSPI contains a “frontstop” to ensure that no single event (assuming other contributors are at 

their baseline performance) can result in an indicator changing from green to white. In the BRIIE no 

single initiating event occurrence can trip either a Tier 1 prediction limit or the Tier 2 threshold. (The 

methods used to develop the Tier 1 prediction limits and the Tier 2 threshold were developed independent 

of this concern.) Finally, the MSPI approach for calculating current component performance, using a 

Bayesian update with a CNID prior, is the approach used in the BRIIE [Equation (4-3)], and so is the 

approach to estimating ∆CDF [Equation (4-2)]. 

Finally, an expert panel was convened in July 2006 to provide suggestions concerning the BRIIE 

and to recommend thresholds for the Tier 2 BRIIE. The peer review panel process is described in more 

detail in Appendix E. Recommendations from the expert panel included the following: 

1. Maintain the two-tiered approach for BRIIE. 

2. Present Tier 2 BRIIE results for BWRs, PWRs, and the overall industry. 

3. Present Tier 2 BRIIE results on a ∆CDF basis using Equation (4-2) with Bayesian updating 

[Equation (4-3)]. 

4. Use a Tier 2 BRIIE threshold of 1.0E–05/rcry (∆CDF basis), applied only to the overall industry 

result. 

5. Also present BRIIE results on a CDF basis using maximum likelihood estimates (MLEs, events 

divided by rcry) for yearly initiating event frequencies to clearly show the dominant yearly 

contributors to CDF. 
18
19

6. BRIIE HISTORICAL PERFORMANCE 

BRIIE historical performance includes both Tier 1 and Tier 2 results. Both are evaluated on an 

NRC FY basis (October 1 through September 30), similar to other ITP performance indicators. The Tier 1 

results are the yearly initiating event frequencies evaluated at the plant-group (BWR and PWR) or 

industry level. These results are compared with the prediction limits to identify instances where plantgroup or industry performance might have degraded. If a prediction limit is exceeded for an initiating 

event, then an engineering analysis is recommended to determine the causes. The Tier 1 results are 

presented in Appendix A. Because the baseline periods end in CY 2002, only the results for FY 2003 

through FY 2005 are compared with the prediction limits. (Results before the baseline period often are 

above the prediction limits because of improved plant-group performance reflected in the baselines.) 

Two of the initiating events exceeded their Tier 1 prediction limits: TRAN (PWR) exceeded the 

prediction limit in FY 2003, and so did loss of offsite power (LOOP). Results for these initiating events 

are presented in Figure 6-1 and Figure 6-2. Results in those figures are presented as yearly frequencies 

(events divided by rcry), rather than as events per year. Therefore, the prediction limits were also 

converted to frequencies, using the prediction limit counts and assumed rcry listed in Table 4-4. An 

engineering analysis of TRAN (PWR) performance in FY 2003 was not performed. However, as part of 

the existing ITP, the ex-AEOD indicator “Automatic Reactor Scrams while Critical” also exceeded its 

prediction limit in FY 2003. That indicator covers both BWRs and PWRs and includes events such as 

LOOP. The discussion in SECY-04-0052, Attachment 2 illustrates the type of engineering analysis that 

might be performed when Tier 1 prediction limits are exceeded. 

LOOP exceeded its prediction limit in FY 2003 because of the widespread grid disturbance on 

August 14, 2003. That single event resulted in eight LOOPs at plants that were in critical operation at the 

time. Therefore, that single grid event resulted in the prediction limit of eight being reached. Four 

additional LOOPS in FY 2003 also occurred, resulting in a total of 12 LOOP events. 
20

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

Fiscal Year

0.0

0.8

1.6

2.4

3.2

Events per reactor critical year

Rate for PWR general transients

Baseline industry average (EB)

95% prediction limit

Baseline period: CY1998-2002

Baseline log model p-value = 0.26 TRANP-07-Dec-2006

Figure 6-1. Tier 1 historical performance for TRAN (PWR). 

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

Fiscal Year

0.00

0.04

0.08

0.12

0.16

0.20

Events per reactor critical year

Rate for loss of offsite power

Baseline industry average (Jeffreys)

95% prediction limit

Baseline period: CY1997-2004

Baseline log model p-value = 0.0031 LOOP-07-Dec-2006

Figure 6-2. Tier 1 historical performance for LOOP. 

Historical performance of the BRIIE Tier 2 [∆CDF basis, Equation (4-2)] is shown in Figure 6-3 

for FY 1988 through FY 2005. Using the ∆CDF format, the BWR, PWR, and industry baselines are 0.0. 

Therefore, bars extending above the zero horizontal line represent worse than baseline performance in 

terms of CDF, and bars extending below the zero horizontal line represent better than baseline 
21

performance. Looking at the FY 2003 through FY 2005 results, none of those years resulted in an 

industry ∆CDF greater than the threshold of 1.0E–05/rcry. (The BWR result for FY 2003 lies slightly 

above this threshold, but because the PWR result is lower, the industry result is significantly below the 

threshold.) Note that the industry result is a weighted average of the PWR and BWR results, using the 

numbers of plants as weights as indicated by Equation (4-4). 

Figure 6-4 for BWRs and Figure 6-5 for PWRs show the breakdowns of CDF by initiating events. 

These figures were recommended by the expert panel for illustration purposes, to qualitatively show the 

individual initiating event contributions to CDF. In these figures, the current frequency for each initiating 

event is just the MLE (events divided by rcry). Using such an approach, if no event occurs during the 

fiscal year in question, then the contribution to CDF for that initiating event is zero. (If a Bayesian update 

approach were to be used to generate this figure, even with no events there would still be a non-zero 

estimate for the current frequency, so there would be a non-zero contribution to CDF.) LOOP events drive 

the BWR and PWR CDFs in FY 2003 and FY 2004. Table 6-1 and Table 6-2 show the initiating event 

counts for each plant group by fiscal year (limited to the 103 currently-operating plants). 

-1.0E-05

-5.0E-06

0.0E+00

5.0E-06

1.0E-05

1.5E-05

2.0E-05

1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Fiscal Year

BRIIE Tier 2 Delta CDF (1/rcry) 

BWR PWR Industry

Figure 6-3. BRIIE Tier 2 (∆CDF) historical performance. 
22

0.0E+00

5.0E-06

1.0E-05

1.5E-05

2.0E-05

2.5E-05

198819891990199119921993199419951996199719981999200020012002200320042005

Fiscal Year

CDF (1/rcry)

LOAC LOCHS LODC LOIA LOMFW LOOP SGTR SORV TRANS VSLOCA

Figure 6-4. BWR CDF breakdown by FY. 

Table 6-1. BWR initiating event counts by FY. 

Initiating 

Event 

(note a) 

1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 

LOAC 2 1 1 2 1 

LOCHS 15 15 12 10 9 14 8 10 5 4 7 7 8 2 6 9 8 7 

LODC 

LOIA 2 2 3 1 2 1 

LOMFW 10 4 3 8 4 3 1 3 3 2 6 3 3 1 2 2 4 

LOOP 3 1 1 3 1 1 8 2 

SGTR 

SORV 1 1 2 3 2 1 2 1 1 

TRANS 70 56 56 61 48 48 44 51 38 24 26 28 30 23 24 28 22 24 

VSLOCA 1 1 

Note a – These event counts include only the 34 BWRs that are currently operating. 
23

0.0E+00

5.0E-06

1.0E-05

1.5E-05

2.0E-05

2.5E-05

3.0E-05

3.5E-05

4.0E-05

198819891990199119921993199419951996199719981999200020012002200320042005

Fiscal Year

CDF (1/rcry)

LOAC LOCHS LODC LOIA LOMFW LOOP SGTR SORV TRANS VSLOCA

Figure 6-5. PWR CDF breakdown by FY. 

Table 6-2. PWR initiating event counts by FY. 

Initiating 

Event 

(note a) 

1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 

LOAC 1 2 

LOCHS 6 12 9 6 6 5 9 4 7 3 3 3 7 8 3 5 7 4 

LODC 1 

LOIA 2 3 1 1 1 1 1 1 1 1 

LOMFW 11 14 12 12 8 3 8 7 7 9 7 5 5 4 1 7 5 4 

LOOP 3 5 3 2 3 2 3 1 1 1 1 1 4 3 

SGTR 1 1 1 

SORV 1 1 

TRANS 164 134 122 105 116 92 78 80 74 54 41 57 42 46 32 68 45 33 

VSLOCA 1 1 1 

Note a – These event counts include only the 69 PWRs that are currently operating. 
24
25

7. RESPONSES TO TIER 1 AND TIER 2 RESULTS 

Table 7-1 shows the relationships between the ROP, the ITP, and the report to Congress. The table 

shows the potential actions that the ITP might take if a BRIIE Tier 1 prediction limit or Tier 2 threshold is 

exceeded. Two examples illustrate how the ITP might treat initiating event performance changes. 

Table 7-1. Relationship between the ROP, ITP and report to Congress. 

Reactor Oversight Process Industry Trends 

Program Report to Congress 

Agency Responses to Individual 

Occurrences of Initiating Events 

Annual Occurrences 

of 

Initiating Events 

Integrated Indicator 

Initiating Event Class 

Early Notification (10CFR 50.72) 

Incident Investigation (MD 8.3) 

Routine or Special Inspections 

Significance Determination Process 

Licensee Event Report (10CFR 50.73) 

Accident Sequence Precursor Analysis 

Action To Be Taken 

If Tier 1 Industry 

Prediction Limit Is 

Exceeded for 

Indicated Initiating 

Event Category: 

Action To Be Taken 

If Tier 2 ∆CDF 

Threshold Is 

Exceeded: 

 General Transient X X 

 Loss of Condenser Heat Sink X X 

 Loss of Main Feedwater X X 

 Loss of Offsite Power X X X X X X 

 Loss of Vital AC Bus X X X X X X 

 Loss of Vital DC Bus X X X X X X 

 Stuck Open SRV X X X X X X 

 Loss of Instrument Air X X X X X X 

 Very Small LOCA X X X X X X 

 Steam Generator Tube 

Rupture X X X X X X 

Investigate to 

determine reasons for 

exceedance. 

Document actions 

taken in ROP. 

Document reasons for 

the exceedance and 

what has been done to 

address them. 

For the first example, suppose four very small loss-of-coolant accident (VSLOCA) events occur in 

FY 2007 at separate plants. This initiating event is historically rare. The 95% industry prediction limit is 

two events (Table 4-4). Because the number of actual events exceeds the prediction limit, this initiating 

event is a candidate for further investigation. 

Because VSLOCAs do not occur very often, NRC would look at each event in more detail after it 

had occurred. Thus, NRC would have inspectors and staff reviewing each event. The ITP would look at 
26

these events to see if there were similarities among the events and to provide any lessons learned from 

this evaluation. These lessons would be communicated to the industry via some type of generic 

communication. Further regulatory action would probably not be necessary since the NRC investigated 

each event in detail. 

If all of these events had occurred at PWRs, the incremental increase to the FY 2007 PWR Tier 2 

∆CDF would be approximately 8.0E–09/ rcry (per PWR) for this hypothetical case. The incremental 

decrease for BWRs (assuming no additional VSLOCAs) would be essentially zero. Therefore, the 

incremental increase for the industry from VSLOCAs would be less than 8.0E–09/rcry. This is 

significantly below the recommended industry threshold of 1.0E–05/rcry. The BRIIE report to Congress 

would identify VSLOCA as a departure from expected baseline performance, but would indicate that this 

departure is not risk significant. 

As another example, suppose five LODC events occur in FY 2007, with two involving BWRs and 

three involving PWRs. This example exceeds the Tier 1 prediction limit for LODC of two events 

(Table 4-4). The incremental increase to the BWR Tier 2 ∆CDF would be approximately 1.7E–06/ rcry 

(per BWR) for this hypothetical case. For PWRs, then the incremental increase is approximately 

3.7E-06/rcry. The industry average incremental increase is 

[(1.7E–06/rcry)(34) + (3.7E–06/rcry)(69)]/(34 + 69) = 3.0E–06/rcry. 

This is below the Tier 2 threshold at the industry level of 1.0E–05/rcry recommended by the expert panel. 

Therefore, the report to Congress would indicate that the industry LODC performance is a departure from 

baseline performance but the departure is not risk significant. 
27

8. BASES FOR THE BRIIE 

This section summarizes the bases for a variety of decisions made during the BRIIE development 

process. Additional information is provided in the appendices. 

8.1 Selection of Initiating Events 

NUREG/CR-5750 identified initiating events that met the following criteria: 

• Included an unplanned reactor trip 

• Sequence of events started when the reactor is critical and at or above the point of adding heat 

• Occurred during CY 1987 through CY 1995 

• Was reported by a licensee event report (LER). 

In addition, certain rare events such as loss-of-coolant accidents (LOCAs) were added to the list. The 

resulting list of initiating events from that study includes 48 individual initiating events grouped under 13 

categories. 

The NUREG/CR-5750 list was reviewed against Individual Plant Examination (IPE) study CDF 

results (Ref. 22) to identify the subset of risk-significant initiating events presented in NUREG-1753. For 

an initiating event to be risk significant at the industry or plant-group (BWR or PWR) level, it needed to 

have a conditional core damage probability (CCDP) ≥ 1.0E–06 and a contribution to industry-wide or 

plant-group CDF ≥ 1%. That screening process resulted in the list of initiating events presented in 

Table 4-2. 

NUREG-1753 was concerned with monitoring plant-specific performance rather that industry-wide 

or plant-group performance, so it chose only the most frequent risk-significant events (TRAN, LOCHS, 

and LOMFW) as monitorable at the plant level over a 3-year period. Monitorable in this context indicates 

that there is a reasonable probability (approximately ≥ 0.1) of observing the event over the monitoring 

period. For the BRIIE, only 1 year of data is used. However, the BRIIE does not monitor at the plantspecific level; it monitors at the industry or plant-group level. Based on initiating event data for U.S. 

commercial nuclear power plants over CY 1988 through CY 2002, all of the risk-significant events 

identified in NUREG-1753 (except for those outside the scope of this effort, such as internal flooding and 

initiators evaluated under the ASP Program) were chosen for inclusion in the BRIIE. 

A review of SPAR model CDF results for the 103 operating U.S. commercial nuclear power plants 

indicates that the BRIIE initiating events cover approximately 60% of the total internal event CDF from 

these models. Other initiating events within these models (covering the remaining 40% of CDF) that are 

not included within the BRIIE include such events as loss of service water, loss of component cooling 

water, medium and large LOCAs, and interfacing system LOCAs. These events are rare and generally 

would not be expected over the lifetimes of the plants. Therefore, such events are not monitorable on a 

yearly basis. 

8.2 Determination of Initiating Event Baselines 

Initiating event data from LERs are available in the initiating event database (IEDB) (Ref. 23), 

covering CY 1987 through the present. NUREG/CR-6928 (Ref. 17) established baseline periods and 

associated frequency distributions for all of the initiating events within the IEDB, using methods first 

developed within the ITP. Those same baseline periods and associated frequency distributions are used 
28

for the BRIIE and are presented in Table 4-3. 

In NUREG/CR-6928, baseline periods to characterize current industry performance were chosen 

using the following process. The Reliability and Availability Database System (RADS, Ref. 24) software 

was used to search the IEDB for specific initiating event information and process that information. RADS 

processes initiating event data to determine total number of events and total rcry over the period specified. 

In addition, RADS presents yearly results for the period chosen. 

Initial RADS searches were performed using data from CY 1988 through CY 2002. These data 

were then examined to determine an optimized baseline period (ending in CY 2002). Optimization in this 

case indicates that yearly data were examined, starting with CY 2002 and working backward in time, to 

identify a baseline period with performance representative of CY 2000. In addition, a minimum of 5 years 

was specified for potential baseline periods. Often the initiating event data indicate more events in the 

early years and fewer events in the latter years, so the early years with poorer performance were not 

included in the baseline period used to quantify the initiating event frequencies. Statistical trend 

evaluations were performed for all potential baseline periods. The starting year that resulted in the 

weakest evidence for existence of a trend was then chosen. Additionally, if there were no events or only 

one event during CY 1988 through CY 2002, then the entire period was chosen as the baseline. Finally, if 

there were only two events and they occurred during the first 3 years (probability of this is less than 0.05 

assuming a constant occurrence rate), then the baseline period started with the first year with no events. 

This optimization of the period used to characterize current performance resulted in baseline periods with 

start years of CY 1988 to CY 1998, but all ending in CY 2002. Appendix A presents the initiating event 

data by year and the resulting baseline periods obtained using this process. 

Once the baseline period was determined, RADS again was used to collect the initiating event data 

over that period. These data (total events and total reactor critical years by plant) were then analyzed 

statistically to determine gamma frequency distributions. An empirical Bayes analysis with a KassSteffey adjustment (Ref. 25) was performed to assess plant-to-plant variation and obtain a frequency 

distribution. If the empirical Bayes analysis failed to converge (indicating little variation between plants), 

then the frequency distribution was obtained using a Bayesian update of the Jeffreys noninformative prior 

with the baseline period data. Finally, if there were three or fewer events over the baseline period and the 

empirical Bayes analysis failed, then the CNID was assumed, with a mean obtained from the Bayesian 

update of the Jeffreys noninformative prior. Use of the CNID in these cases reflected the belief that 

significant plant-to-plant variation probably exists for these less frequent initiating events, but there were 

insufficient data to determine that actual variability. The statistical analysis process is explained in more 

detail in Appendix B and NUREG/CR-6928. 

In NUREG/CR-6928, the IEDB events for six initiating events were reviewed to screen out events 

that were not applicable with respect to the SPAR event tree modeling of such events. This screening 

effort was needed only for those six initiating events; other modeling in SPAR agreed with the initiating 

event definitions used in the IEDB. 

8.3 Tier 1 Prediction Limits 

Tier 1 prediction limits for the BRIIE initiating events are presented in Table 4-4. The prediction 

limits are performance based, and include both aleatory uncertainty (the randomness of the event count in 

the future year) and epistemic uncertainty (lack of perfect knowledge of the value of the baseline 

frequency). They represent upper limits of initiating event counts in a year that, if reached or exceeded, 

indicate a potential degradation in plant-group or industry performance. For example, for TRAN (PWR), 

the expected number of events in a future year is 46.6 (assuming the 69 PWRs are in critical operation an 
29

average of 90% of the calendar time). The predictive distribution for such events indicates that the 95% 

prediction limit is 60, indicating there is a 0.05 probability that 60 or more events will be observed in a 

year if the baseline frequency distribution applies. Therefore, if 60 or more TRAN (PWR) events are 

observed in a year, then there is significant evidence that the plant-group performance has degraded. 

A discussion of predictive distributions and prediction limits is presented in Appendix B. These 

distributions can be generated assuming there is no significant plant-to-plant variation in the baseline 

frequency distribution, resulting in closed form equations (a gamma-Poisson distribution). They can also 

be generated assuming there is plant-to-plant variation. However, in that case simulation must be used 

unless all plants have the same time (rcry). Both approaches were investigated. Because of the belief that 

there is always plant-to-plant variation in initiating event frequencies (even when the baseline data set 

may not appear to indicate such variation), the second approach (assuming plant-to-plant variation) was 

used. 

8.4 Tier 2 Birnbaum Importances 

The BRIIE Tier 2 monitoring of initiating event performance requires an initiating event 

importance measure to convert changes in performance (relative to baseline performance) to a ∆CDF risk 

index. Similar to the MSPI Program, the BRIIE Tier 2 uses the Birnbaum importance measure. This 

importance measure is the derivative of CDF with respect to the initiating event of interest. Therefore, the 

Birnbaum is a measure of the sensitivity of CDF to changes in the initiating event frequency. 

Birnbaum importance measures for BRIIE were obtained from the SPAR models covering the 103 

operating commercial nuclear reactors in the U.S. The SAPHIRE code (Ref. 26) calculates Birnbaum 

importance measures for both initiating events and basic events in the SPAR models. To calculate the 

Birnbaum for an event, SAPHIRE evaluates the CDF with the event probability (or frequency) set to 1.0 

and 0.0 (or 1.0/rcry and 0.0/rcry). The difference in CDF is the Birnbaum importance measure. Note that 

this importance measure is unitless for initiating events. 

The BRIIE Tier 2 requires plant-group (BWR or PWR) average Birnbaums. Therefore, individual 

plant Birnbaums were quantified using the SPAR models. Then the BWR-average Birnbaum was 

determined by summing the results across the 34 BWRs and dividing by 34. A similar process was used 

for the 69 PWRs. For plants with no Birnbaum (the SPAR model did not include the initiating event in 

question), the Birnbaum was assumed to be zero. This is appropriate because the SPAR model would 

have included the initiating event in question if it contributed significantly to the overall CDF. The SPAR 

models used for the calculations were the internal event models released in December 2005. The SPAR 

models are continually being improved and modified, so a re-evaluation of Birnbaums with a different set 

of models might result in some different plant-specific Birnbaums. However, the plant-group averages are 

expected to be less sensitive to typical SPAR model changes. 

Plant-group average Birnbaums obtained from the SPAR models are presented in Table 4-5. 

Additional details concerning the generation of Birnbaum importance measures are provided in 

Appendix C. 

8.5 BRIIE Tier 2 Formulations 

Many different formulations for the BRIIE Tier 2 were considered before deciding to use 

Equations (4-2) and (4-3) as the final quantification approach. Four different approaches addressed 

whether to use plant-group (BWR or PWR) or plant-specific Birnbaums and initiating event current 

frequencies. These four different approaches are identified in Table 8-1. Equations for each of the four 
30

approaches are presented following the table. 

Table 8-1. Plant-specific versus group approaches to quantifying the BRIIE Tier 2. 

Initiating Event Current Birnbaum Importance Measure 

Frequency Plant Specific Plant Group 

Plant Specific Equation (8-1) Equation (8-2) 

Plant Group Equation (8-3) Equation (8-4) 

If the BRIIE Tier 2 is calculated by evaluating results on a plant-specific basis (using plant-specific 

current initiating event frequencies and plant-specific Birnbaum importance measures) and then averaging 

the results, the following equation applies: 

∑ ∑= =

= −

m

i

N

u

Tier Bui uic ib N

BRIIE

1 1

2 ( ( )) 1 λ λ . (8-1) 

If plant-specific current frequencies are combined with group-average Birnbaum importance measures, 

the result is the following: 

∑ ∑ ∑ = = =

= − = −

N

u

m

i

uic ib i ic ib

m

u

Tier Bi B

N

BRIIE

1 1 1

2 ( ) ( ) 1 λ λ λ λ . (8-2) 

If the initiating event data (events and rcry) are aggregated at the plant-group level to determine a group 

initiating event current frequency and plant-specific Birnbaums are used, the following applies: 

∑∑ ∑ == =

= − = −

m

i

N

u

m

i

Tier Bui ic ib Bi ic ib N

BRIIE

11 1

* *

2 ( ) ( ) 1 λ λ λ λ . (8-3) 

Finally, using initiating event data aggregated at the plant-group level and group-average Birnbaums, the 

expression is the following: 

∑=

= −

m

i

BRIIETier Bi ic ib

1

*

2 (λ λ ) . (8-4) 

Terms in the above equations are the following: 

m = number of initiating events covered under the BRIIE 

N = number of plants within the plant group 

λuic = plant-specific current frequency for initiating event i and plant u

* λic = common group-level current initiating event frequency for initiating 

event i

λib = baseline frequency for initiating event i

∑=

=

N

u

ic uic N 1

1 λ λ = arithmetic mean of plant-specific current frequencies 

Bui = plant-specific Birnbaum for initiating event i and plant u
31

∑=

=

N

u

i Bui N

B

1

1

= arithmetic mean of plant-specific Birnbaums for initiating event i. 

Equation (8-4) is the approach chosen for the BRIIE Tier 2 monitoring and is the same as Equation 

(4-2). Note that Equation (8-3) also reduces to Equation (8-4). Of these three approaches (given that two 

of the four result in the same final equation), the most meaningful ones are the purely plant-group-level 

approach modeled by Equation (8-4) and the purely plant-specific approach modeled by Equation (8-1). 

In addition to the plant-specific versus group-level approach issue, there is also an issue related to 

how initiating event current frequencies should be calculated. The preferred approach, Equation (4-3), 

uses a Bayesian update with a CNID prior and the plant-group-level data. An alternative approach is to 

use an MLE (events divided by rcry). These two alternatives can be applied at either the plant-specific 

level or the plant-group level. Therefore, there are four main alternatives as shown in Table 8-2. BRIIE 

Tier 2 results by year were quantified using the four alternatives in Table 8-2. Results indicated that the 

plant-group-level quantification approach combined with using MLEs for the current initiating event 

frequencies was the most sensitive in terms of showing the widest variations in Tier 2 ∆CDF predictions. 

The approach selected, using plant-group-average Birnbaums and a plant-group current frequency 

determined by Bayesian update, was next highest in terms of sensitivity. Quantifying ∆CDF at the plantspecific level (and then averaging the results) and using MLEs for initiating event current frequencies was 

approximately as sensitive as the approach selected, while the plant-specific quantification using 

Bayesian updates to determine current frequencies exhibited very little sensitivity. 

Table 8-2. Evaluation level and frequency calculation approaches to quantifying the BRIIE Tier 2. 

Evaluation Level Initiating Event Current Frequency Calculation 

Bayes MLE 

Plant Specific Plant Specific and Bayes Plant Specific and MLE 

Plant Group Plant Group and Bayes (approach 

selected) 

Plant Group and MLE 

Based on the results of various quantification methods discussed above, the quantification at the 

plant-group level using Bayesian updating to determine initiating event frequencies was chosen as the 

preferred approach to quantifying the BRIIE Tier 2 ∆CDF. Details concerning these analyses are 

presented in Appendix D. Also, the approach selected is consistent with the MSPI Program in terms of 

using a Bayesian update approach with the CNID to determine current performance. 

8.6 Tier 2 Threshold 

The BRIIE Tier 1 prediction limits indicate whether industry or plant-group performance of 

initiating events has degraded relative to the established baselines for performance. These prediction 

limits are performance based and do not provide an indication of whether such performance degradation 

is risk significant. That is the reason the BRIIE Tier 2 ∆CDF estimator was developed. The BRIIE Tier 2 

∆CDF is risk based. 

A threshold for the BRIIE Tier 2 ∆CDF is needed to specify when significant risk-based 

performance degradation has occurred. If the threshold is reached or exceeded, then the result will be 

reported to Congress. The BRIIE Tier 2 ∆CDF threshold was developed using an expert panel approach. 

Information given to the expert panel to help them determine a threshold included the following: 

• BRIIE Tier 2 historical results (BWRs , PWRs, and industry) in Figure 6-3 
32

• BRIIE Tier 2 simulations to determine the 95% prediction limits (BWRs and PWRs) assuming 

current industry performance is maintained (Figure 9-3 and Figure 9-4 in Section 9). 

The expert panel decided that the BRIIE Tier 2 threshold should be 1.0E–05/rcry, applied to the 

industry-level ∆CDF result as defined in Equation (4-4). The threshold is not to be applied at the BWR 

and PWR levels. The expert panel decided on this threshold based on considerations of NRC safety goals 

and Regulatory Guide 1.174 (Ref. 27) and on the desire to be consistent with the MSPI Program and the 

ASP Program. The threshold was derived from coherency with current NRC metrics and the surrogates 

for the safety goals discussed in RG 1.174. Two scenarios were discussed during the deliberations. The 

first was that a single event at 1.0E–03/rcry (∆CDF basis), which is the current ASP indicator threshold 

for reporting significant events to Congress, converts approximately to 1.0E–05/rcry per plant on an 

industry basis (assuming approximately 100 plants). Also if 10% of plants each had a problem equal to 

1.0E–04/rcry (∆CDF basis), this converts again to approximately 1.0E–05/rcry per plant on an industry 

basis. Details concerning the expert panel process are provided in Appendix E. 

8.7 Data Period 

To determine the plant-group average frequencies, a data collection period needs to be specified. 

The ROP uses 1 year of data for its unplanned scrams indicator and 3 years of data for its unplanned 

scrams with loss of normal heat sink indicator. Also, the MSPI uses 3 years of data for components and 

trains within its five monitored safety systems. However, the ROP and MSPI are focused on plantspecific results, while the BRIIE is concerned with plant-group (BWR or PWR) and industry results. With 

34 BWRs and 69 PWRs, the time period for the BRIIE data collection does not necessarily need to be as 

long as the ROP time periods. Both 1 year and 3 years were considered for the BRIIE to estimate 

initiating event current frequencies. A 3-year period provides more data, but since the BRIIE results are 

evaluated yearly, events in a given year impact the BRIIE results for 3 successive years, until they drop 

out of the 3-year period. A 1-year data collection period was chosen for the BRIIE based on the 

following: 

• Data over a 1-year period are considered sufficient to characterize plant-group or industry 

initiating event frequencies for use in a BRIIE-type indicator. 

• A 1-year data collection period eliminates any dependencies between successive-year BRIIE 

results (events that occur impact the BRIIE results only for that specific year). 
33

9. BRIIE TIER 2 UNCERTAINTY AND SENSITIVITY 

The historical performance of the BRIIE over the period FY 1988 through FY 2005 provides a 

limited picture of the expected performance of the BRIIE. To provide additional insight, this section 

considers four topics: 

• The baseline CDF, including a breakdown to show which initiating events contribute most to the 

value and to its uncertainty, and a simulation to show the full uncertainty distribution 

• The predictive distribution for the BRIIE, as evaluated by simulation 

• The sensitivity of BRIIE to various high but plausible counts of each individual type of initiating 

event 

• The sensitivity of BRIIE to non-baseline conditions, that is, the distribution of BRIIE if the 

initiating event frequencies increase by certain assumed amounts. 

9.1 Baseline CDF 

The BRIIE Tier 2 plant-group average Birnbaums and initiating event baseline frequencies can be 

combined to obtain an estimate of baseline CDF for an average plant within the group. That CDF estimate 

is the following: 

∑=

− =

m

i

CDFBRIIE plant average Bi ib

1

, λ (9-1) 

where 

m = number of initiating events covered under the BRIIE 

Bi = plant-group average Birnbaum for initiating event i (Table 4-5) 

λib = baseline frequency for initiating event i. 

CDF estimates using Equation (9-1) can be generated for BWRs and PWRs. However, because the 

BRIIE does not include all initiating events within the SPAR models, the BRIIE baseline CDF estimates 

do not cover all of the internal event model risk. As indicated in Section 8.1, the initiating events 

contained with the BRIIE cover approximately 60% of the overall CDF in the SPAR models. The 

remaining 40% of risk comes from other initiating events such as loss of service water, loss of component 

cooling water, interfacing system LOCAs, medium and large LOCAs, and others. These other initiating 

events are too rare to be monitored within the BRIIE. 

The various initiating event baseline frequencies λib are modeled by Bayesian distributions based 

on the data from the baseline period, as explained in Section 8.2. When these baseline distributions of λi

are inserted into the equation for CDF, the result is a Bayesian distribution for the baseline CDF. 

Elsewhere in this section, when the baseline CDF is given as a number, it is the mean of this distribution. 

The contributions of the initiating events to baseline CDF are shown in Table 9-1 and Table 9-2. 

The terms α and β refer to the baseline gamma(α, β) distributions of the initiating events, with mean α/β. 

The element in the i

th row of the “Mean” column is of the form Biαi/βi. Each element of the “Variance” 

column is of the form Biαi/βi

2

. 
34

Table 9-1. Breakdown of BRIIE baseline CDF for BWRs. 

Initiating Event Mean CDF 

(rcry) 

(note a) 

Variance of CDF α β Average 

Birnbaum 

Importance 

B 

Loss of Offsite Power—LOOP 2.21E–06 43% 3.08E–12 89% 1.6 44.0 6.15E–05 

BWR Loss of Condenser Heat 

Sink—LOCHS (BWR) 

8.47E–07 17% 6.48E–14 2% 11.1 56.4 4.31E–06 

Loss of Vital AC Bus—LOAC 6.03E–07 12% 4.28E–14 1% 8.5 966 6.85E–05 

BWR General Transient—

TRAN (BWR) 

4.57E–07 9% 1.40E–15 0% 149.5 180 5.51E–07 

BWR Loss of Instrument Air—

LOIA (BWR) 

3.47E–07 7% 3.43E–14 1% 3.5 343 3.40E–05 

Loss of Vital DC Bus—LODC 3.05E–07 6% 1.86E–13 5% 0.5 428 2.61E–04 

Loss of Main Feedwater—

LOMFW 

2.22E–07 4% 3.73E–14 1% 1.3 13.8 2.32E–06 

BWR Stuck Open SRV—

SORV (BWR) 

1.02E–07 2% 1.61E–15 0% 6.5 292 4.59E–06 

Very Small LOCA—VSLOCA 8.56E–10 0% 1.46E–18 0% 0.5 322 5.51E–07 

Total 5.09E–06 100% 3.45E–12 100% 

Note a – Other initiating events not covered under the BRIIE also contribute to the overall CDF for BWRs. The total 

CDF in this table is only from the BRIIE initiating events and represents approximately 60% of the overall CDF for 

BWRs. Rcry is reactor critical year. 

Table 9-2. Breakdown of BRIIE baseline CDF for PWRs. 

Initiating Event Mean CDF 

(rcry) 

(note a) 

Variance of CDF α β Average 

Birnbaum 

Importance 

B 

Loss of Offsite Power—LOOP 4.02E–06 33% 1.02E–11 39% 1.6 44.0 1.12E–04 

PWR Stuck Open SRV—

SORV (PWR) 

2.06E–06 17% 8.49E–12 33% 0.5 173 7.14E–04 

PWR Loss of Condenser Heat 

Sink—LOCHS (PWR) 

1.88E–06 15% 9.18E–14 0% 38.5 475 2.32E–05 

Steam Generator Tube 

Rupture—SGTR 

1.38E–06 11% 3.79E–12 15% 0.5 141 3.89E–04 

Loss of Vital DC Bus—LODC 1.10E–06 9% 2.41E–12 9% 0.5 428 9.39E–04 

PWR Loss of Instrument Air—

LOIA (PWR) 

6.44E–07 5% 8.30E–13 3% 0.5 51.0 6.57E–05 

PWR General Transient—

TRAN (PWR) 

5.99E–07 5% 2.02E–14 0% 17.8 23.7 7.97E–07 

Loss of Vital AC Bus—LOAC 3.71E–07 3% 1.62E–14 0% 8.5 966 4.22E–05 

Loss of Main Feedwater—

LOMFW 

1.93E–07 2% 2.80E–14 0% 1.3 13.8 2.01E–06 

Very Small LOCA—VSLOCA 1.24E–09 0% 3.07E–18 0% 0.5 322 7.97E–07 

Total 1.22E–05 100% 2.59E–11 100% 

Note a – Other initiating events not covered under the BRIIE also contribute to the overall CDF for PWRs. The total 

CDF in this table is only from the BRIIE initiating events and represents approximately 60% of the overall CDF for 

PWRs. Rcry is reactor critical year. 

The full Bayesian distributions for BWR and PWR BRIIE CDF were obtained by simulation: A 
35

value of each λi was randomly generated from its baseline distribution and the full set of values was 

inserted into Equation (9-1) to get a value of CDF. This was done many times, and the resulting CDFs are 

shown in Figure 9-1 and Figure 9-2. The vertical line in the figures shows the mean, whose values are 

taken from Table 9-1 and Table 9-2. The 5th and 95th percentiles for BWRs are 2.9E–06 and 8.7E–06/rcry. 

The 5th and 95th percentiles for PWRs are 5.8E–06 and 2.2E–05/rcry. The BRIIE CDF for the average 

BWR is definitely smaller than for the average PWR. Not only is the BWR mean smaller, the mean for 

the average BWR is below the 5th percentile for the average PWR. 

0.0E0 2.0E-6 4.0E-6 6.0E-6 8.0E-6 1.0E-5 1.2E-5 1.4E-5

Baseline CDF, BWRs

Figure 9-1. Simulated baseline BRIIE CDF distribution for an average BWR. 
36

0.0E0 5.0E-6 1.0E-5 1.5E-5 2.0E-5 2.5E-5 3.0E-5 3.5E-5

Baseline CDF, PWRs

Figure 9-2. Simulated baseline BRIIE CDF distribution for an average PWR. 

9.2 Predictive Distribution for BRIIE 

The predictive distribution for BRIIE is the distribution that is conditional on the data that have 

already been observed—the data that determined Bi and λib in Equation (9-1). When this conditional 

distribution is used, Bi and λib are treated as known values, and the uncertainty distribution of λi is 

conditional on the known baseline data. This conditional distribution is the posterior distribution of λi

given xib and tib. The predictive distribution for BRIIE can be simulated as explained below. 

First, one must do some preliminary work for each initiator i (from 1 to m). Use the Jeffreys 

noninformative prior, a gamma(0.5, 0) distribution. If xib events were seen in tib rcry, the posterior 

distribution of λib is gamma(xib + 0.5, tib). Now perform the simulation: 

1. For each initiating event i from 1 to m, generate λi from the posterior distribution of λib. 

2. Generate Xi from its Poisson distribution conditional on λi (using the appropriate rcry exposure 

from Table 4-4). 

3. Calculate λi using Equation (4-3). 

4. Calculate the resulting value of BRIIE using Equation (4-1) to obtain a CDF estimate rather than 

a ∆CDF estimate. 

5. Repeat this many times, obtaining many values from the predictive distribution for BRIIE. 
37

Figure 9-3 shows the simulated predictive distribution for the BWR BRIIE. The calculations 

assume 1 year of data to construct a Bayes estimate of the current initiating event frequencies for BWRs, 

and use Equation (4-2) for BRIIE. This BRIIE is an estimate of the current ∆CDF. The vertical line in the 

graph identifies the mean, which is zero. The 95th percentile is 2.9E–06/rcry. Surprisingly, this percentile 

of the estimator is smaller than the corresponding percentile of the baseline ∆CDF (8.7E–06/rcry −

5.1E-06/rcry = 3.6E–06/rcry). The reason is that we have assumed between-plant variation. Therefore, for 

each initiating event the frequency is randomly generated at each plant, with large values partially 

canceling small ones. The resulting data tend to reflect the mean frequency, yielding an estimate with a 

more concentrated distribution than the distribution of ∆CDF at an average plant. If Equation (4-1) had 

been used instead of Equation (4-2) to define BRIIE, the graph would simply be shifted sideways so that 

the mean is 5.1E–06/rcry (the mean CDF for BWRs from initiating events covered under the BRIIE, as 

indicated in Table 9-1). 

Figure 9-4 shows the simulated predictive distribution for the PWR BRIIE. The mean is marked by 

a vertical line at zero. The 95th percentile is 5.3E–06/rcry, which is smaller than the 95th percentile of the 

baseline ∆CDF (2.18E–05/rcry − 1.22E–05/rcry = 9.6E–06/rcry), just as for BWRs. 

One striking feature of Figure 9-3 is that the distribution is “lumpy”; that is, the histogram has 

multiple local maxima and minima. This is explained as follows. The quantity BRIIE is an estimate of 

∆CDF. The estimated ∆CDF is discrete, because it is constructed from discrete counts. Table 9-1 shows 

that LOOP contributes nearly half the mean and about nearly 90% of the variance of the CDF for BWRs. 

The CDF can be thought of as the sum of two pieces: the piece from LOOP, and the piece from 

everything else. The two pieces have similar means, but the estimate of the piece from LOOP has a highly 

discrete distribution, because typically there are three or fewer LOOP events in one year. This 

discreteness is visible in Figure 9-3. 

This discreteness is not visible in Figure 9-4 for PWRs. Table 9-2 shows that no single initiating 

event dominates the variance as was the case in Table 9-2. Therefore, various event counts combine to 

give a relatively smooth distribution for BRIIE. 

-4E-6 -2E-6 0E0 2E-6 4E-6 6E-6 8E-6

Estimated ∆CDF

Figure 9-3. Simulated predictive distribution of BRIIE for BWRs. 
38

-8.0E-6 -4.0E-6 0.0E0 4.0E-6 8.0E-6 1.2E-5 1.6E-5

Estimated ∆CDF

Figure 9-4. Simulated predictive distribution of BRIIE for PWRs. 

An earlier draft of this report also considered the effect of the uncertainty in the Birnbaum 

importances. If the purpose of this task were to obtain as accurate an uncertainty distribution as possible 

for the baseline CDF (Figure 9-1 and Figure 9-2), then the uncertainty in the Birnbaum importances 

would have to be considered. This uncertainty is not considered here for three reasons: 

1. It is difficult to quantify the uncertainties in the Birnbaum importances. This uncertainty involves 

the uncertainties of all the cut sets in the risk model, as well as the variation between models. 

2. The primary purpose of the BRIIE is to detect risk-significant changes in the initiating event 

frequencies. Preliminary work showed that the ∆CDF version of BRIIE is not very sensitive to 

uncertainty in the Birnbaum importances, at least in the upper tail of the distribution. 

3. The portion of CDF covered by BRIIE is only about 60% of the total CDF, so there is little value 

in trying to estimate it as accurately as possible. 

Therefore, this report considers BRIIE as an index, constructed as an estimate of the ∆CDF that will be 

sensitive to changes in the initiating event frequencies. It treats the SPAR models and the resulting 

Birnbaum uncertainties as given, and it looks for any increases in the estimated BRIIE ∆CDF. 

9.3 Sensitivity of BRIIE to High Counts for Individual Initiators 

A sensitivity study was performed to evaluate the impacts on the BRIIE from individual initiating 

events. For each initiator, the 95% prediction limit (from Table 4-4) was inserted into the BRIIE, while 

keeping other initiating events at their baseline frequencies. (For those initiating events without BWR and 

PWR breakdowns, the prediction limit was divided into BWR and PWR contributions, based on the 

numbers of plants in each plant type.) The results of these sensitivity evaluations are presented in 

Figure 9-5 and Figure 9-6 for BWRs and PWRs, respectively. For BWRs, the largest contributors to the 

BRIIE are LOOP, LOAC, LODC, LOIA, and LOCHS. For PWRs, the largest contributors are SORV, 
39

LOOP, LODC, SGTR, and LOCHS. These events are the same ones that contribute most to the variance 

of BRIIE, as shown in Table 9-1 and Table 9-2. 

0.0E+00

5.0E-07

1.0E-06

1.5E-06

2.0E-06

2.5E-06

3.0E-06

TRAN LOCHS LOMFW LOOP LOAC LODC SORV LOIA VSLOCA

BWR Initiating Event

BRIIE Delta CDF (1/rcry)

Figure 9-5. BWR BRIIE ∆CDF sensitivity to individual initiating event 95% prediction limits. 
40

0.0E+00

1.0E-06

2.0E-06

3.0E-06

4.0E-06

5.0E-06

6.0E-06

TRAN LOCHS LOMFW LOOP LOAC LODC SORV LOIA VSLOCA SGTR

PWR Initiating Event

BRIIE Delta CDF (1/rcry)

Figure 9-6. PWR BRIIE ∆CDF sensitivity to individual initiating event 95% prediction limits. 

9.4 Sensitivity of BRIIE to Non-Baseline Conditions 

The distribution of BRIIE was simulated to see how successful BRIIE was at detecting off-baseline 

situations. Figure 9-7 shows the cumulative distributions of BRIIE under three assumptions. The first is 

that the initiating event frequencies have their baseline distributions. The resulting distribution (solid line 

in Figure 9-7) is the predictive distribution shown earlier. The second assumption (long dash line) is that 

all the initiating event frequencies are 1.5 times the baseline distributions. That is, the baseline α is 

retained, but β is divided by 1.5. The third assumption (short dash line) is that the initiating event 

frequencies are 2.0 times the baseline distributions. As can be seen in Figure 9-7, the distribution of 

BRIIE is moved to the right under the assumed non-baseline conditions. The vertical line in Figure 9-7 is 

the threshold, 1.0E–05/rcry. 

The graph shows at baseline conditions the industry BRIIE virtually will never exceed the 

threshold. Even if the industry has degraded to the point that every initiating event frequency has a 

distribution twice as large as the baseline distribution (i.e. α at the baseline value and β cut in half), then 

there is only a 0.10 probability that the industry data will show enough events to exceed the threshold. 

The threshold suggested by the expert panel was based on consistency with other NRC programs, 

which tend to focus on individual plants. However, the BRIIE looks at a year of industry data. Therefore, 

it is not surprising that BRIIE can detect off-baseline conditions at thresholds lower than 1.0E–05/rcry. 

For example, simulations show that 5.0E–06/rcry is exceeded with probability less than 0.02 under 

baseline conditions, but is exceeded with probability > 0.6 when each initiating event frequency 

distribution is twice as large as the baseline distribution. Values less than 1.0E–05/rcry could be used as 

an early-warning threshold for internal NRC use. 
41

-4.0E-6 -2.0E-6 0.0E0 2.0E-6 4.0E-6 6.0E-6 8.0E-6 1.0E-5 1.2E-5 1.4E-5

Estimated ∆CDF

0.0

0.2

0.4

0.6

0.8

1.0

Probability

baseline parameters

baseline params x 1.5

baseline params x 2

Figure 9-7. Simulated cumulative distribution of BRIIE (∆CDF) for industry under three assumptions. 
42
43

10. SUMMARY 

The existing ITP includes several initiating event indicators, obtained from various NRC programs 

that have differing definitions and overlap. The BRIIE described in this document is an enhancement to 

the ITP in the Initiating Events Cornerstone of Safety. The BRIIE enhances the ITP coverage of initiating 

events by the following: 

• Expanding initiating event coverage from several to nine (BWRs) and ten (PWRs) different 

initiating events, which expands the CDF risk coverage from internal events from less than 20% 

to approximately 60% 

• Eliminating overlapping of initiating event indicators 

• Providing performance-based prediction limits for each of the initiating events (BRIIE Tier 1) 

• Assembling the individual initiating event current performance into an integrated risk measure 

(approximating ∆CDF) that can be used to assess the risk significance of changes in BWR and 

PWR initiating event performance (BRIIE Tier 2) 

• Providing a threshold of risk significance for the integrated risk measure. 

The BRIIE outlined in this report includes a Tier 1 monitoring of initiating event performance and 

a Tier 2 integrated risk indicator based on initiating event performance. Existing NRC programs provide 

the yearly initiating event data and the Birnbaum importance measures needed for the BRIIE, so no 

additional data collection is needed. 
44
45

11. REFERENCES 

1. U.S. Nuclear Regulatory Commission, “Industry Trends,” http://www.nrc.gov/reactors/operating/ 

oversight/industry-trends.html. 

2. U.S. Nuclear Regulatory Commission, “Reactor Oversight Process,” 

http://www.nrc.gov/reactors/operating/ oversight.html. 

3. U.S. Nuclear Regulatory Commission, “Development of an Industry Trends Program for 

Operating Power Reactors,” SECY-01-0111, June 22, 2001. 

4. U.S. Nuclear Regulatory Commission, “Results of the Industry Trends Program for Operating 

Power Reactors and Status of Ongoing Development,” SECY-02-0058, April 1, 2002. 

5. U.S. Nuclear Regulatory Commission, “FY 2002 Results of the Industry Trends Program for 

Operating Power Reactors and Status of Ongoing Development,” SECY-03-0057, April 17, 2003. 

6. U.S. Nuclear Regulatory Commission, “FY 2003 Results of the Industry Trends Program for 

Operating Power Reactors and Status of Ongoing Development,” SECY-04-0052, April 6, 2004. 

7. U.S. Nuclear Regulatory Commission, “FY 2004 Results of the Industry Trends Program for 

Operating Power Reactors and Status of Ongoing Development,” SECY-05-0069, May 17, 2005. 

8. U.S. Nuclear Regulatory Commission, “FY 2005 Results of the Industry Trends Program for 

Operating Power Reactors and Status of Ongoing Development,” SECY-06-0076, March 31, 

2006. 

9. U.S. Nuclear Regulatory Commission, Performance Indicators for Operating Commercial 

Nuclear Power Plants,” NUREG-1187, January 2000. 

10. U.S. Nuclear Regulatory Commission, Precursors to Potential Severe Core Damage Accidents,” 

NUREG/CR-4674 series, December 2003. 

11. U.S. Nuclear Regulatory Commission, “Status of the Accident Sequence Precursor Program and 

the Development of Standardized Plant Analysis Risk (SPAR) Models,” SECY-06-0208, October 

2006. 

12. U.S. Nuclear Regulatory Commission, Performance and Accountability Report - Fiscal Year 

2005, NUREG-1542, Vol. 11, November 2005. 

13. U.S. Nuclear Regulatory Commission, Budget Estimates and Performance Plan Fiscal Year 

2005, NUREG-1100, Vol. 20, February 2004. 

14. U.S. Nuclear Regulatory Commission, NRC Inspection Manual, Manual Chapter 0313, “Industry 

Trends Program,” ML060320705, March 13, 2006. 

15. Hamzehee, H.G. et al., Risk-Based Performance Indicators Results of Phase-1 Development, U.S. 

Nuclear Regulatory Commission, NUREG-1753, April 2002. 

16. Poloski, J.P. et al., Rates of Initiating Events at U.S. Nuclear Power Plants: 1987-1995, U.S. 

Nuclear Regulatory Commission, NUREG/CR-5750, February 1999. 

17. Eide, S.A. et al., Industry-Average Performance for Components and Initiating Events at U.S. 

Commercial Nuclear Power Plants, U.S. Nuclear Regulatory Commission, NUREG/CR-6928, 

February 2007. 

18. Atwood, C.L., “Constrained Noninformative Priors in Risk Assessment,” Reliability Engineering 
46

and System Safety, 1996; 53:37-46. 

19. Dube, D.A. et al., Independent Verification of the Mitigating Systems Performance Index (MSPI) 

Results for the Pilot Plants, U.S. Nuclear Regulatory Commission, NUREG-1816, February 

2005. 

20. Advisory Committee on Reactor Safeguards Reliability and PRA Subcommittee, Official 

Transcript of Proceedings, Wednesday, May 7, 2003, Neal R. Gross and Co., Inc. 

21. Andersen, J.W. memorandum to S.A. Richards, “Summary of Public Workshop on Integrated 

Industry Initiating Event Performance Indicator for the NRC’s Industry Trends Program,” U.S. 

Nuclear Regulatory Commission, ML032390217, September 9, 2003. 

22. Su, T.M. et al., Individual Plant Examination Database – User’s Guide, U.S. Nuclear Regulatory 

Commission, NUREG-1603, April 1997. 

23. U.S. Nuclear Regulatory Commission, “Reactor Operational Experience Results and Databases, 

Study Results,” http://www.nrcoe.inel.gov/results. 

24. Rasmuson, D.M. et al., “An Overview of the Reliability and Availability Data System (RADS),” 

International Topical Meeting on Probabilistic Safety Analysis PSA’05, American Nuclear 

Society, Inc., 2005. 

25. Atwood, C.L. et al., Handbook of Parameter Estimation for Probabilistic Risk Assessment, U.S. 

Nuclear Regulatory Commission, NUREG/CR-6823, September 2003. 

26. Smith, C.L. et al., Testing, Verifying, and Validating SAPHIRE Versions 6.0 and 7.0, U.S. 

Nuclear Regulatory Commission, NUREG/CR-6688, October 2000. 

27. U.S. Nuclear Regulatory Commission, “An Approach for Using Probabilistic Risk Assessment in 

Risk-Informed Decisions on Plant-Specific Changes to the Licensing Basis,” Revision 1, 

Regulatory Guide 1.174, November 2002. 
47

GLOSSARY 

Bayesian update—the process by which a posterior initiating event distribution is obtained using Bayes 

Theorem. The process uses a prior initiating event frequency distribution representing industry-average 

performance and evidence [plant-specific, plant-type, or industry data (events and reactor critical years)] 

to obtain the posterior distribution. 

Baseline Risk Index for Initiating Events (BRIIE)—an industry-level estimate of change in core damage 

frequency (expressed on a per plant basis) resulting from changes in initiating event frequency 

performance relative to established industry-average baseline performance. The BRIIE is quantified using 

Equations (4-2), (4-3), and (4-4). 

Birnbaum importance measure—for initiating events, the Birnbaum importance measure is the derivative 

with respect to the initiating event of interest of the standardized plant analysis risk (SPAR) core damage 

frequency from internal events. The Birnbaum is calculated by evaluating the SPAR core damage 

frequency with the initiating event frequency set to 1.0/rcry and 0.0/rcry. The difference in core damage 

frequency (divided by the difference in frequency) is the Birnbaum. In this context, the Birnbaum has no 

units. 

Calendar year (CY)—the calendar year starts on January 1 and ends on December 31. 

Fiscal year (FY)—for the U.S. Nuclear Regulatory Commission, the fiscal year starts on October 1 of the 

prior calendar year and ends on September 30. 

Initiating event—an unplanned event that occurs while a nuclear power plant is in critical operation and 

requires that plant to shut down to achieve a stable state. Definitions for individual initiating events used 

in this report are found in NUREG/CR-5750, with modifications as explained in NUREG/CR-6928. 

Initiating event baseline frequency—the established industry-average baseline frequency documented in 

NUREG/CR-6928. These frequencies were established using industry data over baseline periods ending 

in 2002 (with start years ranging from 1988 to 1998, depending whether trends existed) and are 

considered to represent up-to-date industry performance. 

Initiating event current frequency—the frequency estimate based on industry data for a given year. The 

current frequency is estimated using a Bayesian update with the prior gamma distribution (a constrained 

noninformative distribution or CNID) characterized by the mean frequency from the baseline distribution 

and α = 0.5., as indicated in Equation (4-3). Industry data for the given year are collected by plant type 

(boiling water reactor or pressurized water reactor), and the current frequency is then evaluated for each 

plant type. The current frequency estimate is then compared with the baseline frequency to determine the 

change in plant-type performance for that year, as indicated in Equation (4-2). 

Initiating event historical frequency—the historical frequency is the frequency estimate based on industry 

data for a past (historical) year. This frequency is quantified similar to the “current” frequency but using 

data from a past year. 

Prediction limit—an upper bound limit for the number of industry-wide or plant-type initiating events 

occurring within a year that, if reached or exceeded, indicates a potential degradation in initiating event 

performance. The Tier 1 portion of the BRIIE compares industry or plant-type yearly data with 

established prediction limits for the initiating events covered under the program. The prediction limits 

used in the BRIIE represent the 95% limits, indicating that given baseline performance, there is a 0.05 

probability that the prediction limit will be reached or exceeded for a given year. These prediction limits 

incorporate both uncertainty in the baseline frequency (epistemic uncertainty) and randomness of the 
48

event count in a future year (aleatory uncertainty). 

Tier 1—that portion of the BRIIE involving performance monitoring of individual initiating events at the 

industry or plant-group level (depending upon the initiator). Yearly event counts for each initiating event 

covered within the program are compared with established prediction limits. If an initiating event meets 

or exceeds its prediction limit, then performance is declared to be degraded, and an engineering analysis 

is performed to identify potential causes. 

Tier 2—that portion of the BRIIE involving integration of individual initiating event performance into a 

risk index [the BRIIE, Equations (4-2), (4-3), and (4-4)] and comparison with a risk-significant threshold. 

If the threshold is reached or exceeded, then the degradation in initiating event performance for the year 

in question is declared to be risk significant and this is reported to Congress. In such cases, engineering 

analyses will be performed to identify causes and potential remedies. 
A-1

Appendix A 

Initiating Event Data and Baselines 
A-2
A-3

Table of Contents 

Appendix A...............................................................................................................................................A-5

Figures 

Figure A-1. PWR general transient—TRAN (PWR)................................................................................A-5

Figure A-2. BWR general transient—TRAN (BWR)...............................................................................A-5

Figure A-3. PWR loss of condenser heat sink—LOCHS (PWR).............................................................A-6

Figure A-4. BWR loss of condenser heat sink—LOCHS (BWR). ...........................................................A-6

Figure A-5. Loss of main feedwater—LOMFW.......................................................................................A-7

Figure A-6. Loss of offsite power—LOOP. .............................................................................................A-7

Figure A-7. Loss of vital ac bus—LOAC. ................................................................................................A-8

Figure A-8. Loss of vital dc bus—LODC.................................................................................................A-8

Figure A-9. PWR stuck open SRV—SORV (PWR). ...............................................................................A-9

Figure A-10. BWR stuck open SRV—SORV (BWR)..............................................................................A-9

Figure A-11. PWR loss of instrument air—LOIA (PWR)......................................................................A-10

Figure A-12. BWR loss of instrument air—LOIA (BWR).....................................................................A-10

Figure A-13. Very small LOCA—VSLOCA. ........................................................................................A-11

Figure A-14. PWR steam generator tube rupture—SGTR (PWR). ........................................................A-11
A-4
 A-5

Appendix A 

Initiating Event Data and Baselines 

This appendix presents graphs for each initiating event covered under the Baseline Risk Index for 

Initiating Events (BRIIE). Each graph presents fiscal year (FY) data, the baseline period chosen to 

estimate current plant-group or industry-average performance, the baseline performance, and the 95% 

prediction limit. The initiating event data are based on operating experience at U.S. commercial nuclear 

power plants from FY 1988 through FY 2005, as reported in licensee event reports (LERs). 

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

Fiscal Year

0.0

0.8

1.6

2.4

3.2

Events per reactor critical year

Rate for PWR general transients

Baseline industry average (EB)

95% prediction limit

Baseline period: CY1998-2002

Baseline log model p-value = 0.26 TRANP-07-Dec-2006

Figure A-1. PWR general transient—TRAN (PWR). 

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

Fiscal Year

0.0

0.8

1.6

2.4

3.2

Events per reactor critical year

Rate for BWR general transients

Baseline industry average (Jeffreys)

95% prediction limit

Baseline period: CY1997-2002

Baseline log model p-value = 0.53 TRANB-07-Dec-2006

Figure A-2. BWR general transient—TRAN (BWR). 
 A-6

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

Fiscal Year

0.00

0.08

0.16

0.24

0.32

0.40

Events per reactor critical year

Rate for PWR loss of heat sink

Baseline industry average (Jeffreys)

95% prediction limit

Baseline period: CY1995-2002

Baseline log model p-value = 0.73 LOCHSP-07-Dec-2006

Figure A-3. PWR loss of condenser heat sink—LOCHS (PWR). 

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

Fiscal Year

0.0

0.2

0.4

0.6

0.8

1.0

Events per reactor critical year

Rate for BWR loss of heat sink

Baseline industry average (EB)

95% prediction limit

Baseline period: CY1996-2002

Baseline log model p-value = 0.76 LOCHSB-07-Dec-2006

Figure A-4. BWR loss of condenser heat sink—LOCHS (BWR). 
 A-7

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

Fiscal Year

0.0

0.1

0.2

0.3

0.4

0.5

Events per reactor critical year

Rate for loss of feedwater

Baseline industry average (EB)

95% prediction limit

Baseline period: CY1993-2002

Baseline log model p-value = 0.22 LOMFW-07-Dec-2006

Figure A-5. Loss of main feedwater—LOMFW. 

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

Fiscal Year

0.00

0.04

0.08

0.12

0.16

0.20

Events per reactor critical year

Rate for loss of offsite power

Baseline industry average (Jeffreys)

95% prediction limit

Baseline period: CY1997-2004

Baseline log model p-value = 0.0031 LOOP-07-Dec-2006

Figure A-6. Loss of offsite power—LOOP. 
 A-8

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

Fiscal Year

0.00

0.01

0.02

0.03

0.04

0.05

0.06

Events per reactor critical year

Rate for loss of vital AC bus

Baseline industry average (Jeffreys)

95% prediction limit

Baseline period: CY1992-2002

Baseline log model p-value = 0.998 LOAC-07-Dec-2006

Figure A-7. Loss of vital ac bus—LOAC. 

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

Fiscal Year

0.000

0.008

0.016

0.024

0.032

0.040

Events per reactor critical year

Rate for loss of vital DC bus

Baseline industry average (Jeffreys)

95% prediction limit

Baseline period: CY1988-2002

Baseline log model p-value = 0.43 LODC-07-Dec-2006

Figure A-8. Loss of vital dc bus—LODC. 
 A-9

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

Fiscal Year

0.00

0.01

0.02

0.03

0.04

0.05

Events per reactor critical year

Rate for PWR stuck open SRV

Baseline industry average (Jeffreys)

95% prediction limit

Baseline period: CY1988-2002

Baseline log model p-value = 0.49 SORVP-07-Dec-2006

Figure A-9. PWR stuck open SRV—SORV (PWR). 

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

Fiscal Year

0.00

0.04

0.08

0.12

0.16

0.20

Events per reactor critical year

Rate for BWR stuck open SRV

Baseline industry average (Jeffreys)

95% prediction limit

Baseline period: CY1993-2002

Baseline log model p-value = 0.78 SORVB-07-Dec-2006

Figure A-10. BWR stuck open SRV—SORV (BWR). 
 A-10

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

Fiscal Year

0.00

0.02

0.04

0.06

0.08

0.10

Events per reactor critical year

Rate for PWR loss of instrument air

Baseline industry average (Jeffreys)

95% prediction limit

Baseline period: CY1997-2002

Baseline log model p-value = 0.94 LOIAP-07-Dec-2006

Figure A-11. PWR loss of instrument air—LOIA (PWR). 

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

Fiscal Year

0.00

0.04

0.08

0.12

0.16

0.20

Events per reactor critical year

Rate for BWR loss of instrument air

Baseline industry average (Jeffreys)

95% prediction limit

Baseline period: CY1991-2002

Baseline log model p-value = 0.11 LOIAB-07-Dec-2006

Figure A-12. BWR loss of instrument air—LOIA (BWR). 
 A-11

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

Fiscal Year

0.000

0.008

0.016

0.024

0.032

0.040

Events per reactor critical year

Rate for very small LOCA

Baseline industry average (Jeffreys)

95% prediction limit

Baseline period: CY1992-2002

Baseline log model p-value = 0.73 VSLOCA-07-Dec-2006

Figure A-13. Very small LOCA—VSLOCA. 

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

Fiscal Year

0.00

0.01

0.02

0.03

0.04

0.05

Events per reactor critical year

Rate for PWR steam generator tube rupture

Baseline industry average (Jeffreys)

95% prediction limit

Baseline period: CY1991-2002

Baseline log model p-value = 0.97 SGTR-07-Dec-2006

Figure A-14. PWR steam generator tube rupture—SGTR (PWR). 
 A-12
B-1

Appendix B 

Initiating Event Prediction Limits 
B-2
B-3

Table of Contents 

Appendix B ............................................................................................................................................... B-5

B.1 Introduction............................................................................................................................... B-5

B.2 Basics ........................................................................................................................................ B-7

B.3 Method of Choice ..................................................................................................................... B-9

B.4 References............................................................................................................................... B-10

Tables 

Table B-1. BRIIE initiating event baseline data and frequency distributions........................................... B-6

Table B-2. BRIIE Tier 1 performance-based prediction limits................................................................. B-9
B-4
B-5

Appendix B 

Initiating Event Prediction Limits 

B.1 Introduction 

The initiating event baseline frequency distributions used in the Baseline Risk Index for Initiating 

Events (BRIIE) are presented in Table B-1. These baseline frequency distributions are identical to those 

presented in NUREG/CR-6928 (Ref. B-1). The distributions were generated in several ways, but in 

general they are interpreted as representing the plant-to-plant variation in initiating event frequency. The 

following general principles were used to estimate the amount of plant-to-plant variation: 

1. For several cases—TRAN [pressurized water reactor (PWR)], LOCHS [boiling water reactor 

(BWR)], and LOMFW—the plant-to-plant variation was quantified using the empirical Bayes 

(EB) method with the Kass-Steffey adjustment. Both the mean of the gamma distribution and the 

α parameter were obtained from the EB results. These three cases all contained many events over 

their baseline periods. Using the EB method with the Kass-Steffey adjustment, the resulting 

distribution reflects both plant-to-plant variation and uncertainty in the mean frequency. 

2. At the other extreme, cases with very few events (two or fewer over the baseline period)—

LODC, SORV (PWR), VSLOCA, and SGTR—cannot possibly result in quantitative estimates of 

the plant-to-plant variation because of the limited data. These initiating events are rare and the 

plant-to-plant variation is believed to be large, although the data are insufficient to validate this 

belief. For these initiating events, the constrained noninformative distribution (CNID) was 

chosen. The CNID assumes α = 0.5 (indicating a wide distribution) and uses a mean obtained 

from the data. In this case, the mean was determined using a Bayesian update of the Jeffreys 

noninformative prior [adding 0.5 to the number of events and dividing by the reactor critical year 

(rcry) exposure]. 

3. Other cases—TRAN (BWR), LOCHS (PWR), LOAC, SORV (BWR), and LOIA (BWR)—have 

enough data to indicate that the plant-to-plant variation is small but not quantifiable using the EB 

method. For those cases, the baseline data were pooled and used to update the Jeffreys 

noninformative prior. (Baseline period data presented in Table B-1 represent the pooled data.) 

However, even for these cases, the resulting distribution was interpreted as reflecting plant-toplant variation as well as uncertainty about the mean value. 

Among the initiating events listed in Table B-1, there are two special cases that do not explicitly 

follow the general principles listed above. For LOIA (PWR) there were three events in the baseline 

period. However, in that case, one plant experienced two of the three events. This indicated more plant-toplant variability than is assumed from the third principle listed above. Therefore, the CNID was used in 

that case. In addition, for LOOP the distribution was obtained directly from NUREG/CR-6890 (Ref. B-2). 

The following sections describe possible methods for determining 95% prediction limits given 

these baseline frequency estimates and assumptions concerning plant-to-plant variation. 
B-6

Table B-1. BRIIE initiating event baseline data and frequency distributions. 

Baseline Data 

(note a) 

Gamma Frequency Distribution 

(note a) 

Initiating Event Identifier Baseline 

Period 

Events Exposure 

(rcry) 

Mean 

(1/rcry) 

α β 

(rcry) 

General Transient 

—PWR 

TRAN 

(PWR) 

1998–2002 228 304.0 7.51E–01 17.77 23.66 

General Transient 

—BWR 

TRAN 

(BWR) 

1997–2002 149 180.2 8.30E–01 149.5 180.2 

Loss of 

Condenser Heat 

Sink—PWR 

LOCHS 

(PWR) 

1995–2002 38 475.0 8.11E–02 38.50 475.0 

Loss of 

Condenser Heat 

Sink—BWR 

LOCHS 

(BWR) 

1996–2002 41 208.6 1.97E–01 11.08 56.38 

Loss of Main 

Feedwater 

LOMFW 1993–2002 84 881.9 9.59E–02 1.326 13.83 

Loss of Offsite 

Power 

LOOP 1997–2004 24 724.3 3.59E–02 1.580 44.02 

Loss of Vital AC 

Bus 

LOAC 1992–2002 8 965.8 8.80E–03 8.500 965.8 

Loss of Vital DC 

Bus 

LODC 1988–2002 1 1282.4 1.17E–03 0.500 427.5 

Stuck Open SRV 

—PWR 

SORV 

(PWR) 

1988–2002 2 866.6 2.88E–03 0.500 173.3 

Stuck Open SRV 

—BWR 

SORV 

(BWR) 

1993–2002 6 291.7 2.23E–02 6.500 291.7 

Loss of 

Instrument Air 

—PWR 

LOIA 

(PWR) 

1997–2002 3 356.9 9.81E–03 0.500 50.99 

Loss of 

Instrument Air 

—BWR 

LOIA 

(BWR) 

1991–2002 3 343.3 1.02E–02 3.500 343.3 

Very Small 

LOCA 

VSLOCA 1992–2002 1 965.8 1.55E–03 0.500 321.9 

Steam Generator 

Tube Rupture 

—PWR 

SGTR 

(PWR) 

1991–2002 2 706.4 3.54E–03 0.500 141.3 

Note a – Rcry is reactor critical year. 
B-7

B.2 Basics 

If an event rate λ has a gamma(α, β) distribution, and if X given λ has a Poisson(λt) distribution, for 

some known t, then the unconditional gamma-Poisson distribution of X is negative binomial: 

α x α x p p

α x

α x

p p

α

α x

X x (1 ) ( ) !

( ) (1 ) 1

1

Pr( ) − Γ

Γ + − = ⎟

⎟

⎠

⎞

⎜

⎜

⎝

⎛

−

+ − = = , x = 0, 1, 2, ... (B-1) 

where p = β/(β+t). Denote this as negative binomial(p, α). The mean of the negative binomial is 

E(X) = α(1 − p)/p 

and the variance is 

var(X) = E(X)/p . 

When we have many values Xi, such as the counts from many plants, each Xi has an unconditional 

negative binomial(pi, α) distribution, where pi = β/(β+ti). However, the distribution of the sum of event 

counts, X = ΣXi, depends on the model, which may have a common λ for all the Xis or a distinct value of λ

for each Xi. 

The Bayesian predictive distribution of X is defined as follows. Suppose that X has a distribution 

that depends on a parameter that we denote generically as θ. In the simplest model, θ is just the event rate 

λ, but in a hierarchical model θ is the two-dimensional pair (α, β). Let θ have a Bayesian distribution g

that is evaluated using baseline data. Then the predictive distribution of X is defined as 

∫ Pr(X = x) = Pr(X = x | θ)g(θ)dθ . 

Simple Model Assuming λ Is the Same for All Plants 

In the simple model, λ is assumed to be the same for all plants. Conditional on λ, X = ΣXi has a 

Poisson(λt) distribution, where t = Σti. If the Jeffreys prior is used and updated with xbase events in the 

baseline period of tbase reactor critical years, the posterior distribution of λ is gamma(α, β), where α = xbase

+ 0.5 and β = tbase. Use this distribution of λ, and let t be the number of rcry in some future prediction 

period. The predictive distribution of X is negative binomial(p, α), where p = β/(β+t). The mean is 

E(X) = α(1 − p)/p = tα/β = t(xbase + 0.5)/tbase. (B-2) 

The variance is 

var(X) = α(1 − p)/p

2

 = E(X)(β+ t)/β = E(X)(1 + t/tbase). (B-3) 

A Poisson count would have variance equal to the mean. X has a variance somewhat larger than a Poisson 

random variable with the same mean. 

Complex Model Assuming λ Is Different for Each Plant 

In the hierarchical model, each plant has a different event rate, with the rate for the i

th plant 

generated randomly from a gamma(α, β) distribution. Then the Xis are independent negative binomial 

random variables. For the prediction period, we “shuffle the deck,” that is, we assume that the same 

gamma distribution applies, but the high plants during the baseline period are not necessarily the high 

plants during the prediction period. Also, we assume the same value of ti for every plant in the prediction 

period, denoted as t1, and let m denote the number of plants. It is shown below that the unconditional 

distribution of X is negative binomial(p1, mα), with p1 = β/(β+t1). This can be used to obtain a prediction 

limit. 

The moment-generating function of Xi is 
B-8

[ ]

α

s

i

i

X i

p e

p

m s E sX i ⎥

⎦

⎤ ⎢

⎣

⎡

− − ≡ = 1 (1 ) ( ) exp( ) (B-4) 

with pi = β/(β+ti). This fact is relatively easy to derive from the distribution in Equation (B-1). The 

moment-generating function of X is the product of the individual moment-generating functions. When the 

tis are all equal to the same value, t1, we have 

[ ][ ]

mα

s

m

i

X i i

p e

p m s E s X E sX ⎥

⎦

⎤ ⎢

⎣

⎡

− − ≡ Σ = ∏ = = 1 (1 ) ( ) exp( ) exp( )

1

1

1

 (B-5) 

This shows that the unconditional distribution of X is negative binomial(p1, mα).] 

It is interesting to work directly with the moments. Suppose, for any values α*

 and β*, that the count 

at plant i, Xi, has a negative binomial(p, α*) distribution, where p = β*/(t1 + β*). In the present application, 

α*

 and β* are the EB estimates. By the formula for the mean of a negative binomial, the predictive mean 

count in a future time t1 is 

E(Xi) = α*

(1 − p)/p = t1α*/β*. (B-6) 

Therefore, the predictive mean of ΣXi is tα*

/β*. In practice the values of α* and β* are highly correlated, 

and it turns out that the predictive mean of ΣXi is almost exactly equal to the observed baseline count 

multiplied by t/tbase, where t is the rcry for all the plants in the prediction period, and tbase is the rcry of all 

the plants in the baseline period. This is very close to Equation (B-2) 

By the formula for the variance of a negative binomial, 

var(Xi) = t1(1−p)/p

2

 = E(Xi)[(β* + t1)/β*] = E(Xi)[1 + t1/β*] . (B-7) 

Therefore, we have 

var(ΣXi) = E(ΣXi)[1 + t1/β*

] . (B-8) 

As before, t1 is the rcry for one plant in the prediction period. If t1 is small compared to β*

, then we do not 

need an accurate estimate of β*

. The variance is only slightly larger than the mean. Therefore, when t1 is 

small compared to β*

, the predictive distribution is only slightly wider than the distribution of a Poisson 

random variable with mean equal to xbase×t/tbase. This result may be compared with Equation (B-3) for the 

non-hierarchical case. 
B-9

B.3 Method of Choice 

Given the assumption that there is plant-to-plant variation for initiating event frequencies, the 

complex model approach discussed above was used. Results are summarized in Table B-2. 

Table B-2. BRIIE Tier 1 performance-based prediction limits. 

Initiating Event Identifier Mean 

(1/rcry) 

Estimated rcry 

Per Year for 

Plant Group 

(note a) 

Expected 

Number of 

Events Per Year 

(note b) 

95% 

Prediction 

Limit 

General Transient—PWR TRAN (PWR) 7.51E-01 62.1 46.6 59 

General Transient—BWR TRAN (BWR) 8.30E-01 30.6 25.4 35 

Loss of Condenser Heat 

Sink—PWR 

LOCHS (PWR) 8.11E-02 62.1 5.0 10 

Loss of Condenser Heat 

Sink—BWR 

LOCHS (BWR) 1.97E-01 30.6 6.0 11 

Loss of Main Feedwater LOMFW 9.59E-02 92.7 8.9 15 

Loss of Offsite Power LOOP 3.59E-02 92.7 3.3 8 

Loss of Vital AC Bus LOAC 8.80E-03 92.7 0.8 3 

Loss of Vital DC Bus LODC 1.17E-03 92.7 0.1 2 

Stuck Open SRV—PWR SORV (PWR) 2.88E-03 62.1 0.2 2 

Stuck Open SRV—BWR SORV (BWR) 2.23E-02 30.6 0.7 3 

Loss of Instrument Air 

—PWR 

LOIA (PWR) 9.81E-03 62.1 0.6 3 

Loss of Instrument Air 

—BWR 

LOIA (BWR) 1.02E-02 30.6 0.3 2 

Very Small LOCA VSLOCA 1.55E-03 92.7 0.1 2 

Steam Generator Tube 

Rupture—PWR 

SGTR (PWR) 3.54E-03 62.1 0.2 2 

Note a – There are 34 BWRs and 69 PWRs (total of 103) in the U.S. commercial nuclear power plant industry. The 

rcry estimates represent 90% critical operation during a calendar year. Rcry is reactor critical year. 

Note b – The expected number of events is the mean frequency multiplied by the plant group rcry. 
B-10

B.4 References 

B-1. Eide, S.A. et al., Industry-Average Performance for Components and Initiating Events at U.S. 

Commercial Nuclear Power Plants, U.S. Nuclear Regulatory Commission, NUREG/CR-6928, 

February 2007. 

B-2. Eide, S.A. et al., Reevaluation of Station Blackout Risk at Nuclear Power Plants, U.S. Nuclear 

Regulatory Commission, NUREG/CR-6890, December 2005. 
C-1

Appendix C 

Initiating Event Birnbaum Importance Measures 
C-2
C-3

Table of Contents 

Appendix C ............................................................................................................................................... C-5

Tables 

Table C-1. Plant-group average Birnbaum importance measures. ........................................................... C-6
C-4
C-5

Appendix C 

Initiating Event Birnbaum Importance Measures 

The Baseline Risk Index for Initiating Events (BRIIE) requires Birnbaum importance measures for 

each of ten types of initiating events. Birnbaum estimates were obtained from the standardized plant 

analysis risk (SPAR) models of U.S. commercial nuclear power plants. SPAR models cover at power, 

internal event core damage frequency (CDF). Contributions to CDF from shutdown and from external 

events are not included at this time. There are 72 SPAR models covering the 103 operating plants [34 

boiling water reactors (BWRs), and 69 pressurized water reactors (PWRs)]. 

The SPAR Rev. 3i models have been updated to SPAR Rev. 3 models. Results from plant visits 

have been incorporated into all SPAR Rev. 3 models. Birnbaum estimates are based on the SPAR Rev. 

3.21 models released in December 2005. 

The BRIIE measures the change in CDF, or ∆CDF, resulting from changes in individual initiating 

event frequencies. For a given initiator, the ∆CDF is the Birnbaum times the change in initiator frequency 

(current value minus baseline value). Initiating event frequencies are presented as events per reactor 

critical year (rcry), and the associated Birnbaums have no units. Table C-1 shows the Birnbaum 

importance measures for each initiator. 
C-6

Table C-1. Plant-group average Birnbaum importance measures. 

Birnbaum 

Importance 

(note a) 

Initiating Event 

BWRs PWRs 

Initiator 

Modeled 

Explicitly 

in SPAR? 

Birnbaum 

Importance 

Obtained 

How? 

Comments 

General Transient 

—TRAN 

5.51E-07 7.97E-07 Yes 

(note b) 

Loss of Condenser Heat 

Sink—LOCHS 

4.31E-06 2.32E-05 Yes/No 

(note b)

Loss of Main Feedwater 

—LOMFW 

2.32E-06 2.01E-06 Yes/No 

(note b)

BWR: SPAR 

output 

PWR: cut set 

slicing 

All BWR models explicitly 

model general transients, 

loss of condenser heat sink, 

and loss of main feedwater. 

About half of the PWR 

models do this; the others 

must be determined by cut 

set slicing. 

Loss of Offsite Power 

—LOOP 

6.15E-05 1.12E-04 Yes Directly from 

SPAR output 

Loss of Vital AC Bus 

—LOAC 

6.85E-05 4.22E-05 Yes Directly from 

SPAR output 

SPAR models include this 

initiator if it is risk 

significant at the plant in 

question. Thirty four plant 

models include this 

initiator. 

Loss of Vital DC Bus 

—LODC 

2.61E-04 9.39E-04 Yes Directly from 

SPAR output 

PWR results dominated by 

4 plants (out of 66 PWRs 

with LODC included in the 

SPAR models) 

Stuck Open SRV 

—SORV 

4.59E-06 7.14E-04 No BWR: SPAR 

output 

PWR: cut set 

slicing 

Loss of Instrument Air 

—LOIA 

3.40E-05 6.57E-05 Yes Directly from 

SPAR output 

Very Small LOCA 

—VSLOCA 

5.51E-07 7.97E-07 Yes Directly from 

SPAR output 

Steam Generator Tube 

Rupture—SGTR 

Not 

applicable 

3.89E-04 Yes Directly from 

SPAR output 

SPAR models for this 

initiator are thought to be 

conservative (result in high 

CDF estimates).

Note a – Per plant 

Note b – In some of the PWR general transient event trees, top events also cover loss of main feedwater, loss of 

condenser heat sink, and stuck open SRV. Therefore, the Birnbaum obtained directly from the SPAR output for the 

general transient initiator reflects importances from four types of initiating events. To obtain the correct Birnbaum 

for the general transient initiator, cut set slicing was used.
D-1

Appendix D 

BRIIE Tier 2 Formulations 
D-2
D-3

Table of Contents 

Appendix D...............................................................................................................................................D-5

D.1 BRIIE Calculations Using Group and Plant-Specific Methods ................................................D-6

D.1.1 Comparison of BRIIE Averaging Method ........................................................................D-7

D.1.2 Comparison of Bayesian and MLE Frequency Estimates.................................................D-9

D.2 Industry Total Calculations.....................................................................................................D-12

D.2.1 Comparison of BRIIE Averaging Method ......................................................................D-12

D.2.2 Comparison of Bayesian and MLE Frequency Estimates...............................................D-12

D.3 Comparison of Bayesian and MLE Frequency Estimates Using Core Damage Frequency ...D-14

D.4 Conclusions.............................................................................................................................D-21

D.5 References...............................................................................................................................D-22

Figures 

Figure D-1. Comparison of BWR BRIIE results using plant-specific and group-average BRIIE 

calculations. ..............................................................................................................................................D-7

Figure D-2. Comparison of PWR BRIIE results using plant-specific and group-average BRIIE 

calculations. ..............................................................................................................................................D-8

Figure D-3. BWR group-average calculation showing Bayesian and MLE results................................D-10

Figure D-4. PWR group-average calculation showing Bayesian and MLE results. ...............................D-10

Figure D-5. BWR plant-specific calculation showing Bayesian and MLE results.................................D-11

Figure D-6. PWR plant-specific calculation showing Bayesian and MLE results. ................................D-11

Figure D-7. Comparison of industry BRIIE results using industry-average and plant-specific BRIIE 

calculations. ............................................................................................................................................D-12

Figure D-8. Industry average calculation using industry-average method showing Bayesian and MLE 

results. .....................................................................................................................................................D-13

Figure D-9. Industry calculation using plant-specific method showing Bayesian and MLE results. .....D-13

Figure D-10. BWR group-average CDF calculation showing Bayesian results by initiating event. ......D-16

Figure D-11. PWR group-average CDF calculation showing Bayesian results by initiating event. ......D-16

Figure D-12. BWR group-average CDF calculation showing MLE results by initiating event. ............D-17

Figure D-13. PWR group-average CDF calculation showing MLE results by initiating event..............D-17

Figure D-14. BWR plant-specific CDF calculation showing Bayesian results by initiating event. .......D-18

Figure D-15. PWR plant-specific CDF calculation showing Bayesian results by initiating event.........D-18

Figure D-16. BWR plant-specific CDF calculation showing MLE results by initiating event...............D-19

Figure D-17. PWR plant-specific CDF calculation showing MLE results by initiating event...............D-19

Tables 

Table D-1. Matrix of grouping and calculation methods..........................................................................D-6

Table D-2. Matrix of Birnbaum and IE count average figures. ................................................................D-9

Table D-3. Matrix of group-average calculation methods for CDF........................................................D-14

Table D-4. RIIE figure matrix.................................................................................................................D-15

Table D-5. BWR initiating event counts.................................................................................................D-20

Table D-6. PWR initiating event counts. ................................................................................................D-20
D-4
 D-5

Appendix D 

BRIIE Tier 2 Formulations 

The Baseline Risk Index for Initiating Events (BRIIE) Tier 2 (or BRIIE for short) measures the 

change in core damage frequency (CDF), or ∆CDF, resulting from changes in individual initiating event 

frequencies. For a given initiator, the ∆CDF is the Birnbaum importance measure times the change in 

initiator frequency (current value minus baseline value). If initiating event frequencies are presented as 

events per reactor critical year (rcry), then the BRIIE has units of ∆CDF (per rcry) for the average plant. 

This appendix explores the various potential methods for quantifying the BRIIE. In addition, two 

methods used to calculate the initiating event current frequency are explored: Bayesian update and 

maximum likelihood estimate (MLE). Therefore, this appendix investigates the sensitivity of the BRIIE to 

various modeling assumptions (or formulations). 

Finally, a variation of the BRIIE calculation that does not compare the current frequency to the 

baseline frequency is investigated for illustrative purposes. 
 D-6

D.1 BRIIE Calculations Using Group and Plant-Specific Methods 

The BRIIE can be calculated for groups of plants, depending on which group the Birnbaum and 

initiating event frequency are based on. The most obvious groups are 

• Industry – the BRIIE represents an average of all plants 

• Plant type – the BRIIE represents an average boiling water reactor (BWR) or pressurized water 

reactor (PWR) plant. 

The recommended BRIIE methodology is to calculate results at the plant-group (BWR or PWR) level and 

then combine those results into an industry average result. Given a specific group, the BRIIE is calculated 

using a group-specific average Birnbaum and group current frequency for each initiating event. However, 

the BRIIE could be calculated by averaging plant-specific results (using plant-specific Birnbaums and 

plant-specific current frequencies) within the group. Both calculation approaches are listed in Table D-1. 

Table D-1. Matrix of grouping and calculation methods. 

Group Calculation Method Designation 

Plant Group Type-Average BWR-BRIIE-A 

PWR-BRIIE-A 

Individual Plant Plant-Specific BWR-BRIIE-P 

PWR-BRIIE-P 

The group-average BRIIE (∆CDF) is given by the following equation: 

∑ ( ) =

= −

m

i

BRIIEA Bi ic ib

1

* λ λ , (D-1) 

where 

m= number of initiating events covered in the BRIIE 

Bi = group-average Birnbaum for initiating event i

= * λic group current frequency for initiating event i

λib = baseline frequency for initiating event i. 

The plant-specific BRIIE (∆CDF) is given by the following equation: 

( )

n

B

BRIIE

n

j

m

i

i j ip j ib

P

∑∑= =

−

= 1 1

, λ , λ

, (D-2) 

where 

n= number of plants in the group 

Bi, j = plant-specific Birnbaum for initiating event i, plant j

λip, j = plant-specific current frequency for initiating event i, plant j. 
 D-7

Equation (D-1) uses initiating event Birnbaums averaged across the plants within the group (e.g., 

BWRs) and the group current frequency for each initiating event. The products of the Birnbaum and 

change in initiating event frequency (current minus baseline) are summed to obtain the BRIIE. 

Equation (D-2) uses the plant-specific Birnbaum and plant-specific current frequency of the 

initiating event compared to the baseline frequency. The sum of these products is the BRIIE for that plant 

for that year. The sum over all plants gives the integrated ∆CDF for that year and this divided by the 

number of plants gives the per plant ∆CDF for that year. The results of Equations (D-1) and (D-2) would 

be equal if the Birnbaums and initiating event frequencies were equal for all plants. However, Equation 

(D-2) emphasizes the coincidence of the Birnbaum importance and the occurrence of an initiating event at 

each individual plant as compared with Equation (D-1). 

D.1.1 Comparison of BRIIE Averaging Method 

Figure D-1 and Figure D-2 show the comparison between Equations (D-1) and (D-2) for BWRs 

and PWRs. There are significant differences between the two equation variations. In some cases, the 

yearly results flip from negative to positive (or vice-versa). This generally occurs only when the 

calculation results are close to zero. 

-4.0E-06

-2.0E-06

0.0E+00

2.0E-06

4.0E-06

6.0E-06

8.0E-06

1.0E-05

1.2E-05

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

Fiscal Year

BRIIE (1/rcry)

BWR-BRIIE-P BWR-BRIIE-A

Figure D-1. Comparison of BWR BRIIE results using plant-specific and group-average BRIIE 

calculations. 

The average BRIIE method generally results in a larger prediction of the average plant ∆CDF. The 

average BRIIE calculation effectively applies observed initiating events to plants that did not actually 

observe them. In addition, the average initiating event Birnbaum is also applied, giving less credit to those 

plants that have lower Birnbaums and minimizing the impact of the plants with high Birnbaums. The 

combined effect is to predict larger results for the average BRIIE versus the plant-specific BRIIE. 
 D-8

-1.0E-05

-5.0E-06

0.0E+00

5.0E-06

1.0E-05

1.5E-05

2.0E-05

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

Fiscal Year

BRIIE (1/rcry)

PWR-BRIIE-P PWR-BRIIE-A

Figure D-2. Comparison of PWR BRIIE results using plant-specific and group-average BRIIE 

calculations. 

In FY 2003, Figure D-1 shows a significant difference (approximately a factor of 14) in the 

calculated BWR BRIIE between the group-average and the plant-specific methods. In FY 2003, there 

were eight BWR loss of offsite power (LOOP) events. These are the major contributors to the increase of 

the BRIIE over the baseline. We will investigate the LOOP calculation. 

The group-average LOOP Birnbaum for a BWR plant is 6.15E–05. The plant-specific average 

Birnbaum for the eight BWR plants that recorded a LOOP is 3.8E–05. In addition, only one of those eight 

plants has a Birnbaum larger than the group-average Birnbaum. The Birnbaum difference only accounts 

for a factor of approximately 1.6 in the FY 2003 LOOP BRIIE calculation between plant-specific and 

type average methods. 

In the plant-type average method, the Bayesian update of the LOOP frequency is performed over 

the plant type. The plant-type average LOOP frequency for FY 2003 is 1.9E–01/rcry, which is 1.6E–

01/rcry greater than the LOOP baseline. 

In the plant-specific method, the Bayesian update of the LOOP frequency is performed for each 

plant. The plants that did not experience a LOOP event that year evaluate to approximately 3.37E–02/rcry 

current LOOP frequency, which is 2.2E–03/rcry less than the baseline (making the BRIIE contribution 

from these plants negligible). The plants that do experience LOOPS evaluate to 1.0E–01/rcry current 

LOOP frequency, which is 6.5E–02/rcry greater than the baseline (a factor of 2.5 less than the plant-type 

average frequency). 

Therefore, the difference in the LOOP frequency calculation and Birnbaum accounts for about 4 of 

the factor of 14 ∆CDF difference in FY 2003. The rest of the difference is because the plant-specific 

result is divided by the number of plants in the group (34 BWRs) to get the average BWR result. 
 D-9

D.1.2 Comparison of Bayesian and MLE Frequency Estimates 

Previous BRIIE summaries used a Bayesian update to calculate the current frequency for the 

initiating event. The Bayesian update is calculated using the following equation: 

( )

( )t

c

i

i i

i +

+ = β

α λ

* , (D-3) 

where 

αi = alpha parameter of the baseline gamma distribution for initiating event i

ci = the plant count(s) for initiating event i in the current year 

βi = beta parameter of the baseline gamma distribution for initiating event i

t = critical time (rcry) for current year at the plant(s). 

In Equation (D-3), the quantities ci and t refer to either an entire plant group or to a single plant, 

depending upon the BRIIE quantification approach being used. 

We decided to investigate the use of another way of expressing the current frequency at a plant in 

the current year, the MLE: 

t

ci

i =* λ . (D-4) 

Similar to Equation (D-3), the quantities ci and t in Equation (D-4) refer to either an entire plant group or 

to a single plant, depending upon the BRIIE quantification approach being used. 

The MLE is calculated using the current year’s initiating event counts for the plant (or group of 

plants) and that plant’s (or group of plants’) critical operating history for the year. Essentially, the 

occurrence of no particular initiating event would result in a zero current frequency and one observed 

event would result in a greater than 1.0/rcry current frequency (plants rarely have a full 1.0 rcry during a 

year). The MLE BRIIE (∆CDF) calculation variation would in effect credit a plant for the lack of 

observed initiating events (negative BRIIE contribution) and strongly penalize a plant for any observed 

initiating events (positive BRIIE contribution). This should give the maximum sensitivity to the 

importance of initiating events occurring over time at various plants. 

The following charts show comparisons of the calculation results (MLE versus Bayesian) for the cases of 

Birnbaums and initiating events averaged over the plant type and calculation method as shown in 

Table D-2. 

Table D-2. Matrix of Birnbaum and IE count average figures. 

Plant Type Group-Average Birnbaum and IE 

Count 

Plant-Specific Birnbaum and IE 

Count 

BWR Figure D-3 Figure D-5 

PWR Figure D-4 Figure D-6 
 D-10

-5.0E-06

0.0E+00

5.0E-06

1.0E-05

1.5E-05

2.0E-05

1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Fiscal Year

BRIIE (1/rcry)

BRIIE, Bayesian BRIIE, MLE

Figure D-3. BWR group-average calculation showing Bayesian and MLE results. 

-2.0E-05

-1.0E-05

0.0E+00

1.0E-05

2.0E-05

3.0E-05

4.0E-05

5.0E-05

1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Fiscal Year

BRIIE (1/rcry)

BRIIE, Bayesian BRIIE, MLE

Figure D-4. PWR group-average calculation showing Bayesian and MLE results. 
 D-11

-1.0E-05

-5.0E-06

0.0E+00

5.0E-06

1.0E-05

1.5E-05

2.0E-05

1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Fiscal Year

BRIIE (1/rcry)

Plant-Specific Birnbaum, Bayesian Plant-Specific Birnbaum, MLE

Figure D-5. BWR plant-specific calculation showing Bayesian and MLE results. 

-3.0E-05

-2.0E-05

-1.0E-05

0.0E+00

1.0E-05

2.0E-05

3.0E-05

4.0E-05

1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Fiscal Year

BRIIE (1/rcry)

Plant-Specific Birnbaum, Bayesian Plant-Specific Birnbaum, MLE

Figure D-6. PWR plant-specific calculation showing Bayesian and MLE results. 

The MLE method shows larger results than the Bayesian method. In the MLE calculation when no 

events occur, the difference between the observed frequency (MLE) and the baseline frequency is the 

negative of the baseline frequency. This decreases the total BRIIE significantly. When initiating events 

occur, the MLE is ≥ 1.0/rcry. The difference between the observed frequency (MLE) and the baseline 

frequency in those cases is large. This increases the total BRIIE significantly. The net effect, in general, is 

that the calculated BRIIE (absolute value) is larger for the MLE method. 
 D-12

D.2 Industry Total Calculations 

The previous discussions have shown plant-type calculations using group-average and plantspecific calculations. The true average plant in the US is not PWR or BWR, but a weighted average of the 

two. This section will display results for the industry, comparing the same types of calculations. 

D.2.1 Comparison of BRIIE Averaging Method 

Figure D-7 shows BRIIE results at an industry level using industry-average and plant-specific 

averaging methods. The phenomena shown in Figure D-1 and Figure D-2 generally are dampened in 

Figure D-7. The large peak in FY 2003 in Figure D-1 is no longer dominant. The large peak in FY 1989 

from Figure D-2 is still dominant. 

-1.0E-05

-5.0E-06

0.0E+00

5.0E-06

1.0E-05

1.5E-05

2.0E-05

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

Fiscal Year

BRIIE (1/rcry)

Plant-Specific, Bayesian Industry Average, Bayesian

Figure D-7. Comparison of industry BRIIE results using industry-average and plant-specific BRIIE 

calculations. 

D.2.2 Comparison of Bayesian and MLE Frequency Estimates 

Figure D-8 and Figure D-9 show comparisons of the calculation results (MLE versus Bayesian) for 

the cases of Birnbaums and initiating events averaged over the industry or plant-specific groups. 
 D-13

-1.5E-05

-1.0E-05

-5.0E-06

0.0E+00

5.0E-06

1.0E-05

1.5E-05

2.0E-05

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

Fiscal Year

BRIIE (1/rcry)

Industry Average, Bayesian Industry Average, MLE

Figure D-8. Industry average calculation using industry-average method showing Bayesian and MLE 

results. 

-2.0E-05

-1.5E-05

-1.0E-05

-5.0E-06

0.0E+00

5.0E-06

1.0E-05

1.5E-05

2.0E-05

2.5E-05

3.0E-05

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

Fiscal Year

BRIIE (1/rcry)

Plant-Specific, Bayesian Plant-Specific, MLE

Figure D-9. Industry calculation using plant-specific method showing Bayesian and MLE results. 
 D-14

D.3 Comparison of Bayesian and MLE Frequency Estimates Using Core Damage 

Frequency 

Several different quantification methods were considered for evaluating the BRIIE. One method 

related to CDF represents an absolute measure of CDF for the selected initiating events. This section 

presents the results of Sections D.1 and D.2 using CDF rather than ∆CDF. 

To keep the different calculation types straight, this measure will be called the Risk Index for 

Initiating Events, or RIIE since there is no baseline term in the equations. These RIIE metrics can be 

calculated using varying levels of averages. All of the results represent CDF (per rcry) for the average 

plant. 

Table D-3. Matrix of group-average calculation methods for CDF. 

Group Calculation Method Designation 

Plant Group Group Average BWR-RIIE-A 

PWR-RIIE-A 

Individual Plant Plant-Specific BWR-RIIE-P 

PWR-RIIE-P 

The average RIIE, related to CDF, is given by the following equation: 

∑ ( ) =

=

m

i

RIIEA Bi ic

1

* λ , (D-5) 

where 

m = number of initiating events covered in the BRIIE 

Bi = group-average Birnbaum for initiating event i 

= * λic common group current frequency for initiating event i. 

The plant-specific RIIE, related to CDF, is given by the following equation: 

 

( )

n

B

RIIE

n

j

m

i

i j ip j

P

∑∑= = = 1 1

, λ ,

, (D-6) 

where 

n = number of plants in the group 

Bi, j = plant-specific Birnbaum for initiating event i, plant j 

λip, j = plant-specific current frequency for initiating event i, plant j. 

The following charts show the calculation of an absolute risk parameter (RIIE) using both the 

Bayesian and MLE methods. The absolute risk lends itself to comparisons of contributions of each 

initiator. Table D-4 shows the matrix of the calculations and the charts. 
 D-15

Table D-4. RIIE figure matrix. 

Plant Type Frequency Calculation 

Type 

Average Birnbaum and IE 

Count 

Plant-Specific Birnbaum and 

IE Count 

BWR Bayesian Figure D-10 Figure D-14 

PWR Bayesian Figure D-11 Figure D-15 

BWR MLE Figure D-12 Figure D-16 

PWR MLE Figure D-13 Figure D-17 
 D-16

0.0E+00

2.0E-06

4.0E-06

6.0E-06

8.0E-06

1.0E-05

1.2E-05

1.4E-05

1.6E-05

1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Fiscal Year

CDF (1/rcry)

LOAC LOCHS LODC LOIA LOMFW LOOP SGTR SORV TRANS VSLOCA

Figure D-10. BWR group-average CDF calculation showing Bayesian results by initiating event. 

0.0E+00

5.0E-06

1.0E-05

1.5E-05

2.0E-05

2.5E-05

3.0E-05

3.5E-05

1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Fiscal Year

CDF (1/rcry)

LOAC LOCHS LODC LOIA LOMFW LOOP SGTR SORV TRANS VSLOCA

Figure D-11. PWR group-average CDF calculation showing Bayesian results by initiating event. 
 D-17

0.0E+00

5.0E-06

1.0E-05

1.5E-05

2.0E-05

2.5E-05

1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Fiscal Year

CDF (1/rcry)

LOAC LOCHS LODC LOIA LOMFW LOOP SGTR SORV TRANS VSLOCA

Figure D-12. BWR group-average CDF calculation showing MLE results by initiating event. 

0.0E+00

5.0E-06

1.0E-05

1.5E-05

2.0E-05

2.5E-05

3.0E-05

3.5E-05

1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Fiscal Year

CDF (1/rcry)

LOAC LOCHS LODC LOIA LOMFW LOOP SGTR SORV TRANS VSLOCA

Figure D-13. PWR group-average CDF calculation showing MLE results by initiating event. 
 D-18

0.0E+00

1.0E-06

2.0E-06

3.0E-06

4.0E-06

5.0E-06

6.0E-06

1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Fiscal Year

CDF (1/rcry)

LOAC LOCHS LODC LOIA LOMFW LOOP SGTR SORV TRANS VSLOCA

Figure D-14. BWR plant-specific CDF calculation showing Bayesian results by initiating event. 

0.0E+00

2.0E-06

4.0E-06

6.0E-06

8.0E-06

1.0E-05

1.2E-05

1.4E-05

1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Fiscal Year

CDF (1/rcry)

LOAC LOCHS LODC LOIA LOMFW LOOP SGTR SORV TRANS VSLOCA

Figure D-15. PWR plant-specific CDF calculation showing Bayesian results by initiating event. 
 D-19

0.0E+00

2.0E-06

4.0E-06

6.0E-06

8.0E-06

1.0E-05

1.2E-05

1.4E-05

1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Fiscal Year

CDF (1/rcry)

LOAC LOCHS LODC LOIA LOMFW LOOP SGTR SORV TRANS VSLOCA

Figure D-16. BWR plant-specific CDF calculation showing MLE results by initiating event. 

0.0E+00

5.0E-06

1.0E-05

1.5E-05

2.0E-05

2.5E-05

3.0E-05

1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Fiscal Year

CDF (1/rcry)

LOAC LOCHS LODC LOIA LOMFW LOOP SGTR SORV TRANS VSLOCA

Figure D-17. PWR plant-specific CDF calculation showing MLE results by initiating event. 
 D-20

Table D-5 and Table D-6 show the counts of the initiating events for each plant type. The most 

important property to notice is that the MLE charts only show bar height for the initiating events that have 

occurred. The Bayesian update will always have a bar height whether there is an initiating event or not. 

Table D-5. BWR initiating event counts. 

Initiating 

Event 

(note a) 

1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 

LOAC 2 1 1 2 1 

LOCHS 15 15 12 10 9 14 8 10 5 4 7 7 8 2 6 9 8 7 

LODC 

LOIA 2 2 3 1 2 1 

LOMFW 10 4 3 8 4 3 1 3 3 2 6 3 3 1 2 2 4 

LOOP 3 1 1 3 1 1 8 2 

SORV 1 1 2 3 2 1 2 1 1 

TRANS 70 56 56 61 48 48 44 51 38 24 26 28 30 23 24 28 22 24 

VSLOCA 1 1 

Note a –These event counts include only the 34 BWRs currently operating. 

Table D-6. PWR initiating event counts. 

Initiating 

Event 

(note a) 

1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 

LOAC 1 2 

LOCHS 6 12 9 6 6 5 9 4 7 3 3 3 7 8 3 5 7 4 

LODC 1 

LOIA 2 3 1 1 1 1 1 1 1 1 

LOMFW 11 14 12 12 8 3 8 7 7 9 7 5 5 4 1 7 5 4 

LOOP 3 5 3 2 3 2 3 1 1 1 1 1 4 3 

SGTR 1 1 1 

SORV 1 1 

TRANS 164 134 122 105 116 92 78 80 74 54 41 57 42 46 32 68 45 33 

VSLOCA 1 1 1 

Note a – These event counts include only the 69 PWRs currently operating. 
 D-21

D.4 Conclusions 

Two different calculations are helpful to explain the whole story. The BRIIE (∆CDF) Figure D-7, 

industry average results, shows average plant risk on a yearly basis, but cannot show each initiating event 

contribution. Therefore, the RIIE is helpful in explaining what contributes to the BRIIE going up or 

down. 

There are significant differences between the type-average and the plant-specific approaches. The 

type-average approach produces larger BRIIE values, but does not use all of the available information. 

The plant-specific approach matches up occurrences of initiating events and the appropriate Birnbaum 

values. Where the event was important (high Birnbaum value), it shows up as important. However, if the 

event is not important (low Birnbaum value), then the BRIIE contribution is small. 

The use of Bayesian updating versus MLE for the current frequency was examined. The MLE 

method tends to produce larger results in the BRIIE when initiating events occur. To be consistent with 

the approach taken for the Mitigating Systems Performance Index (MSPI), the Bayesian update approach 

was chosen. This approach was also recommended in NUREG-1753 (Ref. D-1). However, MLE CDF 

results by plant type (BWR or PWR) are helpful in indicating the contributors to yearly BRIIE results. 
 D-22

D.5 References 

D-1. Hamzehee, H.G. et al., Risk-Based Performance Indicators Results of Phase-1 Development, U.S. 

Nuclear Regulatory Commission, NUREG-1753, April 2002. 
E-1

Appendix E 

Summary of Expert Panel Meeting 
E-2
E-3

Table of Contents 

Appendix E ............................................................................................................................................... E-5

E.1 Specific Questions Addressed................................................................................................... E-6

E.2 Conclusions Reached ................................................................................................................ E-7

E.3 References................................................................................................................................. E-9
E-4
E-5

Appendix E 

Summary of Expert Panel Meeting 

A panel of experts was convened on July 18 and 19, 2006, in conformance with the completion 

plan for the Baseline Risk Index for Initiating Events (BRIIE) development effort. The panel consisted of 

senior managers and staff with extensive knowledge of probabilistic risk assessment (PRA) methods and 

U.S. Nuclear Regulatory Commission (NRC) policies. Panel members were the following: 

• Richard Barrett – Office of Nuclear Regulatory Research (RES) 

• Nilesh Chokshi – RES 

• Thomas Boyce – Office of Nuclear Reactor Regulation (NRR) 

• James Vail – NRR 

• Robert Haag – Region II. 
E-6

E.1 Specific Questions Addressed 

The panel was convened with the objective of reviewing the BRIIE effort and establishing 

threshold values for BRIIE Tier 2 that would trigger an indication of risk-significant, degrading industry 

performance in initiating events. The specific questions directed to the panel to arrive at the threshold 

were the following: 

1. What form of the BRIIE Tier 2 should be used—absolute core damage frequency (CDF) or 

∆CDF? 

2. Which form of the initiating event current frequency should be used—maximum likelihood 

estimator (MLE) or Bayesian update? 

3. What initiating events should be included in the BRIIE—the current list, all those contained in the 

standardized plant analysis risk (SPAR) models, or a subset of all initiating events? (A constant 

would be added to the absolute form of BRIIE to account for those initiating events that would 

not be monitored, so that it is almost equal to the internal events estimated CDF. This would 

subtract out in the difference form of the BRIIE.) 

4. Which initiating events should be monitored—those that occur frequently or occasionally (i.e., 

they have been observed in the past) or all initiating events? 

5. Does it make sense to monitor initiating events that have not occurred [e.g., a loss-of-coolant 

accident (LOCA) event]? How should such an event be treated in the BRIIE if it did occur? 

6. Given the characteristics of the BRIIE and the simulation results, what are appropriate CDF and 

∆CDF thresholds? 

7. What threshold values should be used for reporting to Congress? 

8. How often should the initiating event baseline performance be updated? (An example is the 

LOOP initiating event. Years 2003 and 2004 were different from the years 1997-2002 and 2005.) 

9. Does the two-tier BRIIE make sense? 
E-7

E.2 Conclusions Reached 

Discussions during the meeting addressed all these questions. After considerable collegial debate 

on the pros and cons of the proposed alternatives for each of these issues, the panel reached the following 

conclusions: 

1. Maintain the two-tier process for the BRIIE. The panel felt that both tiers are required to provide 

an accurate and full picture of industry performance. The two-tiered process provides trending 

information and action levels for NRC engagement if the 95% prediction limit of Tier 1 is 

reached, as outlined in the NRC Inspection Manual, Manual Chapter 0313, “Industry Trends 

Program” (Ref. E-1). Tier 2 does not provide strict trending information but provides a risk 

perspective of industry performance as a deviation from a baseline value and the proximity of the 

deviation to a set threshold. 

2. The presentation for BRIIE should be in a bar graph that provides three separate values for each 

year: one bar providing industry-wide results, one bar for boiling water reactor (BWR) results, 

and the third bar for pressurized water reactor (PWR) results. All three bars for each year should 

be presented on one graph. The panel indicated that this form of presentation provides more 

information than simply aggregating industry-wide results into one number or presenting BWR 

and PWR results individually. 

3. The BRIIE should be in the form of ∆CDF with Bayesian updating. The absolute CDF form 

using MLE would be calculated and provided as a communication tool to stakeholders to provide 

additional insights. The panel preferred the ∆CDF form of BRIIE because the absolute CDF form 

of BRIIE would result in different levels for BWRs and PWRs. The ∆CDF form shows the 

change from the baseline for both types of reactors and hence is more understandable. The panel 

discussed and decided that infrequently occurring initiating events [loss of service water 

(LOSW), loss of component cooling water (LOCCW), small loss-of-coolant accident (SLOCA), 

medium loss-of-coolant accident (MLOCA), large loss-of-coolant accident (LLOCA), interfacing 

system loss-of-coolant accident (ISLOCA), reactor vessel rupture (RXVRUPT), excessive lossof-coolant accident (X-LOCA), and others] would not be included in the calculation of BRIIE. 

These events, if they occur, would be captured in the calculation of absolute CDF with MLE. 

4. The threshold for reporting to Congress should be set at 1.0E–05 per reactor critical year (rcry). It 

should be associated only with the ∆CDF BRIIE calculations. This threshold value was derived 

from considerations of the NRC safety goals, Regulatory Guide 1.174 (Ref. E-2), and consistency 

with the Reactor Oversight Process (Ref. E-3) and Accident Sequence Precursor (ASP) Program 

(Ref. E-4). The threshold was derived from coherency with current agency metrics and the 

surrogates for the safety goals discussed in Regulatory Guide 1.174. Two scenarios were 

discussed. The first was that a single event at 1.0E–03/rcry ∆CDF (e.g., the current ASP indicator 

threshold for reporting significant events to Congress) would make the aggregate industry 

performance about equal to 1.0E–05/rcry ∆CDF (1.0E–03/rcry divided by 100 plants). Also, if 

10% of plants had a problem at about 1.0E–04/rcry ∆CDF, then this would also make industry 

performance about equal to 1.0E–05/rcry ∆CDF. The 10% number was chosen to provide a 

distinction between an industry problem and issues with individual plants. 

The panel was concerned about the loss of offsite power (LOOP) grid-related events of August 

2003. The panel felt that these August 2003 events were outliers and may not be representative of 

baseline industry performance. As a sensitivity study, the BRIIE Tier 2 results by fiscal year were 

recalculated using a LOOP baseline that excluded the August 2003 grid-related events. Excluding those 
E-8

events resulted in a LOOP baseline frequency of 2.49E–02/rcry, compared with the existing baseline of 

3.59E–02/rcry. BRIIE results using the sensitivity case LOOP frequency were similar to those using the 

existing baseline frequency. Therefore, the panel felt that the BRIIE should use the existing baseline 

LOOP frequency to be consistent with NUREG/CR-6890 (Ref. E-5) and the SPAR models. 
E-9

E.3 References 

E-1. U.S. Nuclear Regulatory Commission, NRC Inspection Manual, Manual Chapter 0313, “Industry 

Trends Program,” ML060320705, March 13, 2006. 

E-2. U.S. Nuclear Regulatory Commission, “An Approach for Using Probabilistic Risk Assessment in 

Risk-Informed Decisions on Plant-Specific Changes to the Licensing Basis,” Revision 1, 

Regulatory Guide 1.174, November 2002. 

E-3. U.S. Nuclear Regulatory Commission, “Reactor Oversight Process,” 

http://nrc.gov/reactors/operating/ oversight.html. 

E-4. U.S. Nuclear Regulatory Commission, “Status of the Accident Sequence Precursor Program and 

the Development of Standardized Plant Analysis Risk (SPAR) Models,” SECY-06-0208, October 

2006. 

E-5. Eide, S.A. et al., Reevaluation of Station Blackout Risk at Nuclear Power Plants, U.S. Nuclear 

Regulatory Commission, NUREG/CR-6890, December 2005. 
E-10
F-1

Appendix F 

Resolution of Comments 
F-2
F-3

Table of Contents 

Appendix F ............................................................................................................................................... F-5

Tables 

Table F-1. Comment resolution. ............................................................................................................... F-9
F-4
F-5

Appendix F 

Resolution of Comments 

Various organizations were invited to comment on the draft report Development of an Integrated 

Industry Initiating Event Indicator (S. Eide et al., March 13, 2003). Comments were received in 2003 

from the following organizations: 

· U.S. Nuclear Regulatory Commission, 

· Division of Safety and Systems Analysis (NRR DSSA) 

· Region I 

· Region II 

· Probabilistic Safety Assessment Branch (NRR SPSB) 

· Inspection Program Branch (NRR IIPB) 

· Division of Systems Analysis and Regulatory Effectiveness (RES DSARE) 

· Nuclear Energy Institute (NEI) 

· Union of Concerned Scientists (UCS) 

The questions and responses received were based on the March 13, 2003, draft report and 

comments received in response to the public meeting held on July 24, 2003. Since then, many things have 

transpired that have influenced the BRIIE development. Some of these are associated with the Mitigating 

Systems Performance Index (MSPI), such as improvements to the standardized plant analysis risk (SPAR) 

models. Thus, the final report differs significantly from the draft report. The final report’s BRIIE was 

called industry integrated initiating event indicator (IIIEI) in the draft report, and the final report 

Equations (8-1) through (8-4) were numbered 2 through 5 in the draft report. In the discussion below, the 

comments and responses are rephrased to use the terminology of the final report. 

In addition to general comments, reviewers were asked to respond specifically to 11 questions listed 

in Section 6 of the draft report. Those questions are listed below, along with a summary of the reviewers’ 

responses. (It is important to note that many of the questions would not be asked today in light what we 

have learned since the draft report was written and issued for review.) 

1. Is Equation (8-4) rather than Equation (8-1) or (8-2) most appropriate for quantifying the BRIIE? 

Most of the reviewers felt that Equation (8-1) (Equation 2 in the March 13, 2003, draft report), 

plant-specific calculations rolled up into a plant-group [boiling water reactor (BWR) or 

pressurized water reactor (PWR)] or industry average, was the best way to quantify the BRIIE. 

However, sample results using each of the different quantification methods were not presented in 

the March 13, 2003, draft report. Since then, the historical BRIIE was calculated for fiscal year 

(FY) 1988 through FY 2005 (Appendix D) using all four equations. Results indicate that the 

BRIIE is much more sensitive to yearly performance if Equation (8-4) were used, based on plantgroup industry initiating event frequencies. (The same formula is obtained regardless of whether 

plant-specific Birnbaum importances or plant-group average importances are used.) Therefore, 

the final report recommends that this plant-group average approach be used. 
F-6

The reviewers were not provided the results in Appendix D, which compare the recommended 

Bayesian update process for determining the current initiating event frequency with the 

alternative maximum likelihood estimate (MLE, events divided by reactor critical years or rcry). 

Those sensitivities were performed after the draft report was reviewed. 

2. Is the method for determining baseline performance adequate (Sections 8.1 and 8.2)? 

One reviewer thought the justification for the baseline periods was not strong enough. Another 

reviewer thought the baseline periods were appropriate. The final report uses the methodology 

and baseline period results contained in NUREG/CR-6928. Those results are similar to the 

baseline performance estimates presented in the draft report, except for several initiating event 

categories that underwent a review of the individual event reports to ensure that only those events 

applicable to the standardized plant analysis risk (SPAR) modeling assumptions were included. 

Manual Chapter 0313 recommends using the baseline method and prediction limits. 

3. Is the proposed method for calculating current frequencies for the initiating events (Bayes update 

with 3 years of data in the draft report) appropriate (Section 4.3, Equation 4 and Section 8.7)? 

Two reviewers indicated the method was appropriate, one said more than 3 years of data should 

be used, and one said not enough information was presented to answer this question. The final 

report suggests using a Bayes update with 1 year of data. One year of data is now recommended 

because this eliminates any dependencies between years when calculating the Tier 1 and Tier 2 

performance indicators. Also, this approach maximizes the sensitivity of the indicators. 

4. Should core damage frequency (CDF) or ∆CDF be used as the measure for the BRIIE (Section 

4.3 and Appendix D)? 

Two reviewers favored ∆CDF, two were neutral, and one favored CDF. However, most agreed 

that ∆CDF may be more transparent. The final report presents BRIIE in terms of ∆CDF. The 

expert panel recommended that ∆CDF be used. 

5. Given the characteristics of the BRIIE and the simulation results (Section 9), what might be 

appropriate CDF and ∆CDF action thresholds? 

Most reviewers felt that development of the Tier 2 BRIIE thresholds would be a challenge. One 

reviewer thought the thresholds should be developed at lower plant-group levels, rather than at 

the industry-wide level. Finally, one reviewer did not think thresholds would be appropriate given 

the high plant-to-plant variability in Birnbaum importance measures. The final report 

recommends the Tier 2 threshold that was developed by an expert panel process (Section 8.6 and 

Appendix E), which was the process recommended in the March 13, 2003, draft report. 

6. Should the plant-group average Birnbaum importances be obtained from the SPAR models or 

from industry risk models? 

Three reviewers said use the SPAR models, while a fourth said use both. The final report uses 

Birnbaum importances that were obtained from updated SPAR models. 

7. If the Birnbaum importance measures are obtained from the industry, how will the differences 

between the two models (industry and SPAR) be addressed? 
F-7

Most reviewers felt the SPAR models should be used to determine Birnbaum importance 

measures. However, several thought that comparisons should be made with plant probabilistic 

risk assessment (PRA) results and differences addressed. The final report uses Birnbaum 

importances obtained from updated SPAR models. The updated SPAR models are expected to 

agree reasonably well with plant PRA models, especially at the plant-group level. 

8. How often should initiating event baseline performance be updated? 

One reviewer thought the baseline, if reflecting current industry performance that is deemed 

acceptable to NRC, should not need to be updated. Another thought that the baseline should be 

reviewed every 3 years. The final report does not address this implementation issue. However, a 

comprehensive review of industry performance (both initiating event frequencies and component 

unreliability and unavailability) should be performed periodically, and results from such a review 

could be used to determine whether the baselines still reflect industry performance. 

9. How often should the Birnbaum importance measures be updated? 

Most reviewers indicated that the Birnbaum importance measures should be kept up to date. One 

reviewer thought the Birnbaums should be reevaluated whenever plant conditions change 

significantly, while another suggested reviewing the Birnbaums every 3 years. The final report 

does not address this implementation issue. 

10. Is the treatment of uncertainties adequate in Section 9 (Appendix E of March 13, 2003, report)? 

Few reviewers commented on this question. Those who did indicated that the treatment of 

uncertainties is inadequate, and modeling uncertainties should be addressed. Additional 

uncertainty analyses performed since the draft report was issued for review indicate that 

incorporating uncertainty in the Birnbaums (representing plant-to-plant variation) did not 

significantly increase the uncertainty in the BRIIE. Also, because the BRIIE is meant to measure 

the risk impacts of changes in plant-group current initiating event frequencies, the plant-group 

average Birnbaums and the baseline initiating event frequencies are treated as known and 

constant. 

11. Should the thresholds be set so that no one event in a 3-year period would cause the threshold to 

be exceeded? 

Two reviewers felt a single event should not result in exceeding a threshold. Two others thought 

that selecting thresholds should not be influenced by whether a single event could reach or exceed 

the threshold. The Tier 1 prediction limits in Section 4, Table 4-4 are all two events or greater, so 

no single event can trigger a Tier 1 response. Also, the sensitivity results in Section 9 indicate that 

if a given initiating event is at its prediction limit, the BRIIE (assuming all other initiating events 

are at their baseline performance) will not exceed the threshold of 1.0E–05/rcry (per reactor) 

recommended by the expert panel. 

Specific comments from these groups and the comment resolutions are presented in Table F-1. Note 

that in some cases the comments are paraphrased and in many cases the listed comment was extracted 

from longer commentary. Also, comments from RES DSARE were generated after the public meeting on 

the proposed enhancements to the Industry Trends Program (ITP) coverage of the Initiating Events 
F-8

Cornerstone of Safety (held on July 30, 2003). The DSARE comments cover both the March 13, 2003, 

draft report and additional information presented at the public meeting. 

The comments that answer each question listed previously are grouped together, by question, at the 

end of the table. The comments on other topics are so diverse that no grouping by topic was helpful. 

Instead, those comments are presented in the order given by the reviewers.
F-9 

Table F-1. Comment resolution. 

Comment Source Comment Resolution 

General Comments 1. It would seem to me that tracking individual initiating events is likely to be more fruitful for internal agency uses. 

NRR DSSA We agree with the reviewer. That is the reason the Tier 1 activities under the enha

ITP (not documented in the March 13 draft report) were developed. They involve trending of individual initiating events and comparison of yearly industry performwith prediction limits. This is the method currently used in the ITP to assess short trends. It is documented in the NRC Inspection Manual Chapter 0313. 

General Comments 1. In other words, we would have to wait till we got a lot of events for this [integrated] indicator to change significantly. 

NRR DSSA Sensitivity analyses summarized in Section 9 of the final report indicate that the P

BRIIE is sensitive to loss of offsite power (LOOP), stuck open safety relief valve (SORV), loss of dc bus (LODC), and steam generator tube rupture (SGTR). The BBRIIE is most sensitive to LOOP. However, the reviewer is correct in that these initiating events would need to significantly exceed their prediction limits (Sectioorder for the BRIIE to exceed the threshold of 1.0E–05/rcry (or several different ewould need to occur). 

General Comments 1. If it [the integrated indicator] is used to provide information to Congress, it must be very carefully qualified as not representing the estimate of the risk (in terms of core damage), since a potentially significant contribution from external events, particularly fires, and from other modes of operation (low power and shutdown) are not addressed. 

NRR DSSA Agreed. The scope of the BRIIE is explicitly stated in the Executive Summary. Th

BRIIE estimates change in that portion of the CDF corresponding to an explicit likinds of initiating events. Words have been added to the final report to explain th

General Comments 2. Would it not be better to calculate the changes in CDF given all the changes, including those in the conditional core damage probabilities (CCDPs) [representing changes in mitigating system performance]? This is in fact a more natural match to the original Safety Goal policy statement and its subsidiary objective. 

NRR DSSA The ITP enhancement effort has focused first on the Initiating Events Cornerstone 

Safety. Follow-on work (not currently being pursued) may also address the MitigSystems Cornerstone of Safety. (The mitigating systems impact the CCDPs mentithe comment.) At present, there is no plan to integrate ITP performance indicatorthe cornerstone of safety level. However, the proposed higher-level integration coperformed if desired. 

General Comments 3. The report needs to be thoroughly edited. NRR DSSA The proposed technical editing was performed. General Comments 4. The report should be cleaned up with respect to its use of statistical jargon. Examples include “predictive distribution” and “p-value”. 

NRR DSSA More information is provided in the report concerning predictive distributions. A 

definition of p-value was added to the report. Statistically significant is defined in of the ‘p-value.’ A p-value is a probability indicating whether to accept or reject thypothesis that there is no trend in the data. P-values of less than or equal to 0.05 indicate that we are 95% confident that there is a trend in the data (reject the null hypothesis of no trend.) 

General Comments 5. As a note of clarification, Regulatory Guide 1.174 does not provide goals for ∆CDF, it provides 

guidelines. 

NRR DSSA The sentence in question was removed from the report. 
F-10 

Comment Source Comment Resolution 

General Comments 6. Given that the CDF is a linear function of the initiating event frequencies, the first part of Appendix A, through Equation A-3, is superfluous. 

NRR DSSA The March 2003 draft’s Appendix A was removed from the report. 

Recommendations. It appears to me that this indicator can, at best, only be a lagging indicator. A more pro-active approach is one based on identifying significant events on a plant-specific basis and then assessing whether those events have a potential for affecting other plants. 

NRR DSSA This is true. However, an indicator that lags by only one year still has worth. The 

existing historical downward trends are of interest, and any start of an upward trewould be of great interest. Also, preliminary results can be calculated as the data in, reducing the lag further. The suggested pro-active approach is not precluded buse of industry trending. 

Recommendations. This indicator should be exhaustively exercised to understand what sort of changes in trends would actually trip the threshold, or result in an increasing trend. This can then be played out against realistic expectations (admittedly very subjective) to test the value of the indicator. 

NRR DSSA The final report presents both Tier 1 prediction limits and a Tier 2 industry-level 

threshold for the BRIIE. The Tier 2 threshold was established by an expert panel process. Section 9 presents the impacts on the BRIIE Tier 2 if each initiating evenreaches its Tier 1 prediction limit. Simulations and sensitivity studies were conduevaluate the behavior of the BRIIE. These may not have been as exhaustive as suby the reviewer, but we feel they were adequate to provide the necessary insights BRIIE Tier 2 behavior. 

The document title should be changed to “Development of an Industry Aggregate Risk Indicator” 

Region II The title of the final report is “Baseline Risk Index for Initiating Events (BRIIE).” 

Because the revised report addresses both Tier 1 (individual initiating events) and (aggregation of initiators into a single risk-informed performance indicator) activthe suggested title change might appear too limited in scope. 

Instead of using average risk, don’t divide by the number of plants, and show the aggregate risk. 

Region II This is a question of taste, because either value can be calculated from the other. T

report uses risk per plant because most analysts are familiar with acceptable valuerisk per plant. 

The index inputs can be used to derive a companion indicator that reflects industry aggregate large early release frequency (LERF). Work on a similar approach is underway in the Significance Determination Process (SDP) Program to provide LERF output from the SDP sheets. 

Region II The scope of the effort was to focus on initiating events and CDF for at-power 

operation. Thus, LERF is not addressed. Follow-on work (not currently being purcould expand the BRIIE to include LERF. 

When determining the Birnbaums, be aware of potential truncation effects of the risk models. 

RII The Birnbaum importances were determined using a truncation level of 1.0E-12/r

the SPAR rev. 3.21 models. That truncation level is believed to be appropriate for determining initiating event Birnbaums. The Birnbaum importances, listed in AppC, are all at least five orders of magnitude larger that the truncation level. This shprovide sufficient accuracy. During the implementation phase of BRIIE, the Birnbwill be reevaluated. 

Some plants have fault trees, instead of a single basic event with a frequency for their initiating event in the model. For those calculated initiating events (IEs), Birnbaum does not work. Use CCDP/IE frequency. 

Region II None of the SPAR models at present use fault trees for initiating events. In the fu

fault trees may be used for certain IEs. We will evaluate this issue at that time. 
F-11 

Comment Source Comment Resolution 

For charts and tables that use simulation versus actual, indicate simulation on the chart, so it is not taken out of context accidentally by the reader. 

Region II This has been done in the final report in the figure titles.

In Appendix A, page 35, include some language explaining what Birnbaum represents in nonmathematical language. 

Region II The Birnbaum descriptions were modified to characterize them both in simple ter

by evaluation method within SAPHIRE. 

As information becomes available, modify the Birnbaums to include internal flooding and fire and external event importance. 

Region II The current scope of BRIIE focuses at power, internal events only This recommen

could be incorporated into follow-on efforts. Most SPAR models do not now inclexternal events, but fire, flood, and seismic events may be added within a few yea

Overall, a well written paper on a new approach to an industry health indicator. In this reviewer’s opinion this is well worth additional funding. 

Region II No comment necessary. 

General Comment 1. The subject report seems to be predominantly focused on developing the technical and mathematical structure for addressing the third objective [communicate industry-level information to Congress...] without establishing the first two objectives [collect and monitor industry-wide data..., and assess the safety significance and causes of any statistically significant adverse industry trends...]. 

NRR SPSB The March 13, 2003, draft report primarily addressed development of the BRIIE T

BRIIE results that reach or exceed a Tier 2 threshold will be reported to Congressrevised report provides equal coverage to the Tier 1 activities, monitoring individThe Tier 1 process clearly involves data collection, monitoring, and trending. 

General Comment 2. The subject report does not address limitations of this concept or limitations of the developed methodology and its associated uncertainties, which should be presented. These limitations include: 1. Not all initiating events are addressed 2. Not all of the initiating events considered in the report 

may be the result of deficient licensee performance (e.g., grid-related LOOP events) 

3. External events are excluded 4. Industry averages may conceal information about groups 

of plants 

5. The current SPAR models may not reflect changes and 

improvements in plant design and operational procedures 

6. No strong justification is provided for using predictive 

distributions rather than historical trending. 

NRR SPSB All these comments are addressed elsewhere in this table: 

1. See NRR DSSA General Comment 1 and NRR SPSB Implementation Comm2. See NRR SPSB Implementation Comment 2. 3. See NRR SPSB Implementation Comment 3. 4. See NRR SPSB Implementation Comment 4. 5. See NRR SPSB Implementation Comment 5. 6. See NRR SPSB Implementation Comment 6. 

General Comment 3 does not exist because it was omitted in the comments. 
F-12 

Comment Source Comment Resolution 

General Comment 4. Thresholds should be developed to be applicable on a lower industry level in addition to the overall industry-wide level (e.g., reactor type or vendor). 

NRR SPSB The expert panel considered this suggestion. They decided to recommend a single 

Tier 2 threshold of 1.0E–05/rcry for the Industry. This threshold is used only for reporting to Congress. The panel also recommended that BRIIE results for PWRs BWRs be presented, but that the threshold should not be applied to them. Follow-work (not currently being pursued) could be performed to identify the characteris

BRIIE results for subsets of plants, to be used internally by NRC. A sentence was to Section 1 to warn the reader that the BRIIE as presently characterized is not de

to detect trends or deviations from normal performance at plant group levels below used in the BRIIE (industry, BWR, and PWR). 

General Comment 5. Because of significantly decreasing initiating event frequencies over the years, it may not be appropriate to try to determine a predictive distribution on the industry-average based solely on the last few years. 

NRR SPSB The methods used to determine baseline periods (ending in calendar year 2002) fo

initiating events is believed to be appropriate. Manual Chapter 0313 outlines the uthe baseline method in the Industry Trends Program. It is also described in detail NUREG/CR-6928 and is focused on determining a baseline frequency that most appropriately represents current industry performance. (These baselines will be reperiodically to determine if industry performance has changed.) The predictive distribution is simulated from the statistics of this baseline frequency, addressing aleatory and epistemic uncertainty. We believe that the baseline frequency statistiappropriate for simulating the predictive distributions. 

Detailed Comment 1. Baseline periods short and long seem to be the same (1988 - 2001). (Executive summary, p. x in the draft report) 

NRR SPSB This was a typo in the draft report. That sentence is not in the final report. 

Detailed Comment 2. We recommend that the same data be used in all four equations for the BRIIE in order to determine the magnitude of the differences. This information can then be used to decide on which formula to use for estimating the BRIIE. (Section 3.3, p. 9 in the draft report) 

NRR SPSB In the final report, all four equations were used to determine the historical perform

of the BRIIE. The recommended choice is the industry average approach, Equatio4). This equation provides the most sensitivity to the occurrence of less frequent initiating events with high Birnbaum importances. 

Detailed Comment 3. In the equation defining the average industry Birnbaum, the summation should be about u and not i. (Section 3.3, p. 9 in the draft report) 

NRR SPSB This was corrected in the final report. 

Detailed Comment 4. Need to define the p-value. (Section 3.4, p. 11 in the draft report) 

NRR SPSB The final report uses initiating event baseline periods determined in NUREG/CR-

The baseline period determination and p-value information are discussed in detail document. The report no longer uses the term “p-value”. 

Implementation Comment 1. The report should address how new IEs of significance will be identified and integrated into the program, even if their current importance ranking is relatively low. 

NRR SPSB If new IEs of significance occur, the issue of including them in the SPAR models 

need to be addressed. When new IEs are added to the SPAR models, then they caevaluated as potential candidates for inclusion in the BRIIE, based on the steps usidentify the existing BRIIE Program IEs (as listed in Section 8.1). A paragraph wadded to explain the process by which new IEs might be added. 
F-13 

Comment Source Comment Resolution 

Implementation Comment 2. The report needs to address how it will address events and trends that are not directly related to the performance or authority of the licensee (e.g., grid-related LOOP events). 

NRR SPSB As presently described, the Tier 1 and Tier 2 performance indicators do not distin

events not directly related to licensee performance. For example, grid-related LOOevents are not distinguished from other causes of LOOP. However, if the predictilimits or the threshold are exceeded, then follow-on evaluations would identify w

the increased number of events was the result of events not directly related to liceperformance. Examples of potential NRC responses are presented in Section 7 of final report. 

Implementation Comment 3. The report does not address external events, which for some plants are the overwhelmingly dominant contributor to plant risk. 

NRR SPSB The BRIIE was developed as an at power Level 1 indicator. This could be addres

potential follow-on work (not currently being pursued). Most of the SPAR modelnot now include external events. Fire, flood, and seismic event risk models are bedeveloped. Fires and floods may occur frequently enough to allow trending at the industry level. 

Implementation Comment 4. The integrated data (industry averages) may conceal information about groups of plants contribution to changes in the BRIIE. 

NRR SPSB As presently formulated, if the Tier 1 prediction limits or the Tier 2 threshold are 

exceeded, then no further examination is required. If a subset of the industry (at a 

lower than BWRs and PWRs) exhibits a change in performance, but the rest of thindustry masks this change, then the industry-wide performance indicators will inno significant industry-wide change in performance. This balancing of performanbelieved to be appropriate at the industry level. Follow-on work (not currently bepursued) could investigate whether performance indicators should also be formula lower, group-of-plants level. If a Tier 1 prediction limit or the Tier 2 threshold iexceeded, analysis of the data, based on groups of plants, could be performed to determine the causes. 

Implementation Comment 4. If a good or bad performer is shut down for a long period of time (e.g., six months or a year) to address an issue, will the calculated industry average consider the repercussions of the issue on the BRIIE results? 

NRR SPSB Periods of shutdown would be automatically incorporated in the calculation of the 

industry-wide (or BWR or PWR) critical years of operation for the fiscal year in question and fewer initiating events from the plant in question. Hypothetically, if poorly performing plant had been the source for several less frequent initiating evoccurrences (e.g., loss of DC bus) that substantially increased the BRIIE, and that 

was then shut down for an extended length of time, the BRIIE would show an improvement. (In this case, the improvement in the BRIIE would have been the rethe poorly performing plant being shut down, rather than the result of improving iperformance. If that plant then came back on line without an improvement in 

performance, then the BRIIE would show a subsequent degradation.) Issues such will be addressed as they arise during implementation of the BRIIE. 

Implementation Comment 5. The report needs to present how PRA quality, SPAR modeling conservatisms, and uncertainties in results and model/plant fidelity will be addressed within the program to ensure that the results are representative of the existing plant designs and conditions. 

NRR SPSB Work associated with implementation of the Mitigating Systems Performance Ind

(MSPI) Program included the issue of PRA quality and SPAR modeling versus lirisk models. That led to a comprehensive effort to update the SPAR models based cut set level comparison with licensee risk models (with this comparison leading tSPAR model and licensee model changes). During the implementation phase of Bthe Birnbaums will be reevaluated using the most up-to-date SPAR models and parameter estimates. 
F-14 

Comment Source Comment Resolution 

Implementation Comment 6. There is not a strong justification provided for using predictive analyses with simulations instead of using historical performance, and the uncertainties in the parameters used in these analyses are not presented. 

NRR SPSB The predictive distribution gives a comparison of current performance with baseli

performance, accounting correctly for uncertainties. Historical performance was considered a valuable input to the expert panel process to determine thresholds foBRIIE. However, the approach of simulating a predictive distribution also providvaluable insight into the characteristics of the BRIIE. (Inputs for these simulationsummarized in Section 9 of the final report. Uncertainties in these inputs are also discussed.) Both types of information were provided to the expert panel. 

Implementation Comment 6. It is not even justified why the baseline period is what it is for each IE. 

NRR SPSB The draft report did not present sufficient information in this area. Since issuing t

report in 2003, the NRC is using the baseline concept. This concept is documenteITP Inspection Manual Chapter 0313. BRIIE uses the initiating event baseline perand frequencies presented in NUREG/CR-6928. Those baselines have been reviewNRC. 

Implementation Comment 6. The trending should also be both long term (over the past 15 years) and short term (over the past few years). 

NRR SPSB In the final report, Appendix A presents initiating event frequency data over the p

FY 1988 through FY 2005, even though the baseline periods chosen may be much shorter. These plots show both the long-term and short-term trends (or lack thereoITP manual chapter suggests using a 10-year window. If the ITP chooses to add BIEs and BRIIE to its trending effort, a 10-year window may be appropriate for theadditional performance indicators. Words were added to Section 1 to indicate that BRIIE is not focused on trending, but on comparing yearly results against either prediction limits or a threshold. The ITP will make a decision whether to also trenBRIIE IEs and the BRIIE Tier 2 results. 

Implementation Comment 6. A slight increase in some IEs over the next couple of years should not be considered a negative trend, as it may just be that the initiating event frequency is leveling out. The criteria for determining and addressing these conditions need to be presented. 

NRR SPSB Trending methods already used by the ITP, and documented in the ITP manual ch

can be used to determine whether adverse trends exist in the individual initiating However, BRIIE focuses on whether the individual initiating events reach their prediction limits within a year and whether the BRIIE reaches its risk threshold. 

Concept Comment 1. Do we want to include data that are not related to a licensee performance deficiency? 

NRR IIPB Yearly initiating event data collection will not include a determination whether ea

event was related to licensee performance. However, if Tier 1 or Tier 2 thresholds reached, then the follow-on review should identify which events were outside the 

of the licensees. 

Concept Comment 1. Where does LERF factor in this high level look at the industry’s performance? 

NRR IIPB At present, LERF is not considered in BRIIE. [The same is true of the existing IT

performance indicators, except perhaps for more recent Accident Sequence Precu(ASP) evaluations.] However, LERF could be considered in follow-on work (not currently being pursued). The final report states that the BRIIE as outlined does ninclude LERF considerations. 
F-15 

Comment Source Comment Resolution 

Relationship Comment 1. The relationship is clear between the ROP and the BRIIE, but it is not as clear between the ITP and the BRIIE. 

NRR IIPB BRIIE is part of the ITP. The Tier 1 portion of BRIIE is perhaps more typical of t

existing ITP (monitoring of individual performance indicators using performanceprediction limits), while the BRIIE Tier 2 is probably more like the ROP MitigatiSystems Performance Index (MSPI). Sections 1 through 4 more clearly discuss thexisting ITP and how BRIIE fits into the ITP. The BRIIE development efforts havreported in the annual ITP Commission paper for the last several years. 

Summary Comment 1. The proposed indicator is inconsistent with the guidelines [in the Commission’s White Paper on “Risk-Informed and Performance-Based Regulation”] because no threshold has been proposed that demonstrates the ability to preserve safety margins, while providing time to take corrective action, if the threshold is tripped. 

RES DSARE The ITP is intended to complement the present ROP. Because it looks at historica

industry-wide data, its primary role is not to detect specific problems, but rather to provide assurance that the industry as a whole remains safe. BRIIE with its Tier 1 prediction limits is designed to detect performance degradation at the industry (or or PWR) level. The prediction limits are tripped significantly before the BRIIE Ti∆CDF threshold of 1.0E–05/rcry is reached. Therefore, to a certain extent the Tie

portion of BRIIE provides time to take corrective action before the Tier 2 threshoreached (assuming a gradual degradation rather than an abrupt, large degradation 

occurs). See Section 7. 

Summary Comment 2. It appears that uncertainty is significantly underestimated in the report [compared with Regulatory Guide (RG) 1.174 (Ref. F-3) direction to include both aleatory, epistemic, and completeness uncertainties]. 

RES DSARE The final report presents an uncertainty analysis result in Section 9 that addresses 

the uncertainty in the baseline initiating event frequencies. That analysis is not me

indicate the overall uncertainty in the BRIIE Tier 2 CDF. The BRIIE Tier 2 treats Birnbaum importance measures and baseline initiating event frequencies as constTherefore, including additional uncertainties would be misleading in terms of the The uncertainty analysis is presented mainly for comparisons purposes with the BTier 2 predictive distribution, which is believed to be the most representative of fuBRIIE Tier 2 results (assuming future industry performance remains at its baselinperformance). 

Summary Comment 3. The subject report does not clearly indicate how the safety goal policy (as amended) will apply for the performance indicator. 

RES DSARE The safety goals and other guidelines (e.g., RG 1.174) were used by the expert pa

guidance in setting the BRIIE Tier 2 threshold of 1.0E–05/rcry. 

General Comment 1. It is not clear to all concerned that what is being monitored is the variability of a statistic and not safety of the operating fleet of power reactors. 

RES DSARE In the final report, the BRIIE Tier 1 comparison with performance-based predictio

limits associated with the individual initiating events monitors variability. The TiBRIIE then interprets the Tier 1 results in terms of safety of the operating fleet (wrespect to the Initiating Events Cornerstone of Safety). 

General Comment 2. It should be recognized that this hierarchical structure [aggregation of plant-level information into industry-wide results] is quite different from the hierarchy of the regulatory framework associated with the ROP, which dis-aggregates the safety mission of the NRC into the cornerstones. 

RES DSARE The ITP aggregates plant-level information into industry-wide results. The purpo

the BRIIE is to address only the Initiating Events Cornerstone of Safety. Aggregaabove this cornerstone of safety is not being addressed at this time. 
F-16 

Comment Source Comment Resolution 

General Comment 3. It is not clear how inspection results will be factored into the ITP. 

RES DSARE As structured, BRIIE does not directly include inspection results. However, if Tie

prediction limits or the Tier 2 ∆CDF threshold are reached or exceeded, then 

engineering analyses will be performed to attempt to determine the causes. These engineering analyses would involve applicable inspection results. The ITP is to complement the ROP, not just summarize data from the ROP. 

Summary Comment 1. We believe the concept of the BRIIE—assessing initiating events based on their risk worth (or Birnbaum in the draft report)—is appropriate and believe that this approach should be applied in developing a plant specific performance indicator to replace the current “Scrams with Loss of Normal Heat Removal.” 

NEI The ITP work does not consider plant-specific risk. The suggested work is possib

it is not in the scope of this project. 

Summary Comment 2. We believe the use of this approach [BRIIE approach] for assessing industry performance is not appropriate. It is not clear what the BRIIE result really represents and it is clearly not an “actionable” measure. 

NEI The BRIIE is like a report card for the industry as a whole. The resulting action, i

required, would be to tell Congress that last year the industry had a worse report cthan we wanted. Any corrective or preventive actions would be based not on the Bbut on engineering analyses or on plant-specific or event-specific investigations. 

Summary Comment 3. Thus the BRIIE seems to merely add an additional level of assessment which appears to be of relatively low value, and may be difficult to explain to stakeholders. We believe that the current body of indicators and inspection findings provide a very robust measure of industry performance. 

NEI The Tier 1 results are more useful than the BRIIE Tier 2, in terms of discovering, 

understanding, and correcting problems. However, some tool is needed for determwhen the NRC tells Congress that problems are risk significant—a tool that encompasses the industry and that does not raise many false alarms. For this, BRIseems appropriate. 

Summary Comments 1. We do not believe that the BRIIE will add much value to the Industry Trends Program for the simple reason that it fails to monitor sufficiently meaningful data. Its focus seems almost completely decoupled from the reality of the NRC’s oversight program and thus, if used, will not assist the agency fulfill its mission. Since September 1984, 24 nuclear power reactors have been shut down for longer than a year. [A list of those reactors and their shutdown periods was provided.] These outages are prima facie evidence of unacceptable safety levels—it took extensive efforts lasting over a year to restore the safety levels at these reactors. However, none of these extended shutdowns were precipitated by the occurrence of one of the initiating events to be covered by the BRIIE. Therefore, the BRIIE would not have enabled the agency to avoid any of these costly safety shutdowns. 

UCS The ITP is intended to complement the present ROP. Because it looks at historica

industry-wide data, its primary role is not to detect specific problems, but rather to provide assurance that the industry as a whole remains safe, and (less frequently) provide an indication of any adverse trends that are starting to develop. Since the of the 2003 draft report, the Tier 1 concept has been further developed. The Tier indicators are likely to discover such adverse trends before they affect the BRIIE strongly. Both Tier 1 and Tier 2 use industry-wide data, and therefore are less focon particular problems than are existing oversight programs. 
F-17 

Comment Source Comment Resolution 

Summary Comments 2. Setting aside, for the moment, the fact that the BRIIE is monitoring the wrong things, we believe it is monitoring the wrong things in the wrong way. It attempts to allow comparisons between events using importance factors. That might work on a plant-specific basis, but it cannot work on a reactor type basis (e.g., BWR or PWR). This “one size fits all” approach is wrong because it will downplay certain events at some plants and overplay those events elsewhere. 

UCS The objection seems to be that the industry mean is presented instead of the indiv

plant-specific ∆CDF values. The BRIIE Tier 2 was calculated using several appro

in the development stage. One approach was to calculate the BRIIE Tier 2 for eac

individual plant and then average those results. That approach was not as sensitivproposed approach, in terms of predicting the largest variations from the baselineTherefore, the plant-type average approach was selected. The comment is correct this approach has the potential to downplay certain events and overplay others, buleast for the historical data analyzed (covering 1988–2005) the plant-type average approach resulted in higher ∆CDF peaks that the plant-specific approach. 

Summary Comments 3. In addition, this indicator is way too convoluted for public consumption. 

UCS The BRIIE Tier 2 indicator is similar to the MSPI, which has been implemented a

enhancement to the ROP. In fact, it is simpler than the MSPI because it does not hfront stops and backstops. The ITP Commission paper as provided information an

summaries of the BRIIE develop efforts. 

Question 1: Is Equation (8-4) rather than Equation (8-1) or (8-2) most appropriate for quantifying the BRIIE?I cannot see why the Equation (8-1) would not be used rather than Equation (8-2) or (8-4). 

NRR DSSA 

The sensitivity to the method of calculating the baseline and annual values of the BRIIE needs to be investigated. It may be more appropriate to use the plant-specific method [Equation (8-1)], to better account for the differences between plants. Using the industry mean values could minimize the effects of plants that may be in the tails of the plant by plant initiating event distributions of frequency or Birnbaum importance measure. 

Region I 

All of the various equations have meaning. Equation (8-1) would be closest to true industry aggregate risk. Equation (8-2) is an indicator that should represent the average state of industry design, and the adequacy of the average design to respond to the challenges of the particular location. Equation (8-3) represents what we currently see nationwide for rare events’ treatment in PRAs. Finally, Equation (8-4) can be useful as an indicator, but evaluations need to be made comparing the results with the output from Equation (8-1). 

Region II 

The average values may not identify the importance of the extremes. Consider using more plant specific data and let the outliers be revealed. 

NRR IIPB 

The basis for responding to this question has not been sufficiently established in the report. 

RES DSARE 

The BRIIE Tier 2 was calculated using several approaches in the development staOne approach was to calculate the BRIIE Tier 2 for each individual plant and then average those results [Equation (8-1) approach]. That approach was not as sensitithe proposed approach [Equation 8-4)], in terms of predicting the largest variationthe baselines. Therefore, the plant-type average approach [Equation (8-4)] was seSee Appendix D. 
F-18 

Comment Source Comment Resolution 

Equation (8-1) is sufficient for quantifying the BRIIE. More complex formulations would have limited value for the generation of an industry average performance indicator. 

NEI 

Question 2. Is the method for determining baseline performance adequate (Section 8.2)?However, it would be preferable to see the [baseline] period hypothesized on the basis of stability of industry/regulatory practice, with statistical tests used to test the validity rather than have the statistical tests drive the approach without any discussion of reasons why the observed effects are to be expected. 

NRR DSSA What is suggested is probably the ideal way to identify appropriate baseline perio

determining up-to-date baseline frequencies. However, the review of industry/regpractices and their impact on initiating event frequency trends is beyond the scope present effort. The statistical approach to identifying appropriate baseline periods believed to be adequate for our purposes. The baseline approach is discussed in thManual Chapter 0313. 

In calculating the annual BRIIE, using the individual SPAR Birnbaum importance measure at the time of each initiating event may more closely represent the actual consequence of events and provide more information relative to actual industry performance trends. 

Region I What is suggested would more accurately reflect the plant-specific risk of individ

initiating event occurrences. However, if SPAR Birnbaums were to be re-evaluatereflect plant-specific conditions at the time of each initiating event occurrence, th

would require on average over 100 such evaluations each year. One goal of BRIIE use data already being collected each year with minimal extra effort. Therefore, th

report suggests that a set of SPAR plant-specific baseline Birnbaums be establishthen used year after year. These plant-specific Birnbaum importances end up influthe BRIIE only through the industry average, as seen by comparing Equations (8-(8-4). (The implementation issue concerning possible periodic updates to these baBirnbaums is not addressed in the final report.) 

Coincide the minimum period of data collection with refueling outages. Also, 4 years (subjectively) as a minimum may be premature for Bayesian updating. 

NRR IIPB This response appears to be addressing Question 3, rather than Question 2. If not 

this response addresses the minimum number of years to use to establish a baselinperiod), then the final report uses 5 years as a minimum baseline period. Longer pare used whenever the frequency appears constant during that longer period. 

The method is not adequate because the focus appears to be on assuring stability of the indicator when the underlying data are (or could be) highly variable, rather than focusing on the safety significance. 

RES DSARE This comment appears to be addressing Question 3 rather than Question 2. 

The method described appears to satisfy the desirable characteristics outlined in Section 3.4 (draft report). The decision rules in section 3.4 do take some time to understand and may not be transparent enough to allow them to be easily understood by the public. 

NEI The baseline periods and baseline frequency distributions in the final repot are tho

presented in NUREG/CR-6928. A summary of the methods used in that report is presented in the final report. 

Question 3: Is the proposed method for calculating current frequencies for the initiating events (Bayes update with 3 years of data) appropriate?
F-19 

Comment Source Comment Resolution 

The method for calculating the initiating event frequencies is as good as any, particularly for the rare events, e.g., loss of instrument air. 

NRR DSSA 

If all the underlying assumptions and premises are considered acceptable, then this method appears to be acceptable. 

RES DSARE 

There were not enough sensitivity studies performed to determine the answer to this question. 

NEI 

The final report suggests that a Bayes update with 1 year of industry data be used calculate initiating event current frequencies. One year, rather than 2 or 3 years, wchosen. The use of a 1-year period eliminates data dependencies between yearly rand provides increased sensitivity to industry performance during the year in quesNote that the baseline is used to estimate the mean industry initiating event frequeThe use of 1 year is for the actual initiating event contribution to BRIIE for that y

Three years does not appear to represent an adequate data collection period, especially considering the cyclic nature of performance that some licenses have demonstrated. 

NRR IIPB See the above response, which applies here as well. In addition, if a plant exhibits 

performance, then the rolling, 1-year data collection period would reflect poor performance during the “bad” portion of the cycle and good performance during t“good” portion of the cycle. Since a yearly measure of performance is desired, thicharacteristic is appropriate. The mean industry initiating event frequency is estimusing the baseline method and includes several years. 

Question 4: Should CDF or ∆CDF be used as the measure for the BRIIE?I’m not sure it matters whether CDF or ∆CDF is used, though the variation from year to year may show up more clearly in the ∆CDF plot. 

NRR DSSA 

Use delta CDF because it is consistent with the ROP. NRR IIPB The delta CDF is the preferred metric because it is consistent with the guidance in RG 1.174. Besides, delta CDF is a more transparent metric than CDF. 

RES DSARE 

It does not appear to matter if CDF or delta CDF is used. Region I  CDF should be used for the measure for the BRIIE. NEI 

The final report presents BRIIE Tier 2 results in terms of ∆CDF, which was the 

recommendation of the expert panel. Both approaches are discussed in the report, either approach could have been used. 

Question 5: Given the characteristics of the BRIIE and the simulation results, what might be appropriate CDF and ∆CDF action thresholds? It’s not clear how you would establish action thresholds. If they are to be established by an expert panel, one useful source of information would be a set of sensitivity studies that demonstrate how the indicator would change given postulated occurrences of initiating events of various types. 

NRR DSSA 

The development of a threshold will be a challenge. Region I 

The final report used an expert panel process for the establishment of the BRIIE T∆CDF threshold of 1.0E–05/rcry applied at the industry level. The expert panel wsupplied sensitivity study and uncertainty results to aid in its determinations. This information is contained in the report. 
F-20 

Comment Source Comment Resolution 

Proposed action thresholds should include site specific Birnbaum times site specific initiating event frequency compared to the industry average result as a measure. This will measure the impact of high Birnbaum and high initiating event at the same plant, and its impact on industry risk. 

Region II The sensitivity results comparing a plant-specific approach [Equation (8-1)] to the 

recommended plant-group average approach [Equation (8-4)] were supplied to thexpert panel. 

Given the extremely high variability in the Birnbaum importance measure, the concept of an action threshold is not applicable. 

NRR DSARE The expert panel felt that a threshold of 1.0E–05/rcry applied at the industry level 

appropriate for reporting to Congress. 

It is difficult to discuss industry wide action thresholds without any context that defines what the actions may be. This question cannot be adequately addressed without this context. A simplistic response is that the surrogate safety goal of 1.0E–04 is the appropriate action limit. It is generally recognized that this goal is a conservative reflection of the actual safety goals. 

NEI Actions to be taken given that BRIIE Tier 1 prediction limits are exceeded or BRI

2 threshold is exceeded are summarized in Section 7. Note that if the BRIIE Tier threshold is exceeded, the NRC will already have taken some type of action. 

No comment. NRR IIPB No response necessary. Question 6: Should the industry-average Birnbaum importances be obtained from the SPAR models or from industry risk models?The Birnbaum importance measures should be obtained from the SPAR3 models. The SPAR models are performed to a consistent standard even if they are less detailed than the licensee PRAs. Licensee PRAs vary in the level of detail and style of modeling and would introduce an additional source of variability that might mask the significance of some initiating events. 

NRR DSSA 

 Industry average Birnbaum importances should be obtained from both the SPAR models and industry risk models. The results should be compared and then a decision should be made as to which set to use. 

NRR IIPB 

The final report uses Birnbaum importance measures obtained from the SPAR Remodels. Before the BRIIE is officially implemented, final Birnbaum values shoulobtained from the most up-to-date SPAR models. As part of the SPAR model proand the implementation of the MSPI, importance measures from the SPAR modelbeing compared to corresponding importance measures from licensee models. As the implementation phase, it may be possible to perform comparisons using a subthe plant risk models. 
F-21 

Comment Source Comment Resolution 

The industry average Birnbaum importance values should be obtained from the SPAR models. However, benchmarks performed for the MSPI pilot plants demonstrated that additional benchmarks are necessary before the SPAR models could produce Birnbaum importance values that are representative of the actual plant configurations. 

NEI 

Section 6 Questions 6, 7, and 9. The use of the SPAR models is preferred, since this will be an NRC indicator. As time passes there will be a need to ensure that the SPAR models remain consistent with changes in plant design and operation. 

Region I 

It is premature to obtain an answer to this question. It would appear that the variations within the industry risk models are likely to be so large that treating them as a monolithic group may not be appropriate. 

RES DSARE The expert panel felt that a BRIIE tier 2 threshold of 1.0E–05/rcry applied at the i

level was appropriate for the report to Congress. 

Question 7: If the Birnbaum importance measures are obtained from the industry, how will the differences between the two models (industry and SPAR) be aIn light of the answer to the previous question, this question is moot. 

NRR DSSA 

 See the response to Question 6. That is, use both and compare them to ensure consistency. 

NRR IIPB 

The use of the SPAR models is preferred, since this will be an NRC indicator. As time passes there will be a need to ensure that the SPAR models remain consistent with changes in plant design and operation. . 

Region I 

No further response necessary. 

Differences in the models must be addressed for the indicator to have any integrity no matter which set of models is used to supply the importance measures. 

NEI 

See the response to Question 6. That is, variations within industry risk models mean that treating the industry as a whole is probably inappropriate. 

RES DSARE 

Differences in the models were addressed as part of the MSPI implementation proThat process led to changes in both the licensee and SPAR models. 

Question 8: How often should initiating event baseline performance be updated?It’s not clear that the baseline performance would ever need to be updated if we, NRC, believe that the current performance is acceptable. 

NRR DSSA The final report does not provide recommendations concerning whether to period

update the initiating event baseline frequencies. This question is more of an implementation issue rather than a basis issue. One possibility might be to reinves
F-22 

Comment Source Comment Resolution 

Once the final method of calculating the BRIIE baseline, annual value, and the threshold is developed, a sensitivity study should be conducted on the methods of updating the baseline. 

Region I 

The baselines should be updated every two refueling outage cycles (36 months). 

NRR IIPB 

It would appear that having a predictable period, such as a year, should be preferable to having any other criterion that may increase the variability. 

RES DSARE 

If CDF is used as the metric (as opposed to ∆CDF), it is not clear what the purpose of the baseline is. The answer to this question should be reflective of the purpose of the baseline. If the baseline is used to provide the early warning threshold, then it should be updated as often as necessary to be reflective of current industry performance. 

NEI 

industry performance every 5 years to decide if new baselines are appropriate. 

Question 9: How often should the Birnbaum importance measures be updated? If the Birnbaum measures are obtained from the SPAR models, will they be updated? What is the plan for updating the SPAR models? How would trending be performed if Birnbaum measures are changed? 

NRR DSSA 

The Birnbaums should be updated coincidently with the baselines. 

NRR IIPB 

See the response to Question 8, i.e. keep the update times predictable. 

NRR DSARE 

When the plant designs and operating methods change, on a continuous basis. 

NEI 

The use of the SPAR models is preferred, since this will be an NRC indicator. As time passes there will be a need to ensure that the SPAR models remain consistent with changes in plant design and operation. 

Region I 

The final report uses Birnbaum importance measures obtained from the SPAR moBefore the official implementation of the BRIIE, final Birnbaum values should be obtained from the most up-to-date SPAR models. The final report does not make recommendation about updating the Birnbaum importances. This is more of an implementation issue than a basis issue. The SPAR models are being updated and improved in a systematic and controlled manner. One possibility might be to revisit the SPAR Birnbaums every 5 years to whether the baseline Birnbaums should be changed. Another possibility would be review the baseline Birnbaums at the same time that the baseline initiating event frequencies are reviewed. If changes are made, then succeeding yearly BRIIE resu

would use the updated baseline Birnbaums, and the yearly BRIIE results that had previously calculated (using an older set of baseline Birnbaums) would remain unchanged. 

Question 10: Is the treatment of uncertainties adequate?
F-23 

Comment Source Comment Resolution 

In this report, uncertainty is fed in through the generation of the various distributions. What will be more interesting is how the authors propose to use this information to both set, and provide the approach for comparison with, the thresholds. 

NRR DSSA 

Once a final method of determining the baseline and the annual BRIIE is developed, the related uncertainties in the SPAR models and the BRIIE calculations need to be reviewed and understood as the threshold is developed. 

Region I 

The expert panel was provided the uncertainty and sensitivity information presentthe final report. The expert panel used these insights, along with other informationhelp them set the Tier 2 threshold. 

Because only uncertainties in the initiating event frequencies have been included, the treatment of uncertainties is inadequate. 

RES DSARE 

No. Modeling uncertainties must be addressed. NEI 

In the final report, the BRIIE Tier 2 is regarded as an index, similar to the MSPI. purpose is to detect degradation in industry initiating event performance and charthe risk significance in terms of ∆CDF. Therefore, we ask what varies from year t

and consider only those uncertainties. The baseline data, the Birnbaum importancthe SPAR models are (virtually) constant from year to year, and therefore regardeknown. When examining whether a high value of the BRIIE Tier 2 is risk significonly relevant uncertainty is the uncertainty in the corresponding initiating event 

frequencies, because the initiating event frequencies are the only quantities that chin the calculation of BRIIE from year to year. 

 No comment. NRR IIPB No response necessary Question 11: Should the thresholds be set so that no one event in a three year period would cause the threshold to be exceeded?It makes no sense for one event at one plant to trip an industry wide threshold. It makes more sense to investigate the one event first to see if it reflects a plant specific problem or one that could have generic implications. 

NRR DSSA 

No. Any event of sufficient importance (i.e., delta CDF of 1E3) should trigger the threshold. NRR IIPB Such an artificial constraint would do disservice to the cause of regulation and damage to public confidence. 

RES DSARE 

 If one event in a 3-year period could cause a threshold to be exceeded, then the method is fundamentally flawed. It would indicate the 3-year period [for data collection] is not sufficient. 

NEI 

Section 6 Questions 10 and 11. Once a final method of determining the baseline and the annual BRIIE is developed, the related uncertainties in the SPAR models and the BRIIE calculations need to be reviewed and understood as the threshold is developed. 

Region I 

The sensitivity studies in the final report indicate that no single event can trip eithTier 1 prediction limits or the Tier 2 ∆CDF threshold. Therefore, no artificial rest

needed. 
F-24