(Information) 

April 10, 2014 SECY-14-0042

FOR: The Commissioners 

FROM: Eric J. Leeds, Director 

Office of Nuclear Reactor Regulation 

SUBJECT: FISCAL YEAR 2013 RESULTS OF THE INDUSTRY TRENDS 

PROGRAM FOR OPERATING POWER REACTORS 

PURPOSE: 

The purpose of this paper is to inform the Commission of the results of the U.S. Nuclear 

Regulatory Commission’s (NRC’s) Industry Trends Program (ITP) for fiscal year (FY) 2013. 

This paper also provides a response to a staff requirements memorandum (SRM) issued by the 

Commission on June 13, 2013, (Agencywide Documents Access and Management System 

(ADAMS) Accession No. ML13164A337) in response to the May 29, 2013, “Briefing on the 

results of the Agency Action Review Meeting (AARM)” (ADAMS Accession No. ML13155A441). 

This paper does not address any new resource implications. 

BACKGROUND: 

The NRC staff implemented the ITP in 2001 to monitor for adverse trends in safety performance 

based on industry-level indicators. After the NRC assesses statistically significant adverse 

trends for safety significance, it responds, as necessary, to any identified safety issues, 

including adjustments to the inspection and licensing programs. One important output of the 

ITP is the annual agency performance measures reported to Congress on the number of 

statistically significant adverse industry trends in safety performance. This outcome measure is 

part of the NRC Performance and Accountability Report (PAR). In addition, the NRC annually 

CONTACT: Andrew Waugh, NRR/DIRS 

 301-415-5601 

 Luis A. Cruz, NRR/DIRS 

 301-415-3982

POLICY ISSUE
The Commissioners - 2 - 

reviews the results of the ITP and any actions taken or planned during the AARM. The NRC 

reports the findings of this review to the Commission. This paper is the 14th annual report to 

the Commission on the ITP. 

NRC Inspection Manual Chapter (IMC) 0313, “Industry Trends Program,” dated May 29, 2008 

(ADAMS Accession No. ML080860540) contains details of the ITP, including definitions of 

monitored indicators and program descriptions. 

DISCUSSION: 

Using the ITP, the staff monitors industry safety performance to identify and address any 

statistically significant adverse industry trends. The ITP indicators are based on the best 

available data. A statistically significant adverse trend exists if the slope of the regression line 

fitted to the long-term indicator data has a positive value and the fit of the regression line is 

statistically significant. 

The ITP also uses precursor events identified by the Accident Sequence Precursor (ASP) 

Program to assess industry performance. The staff analyzes the occurrence rate of precursors 

to determine if an adverse trend exists. The staff uses results from the ASP program as one of 

the agency’s safety-goal performance measures reported in the NRC Performance and 

Accountability Report. 

In addition to the long-term indicators, the ITP uses a statistical approach based on prediction 

limits to identify potential short-term year-to-year emergent issues before they become 

long-term trends. The short-term prediction limits are determined from a predictive distribution 

derived using information from an established baseline period. These prediction limits are 

reevaluated each year. 

The ITP complements the Reactor Oversight Process (ROP). The ITP monitors industry level 

performance, whereas the ROP provides oversight to individual plants commensurate with their 

safety performance. 

The Office of Nuclear Regulatory Research (RES) provides indirect support to the ITP in the 

areas of operating experience data and models that are developed and budgeted under other 

RES programs, such as the Standardized Plant Analysis Risk Model Development Program, the 

ASP Program, and the Reactor Operating Experience Data Collection and Analysis Program. 

The ITP uses the results of RES work in the ASP Program to assess industry performance, 

although the funding and performance of RES work are separate from the ITP. 

FY 2013 LONG-TERM INDUSTRY TRENDS: 

Based on the ITP indicators and the ASP program results, the staff did not identify any 

statistically significant adverse trends in industry safety performance. The graphs in 

Enclosure 1 show the long-term ITP indicator trends and the ASP data. The staff removed the 

trendlines from the graphs in Enclosure 1 that did not have a statistically significant trend. The 

staff evaluated both linear and exponential trendlines for each set of data and used the trendline 

showing the highest degree of statistical significance. The staff observed that a number of 

indicators displayed a statistically significant improving trend. The staff considers this 
The Commissioners - 3 - 

improvement of industrywide performance to be reflective of a number of industry initiatives as 

well as the ROP’s effectiveness in inspecting, measuring, assessing, and responding to plant 

performance. 

The ASP Program considers an event with a conditional core-damage probability (CCDP) or an 

increase in core-damage probability (ΔCDP) greater than or equal to 1×10-6 to be a precursor. 

The RES staff evaluated precursor data from FY 2003 through FY 2012 and identified no 

statistically significant trends for the occurrence rate of all precursors during that period 

(Figure 14 of Enclosure 1). Additional information can be found in Section 4.1 of Enclosure 1 of 

SECY-13-0107, “Status of the Accident Sequence Precursor Program and the Standardized 

Plant Analysis Risk Models,” dated October 4, 2013 (ADAMS Accession No. ML13232A092). 

The evaluation of FY 2013 precursor data is still ongoing and not included in this paper. 

The ASP Program also provides the basis for the safety performance measure of zero 

significant precursors of a nuclear reactor accident. This is one measure that is associated with 

the safety goal in the NRC’s annual PAR (ADAMS Accession No. ML13350A620). A significant

precursor is an event that has a probability of at least 1 in 1,000 (i.e., CCDP or ΔCDP greater 

than or equal to 1×10-3) of leading to a reactor accident. The RES staff is completing 

preliminary analyses on precursor events and anticipates no significant precursors in FY 2013. 

The RES staff identified five precursors that might meet the criteria for significant events (i.e., a 

significant event in the ITP is one that has a CCDP or ΔCDP greater than or equal to 1×10-5). 

The RES staff will complete its evaluation of FY 2013 precursors to obtain final results. The 

staff will update the FY 2013 precursor data and will report any changes to the ITP analysis in a 

memorandum to the Commission after receiving the final ASP results from RES. The staff 

recognizes that a statistically significant adverse trend in the significant events indicator will 

exist if the final ASP results for these five precursor events indicate that these meet the ITP 

significant events criteria. 

FY 2013 SHORT-TERM INDUSTRY PERFORMANCE: 

The staff uses a statistical approach based on prediction limits to identify potential short-term 

year-to-year emergent issues before they become long-term trends. Enclosure 2 shows the 

short-term results and the prediction limits for each of the ITP indicators. None of the indicators 

exceeded its prediction limit in FY 2013. Short-term FY 2013 data did not reveal any emerging 

trends that warranted additional analysis or significant adjustments to the nuclear reactor safety 

inspection or licensing programs. However, the staff observed that the short-term data for the 

safety system failures indicator was close to exceeding its short-term prediction limit. The staff 

noticed an increase in the reporting of safety system failures after issuing NUREG-1022, Rev. 3, 

“Event Report Guidelines: 10 CFR 50.72 and 50.73,” (ADAMS Accession No. ML13032A220), 

which clarified the criteria for event reporting. 

FY 2013 RESULTS OF BASELINE RISK INDEX FOR INITIATING EVENTS: 

In 2008, the NRC staff implemented the Baseline Risk Index for Initiating Events (BRIIE) as part 

of the ITP. The BRIIE (1) tracks several types of events that could potentially initiate a 

challenge to a plant’s safety systems, (2) assigns a value to each initiating event (IE) according 

to its relative importance to the plant’s overall risk of damage to the reactor core, and 

(3) calculates an overall indicator of industry safety performance. 
The Commissioners - 4 - 

The BRIIE concept provides a two-level approach to industry performance monitoring. The first 

level (referred to as Tier 1 performance monitoring) tracks and counts the number of times that 

the IEs that affect plant safety occur in nuclear power plants during the year. Nine IE categories 

are monitored for boiling-water reactors and 10 for pressurized-water reactors. The number of 

times that each event occurs is compared to a predetermined number of occurrences for that 

event. The predetermined number of occurrences is calculated from a predictive distribution 

derived using information from an established baseline period; it is reevaluated on an annual 

basis. If the predetermined number is exceeded, one can infer the possible degradation of 

industry safety performance. This annual tracking allows the NRC to intervene and engage the 

nuclear industry before any long-term adverse trends in performance emerge. 

 

The second level (referred to as Tier 2 performance monitoring) addresses the risk to plant 

safety and core damage that each of the initiating events contributes. Each event is assigned 

an importance value, a ranking based on its relative contribution to overall risk to plant safety. 

The greater the contribution of the event to overall risk, the higher the importance value it is 

assigned. Using statistical methods, the importance values are combined with the number of 

times the events occur during the year to calculate a number that indicates how much the 

overall industry risk of damage to the reactor core has changed from a baseline value. If the 

BRIIE-combined industry value reaches or exceeds a threshold value of 1×10-5 per reactor 

critical year, the NRC informs Congress of this performance outcome, along with actions that 

have already been taken or are planned in response, in the NRC Performance and 

Accountability Report. 

Enclosure 3 includes Tier 1 and Tier 2 BRIIE results. None of the IEs tracked in Tier 1 

exceeded its prediction limit in FY 2013. For Tier 2, Figure 15 of Enclosure 3 shows that the 

combined industry BRIIE value for FY 2013 (-2.52×10-6 per reactor critical year) is negative, 

which indicates that industry performance was better than baseline. The combined industry 

BRIIE value is below the established reporting threshold of ∆CDF = 1.0×10-5 per reactor critical 

year. 

PROGRAM REVIEW: 

On June 13, 2013, the Commission issued an SRM (ADAMS Accession No. ML13164A337) in 

response to the May 29, 2013, “Briefing on the results of the Agency Action Review Meeting 

(AARM)” (ADAMS Accession No. ML13155A441). In this SRM the Commission directed the 

staff to perform a review of the ITP as follows: 

As part of the Reactor Oversight Process (ROP) Self-Assessment for calendar year (CY) 2013, 

the staff should review implementation of the Industry Trends Program over its history for 

lessons learned and inform the Commission of any program enhancements and/or resource 

reductions that may be warranted. 

The staff focused its review in the following areas: (1) program objectives, (2) indicators used in 

the program, (3) long-term analysis, (4) short-term analysis, (5) implementation of the BRIIE, 

and (6) additional considerations. The staff completed its review of the implementation of the 

ITP for lessons learned and concluded that: 

• No changes to or a realignment of the ITP objectives (as listed in the program guidance) 

are warranted. 
The Commissioners - 5 - 

• An indicator representative of the Public Radiation Safety cornerstone and the Security 

cornerstone of the ROP should be included in the ITP. 

• The potential use of ROP Performance Indicators (PIs) to supplement or replace ITP 

indicators and unnecessary duplication among ITP indicators should be evaluated. This 

may offer an opportunity for the NRC to achieve better resource efficiencies. 

• The ASP outcomes should not be replaced with Management Directive (MD) 8.3, “NRC 

Incident Investigation Program” (ADAMS Accession No. ML031250592), results for ITP 

purposes. 

• The significant events indicator data before FY 2007 do not reflect the same definition 

for the indicator, but no adjustments to the data or its sources are warranted at this time. 

• No changes to the ITP long-term analysis process are warranted. 

• No changes to the ITP short-term analysis process are warranted. 

• No changes to the BRIIE process are warranted. 

• The staff should explore ways to incorporate industrywide performance of new reactors 

in the ITP. 

The staff’s review of the program, conclusions, and commitments for addressing its conclusions 

are discussed in more detail in Enclosure 4 of this paper. 

COMMITMENTS: 

The staff commits to the following: 

• Consider the use of the Public Radiation Safety cornerstone ROP performance indicator 

(i.e., Radiological Effluence Occurrence) in the ITP. 

• Consider the use of the Security cornerstone ROP performance indicator (i.e., Protected 

Area Security Equipment Performance Index) in the ITP. 

• Evaluate the possibility of supplementing and/or replacing ITP indicators with ROP PIs 

where applicable. 

• Evaluate the use of performance data for new reactors in the ITP, as applicable. 

The staff intends to report to the Commission on its progress towards meeting these 

commitments in future ITP SECY papers. 

CONCLUSION: 

As discussed in this paper, for FY 2013, the staff identified no statistically significant adverse 

trends in industry safety performance. Specifically, no ITP indicator exceeded its prediction 

limit, and the BRIIE value remained below the threshold for a report to Congress. The staff will 
The Commissioners - 6 - 

update the FY 2013 significant events data after receiving the final ASP results from RES and 

will report any changes to the ITP analysis in a memorandum to the Commission. 

RESOURCES:

For ITP activities, resources are included in the FY 2014 enacted budget of 0.5 full-time 

equivalent (FTE) and $595K; and in the FY 2015 Request of 0.5 FTE and $595K. These 

resources are to conduct ongoing ITP implementation in FY 2014 and FY 2015. No additional 

funds beyond those already budgeted will be required for the program. Any additional 

resources required in the future years will be addressed during the Planning, Budgeting, and 

Performance Management process. 

COORDINATION: 

The Office of the Chief Financial Officer has reviewed this paper and concurs. The Office of the 

General Counsel has reviewed this paper and has no legal objection. 

 /RA/ 

 Eric J. Leeds, Director 

Office of Nuclear Reactor Regulation 

Enclosures: 

1. Fiscal Year 2013 Long-Term Industry Trend 

Results 

2. Fiscal Year 2013 Short-Term Industry 

Performance 

3. Summary of Baseline Risk Index for 

Initiating Events: Annual Graphs through 

Fiscal Year 2013 

4. Staff’s Review of the Industry Trends 

Program 
The Commissioners - 6 - 

update the FY 2013 significant events data after receiving the final ASP results from RES and 

will report any changes to the ITP analysis in a memorandum to the Commission. 

RESOURCES:

For ITP activities, resources are included in the FY 2014 enacted budget of 0.5 full-time 

equivalent (FTE) and $595K; and in the FY 2015 Request of 0.5 FTE and $595K. These 

resources are to conduct ongoing ITP implementation in FY 2014 and FY 2015. No additional 

funds beyond those already budgeted will be required for the program. Any additional 

resources required in the future years will be addressed during the Planning, Budgeting, and 

Performance Management process. 

COORDINATION: 

The Office of the Chief Financial Officer has reviewed this paper and concurs. The Office of the 

General Counsel has reviewed this paper and has no legal objection. 

 Eric J. Leeds, Director 

Office of Nuclear Reactor Regulation 

Enclosures: 

1. Fiscal Year 2013 Long-Term Industry 

 Trend Results 

2. Fiscal Year 2013 Short-Term Industry 

 Performance 

3. Summary of Baseline Risk Index for Initiating 

 Events: Annual Graphs through Fiscal Year 2013 

4. Staff’s Review of the Industry Trends Program 

ADAMS Accession Number: ML14069A110 WITS 200100034 *concurred via email 

OFFICE NRR/DIRS/IPAB NRR/DIRS/IPAB BC:NRR/DIRS/IPAB Tech Editor* BC:NRR/DIRS/IOEB*

NAME AWaugh LCruz RFranovich JDougherty HChernoff 

(EThomas for) 

DATE 02/28/2014 02/28/2014 03/19/2014 03/7/2014 03/28/2014 

OFFICE D:NRR/DIRS OCFO* OGC (NLO) RES* D:NRR 

NAME HNieh RAllwein BMizuno BSheron 

(RCorreia for) ELeeds 

DATE 03/27/2014 04/01/2014 03/25/2014 03/28/2014 04/10/2014 

OFFICIAL RECORD COPY
Enclosure 1 

FISCAL YEAR 2013 LONG-TERM INDUSTRY TREND RESULTS 

The staff of the U.S. Nuclear Regulatory Commission did not observe any statistically significant 

adverse trends in the Industry Trends Program performance indicator data from the most recent 

10 years (fiscal years 2004–2013), as indicated by the figures below. 

Figure 1. Automatic scrams while critical

0.56

0.47

0.32

0.48

0.29

0.36

0.44 0.45

0.30

0.38

0.00

0.50

1.00

1.50

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Scrams per Unit

Fiscal Year

Industry Average

Industry Average Trend

Industry Average
- 2 - 

Figure 2. Safety-system actuations 

Figure 3. Significant events

0.24

0.38

0.22

0.25

0.14

0.23

0.18 0.19

0.17

0.29

0

0.2

0.4

0.6

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Actuations per Unit

Fiscal Year

Industry Average

Industry Average Trend

Industry Average

0.04

0.05

0.03

0.02

0.03

0.02

0.10

0.13

0.10

0.02

0

0.04

0.08

0.12

0.16

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Events per Unit

Fiscal Year

Industry Average
- 3 - 

Figure 4. Safety-system failures 

Figure 5. Forced outage rate 

As discussed in this paper, Figures 1 through 5 do not display a trendline because these graphs 

do not have a statistically significant trend. 

0.78

0.99

0.60

0.68 0.71 0.71

0.96 0.96

0.88

1.01

0.0

0.5

1.0

1.5

2.0

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Failures per Unit

Fiscal Year

Industry Average

Industry Average Trend

Industry Average

1.88

2.34

1.47 1.41 1.34

2.21

1.74 1.80

2.77

2.98

0

2

4

6

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Forced Outage Rate (%)

Fiscal Year

Industry Average

Industry Average Trend

Industry Average
- 4 - 

Figure 6. Equipment forced outages 

Figure 7. Collective radiation exposure 

0.15

0.13

0.10 0.11

0.08

0.09 0.10

0.09 0.09

0.08

0

0.1

0.2

0.3

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Equipment Forced Outages per

1,000 Commercial Critical Hours

Fiscal Year

Industry Average

Industry Average Trend

Industry Average

Industry Average Trend

100

117

93

109

96

87 91 90

72 80

0

50

100

150

200

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Average Exposure per Unit

(Person-REM)

Fiscal Year

Industry Average

Industry Average Trend

Industry Average

Industry Average Trend
- 5 - 

Figure 8. Unplanned power changes

Figure 9. Reactor coolant system activity

1.00 0.97 0.91 0.91 0.92

0.56

0.86 0.89

0.70 0.76

0.0

0.5

1.0

1.5

2.0

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Power Changes per 7,000 Critical Hours

Fiscal Year

Industry Average

Industry Average Trend

Industry Average

Industry Average Trend

0.42 0.41

0.60

0.45

0.48

0.32

0.19

0.13 0.09

0.08

0.0

0.5

1.0

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Maximum Percentage of Technical

Specification Limits

Fiscal Year

Industry Average

Industry Average Trend

Industry Average

Industry Average Trend
- 6 - 

Figure 10. Reactor coolant system leakage

Figure 11. Drill and exercise performance 

4.64 4.45 4.08 4.35 4.25 3.97 4.14 4.05 4.08 3.99

0

2

4

6

8

10

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Maximum Percentage of

Technical Specification Limits

Fiscal Year

Industry Average

Industry Average Trend

Industry Average

Industry Average Trend

95.69 95.82

96.06 96.19 96.22

97.06 96.89

97.30 97.51 97.54

92

94

96

98

100

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Percentage Timely and Accurate

Fiscal Year

Industry Average

Industry Average Trend

Industry Average

Industry Average Trend
- 7 - 

Figure 12. Emergency response organization drill participation

Figure 13. Alert and notification system reliability

97.90

98.20

97.89

97.55

98.16

98.65 98.77

99.43

99.33

99.44

94

96

98

100

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Percentage of Key Personnel

Fiscal Year

Industry Average

Industry Average Trend

Industry Average

Industry Average Trend

99.23

99.34

99.48 99.45

99.53 99.57 99.59

99.68

99.77 99.77

98

99

100

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Percentage of Tests Successful

Fiscal Year

Industry Average

Industry Average Trend

Industry Average

Industry Average Trend
- 8 - 

Figure 14. Accident sequence precursors

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

2003 2004 2005 2006 2007 2008 2009 2010 2011 2012

Precursors Occurrence per Unit 

Fiscal Year 
Enclosure 2 

FISCAL YEAR 2013 SHORT-TERM INDUSTRY PERFORMANCE

The annual industry trend analysis compares data for the most recent year to established 

short-term “prediction limits.” The prediction limits are 95th percentiles of predictive distributions 

for the data. The predictive distributions are statistical probability distributions that describe 

expected future performance. They are derived from performance during “baseline” periods for 

each performance indicator (PI). Baseline periods are periods for each PI during which the data 

can be regarded as fairly constant and indicative of “current” performance. 

The results of the evaluation for fiscal year (FY) 2013 Industry Trends Program PIs, using the 

established prediction limits, indicate that no PI exceeded its associated prediction limit in 

FY 2013, as shown in the following figures for each PI with its FY 2013 data and associated 

prediction limit. 

Figure 1. Automatic scrams while critical

0.00

0.25

0.50

0.75

1.00

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Scrams per Unit

Fiscal Year

Industry Average

Prediction Limit (0.673)

Baseline Industry Average (0.534), 1997-2002
- 2 - 

Figure 2. Safety-system actuations 

Figure 3. Significant events

0.00

0.20

0.40

0.60

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Actuations per Unit

Fiscal Year

Industry Average

Prediction Limit (0.375)

Baseline Industry Average (0.275), 1998-2005

0.00

0.05

0.10

0.15

0.20

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Events per Unit

Fiscal Year

Industry Average

Prediction Limit (0.106)

Baseline Industry Average (0.524), 2000-2004
- 3 - 

Figure 4. Safety-system failures 

Figure 5. Forced outage rate 

0.00

0.50

1.00

1.50

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Failures per Unit

Fiscal Year

Industry Average

Prediction Limit (1.04)

Baseline Industry Average (0.859), 2001-2004

0

2

4

6

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Forced Outage Rate %

Fiscal Year

Industry Average

Prediction Limit (4.28)

Baseline Industry Average (2.40), 2001-2004
- 4 - 

Figure 6. Equipment forced outages 

Figure 7. Collective radiation exposure

0.00

0.05

0.10

0.15

0.20

0.25

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Equipment Forced Outage Rate per 

1,000 Commercial Critical Hours

Fiscal Year

Industry Average

Prediction Limit (0.180)

Baseline Industry Average (0.142), 1998-2004

0

50

100

150

200

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Average Exposure per Unit (Person-REM)

Fiscal Year

Industry Average

Prediction Limit (139)

Baseline Industry Average (119), 1999-2002
- 5 - 

Figure 8. Unplanned power changes per 7,000 critical hours 

Figure 9. Reactor coolant system activity 

0.0

0.5

1.0

1.5

2.0

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Unplanned Power Changes per

7,000 Critical Hours

Fiscal Year

Industry Average

Prediction Limit 1.30

Baseline Industry Average (1.11), 2001-2004

0.00

0.25

0.50

0.75

1.00

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Maximum Percentage of Technical 

Specification Limits

Fiscal Year

Industry Average

Prediction Limit (0.646)

Baseline Industry Average (0.462), 2001-2004
- 6 - 

Figure 10. Reactor coolant system leakage 

Note that the 2000 steam generator tube rupture event at Indian Point Nuclear Generating 

Unit 2 was not included in the short-term data for determining prediction limits in Figure 10. This 

event was excluded from the development of the prediction limit models because it was 

considered as an outlier that could overly influence the statistical analysis of the industrywide 

data. This treatment results in a more conservative prediction limit. 

Figure 11. Drill and exercise performance

0

2

4

6

8

10

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Maximum Percentage of Technical 

Specification Limits

Fiscal Year

Industry Average

Prediction Limit (6.32)

Baseline Industry Average (5.14), 2000-2004

92

94

96

98

100

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Percentage Timely and Accurate

Fiscal Year

Industry Average

Prediction Limit (95.21)

Baseline Industry Average (95.62), 2000-2004
- 7 - 

Figure 12. Emergency response organization drill participation 

Figure 13. Alert and notification system reliability 

94

96

98

100

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Percentage of Key Personnel

Fiscal Year

Industry Average

Prediction Limit (97.50)

Baseline Industry Average (97.87), 2003-2007

97

98

99

100

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Percentage of Tests Successful

Fiscal Year

Industry Average

Prediction Limit (98.35)

Baseline Industry Average (98.87), 2001-2004
Enclosure 3 

SUMMARY OF BASELINE RISK INDEX FOR INITIATING EVENTS: 

ANNUAL GRAPHS THROUGH FISCAL YEAR 2013 

The Baseline Risk Index for Initiating Events (BRIIE) addresses the initiating event (IE) 

cornerstone in the U.S. Nuclear Regulatory Commission’s (NRC’s) Reactor Oversight Process 

(ROP) for monitoring commercial nuclear power plants. It is based on plant performance for the 

10 initiator events listed in the table below. 

 INITIATOR ACRONYM APPLICABLE PLANTS 

 General transient TRAN Both plant types, separately 

 Loss of condenser heat sink LOCHS Both plant types, separately 

 Loss of main feedwater LOMFW Both plant types 

 Loss of offsite power LOOP Both plant types 

Loss of vital alternating 

current bus LOAC Both plant types 

Loss of vital direct current 

bus LODC Both plant types 

Stuck-open safety or relief 

valve SORV Both plant types, separately 

 Loss of instrument air LOIA Both plant types, separately 

Very small loss-of-coolant 

accident VSLOCA Both plant types 

Steam generator tube 

rupture SGTR Pressurized-water reactors 

(PWRs) only 

The BRIIE program, described in NUREG/CR-6932, “Baseline Risk Index for Initiating Events 

(BRIIE),” issued June 2007, consists of two levels or tiers. The first tier considers individual IEs 

and evaluates performance based on statistical prediction limits. This evaluation is for the 

ongoing monitoring and early detection of possible industry-level deficiencies. A second tier is a 

risk-based integrated measure evaluated for each plant type. Because four of the initiators 

have separate data for each plant type, there are a total of 14 Tier 1 graphs. 

The units for the Tier 1 IE frequency graphs are event counts for a fiscal year divided by the 

industry critical time for the year. The Tier 1 graphs also show the average frequency for an 

established “baseline period” and 95-percent prediction limits for a future year if occurrences 

continue at the same rate as in the baseline period. If industry data shift as time progresses, 

the baseline periods used to determine the prediction limits might no longer be relevant. The 

periods originally were developed to describe, roughly, calendar years 1998–2002. 

The prediction limits depend on the expected critical years of reactor operation in the upcoming 

year and the baseline occurrence rate for each indicator. A rate can exceed a limit by having 

more events than expected or by having the same number of events and less critical time than 

expected. In recent years, U.S. nuclear power plant availability has been approximately 

90 percent at the industry level. This figure enters into the calculations that determine the 

bounds on the number of events that might be expected. 

None of the fiscal year (FY) 2013 occurrence rates exceeded their prediction limits. 

The Tier 2 integrated index includes, for each plant type, the relative contribution of each 

initiator to the risk of core damage, based on the events that occurred in each fiscal year. 
- 2 - 

The event frequencies are converted to core-damage frequency (CDF) estimates by multiplying 

by Birnbaum risk coefficients. These coefficients are industry averages of the contribution to 

core damage from each initiator as reflected in the industry’s standardized plant analysis risk 

models. 

Figure 15 shows annual differences in estimated industry CDF compared to the established 

baseline levels of these quantities. The combined industry BRIIE value for FY 2013 (−2.52×10−6

per reactor critical year) indicates better than baseline industry performance. The combined 

industry BRIIE value is below the established reporting threshold of ∆CDF = 1.0×10−5 per 

reactor critical year. 

Figure 1. Pressurized-water reactor (PWR) general transients

0.717

0.515 0.524

0.649

0.559 0.568

0.883

0.655

0.439

0.566

0.0

0.5

1.0

1.5

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Events per reactor critical year

Fiscal Year Baseline period: 1998 - 2002

Baseline industry average

95% prediction limit for one year
- 3 - 

Figure 2. Boiling-water reactor (BWR) general transients 

Figure 3. PWR loss of condenser heat sink

0.808 0.837

0.558

0.971

0.546

0.839

0.634

0.472

0.646 0.665

0.0

0.5

1.0

1.5

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Events per reactor critical year

Fiscal Year

Baseline industry average

95% prediction limit for one year

0.112

0.064 0.064

0.032

0.016

0.047

0.082

0.033 0.034 0.034

0.00

0.05

0.10

0.15

0.20

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Events per reactor critical year

Baseline period: 1995 - 2002 Fiscal Year

Baseline industry average

95% prediction limit for one year 
- 4 - 

Figure 4. BWR loss of condenser heat sink 

Figure 5. Loss of main feedwater

0.155 0.161

0.031

0.157

0.091 0.093

0.060 0.063 0.062 0.063

0.0

0.1

0.2

0.3

0.4

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Events per reactor critical year

Fiscal Year Baseline period: 1996 - 2002

Baseline industry average

95% prediction limit for one year 

0.084 0.086

0.042

0.063

0.042 0.042

0

0.022

0.011

0.044

0.00

0.05

0.10

0.15

0.20

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Events per reactor critical year

Baseline period: 1993 - 2002 Fiscal Year

Baseline industry average

95% prediction limit for one year 
- 5 - 

Figure 6. Loss of offsite power 

The prediction limit for loss of offsite power was calculated under the assumption that the eight 

at-power events that occurred during the 2003 blackout were a single event. This treatment 

results in a more conservative prediction limit.

Figure 7. Loss of vital alternating-current bus

0.053

0

0.021

0.011

0

0.031

0

0.075

0.055

0.044

0.00

0.05

0.10

0.15

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Events per reactor critical year

Fiscal Year Baseline period: 1997 - 2004

Baseline industry average

95% prediction limit for one year 

000000

0.021

0

0.011

0

0.00

0.02

0.04

0.06

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Events per reactor critical year

Fiscal Year Baseline period: 1992 - 2002

Baseline industry average

95% prediction limit for one year
- 6 - 

Figure 8. Loss of vital direct-current bus

Figure 9. PWR stuck-open safety or relief valve

0000000

0.011

0 0

0.00

0.01

0.02

0.03

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Events per reactor critical year

Fiscal Year Baseline period: 1988 - 2002

Baseline industry average

95% prediction limit for one year

0000000000

0.00

0.01

0.02

0.03

0.04

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Events per reactor critical year

Fiscal Year Baseline period: 1988 - 2002

Baseline industry average

95% prediction limit for one year
- 7 - 

Figure 10. BWR stuck-open safety or relief valve

Figure 11. PWR loss of instrument air 

0.031

0000

0.031

0000

0.00

0.03

0.06

0.09

0.12

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Events per reactor critical year

Fiscal Year Baseline period: 1993 - 2002

Baseline industry average

95% prediction limit for one year

0 0

0.016 0.016

0 0

0.016

000

0.00

0.02

0.04

0.06

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Events per reactor critical year

Fiscal Year

Baseline period: 1997 - 2002

Baseline industry average

95% prediction limit for one year
- 8 - 

Figure 12. BWR loss of instrument air

Figure 13. Very small loss-of-coolant accident

0 0

0.031

0000000

0.00

0.02

0.04

0.06

0.08

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Events per reactor critical year

Fiscal Year

Baseline period: 1991 - 2002

Baseline industry average

95% prediction limit for one year

0

0.011

00000000

0.00

0.01

0.02

0.03

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Events per reactor critical year

Fiscal Year Baseline period: 1992 - 2002

Baseline industry average

95% prediction limit for one year
- 9 - 

Figure 14. PWR steam generator tube rupture 

Figure 15. BRIIE Tier 2 (ΔCDF) 

0000000000

0.00

0.01

0.02

0.03

0.04

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Events per reactor critical year

Fiscal Year Baseline period: 1991 - 2002

Baseline industry average

95% prediction limit for one year

-1.0E-05

-5.0E-06

0.0E+00

5.0E-06

1.0E-05

1.5E-05

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Delta CDF (1/rcry)

Fiscal Year

PWR BWR Industry Using Bayesian updates of annual frequencies

Industry BRIIE threshold for one year of data: 1.E-5
Enclosure 4 

STAFF’S REVIEW OF THE INDUSTRY TRENDS PROGRAM 

BACKGROUND: 

The U.S. Nuclear Regulatory Commission (NRC) staff developed and implemented the Industry 

Trends Program (ITP) in 2001 to (1) monitor, assess, and respond to trends in industrywide 

performance indicators (PIs) for operating power reactors and (2) communicate the results to 

Congress and other stakeholders. The program is also used to provide an agency performance 

measure on the number of statistically significant adverse trends in nuclear safety performance. 

The staff reports the results of the program to the Commission annually. The reporting of the 

ITP results is preceded by annual reports developed by the former Office for Analysis and 

Evaluation of Operational Data (AEOD) such as the NUREG-1187, “Performance Indicators for 

Operating Commercial Nuclear Power Plants,” and NUREG-1272, “AEOD’s Annual Report,” 

series. 

On June 13, 2013, the Commission issued a staff requirements memorandum (SRM) 

(Agencywide Documents Access and Management System (ADAMS) Accession 

No. ML13164A337) in response to the May 29, 2013, “Briefing on the results of the Agency 

Action Review Meeting (AARM)” (ADAMS Accession No. ML13164A337), directing the staff to 

perform a review of the ITP as follows: 

As part of the Reactor Oversight Process (ROP) Self-Assessment for calendar year (CY) 2013, 

the staff should review implementation of the Industry Trends Program over its history for 

lessons learned and inform the Commission of any program enhancements and/or resource 

reductions that may be warranted.

This enclosure provides the staff’s review of the ITP in response to the Commission’s June 13, 

2013, SRM. 

HISTORICAL REVIEW PROCESS AND FOCUS AREAS: 

The staff reviewed Commission papers on the results of the ITP and program guidance for 

lessons learned in the implementation of the program. The staff consulted previous ITP 

program managers and other internal stakeholders in the development of its conclusions and 

recommendations. As part of its review, the staff summarized the results presented in previous 

ITP SECY papers to support the evaluation of the program and to document that information as 

a knowledge-management resource (ADAMS Accession No. ML14087A279). SECY-01-0111, 

“Development of an Industry Trends Program for Operating Power Reactors” (ADAMS 

Accession No. ML011570324), describes the development of the program, and Inspection 

Manual Chapter (IMC) 0313, “Industry Trends Program” (ADAMS Accession 

No. ML080860540), contains guidance governing the program. 

The staff focused its review of the ITP based on the areas discussed in both the annual 

reporting of the ITP results and IMC 0313 guidance. The staff evaluated the following focus 

areas to identify lessons learned, and potential enhancements or resource reductions: 

• Program objectives, 

• Indicators used, 

• Long-term analysis, 

• Short-term analysis, 

• Baseline Risk Index for Initiating Events (BRIIE) implementation, and 
- 2 - 

• Additional considerations. 

PROGRAM OBJECTIVES: 

The staff developed the ITP as a means to confirm the safety performance of operating nuclear 

power plants and to increase public confidence in the efficacy of NRC processes. The ITP 

complements the ROP plant-specific oversight of safety performance by monitoring and 

assessing industry-wide trends in safety performance. The specific program objectives 

described in SECY-01-0111 are as follows: 

• Collect and monitor industrywide data that can be used to assess whether operating 

plant safety performance is being maintained and provides feedback for the ROP. 

• Assess the significance and causes of any statistically significant adverse industry 

trends, determine whether they represent an actual degradation in overall industry safety 

performance, and respond appropriately to any safety issues that may be identified. 

• Communicate industrywide information to Congress and other stakeholders in an 

effective and timely manner. 

The ITP objectives are defined in IMC 0313 as follows: 

• Collect and monitor industrywide data that can be used to assess whether the nuclear 

industry is maintaining the safety performance of operating plants and provide NRC 

feedback to its nuclear reactor safety inspection and licensing programs. 

• Assess the safety significance and causes of any statistically significant adverse industry 

trends, determine whether the trends represent an actual degradation in overall industry 

safety performance, and respond appropriately to any safety issues that may be 

identified. 

• Communicate industry-level information to Congress and other stakeholders in an 

effective and timely manner. 

• Support the NRC’s performance goal of ensuring safety while enhancing openness in 

the agency’s regulatory processes. 

The staff compared the original objectives of the ITP to those that guide program 

implementation today and concluded that they have remained consistent over the last 13 years. 

The staff did note that an objective to promote openness in the agency’s regulatory processes 

has existed in IMC 0313 since its inception. The staff also reviewed ITP results over the history 

of the program and found that the methods and conclusions documented therein have 

continued to satisfy the stated program objectives. The staff is not considering any changes to 

or a realignment of the ITP objectives as listed in the program guidance. No enhancements or 

resource reductions are warranted in this area. 

INDICATORS USED IN THE ITP: 

The staff reviewed the ITP indicators to evaluate their effectiveness, and identify lessons 

learned and potential enhancements or resource reductions. The staff examined the 
- 3 - 

effectiveness of the ITP indicators by evaluating: (1) their ability to provide feedback to the ROP, 

(2) the efficiency of their data sources, and (3) the results yielded since the program was 

implemented. 

1. Feedback to the ROP 

The staff evaluated all of the ITP indicators to determine whether these can provide feedback to 

the ROP consistently across all cornerstones and found that the current set of ITP indicators 

does not provide industry performance insights in either the Public Radiation Safety cornerstone 

or the Security cornerstone. To more fully reflect ROP effectiveness, the ITP should include an 

indicator representative of each of these two cornerstones of the ROP. The staff recognized 

that there are ROP PIs for each of these two cornerstones that can be used as industrywide 

indicators in the ITP and will consider these for use in the ITP as a potential enhancement. 

2. Data sources 

The staff reviewed the data sources used by the ITP indicators and identified areas where 

improvements could be made to increase the efficiency of the data-collection process. Six of 

the ITP indicators use data obtained through the ROP PIs. Three of the ITP indicators use data 

obtained from Licensee Event Reports (LERs) in accordance with 10 CFR 50.73, “Licensee 

Event Report System”, and Event Notifications (ENs) in accordance with 10 CFR 50.72, 

“Immediate Notification Requirements for Operating Nuclear Power Reactors.” Three of the ITP 

indicators use data from the Institute of Nuclear Power Operations (INPO). Two ITP indicators 

use other agency processes. The data sources for each indicator can be found in IMC 0313 

and are summarized in Table 1 at the end of this Enclosure. 

The staff identified two ITP indicators that use LERs and ENs as data sources and for which a 

related ROP PI exists. These are the Automatic Scrams while Critical and Safety-System 

Failures indicators. However, the staff understands that there are differences between the 

definitions of these ITP indicators and the definitions of their related ROP PIs (Unplanned 

Scrams and Safety Systems Functional Failures, respectively). The staff will evaluate the 

feasibility of replacing or supplementing the data sources for these indicators with the 

information obtained through the ROP PI program. The staff recognizes this as an opportunity 

to possibly reduce resources in the ITP data collection and analysis processes. However, this 

transition would increase the ITP’s dependency on the ROP PI program, which the industry 

currently supports through voluntary reporting of ROP PI data. The staff will evaluate whether 

unnecessary duplication exists among the ITP indicators and will consider the potential use of 

ROP PIs to supplement or replace ITP indicators where possible. 

The staff also evaluated the data sources used for the significant events indicator. As noted in 

SECY-09-0048, “Fiscal Year 2008 Results of the Industry Trends Program for Operating Power 

Reactors and Status of the Ongoing Development of the Program” (ADAMS Accession 

No. ML090420101), the staff revised IMC 0313 to clarify the definition of the significant events 

indicator in 2008. The revised definition provided more objective criteria to identify significant 

events. One of the criteria used in the revised definition is “an event with a Conditional Core 

Damage Probability (CCDP) or increase in core damage probability (ΔCDP) of 1×10-5 or higher.” 

The staff has historically used the Accident Sequence Precursor (ASP) program results to 

define the CCDP or ΔCDP of potential significant events. The annual results of the ASP 

program are issued on a fiscal-year basis, at the end of the following fiscal year. ITP results are 

reported on a fiscal-year basis, midway through the following fiscal year. Therefore, most of the 

ASP results are not finalized before the annual ITP reporting, and the staff has had to update 

the Commission through a separate memorandum when there are any changes in the analysis 

of the significant events indicator after the ASP analyses are completed. 
- 4 - 

 As part of this review, the staff considered using results from Management Directive (MD) 8.3, 

“NRC Incident Investigation Program” (ADAMS Accession No. ML031250592), as a data source 

for CCDP or ΔCDP values for potential significant events. This possibility was considered 

because the MD 8.3 program provides early risk-significance results that would support more 

timely ITP analysis. However, the staff noted that not all of the events considered under the 

ASP undergo an MD 8.3 evaluation. For these events, ASP would continue to be the relied 

upon data source; however, MD 8.3 results can be used to inform the staff in its evaluation of 

the significant events indicator. In addition, the ASP analysis is more robust than the MD 8.3 

analysis, and it provides more accurate risk significance results. Using a combination of MD 8.3 

and ASP program data would introduce data of varying pedigree without the consistent benefit 

of more timely results. Therefore, the staff concludes that ASP outcomes should not be 

replaced with MD 8.3 results for ITP purposes.

The staff also examined, as part of its review, the significant events data before 2007 and 

determined that these older data do not reflect the current definition for the indicator, which was 

clarified in 2008. The staff reviewed the significant events data between FY 2004 and FY 2007 

and concluded that realigning these data to reflect the revised definition for significant events 

would result in a less conservative approach for the long-term trending of this indicator. If the 

data are updated to reflect the current definition of significant events, the number of significant 

events required to meet the statistically significant trend threshold would be higher. The staff 

concludes that no changes to the significant events indicator data sources are warranted. The 

staff notes, as a lesson learned, that the significant event indicator data before FY 2007 do not 

reflect the same definition for the indicator, but is not considering any adjustments at this time.

3. ITP Indicator Results 

The staff also reviewed the historical behavior of each indicator and the staff’s actions in 

response to exceeded short-term prediction limits. The staff observed that most of the 

indicators exhibit a statistically significant decreasing trend, which represents an overall 

improvement in industry safety performance. However, the staff noted that the following 

indicators have exceeded their short-term prediction limits since the program was implemented: 

• Safety-System Actuations: The short-term prediction limit for this indicator was 

exceeded in 2003 and 2005. The staff observed that the Northeast blackout of 

August 14, 2003 was a driver in exceeding the prediction limit in 2003. The staff’s 

analysis did not reveal any safety issue warranting immediate regulatory action, and the 

staff noted that the agency was addressing this issue through an action plan issued on 

February 4, 2004, for resolving electrical grid concerns (ADAMS Accession 

No. ML040160437). The staff also analyzed the indicator’s short-term data for 2005, 

when the short-term prediction limit was exceeded again. As discussed in SECY-06-

0076, “FY 2005 Results of the Industry Trends Program for Operating Power Reactors 

and Status of the Ongoing Development of the Program” (ADAMS Accession 

No. ML060450235), the staff did not identify any driving factors behind the increase in 

safety-system actuations. Therefore, the staff concluded that the trend in this indicator 

was not safety-significant. 

• Automatic Scrams while Critical: The short-term prediction limit for this indicator was 

exceeded in 2001 and 2003. The staff analyzed the short-term data for this indicator in 

both instances and did not identify any safety issues associated with exceeding the 

prediction limit. The staff also noted that the Northeast blackout of August 14, 2003, was 

a driver in exceeding the prediction limit in 2003. 
- 5 - 

• Collective Radiation Exposure: The short-term prediction limit for this indicator was 

exceeded in 2001. The staff reviewed the short-term data for this indicator and did not 

identify any safety issues associated with exceeding the prediction limit. 

• Significant Events: The short-term prediction limit for this indicator was exceeded in 

2011. The Operating Experience Branch staff reviewed significant events from the 

previous 5 years to identify any trends of concern. The review revealed that the number 

of events involving an initiator and subsequent complications increased since 2010, and 

that nonsafety-related system failures and corrective-action program weaknesses 

contributed to most of those significant events involving an initiator and complications. 

The staff’s insights were used as input to the ROP enhancement effort. 

Based on its review of the results achieved since the ITP’s inception, the staff did not identify 

any analytical flaws or unfounded conclusions. The staff found that the evaluations performed 

in response to the exceeded short-term prediction limits met program objectives; the evaluations 

provided reasonable insights into potential safety issues, and agency responses to these 

occurrences were well supported. No additional enhancements or resource reductions are 

warranted with regards to the ITP indicators.

LONG-TERM ANALYSIS: 

Appendix C, “Long Term Trending,” to IMC 0313 describes the staff’s approach to long-term 

trending analysis. Appendix C to IMC 0313 indicates that long-term analysis of the ITP 

indicators is performed on a 10-year basis to ensure that enough data are available for a 

meaningful statistical analysis. The staff also concluded in SECY-06-0076, “FY 2005 Results of 

the Industry Trends Program for Operating Power Reactors and Status of the Ongoing 

Development of the Program” (ADAMS Accession No. ML060450235), that the use of a 10-year 

period for long-term trending ensures that data more than 10 years old do not overly influence 

the trend determination. As part of this review, the staff considered the implementation of a 

shorter timeframe for long-term trending, but concluded that it would result in less conservative 

thresholds for the statistical significance of long-term trends. The staff reviewed the ITP 

indicators data for the last two decades and identified a significant improvement in industry 

performance when comparing the data from the last ten years to the data of the previous 

decade (i.e., ten to twenty years old data). Including performance data older than ten years in 

the ITP long-term analysis would reduce the ability of the program to detect any incremental 

changes in industry performance. The staff maintains its view that using a time period of more 

than 10 years would not be representative of current industry performance, as intended by the 

program. In addition, using older data would not yield a reliable, contemporary measure of the 

agency's safety performance, which is based on the number of statistically significant adverse 

trends. However, the staff recognizes that data beyond a 10-year period can be used to inform 

the review of exceeded short-term prediction limits or any emerging statistically significant 

long-term adverse trends. 

The staff also recognizes that it can gain insights on industry performance by completing an 

additional evaluation of trends in industry performance using a shorter time period; i.e., by 

evaluating “mid-term trending.” The Operating Experience staff in the Office of Nuclear Reactor 

Regulation (NRR) has broad responsibility to look at adverse data trends including “mid-term” 

trending and evaluation of industry performance data over a three-to-five year period. These 

reviews and insights have been and will continue to be integrated into the ITP. No 

enhancements or resource reductions are warranted in this area.
- 6 - 

SHORT-TERM ANALYSIS: 

As described in IMC 0313, the staff adopted a statistical approach using prediction limits to 

identify potential short-term emergent issues before they manifest themselves as long-term 

trends. Appendix B, “Prediction Limits,” to IMC 0313 describes the process for developing 

short-term prediction limits for the ITP indicators. The prediction limits for each indicator are 

developed using information from a baseline period, which contains at least 4 years of data. 

A statistical test is performed on groups of data, and the period with the most constant 

frequency is selected as the baseline period. A predictive distribution is calculated for this 

period, and the value representing 95 percent of the upper limit of the distribution is defined as 

the prediction limit for the indicator. Prediction limits are reassessed and updated accordingly 

on a yearly basis. 

As discussed in the “Indicators Used in the ITP” section of this enclosure, some of the ITP 

indicators have exceeded their short-term prediction limits, and the staff has responded 

accordingly to identify any safety issues derived from exceeding that limit. The staff concluded 

that the approach used to determine prediction limits and perform a short-term analysis is 

reasonable and has provided appropriate results. No enhancements or resource reductions are 

warranted in this area.

BRIIE IMPLEMENTATON: 

The staff implemented the BRIIE for the first time in SECY-09-0048, “Fiscal Year 2008 Results 

of the Industry Trends Program for Operating Power Reactors and Status of the Ongoing 

Development of the Program” (ADAMS Accession No. ML090420101). The BRIIE was 

implemented to enhance the ITP by incorporating a mechanism to determine the risk 

significance in safety performance. The BRIIE concept provides a two-level approach to 

industry performance monitoring (i.e., Tier 1 and Tier 2). Both Tier 1 and Tier 2 of the BRIIE are 

explained in the annual reporting of the ITP results to the Commission. The Tier 2 BRIIE value 

is reported as a change in core-damage frequency (ΔCDF). The staff established a threshold 

ΔCDF value of 1×10-5 for reporting to Congress. The BRIIE Tier 2 threshold value has not been 

exceeded since the BRIIE was implemented. However, the staff noted that, had BRIIE been 

established five years earlier, the reporting threshold would have been crossed in 2003 because 

of the increase in initiating events resulting from the Northeast blackout. 

The staff reviewed the BRIIE data since its implementation to identify any instances in which a 

Tier 1 short-term prediction limit was exceeded. The staff observed that the Loss of Offsite 

Power BRIIE indicator exceeded its short-term prediction limit in FY 2011. The staff reviewed 

the FY 2011 data for this indicator in SECY-12-0056, “Fiscal Year 2011 Results of the Industry 

Trends Program for Operating Power Reactors” (ADAMS Accession No. ML12065A340), and 

concluded that the events identified by the indicator resulted from natural phenomena occurring 

at multi-unit sites and were not representative of degraded industry performance. The staff did 

not recommend program adjustments as a result of this. 

The staff concluded that the approach used for evaluating Tier 1 of the BRIIE effectively 

prompted further analysis when the CY 2011 ITP review was being performed. The staff is 

confident that the BRIIE will continue to reveal risk-significant changes in industry performance 

when prediction limits are exceeded. This will lead to further analysis of underlying causes and, 

if warranted, regulatory actions to effectively address them. No enhancements or resource 

reductions are warranted in this area.
- 7 - 

ADDITIONAL CONSIDERATIONS: 

The staff recognizes that there is an effort in place to evaluate the appropriateness of ROP PIs 

for new reactors. The staff issued a commission paper addressing this matter on 

December 17, 2013: SECY-13-0137, “Recommendations for Risk-Informing the Reactor 

Oversight Process for New Reactors” (ADAMS Accession No. ML13263A351). The staff should 

evaluate the results of this effort to determine the impacts of using ROP PI data from new 

reactors in the ITP. The staff also recognizes that a similar evaluation of applicability of new 

reactors’ performance data for the BRIIE analysis should be performed in the near future. The 

staff should explore ways to incorporate industrywide performance of new reactors in the ITP. 

CONCLUSION: 

The staff reviewed the implementation of the ITP over its history for lessons learned and 

concluded that: 

• No changes to or a realignment of the ITP objectives (as listed in the program guidance) 

are warranted. 

• An indicator representative of the Public Radiation Safety cornerstone and the Security 

cornerstone of the ROP should be included in the ITP. 

• The potential use of ROP PIs to supplement or replace ITP indicators and unnecessary 

duplication among ITP indicators should be evaluated. This may offer an opportunity for 

the NRC to achieve better resource efficiencies. 

• The ASP outcomes should not be replaced with MD 8.3 results for ITP purposes. 

• The significant events indicator data before FY 2007 do not reflect the same definition 

for the indicator, but no adjustments to the data or its sources are warranted at this time. 

• No changes to the ITP long-term analysis process are warranted. 

• No changes to the ITP short-term analysis process are warranted. 

• No changes to the BRIIE process are warranted. 

• The staff should explore ways to incorporate industrywide performance of new reactors 

in the ITP. 

The staff’s proposed plan to pursue these potential enhancements and/or resource reductions is 

included as Table 2 of this Enclosure. 
- 8 - 

Table 1. Indicators Used in the Industry Trends Program 

Origin: Ex-AEOD = indicators developed by the AEOD Office 

 ROP PI = ROP performance indicators 

 IMC 0313 = indicator defined through IMC 0313 

 ASP = indicator derived from the Accident Sequence Precursor program 

Cornerstone: IE = Initiating Events MS = Mitigating Systems BI = Barrier Integrity 

 ORS = Occupational Radiation Safety EP = Emergency Preparedness 

Data Source: LERs = Licensee Event Reports ENs = Event Notifications 

 ROP AM = ROP Action Matrix Input ASP = Accident Sequence Precursor Program 

 AO = Abnormal Occurrences Report INES = International Nuclear Event Scale 

 MOR = Monthly Operating Reports ROP PI = ROP Performance Indicators 

INPO = reported by the Institute of Nuclear Power Operations 

Behavior: SST = statistically significant trend NSST = not statistically significant trend 

 

ITP Indicator Origin Corner-stone Data Source Trend 

Behavior 

Prediction 

Limit Baseline 

Period 

Prediction 

Limit 

Crossed 

Automatic Scrams 

while Critical Ex-AEOD IE LERs and ENs SST: Negative 

slope 1997-2002 2001 and 2003 

Safety-System 

Actuations Ex-AEOD MS LERs and ENs SST: Negative 

slope 1998-2005 2003 and 2005 

Significant Events IMC 0313 ALL ROP AM, ASP, 

AO, INES 

NSST: Positive 

slope 2000-2004 2011 

Safety-System 

Failures Ex-AEOD MS LERs and ENs NSST: Positive 

slope 2001-2004 No 

Forced Outage Rate Ex-AEOD MS MOR NSST: Positive 

slope 2001-2004 No 

Equipment Forced 

Outages Ex-AEOD MS MOR SST: Negative 

slope 1998-2004 No 

Collective Radiation 

Exposure Ex-AEOD ORS INPO SST: Negative 

slope 1999-2002 2001 

Unplanned Power 

Changes ROP PI IE ROP PI SST: Negative 

slope 2001-2004 No 

Reactor Coolant 

System Activity ROP PI BI ROP PI SST: Negative 

slope 2001-2004 No 

Reactor Coolant 

System Leakage ROP PI BI ROP PI SST: Negative 

slope 2000-2004 No 

Drill and Exercise 

Performance ROP PI EP ROP PI SST: Negative 

slope 2000-2004 No 

Emergency-Response 

Organization Drill 

Participation 

ROP PI EP ROP PI SST: Negative 

slope 2003-2007 No 

Alert and Notification 

System Reliability ROP PI EP ROP PI SST: Negative 

slope 2001-2004 No 

Accident-Sequence 

Precursor Rate ASP ALL ASP NSST: 

Negative slope N/A No 
- 9 -

Table 2. Staff’s Plan to Address Conclusions from the Industry Trends Program Review 

Review Area No. Conclusions Commitment Estimated Completion 

Objectives 1 No changes are warranted. N/A N/A Indicators 2 Include an indicator reflecting the Public Radiation Safety cornerstone. Consider using the Public Radiation Safety cornerstone ROP performance indicator (i.e., Radiological Effluence Occurrence). April 2015 – FY2014 ITP Results SECY Indicators 3 Include an indicator reflecting the Security cornerstone. Consider using the Security cornerstone ROP performance indicator (i.e., Protected Area Security Equipment Performance Index). April 2015 – FY2014 ITP Results SECY Indicators 4 

Consider the potential use of ROP PIs to supplement/replace ITP indicators where possible and evaluate ITP indicators for unnecessary duplication among these. 

Evaluate the feasibility of supplementing and/or replacing ITP indicators with ROP performance indicators where applicable. 

April 2015 – FY2014 ITP Results SECY 

Indicators 5 

ASP outcomes for significant events should not be replaced with MD 8.3 results. 

N/A N/A 

Indicators 6 

No adjustments to the significant events data or its sources are warranted at this time. 

N/A N/A 

Long -Term Analysis 7 No changes are warranted. N/A Short-Term Analysis 8 No changes are warranted. N/A N/A BRIIE 9 No changes are warranted. N/A N/A Additional Considerations 10 Explore ways to incorporate new reactors’ performance data into ITP. Evaluate the use of performance data for new reactors in the ITP, as applicable (e.g., ROP PIs for new reactors, BRIIE adequacy). Within a year of establishment of new reactors PIs.