POLICY ISSUE

INFORMATION

February 15, 2002 SECY-02-0030

FOR: The Commissioners

FROM: William D. Travers

Executive Director for Operations

SUBJECT: SUMMARY REPORT ON NRCíS HISTORICAL EFFORTS TO 

DEVELOP AND USE PERFORMANCE INDICATORS

PURPOSE:

To report on the staffís historical efforts to develop and use performance indicators (PIs), in

compliance with a staff requirements memorandum (SRM) dated August 2, 2001.

SUMMARY:

The staff began formal efforts to develop PIs in 1986 in response to direction from the

Commission. The staff conducted a survey of the various NRC offices and the regions to

identify power reactor PIs then in use and established an interoffice task group to develop a

single, coordinated NRC PI program. To minimize the burden of data collection on licensees,

the task group used the Institute of Nuclear Power Operationsí, (INPOís) PI definitions

whenever possible (INPO started its industry-wide PI program in 1985). The task group

developed a set of PIs for use by senior NRC managers to monitor trends in operational

performance at each operating plant. The first report, issued in February 1987, presented

quarterly data from calendar years 1985 and 1986. This program was expanded and improved

upon over the years, incorporating two new indicators and developing improved methods for

calculating and displaying the data. The program was discontinued in September 1999, when

the new Reactor Oversight Process (ROP) PIs came into use.

Contacts: Donald E. Hickman, NRR/DIPM

 301-415-8541

 Petteri Tiippana, NRR/DIPM (Foreign Assignee from the Radiation and Nuclear

 301-415-2963 Safety Authority, Finland)
-2-

The ROP PIs were developed as part of a joint effort by the NRC and stakeholders to

fundamentally restructure the NRCís process for overseeing the performance of operating

reactors. 

The ROP PIs provide a broad sample of data to assess licensee safety performance and help

determine the appropriate level of regulatory engagement. In developing these PIs, the staff

worked closely with all stakeholders, considered other PIs used by individual licensees, and

evaluated PIs used by international organizations and foreign nuclear plant operators and

regulators. The staff continues to expand and improve upon the ROP PIs through ongoing

public meetings.

BACKGROUND:

In an SRM dated August 2, 2001, the Commission directed the staff to provide a summary

report on NRCís historical efforts to develop and use PIs. The Commission requested that in

the report the staff should describe its efforts to work with the industry and other stakeholders

in developing the set of PIs used in the ROP and to probe the industry to identify areas

industry is trending that might be of value to the NRCís ROP. The SRM further directed that

the report highlight international use of PIs such as Ontario Powerís public exposure PI and

Finlandís advanced PIs.

DISCUSSION:

The NRC staff used PIs for many years to obtain insights regarding the performance of plants. 

The first official NRC PI program began in 1986, in response to an SRM dated March 10,

1986, in which the Commission directed the staff to provide (1) the current status of PI use by

various staff groups, including the use of specific indicators by specific offices; (2) the status of

work to develop PIs and (3) a listing of the PIs the staff believed the NRC should monitor, with

a plan and a schedule for that monitoring, and 4) a recommendation that a single group be

responsible for monitoring and evaluating PIs.

The staff responded to the Commission in SECY-86-144, ìPerformance Indicators,î dated

May 5, 1986, which listed approximately 80 PIs then in use or under development by various

NRC offices. The paper included a plan for developing a structured, integrated NRC PI

program for making timely regulatory decisions about the performance of nuclear power

plants. The Office of Inspection and Enforcement (IE) was assigned to coordinate and

implement this proposed PI program. The staff also informed the Commission that an

interoffice task group chaired by IE had been established to develop the set of PIs and that the

staff would provide a final proposal by the end of September 1986.

In SECY-86-317, ìPerformance Indicators,î dated October 28, 1986, the staff described the

efforts of the interoffice task group to review previous work, select indicators, conduct a trial

program, prepare a sample report, and develop an implementation plan. The task group

developed a list of desired attributes of PIs (see Attachment 1) and used a logic model (low

frequency of transients, high reliability/availability of safety systems, and low frequency of

common-cause failures) to identify potential indicators. The group selected 17 indicators to be

used in a trial program for 50 plants. The due date of the final proposal was extended to the 
-3-

end of October 1986 to allow the trial program to be expanded to include all plants. The task

group then selected 8 of the 17 indicators for implementation. These were as follows:

1. automatic scrams while critical (identical to the INPO indicator)

2. safety system actuations (identical to the INPO indicator)

3. significant events

4. safety system failures

5. forced outage rate

6. maintenance backlog (identical to the INPO indicator)

7. enforcement action index

8. equipment forced outages per 1000 critical hours

Throughout this effort the task group sought industry feedback and communicated often with

INPO. As a result of industry comments, the staff decided to remove the maintenance backlog

indicator from the set presented to the Commission and to place a high priority on collecting

additional data to determine the best means of monitoring maintenance effectiveness.

The task group concluded that this program would provide an objective view of operational

performance and would enhance the staffís ability to recognize changes in the safety

performance of plants. The task group considered the program a tool to be used in

conjunction with other information to provide input to management decisions regarding the

need to adjust regulatory programs. The staff briefed the Commission on the recommended

program in November 1986.

The Commissionís December 30, 1986, SRM in response to SECY-86-317 approved the PI

plan with several changes, including the deletion of the enforcement action index from the list

of PIs. The Commission also directed the staff to continue trying to develop new PIs

particularly in the areas of maintenance and training.

In February 1987, the first quarterly PI report was issued containing quarterly data from the

first quarter of 1985 through the fourth quarter of 1986. In the spring of 1987 NRC and INPO

developed a coordination plan for use of PIs. The plan established principles for proper use of

indicators. Collective radiation exposure (CRE) data was added to the PI program in the report

for the first quarter of 1987, using the same definition as the INPO indicator of the same name. 

Because the NRC required licensees to report radiation exposure data annually, this PI lagged

behind the other indicators by about 1 year.

In a major reorganization of the NRC staff in April 1987, IE was disbanded and the PI program

was transferred to the Office for Analysis and Evaluation of Operational Data (AEOD). AEOD

continued to provide quarterly PI reports and to improve and enhance these reports. The staff 

reached an agreement with INPO to allow the NRC to receive the CRE data that licensees

reported to INPO quarterly. This reduced the CRE lag time to one quarter. This change was

first implemented in the PI report for the first quarter of 1989. (The staff periodically compared

the data received from INPO with the data reported annually to the NRC and found that they

typically differed by 5 percent or less.)

The staff proposed a new PI based on programmatic causes of events. Adoption of the PI

was recommended to the Commission in SECY-89-211, ìPerformance Indicators - Cause

Codes,î dated July 14, 1989. Each licensee event report was to be analyzed to identify

possible deficiencies in one or more of six programmatic areas. These areas were

administrative control problems, licensed operator errors, other personnel errors, maintenance
-4-

problems, design/construction/installation/ fabrication problems, and miscellaneous. (These

six cause codes were treated as a single PI.) In an SRM dated August 10, 1989, the

Commission approved the inclusion of cause code trends in the quarterly performance

indicator report, but disapproved the staffís recommendation to display cause code deviations

from the peer group average using nuclear steam supply systems as peer groups for plant-toplant comparisons. The Commission directed the staff to develop more appropriate peer

groups. Cause code trends were first included in the PI report for the second quarter of 1989. 

The staff began developing appropriate peer groups for cause code deviations.

Meanwhile the Office of Nuclear Reactor Regulation (NRR) continued to work on an indicator of

maintenance effectiveness. The most promising method was to analyze the data reported by

licensees to the INPO Nuclear Plant Reliability Data System (NPRDS) to identify maintenancepreventable failures at each plant. Significant increases in the number of such failures (above a

preset threshold) were counted and trended over time. The graphs produced by this method

provided some useful insights into plant maintenance-related performance. The staff used the

plots in several Senior Management Screening Meetings. However, the graphs did not change

appreciably over time and thus were incapable of identifying trends in performance. In addition,

the plots were expensive to produce because they required analysts to review all NPRDS

records. After about 2 years of testing, this effort was abandoned. Also during this time, the

Office of Research (RES) worked on potential indicators of training and management

effectiveness. After several unsuccessful years in attempting to develop suitable indicators, this

effort too was abandoned.

Through experience with the cause codes, the staff observed that many types of plant events

correlated well with the refueling cycle periodicity. For example, safety system actuations were

observed to occur more often when a plant was shut down. This was attributed to the

increased maintenance and testing of actuation instrumentation during an outage. The staff

therefore initiated a study to address the effects of operating cycles on PIs. This work was

combined with the work to develop peer groups, and the result was a new peer

group/operational cycle methodology that would require a complete overhaul of the way the

indicators were calculated and displayed. A trial program was conducted in 1992 and the

results submitted to the Commission in SECY-92-425, ìPerformance Indicator Program - Peer

Group and Operating Cycle Phase Enhancements,î dated December 23, 1992. That proposal

was approved in an SRM dated March 12, 1993, and the first peer group/operational cycle

report was issued for the first quarter of 1993. Quarterly PI reports were issued through June

1993, when the frequency was changed to twice a year. After publication of the third quarter

1995 report, the frequency was changed to once a year.

AEOD was disbanded in 1999 and the PI program was transferred to RES. In 1999, new PIs

were incorporated into the ROP and a pilot program was begun in June of that year. The old

NRC PI program was then terminated and the last PI report was issued in January 2000, with

data through the third quarter 1999. Although the PI program was terminated, the database

continues to be updated for use in the Industry Trends program as described in SECY-01-0111,

ìDevelopment of an Industry Trends Program for Operating Power Reactors.î 
-5-

Staff Efforts To Work With the Industry and Other Stakeholders When Developing PIs Used in

the ROP

The ROP PIs are used in a fundamentally different way than they were used in the old AEOD

program. The AEOD PIs were a set of indicators that provided insights on individual plant

operational performance. They were used by senior NRC management as a confirmatory tool

to support regulatory decisions made as a result of the Systematic Assessment of Licensee

Performance (SALP) and the Senior Management Meeting (SMM) processes. In the ROP, PIs

provide an objective input that is used, in conjunction with inspection findings, to identify plant

performance deficiencies that warrant increased NRC regulatory response. This concept was

discussed in a public workshop in September and October of 1998 and developed by an NRC

technical framework task group in October, November, and December 1998.

With the input of internal and external stakeholders, the staff refined the technical framework

and finalized the selection of PIs to be used in the ROP. Most of the indicators that were

chosen either had been in use for some time in NRC or INPO programs or could be readily

developed from available data.

At the start of initial implementation, the staff, industry, and other stakeholders fully expected

that, as experience was gained, the PIs would evolve. Since early 1999 the staff has worked

closely with stakeholders in public meetings to review experience and make changes as

warranted in accordance with a formally established change process. The staff has held

several public workshops on the PI program, obtained insights from a Pilot Program Evaluation

Panel and an Initial Implementation Evaluation Panel representing all stakeholders, solicited

comments through Federal Register notices, and conducted surveys of stakeholders. These

efforts to involve its stakeholders has resulted in a number of beneficial refinements and

improvements in the PI program.

In addition, RES initiated a study in early 1999 to develop risk-based performance indicators

(RBPIs). The purpose of the RBPI development study was to examine the technical feasibility

of providing improved performance indicators for potential implementation in the ROP. The

Phase 1 of this study was completed in November 2001. Based on the feedback obtained

from internal and external stakeholders, several aspects of the unreliability and unavailability

indicators from the RBPI development study were identified as potential enhancements to the

current set of the ROP performance indicators. As a result, the NRC staff is currently working

with the industry to start a pilot program in mid-2002. This pilot program will include

unreliability indicators for mitigating systems in the ROP, as well as changes to the current

ROP PIs for unavailability. RES is also conducting some follow-on development work to

provide performance indicators for the containment integrity.

Staff Efforts To Identify Areas that Industry is Trending

Over the years the staff has monitored performance indicator reports from individual licensees

to look for PIs that might be suitable for NRC use. Most of these reports contained a large

number of indicators (as many as 100 or more). The majority of these indicators monitored

details of plant work processes in order to help manage these processes. Some of the

indicators that the NRC used before starting the formal NRC PI program in 1986 were of this

type. These indicators included maintenance backlog, licensed operator exam pass rate,

gross heat rate, and man-rem exposure due to maintenance. While these work process

indicators may be useful for licenseesí purposes, the staff concluded they were inappropriate

for the NRC to track because they did not have a logical connection to plant safety or/and they
-6-

could be easily manipulated, resulting in unintended consequences. In addition, those plantspecific PIs would most likely not be suitable for industry-wide use. The types of indicators

needed by regulators and plant operators are quite different, as was identified by the

International Atomic Energy Agency (IAEA) meetings on PIs in the early 1990s (see below). 

A small number of licensee indicators are higher level indicators for use by upper

management. These include the World Association of Nuclear Operators (WANO) PIs which

are more appropriate for NRC use and are, in fact, the same as or similar to those the staff

has been using.

International and Foreign Use of PIs

Nuclear Energy Agency Activities Related to Performance Indicators

The Nuclear Energy Agency (NEA) of the Organization for Economic Cooperation and

Development has been working on PIs for several years through its Committee on the Safety

of Nuclear Installations (CSNI) and its Committee of Nuclear Regulatory Authorities (CNRA). 

The CNRAís Working Group on Inspection Practices (WGIP) explored the use of indicators for

regulatory purposes at its Baltimore 2000 workshop. The summary report of that meeting

(proceedings of the International Workshop in Baltimore, MD, May 15-17, 2000) summarizes

PI use by participating countries, lists desired attributes of good PIs, and provides guidance on

using indicators.

In 1999, CSNIís Working Group on Operating Experience (WGOE) discussed the possibility of

organizing an international workshop on the use of indicators by nuclear regulators, utilities,

and researchers. As a result, NEA, IAEA, and WANO jointly sponsored a workshop in October

2000. The result of this workshop was a proposal to investigate the possibility of developing a

set of indicators for use by regulators comparable to the set of WANO indicators used by

utilities worldwide to trend and compare plant performance.

Based on recommendations from CNRA and CSNI, NEA decided to establish two task groups,

one under the CSNIís WGOE to use indicators of plant performance or external indicators to

develop indicators of regulatory effectiveness and another under the CNRA to use indicators of

regulatory performance or internal indicators to develop measures of regulatory efficiency.

The work on external indicators by the WGOE used input from regulators from Finland,

France, Spain, Sweden, the United Kingdom, and the United States. The group found that six

performance measures were being monitored by most participating countries:

1. number of scrams

2. availability of safety systems

3. reactor coolant system integrity

4. collective radiation exposure

5. power reductions

6. fuel integrity

Because different countries defined these PIs differently, they were not considered ready for

international use. The results of the WGOEís work were presented in NEA/CSNI/R(2001)11,

ìPerformance Indicators Report.î The task group continues to work to develop common

definitions. Because the indicators will be used internationally, it is likely that the definitions will

be similar to those used by WANO.
-7-

IAEA Activities on PIs

The IAEA has been working on the development of indicators for over 10 years. In March

1990 the IAEA convened the first of a series of meetings on PIs that were attended by a few

regulators but mostly by plant operators (about two-thirds of the participants). As a result of

these meetings it became apparent, with respect to PIs, that the needs of regulators were

different from the needs of plant operators. In 1995 the IAEA began a program to develop a

set of international PIs to be used by plant operators to manage processes in the plant. The

PIs were tested in a pilot program at four nuclear plants of different designs in different

countries. Several documents were published as a result of this program, the latest being

IAEA TECDOC 1141, ìOperational Safety PIs for Nuclear Power Plants.î This document was

intended to assist plant operators in developing and implementing a set of safety PIs suitable

for monitoring and trending plant safety performance. There are 50 to 60 of these PIs,

depending upon the individual plant needs. They are low-level indicators, measuring number

of corrective work orders issued, number of hours in simulator training for licensed operators,

etc. Although, these indicators are useful for licensee management, they are not suitable for

NRC use.

Ontario Powerís Public Dose PI

The radiological impact of the operation of nuclear power plants on the environment has

historically been stated in terms of the effect on peopleís health, with the assumption that if

people are safe, then all other species are also reasonably assured of being safe. Ontario

Power Generation (OPG) calculates and reports a dose to the public for each OPG site. The

indicator is calculated quarterly and reported as a year-to-date value for each site in

microSieverts (Sv). The Atomic Energy Control Board (AECB) of Canada and OPG recognize

a dose rate of 50 Sv (5 millirem)/year as being of minimal health risk to an individual member

of the public. The staff at each site calculates a dose to the public based on both station

emissions data and on-site environmental radiological monitoring data. Obtaining the latter

requires the collection of air, water, milk, fish, soil, and vegetation samples and a complex

mathematical calculation. The dose typically applies to persons living just outside the site

boundary who are at their residence 24 hours a day, drink local water and milk, and eat local

fish and produce. The OPG PI is very similar to the NRCís ROP PI. Both use calculations

based on effluent release data and both are reported quarterly. Both calculate the dose to a

member of the public at the site boundary. The difference is that the Canadians also calculate

doses to the public based on environmental monitoring data. This usually produces much lower

doses than those calculated from effluents. 

PIs in Finland

The Radiation and Nuclear Safety Authority in Finland (STUK) began developing a PI system

in the mid 1990s. The goal of the project was to develop a tool that could be used to provide

quantitative indications of both the levels of plant safety and the efficiency of regulatory

activities. Plant safety PIs are based on the defense-in-depth principle and are organized into

three areasósafety and quality of normal operations, operating events, and barrier integrity. 

PIs measuring the efficiency of regulatory activities are grouped according to STUKís main

activities.

STUK uses plant safety PIs as a tool to gain insights into plant performance that goes beyond

the inspection results and safety assessments. STUK has used them to focus regulatory

resources on areas that have shown degrading safety performance. In addition, indicators
-8-

have occasionally provided the impetus for meetings between plant and regulatory

management. There are no thresholds associated with the PIs; however, some internal goals

have been set. Plant PIs are published in the annual report of STUKís Office of Nuclear

Reactor Regulation. Every plant safety PI has an owner who is responsible for data collection,

calculation, analysis, and reporting of results. Indicators are calculated and reported annually. 

The PI system has a coordinator who prepares an annual summary report based on the

ownersí assessment results and who makes proposals to management for further analysis or

regulatory measures.

One of STUKís goals in developing its PI system was to minimize resources needed for

calculating the PIs. To some extent, STUK has achieved this goal by providing simple

definitions and by using data that licensees submit to STUK according to written reporting

requirements. STUK uses the indicator system as a tool to collect and present quantitative

data for trending performance in each monitored area. However, there are some indicators,

such as the number of single human errors and common-cause failures (human originated and

technical) and the risk-based indicators for operational events, for which calculation is more

complicated and requires expert judgment. Therefore, these indicators cannot be considered

as objective as the others. However, STUK allows and takes into account a certain amount of

inaccuracy when making conclusions based on the indicator results. See Attachment 2 for a

more detailed description and a complete list of the STUK PIs.

The purpose of the STUK PIs (to provide quantitative indications of both the level of plant

safety and the efficiency of regulatory activities) differs from the purpose of the ROP PIs (to

provide an indication of plant safety performance to identify the need for increased regulatory

engagement). The two PI programs differ largely because of these differences in purpose. 

The majority of the STUK PIs are not directly related to risk (e.g., the ratio of preventive to

corrective maintenance and the number of Technical Specification non-compliances). The

more risk-informed STUK indicators are largely included within the ROP PIs in some way (e.g.,

safety system unavailability and reactor coolant activity). Two of the three STUK PIs that are

risk-informed but not included in the ROP PIsórisk-significance of operating events and fire

safetyó are not included because of the low frequency of high risk-significant events and the

difficulty of measuring the magnitude of the risk in a PI. Instead the ROP monitors less risksignificant events that the NRC considers to be precursors to high-risk-significant events

through such PIs as unplanned scrams, scrams with loss of normal heat removal, unplanned

power changes, and safety system functional failures. In addition, events of high risksignificance are reviewed through the inspection program and the Significance Determination

Process (SDP) and through the events assessment and Accident Sequence Precursor

programs. The third risk-informed STUK PI that is not included in the ROP

programócontainment leakageówas in the ROP program but was removed because the

different methods of measuring containment leakage made plant-to-plant comparisons difficult. 

The staff will continue efforts to develop improved barrier integrity PIs.

PIs in Spain and Sweden

The Spanish nuclear regulatory body (CSN) has used a PI system since the mid 1990s. The

system in use is based on the one used by the NRC before the new ROP. It enabled CSN to

compare the performance of Spanish and U.S. plants of similar technology and vintage and to

identify causes for the differences in the performance results. When NRC changed its system
-9-

in 2000, CSN designed a new system that built on the experience of the existing one. The

parameters monitored are shown in Attachment 3.

The Swedish nuclear regulatory body (SKI) is also developing a PI system. The system is

based on different barrier functions in different stages of defense in depth. The structure of

the SKI system is descriptive and includes the most essential aspects and barriers of defense

in depth. However, some of the most interesting indicators, including some using probabilistic

safety assessment (PSA) techniques, are still under development. The structure of and

indicator areas in the SKI PI system are presented in Attachment 4.

The CSN PIs are very similar to the ROP PIs and the old AEOD PIs. The NRC discontinued

some of the AEOD PIs because they did not add important risk information to the ROP PIs. 

SKIís risk-informed PIs are also similar to the ROP indicators. However, a number of the SKI

PIs are not as risk-informed and, therefore, are not included in the ROP program.

CONCLUSION: 

In conclusion, the history of the staff formally utilizing PIs began over 15 years ago, in

response to direction from the Commission. During those years and up to the incorporation of

PIs into the ROP, PIs were used primarily as a confirmatory tool to support regulatory

decisions made in the SALP and SMM process. With the development of the ROP, PIs have

been used as an objective input, in conjunction with inspection findings, to identify plant

performance deficiencies. The staff has benefitted from its interactions with its domestic and

international stakeholders, although many international stakeholders use PIs for purposes that

differ from PI use in the ROP. The staff will continue to interact with both its domestic and

international stakeholders to gain insights for making future PI improvements. Development of

a 

replacement indicator for mitigating system unavailability is an example of a current PI

improvement initiative. Planned near-term improvements include development of PIs for 

mitigating system unreliability and containment integrity.

/RA by William F. Kane Acting For/

William D. Travers

Executive Director

 for Operations

Attachments: 1. Desired Attributes of Performance Indicators

2. STUK Indicators

3. Performance Indicators in Spain

4. Performance Indicators in Sweden
-9-

PIs in Spain and Sweden

The Spanish nuclear regulatory body (CSN) has used a PI system since the mid 1990s. The

system in use is based on the one used by the NRC before the new ROP. It enabled CSN to

compare the performance of Spanish and U.S. plants of similar technology and vintage and to

identify causes for the differences in the performance results. When NRC changed its system

in 2000, CSN designed a new system that built on the experience of the existing one. The

parameters monitored are shown in Attachment 3.

The Swedish nuclear regulatory body (SKI) is also developing a PI system. The system is

based on different barrier functions in different stages of defense in depth. The structure of

the SKI system is descriptive and includes the most essential aspects and barriers of defense

in depth. However, some of the most interesting indicators, including some using probabilistic

safety assessment (PSA) techniques, are still under development. The structure of and

indicator areas in the SKI PI system are presented in Attachment 4.

The CSN PIs are very similar to the ROP PIs and the old AEOD PIs. The NRC discontinued

some of the AEOD PIs because they did not add important risk information to the ROP PIs. 

SKIís risk-informed PIs are also similar to the ROP indicators. However, a number of the SKI

PIs are not as risk-informed and, therefore, are not included in the ROP program.

CONCLUSION: 

In conclusion, the history of the staff formally utilizing PIs began over 15 years ago, in

response to direction from the Commission. During those years and up to the incorporation of

PIs into the ROP, PIs were used primarily as a confirmatory tool to support regulatory

decisions made in the SALP and SMM process. With the development of the ROP, PIs have

been used as an objective input, in conjunction with inspection findings, to identify plant

performance deficiencies. The staff has benefitted from its interactions with its domestic and

international stakeholders, although many international stakeholders use PIs for purposes that

differ from PI use in the ROP. The staff will continue to interact with both its domestic and

international stakeholders to gain insights for making future PI improvements. Development of

a 

replacement indicator for mitigating system unavailability is an example of a current PI

improvement initiative. Planned near-term improvements include development of PIs for 

mitigating system unreliability and containment integrity.

/RA by William F. Kane Acting For/

William D. Travers

Executive Director

 for Operations

Attachments: 1. Desired Attributes of Performance Indicators

2. STUK Indicators

3. Performance Indicators in Spain

4. Performance Indicators in Sweden

WITS 200100087

Accession #ML020150056 QTE 1/18/02

To receive a copy of this document, indicate in the box: "C" = Copy without enclosures "E" = Copy with enclosures "N" = No copy

OFC: NRR/IIPB NRR/IIPB NRR/IIPB NRR/IIPB

NAME: DEHickman PTiipanna MASatorius MRJohnson

DATE: 1/15/02 1/15/02 1/15/02 1/15/02

OFC: NRR/DIPM NRR/OD NRR/OD EDO

NAME: BABoger BBorchardt SJCollins (RWBorchardt for) WDTravers

DATE: 1/24/02 1/28/02 1/28/02 1/ /02

OFFICIAL RECORD COPY
Desired Attributes of Performance indicators

1. The PIs should be related to nuclear safety and regulatory performance.

2. The data should be readily available to the NRC in a timely manner.

3. The data should not be susceptible to manipulation.

4. The data should be comparable between licensees.

5. The indicators should be worthy goals for licensees.

6. The indicators should reflect a range of performance.

7. The indicators should be independent of each other.

8. The indicators should be leading (i.e., predictive of future performance).

9. The set of PIs should be broad enough for correlation with SALP.

Attachment 1
STUK INDICATORS

Safety of Nuclear Installations

Safety and Quality Culture

A. Component failures and their repairs

1. Number of component failures in technical specification (TS) systems

2. Ratio between preventive and corrective maintenance activities on TS systems

3. Unavailability time of TS components compared to allowed outage times (AOTs)

4. Number of single and multiple maintenance errors (Common Cause Failures

[CCFs])

5. Number of technical CCFs (critical and potential)

B. Number of Technical Specification Deviations

1. Number of noncompliances with TSs

2. Number of TS exemptions

C. Radiation Doses

1. Annual collective dose

2. Annual average of 10 highest doses

D. Safety System Unavailability

1. Unit-specific WANO indicators

E. Radioactive Releases

1. Releases to the atmosphere

2. Releases to the water system

3. Calculated dose, based on releases, to the most exposed person living in the

vicinity of the plant

F. Updating of Plant Documentation

1. Annual rate of plant modifications that are not incorporated into design basis

documents by the time of the next outage following the modification

Attachment 2
-2-

G. Amount of Investments to Plant

1. Annual investment rate in plant modernization

Operating Events

A. Number of reported events

B. Risk significance of operating events

C. Direct causes of events

1. Number of events caused by organizational factors

2. Number of events caused by technical factors

D. Number of Fire Alarms

1. Number of malfunctions

2. Number of real fire alarms

3. Number of fires

4. Number of other rescue missions

Barrier Integrity

A. Fuel integrity

1. Maximum activity of the reactor coolant system in dose-equivalent I-131

2. Number of fuel failures (leaking fuel bundles)

B. Primary System Integrity

1. Volume of identified and unidentified leakage

2. WANO chemistry index

C. Containment Integrity

1. Overall leakage of isolation valves compared to allowed leakage of isolation valves

2. Percentage of isolation valves that passed the first leakage test

3. Overall leakage of containment penetrations and personnel air locks compared to

the allowed overall leakage
-3-

STUKís Indicators of Safety and Quality Culture

STUK has indicators that are used to monitor the quality of maintenance of safety-significant

as well as non-safety-significant systems. For example, STUK monitors common-cause

failures for all systems. For safety-significant systems, STUK monitors the number of

component failures, the average percentage of collective component unavailability time

compared to allowed unavailability time, PRA-based risk significance of component

unavailabilities, and the WANO safety system unavailability indicators. These indicators are

somewhat overlapping. STUK uses them to measure the quality of maintenance and

operations and to describe the safety significance of safety system unavailabilities.

STUK also monitors the quality of operations by counting the number of deviations from TSs.

The deviations include plant configurations and plant operating parameters that are not in

compliance with TSs and noncompliances with administrative rules in the TSs. In addition,

STUK monitors the number of exemptions (approved by STUK) from TSs.

The indicators that STUK uses to monitor radiation doses and radioactive releases are the

same used worldwide. The radiation dose indicator describes collective radiation doses (manSv) of workers and is used for dose trending and public health purposes. In addition, STUK

monitors the average of the 10 highest worker doses. Indicators of radioactive releases

monitor the actual amount of radioactive effluents released into the environment. STUK also

has an indicator to monitor the dose of the most exposed person living in the vicinity of the

plant. The dose is calculated by conservative methods and is based on the actual releases. 

STUK also has requirements to calculate the dose of the most exposed person by analyzing

environmental measurement data from air, water, food, vegetation, and soil. However, this

calculation has not been possible due to small amounts of radionuclides in the environment. 

STUK has recently included two new indicators in its PI system. The first is an indicator that

monitors the quality and timeliness of the plant documentation updating process as a part of

the management of plant modifications. Another new indicator is the rate of investment in the

plant. It was included to follow the effects of electricity market liberalization on plant

modernization. These indicators are not directly related to risk, and setting objective and

risk-informed thresholds for them is difficult.

STUKís Indicators for Operating Events

STUK has indicators that monitor the number of operating events, the risk significance of

operating events, direct causes of operating events, and the number of fire alarms. The number

of operating events is the number of operating event reports submitted to STUK according to the

reporting requirements. There are three categories of reports: special event reports, reactor

scram reports, and operational disturbance reports. Direct causes of operating events are 

either organizational or technical. These PIs are trended to indicate the performance of the
-4-

Σ%

CDF

CDF

T

T i % iuna iuna = 



 



 − 



 



1 100 ⋅ ⋅

plant and of the operating organization. The number of fire alarms indicator is used to trend the

number of fires, fire alarms, and fire detection system malfunctions.

STUK has performed follow-up risk studies for operational events to explore the risk

significance of different types of operating events, and as a result of these studies, has

developed a risk-based indicator to monitor operating events. The indicator is calculated by

using living PRA models and computer codes. Basically, the indicator identifies the risk

significance of the unavailability of components due to various causes. The indicator is defined

as the annual sum of the increases in risk above the baseline risk due to component

unavailability. Results are presented as a percentage of the baseline risk in a normal plant

configuration. The equation and figure below clarify the calculation of the risk indicator:

where

i = component unavailability in question

CDF = mean (baseline) Core Damage Frequency

CDFiuna = new Core Damage Frequency during component

unavailability

Tiuna = component unavailability time

T = total time when reactor is critical during the year

Figure 1. The shadowed area is the additional risk caused by a safety significant

system component unavailability. The annual indicator is the sum of all individual

unavailabilities during the year.

This is the most risk informative indicator in STUKís indicator system and STUK has allocated

significant resources to its development and calculation. With this indicator, STUK follows the

risk of operating events in four categories: the risk due to component failures, to component

on-line preventive maintenance, to events, and to exemptions from TSs. The indicator takes
-5-

into account simultaneous unavailabilities of components due to any cause, dependencies

between primary systems and their support systems, unavailability time, and the statistical

unreliability of other systems. The calculation of the indicator has enabled STUK to identify

risk-significant operating events, systems, components and plant configurations, and to identify

limitations in PRA and areas of PRA that need to be developed. STUK has set an internal goal

for this indicator that the risk-significance of operating events should not exceed 5% of the

normal annual risk. However, this indicator includes uncertainties that need to be taken into

account when assessing results. Some of the uncertainties are the result of the inherent

uncertainties of all PRA results. In addition, there are uncertainties related to the modeling of

unavailabilities. Due to limitations of PRA models, modeling of unavailabilities of components

and systems is not always straightforward and conservative expert judgment needs to be used.

STUKís Indicators of Barrier Integrity

For barrier integrity, STUK uses indicators that are similar to those used in the ROP. These

include integrity of nuclear fuel (I-131 activity in reactor coolant system) and integrity of reactor

coolant system (RCS) (volume of identified leakage). STUK also trends the number of fuel

failures during operating cycles to get more insights into fuel quality and activity levels in the

RCS. In addition to identified leakage of the reactor coolant system, STUK is monitoring the

volume of unidentified leakage compared to the TS limit. 

STUK has indicators that monitor the leak-tightness of the containment. One of them is the

overall leakage of the isolation valves compared to the leakage allowed in the TSs. Another is

the leakage rate of containment penetrations and personnel air locks compared to the allowed

leakage rate. The leakage rate is measured when isolation valves and containment

penetrations are tested during the outage. These indicators provide information regarding the

status of the containment leak-tightness during the previous operating cycle. In addition to

these indicators, STUK monitors the number of leaking valves (percentage of the isolation

valves that did not passed the first leakage test), giving more insights into the containment leaktightness PI data (whether the cause of leakage is one or several leaking valves). 
PERFORMANCE INDICATORS IN SPAIN 

Performance Stability and Reliability of Mitigating Systems

A. Unit capability factor

B. Nonscheduled shutdowns/year

C. Number of scrams/7000 critical hours (manual and automatic scrams)

D. Nonscheduled safety systems actuations/year

E. Safety system failures/year

F. Safety system unavailability/year

Barrier Integrity

A. Reactor coolant system activity

B. Reactor coolant system identified leakage

Radiological Impact

A. Collective radiation exposure to workers

B. Volume of low- and medium-level solid radioactive waste

C. Activity of gaseous radioactive releases

D. Activity of liquid radioactive releases

Attachment 3
PERFORMANCE INDICATORS IN SWEDEN 

Prevention of Abnormal Failures

A. Robust design and construction

1. Number of fuel failures

2. Primary pressure boundary

B. High quality in maintenance and operations

1. Rate of violations of technical specifications

2. Rate of maintenance problems (repeated or overdue maintenance)

C. Initiating events

1. Number of scrams

2. Number of safety system initiations

Control of Abnormal Operation and Detection of Failures

A. Robust supervision systems

1. Unavailability of supervision and protection systems

2. Number of incidents with failing system during scram

Control of Accidents within Design Basis

A. Effective safety systems

1. Unavailability of safety systems

2. Unavailability of separating barriers

3. Number of leaking containment isolation valves

B. High quality in maintenance and operations

C. Effective emergency operating procedures

Attachment 4
-2-

Control of Severe Plant Conditions, Including Prevention of Accident Progress, and Mitigating of

the Consequences of Severe Accidents

A. Consequence mitigating systems

1. Unavailability of consequence mitigating systems

2. Unavailability of supervision systems

B. High quality in maintenance and operations

C. Physical protection

D. Effective accident management

Mitigation of Radiological Consequences of Significant Releases of Radioactive Materials

A. Preparatory measures for effective dissemination of information to protect the population

Global Safety

A. Unplanned loss of production