POLICY ISSUE

INFORMATION

July 18, 2003 SECY-03-0122

FOR: The Commissioners

FROM: William D. Travers 

Executive Director for Operations

SUBJECT: STATUS REPORT ON DRAFT REGULATORY GUIDE, DG-1122 “AN

APPROACH FOR DETERMINING THE TECHNICAL ADEQUACY OF

PROBABILISTIC RISK ASSESSMENT RESULTS FOR RISK-INFORMED

ACTIVITIES,” AND DRAFT STANDARD REVIEW PLAN CHAPTER 19.1,

“DETERMINING THE TECHNICAL ADEQUACY OF PROBABILISTIC RISK

ASSESSMENT RESULTS FOR RISK-INFORMED ACTIVITIES”

PURPOSE:

To provide the Commission with the status of Draft Regulatory Guide DG-1122, “An Approach

for Determining the Technical Adequacy of Probabilistic Risk Assessment Results for RiskInformed Activities,” including the status of Draft Standard Review Plan (SRP) Chapter 19.1,

“Determining the Technical Adequacy of Probabilistic Risk Assessment Results for RiskInformed Activities,” and the status of the staff’s endorsement of the probabilistic risk

assessment (PRA) standards and the industry’s peer review program.

BACKGROUND:

PRA standards have been under development by the American Society of Mechanical

Engineers (ASME) and the American Nuclear Society (ANS). On April 5, 2002, ASME issued a

standard for a full-power, internal events (excluding fire) Level 1 (i.e., core damage frequency

(CDF) analysis) PRA and a limited Level 2 (i.e., large early release frequency (LERF) analysis)

PRA. ANS is developing standards for PRAs for evaluating external events and internal fire

and risk from low-power and shutdown modes of operation. In addition, reactor owners’ groups

have been developing and applying a PRA peer review program for several years that

addresses a Level 1 and a limited Level 2 PRA for full-power operation only and for internal

events only excluding internal floods and fires.

Contact:

Mary Drouin, RES/DRAA

415-6675
The Commissioners 2

In SECY-02-0070 (April 24, 2002), the staff indicated its plan to develop a new Regulatory

Guide and Standard Review Plan chapter that would provide guidance to licensees and the

staff, respectively, on how to use the standards and other industry programs in evaluating the

technical appropriateness of PRA results for risk-informed applications. In November 2002, the

staff published DG-1122, along with the associated Draft SRP Chapter 19.1 for public review

and comment. This paper provides a status of DG-1122, SRP Chapter 19.1, the PRA

standards development, and industry peer review program, and stakeholder interaction on

these items.

DISCUSSION:

Draft Regulatory Guide DG-1122 and Draft SRP Chapter 19.1

The purpose of DG-1122 is to describe one acceptable approach for determining that the

quality of the PRA (in toto or for those parts that are used to support an application) is sufficient

to provide confidence in the results such that they can be used in regulatory decision making

for light water reactors. It is also intended to reflect and endorse guidance provided by

standards-setting and nuclear industry organizations. As noted in DG-1122, guidance is

provided in four areas:

(1) A minimal set of functional requirements of a technically acceptable PRA.

(2) The NRC position on consensus PRA standards and industry PRA program documents.

(3) Demonstration that the PRA (in toto or specific parts) used in regulatory applications is of

sufficient technical adequacy.

(4) Documentation that the PRA (in toto or specific parts) used in regulatory applications is of

sufficient technical adequacy.

DG-1122 guidance is for a “full-scope” PRA that includes a Level 1 (i.e., CDF) analysis and a

Level 2 (i.e., radionuclide release frequency) analysis, addressing all operating modes (i.e., fullpower, low-power, and shutdown), and internal (transients, loss of coolant accidents, floods and

fires) and external (seismic, high winds, floods) events. The attributes and characteristics

provided for each technical element prescribe a PRA that reflects:

• the as-built and as-operated plant, 

• the operating history of the plant,

• the plant-specific response to an event, and

• the sources and impacts of key uncertainties.

SRP Chapter 19.1 provides guidance to the staff on how to assess the adequacy of the PRA

results used to support an application. It focuses the staff review on aspects of the PRA model

that are not performed in accordance with the standards (as modified to represent the staff

position), and on the assessment of the assumptions that have been demonstrated to be critical

to the application.

DG-1122 does not provide guidance on how PRA results are used in the application-specific

decisionmaking processes; that guidance is provided in such documents as:
The Commissioners 3

• RG 1.174, “An Approach for Using Probabilistic Risk Assessment in Risk-Informed

Decisions on Plant-Specific Changes to the Licensing Basis”

• RG 1.175, “An Approach for Plant-Specific, Risk-Informed Decisionmaking: Inservice

Testing”

• RG 1.177, “An Approach for Plant-Specific, Risk-Informed Decisionmaking: Technical

Specifications”

• DG-1121, “Guidance for Categorizing Structures, Systems, and Components in Nuclear

Power Plants According to Their Safety Significance”

Consequently, DG-1122 is a supporting document to other NRC regulatory guides that address

risk-informed activities. The attached figure shows the relationship of DG-1122 and riskinformed activities, application-specific guidance, consensus PRA standards, and industry

programs.

Stakeholder Interaction

The staff has interfaced with various stakeholders on a continual basis. This interaction has

involved:

• direct participation with ASME and ANS in the development of the standards

• observing industry PRA peer reviews

• numerous public meetings and workshops in developing DG-1122 and SRP Chapter 19.1

• briefings to the Advisory Committee on Reactor Safeguards (ACRS).

As noted above, DG-1122 and SRP Chapter 19.1 were issued for public review and comment in

November 2002. Comments were received from six different organizations of which the

majority of the comments concerned staff positions on the ASME standard in Appendix A (see

discussion below on ASME standard). Minor comments (e.g., editorial in nature) were

submitted on Appendix B (staff positions on NEI-00-02) of DG-1122 (see discussion below on

industry peer review program), and no comments were submitted on SRP Chapter 19.1. There

was a consensus among the stakeholders that the staff should move forward to publish the

regulatory guide “for trial use” and test the guide via pilot applications. South Texas Nuclear

Electric Generating Station has submitted a letter of intent indicating their desire to participate

in a pilot; other licensees have expressed interest.

The ACRS, on April 21, 2003, issued a letter to the staff with recommendations on DG-1122. 

Three of the recommendations supported the staff position regarding the ASME standard (see

discussion below on ASME standard). The remaining three recommendations were:

• providing guidance on how to perform sensitivity and uncertainty analyses.

• clarifying how the capability categories are consistent with the provision that the event

probabilities reflect the actual operating history and experience of the plant as well as

applicable generic experience.

• providing guidance on acceptable qualitative characterization of risk contributions not

calculated in limited-scope PRAs.

The ACRS, in a subsequent letter on May 16, 2003, “Improvement of the Quality of Risk

Information for Regulatory Decisionmaking,” provided the staff with three similar
The Commissioners 4

recommendations. The staff agreed in responses to both ACRS letters, to include guidance, as

appropriate, on these items in DG-1122.

On June 13, 2003, an industry peer review of the San Onofre Nuclear Generating Station

(SONGS) PRA was completed. NRC staff were present during the peer review, but as

observers only. This peer review was the first review to use the ASME standard as the basis in

evaluating a PRA. This peer review, therefore, both evaluated the SONGS PRA, and tested the

ASME standard. Based on the staff observations, the staff provides the following comments:

• The ASME standard needs additional guidance in interpreting and applying some of the

requirements. The peer review team intends to provide this feedback to ASME. The staff

also intends to add this guidance, where appropriate, in DG-1122.

• The industry members believed that the standard had “raised the bar” with respect to PRA

quality. While they did not necessarily believe that this was inappropriate, they believed

that the consequences are that each licensee will have to modify its PRA to some degree,

some more than others, to meet what the standard represents as “current good practices.”

(A set of principles and objectives were established by ASME (see DG-1122). One

principle states that “The standard should be based on current good practices.”)

Standards Development and Industry Peer Review Program

ASME and ANS are the two professional societies currently writing the PRA standards. ASME

is responsible for the development of a PRA standard for a Level 1 analysis and a limited

Level 2 analysis, considering full-power operation and internal events (excluding internal fire). 

ANS is responsible for PRA standards for external events, low-power shutdown, and internal

fires. The ASME and ANS standards are intended to be used together to cover the different

aspects of PRA scope. 

The status of the ASME and ANS PRA standards is discussed below:

• ASME standard: ASME RA-S-2002 (“Standard for Probabilistic Risk Assessment for

Nuclear Power Plant Applications”) was published on April 5, 2002. The staff position was

published for public review and comment in Appendix A of DG-1122 in November 2002. 

Subsequently, the staff and the ASME standard’s committees have achieved resolution 

the staff’s positions except on three significant items. It is the staff’s understanding that

the resolutions will be incorporated into the standard in an addendum to be published in

September 2003. The three items of disagreement include:

(1) The Definition and Usage of the Terms Dominant, Important, Key and

Significant. These terms are used in an inconsistent manner in the ASME standard,

and used to determine whether a requirement is necessary. The staff believes a

definition is needed and one that removes as much subjectivity as possible. The

ASME standard’s committees have acknowledged the inconsistencies in the usage of

these terms, and plans to address the resolution in the addendum. The ASME

standard’s committees, however, do not agree that a quantitative definition is needed,

and therefore, does not intend to include one in the addendum. The staff position

includes quantitative definitions and it is the staff intent to test the definitions in pilot
The Commissioners 5

applications of the guide. The ACRS in their April 21 2003, letter to the Commission,

notes that the terms are “critical to the application of the standard” and that “clear

definitions of the terms. . . should be included in the draft final Regulatory Guide

before issuing it for trial use.”

(2) Peer Review Assessment of Key Assumptions. The ASME standard requires a

peer review as part of the process. The purpose of the peer review, as stated in the

standard, “is to assess the PRA to the extent necessary to determine if the

methodology and its implementation meet the requirements of this standard.” To

achieve this purpose, the staff believes that the peer review needs to evaluate the

validity of the key assumptions in the PRA. The ASME standard’s committees do not

believe it is necessary to add such a statement to the purpose. The staff position on

the purpose of the peer review includes an assessment by the peer review team to

evaluate the validity of the key assumptions. The ACRS in their April 21, 2003, letter

to the Commission, notes that “PRAs rely on numerous assumptions that are often

critical to the validity of the results. . . . We agree with the staff that such an

assessment should be required.”

(3) Topics to be Peer Reviewed. The ASME standard provides a list of “suggestions”

for the peer review team to consider for review. The ASME standard’s committees

believe that the “peer review teams must be allowed to select the scope and level of

detail for the review and not be bound by prescriptive requirements. A peer review is

not an audit.” The staff agrees, however, the staff believes that the standard needs to

provide a minimum set of topics for the peer review team. The staff does not believe

that a minimum required set of topics is prescriptive, nor makes the review an audit. 

The staff believes that a minimum set ensures the topics are examined, provides for

uniformity and consistency between peer reviews, and allows the team flexibility in

determining the scope and level of detail of each topic based on the peer reviewer

expertise. The staff position includes the suggested list in the standard as topics that

the peer review team needs to review. The ACRS in their April 21, 2003, letter to the

Commission, notes that “The staff argues that these suggestions should, in fact, be

requirements. . . We agree. . . .”

• ANS Standard - External Events: ANS has completed the standard for external events

which addresses seismic, high winds, and floods. The standard is in the process of being

published, and it is the staff’s understanding that issuance is imminent (July 2003).

• ANS Standard - Low Power Shutdown: It is the staff’s understanding that a draft standard

for public review and comment is to be issued by December 2003.

• ANS Standard - Internal Fire: It is the staff’s understanding that a draft standard for public

review and comment is to be issued by September 2004.

NEI issued NEI-00-02 (“Probabilistic Risk Assessment (PRA) Peer Review Process Guidance”),

which is a process for a PRA peer review and it contains subtier criteria that the peer review

team uses to judge the technical adequacy of the PRA. This process is for a peer review of a

Level 1 and a limited Level 2 PRA for full-power operation and internal events (excluding

internal floods and fires) only. The ASME standard (which references NEI-00-02 as an
The Commissioners 6

example of an acceptable review methodology) was issued after the peer reviews had been

performed on the majority of the industry’s PRAs. The staff noted that there are discrepancies

between the ASME standard and the subtier criteria in NEI-00-02. Consequently, NEI

submitted draft industry guidance for self-assessment. The purpose of this self-assessment is

to provide guidance to industry “for using the results of existing PRA peer reviews (per NEI-00-

02), along with supplemental self-assessments, to satisfy the peer review requirements of the

ASME standard as endorsed by the NRC.” This self-assessment includes guidance for the

process, a comparison of the subtier criteria in NEI-00-02 against the requirements in the

ASME standard, and the industry actions needed to address the identified discrepancies.

It is the staff’s understanding that NEI will finalize this draft guidance during the pilot application

of DG-1122 and incorporate it into a revision of NEI-00-02.

CONCLUSIONS:

The following summarizes the status, and staff's accomplishments, to date related to its effort

to address the issue of technical adequacy of PRA for risk-informed activities:

• The staff is working closely with professional societies in developing PRA standards.

• The staff is interacting frequently with stakeholders to solicit their input.

• The staff has developed DG-1122 and SRP Chapter 19.1. It is the staff’s intent to issue

the regulatory guide for trial use and test the guide in one or more pilot applications. The

staff intends to issue the guide in early fall of 2003 after ASME issues the addendum to the

ASME standard. However, if ASME delays issuing its addendum, the staff intends to move

forward with issuing the guide.

/RA by William F. Kane Acting For/

William D. Travers

Executive Director 

 for Operations

Attachment: As noted
The Commissioners 6

Level 1 and a limited Level 2 PRA for full-power operation and internal events (excluding

internal floods and fires) only. The ASME standard (which references NEI-00-02 as an

example of an acceptable review methodology) was issued after the peer reviews had been

performed on the majority of the industry’s PRAs. The staff noted that there are discrepancies

between the ASME standard and the subtier criteria in NEI-00-02. Consequently, NEI

submitted draft industry guidance for self-assessment. The purpose of this self-assessment is

to provide guidance to industry “for using the results of existing PRA peer reviews (per NEI-00-

02), along with supplemental self-assessments, to satisfy the peer review requirements of the

ASME standard as endorsed by the NRC.” This self-assessment includes guidance for the

process, a comparison of the subtier criteria in NEI-00-02 against the requirements in the

ASME standard, and the industry actions needed to address the identified discrepancies.

It is the staff’s understanding that NEI will finalize this draft guidance during the pilot application

of DG-1122 and incorporate it into a revision of NEI-00-02.

CONCLUSIONS:

The following summarizes the status, and staff's accomplishments, to date related to its effort

to address the issue of technical adequacy of PRA for risk-informed activities:

• The staff is working closely with professional societies in developing PRA standards.

• The staff is interacting frequently with stakeholders to solicit their input.

• The staff has developed DG-1122 and SRP Chapter 19.1. It is the staff’s intent to issue

the regulatory guide for trial use and test the guide in one or more pilot applications. The

staff intends to issue the guide in early fall of 2003 after ASME issues the addendum to the

ASME standard. However, if ASME delays issuing its addendum, the staff intends to move

forward with issuing the guide.

/RA by William F. Kane Acting For/

William D. Travers

Executive Director 

 for Operations

Attachment: As noted

DOCUMENT NAME:C:\ORPCheckout\FileNET\ML031760084.wpd

June 24, 2003 (12:14PM)

Distribution: DRAA Chron, PRAB r/f, Drouin, Drouin r/f, Giitter, Newberry/Cunningham,Tschiltz, Parry,

West, Matthews/Gillespie, Black/M. Johnson, Thadani/Strosnider, Collins/Borchardt

OAR in ADAMS? (Y or N) Y ADAMS ACCESSION NO.:__ML031760084__ TEMPLATE NO. EDO-002

Publicly Available? (Y or N) Y DATE OF RELEASE TO PUBLIC SENSITIVE? N 

To receive a copy of this document, indicate in the box: "C" = Copy without attachment/enclosure "E" = Copy with attachment/enclosure "N" = No copy

*See previous concurrence

OFFICE DRAA/PRAB Tech Editor DRAA/PRAB DRAA

NAME Drouin* ABerank* Giitter* Cunningham/Newberry*

DATE 06/ 5 /03 06/16/03 06/ 13 /03 06/ 13 /03

OFFICE NRR RES DEDO EDO

NAME Black* Thadani* Paperiello Travers

DATE 06/ 20 /03 06/ 24 /03 07/17/03 07/18/03
Attachment 1

Relationship of DG-1122 to Other

Risk-Informed Guidance