Level 3 PRA Project QA Plan, Rev. 0

Level 3 PRA Project

QA Plan

Rev. Date Description

0 08/11/2016 Original issuance*

*Earlier versions of the Level 3 PRA project QA plan appear as Section 18 of the Level 3 PRA 

project technical analysis approach plan (ADAMS Accession No. ML13296A064)
Level 3 PRA Project QA Plan, Rev. 0

.
Level 3 PRA Project QA Plan, Rev. 0

i

Contents

List of Acronyms ........................................................................................................................iv

1 Established Methods, Tools and Data .................................................................................. 1

2 Qualified Personnel.............................................................................................................. 1

3 PRA Model Configuration Control......................................................................................... 2

4 Technical Reviews ............................................................................................................... 3

4.1 Technical Advisory Group............................................................................................ 3

4.2 Internal Self-Assessment............................................................................................. 5

4.3 External Peer Reviews .............................................................................................. 16

4.3.1 PRA Standard Peer Review ........................................................................ 16

4.3.2 Independent Expert Peer Review................................................................ 18

4.4 Advisory Committee on Reactor Safeguards ............................................................. 19

4.5 Public Review and Comment..................................................................................... 19

5 Documentation Control....................................................................................................... 19

5.1 Storage and Access of Project Information................................................................ 20

5.2 Upload of Information onto the SharePoint Site......................................................... 21

5.3 Documentation Control of Licensee Information ........................................................ 21

5.4 Documentation Backup ............................................................................................. 22

5.5 Use of External Storage Media.................................................................................. 22

5.6 Working Document Folders ....................................................................................... 22

5.7 Use of Templates and Forms for Documentation....................................................... 23

5.8 Site Visits .................................................................................................................. 26

5.9 Documentation Control for NRC Contractors............................................................. 28

5.10 Non-Disclosure Agreement to Allow Access to Proprietary Information ..................... 29

5.11 Project Documentation Markings............................................................................... 29

5.12 Guidance for Addressing Potential Technical Issues ................................................. 30

5.13 Future Plant Modifications ......................................................................................... 35

5.14 Organization of the Various Types of Information on the SharePoint Site.................. 35

6 Technical Reports .............................................................................................................. 37

6.1 Staff Technical Reports ............................................................................................. 37

6.2 Contractor Technical Reports .................................................................................... 42

7 QA Program Implementation Audits ................................................................................... 43
Level 3 PRA Project QA Plan, Rev. 0

ii

LIST OF TABLES

Table 1 TAG Review Documentation Template........................................................................ 4

Table 2 Self-Assessment and Peer Review Criteria Where Standards Do Not Exist ................ 7

Table 3 Self-Assessment Documentation Template................................................................. 8

Table 4 Overall Results of Self-Assessment Process............................................................... 9

Table 5 Example: Mapping of the HLRs of SFP PRA and At-Power Level 1 PRA..................10

Table 6 Applicability of SRs of the At-Power Level 1 PRA to the SFP PRA.............................11

Table 7 External Peer Review Documentation Template ........................................................17

Table 8 Documentation of Meetings and Discussions .............................................................23

Table 9 Documentation for Level 1 Internal Events Initiating Event Analysis...........................24

Table 10 Issue Documentation Template................................................................................34

Table 11 Summary of Level 3 PRA Project Information on SharePoint Site ............................36

Table 12 Sign-Off Sheet for Staff Technical Reports...............................................................38

Table 13 Document Checklist .................................................................................................39

Table 14 QA Review and Acceptance Form for Staff (Supporting) Technical Reports.............42

Table 15 QA Review and Acceptance Form for Contractor Technical Reports........................43

LIST OF FIGURES

Figure 1 Process Used for Self-Assessment ............................................................................. 5
Level 3 PRA Project QA Plan, Rev. 0

iv

List of Acronyms

ACRS Advisory Committee on Reactor Safeguards

CDF Core Damage Frequency

COR Contracting Office Representative

DCS Dry Cask Storage

DORL Division of Operating Reactor Licensing

DRA Division of Risk Analysis

HLR High Level Requirement

IEPR Independent Expert Peer Review

INL Idaho National Laboratory

LERF Large Early Release Frequency

LRF Large Release Frequency

NRC U.S. Nuclear Regulatory Commission

NRR Office of Nuclear Reactor Regulation

PRA Probabilistic Risk Assessment

PRAB Probabilistic Risk Assessment Branch

PWR Pressurized Water Reactor

PWR OG PWR Owner’s Group

QA Quality Assurance

QC Quality Control

RES Office of Nuclear Regulatory Research

RG Regulatory Guide

SFP Spent Fuel Pool

SNC Southern Nuclear Operating Company

SPAR Standardized Plant Analysis Risk

SR Supporting Requirement

SRA Senior Reactor Analyst

SSC Structure, System, and Component

TAAP Technical Analysis Approach Plan

TAG Technical Advisory Group
Level 3 PRA Project QA Plan, Rev. 0
Level 3 PRA Project QA Plan, Rev. 0

1

QUALITY ASSURANCE PLAN

Quality Assurance (QA) is a key factor in any analysis to ensure and demonstrate the 

technical acceptability of the analysis and probabilistic risk assessment (PRA) model fidelity. 

The objective of QA is to ensure that both the technical approach (methods, tools, data) is 

appropriate, and that implementation of the technical approach is appropriately performed. 

To achieve this objective, QA involves seven major elements which are discussed in the 

following sections:

• Section 1 – Use of established methods, tools and data

• Section 2 – Qualified personnel

• Section 3 – PRA model configuration control

• Section 4 – Technical review of the methods, tools, data, and developed models

• Section 5 – Documentation control

• Section 6 – Technical reports

• Section 7 – QA program implementation audits

1 ESTABLISHED METHODS, TOOLS AND DATA

The PRA model will generally be based on state-of-practice methods, tools (e.g., computer 

codes) and data, that is, those that have been established and accepted (including 

verification and validation where applicable) in the risk community (i.e., U.S. Nuclear 

Regulatory Commission (NRC) and industry). Examples of sources include:

• Consensus standards

• Internal and external guidance documents

• Accepted generic structures, systems and components (SSCs) performance data 

(where plant specific data is not available)

• Validated codes

For each technical task1

, the method, tools and data being used will be documented along 

with the basis for their acceptability (e.g., NRC endorsement). This documentation is 

identified in each technical task in Technical Analysis Approach Plan (TAAP) report and 

described in Section 5.

2 QUALIFIED PERSONNEL

Qualified individuals are needed to perform the work. Their qualifications depend on whether 

the analyst is (1) a performer or (2) a reviewer. 

A performer is an individual who develops some aspect of the PRA model. Their role, either 

as a team leader, a task leader, or an analyst will need to have some level of expertise. 

Certainly, an analyst can develop the qualifications with on the job training; however, the task 

and team leaders need to be more experienced personnel who bring actual experience in the 

area they are leading. If an analyst has little to no experience, their work will be closely 

supervised and monitored by their task leader. PRA consensus standards and Regulatory 

 1 Technical tasks are the technical steps that will be performed to accomplish the technical element.
Level 3 PRA Project QA Plan, Rev. 0

2

Guide (RG) 1.2002 do not prescribe qualifications for the team performing the actual work. 

Moreover, one of the major objectives of the Level 3 PRA project is to train inexperienced 

staff in how to construct a PRA model. 

A reviewer is an individual who has some role in reviewing the actual work and making 

judgments with regard to its technical acceptability. In this regard, these individuals must 

have a certain level of expertise and on the job training is not acceptable. Both RG 1.200 

and the PRA standards provides peer review personnel qualifications. These requirements 

should be met unless otherwise justified.

3 PRA MODEL CONFIGURATION CONTROL

Ensuring that the analysts are using the same information and same models and that the 

reviews are being performed on the most recent model and documentation is important in 

ensuring the fidelity of the PRA model. Developing a PRA model involves numerous tasks 

being performed by many different analysts. It is, therefore, essential that the information 

collected and the models developed for this project be controlled so that all of the analysts 

use the same information and models. The control of the developed models is discussed in 

this section. The control of information is discussed in Section 5.

The Idaho National Laboratory (INL) will host and maintain the SAPHIRE-based models 

developed as part of the Level 3 PRA project. INL will provide the necessary technical 

management and oversight to ensure efforts by INL or NRC staff (including work performed 

by other NRC contractors and provided to INL by the NRC) to create, revise or otherwise 

modify the Level 3 PRA project models are coordinated and the models are properly 

integrated. These model enhancements may include the creation, addition, revision or other 

modification of a low-power/shutdown model, all-hazards model (e.g., fire, external flooding, 

seismic, etc.), Level 2 PRA model, multi-unit model, spent fuel pool model, or other extended 

model applicable to the construct of the overall Level 3 PRA project model.

To the extent practicable, the methodology, quality, and philosophy used to develop the 

current set of Standardized Plant Analysis Risk (SPAR) models for the 100 operating 

commercial nuclear power plants will be used to develop the external event model, lowpower/shutdown model, extended Level 1 PRA model, and Level 2 PRA model for the Level 

3 PRA project. This includes model construct, event nomenclature, assumptions, preferred 

technical positions, and other key aspects of the existing models to allow NRC staff the ease 

of use of the models.

INL will identify a single point of contact to act as the Level 3 PRA project model coordinator 

(“Coordinator”). The Coordinator will maintain a log and track all permanent revisions to the 

model including the reason for the revision, assumptions, deviations from preferred technical 

positions, and any other information deemed important to understanding the model or the 

revision to the model. The Coordinator will ensure that the appropriate model revision is 

being used and that the effort results in a properly integrated model. The Coordinator will 

also coordinate INL model integration activities. Version control software, suitable to this task 

and with sufficient documentation capabilities, may be used by INL, subject to approval by 

the NRC staff.

 2 Regulatory Guide1.200, “An Approach For Determining The Technical Adequacy Of Probabilistic Risk 

Assessment Results For Risk-Informed Activities,” Revision 1, U.S. Nuclear Regulatory Commission, Washington, 

DC, January 2007.
Level 3 PRA Project QA Plan, Rev. 0

3

When multiple revisions to the enhanced Vogtle model are planned by INL or NRC staff, INL 

will coordinate the activities of the different modelers. This is to ensure that the model 

developers use the appropriate model version(s) and that the final product does not include 

models that were constructed based on an obsolete model version.

INL will also perform quality control (QC) and QA reviews of the new or revised models. This 

is to ensure that the model represents the as-built, as-operated plant to the extent 

practicable. Similar QA criteria and processes used for the existing SPAR models will be 

used to review the Level 3 PRA project models. This includes (as appropriate and as 

practical) satisfying the criteria and processes in the Standardized Plant Analysis Risk 

(SPAR) Model QA Plan,

3 the latest approved INL QC/QA processes, applicable sections of 

Volume 3 of the RASP Handbook,

4 RG 1.200, and other applicable guidance.5

4 TECHNICAL REVIEWS

In ensuring technical acceptability, different types of review will be performed. These involve 

five types which are discussed in the following sections:

• Section 4.1 – review by a Technical Advisory Group

• Section 4.2 – internal self-assessment

• Section 4.3 – external peer reviews

• Section 4.4 – review by the Advisory Committee on Reactor Safeguards

• Section 4.5 – public review and comment

Each of these reviews has different objectives and scope which are described below.

4.1 Technical Advisory Group

The objective of the Technical Advisory Group (TAG), as specified in the TAG charter,6 is to: 

(1) review progress in the development of the Level 3 PRA, and (2) provide insight, advice, 

and guidance on (a) the technical bases, tools, methods, models, and data for the project, (b) 

the interpretation of the results of the various PRA models and the overall PRA model, and 

(c) the response to comments received from the external peer reviews of the study. In this 

role, the TAG will serve as an ongoing review team that will provide review and feedback as 

the project progresses. Also, as part of its initial review responsibility, the TAG will review the 

TAAP to provide feedback on the approach being used to perform the work.

 3 “Standardized Plant Analysis Risk (SPAR) Model QA Plan,” Revision 0, U.S. Nuclear Regulatory Commission, 

Washington, DC, September 2006 (not publicly available).

4 “Risk Assessment of Operational Events Handbook, Volume 3 – SPAR Model Reviews,” Revision 1, U.S. 

Nuclear Regulatory Commission, Washington, DC, September 2007 (not publicly available).

5 For example: American Nuclear Society, “American National Standard External-Events PRA Methodology,” 

ANSI/ANS-58.21-2003, December 2003.

6 Charter for the Technical Advisory Group on the Full-Scope Site Level 3 Probabilistic Risk Assessment Project, 

ADAMS Accession Number ML120410123 (not publicly available).
Level 3 PRA Project QA Plan, Rev. 0

4

As stated earlier, the approach used for the Level 3 PRA project will be based on plant 

information and established methods, tools and data. Where the plant information or the 

methods, tools or data do not exist to develop certain aspects of the PRA model, other 

sources such as expert opinion will be used. The TAG will play a key role in addressing the 

acceptability of such proposed approaches. Furthermore, it is expected that the TAG will 

play a fundamental role in resolving technical or programmatic issues that may arise.

The TAG will consist of senior technical staff in the area of PRA, and in supporting technical 

areas (e.g., seismic hazard and plant response), as well as an experienced PRA 

representative from the Electric Power Research Institute and from industry7

. The Office of 

Nuclear Regulatory Research (RES)/Division of Risk Analysis (DRA) staff will chair and 

coordinate the TAG, which will meet periodically. The TAG Chairman will be responsible for 

leading and moderating the TAG meetings, and will serve as the TAG spokesperson, as 

necessary, in briefings to NRC and project management. The TAG Coordinator, in 

consultation with the Level 3 PRA Project Program Manager and the TAG Chairman, will 

develop and disseminate the agenda for each TAG meeting. The TAG Coordinator will also 

be responsible for organizing and recording the minutes of the TAG meetings and 

maintaining an electronic repository to provide reports, publications, and other technical 

information as background for all TAG meetings.

Table 1 provides a template for the TAG review documentation. This template (or a similar 

documentation format) is to be used to document the results of the TAG reviews performed 

for the Level 3 PRA project.

Table 1 TAG Review Documentation Template

SR Finding Recommended Resolution Implemented Resolution

Reviewer: Responsible Analyst:

Risk Source: Hazard: [e.g., internal events] Level: [1, 2 or 3]

Technical Element: Date:

 7 This individual was initially a staff member of NextEra Energy Resources and then became an employee of 

Westinghouse.

Reactor, Spent Fuel Pool, Dry Cask Storage, Integrated Risk

Describe the finding, 

what is the issue, why it 

is a concern; 

explanation needs to 

clearly explain the 

concern and the basis 

for the concern.

Describe the recommendation to 

resolve the concern; the explanation 

needs to be sufficiently detailed so 

that the analyst understands what 

needs to be revised in the PRA to 

resolve the concern.

Analyst describes the response to 

the finding and recommendation, 

describing how it was resolved; the 

explanation should not be just an 

“accept,” but an explanation of 

exactly how it was resolved (e.g., 

how the PRA model was revised).

List the applicable supporting requirement (SR) using the standard index number; if an SR is not applicable, then use the 

technical element 2 to 4 digit abbreviation (xxxx) and the finding numbered sequentially (yy) with an “T” (i.e., xxxx-yy-T). If 

criteria were developed and used, then reference the criterion number (see Table 2).
Level 3 PRA Project QA Plan, Rev. 0

5

4.2 Internal Self-Assessment

The objective of the internal self-assessment is to further ensure the technical acceptability of 

the work as the PRA model is being developed. The PRA model will be developed based on 

established and accepted methods, tools, and data as documented in, for example, 

consensus standards and guidance documents. For each technical element, a review of the 

work is performed using the process described below. 

The full-scope site Level 3 PRA model consists of models developed by the Southern 

Nuclear Operating Company (SNC) for Vogtle Units 1 and 2, and those developed internally 

by the NRC. Parts of the Vogtle PRA model have received an industry peer review, using the 

ASME/ANS Level 1 PRA Standard.8

 The self-assessment process will take advantage of the 

industry peer review. Figure 1 provides the process for self-assessment. This process 

involves 5 steps as discussed below.

Figure 1 Process Used for Self-Assessment

Generally the self-assessment is performed by the technical element leader, responsible 

analyst, or may be performed by an internal NRC “team.” If the work is performed by a 

contractor, the self-assessment is performed by an NRC team (with contractor support). The 

purpose of using an NRC team instead of the contractor to perform the self-assessment is for 

the NRC to have ownership of the work; that is, to understand the details of constructing the 

model.

In Step 1, the self-assessment reviewer determines whether an independent industry peer 

review was performed. This decision will determine the scope of the self-assessment; that is, 

the analyst is determining whether the self-assessment can take advantage of the 

independent peer review performed on the Vogtle PRA. If an independent peer review was 

not performed, then the reviewer needs to perform a complete self-assessment (Step 4). If 

 8 ASME/ANS RA-Sa-2009, “Standard for Level 1/Large Early Release Frequency Probabilistic Risk Assessment for Nuclear 

Power Plant Applications,” Addendum A to RA-S-2008, ASME, New York, NY, American Nuclear Society, La Grange Park, 

Illinois, February 2009.

PRA model 

(technical task) 

acceptable

Self-assessment 

complete

No

Yes

Yes

Step 1:

Step 4: Step 2:

Step 3: Step 5:

Revise PRA model 

(Technical task)

accordingly

No

Perform a 

selfassessment 

Document the resolution 

of the findings and the 

self assessment

Was an independent 

industry peer review 

performed?

Were the peer review 

findings adequately 

addressed or not 

significant?
Level 3 PRA Project QA Plan, Rev. 0

6

an independent peer review was performed, then the significance of the peer review results 

and resolutions needs to be assessed (Step 2).

In Step 2, the reviewer determines if the findings from the peer review were addressed and if 

they appropriately addressed the issue. If the peer review findings were adequately 

addressed or were not adequately addressed but determined not to be significant to the PRA, 

then the reviewer goes to Step 4 to perform the self-assessment. If the peer review findings 

are determined to not be adequately addressed and are significant to the PRA, then the 

reviewer needs to revise the PRA model to correct the issue (Step 3). 

Significance can be determined both qualitatively and quantitatively, as follows:

Qualitative –

• The finding can result in changing the basic structure of the PRA model (e.g., success 

criteria such that the accident sequence progression is changed, different initiating 

events and/or frequencies, different human events and/or frequencies, different 

equipment failure probabilities).

Quantitative –

• Significant accident sequences are impacted. A significant sequence is one of the set 

of sequences, defined at the functional or systemic level that, when ranked, compose 

95% of the core damage frequency (CDF) or the large early release frequency/large 

release frequency (LERF/LRF), or that individually contribute more than ~1% to the 

CDF or LERF/LRF.

• Significant basic event/contributors are impacted. Significant basic events (i.e., 

equipment unavailabilities and human failure events) are those that have a FussellVesely9 importance greater than 0.005 or a risk-achievement worth greater than 2. 

In Step 3, the reviewer revises the PRA model to resolve the inadequacy. After the PRA is 

revised, the reviewer goes to Step 4 to perform the self-assessment.

In Step 4, the self-assessment is performed using the guidance in RG 1.200. As such, the 

self-assessment:

• Uses a set of desired PRA characteristics and attributes as the basis for review

• Uses a minimum list of review topics to ensure coverage, consistency, and uniformity

• Reviews PRA methods

• Reviews application of methods

 9 Risk Reduction Worth: “The decrease in risk if a plant feature (e.g., system or component) were assumed to be optimized or 

were assumed to be made perfectly reliable. Depending on how the decrease in risk is measured, the risk reduction worth can 

either be defined as a ratio or an interval.” Risk Achievement Worth: The increase in risk if a plant feature (e.g., system or 

component) was assumed to be failed or was assumed to be always unavailable. Depending on how the increase in risk is 

measured, the risk achievement worth can either be defined as a ratio or an interval. Sometimes risk achievement worth is 

referred to as “risk increase.” Fussell-Vesely: For a specified basic event, Fussell-Vesely importance is the relative contribution 

of a basic event to the calculated risk. This relative or fractional contribution is obtained by determining the reduction of setting 

the probability of the basic event to zero. Birnbaum Importance: “An indication of the sensitivity of the accident sequence 

frequency to a particular basic event.” 
Level 3 PRA Project QA Plan, Rev. 0

7

• Reviews assumptions and assesses their validity and appropriateness

• Determines if the PRA represents the as-built and as-operated plant

• Reviews results of each PRA technical element for reasonableness

• Reviews PRA maintenance and update process

• Reviews PRA modification attributable to use of different model, techniques, or tools

• Reviews against modifications to the standard, if there is a standard

In evaluating the above, if a standard exists, then the requirements in the standard are used 

as the basis for the self-assessment in determining whether, for example, the desired 

attributes and characteristics provided in RG 1.200, Section 1 are met. If a PRA standard 

does not exist for a particular hazard or technical element, then criteria are developed to 

perform the self-assessment. These criteria are detailed enough to judge the technical 

acceptability of the work. They should be of consistent detail as in the standard for hazards 

or technical elements addressed by a standard. These criteria are documented using Table 

2 (or a similar documentation format). Once the self-assessment (Step 4) is complete, the 

reviewer should go to Step 5 to document the results.

Table 2 Self-Assessment and Peer Review Criteria Where Standards Do Not Exist

Criteria 

#

Criteria

Source of Risk: Hazard:

PRA Level: Technical Element:

In Step 5, the reviewer documents the self-assessment using Tables 3 and 4 (or a similar 

documentation format). Table 3 can be generated using the ePSA Risk and Reliability 

software. This program populates some of the fields in the table automatically based on the 

ASME/ANS Level 1 PRA standard. For those parts of the PRA not covered by this standard, 

the ePSA software cannot be used, and the analyst will have to create the table using the 

template and the criteria developed and documented in Table 2. The purpose of Table 4 is to 

provide a high level summary of the conclusions of the self-assessment. 

After Step 5, the initial self-assessment is complete.

The elements of the Level 1 PRA that require complete or focused review can be assessed 

using the guidance in RG 1.200 supported by the requirements provided in the ASME/ANS 

PRA Standard. For those aspects of the PRA models that do not have a final consensus 

standard, but do have a standard that is being developed, they will be reviewed using the 

high level requirements stipulated in the latest draft of the specific standards. This process 

In numbering the criteria, use the technical element 2-4 digit abbreviation 

(xxxx) and the criteria numbered sequentially (yy) with a “C” (i.e., xxxx-yy-C).
Level 3 PRA Project QA Plan, Rev. 0

8

will be used for the self-assessment review of the Level 2, Level 3, and low power and 

shutdown PRA. The PRA models for which a standard does not exist or is not being 

developed (i.e., dry cask storage [DCS], spent fuel pool [SFP]), elements of these models 

that have similar bases as compared to those of the Level 1 PRA (e.g., initiating event 

analysis, data analysis, human reliability analysis, accident sequence analysis, consequence 

analysis, source term determination, quantification/uncertainty analysis, etc.) can be 

reviewed using the requirements for the similar technical areas in the Level 1 through Level 3 

PRA standards bearing in mind the differences in the requirements related to reactor versus 

those for the DCS/SFP.

Table 3 Self-Assessment Documentation Template

Section Finding 

ID

Cat II 

Requirement

SelfAssessmentFinding Comment Resolution

Describe the assessment, describe what was done, whether a concern was found or not. Describe the 

concern, why it is a concern; explanation needs to clearly explain the concern and the basis for the 

concern. If no concern is found, describe the basis for why it is believed the requirement (or criterion) 

was met. Describe the proposed fix to resolve the concern; the explanation needs to be sufficiently 

detailed so that it is understood what needs to be revised in the PRA to resolve the concern.

The supporting requirement for Capability Category II from the ASME/ANS 

Level 1 PRA standard is generated automatically if using the ePSA 

software; however, if this standard does not apply, the template is used 

and the defined criteria are entered manually from Table 2.

As a result of the self-assessment, if a 

concern is found, i.e., a “finding,” then 

a “Y” is marked; if there is no finding, 

then an “N” is marked.

Any additional explanations 

that are relevant to the selfassessment are discussed.

Analyst describes the response to the finding and the proposed fix, describing how it 

was resolved; the explanation should not be just an “accept,” but an explanation of 

exactly how it was resolved (e.g., how the PRA model was revised).
Level 3 PRA Project QA Plan, Rev. 0

9

Table 4 Overall Results of Self-Assessment Process

# Criteria Conclusion

Reviewer: Responsible Analyst:

Risk Source: Hazard: Level:

Technical Element: Date:

Vogtle Industry Peer Review

1 Was an independent peer review 

performed on the Vogtle PRA?

2 Was the scope of the peer review 

adequate?

3 Did the peer review meet the staff 

position defined in Regulatory Guide 

1.200 for an acceptable peer 

review?

4 Were the peer review findings 

adequately addressed in the PRA?

General Conclusions

5 Is the identified list of information 

needed to accomplish the task 

reasonably complete?

6 Does the plant information 

appropriately represent the as-built 

and as-operated plant?

7 Was the plant information used in an 

acceptable manner?

8 Are the assumptions for each task 

identified?

9 Are the assumptions for each task 

adequately justified (appropriate)?

10 Do the results (both interim and 

final) appear reasonable given the 

design, operation and historical 

performance of the plant?

Specific Conclusions

For example, the initiating event analysis for a SFP PRA uses similar techniques and 

processes as those used for a Level 1 reactor PRA. The high level requirements for the 

reactor PRA model can be used for the SFP PRA model (the specifics of SFP are presented 

in parenthesis) as indicated below: 

HLR-IE-A – The initiating event analysis shall provide a reasonably complete identification 

of initiating events.

HLR-IE-B – The initiating event analysis shall group the initiating events so that events in 

the same group have similar mitigation requirements to facilitate an efficient 

but realistic estimation of CDF (or fuel damage frequency)

Describe the conclusion and the basis for the 

conclusion; may refer to self-assessment table..

Describe unique or specific conclusions, if any, and the 

basis for the conclusion.
Level 3 PRA Project QA Plan, Rev. 0

10

HLR-IE-C – The initiating event analysis shall estimate the annual frequency of each 

initiating event or initiating event group

Individual supporting requirements can be tailored for use in SFP PRA self-assessment 

Table 5 provides an example self-assessment process for the SFP PRA. In the absence of 

any standard, the technical elements of the SFP PRA defined in the TAAP are compared to 

the similar elements of the Level 1 reactor at-power internal events PRA discussed in the 

ASME/ANS Standard. Tables 5 and 6 identify both the high level requirements and the 

supporting requirements that are common and applicable for the self-assessment review of 

the SFP PRA.

Table 5 Example: Mapping of the HLRs of SFP PRA and At-Power Level 1 PRA

Task 

#

At-Power Level 1 PRA Technical 

Elements (HLR) SFP PRA Technical Elements 

1 IE Analysis IE Analysis

• Identification

• Grouping

• Analysis

• Identification10

• Grouping

• Analysis

• Operating Cycle Discretization 11

2 Accident Sequence Analysis Accident Sequence Analysis

• CDF Accident Scenario Description 

• Treatment of Dependencies

• Fuel Uncovery Accident Scenario 

Description 

• Treatment of Dependencies

3 Systems Analysis Systems Analysis

• Treatment of Causes for System failure

• Treatment of CCF

• Treatment of Dependencies

• Treatment of Causes for System Failure

• Treatment of CCF

• Treatment of Dependencies

4 Success Criteria Structural Analysis

• Defining Overall SSC and Human 

Action Success Criteria

• Using Thermal/Hydraulic, Structural and 

other supporting Engineering Bases to 

Drive SC

• Defining Overall SSC and Human Action 

Success Criteria

• Using Thermal/Hydraulic, Structural and 

other supporting Engineering Bases to Drive 

SC

• Identification of FP failure modes and 

locations

• SFP Structural Integrity Analysis

• SSCs Structural Integrity Analysis

 10 Includes hazard and low-likelihood event screening.

11 Discretizing the reactor operating cycle into a finite set of operating cycle phases (OCPs) can be considered to 

be akin to the plant operating states considered in a low power and shutdown PRA, with respect to the amount 

of decay heat that needs to be considered. This process determines the time available to respond to an 

accident, before fuel damage occurs. 
Level 3 PRA Project QA Plan, Rev. 0

11

Table 5 Example: Mapping of the HLRs of SFP PRA and At-Power Level 1 PRA

Task 

#

At-Power Level 1 PRA Technical 

Elements (HLR) SFP PRA Technical Elements 

5 Data Analysis Data Analysis

6 Human Reliability Analysis Human Reliability Analysis

• Identifying routines of activities

• Screening of activities

• Defining HFEs

• Assessing HFE Probability

• Identifying Operator Accident Response

• Defining Response HFEs

• Assessing Response HFE Probability

• Modeling Recovery Actions

• Identifying routines of activities

• Screening of activities

• Defining HFEs

• Assessing HFE Probability

• Identifying Operator Accident Response

• Defining Response HFEs

• Assessing Response HFE Probability

• Modeling Recovery Actions

7 Quantification Quantification

Table 6 Applicability of SRs of the At-Power Level 1 PRA to the SFP PRA

Technical 

Element HLR

Supporting 

Requirement

Applies 

(Y/N) Comment

1 IE-A IE-A1 Y Except instead of core damage (CD) it 

considers fuel damage (FD)

IE-A2 Y Except the IE categories reduce to fuel 

uncovery and loss of power

IE-A3 Y

IE-A4 Y

IE-A5 Y

IE-A6 Y

IE-A7 Y

IE-B IE-B1 Y

IE-B2 Y

IE-B3 Y

Note: The timing and the effect on the 

operability and performance of operators and 

relevant mitigating systems is one criterion to 

consider. The operating cycle discretization 

influences this timing factor.

IE-B4 Y

IE-B5 N

IE-C IE-C1 Y

IE-C2 Y

IE-C3 Y

IE-C4 Y

IE-C5 Y

IE-C6 Y Screening the low-frequency events

IE-C7 Y

IE-C8 Y

IE-C9 Y
Level 3 PRA Project QA Plan, Rev. 0

12

Table 6 Applicability of SRs of the At-Power Level 1 PRA to the SFP PRA

Technical 

Element HLR

Supporting 

Requirement

Applies 

(Y/N) Comment

IE-C10 Y

IE-C11 Y

IE-C12 Y

IE-C13 Y

IE-C14 N

2 AS-A AS-A1 Y

AS-A2 Y Except that instead of preventing core 

damage, fuel damage should be considered

AS-A3 Y

AS-A4 Y

AS-A5 Y

AS-A6 Y

AS-A7 Y

AS-A8 Y

Except that instead of the core damage end 

state, the fuel damage end state should be 

considered

AS-A9 Y

AS-A10 Y

AS-A11 Y

AS-B AS-B1 Y

AS-B2 Y Except for examples

AS-B3 Y

AS-B4 Y

AS-B5 Y

AS-B6 Y

AS-B7 Y Except examples (b) and (c)

3 SC-A SC-A1 N Applies to fuel damage

SC-A2 Y Modifies the parameters and SCs to be used 

in determining the fuel damage

SC-A3 Y

SC-A4 Y If applicable

SC-A5 Y

SC-A6 Y

SC-B SC-B1 Y

SC-B2 Y

SC-B3 Y

SC-B4 Y Except for fuel damage

SC-B5 Y

4 SY-A SY-A1 Y

SY-A2 Y

SY-A3 Y

SY-A4 Y

SY-A5 Y Except for fuel damage

SY-A6 Y
Level 3 PRA Project QA Plan, Rev. 0

13

Table 6 Applicability of SRs of the At-Power Level 1 PRA to the SFP PRA

Technical 

Element HLR

Supporting 

Requirement

Applies 

(Y/N) Comment

SY-A7 Y

SY-A8 Y

SY-A9 Y

SY-A10 Y

SY-A11 Y

SY-A12 Y

SY-A13 Y

SY-A14 Y

SY-A15 Y

SY-A16 Y

SY-A17 Y

SY-A18 Y

SY-A19 Y

SY-A20 Y

SY-A21 Y

SY-A22 Y

SY-A23 Y

SY-A24 Y

SY-B SY-B1 Y

SY-B2 Y

SY-B3 Y

SY-B4 Y

SY-B5 Y

SY-B6 Y

SY-B7 Y

SY-B8 Y

SY-B9 Y

SY-B10 Y

SY-B11 Y

SY-B12 Y

SY-B13 Y

SY-B14 Y

SY-B15 Y

5 HR-A HR-A1 Y

HR-A2 Y

HR-A3 Y

HR-B HR-B1 Y

HR-B2 Y

HR-C HR-C1 Y

HR-C2 Y

HR-C3 Y

HR-D HR-D1 Y

HR-D2 Y
Level 3 PRA Project QA Plan, Rev. 0

14

Table 6 Applicability of SRs of the At-Power Level 1 PRA to the SFP PRA

Technical 

Element HLR

Supporting 

Requirement

Applies 

(Y/N) Comment

HR-D3 Y

HR-D4 Y

HR-D5 Y

HR-D6 Y

HR-D7 Y

HR-E HR-E1 Y

HR-E2 Y Except for preventing or mitigating fuel 

damage

HR-E3 Y

HR-E4 Y

HR-F HR-F1 Y

HR-F2 Y

HR-G HR-G1 Y

HR-G2 Y

HR-G3 Y

HR-G4 Y

HR-G5 Y

HR-G6 Y

HR-G7 Y

HR-G8 Y

HR-H HR-H1 Y

HR-H2 Y

HR-H3 Y

6 DA-A DA-A1 Y

DA-A2 Y

DA-A3 Y

DA-A4 Y

DA-B DA-B1 Y

DA-B2 Y

DA-C DA-C1 Y

DA-C2 Y

DA-C3 Y

DA-C4 Y

DA-C5 Y

DA-C6 Y

DA-C7 Y

DA-C8 Y

DA-C9 Y

DA-C10 Y

DA-C11 Y

DA-C12 Y

DA-C13 Y

DA-C14 Y

DA-C15 Y
Level 3 PRA Project QA Plan, Rev. 0

15

Table 6 Applicability of SRs of the At-Power Level 1 PRA to the SFP PRA

Technical 

Element HLR

Supporting 

Requirement

Applies 

(Y/N) Comment

DA-C16 Y

DA-D DA-D1 Y

DA-D2 Y

DA-D3 Y

DA-D4 Y

DA-D5 Y

DA-D6 Y

DA-D7 Y

DA-D8 Y

7 QU-A QU-A1 Y

QU-A2 Y Except for fuel damage frequency

QU-A3 Y Except for fuel damage frequency

QU-A4 Y Except for fuel damage frequency

QU-A5 Y

QU-B QU-B1 Y

QU-B2 Y

QU-B3 Y The example applies to fuel damage 

frequency

QU-B4 Y

QU-B5 Y

QU-B6 Y Except for fuel damage frequency

QU-B7 Y

QU-B8 Y

QU-B9 Y

QU-B10 Y

QU-C QU-C1 Y

QU-C2 Y

QU-C3 Y

QU-D QU-D1 Y

QU-D2 Y

QU-D3 Y

QU-D4 Y

QU-D5 Y

QU-D6 Y Except for fuel damage frequency

QU-D7 Y

QU-E QU-E1 Y

QU-E2 Y

QU-E3 Y Except for fuel damage frequency

QU-E4 Y
Level 3 PRA Project QA Plan, Rev. 0

16

4.3 External Peer Reviews

The objective of the external peer reviews is to provide independent reviews of the technical 

acceptability of the developed PRA model and its results. There are two types of peer review 

planned which are discussed in the following sections:

• Section 4.3.1 – PRA Standard Peer Review

• Section 4.3.2 – Independent Expert Peer Review

The first peer review is similar to the peer reviews performed by industry and follows the peer 

review process as required by the ASME/ANS PRA standard and employs the NEI peer 

review guidance. The purpose of the ASME/ANS peer review is “to assess the PRA to the 

extent necessary to determine if the methodology and its implementation meet the 

requirements of this standard.” And “. . . to determine strengths and weaknesses in the PRA.” 

The peer reviewers are industry individuals whose qualifications as acceptable peer 

reviewers are provided in the ASME/ANS PRA standard as endorsed in RG 1.200. 12 A 

major qualification includes independence from the team who developed the PRA model 

under review. 

The second peer review is also an independent review performed by a team of experts. 

Many of these reviewers are likely to come from academia and national laboratories.

4.3.1 PRA Standard Peer Review

ASME/ANS have developed PRA standards which provide the necessary technical 

requirements for what constitute a technically acceptable PRA based on state-of-the practice 

methods. One objective of the Level 3 PRA project is to develop a PRA based on current 

state-of-the-practice methods.

To the extent practical, the PRA standard peer reviews will be conducted for all major parts of 

the Level 3 PRA project at various points throughout the performance of the study. This 

approach will allow peer review findings to be addressed in a timely manner. It will, as 

opposed to performing one large, comprehensive external peer review at the end of the 

project, minimize the extent of potential re-work.

Where PRA standards (either “final” or “draft for trial use”) are available, they will provide the 

basis for the peer review. If a standard is in “draft for trial use” stage, the peer review part of 

the standard will be reviewed and additional guidance will be developed, if needed, to make it 

acceptable to the staff. If a PRA standard does not exist (e.g., spent fuel pool), review 

criteria will be developed to support the peer review of the PRA scope item.

The reviews will be performed consistent with the process described in RG 1.200 and 

supplemented with other related guidance. The peer review teams will be comprised of 

individuals who are independent from the project. It is envisioned that the standard peer 

reviews will be performed by industry (e.g., the Pressurized Water Reactor (PWR) Owner’s 

Group (OG) and consultants), supplemented by NRC staff (e.g., Regional senior reactor 

analysts (SRAs)). In determining whether the technical requirements in the standard have 

been met, the level of detail of the PRA model review goes beyond the technical bases, tools, 

methods, models, assumptions and data for the project, as well as interpretation of the study 

 12Regulatory Guide 1.200, Rev. 2, “An Approach for Determining the Technical Adequacy of Probabilistic Risk Assessment 

Results for Risk-Informed Activities,” March 2009.
Level 3 PRA Project QA Plan, Rev. 0

17

results. It also involves reviewing how the various models (e.g., accident sequence 

development, systems analyses) were constructed. In this regard, actual Vogtle-specific 

information is needed. The peer reviewers are required to sign a non-disclosure agreement 

since this information is proprietary. 

The scope of the peer review will be documented prior to each peer review and provided to 

the peer review team. Table 7 provides a suggested format for documenting the peer review 

findings (it is the same as the TAG review documentation template previously provided in 

Table 1).

It is expected that the peer review team will generate a peer review report. This report will 

describe the process, team members (and their qualifications), and basis for review findings. 

It is further expected that the Level 3 PRA project task leader will review the peer review 

findings and document how each finding will be resolved. The results of the standard peer 

reviews will be provided to the Level 3 Program Manager and to the Document Controller.

Table 7 External Peer Review Documentation Template

SR Finding Recommended Resolution Implemented Resolution

Reviewer: Responsible Analyst

Risk Source: Level : [1,2,3] Hazard: [e.g., internal events]

Technical Element: Date:

o High Significance -- the issue needs resolution to ensure the technical adequacy of the 

PRA, the capability of the PRA, or the robustness of the PRA update process.

o Medium Significance -- The issue needs resolution to maintain maximum flexibility in 

PRA applications and consistency with Industry practices (as endorsed by the NRC) or 

simply to enhance the PRA’s technical capability as time and resources permit. It is 

unlikely that the technical adequacy of the PRA is impacted. 

o Low Significance -- The issue that does not impact the technical adequacy of the PRA.

Describe the finding, 

what is the issue, why 

it is a concern; 

explanation needs to 

clearly explain the 

concern and the basis 

for the concern.

Describe the 

recommendation to 

resolve the concern; 

the explanation needs 

to be sufficiently 

detailed so that the 

analyst understands 

what needs to be 

revised in the PRA to 

resolve the concern.

Analyst describes the response to the 

finding and recommendation, describing 

how it was resolved; the explanation 

should not be just an “accept,” but an 

explanation of exactly how it was resolved 

(e.g., how the PRA model was revised).

The level of significance of the concern 

should be listed including the basis for 

level of significance assessed; see below 

for explanation of significance.

List the applicable supporting requirement (SR) using the standard index number; if an SR is 

not applicable, then use the technical element 2-4 digit abbreviation (xxxx) and the finding 

numbered sequentially (yy) with an “P” (i.e., xxxx-yy-P). If criteria were developed and used, 

then reference the criterion number (see Table 2).

Reactor, Spent Fuel Pool, Dry Cask Storage, Integrated Risk
Level 3 PRA Project QA Plan, Rev. 0

18

4.3.2 Independent Expert Peer Review

The purpose of the Independent Expert Peer Review (IEPR) is not to determine if the 

ASME/ANS PRA standard requirements were met, but to perform a high level peer review 

comprised of known national or international PRA experts primarily from academia and 

national laboratories. The main objective is to determine strengths and weaknesses in the 

PRA. In this regard, the IEPR involves reviewing the technical bases, tools, methods,

models, assumptions and data for the project, as well as interpretation of the study results. It 

does not involve reviewing how the various models (e.g., accident sequence development, 

systems analyses) were constructed

This IEPR is intended to be performed at the end of the project. 

The scope of the peer review will be documented prior to each peer review and provided to 

the IEPR team. The documentation of the IEPR will include the following:

• Identification of the reviewer, the part of the Level 3 PRA project reviewed (which 

radiological sources, PRA Level(s), operating state(s), and hazard(s)).

• Description of the findings, what is the issue, why it is a concern (i.e., the basis for the 

concern).

• Identification of the level of significance of the issue and the basis for the significance. 

The significance will be identified as:

o High Significance -- the issue needs resolution to ensure the technical 

adequacy of the PRA, the capability of the PRA, or the robustness of the PRA 

update process.

o Medium Significance -- The issue needs resolution to maintain maximum 

flexibility in PRA applications and consistency with Industry practices (as 

endorsed by the NRC) or simply to enhance the PRA’s technical capability as 

time and resources permit. It is unlikely that the technical adequacy of the 

PRA is impacted. 

o Low Significance -- The issue that does not impact the technical adequacy of 

the PRA.

• Description of the proposed recommendation to resolve the concern; the explanation 

needs to be sufficiently detailed so that the analyst understands what needs to be 

revised in the PRA to resolve the concern.

It is expected that the IEPR team will generate a peer review report. This report will describe 

the process, team members (and their qualifications), and basis for review findings. It is 

further expected that the Level 3 PRA project task leader will review the IEPR findings and 

document how each finding will be resolved; the explanation should not be just an “accept,” 

but an explanation of exactly how it was resolved (e.g., how the PRA model was revised). 

The results of the IEPR will be provided to the Level 3 Program Manager and to the 

Document Controller.
Level 3 PRA Project QA Plan, Rev. 0

19

4.4 Advisory Committee on Reactor Safeguards

The objective of the Advisory Committee on Reactor Safeguards (ACRS) review for the Level 

3 PRA project is to: (1) monitor progress in the development of the Level 3 PRA and (2) 

provide insight, advice, and guidance on the technical bases, tools, methods, models, 

assumptions and data for the project, as well as on interpretation of the study results. 

The ACRS Reliability and PRA Subcommittee will be briefed approximately twice a year to 

obtain their feedback on the technical approaches and assumptions employed in the Level 3 

PRA project. 

4.5 Public Review and Comment

As part of the documentation, a final summary of the results of the Level 3 PRA project will 

be published. This report will provide the various results of the study, and will also 

summarize the various tools, methods, models, assumptions and data used. This summary 

report (or reports) will be published for public review and comment.

A public meeting will be held to brief the public on the report(s) and answer questions. A 

second meeting will be held to provide responses to the public comments.

Each team leader is responsible for addressing the public comments associated with their 

part of the study.

5 DOCUMENTATION CONTROL

Documentation control is a key factor in any analysis to ensure and demonstrate the 

technical acceptability of the analysis. For each technical task, the method, tools, data and 

other information being used will be documented along with the basis for their acceptability 

(e.g., NRC endorsement). The documentation for each technical task is identified in the 

TAAP, and the document control process for this project is described in this section.

As mentioned above, the information to be documented includes the following:

• Methods

• Tools

• Data

• Other information - this includes the various information (other than methods, tools 

and data) used to develop the PRA model; for example:

 plant design information reflecting the normal and emergency configurations of 

the plant

 plant operational information with regard to plant procedures and practices

 plant history (plant, system, and component performance)

 plant test and maintenance procedures and practices

 engineering aspects of the plant design

• Analytical work

• Results

Given the large amount of information of various types required to construct and report the 

results of the Level 3 PRA project, an appropriate medium is needed to store and access this 
Level 3 PRA Project QA Plan, Rev. 0

20

information. This medium has to have the ability for the project analysts to store, retrieve, 

edit, and control the information. SharePoint has been selected to be the medium, and the 

primary repository for Level 3 PRA project information will be referred to as the Level 3 PRA 

SharePoint site.

The Level 3 PRA project Documentation Coordinator will primarily be responsible for 

document control. The Documentation Coordinator will be in charge of the various tasks 

needed to ensure the SharePoint site runs smoothly and remains organized, and will be 

responsible for receiving information from the licensee, processing it, and ensuring that the 

information gets to contractors and the SharePoint site in a reasonable timeframe, as well as 

ensuring that vital information is routinely backed up.

Documentation control for this project involves the following major elements, each of which is 

described in a separate section below:

• Section 5.1 – Storage and access of project information

• Section 5.2 – Upload of information onto the SharePoint site

• Section 5.3 – Documentation control of licensee information 

• Section 5.4 – Documentation backup

• Section 5.5 – Use of external storage media 

• Section 5.6 – Working document folders

• Section 5.7 – Use of templates and forms for documentation

• Section 5.8 – Site Visits

• Section 5.9 – Documentation control for NRC Contractors

• Section 5.10 – Non-disclosure agreement to allow access to proprietary information

• Section 5.11 – Project documentation markings

• Section 5.12 – Guidance for addressing potential technical issues

• Section 5.13 – Future plant modifications

• Section 5.14 – Organization of the various types of information on the SharePoint site 

5.1 Storage and Access of Project Information

As mentioned above, SharePoint has been selected as the medium to store and access the 

Level 3 PRA project information. SharePoint has the necessary flexibility to organize and 

store the information in a manner consistent with the needs of the project. It also allows for 

dynamic changes to the organization and site as new needs arise over the course of the 

project. 

Moreover, controls can be used to limit access to the information; for example, who is 

allowed to access the information and who is allowed to edit documents. These controls will 

help ensure that files are not accidentally deleted or edited without the author’s approval. 

SharePoint also has an established backup procedure that ensures data integrity. Therefore, 

SharePoint provides a mechanism to ensure that information will not be lost or corrupted. 

The information stored on the SharePoint site is only accessible by the project team 

members who have access to the NRC’s local area network.
Level 3 PRA Project QA Plan, Rev. 0

21

5.2 Upload of Information onto the SharePoint Site

As the work progresses, the project team members will occasionally need to place files onto 

the Level 3 PRA Project SharePoint site. These files will include information that only the 

individual analyst will need to access, or that needs to be shared with other members of the 

task team or with the entire project team. Moreover, there may need to be restrictions, for 

example, on who has permission to edit these files. 

Although most team members may not edit or modify most of the files stored on the 

SharePoint site, any project team member has permission to upload files into the temporary 

storage location titled, “Inbox.” Once a file is uploaded into the Inbox, the Documentation 

Coordinator will move the file from the Inbox to its proper read-only location. In order to 

upload files, there is a link on the right hand side of the front page that is titled, “Inbox: 

Upload documents to the L3PRA website.” This page can also be found by clicking: 

http://portal.nrc.gov/edo/res/dra/L3PRA/Inbox_Library/Forms/AllItems.aspx

Once on the Inbox page, the “upload” button is clicked and the analyst chooses the files to be 

placed on the site. In uploading each file, a brief description of the file and the last edited 

date is included in the “Notes” section. The restrictions on who has access, edit capability, 

etc., can be found in Table 12 for the different types of information.

5.3 Documentation Control of Licensee Information

The information received from the licensee will also be stored on the SharePoint site. The 

information on the SharePoint site will be read-only, with the exception of the personal 

working files (discussed in Section 5.6). This administrative control will prevent inadvertent 

changes to information obtained from the licensee. All information received from the licensee 

will also be maintained on read only CD-ROMs or DVDs so that, in the event of an 

inadvertent change on SharePoint, the original data can be restored. Moreover, there is 

information received from the licensee which is proprietary and not available to the public, 

and therefore needs to be protected. When information is received from the licensee in 

support of this project, a proprietary determination is conducted for each submittal.13 Once 

this proprietary determination is conducted and approved by the Office of the General 

Counsel, the information is placed on the SharePoint site for all NRC Level 3 PRA Project 

Team members. The specific SharePoint folder that contains this information is clearly 

marked as “Proprietary.” If this information is needed by a contractor to perform their work, 

the information is then copied onto an encrypted external media device (usually a CD-ROM, 

marked as “Proprietary,” if applicable) and sent to the contractor along with a notice, if 

applicable, that the CD-ROM contains proprietary information and should be handled 

appropriately. 

In addition, the licensee may occasionally send updated information, or may resend the same 

information. These occurrences may cause confusion as to which version of the information 

is the most current. It is, therefore, essential that the information be administratively 

controlled such that different information is not being used in developing the model by 

different analysts. The use of SharePoint for file hosting will greatly simplify this process. The 

 13 RES Office Instruction ADM-003, Revision 1, “Procedures for Handling Request to Withhold Proprietary 

Information,” May 11, 2012, ADAMS Accession Number ML12132A139 (not publicly available).
Level 3 PRA Project QA Plan, Rev. 0

22

Documentation Coordinator will ensure that the data on the SharePoint site is the most 

current, up-to-date information that the NRC has received from the licensee, and will notify 

the entire project team when new information from the licensee is added to the SharePoint 

site. This notification will identify what information is being added and whether it updates any 

information currently existing on the site. 

5.4 Documentation Backup 

Using SharePoint to store and access all the information connected with the Level 3 PRA 

Project will ensure a high level of data integrity. The files on SharePoint are backed-up 

several times a week and copies are maintained both onsite and offsite. If SharePoint is 

corrupted, this process ensures that there will be minimal loss of information, and progress of 

the project can continue given an extreme event. In addition to this automatic NRC backup 

of the information, once a week the Documentation Coordinator will copy all of the 

information on the Level 3 PRA SharePoint site onto an external media device. This backup 

of the files will be stored onsite for rapid recovery of files. Information that is not able to be 

placed on the SharePoint site will also be backed up and maintained. 

5.5 Use of External Storage Media

There may be types of information that are not permitted to be uploaded onto the NRC’s 

SharePoint Site. This type of information generally involves large files and executable files

(e.g., Access Database files and files that end in “.exe”). Therefore, an external media 

storage device that has been approved for use on NRC equipment will be available, on 

request, for project team members to back up these files. This external media device will be 

stored and maintained by the Document Coordinator.

In addition, some Level 3 PRA team members may develop work products that will not be 

able to be backed up onto the “working documents” section of SharePoint, described in 

Section 5.6. An example of this type of work is the MELCOR calculations being completed 

on high performance computers. The personnel working on these types of files will be given 

a separate external media device that will allow them to regularly back up their work.

5.6 Working Document Folders

For this project, there is a tremendous amount of information that is part of the technical work 

performed (e.g., code calculations) that is essential to retain. This information is critical in 

being able to understand how the PRA model was ultimately constructed. To ensure that this 

information is not lost, each analyst will store their work on the SharePoint site. The site will 

have a section with a separate folder assigned to each major technical area of the study. 

These working document folders will be viewable by all members of the project team; 

however, write access will only be available to the cognizant task leader. At their request, 

task leaders can request the Documentation Coordinator to provide write access for their 

folder to other ream members (e.g., if multiple team members are collaborating on the 

development of a document or file).

Each analyst of the Level 3 PRA project will store their working files and other important 

information relevant to the project in the associated working document folder on the 

SharePoint site instead of their personal computer or some other location. Given the back-up 

features in place for the Level 3 PRA project information on the SharePoint site, this will 
Level 3 PRA Project QA Plan, Rev. 0

23

ensure that all the necessary information being used in the project is properly saved and 

stored.

5.7 Use of Templates and Forms for Documentation

As the work is being performed and decisions are being made in constructing the PRA, it is 

important to document this information. To ensure the needed amount of information is 

documented and that it is documented consistently among the analysts, documentation 

templates/forms have been created. These templates and forms (or similar documentation 

formats), which will be stored on the SharePoint site, address the following information:

• Results and resolution of reviews (i.e., TAG, self-assessment, and external peer 

reviews) – see Tables 1, 3, 4, and 7

• Criteria used for self-assessment (where no standard exists) – see Table 2

• Results of meetings: TAG, internal discussions, SNC, briefings, ACRS – see Table 9

• Working files – see Tables 9 and 10

• Technical issues and their resolution – see Table 1

During meetings, discussions, and briefings, there can be significant decisions made with 

regard to the PRA. It is essential to document this information. Table 9 provides a template 

for documenting meetings and discussions. In many instances, there may be issues that are 

identified and need to be addressed. These issues will be documented via the process 

described in Section 5.10. 

In performing the work to develop the PRA model, various information, assumptions, etc., are 

used at different stages (e.g., for the different technical elements). It is essential to document 

this information. Table 10 provides a template for documenting this information, using 

initiating event analysis as an example.

Table 8 Documentation of Meetings and Discussions

DATE:

TOPIC:

SUMMARY OF MEETING/DISCUSSION:

A high level summary of the major points.
Level 3 PRA Project QA Plan, Rev. 0

24

Table 8 Documentation of Meetings and Discussions

CONCLUSIONS

Num Decision Basis for Decision

ACTION ITEMS

Num Item Assignee Due Status

Table 9 Documentation for Level 1 Internal Events Initiating Event Analysis

Sources of Information (Inputs)

Source Description

Data

Item Value Distribution Description

Assumptions

Describe the source of information (inputs) used 

in the technical elements, the actual input may 

be attached; inputs from other tasks should also 

be included.

List each event that has a parameter value, 

provide its value and uncertainty interval and 

describe the basis for both; this may be an 

attachment to the table.

List and describe each decision made during the meeting/discussions and the bases for the 

decision; include in the discussion on the decision where and how the PRA model is impacted; 

can be a high level discussion (e.g., revised Level 1 internal events success criteria).

Describe each action item identified during the meeting/discussion, who is assigned the action 

item, the due date of the action item, and the status of the action item, including the date for the 

reported status. When completed, not “complete” with the completion date.

Describe each assumption, give the basis fro the assumption, and describe how the PRA model 

would be impacted (e.g., new initiating event, revised success criteria)
Level 3 PRA Project QA Plan, Rev. 0

25

Table 9 Documentation for Level 1 Internal Events Initiating Event Analysis

Note:

An assumption is a decision or judgment that is made in the development of the PRA model. An 

assumption is either related to a source of model uncertainty or is related to scope or level of detail. 

An assumption related to a model uncertainty is made with the knowledge that a different 

reasonable alternative assumption exists. A reasonable alternative assumption is one that has 

broad acceptance within the technical community and for which the technical basis for consideration 

is at least as sound as that of the assumption being made. An assumption related to scope or level 

of detail is one that is made for modeling convenience.

Sources of Model Uncertainty

Source Characterization

Note:

A source of model uncertainty is one that is related to an issue for which there is no consensus 

approach or model (e.g., choice of data source, success criteria, reactor coolant pump seal loss-ofcoolant accident model, human reliability model) and where the choice of approach or model is 

known to have an impact on the PRA results in terms of introducing new accident sequences, 

changing the relative importance of sequences, or significantly affecting the overall CDF, LERF, or 

LRF estimates that might have an impact on the use of the PRA in decision-making.

Documentation Criteria

Criteria Documentation Description

DOCUMENT the processes used to select, group, and screen the initiating events and to model and 

quantify the initiating event frequencies, including the inputs, methods, and results. This 

documentation includes

the functional categories considered and the 

specific initiating events included in each

the systematic search for plant-unique and plantspecific support system initiators

the systematic search for RCS pressure boundary 

failures and interfacing system LOCAs

the approach for assessing completeness and 

consistency of initiating events with plant-specific

experience, industry experience, other comparable 

PRAs and FSAR initiating events

the basis for screening out initiating events

the basis for grouping and subsuming initiating 

events

The final grouping of initiators for which accident 

sequence development will be performed

the dismissal of any observed initiating events, 

including any credit for recovery

the derivation of the initiating event frequencies and 

the recoveries used

List each source of model uncertainty, describe the source, 

and describe how the PRA model would be impacted (e.g., 

new initiating event, different grouping).

Provide a brief discussion of how the criteria 

were met; can reference another document 

that provide the necessary information.
Level 3 PRA Project QA Plan, Rev. 0

26

Table 9 Documentation for Level 1 Internal Events Initiating Event Analysis

the approach to quantification of each initiating 

event frequency

the frequencies quantified for initiating event group

the justification for exclusion of any data

Other Documentation Criteria

5.8 Site Visits

During the course of developing the Level 3 PRA model, it will be necessary for cognizant 

staff members to visit either SNC headquarters, the Vogtle plant site, or the surrounding 

Vogtle plant site area. The purpose of these visits is to (1) gather additional information not 

obtainable via documentation, and/or (2) confirm understanding of information provided.

A site visit generally involves:

• Discussions with various on-site personnel (e.g., engineering, operations, 

maintenance) and off-site personnel (e.g., local law enforcement regarding 

evacuation)

• Walk-down of the site and/or the surrounding evacuation area

To ensure that the purpose of the visit is achieved, the team leads participating in the site 

visit will prepare a site visit plan prior to the visit. This plan will be forwarded to SNC (or other 

appropriate organization) so that the licensee (or other organization) is prepared for the visit. 

The site plan will include the following:

• Dates of visit

• Names of NRC staff and contractors attending, including their role and responsibility 

in the Level 3 PRA project

• SNC or other organization personnel to be interviewed

• The places at the site (or surrounding area) to be visited

• List of questions and issues to be discussed

List any unique documentation requirements.
Level 3 PRA Project QA Plan, Rev. 0

27

It is equally important to document the results of the site visit. This documentation will 

include the following:

• Dates of the visit

• Names of NRC staff and contractors on the site visit

• Names of SNC and other organization personnel (including their position) interviewed

• Specific questions and issues discussed along with a summary of the discussion

• Site areas visited with specific observations

• Summary of discussions; should identify the specific topic and details of the 

discussion

• General observations and conclusions made as a result of the visit

If the intent of the visit is to access the actual Vogtle site, it is preferred that the NRC staff 

have unescorted access so as not to be a burden to SNC. To obtain unescorted access, the 

following must be performed:

• Each NRC staff member on the site visit must have completed NRC site access 

training (i.e., H-100 (NRC Site Access Training) or H-101 (NRC online Site Access 

Refresher Training), as appropriate) within the last 12 months

• Region II must be notified. This notification will be performed by the NRC Level 3 

PRA project manager, and will include the following information for each traveler:

 Name (as it appears on NRC badge)

 NRC badge number

 Clearance level (L, L(h), Q, or NC)

 Site access training

 Completion date of training

 Type of training (H-100, H-101, or non-NRC training at a specified 

power plant)

 Nuclear power plant/site to be visited

 Date(s) of visit

• The Region will notify the security department at the Vogtle site, by letter, of the 

upcoming visit. The letter will inform plant security that the NRC staff have the 

necessary access training and to provide them with a badge allowing unescorted 

access.

It is expected that the NRC contractors will be escorted (by NRC staff). However, Region II 

should still be notified of their participation in the visit, so that they are included in the letter 
Level 3 PRA Project QA Plan, Rev. 0

28

that the Region sends to plant security. This will facilitate the badging process. It is also 

expected that all contractors will complete the NRC site access training so that they do not 

have to undergo such training at the site. The information to be provided to the Region for 

each contractor includes:

• Name (as it appears on driver’s license)

• Company

• Site access training

 Completion date of training

 Type of training (H-100, H-101, or non-NRC training at a specified power 

plant)

• Nuclear power plant/site to be visited

• Date(s) of visit

5.9 Documentation Control for NRC Contractors

This project will involve a substantial amount of work developed by NRC contractors. For 

example, the SPAR models and SAPHIRE program were developed and are hosted by INL 

for the NRC under previous contracts. Under the Level 3 PRA contract, INL will also host the 

models for this project. It is expected that the NRC Contractors working on this project will 

have their own internal information and document control system. It is the Contracting 

Officer’s Representative’s (COR’s) responsibility to ensure that the contractor has an 

adequate plan to store and backup their work. The COR should document this finding using 

the review template.

When a document comes to the NRC from a contractor, it will be sent to the COR and 

technical lead. The technical lead will decide whether the information should be stored only 

on the SharePoint site, or also in ADAMS. In making this determination, the technical lead 

will need to consider the following factors:

• Status of the information (e.g., draft, mark-up, final product)

• Whether the document is a deliverable specified in the contract

• Likelihood that the information will ultimately be contained, in whole or in part, in 

another stored document

As general guidance, final products and other contract deliverables should be stored in 

ADAMS (as well as on the SharePoint site). Most other information will just be stored on the 

SharePoint site. Information will be stored on the SharePoint site using the procedure 

outlined in Section 5.2. The technical analyst will make the determination whether the 

information should be stored in their working document folder in SharePoint or in some other 

SharePoint location (if the latter, this should be coordinated with the Documentation 

Coordinator). Generally, contractor information that is final and is being used as reference 

material should be stored in, for example, a SharePoint location for the associated technical 

element. Contractor information that is not final should be stored in the technical analyst’s 
Level 3 PRA Project QA Plan, Rev. 0

29

associated working document folder. Additional information on the review and acceptance of 

contractor technical reports is provided in Section 6.2.

5.10 Non-Disclosure Agreement to Allow Access to Proprietary 

Information

To support the Level 3 PRA project, the NRC has collected a substantial amount of 

proprietary information about the Vogtle plant and its PRA. To ensure that the staff does not 

violate the conditions under which the licensee has provided this information, each project 

team member receives the following electronic message which they must acknowledge 

before being granted access to the proprietary information area of the Level 3 PRA 

SharePoint site:

The proprietary information submitted by SNC for Vogtle Units 1 and 2 was 

provided to the NRC on a voluntary basis and can only be used to support the 

Level 3 PRA project. In no circumstances can this information be used to 

support a regulatory decision (including, but not limited to, inspection activities 

and license reviews). Furthermore, this information shall not be redistributed 

beyond the Level 3 PRA project team. Please acknowledge your 

understanding of this information by clicking the vote button above.

5.11 Project Documentation Markings

All documents generated as part of this project (either by staff or contractors) that contain 

licensee-provided proprietary information should have each page marked with a header and 

footer that states "Official Use Only – Proprietary Information."

In addition, all documents (by either staff or contractors) that contain licensee-provided 

proprietary information and that are placed in ADAMS, should include the following disclaimer 

on the cover page:

“This document contains proprietary information voluntarily supplied by Southern Nuclear 

Operating Company to support the Level 3 PRA Project. Per NRR Office Instruction LIC204, Revision 3 (January 2007), and RES Office Instruction ADM-003, Revision 1 (May 

2012), this information should not be used to support an NRC review and approval of a 

licensee application or a document, or for any other NRC decision.”

It should be further noted that the proprietary information submitted by SNC for Vogtle Units 1 

and 2 was provided to the NRC on a voluntary basis and can only be used to support the 

Level 3 PRA project. In no circumstances will this information be used to support a 

regulatory decision (including, but not limited to, inspection activities and license 

reviews). Aside from submitting documents into ADAMS with the disclaimer above, 

documents containing SNC proprietary information should not be distributed beyond the 

Level 3 PRA project team.
Level 3 PRA Project QA Plan, Rev. 0

30

5.12 Guidance for Addressing Potential Technical Issues

In developing the Level 3 PRA model, technical issues will arise that may impact the PRA 

results or insights. These issues can include:

• potential issues that may call into question the technical rigor or adequacy of the SNC 

Vogtle PRA14 (e.g., potential model errors or deficiencies that may require changes to 

the model) or related quality control activities (e.g., self-assessment or peer review)

• issues that require a decision by the Level 3 PRA Project Management Team or 

discussion with the Level 3 PRA TAG or other experts (e.g., selection of significant 

assumptions or a choice between different analytical methods, models, or 

approaches); further technical analysis beyond that described in the TAAP; and/or 

coordination across technical areas.

An important consideration is that these issues are likely to involve proprietary PRA and plant 

information submitted by SNC that must be protected from public disclosure or misuse. SNC 

has voluntarily submitted substantial amounts of proprietary PRA and plant information to the 

NRC in support of the Level 3 PRA project and, for the reasons detailed below, this 

information is not to be used to support regulatory decisionmaking:

• Under the requirements specified in 10 CFR 2.390, “Public inspections, exemptions, 

requests for withholding,” proprietary information submitted will be withheld from 

public disclosure if it is of a type normally held in confidence by SNC. All proprietary 

information submitted by SNC is reviewed and controlled as described in Section 5.3. 

Non-proprietary versions of these documents, which would normally be submitted to 

support a license amendment or regulatory use, will not be developed to support this 

research project.

• Information submitted by SNC for this project does not support any regulatory 

decision and is not required to be done under oath and affirmation or docketed, as 

would normally be done for a licensing submittal (e.g., see 10 CFR 50.30).

• This information is not being submitted either to support a licensing application or by 

the Commission's regulations, orders, or license conditions, and consequently the 

requirements of 10 CFR 50.9, “Completeness and accuracy of information,” do not 

apply. 

Consequently, it is important to have a process for addressing potential issues that also 

ensures that appropriate separation between the Level 3 PRA project and regulatory 

decisionmaking is maintained. For this project, a process has been developed for resolving 

technical issues, communicating technical concerns to SNC staff, and turning issue follow-up 

over to the appropriate regulatory process when appropriate.

For the purposes of this process, the following terms are used:

 14 The Level 3 PRA Project is associated with only Units 1 and 2 at the Vogtle site. Units 3 and 4 

(currently under construction) are outside of the scope of this project.
Level 3 PRA Project QA Plan, Rev. 0

31

• Level 3 PRA Project Management Team – In this context, refers to the Level 3 PRA 

project Program Manager, Principle Technical Advisor, and RES/DRA/PRAB Branch 

Chief.

• Cognizant staff – project team members that include, at a minimum, the technical 

lead, but may also include other technical analysts on the project team that are 

involved with identification or resolution of the issue.

The following process is used to ensure that issues identified in the performance of the Level 

3 PRA project that have the potential to impact regulatory decisionmaking are handled in an 

appropriate manner.

1. When a Level 3 PRA project staff member or contractor identifies an issue (or 

potential issue), the cognizant staff will assess what impact the issue could have on 

the PRA (i.e., the significance of the issue) and whether the issue could call into 

question the technical rigor or adequacy of the SNC Vogtle PRA or related quality 

control activities. The cognizant staff will then summarize the issue and its potential 

impact on the PRA in a document (see Table 11). This documentation shall be 

forwarded to the Level 3 PRA Project Management Team as soon as practical after 

the issue is identified, at which point the issue will be added to the Level 3 PRA 

project issue tracking spreadsheet.

General guidance for determining whether an issue should be documented and 

tracked includes:

• Issue may call into question the technical rigor or adequacy of the SNC Vogtle 

PRA

• Issue involves a choice between different analytical methods, models, or 

approaches

• Issue requires additional work beyond that described in the TAAP

• Issue requires coordination across technical areas (e.g., an unresolved 

technical issue that has the potential to materially impact modeling decisions 

made in two or more technical areas)

• Issue warrants communication to the entire Level 3 Project team for 

awareness

• Any other issue a project team member determines should be included or 

would be of interest to the Level 3 PRA Project Management Team

2. For those issues that potentially question the technical adequacy of the SNC Vogtle 

PRA, the Program Manager will coordinate a meeting or discussion with SNC to 

address the identified issue. The meeting or discussion will include SNC staff (as 

identified by SNC), the Level 3 PRA Project Management Team, and the cognizant 

staff. The results of the meeting or discussion will be documented in accordance with 

project procedures (see Table 9 for documenting discussions/meetings). To facilitate 

the discussion, the Program Manager may forward the summary description of the 

issue (in its entirety or in part) to SNC prior to the meeting or discussion. In 

accordance with project communication protocols, the meeting/discussion will be 

coordinated with the NRR/DORL Project Manager and the SNC Licensing Director.
Level 3 PRA Project QA Plan, Rev. 0

32

a. Following the discussion with SNC (and after reviewing any additional 

information identified by SNC), the Level 3 PRA management team and 

cognizant project staff may determine that the issue is adequately resolved 

because, for example:

i. SNC provided additional information or clarification to resolve the 

issue.

ii. SNC and the NRC used different methods or approaches, both of 

which are acceptable.

iii. The issue was determined to not have a significant impact on the PRA 

results or insights.

If the issue has not been resolved, it will be evaluated by the Level 3 PRA 

Project Management Team to determine if a technical inadequacy (i.e., error) 

of the SNC Vogtle PRA actually exists. It will be assumed that any technical 

inadequacy issue that is not resolved by the cognizant staff will be considered 

to be an error in the SNC Vogtle PRA unless the Level 3 PRA Project 

Management Team determines that the issue has been resolved.

b. Once an error of the SNC Vogtle PRA has been identified, appropriate SNC 

staff will be contacted (in accordance with project communication protocols 

and in coordination with the Office of Nuclear Reactor Regulation (NRR)/

Division of Reactor Licensing (DORL) Project Manager) and informed of the 

details of the error, including the potential for the error to impact PRA results 

and/or insights or call into question the adequacy of quality control activities. 

SNC will be requested to conduct a review of the error and inform the 

NRR/DORL Project Manager of the result of this review. This review will 

include consideration of any licensing and/or regulatory applications of the 

PRA. The Level 3 PRA cognizant staff will prepare a written summary of the 

notification of SNC staff of the error which the RES/DRA/PRAB Branch Chief 

will forward to the NRR/DORL Vogtle Project Manager and appropriate 

NRR/DORL Branch Chief (either by formal memo or email notification). 

Once the error has been communicated to SNC and the NRR/DORL Project 

Manager, the Level 3 PRA project team is not responsible for any further 

follow-up on the potential regulatory implications of the error. 

c. Once the error has been turned over to NRR/DORL, it is recognized that the 

Level 3 PRA project team may proceed with an appropriate technical 

resolution consistent with the overall project objectives. The error will continue 

to be documented and tracked using Table 11 as the error is resolved within 

the context of the Level 3 PRA project. 

3. For those issues that require a decision, further technical analysis, and/or discussion 

beyond the cognizant staff, the cognizant staff member who has the lead for the issue 

will set up a project team meeting to discuss the issue. This meeting should include 

all cognizant staff and the Level 3 PRA Project Management Team. For those issues 

requiring further technical analysis, the cognizant staff, in consultation with the Level 3 

PRA Project Management Team, will determine what technical analysis will be 

performed to resolve the issue. In determining what analysis to perform, 

consideration will be given to the potential impact that the issue may have on the PRA 
Level 3 PRA Project QA Plan, Rev. 0

33

results or insights, the resources required for the additional analysis, and the 

availability of requisite staff.

The results of the meeting or discussion will be documented in accordance with 

project procedures (see Table 9 for documenting discussions/meetings). If the 

cognizant staff and Level 3 PRA Project Management Team cannot resolve the issue 

during the meeting, then one or more of the following actions will be taken:

a. The cognizant staff member who has the lead for the issue will organize a 

meeting with other knowledgeable staff or contractors.

b. The Level 3 PRA Program Manager will communicate to the TAG coordinator 

that the project team wishes to discuss the issue with the TAG.

c. The Level 3 PRA Program Manager will coordinate a meeting or discussion 

with SNC to get more information related to the issue, as needed. The 

meeting or discussion will include SNC staff (as identified by SNC), the Level 

3 PRA Project Management Team, and the cognizant staff. To facilitate the 

discussion, the Program Manager may forward the summary description of the 

issue (in its entirety or in part) to SNC prior to the meeting or discussion. In 

accordance with project communication protocols, the meeting/discussion will 

be coordinated with the NRR/DORL Project Manager and the SNC Licensing 

Director.

For all of the above actions, the results of any meetings or discussions will be 

documented in accordance with project procedures (see Table 9 for documenting 

discussions/meetings), and the issue tracking spreadsheet will be updated 

accordingly. Also, as part of the resolution of the issue, it is possible that a potential 

error or deficiency may be identified in the SNC Vogtle PRA or related quality control 

activities. If so, this issue will be addressed as discussed in Step 2, above. 

4. The different types of issues discussed above are to be tracked using Table 11 (or 

similar format). This process involves the following:

• Once the cognizant staff has entered the issue into Table 11, the table is 

forwarded to the Documentation Coordinator.

• When there is any new information related to the issue, that information is 

forwarded to the Documentation Coordinator.

• The Documentation Coordinator will update and maintain the master list of all 

the issues, which will reside on the Level 3 PRA project SharePoint site.
Level 3 PRA Project QA Plan, Rev. 0

34

Table 10 Issue Documentation Template

Technical Analyst ___________________

Status

● High Significance -- An issue needing resolution to ensure the technical adequacy of the PRA, the capability of the PRA, or the robustness of the PRA 

update process.

● Medium Significance -- An issue whose resolution is needed to maintain maximum flexibility in PRA applications and consistency with Industry practices 

(as endorsed by the NRC) or simply to enhance the PRA’s technical capability as time and resources permit. It is unlikely that the technical adequacy of the PRA is impacted. 

● Low Significance -- An issue that does not impact the technical adequacy of the PRA.

Implemented 

Resolution

Recommended 

Resolution

Level of 

Significance

Issue Num

Describe the concern, what is the 

issue, why it is a concern; 

explanation needs to clearly explain 

the concern and the basis for the 

concern. If the issue is associated 

with a specific SR, then note and. 

Describe the sources of information 

(inputs) used in the technical 

element, the actual input may be 

attached; inputs from other tasks 

should also be included.

Describe the 

recommendation to resolve 

the concern; the explanation 

needs to be sufficiently 

detailed so that the analyst 

understands what needs to 

be revised in the PRA to 

resolve the concern, and the 

basis for the 

recommended.A high A A high level summary of the 

major points discussed 

Describe the response to the 

issue and recommendation, 

describing how it was 

resolved; the explanation 

should not be just an 

“accept,” but an explanation 

of exactly how it was 

resolved (e.g., how the PRA 

model was revised), and the 

basis for the Describe each 

assumption, give the basis 

for the assumption, and 

describe how the PRA 

model would be impacted (e.g., new initiating event, 

different grouping).

The level of 

significance of 

the concern 

should be 

listed including 

the basis for 

level of 

significance 

assessed; see 

below for 

explanation 

Provide a brief 

discussion of

how the criteria 

were met; can 

reference 

another 

document that 

provides the 

evidence.

Use the technical element 2-4 digit abbreviation (xxxx) and the issue numbered sequentially (yy) followed by an “I” (i.e., xxxx-yy-I). Note, for issues that may cut 

across multiple technical elements, use the 4 digit abbreviation “CCCC.”

Describe the status, what has 

been accomplished, what still 

needs to be done, and the date 

of the status (i.e., the status will 

continually be updated, each 

time it is updated, the date needs 

to be changed accordingly). 

Once the issue has been 

resolved (i.e., the PRA model 

has incorporated the resolution), the status should state complete 

and the date completed. Part of 

the status is whether the issue 

has been reviewed by the TAG 

(the review by the TAG is 

documented elsewhere).List and 

Describe each action item 

identified during the 

meeting/discussion, who is 

assigned the action item, the due 

date of the action item, and the 

status of the action item, 

including the date for the 

reported status. When 

completed, note “complete” with 

the completion date 
Level 3 PRA Project QA Plan, Rev. 0

35

5.13 Future Plant Modifications 

One objective of the Level 3 PRA project model is to ensure it reflects the as-built, asoperated plant. However, the Level 3 PRA project will take several years to complete, and 

the plant design and operation are likely to change over time. Therefore, the potential exists 

that the Level 3 PRA project model may not reflect the as-built, as-operated plant at the time 

of project completion. Consequently, criteria are needed to determine which future 

modifications under consideration by SNC are incorporated into the model.

The following criteria are used to determine which, if any, future Vogtle plant modifications 

will be included in the NRC Level 3 PRA model:

• The potential modification is risk significant,

• There is a regulatory commitment that the proposed plant change will be completed 

by the time the Level 3 PRA model is completed, 

• If procedures and training are required, they meet the guidelines of RIS 2008-15 and 

they are implemented in a timeframe that does not impede the overall Level 3 PRA 

schedule,

• The effect of the modification has already been evaluated by the NRC (e.g., safety 

evaluation report issued) and accepted, and

• There is sufficient information for the Level 3 PRA project to understand the proposed 

change.

All of the above criteria must be met for the plant modification to be included in the Level 3 

PRA model. If one of the criteria is not met, then the plant modification will not be included. 

However, sensitivity studies may be performed to determine its impact on the PRA. The 

basis for including and not including a plant modification will be documented using Table 10.

5.14 Organization of the Various Types of Information on the 

SharePoint Site

The Level 3 PRA project has different types of information that need to be stored and 

accessed. The various types of information are summarized in Table 12. Also provided in 

Table 12 are the access control settings for the different types of information.
Level 3 PRA Project QA Plan, Rev. 0

36

Table 11 Summary of Level 3 PRA Project Information on SharePoint Site

Brief Description of Folder 

Contents

Access Control*

General L3PRA 

Project 

Documents

• General documents relating to 

the work performed in support 

of this project (e.g., briefings, 

TAAP documents)

Read/Write Access:

Documentation Coordinator

Read-Only Access:

All Level 3 PRA Project Team 

Members

No Access:

All other NRC staff

Reference 

Documents 

(Including Vogtle 

Site Information)

• Plant specific information 

previously available at the 

NRC (e.g., FSAR, IPE, 

IPEEE)

• General non-plant specific 

information (e.g., dry cask 

storage information)

• Proprietary plant specific 

information sent by Vogtle in 

support of this project (e.g., 

PRA models and 

documentation, plant 

procedures, system diagrams)

Read/Write Access:

Documentation Coordinator

Read-Only Access:

All Level 3 PRA Project Team 

Members*

No Access:

All other NRC staff

Task Group 

Technical 

Documents 

• Personal working files Read/Write Access:

Documentation Coordinator

Each team member will have 

read/write access to their own working 

files.**

Read-Only Access:

All Level 3 PRA Project Team 

Members

No Access:

All other NRC staff

Technical 

Advisory Group 

Documents

• TAG information (e.g., 

meeting minutes)

Read/Write Access:

Documentation Coordinator

TAG Coordinator

Read-Only Access:

 All Level 3 PRA Project Team 

Members

No Access:

 All other NRC staff

Inbox: Upload 

Documents to 

the L3PRA 

Website

• Miscellaneous documents 

uploaded to the site that have 

not yet been filed

Read/Write Access:

Documentation Coordinator

Read-Only Access:

All Level 3 PRA Project Team 

Members

No Access:

All other NRC staff

*To access the proprietary information area of this folder, project team members need to 

acknowledge the non-disclosure statement (as discussed in Section 5.9).

**Write access may be shared with other project team members as the request of the owning 

individual.
Level 3 PRA Project QA Plan, Rev. 0

37

6 TECHNICAL REPORTS

Before technical reports are made available outside the Level 3 PRA project, it needs to be 

determined that each is ready for release. The technical report may either be one generated 

by a Level 3 PRA project member or a contractor supporting the Level 3 PRA project.

6.1 Staff Technical Reports

There are two general types of staff technical reports developed as part of the Level 3 PRA 

project. The first type are reports that document major project milestones (e.g., the reactor, 

at-power, Level 1 PRA for internal events). The determination of whether this type of staff 

technical report is ready for release is accomplished in a four-step process and documented 

on a sign-off sheet, as described below. A second type of staff-generated technical report 

are those that support the major project reports (e.g., a report documenting a set of MELCOR 

calculations performed to resolve a specific issue or set of issues). The determination of 

whether this latter type of report is ready for release is addressed at the end of this section.

The sign-off sheet is the cover page of each technical report and involves the following:

1) The task analyst (originator) performs a final check that all the necessary steps have 

been performed. These steps include (1) completion of all the necessary 

documentation and (2) completion of the self-assessment including its documentation. 

Once the task analyst believes the work is ready for release, the analyst signs on the 

release form. By signing the form, the individual is confirming that he/she is the 

individual taking responsibility for the documented work.

2) A separate individual performs a technical review. This individual is usually someone 

associated with the project who has technical knowledge in the subject area. Once 

the findings from the reviewer are adequately addressed and resolved, the reviewer 

then signs off on the release form. By signing the form, the reviewer is confirming 

that he/she has performed a technical review and is approving the technical content, 

except where noted with comments.

3) A review is performed by a member of the Level 3 PRA Project Management Team. 

By signing the form, this individual is approving the document revision and confirming 

that it is ready for external review.

4) Once the above reviews are performed, the document is ready to be released to the 

TAG for review. This signature, generally provided by the Level 3 PRA Program 

Manager, confirms that the document revision has been provided to the TAG 

Chairman.

Each time a new version of the document is produced, a new sign-off sheet should be 

completed. In addition, a revision log should be included in the report and updated with each 

revision. The revision log includes the revision number, the date the Level 3 PRA Project 

Management Team member signed off on the revision, and a description of the major 

changes to the report and the reasons why. For the initial version of the report, “Initial 

issuance” is entered under the description heading.

Table 13 provides the template to be used for the sign-off sheet.
Level 3 PRA Project QA Plan, Rev. 0

38

Table 12 Sign-Off Sheet for Staff Technical Reports

Title of Document

________________________________________________________________________

________________________________________________________________________

_______________________________________________________________________

Revision Number: _______________________

ADAMS ML Number: _______________________

Document Approvals

ORIGINATOR: ___________________________________________________________

 Name (printed) Signature Date 

TECHNICAL

REVIEWER: ___________________________________________________________

 Name (printed) Signature Date

L3PRA MANAGEMENT

REVIEW: ____________________________________________________________

 Name (printed) Signature Date

TAG REVIEW

INITIATED: _____________________________________________________________

 Name (printed) Signature Date

Finally, in order to track the status of the technical reports (and their revision), a document 

checklist template has been developed (see Table 14). The document checklist will be 

completed collaboratively by the document originator and the Level 3 PRA Program 

Manager. The document checklists will not be included in the reports, but will be stored 

separately in a binder that will be maintained by the Level 3 PRA Program Manager.

A major inclusion as part of the documentation is to note the other participants in the work. 

This documentation should note the specific areas they contributed so that there is an 

historical account for all participants in the future. 
Level 3 PRA Project QA Plan, Rev. 0

39

Table 13 Document Checklist

Revision 0 (only)

• Applicable code and model version (e.g., SAPHIRE 8.0.9.525, Model R01) ..........____________

SAPHIRE Version ________________ Model Version ________

• Changes in PRA Model of Record verified (i.e., configuration control of model).....____________

• Site visit trip reports completed ................................................................................____________

• Document

– Consistent with format guidance (RES OI ADM-017 “Preparation of

NUREG Series Reports”)...................................................................................____________

– Proofread (e.g., good, plain English; spell check; grammar check)..................____________

– Originator signature on Reviewer Sheet............................................................____________

– Revision Number included.................................................................................____________

Revision No. ____

– Revision log sheet included after sign-off sheet................................................____________

– Acknowledgement page included......................................................................____________

• All issues on the issue tracking list addressed in the document..............................____________

• Self-assessment performed and documented .........................................................____________

• Model, document, and self-assessment documentation reflect resolution of

self-assessment comments......................................................................................____________

• Technical review performed and technical reviewer signature on Reviewer Sheet ____________

• L3 PRA project management review performed and signature on Reviewer Sheet____________

• Document entered into ADAMS and accession number included on document .....____________

SNC Review

Revision No. ____ Full Review ____ Focused Review ____

• Document sent to SNC for Fact Check and Proprietary Review .............................____________

• SNC comments added to issue tracking list ............................................................____________

• Model, document, and issue tracking list reflect resolution of SNC comments, and

new revision number (if changed)............................................................................____________

Revision No. ____

TAG Review

Revision No. ____ Full Review ____ Focused Review ____

• Document sent to TAG for review............................................................................____________

• Consensus TAG review comments added to issue tracking list ..............................____________

• Model, document, and issue tracking list reflect resolution of consensus TAG

comments, and new revision number (if changed) ..................................................____________

Revision No. ____

PWROG Peer Review

Revision No. ____ Full Review ____ Focused Review ____

• PWROG Peer Review (PPR) readiness letter sent 12 weeks in advance of PPR..____________

• NRC support to PPR team identified and confirmed with PWROG.........................____________

• Material sent to PPR team 4 weeks in advance of PPR..........................................____________

• Logistics for PPR meeting performed (e.g., rooms, equipment, documentation)....____________
Level 3 PRA Project QA Plan, Rev. 0

40

Table 13 Document Checklist

• Outstanding items during PPR addressed and resolved (on-site PPR completed) ____________

• Draft PPR report sent to NRC for review and comment..........................................____________

• NRC comments on PPR report sent to PPR team ..................................................____________

• Final PPR report issued (ADAMS No. ____________________)...........................____________

• PPR comments added to issue tracking list ............................................................____________

• Model, document, and issue tracking list reflect resolution of PPR comments, and

new revision number (if changed)............................................................................____________

Revision No. ____

ACRS Review

Revision No. ____ Full Review ____ Focused Review ____

• Document sent to ACRS..........................................................................................____________

• Presentation prepared and approved for ACRS briefing .........................................____________

• ACRS briefed on: ______________________........................................................____________

• L3 PRA management identified ACRS comments to be added to issue tracking 

list ......................................................................................................................____________

• Model, document, and issue tracking list reflect resolution of ACRS comments, and

new revision number (if changed)............................................................................____________

Revision No. ____

Final L3 PRA Project Management Approval

• L3 PRA project management approval of document as Revision No. ____ ...........____________

Revisions

• Originator modifies model and document as appropriate based on new 

information.........................................................................................................____________

– Applicable code and model version (e.g., SAPHIRE 8.0.9.525, Model R01) ...____________

SAPHIRE Version ________________ Model Version ________

– Verify changes in PRA Model of Record (i.e., configuration control of model).____________

• Originator signature on Reviewer Sheet..................................................................____________

– Revision Number included ................................................................................____________

Revision No. ____

– Revision log sheet updated ...............................................................................____________

• Self-assessment performed and documented for modified portion of model..........____________

• Model, document, and self-assessment documentation reflect resolution of

self-assessment comments .....................................................................................____________

• Technical review performed for modified portion of model and technical reviewer

signature on Reviewer Sheet...................................................................................____________

• Document entered into ADAMS and accession number included on document

(new accession number for each revision) ..............................................................____________

• L3 PRA project management review performed and signature on Reviewer 

Sheet .................................................................................................................____________

Additional reviews required (L3 PRA management enters “full,” ”focused,” or

“none,” as appropriate):

SNC for Fact Check and Proprietary Review ____________

TAG review ____________
Level 3 PRA Project QA Plan, Rev. 0

41

Table 13 Document Checklist

PWROG review ____________

ACRS review ____________

L3 PRA project management approval on revised document issued as 

Revision No. ____....................................................................................................____________

Comments: _____________________________________________________________________

_______________________________________________________________________________

_______________________________________________________________________________

_______________________________________________________________________________

_______________________________________________________________________________

_______________________________________________________________________________

As discussed at the beginning of this section, a second type of staff-generated reports are 

those that support the major project reports (e.g., a report documenting a set of MELCOR 

calculations performed to resolve a specific issue or set of issues). The determination of 

whether this type of staff technical report is ready for release is accomplished in a two-step 

process and documented on a sign-off sheet, as described below.

The sign-off sheet is the cover page of each technical report and involves the following:

1) The task analyst (originator) performs a final check that the report is complete and 

appropriately formatted. Once the task analyst believes the work is ready for release, 

the analyst signs on the release form. By signing the form, the individual is confirming 

that he/she is the individual taking responsibility for the documented work.

2) A separate individual performs a technical review. This individual is usually someone 

associated with the project who has technical knowledge in the subject area. Once 

the findings from the reviewer are adequately addressed and resolved, the reviewer 

then signs off on the release form. By signing the form, the reviewer is confirming 

that he/she has performed a technical review and is approving the technical content, 

except where noted with comments.

Each time a new version of the document is produced, a new sign-off sheet should be 

completed. In addition, a revision log should be included in the report and updated with each 

revision. The revision log includes the revision number, the date the technical reviewer 

signed off on the revision, and a description of the major changes to the report and the 

reasons why. For the initial version of the report, “Initial issuance” is entered under the 

description heading.

Table 15 provides the template to be used for the sign-off sheet.
Level 3 PRA Project QA Plan, Rev. 0

42

Table 14 QA Review and Acceptance Form for Staff (Supporting) Technical Reports

Title of Document

_________________________________________________________________________________________

_________________________________________________________________________________________

______________________________________ 

Revision Number: _______________________

ADAMS ML Number: _______________________

Document Approvals:

ORIGINATOR: ____ ______________________________________________

 Name (printed) Signature Date

TECHNICAL

REVIEWER: __ _________________________________________________

 Name (printed) Signature Date

6.2 Contractor Technical Reports

The determination of whether a contractor technical report is ready for release is 

accomplished through an acceptance review by the Level 3 PRA project task leader. When 

NRC staff sign off on a project document that includes contractor work in either the main 

report or an attachment, or references contractor work, the staff are not necessarily 

guaranteeing the technical adequacy of the contractor work, but are confirming that the work 

is appropriate for the project objectives, that the context of the work is consistent with other 

parts of the overall analysis, and that they can “talk” to the work at a high level to a third party 

audience.

Once the task leader believes the work is ready for release, the leader signs on the QA 

review and acceptance form (see Table 16). By signing the form, the individual is confirming 

that he/she is the individual taking responsibility for the documented work.
Level 3 PRA Project QA Plan, Rev. 0

43

Table 15 QA Review and Acceptance Form for Contractor Technical Reports

Title of Document

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

Revision Number: _______________________

ADAMS ML Number: _______________________

Document Approval

L3PRA TECHNICAL

LEAD: 

____________________________________________________________________

 Name (printed) Signature Date

7 QA PROGRAM IMPLEMENTATION AUDITS

Periodic audits of the implementation of the Level 3 PRA project QA plan will be performed 

and will cover a representative sampling of project activities in order to verify compliance with 

QA plan requirements. The Level 3 PRA Project Management team will determine the 

scheduling of these audits and how they are to be carried out.