Initial Issuance - December 2005

USNRC STANDARD REVIEW PLAN

Standard review plans are prepared for the guidance of the Office of Nuclear Reactor Regulation staff responsible for the review of applications to construct and operate nuclear power plants. 

These documents are made available to the public, as sections of NUREG-0800, as part of the NRC’s policy to inform the nuclear industry and the general public of regulatory procedures and

policies. Standard review plans are not substitutes for regulatory guides or the NRC’s regulations, and compliance with them is not required. The standard review plan sections are keyed to

the Standard Format and Content of Safety Analysis Reports for Nuclear Power Plants (Regulatory Guide 1.70). Not all sections of the standard format have a corresponding review plan.

Published standard review plans will be revised periodically, as appropriate, to accommodate comments and to reflect new information and experience. Comments and suggestions for

improvement will be considered and should be sent to the U.S. Nuclear Regulatory Commission, Office of Nuclear Reactor Regulation, Washington, DC 20555.

Requests for single copies of draft or active SRP sections (which may be reproduced) should be made to the U.S. Nuclear Regulatory Commission, Washington, DC 20555, Attention: 

Reproduction and Distribution Services Section, or by fax to (301) 415-2289, or by email to DISTRIBUTION@nrc.gov. Electronic copies of this section are available through the NRC’s public

Web site at http://www.nrc.gov/reading-rm/doc-collections/nuregs/staff/sr0800/#c15, or in the NRC’s Agencywide Documents Access and Management System (ADAMS),

at http://www.nrc.gov/reading-rm/adams.html, under Accession #ML053550265.

NUREG-0800

(Formerly NUREG-75/087)

U.S. NUCLEAR REGULATORY COMMISSION

STANDARD REVIEW PLAN

OFFICE OF NUCLEAR REACTOR REGULATION

15.0.2 REVIEW OF TRANSIENT AND ACCIDENT ANALYSIS METHODS

REVIEW RESPONSIBILITIES

Primary - Lead Organization with Responsibility for Review of Transient and Accident Analyses

Secondary - Other Organizations with Responsibility for Review of Transient and Accident Analyses

I. AREAS OF REVIEW

In order to establish a licensing basis, licensees must analyze transients and accidents in accordance

with the requirements of 10 CFR 50.34, 10 CFR 50.46, and where applicable, per NUREG-0737,

“Clarification of TMI Action Plan Requirements.” These accidents and transients are described

in the Standard Review Plan (SRP) (NUREG-0800) [Ref. 1]. This section of the SRP describes

the review process and acceptance criteria for analytical models and computer codes used by

licensees to analyze accident and transient behavior. The purpose of the review is to verify that

the evaluation model is adequate to simulate the accident under consideration. This includes

methods to estimate the uncertainty in the calculation, as in the case of a best estimate

loss of coolant accident (LOCA); or to ensure that the results of the analysis are demonstrably

conservative, as in the case of a LOCA in Appendix K to 10 CFR Part 50. This review is independent of

the type of application submitted (e.g., license amendment, topical report, design certification,

or combined license application).

The guidance in this section should be applicable to most of the transients and accidents described

in the SRP. Appendices that provide specific information for specific classes of accidents and transients

will be developed in the future. Guidance to the industry is set forth in Regulatory Guide 1.203,

“Transient and Accident Analysis Methods” [Ref. 2]. The following sections discuss the areas

to be reviewed.

1. Documentation

The development of an evaluation model for use in reactor safety licensing calculations requires a

substantial amount of documentation. This documentation includes/covers (a) the evaluation

model, (b) the accident scenario identification process, (c) the code assessment, (d) the uncertainty

analysis, (e) a theory manual, (f) a user manual, and (g) the quality assurance program.
15.0.2-2 Initial Issuance - December 2005

2. Evaluation Model

An evaluation model is the calculation framework for evaluating the behavior of the reactor

coolant system during a postulated accident or transient. It includes one or more computer

programs and other information necessary for application of the calculation framework

to a specific transient or accident, such as mathematical models used, assumptions included

in the programs, a procedure for treating the program input and output information,

specification of those portions of the analysis not included in the computer programs,

values of parameters, and other information necessary to specify the calculation procedure. 

Evaluation models are sometimes referred to as a licensing methodology.

3. Accident Scenario Identification Process

The accident scenario identification process is a structured process used to identify and rank

the reactor component and physical phenomena modeling requirements based on (a) their

importance to acceptable modeling of the scenario and (b) their impact on the figures

of merit for the calculation (e.g., peak cladding temperature and maximum and average

cladding oxidation thicknesses). It is also used to identify the key figures of merit

or acceptance criteria for the accident.

4. Code Assessment

The code assessment provides a complete assessment of all code models against

applicable experimental data and/or exact solutions in order to demonstrate that the code is

adequate for analyzing the chosen scenario.

5. Uncertainty Analysis

Uncertainty analyses are performed to confirm that the combined code and application

uncertainty is less than the design margin for the safety parameter of interest when the code

is used in a licensing calculation. Examples of safety parameters are peak cladding

temperature (PCT), cladding oxidation thickness, departure from nucleate boiling ratio

(DNBR), and critical power ratio (CPR).

6. Quality Assurance Plan

The quality assurance plan covers the procedures for design control, document control,

software configuration control and testing, and error identification and corrective actions

used in the development and maintenance of the evaluation model. The program also

ensures adequate training of personnel involved with code development and maintenance,

as well as those who perform the analyses.

II. ACCEPTANCE CRITERIA

The acceptance criteria are based on meeting the requirements of the regulations in

10 CFR Part 50 that govern the evaluation models for the specific accident under consideration

(e.g., 10 CFR 50.46 for a LOCA). The following sections discuss the specific criteria.

1. Documentation

The submittal must identify the specific accident scenarios and plant configurations for

which the codes will be used. The evaluation model documentation must be scrutable,
15.0.2-3 Initial Issuance - December 2005

complete, unambiguous, accurate, and reasonably self-contained. Consistent nomenclature

must be used throughout the entire model documentation. Any referenced material must be

readily available from a technical library. Copies of any referenced documents that are not

readily obtainable from a technical library or the NRC Public Document Room, including

proprietary reports, must be included with the documentation or provided upon request. 

The code documentation must be sufficiently detailed that a qualified engineer can

understand the documentation without recourse to the originator as required of any design

calculation that meets the design control requirements of Appendix B to 10 CFR Part 50,

and the documentation requirement in Appendix K to 10 CFR Part 50. It is desirable that

the documentation include the responses to requests for additional information, sorted

according to the review issue so that it is easy to follow the entire review history for a single

issue. The reviewer can help obtain this goal by issuing RAI’s organized by review issue. 

The documentation must include the following components:

A. An overview of the evaluation model which provides a clear roadmap

describing all parts of the evaluation model, the relationships between them,

and where they are located in the documentation.

B. A complete description of the accident scenario including plant initial

conditions, the initiating event and all subsequent events and phases of the

accident, and the important physical phenomena and systems and/or

component interactions that influence the outcome of the accident.

C. A complete description of the code assessment comprising a description of

each assessment test, why it was chosen, success criteria, diagrams of the

test facility that show the location of instrumentation that is used in the

assessment, a code model nodalization diagram, and all code options used in

the calculation.

D. A determination of the code uncertainty for a sample plant accident

calculation. (Appendix K models do not require a determination of the code

uncertainty.)

E. A theory manual that is a self-contained document and that describes

(a) field equations, (b) closure relationships, (c) numerical solution techniques,

(d) simplifications and approximations (including limitations) inherent in

the chosen field equations and numerical methods, (e) pedigree or origin

of closure relationships used in the code, and (f) limits of applicability for all

models in the code.

F. A user manual that provides (a) detailed instructions about how the computer

code is used, (b) a description of how to choose model input parameters and

appropriate code options, (c) guidance about code limitations and options that

should be avoided for particular accidents, components, or reactor types, and

(d) if multiple computer codes are used, documented procedures for ensuring

complete and accurate transfer of information between different elements of

the evaluation model.

G. A quality assurance plan that describes the procedures and controls under

which the code was developed and assessed, and the corrective action

procedures that are followed when an error is discovered.
15.0.2-4 Initial Issuance - December 2005

It is not important that the documentation be provided in exactly the format stated above

but the information in the review package must be clearly organized in a reasonable manner.
15.0.2-5 Initial Issuance - December 2005

2. Evaluation Model

Models must be present for all phenomena and components that have been determined

to be important or necessary to simulate the accident under consideration. The chosen

mathematical models and the numerical solution of those models must be able to predict

the important physical phenomena reasonably well from both qualitative and quantitative

points of view. The degree of imprecision that is allowed in the models will ultimately be

determined by the amount of uncertainty that can be tolerated in the calculation. Models

that cause non-physical predictions to the extent that misinterpretation of the calculated

results or trends in the results may occur, are not acceptable. For Appendix K LOCA

analyses, emergency core cooling system (ECCS) evaluation models must meet the specific

requirements contained in Appendix K to 10 CFR Part 50.

3. Accident Scenario Identification Process

The purpose of the accident scenario identification process is to identify and rank the

reactor component and physical phenomena modeling requirements based on (a) their

importance to the modeling of the scenario and (b) their impact on the figures of merit for

the calculation. The accident scenario identification process must be a structured process. 

It must include evaluation of physical phenomena to identify those that are important in

determining the figure of merit for the scenario. The models that are present in the code

and their degree of fidelity in predicting physical phenomena must be consistent with

the results of this process. For example, if the accident scenario identification process

determines that a certain physical phenomenon is important to the scenario under

consideration, the code must have a relatively accurate model for that phenomenon

and a detailed assessment of that model must be provided. Phenomena that have lower

ranking may be represented by models with larger inherent uncertainty. The formality

and complexity of this process should be commensurate with the complexity

and importance of the event under consideration.

4. Code Assessment

Assessments of all code models intended to be used in the evaluation model must be

provided. All assessments must be performed with the frozen version of the evaluation

model that has been submitted for review. Assessments performed with other versions

of the evaluation model should be justified on a case by case basis because even “small”

changes to the evaluation model can have unintended consequences on calculation results

that were thought to not be impacted by the changes.

Separate effects testing must be performed to demonstrate the adequacy of the physical

models to predict physical phenomena that were determined to be important by the accident

scenario identification process. Separate effects testing must also be used to determine

the uncertainty bounds of individual physical models.

Integral effects testing must be performed to demonstrate that the interactions between

different physical phenomena and reactor coolant system components and subsystems

are identified and predicted correctly.
15.0.2-6 Initial Issuance - December 2005

Assessments against both separate effects tests and integral effects tests must be performed

with the code. All models need to be assessed over the entire range of conditions

encountered in the transient or accident scenario. Assessments must also compare code

predictions to analytical solutions, where possible, to show the accuracy of the numerical

methods used to solve the mathematical models. Code options used in the assessment

calculations must be the same as those used in plant accident calculations.

A scaling analysis must be performed that identifies important non-dimensional parameters

related to geometry and key phenomena. Scaling distortions and their impact on the code

assessment must be identified and evaluated in the assessment. Calculations of actual plant

transients or accidents can be considered, but only as confirmatory supporting assessments

for the evaluation model. This is because the data available from plant instrumentation

is usually not detailed enough to support code assessment of specific models. Plant data

can be used for code assessment if it can be demonstrated that the available instrumentation

provides measurements of adequate resolution to assess the code. The assessment cases

must compare code predictions to all important measured variables in order to show that

good predictions of one test variable do not result from compensating errors. Assessments

must include a description of all assessment cases, specific models that are being assessed

in each case, and acceptance criteria used. Acceptance criteria must be supported

by quantitative analysis whenever possible.

ECCS evaluation models must include a specific assessment to meet the criteria in

Appendix K to 10 CFR Part 50. Small-break ECCS evaluation models must also meet

the assessment requirements of TMI Action Item II.K.3.30, where applicable.

5. Uncertainty Analysis

The uncertainty analysis must address all important sources of code uncertainty, including

the mathematical models in the code and user modeling such as nodalization. The major

sources of uncertainty must be addressed consistent with the results of the accident sequence

identification process. When the code is used in a licensing calculation, the combined code

and application uncertainty must be less than the design margin for the safety parameter

of interest. The analysis must include a sample uncertainty evaluation for a typical plant

application.

In some cases, bounding values are used for input parameters as described in SRP sections

or Regulatory Guides and are used for plant operating conditions such as accident initial

conditions, set points, and boundary conditions.

6. Quality Assurance Plan

The code must be maintained under a quality assurance program that meets the requirements

of Appendix B to 10 CFR Part 50.
15.0.2-7 Initial Issuance - December 2005

III. REVIEW PROCEDURES

1. Assignment of Review Responsibilities

The lead reviewer for the evaluation model should perform an assessment of the submittal

in order to determine what areas of reviewer expertise are required to begin the review process. 

Evaluation models often contain complex mathematical and physical models that

encompass multiple scientific and engineering disciplines. The lead reviewer should identify

qualified personnel and their areas of review responsibility. Reviews often require personnel

with highly specialized areas of expertise, which may not be available in the NRR technical

staff.

2. Acceptance Review

An acceptance review is a preliminary process of assessment of the completeness

and apparent acceptability of quality of the documents submitted in order to commit NRC

to a detailed review of the application. The reviewers of the code or codes comprising

the evaluation model should perform an acceptance review consistent with the guidance

provided in NRC Office Instruction No. LIC-101, Revision 1, “Licence Amendment Review

Procedures,” dated March 27, 2002 (Ref. 3). The reviewers should ensure that all areas

discussed in the section titled Documentation are addressed in the topical report submitted

by a licensee or vendor. The documentation is not required to have the same section titles

or organization as described above, but all of the relevant subject matter must be included. 

The material must be sufficiently detailed to permit the reviewers to begin a review without

immediately requesting additional documentation in order to understand the licensing submittal.

The results of the acceptance review should be documented in a letter to the organization

submitting the evaluation model for review. Submittals that do not contain the required

material should also be processed in accordance with NRC Office Instruction No. LIC-101,

Revision 1, “Licence Amendment Review Procedures” (Ref. 3). The reviewers should document

what material is missing from the submittal. This information should then be provided

in the letter to the submitting organization so that the material can be revised to meet

acceptance review requirements for any future submittals. After a successful completion

of the acceptance review, a detailed review of the code documentation may proceed.

3. Detailed Review

a. Documentation

The reviewers should review the documentation to determine if (i) all documentation listed

in Section II.1 above has been provided, (ii) the evaluation model overview provides

an accurate roadmap of the evaluation model documentation, (iii) all documentation

is accurate, complete, and consistent and, (iv) all symbols and nomenclature have been

defined and consistently used.

The reviewers should confirm that the theory manual is a self-contained document and that

it describes the field equations, closure relationships, numerical solution techniques,

and simplifications and approximations (including limitations) inherent in the chosen field

equations and numerical methods. The reviewers should also confirm that the theory

manual identifies the pedigree or origin of closure relationships used in the code

and the limits of applicability for all models in the code.
15.0.2-8 Initial Issuance - December 2005

The reviewers should confirm that the user manual provides guidance for selecting

or calculating all input parameters and code options. The guidance must be clear

and unambiguous. As a result of the code development or review process many code options

may be determined to be inappropriate for specific licensing calculations. The reviewers

should confirm that the guidance in the manual specifies the required and acceptable code

options for the specific licensing calculations. The reviewers should also confirm that

required input settings are hardwired into the input processor so that the code stops

with an error message if the required input is not provided or if the input is not within

an acceptable range of values or that administrative controls ( an independent reviewer

QA check) are in place that accomplish the same purpose. The reviewers should confirm

that computer codes that are used for multiple accidents and transients include guidelines

that are specific to each transient or accident. Code assessment cases can be examined

to ensure that the modeling used in the assessment cases is consistent with the user

guidelines.

b. Evaluation Model

The reviewers should determine whether the mathematical modeling and computer codes

used to analyze the transient or accident have been previously reviewed and accepted. 

For changes to previously approved models, the reviewers can limit their review to the new

material if they determine that there is nothing new that will invalidate the previous approval,

including the range of applicability for the analysis method. Otherwise the entire model

must be reviewed. For a new model that has not been previously reviewed, the reviewers

initiate an evaluation of the entire analytical model.

The reviewers should determine if the physical modeling described in the theory manual

and contained in the mathematical models is adequate to calculate the physical phenomena

influencing the accident scenario for which the code is used. A scenario will have a set

of governing physical phenomena that drive the results of the calculation. The key physical

phenomena, including constitutive equations needed for model closure, must be defined

for the calculation being performed by the code. Physical phenomena that are important

for one accident scenario may not be important for a different accident scenario. The key

physical phenomena can also be specific to a particular plant design.

The mathematical equations that comprise an evaluation model can be characterized as

being either a field equation or as a constitutive or closure relationship. Field equations are

a set of rigorously derived equations that contain no approximations other than the initial

assumptions used in deriving the equations. The range of applicability of the field equations

is limited only by the validity of the assumptions used in their derivation. An example

of a set of field equations is the set of fluid transport equations for mass, momentum,

and energy that are derived from macroscopic balances of these quantities. Although these

equations are mathematically exact, the equations contain more unknown quantities than

there are equations. Some of these unknown quantities are the equation of state, the stress

tensor, and the heat flux. In order to be able to solve the field equations the unknown

quantities must be expressed in terms of the known quantities from the field equations. 

The equations that relate the unknown quantities to the known quantities are constitutive

or closure relationships. These equations are often models or approximations that are much

more restrictive in their range of validity than the set of field equations that they are used

with. For example, using Newton’s law of cooling as a closure relationship for heat flux
15.0.2-9 Initial Issuance - December 2005

from a heat structure to a fluid will limit the application to model situations where radiation

heat transfer is not significant even though the field equations are valid. The reviewer must

therefore ensure that the field equations of the evaluation model are adequate to describe

the set of physical phenomena that occur in the accident and ensure that the closure

relationships are valid over the full range of conditions encountered during the accident.

The modeling must be consistent with the results of the accident scenario identification

process in that there must be models for all important phenomena in the accident scenario. 

Components and physical phenomena that are identified as being important in the accident

scenario identification process must be modeled with a high degree of fidelity. Phenomena

of lower importance may be represented by less accurate models.

The reviewers should determine if the simplifying assumptions and assumptions used in

the averaging procedure are valid for the accident scenario under consideration. Simplifying

assumptions and averaging are often applied to detailed physical and mathematical modeling

to obtain simplified mathematical models that can be solved more readily and with less

computational effort. Examples of common simplifications are incompressible flow models,

one-dimensional flow models, common two-phase flow models such as the homogeneous

equilibrium model (HEM), drift flux, and the two-fluid model, and simplified reactor kinetics

models such as point kinetics or one-dimensional kinetics. Even models commonly thought

of as detailed models usually contain simplifying assumptions and averaging procedures

applied to first-principles models. Reviewers should confirm that justifications are provided

for all simplifications, assumptions, and averaging.

The reviewers should confirm that the level of detail in the model is equivalent to

or greater than the level of detail required to specify the answer to the problem of interest. 

For example, a one-dimensional flow model can not provide information about the velocity

profile in the vicinity of a pipe bend or the degree of thermal stratification in a horizontal

pipe. A detailed, three-dimensional flow model would be needed to provide this type

of information.

The reviewers should confirm that the equations and derivations are correct. There must

be sufficient text to adequately describe the derivation, including all assumptions

and equations. The derivations must be sufficiently detailed to allow the reviewers

to understand the logical progression of steps involved in the derivation. Simplifying

assumptions must have a technical justification and a range of validity associated with them.

Models that are typically used in nuclear reactor analysis are highly phenomenological

and/or empirical in nature. They are either proposed using physical or engineering judgment

based on observations of experimental data or derived using averaging procedures applied

to detailed first-principle models. These models often represent processes that occur on

length and time scales that are too small to be resolved in the computation or processes

that we do not have sufficient understanding to model from first principles. These models

require closure relationships based on information from experimental measurements or

detailed first-principle calculations. The reviewers should ensure that the range of validity

of the closure relationships is specified and is adequate to cover the range of conditions

encountered in the accident scenario. This is especially true of empirical correlations

that are derived directly from experimental data without recourse to any physical modeling.
15.0.2-10 Initial Issuance - December 2005

In most applications, especially those with a large number of processes and parameters,

it is difficult, if not impossible, to design test facilities that preserve total similitude between the

experiment and the nuclear power plant. Therefore, optimum similarity criteria are

identified and scaling rationales developed for selecting existing data or designing and

operating experimental facilities. The reviewers should confirm that the similarity criteria

and scaling rationales are based on the important phenomena and processes identified by

the accident scenario identification process and appropriate scaling analyses.

The reviewers should confirm that scaling analyses were conducted to ensure that the data

and the models will be applicable to the full scale analysis of the plant transient. Scaling

compromises that are identified must be addressed in the bias and uncertainty evaluation. 

The experimental data base must be demonstrated to be sufficiently diverse, so that the

expected plant specific response is bounded and that the evaluation model calculations are

comparable to the corresponding tests in non-dimensional space. This demonstration allows

extending the conclusions relating to code capabilities, drawn from assessments comparing

calculated and measured test data to the prediction of plant specific transient behavior.

c. Accident Scenario Identification Process

The accident scenario identification process is required in order to determine the needed

modeling and assessment requirements for the code. The accident scenario identification

process is also needed to identify and rank the reactor component and physical phenomena

modeling requirements based on their importance to acceptable modeling of the scenario

and their impact on the figures of merit for the calculation. This process is highly dependent

on the type of reactor and the accident scenario of interest.

Often a single computer code is used to analyze multiple accident or transient classes. 

A separate accident scenario identification description is needed for each accident or transient

class for which the code is to be used in order to describe the accident progression

and dominant physical phenomena for that particular accident. This description must explicitly

reference code models and assessment cases that are specifically applicable to each

scenario to avoid confusion when the same code is used for multiple accident scenarios.

The processes and phenomena that evaluation models should simulate are found by examining

experimental data and experience, as well as code simulations related to the specific scenario. 

Independent techniques to accomplish the ranking include expert opinion, selected

calculations, and decision-making methods.

The reviewers should confirm that the process used for accident scenario identification

is a structured process. The reviewers should confirm that the description of each accident

scenario provides a complete and accurate description of the plant initial and boundary

conditions and the accident progression. The specified initial and boundary conditions must

comply with values required by regulations or specified as acceptable in a regulatory guide

or SRP section that covers the accident. The reviewers should confirm that the dominant

physical phenomena influencing the outcome of the accident are correctly identified and

ranked. The reviewers should keep in mind that the initial phases of the accident scenario

identification process can rely heavily on expert opinion and can be subjective. Therefore,

iteration of the process, based on experimentation and analysis, is important.
15.0.2-11 Initial Issuance - December 2005

One example of an acceptable structured process is the phenomena identification and ranking

table (PIRT) process, which is described in NUREG/CR-5249, “Quantifying Reactor Safety

Margins: Application of Code Scaling, Applicability, and Uncertainty Evaluation Methodology

to a Large-Break Loss-of-Coolant Accident” [Ref. 4]. The process is also described in

a series of papers in Nuclear Engineering Design [Ref. 5].

d. Code Assessment

The reviewers should confirm that the code assessment adequately covers all of the important

code models and the full range of conditions encountered in the accident scenarios. 

The assessment must be consistent with the accident scenario identification process in that all

models must have assessment commensurate with their importance and required fidelity. 

The reviewers should also verify that all assessment cases were performed with a single

version of the evaluation model.

The reviewers should confirm that the numerical solution conserves all important quantities. 

Even when the mathematical equations conserve mass, momentum, and energy,

the numerical method used to solve the equations may not conserve any of these quantities. 

The reviewers should also confirm that all code options that are to be used in the accident

simulation are appropriate and are not used merely for code tuning.

The reviewers should ensure that all code closure relationships based in part on experimental

data or more detailed calculations have been assessed over the full range of conditions

encountered in the accident scenario by means of comparison to separate effects test data. 

Scaling analyses may be needed to demonstrate that the assessment results apply to the

full-scale plant accident conditions. Even closure relationships such as equations of state

and material properties, which are based on interpolating functions, need to be assessed

against standards for the properties. For example, a relatively small error in thermodynamic

properties such as phasic densities as a function of temperature and pressure may cause

a larger error when propagated to a neutron kinetics model in a main steamline break

or an anticipated transient without scram (ATWS) calculation.

The reviewers should ensure that integral test assessments properly demonstrate physical

and code model interactions that are determined to be important for the full size plant

accident scenarios. The integral test will usually not be full-scale, and therefore will contain

scaling distortions. These distortions can affect both local and overall elements

of the analysis, such as two-phase flow regime transitions and global dynamic response

of the test facility, when compared to the full scale plant. The reviewers should ensure

that the documentation contains comparisons of all important experimental measurements

with the code predictions in order to expose possible cases of compensating errors. 

Such errors may result in good predictions of key parameters, derived from poor predictions

of contributing parameters. These cases are often indicative of tuning the code

to an integral experiment. One example of this is described in NUREG/CR-5249 [Refs. 4

and 5], in which key parameters, namely large-break LOCA (LBLOCA) reflood peak cladding

temperatures were predicted well, even though an important contributing factor, the core

void fraction during reflood, was not predicted accurately.
15.0.2-12 Initial Issuance - December 2005

In the case of LOCA evaluation models, specific assessment test cases are required in order to

meet the requirements of Appendix K to 10 CFR Part 50. Specific test cases are also

specified in the TMI action items for PWR small-break LOCA (SBLOCA) evaluation models. 

The reviewers should confirm that these assessments have been performed where applicable.

The reviewers should refer to published literature for sources of assessment data for specific

phenomena, accident scenarios, and plant types. These include the ECCS Compendium

[Ref. 6], NEA validation data documents [Refs. 7, 8, and 9], and International Standard

problems.

e. Uncertainty Analysis

The reviewers should confirm that the method for calculating uncertainty contains all

important sources of uncertainty and that a sample uncertainty calculation for a prototypical

plant gives a reasonable estimate of the calculation uncertainty. The reviewers should

confirm that the accident scenario identification process was used in identifying

the important sources of uncertainty.

The reviewers should confirm that sources of code uncertainty have been addressed. 

These include uncertainties in theoretical models or closure relationships determined from

comparison to separate effects tests, uncertainties due to scaling of the basic models

and closure relationships, and uncertainties due to plant nodalization and solution techniques. 

The reviewers can sometimes determine whether the code uncertainty is reasonable

by applying a simple analytical model. This can be done if the dominant contribution

to uncertainty is confined to a small number of models. In other cases, detailed audit

calculations may be needed to confirm the estimate of uncertainty in a calculation.

The reviewers should confirm that sources of calculation uncertainties have been addressed,

including uncertainties in plant model input parameters for plant operating conditions (e.g.,

accident initial conditions, set points, and boundary conditions). Calculation uncertainties

are specific to a given licensing calculation. Therefore, it is sometimes acceptable to use

bounding values in the licensing calculation for these types of uncertainties.

The reviewers should confirm that the uncertainties in the experimental data base have been

addressed. These uncertainties arise from such items as measurement errors and

experimental distortions. For separate effects tests and integral effects tests, the reviewers

should confirm that the differences between calculated results and experimental data

for important phenomena have been quantified for bias and deviation. The reviewers should

confirm that data sets and correlations with experimental uncertainties that are too large

when compared to the requirements for evaluation model assessment are not used.

When the code is used in a licensing calculation, the reviewers should confirm that

the combined code and application uncertainty is less than the design margin for the safety

parameter of interest in the calculation.

For best estimate LOCA analyses, uncertainty determination description and guidance

are described in NUREG/CR-5249 [Ref. 4], Regulatory Guide 1.157, and Appendix A

to Regulatory Guide 1.203 [Ref. 2]. In these examples, the uncertainty analyses discussed

have the ultimate objective of providing a singular statement of uncertainty with respect to
15.0.2-13 Initial Issuance - December 2005

the 10 CFR 50.46 acceptance criteria. This singular uncertainty statement is accomplished

when the individual uncertainty contributions are determined (see Regulatory Guide 1.157,

Ref. 10).

For other events, a complete uncertainty analysis is not required. However, in most cases

the SRP guidance is to use “suitably conservative” input parameters. This suitability

determination may involve a limited assessment of biases and uncertainties. The individual

uncertainty (in terms of range and distribution) of each key contributor is determined from

the experimental data, input to the nuclear power plant model, and the effect on appropriate

figures of merit evaluated by performing separate calculations. The figures of merit

and devices chosen must be consistent.

The NRC has developed the Code Scaling, Applicability, and Uncertainty (CSAU)

methodology for code uncertainty evaluation. The CSAU process has been demonstrated

for LBLOCA [Refs. 4 and 5] and boiling-water reactor ATWS [Ref. 11]. Other methods

of uncertainty evaluation may be acceptable.

f. Quality Assurance Plan

The reviewers should confirm that the evaluation model is maintained under a quality

assurance program that meets the requirements of Appendix B to 10 CFR Part 50. 

As a minimum, the program must address design control, document control, software

configuration control and testing, and corrective actions. The reviewers should confirm

that the quality assurance program documentation includes procedures that address all

of these areas. The reviewers may conduct an audit of the implementation of the code

developer’s quality assurance program.

The reviewers should confirm that independent peer reviews were performed at key steps

in the evaluation model development process. The peer review team should include

programmers, developers, end users, and independent members with recognized expertise

in relevant engineering and science disciplines, code numerics, and computer

programming. Expert peer review team members, who were not directly involved in the

evaluation model development and assessment, can enhance the robustness of the

evaluation models. Further, they can be of value in identifying deficiencies that are

common to large system analysis codes.

4. Independent Analysis

The reviewers may perform independent analyses in order to determine or confirm

the importance of key phenomena and evaluate the impact of uncertainties in these

phenomena on the key figures of merit in the plant calculation. The reviewers may want

an independent determination of the importance of a high-ranking phenomenon,

or to determine if a phenomenon not ranked as high by the licensee must be so ranked. 

Independent analysis may also be used to confirm uncertainty estimates. The Office

of Nuclear Regulatory Research should be consulted, as needed, to accomplish these audits.
15.0.2-14 Initial Issuance - December 2005

5. Review of General Purpose Computer Programs

Very often a general purpose transient analysis computer program, such as RELAP5,

TRAC, or RETRAN, is developed to analyze a number of different events for a wide variety of

plants. These codes can constitute the major portion of an evaluation model for a particular

plant and event. Generic reviews are often performed for these codes to minimize the

amount of work required for plant- and event-specific reviews. A certain amount of generic

assessment may be performed for such a code as part of the generic code development. 

Applying portions of the review process to a general purpose transient analysis computer

program is required to determine its suitability for use as the basis for an evaluation model.

The review process for the general purpose transient analysis computer program,

may indicate that the code models and generic assessment do not include all the appropriate

geometry, phenomena, or the necessary range of variables to demonstrate code adequacy

for some of the intended plant-specific event analyses. In order to avoid problems

in the review, it is important that the submitted code documentation identify the intended

range of applicability of the generic code, including its models and correlations. 

The “generic” assessment that accompanies the code must support the intended range

of applicability of the code. The reviewer must ensure that the code models and assessment

support the use of the code over its stated range of applicability. One goal of the review

should be that evaluation models that use an approved general purpose transient analysis

computer program can efficiently identify the models and assessment that support

the analysis of the specific plant and accident types that the evaluation model will be used

for in order to streamline future reviews of the evaluation models.

6. Review of Small Changes to Existing Evaluation Models

Application of the full review process described in this SRP section may not be needed for

all evaluation models submitted for review by the staff. Some evaluation models submitted

for review are relatively minor modifications to existing evaluation models. The scope and

depth of applying the review process to the evaluation model can be based on a graded

approach. The following attributes of the evaluation model should be considered when

determining the extent to which the full review process may be reduced for a specific

application.

a. Novelty of the revised evaluation model compared to the currently acceptable model.

b. The complexity of the event being analyzed.

c. The degree of conservatism in the evaluation model.

d. The extent of any plant design or operational changes that would require a

reanalysis.

Each of these attributes is discussed in the following sections.
15.0.2-15 Initial Issuance - December 2005

6.1 Novelty of the evaluation model or changes to the model

The level of effort involved in the review process should be commensurate with the extent

of the changes made to an evaluation model. Small changes to a robust time tested

evaluation model component such as a change to a simple heat transfer or drag correlation

(possibly required by an error correction) may not require a full review of the entire

evaluation model. In this case, scaling would only have to be considered within the context of

how well the new model scales to full plant analysis if the model is developed from a reduced

scale test program. Consideration would also have to be given to how well the assessment

cases for the model represent full scale plant conditions. A small subset of the entire code

assessment matrix may be adequate to test the phenomenon that are affected by the model

changes or the new model. Another subset of the code test cases may need to be

performed to ensure that other parts of the model are not inadvertently impacted by the

changes. The impact of any changes due to an error correction would have to be evaluated

for the current license analysis of record. A large model change may require application of

the review process on a much larger scale. Changing models from an equilibrium, drift flux

model to a two fluid, non-equilibrium model would be an example of a significant change

that would require an extensive review for the new evaluation model.

6.2 Complexity of the event

The level of effort involved in applying the development process should be commensurate

with the complexity of the evaluation model. At first glance the review process may seem

too burdensome to apply to simple events. However, application of the review process

to a simple evaluation models will automatically result in a simplified review process. 

In simple events the number of key physical phenomena should also be small and the code

assessment only needs to cover the important phenomena even though the underlying

general purpose transient analysis computer program may have models that cover a much

wider range of conditions. An example of this is the system evaluation of a PWR pump trip

analysis in which the important phenomena may be limited to a few quantities such as

single-phase liquid wall drag and heat transfer, and pump inertia. In this case, very little

assessment would need to be performed and there may be adequate full scale plant data

for the code assessment so there would be no need for a scaling analysis. The other extreme is

an evaluation model for a large-break loss-of-coolant accident where the physical

phenomena and the mathematical models are complex and cover a wide range of conditions. 

An extensive code development process and assessment and an extensive review process

would be required in this case.
15.0.2-16 Initial Issuance - December 2005

6.3 Degree of conservatism

The review process may be simplified if there is a large degree off conservatism

in the evaluation model. The intended results of an analysis can be conservative due to

a combination of code input and modeling assumptions. The amount of assessment required

for a change to an evaluation model may be reduced significantly if the documented degree of

conservatism is large or if the new model can be shown to give more conservative results

than the previous model. However, conservatism in just one part of the evaluation model

such as a heat transfer correlation can not be used to justify conservatism in the evaluation

model as a whole because other parts of the model may be non-conservative and cause

the overall model to be non-conservative. The degree of conservatism in the overall

evaluation model must be quantified and documented for the particular application in order

to justify a reduction in assessment requirements using this argument.

6.4 Extent of plant or operational changes that require a reanalysis

The level of effort required to apply the process should be commensurate with the extent

of changes made to the plant design or operation. Most of the changes to plant equipment

or operations do not cause the plant to operate outside the range of validity of

the evaluation model. In this case no additional development and assessment needs

to be performed. This may not be the case for all changes. Examples of changes that

may require changes to or additional assessment of the evaluation model are fuel bundle

design changes (including grid spacer and intermediate flow mixer design changes),

increases in the peak linear heat generation rate or operational changes that may cause

reliance on a different safety grade trip which requires that accurate prediction of a quantity

not required in the previous analysis. In these cases a limited application of the review

process similar to that described in section 6.1 should be sufficient.

IV. EVALUATION FINDINGS

The reviewers should document all findings in an evaluation model safety evaluation report

(SER) that either accepts the evaluation model for the intended use or rejects the evaluation

model. Acceptance may be subject to limitations determined during the review. 

If the evaluation model is to be accepted, it must be clearly demonstrated that the model

is getting the right answer for the right reasons. There must be no evidence of compensating

errors or arbitrary code tuning that produces the desired result for a single parameter. 

Any errors found in the documentation or issues related to the technical adequacy

of the model must be addressed through the review process. The review process

and the results of communications with the code submitter must also be documented

as appropriate in the SER to provide a traceable history of the review process. Restrictions

and limitations on use of the evaluation model must be explicitly documented in the SER. 

The SER must document areas that were reviewed, the method of review, and the findings

in each area. The SER must also document areas that were not reviewed and provide

the reasons for their omission.

The reviewers should determine if the evaluation model was previously reviewed and if

the evaluation model previously had restrictions on its use or on the use of certain models

in the codes. These previous restrictions must be explicitly addressed in the SER. 

The current review of the evaluation model may result in the imposition of new restrictions
15.0.2-17 Initial Issuance - December 2005

to be placed on its use. These restrictions must be explicitly identified. The applicant’s

documentation must also be revised to reflect these restrictions on use of the code. 

SER restrictions that have not been explicitly incorporated into evaluation model

documentation have been determined to not be legally enforceable. Only statements

in the approved description of the evaluation model are legally enforceable.

V. IMPLEMENTATION

The following is intended to provide guidance to applicants and licensees regarding

the NRC staff’s plans for using this SRP section.

This SRP section will be used by the staff when performing safety evaluations of transient

and accident analysis methods used in safety analysis for licenses under 10 CFR Parts 50

and 52. Except in those cases in which the applicant proposes an acceptable alternative

method for complying with specified portions of the Commissions’s regulations, the method

described herein will be used by the staff in its evaluation of conformance with Commission

regulations.

The provisions of this SRP section apply to reviews of transient and accident analysis

methods submitted by licensees or vendors six months or more after the date of issuance

of this SRP section.

VI. DEFINITIONS

All definitions are in the context of the objectives of this SRP section and may not be

generic to other uses.

Acceptance Review An initial review of a submittal performed to ensure that sufficient

information is included for the review to conduct the review.

Accidents In this SRP section, accidents and transients refer to events that are

defined in NUREG-0800 (Ref. 1) to be analyzed to meet the requirements of

the General Design Criteria (GDC), except for the fuel assembly

misloading event and all radiological consequence analyses.

Codes Calculational procedures that compose an evaluation model.

Code Options User controlled options that control which procedures, models,

correlations, etc., the code uses when performing calculations.

Code Tuning The adjustment of parameters or options included in the code

to achieve a predetermined result of code run.

Compensating A set of errors that, combined, mask the effect of the individual errors.

Errors

Correlation The change in one parameter as a result of a change in another 

Effort parameter.
15.0.2-18 Initial Issuance - December 2005

Closure Equations and correlations required to close the field equations

Relationships so that they may be solved. They relate unknown quantities to the

variables of the field equations. They may include physical correlations

of transport phenomena such as equations that relate the shearing

stresses to the rate of strain (the velocity field).

Constitutive Equations and correlations required to close the field equations so that

Equations they may be solved. They relate unknown quantities to the variables of

the field equations. They may include physical correlations

of transport phenomena such as equations that relate the shearing

stresses to the rate of strain (the velocity field).

Critical Power Ratio The ratio of assembly power at which critical heat flux occurs to the

actual assembly power.

Code Scaling, A process to determine the applicability, scalability, and uncertainty of 

Applicability, a computer code in simulating an accident or other transient. A PIRT 

and Uncertainty process is normally embedded within a CSAU process. See Reference

1.

Empirical Derived from experimental data without recourse to physical modeling.

Evaluation Model Calculational framework for evaluating the behavior of the reactor

system during a postulated Chapter 15 event, which includes one

or more computer programs and all other information needed for use

in the target application.

Field Equations Equations that are solved to determine the transport of mass, energy,

and momentum throughout the system.

Figures of Merit Quantitative standards of acceptance that are used to define

acceptable answers for a safety analysis (e.g., DNBR limits and fuel

temperature limits).

Frozen The condition whereby the analytical tool(s) and associated facility

input decks remain unchanged (and under configuration control)

throughout a safety analysis thereby ensuring traceability of,

and consistency in, the final results.

General Design Design criteria described in Appendix A to 10 CFR Part 50.

Criteria

Homogeneous An analytical model for two-phase flow that assumes (1) both phases 

Equilibrium Model move at the same velocity, (2) the fluid is in thermal equilibrium, and

(3) the flow is isentropic and steady.

Integral Effects An experiment in which the primary focus is on the interactions 

Test between parameters and processes.
15.0.2-19 Initial Issuance - December 2005

Model (without “evaluation” modifier) - Equation or set of equations that

represents a particular physical phenomenon within a calculational

device.

Peak Cladding The maximum fuel element cladding temperature.

Temperature 

Phase State of matter involved in transport process, usually liquid or gas.

Phenomena May refer to a table, or to a process depending on context of use. The 

Identification process relates to determining the relative importance of phenomena 

and Ranking Table (and/or physical processes) to the behavior of a nuclear power plant

following the initiation of an accident or other transient. A PIRT table is a

listing of the results of application of the process.

Request for A documented request, usually in the form of questions, sent by the 

additional NRC to the submitter to obtain more information on areas under

review.

information 

Roadmap A document used to facilitate navigation through a larger and complex

set of documentation.

Scaling The process in which the results from a subscale facility (relative to

a nuclear power plant) and/or the modeling features of a calculational

device are evaluated to determine the degree to which they represent

a nuclear power plant.

Scaling Distortions Errors introduced into the data as a result of scaling the experimental

facility.

Scenario Time sequence of events.

Sensitivity Studies The term is generic to several types of analyses; however, the definition of

most interest here relates to those studies associated with the PIRT

process and used to determine the relative importance of phenomena

or processes. This may also involve analysis of experimental data

that are a source of information used in the PIRT process.

Separate Effects An experiment in which the primary focus is on a single parameter

Test or process.

Safety Evaluation A report by the NRC that evaluates a submittal and either accepts or 

Report rejects the proposals in the submittal.

Standard Review Acceptable plan for NRC reviewers, as set forth in NUREG-0800.

Plan 
15.0.2-20 Initial Issuance - December 2005

Transients In this SRP, accidents and transients refer to those events that are

defined in NUREG-0800 to be analyzed to meet the requirements

of the General Design Criteria (GDC), except for the fuel assembly

misloading event and all radiological consequence analyses.

Uncertainty There are two separate but related definitions of primary interest:

a. The inaccuracy in experimentally derived data typically

generated by the inaccuracy of measurement systems.

b. The inaccuracy of calculating primary safety criteria or related

figures of merit typically originating in the experimental data

and/or assumptions used to develop the analytical tools. 

The analytical inaccuracies are related to approximations

in solving the equations and constitutive relations.

User Manual A document that includes modeling guidelines for the accident under

consideration, procedures for selecting code inputs, and procedures

for transferring information between different pieces of the evaluation

model.
1

 Copies are available for inspection or copying for a fee from the NRC’s Public Document Room (PDR),

which is located at 11555 Rockville Pike, Rockville, Maryland; the PDR’s mailing address is USNRC PDR,

Washington, DC 20555-0001. The PDR can also be reached by telephone at (301) 415-4737 or (800) 397-4205, by

fax at (301) 415-3548, and by email to PDR@nrc.gov. The SRP is also available through the NRC’s public Web

site at http://www.nrc.gov/reading-rm/doc-collections/nuregs/staff/sr0800/.

2

 Single copies of regulatory guides, both active and draft, and draft NUREG documents may be obtained

free of charge by writing the Reproduction and Distribution Services Section, OCIO, USNRC, Washington, DC

20555-0001, or by fax to (301)415-2289, or by email to DISTRIBUTION@nrc.gov. Copies of certain guides

and many other NRC documents are available electronically through NRC’s public Web site at

http://www.nrc.gov/reading-rm/doc-collections/. Documents are also available through the NRC’s Agencywide

Documents Access and Management System (ADAMS), at http://www.nrc.gov/reading-rm/adams.html.

3

 Electronic copies are available in the NRC’s Agencywide Documents Access and Management System

(ADAMS), at http://www.nrc.gov/reading-rm/adams/web-based.html (search for “LIC-101”).

4

 Copies are available at current rates from the U.S. Government Printing Office, P.O. Box 37082, Washington, DC

20402-9328 [telephone (202) 512-1800], or from the National Technical Information Service (NTIS), 5285 Port

Royal Road, Springfield, Virginia 22161 [http://www.ntis.gov, telephone (703) 487-4650]. Copies are available for

inspection or copying for a fee from the NRC’s Public Document Room (PDR), which is located at 11555 Rockville

Pike, Rockville, Maryland; the PDR’s mailing address is USNRC PDR, Washington, DC 20555-0001. The PDR can

also be reached by telephone at (301) 415-4737 or (800) 397-4205, by fax at (301) 415-3548, and by email

to PDR@nrc.gov.

5 Nuclear Engineering and Design is available for electronic download (by free subscription) through Science

Direct, a service of the Reed Elsevier Group, at

http://www.sciencedirect.com/science?_ob=JournalURL&_cdi=5756&_auth=y&_acct=C000050221&_version=1&

_urlVersion=0&_userid=10&md5=5bc1bb0de40acc1a7b0e4129b21d809f.

6

 Copies are available on the public Web site maintained by the OECD Nuclear Energy Agency,

at http://www.nea.fr/html/nsd/docs/1996/csni-r1996-17.pdf.

15.0.2-21 Initial Issuance - December 2005

VII. REFERENCES

1. NUREG-0800, “Standard Review Plan for the Review of Safety Analysis Reports

for Nuclear Power Plants,” June 1987. (Certain updated sections are available

from the NRC.)1

2. USNRC, “Transient and Accident Analysis Methods,” Regulatory Guide 1.203,

December 2005.2

3. NRC Office Instruction No. LIC-101, “Licence Amendment Review Procedures.”3

4. B.E. Boyak, et al., “Quantifying Reactor Safety Margins: Application of Code Scaling,

Applicability, and Uncertainty Evaluation Methodology to a Large-Break Loss-ofCoolant Accident,” NUREG/CR-5249, USNRC, December 1989.4

5. B.E. Boyack, et al., “Quantifying Reactor Safety Margins,” six papers in Nuclear

Engineering and Design, Vol. 119, No. 1, May 1990.5

6. “Compendium of ECCS Research for Realistic LOCA Analysis,” NUREG-1230,

December 1988.4

7. “Separate Effects Test Matrix for Thermal-Hydraulic Code Validation,” Committee

on the Safety of Nuclear Installations, NEA/CSNI/R(93)14, September 1993.6
7

 Electronic copies are available on the public Web site maintained by the OECD Nuclear Energy Agency,

at http://www.nea.fr/html/nsd/docs/1987/csni87-132.pdf.

15.0.2-22 Initial Issuance - December 2005

8. “Integral Test Facility Validation Matrix for the Assessment of Thermal-Hydraulic

Codes for LWR LOCA and Transients,” Committee on the Safety of Nuclear

Installations, NEA/CSNI/R(96)17, July 1996.

9. “CSNI Code Validation Matrix of Thermo-Hydraulic Codes for LWR LOCA and

Transients,” Committee on the Safety of Nuclear Installations, CSNI Report 132,

March 1987.7

10. USNRC, “Best-Estimate Calculations of Emergency Core Cooling System Performance,”

Regulatory Guide 1.157, May 1989.2

11. W. Wulff, et al., “Uncertainty Analysis of Suppression Pool Heating During

an ATWS in a BWR-5 Plant: An Application of the CSAU Methodology Using the BNL

Engineering Plant Analyzer,” NUREG/CR-6200, USNRC, March 1994.4

Paperwork Reduction Act Statement

The information collections contained in the Standard Review Plan are covered by

the requirements of 10 CFR Part 50, which were approved by the Office of Management

and Budget (OMB), approval number 3150-0011.

Public Protection Notification

The NRC may not conduct or sponsor, and a person is not required to respond to,

a request for information or an information collection requirement, unless the requesting

document displays a currently valid OMB control number.