# Regulatory Scenario for the Acceptance of Uncertainty Analysis

Authors: Maria Regina Galetti
Journal: Science and Technology of Nuclear Installations
 (2008)

---

## Abstract

The task of regulatory body staff reviewing and assessing a realistic large break loss-of-coolant accident evaluation model is discussed, facing the actual regulatory licensing environment related to the acceptance of the analysis of emergency core cooling system performance. Especially, focus is directed to the question of how to fulfill the requirement of quantifying the uncertainty in the calculated results when they are compared to the acceptance criteria for this system. As it is recognized that the regulation governing the loss-of-coolant accident analyses was originally developed by the United States Nuclear Regulatory Commission, a description of its evolution is presented. When using a realistic evaluation model to analyze the loss-of-coolant accident, different approaches have been used in the licensing arena. The Brazilian regulatory body has concluded that, in the current environment, the independent regulatory calculation is recognized as a relevant support for the staff decision within the licensing framework of a realistic analysis.


---

## Body

## 1. Introduction

The objective of this paper is to discuss the regulatory licensing environment
related to the acceptance of the analysis of emergency core cooling system (ECCS)
performance in light water reactors when using a realistic or best-estimate
evaluation model. The focus is directed to the question of how to meet the
requirement of quantifying the uncertainty in the calculated results when they
are compared to the acceptance criteria for this system.It also included the experience of the Brazilian nuclear regulatory body
(CNEN) reviewing and assessing the Angra 2 nuclear power plant (NPP) large-break
loss-of-coolant accident (LB-LOCA) analysis, submitted for licensing with a
realistic evaluation methodology.


## 2. Regulating the Use of Be + U

The United States Nuclear Regulatory Commission (USNRC) emergency core cooling
systems acceptance criteria, issued in 1974 [1], is recognized as a highly
conservative approach due to limitations in knowledge at that time. This
relevant aspect was identified and dealt with by the nuclear community through
a huge effort in the reactor-safety research area. For additional details, see [2–6].In 1983, based on experimental programs results, the ability of advanced
computer codes to predict the behavior during a LOCA was demonstrated, and the
conservatism in Appendix K could be quantitatively estimated. Because of this, through
the release of SECY-83-472 [7], the NRC adopted an interim approach for
evaluation models retaining the features of Appendix K which were recognized as
requirements but allowing the use of best-estimate methods, in that, models and
correlations are stated as acceptable. Even still
conservative, this approach was the first step on licensing decision making
based on realistic calculations.On September 16, 1988, the NRC amended the requirements of § 50.46 to 10 CFR [8] reflecting the improved
understanding of the thermal-hydraulic phenomena occurring during the
loss-of-coolant accidents, obtained by the results of extensive research
programs sponsored by the NRC and the nuclear industry. In Brazil, CNEN adopted
this revision which allows, as an option, the use of realistic evaluation
models to calculate the performance of the emergency core cooling system. In
such cases, the LOCA analysis will fulfill the requirement of identifying and
evaluating the uncertainty in the analysis methods and inputs, and this
uncertainty must be considered when comparing the calculated results with the
acceptance criteria so that there is a high probability that the criteria will
not be exceeded.This revision of 10 CFR
50.46 allows licensees or applicants to use either the conservative evaluation
model defined in Appendix K, with its conservative analysis methods, or a
realistic evaluation model (best-estimate plus uncertainty analysis methods). The
Regulatory Guide 1.157 [9] describes acceptable models, correlations, data,
model evaluation procedures, and methods for meeting the specific requirements
for a realistic calculation of ECCS performance during a LOCA.Despite of that, there is still a lack of an established set of specific
regulatory requirements and guidance applied to the acceptance of the
uncertainty calculation related to the results of a realistic evaluation model
used to analyze the LOCA. On January 11, 2001, the Advisory Committee on
Reactor Safeguard (ACRS) of USNRC addressed the question of how the perceived
weaknesses of the thermal-hydraulic codes may affect the regulatory role, and already
emphasized in a Letter Report [10], “We
perceive a need for the staff to be more specific about what are acceptable
methods of deriving and expressing the uncertainties in codes and how these
methods are to be used in the regulatory context”.More recently, NRC has issued section 15.0.2 of the Standard Review Plan
[11] describing the review process and
acceptance criteria for analytical models and computer codes used to analyze the
accident and transient behavior, including methods to estimate the uncertainty
in best-estimate LOCA calculation. Additionally, guidance to the
industry was issued, set forth in Regulatory Guide 1.203 [12]. Despite of that,
as it has been pointed out by ACRS in its January 11, 2001 Letter Report related
to Regulatory Guide 1.157, these new regulatory guidance documents remain very
qualitative and leaves considerable latitude in interpretation.In parallel, NRC has been conducted research, together with industry,
related to the acceptance criteria for ECCS. As an example, it should be
mentioned that the ongoing development
of a performance-based option for the embrittlement criteria in 50.46(b) [13–15], and also the proposed rule for a voluntary alternative to
10 CFR 50.46, related to the definition of LOCA break sizes [16].In the United States,
the first NRC approved best-estimate LOCA methodology was the Westinghouse methodology
[17], patterned after the Code Scaling, Applicability, and Uncertainty evaluation
methodology (CSAU), and uses response surfaces to estimate PCT uncertainty
distribution with the 95th percentile PCT determined from a Monte Carlo sampling and accepted as the licensing basis
PCT. In 1999, it was extended to other plants design (AP600 and 2-loops plants
with upper plenum injection). By 2000, 14 plants in the United States had
Westinghouse BELOCA methodology as a licensing basis and it was also used for
Ringhals unit 2 in Sweden [18].Framatome ANP has submitted its realistic LB-LOCA methodology and got NRC
approval in 2003 [19]. It follows CSAU approach but was the first to use a nonparametric
order statistic method, eliminating the need for response surfaces. By 2006,
there were seven completed realistic LB-LOCA analyses with 3-loop and 4-loop
Westinghouse and Combustion Engineering pressurized water reactors [20].By 2004, Westinghouse updated its methodology to use nonparametric order
statistic, and an advanced statistical treatment of uncertainty method (ASTRUM)
[21] was approved for licensing. In US, by 2006, 24 plants are licensed or
analyzed with Westinghouse 1996 and 1999 BELOCA methodologies and 10 plants are
analyzed or being analyzed with ASTRUM [18].It is worthwhile to mention the ongoing issue at the regulatory arena
with the use of nonparametric order statistics methodology to demonstrate
that the criteria in 10 CFR 50.46(b) are satisfied. The number of ECCS
performance-evaluation model runs accepted to demonstrate a probability that the criteria will not be exceeded is
different in two similar realistic LB-LOCA methodologies approved by the NRC
[19, 21]. Such difference is due to the approach used to demonstrate the simultaneous satisfaction of the first three criteria in
10 CFR 50.46(b), peak cladding temperature, peak local oxidation, and corewide oxidation. There are
still undergoing discussions on this philosophical issue [22–26].In Germany, the use of best-estimate codes is allowed, in combination with conservative initial and boundary
conditions, and efforts are being conducted to include uncertainty evaluation
in the regulation with a revision in the German nuclear regulation. There is
also a recommendation of the Reactor Safety Commission to perform LOCA
licensing analysis [27].In Canada, the Canadian Nuclear
Safety Commission recently conducted a research program that resulted in a
guide for safety assessment and applications of best-estimate analysis and uncertainty
methodology [28].


## 3. Brazilian Regulatory Experience

Angra 2 NPP is a 4-loop Siemens design 1300 MWe pressurized water reactor
that started commercial operation in 2001. The best-estimate LOCA approach was formally adopted by the
utility Eletronuclear (ETN) in 1994. By 1998, when the realistic LB-LOCA
analysis was submitted, based on CSAU methodology to evaluate the uncertainty,
there were only few applications of realistic evaluation models in the
licensing arena.Aiming at performing a consistent safety review and assessment of this
analysis, the Brazilian regulatory body trained its staff and relied upon two
international consultants, the German institute GRS (Gesellschaft fur Anlagen
und Reaktorsicherheit) and the University of Pisa.The cooperation with many international institutions involved in accident-analysis
research provided a relevant technical background for the regulatory staff. In
the same time, the national thermal-hydraulic journey (JONATER), coordinated by
CNEN, has promoted the integration of seven institutions (regulatory body, research
institutes, and utility) of the Brazilian nuclear sector. One result of JONATER
was the use of an uncertainty methodology applied as an exercise for Angra 1
NPP, a Westinghouse 630 MWe 2-loop pressurized water reactor. The uncertainty
bands were estimated with the UMAE [29] method for the results of the small-break
LOCA simulated with the Relap5Mod2 code, as it is shown in Figure 1 [30].Figure 1
JONATER application of UMAE to angra 1 NPP SB-LOCA: uncertainty bands for the PCT.UMAE is an uncertainty methodology based on accuracy extrapolation
resulting from a   comparison between code results and relevant experimental data
obtained in experimental facilities. These facilities were simulated, for the
chosen transient scenario, with an established nodalization that will be the
basis for the nodalization adopted in the plant calculation. The extrapolated accuracy is
superimposed directly to the results of the plant calculation. Uncertainty
bands are constituted by a set of “punctual” error bands in thex-y plane
(where x is the time t and y is YC quantity). Each value YC at a time t can be characterized by an error DYC in the “y”
direction and by an error Dt in the “x” direction. The total uncertainty is the
superimposition of these two errors.As the estimation of Angra 1 small-break LOCA uncertainty bands was an
exercise for the application of an uncertainty methodology, for the accuracy
calculation, only the large scale test facility (LSTF) database was considered
(experimental and Relap5/Mod2 results for the SB-CL-21 test). It is important
to mention that the accuracy should be obtained from more tests to avoid some
poor accuracy that eventually can result for some specific parameter. For
instance, code simulation of the LSTF experiment yielded a result for the heater
rod temperature and time of its occurrence far from the verified experimental
value. Therefore, the lower uncertainty band at the end of the transient for
the peak cladding temperature shows no physical results due to the limited
number of experimental data used.The Angra 2 LB-LOCA analysis presented in the final safety-analysis report
was reviewed by CNEN staff taking into account the two independent reviews
performed by the international consultants. As a result, a preliminary safety-evaluation
report (SER) requested additional information (RAI), with a total of 27 questions
to the applicant, each one is classified according to their significance
to safety [31].Table1 lists the main steps in the review and assessment process of
Angra 2 NPP LB-LOCA analysis.Table 1
Angra 2 NPP LB-LOCA Review.
ActivityDateSubmission of FSAR with the LOCA
analysis1998CNEN’s Preliminary Safety Assessment1998GRS Expert Mission-FSAR LBLOCA Analysis and
Uncertainty Method revision1998IAEA Expert Mission/Pisa Uni-FSAR
LBLOCA Analysis and Uncertainty Method revision1999Issuance of CNEN Preliminary SER
with 27 RAI1999GRS Expert Follow-up Mission1999Answers to the request of
additional information1999Licensing meeting: Utility, Siemens, CNEN, GRS and Pisa University1999Submission of Generic Thermal
Mechanics Analysis of Fuel Failures after a LOCA1999Submission of Angra 2 Specific
Thermal Mechanics Analysis of Fuel Failures after a LOCA1999Emission of CNEN SER about ECCS
Technical Specification2000Pisa University Technical Consultancy to CNEN-Regulatory audit analysis2001Issuance of CNEN SER: additional RAI on core
nodalization and uncertainty quantification2001Independent
LB-LOCA calculation used to check the request of a temporary 6% power uprate2002The Siemens uncertainty methodology applied to Angra 2 followed,
essentially, the CSAU approach (Phenomena Identification Ranking Table, code
capabilities for accident scenario) and used Monte Carlo calculations with
response surface. The treatment of the uncertainties is performed separately
from three basic categories: code uncertainties (statistical quantification of
difference between calculated and measured PCT), plant parameters uncertainties
(statistical variations), and fuel parameters uncertainties (statistical
variations). Some additional parameters related to uncertainties have
been required to be run at combined worst-case conditions. These parameters are
break area and location, axial core power distribution, worst-case single
failure and repair assumption, loss of offsite power, and reactor kinetics.This uncertainty analysis is such that the 95% probability PCT was
generated by using Monte Carlo
to combine uncertainties from the three sources. The two other criteria
(maximum cladding oxidation and hydrogen generation) were calculated
considering conservative assumptions.The number of data points, used to determine code accuracy through the
quantification of the differences between calculated and measured results for LOFT and CCTF experiments,
was one example of RAI from the preliminary SER. It was further required from the applicant to verify the implications of
considering additional relevant experimental data into code integral
uncertainties. Additionally, the applicant presented code uncertainty
quantification with more experimental data.After the issuance of the preliminary SER, the importance of an independent
regulatory calculation was recognized. Together with CNEN staff, the University
of Pisa performed independent calculation [32, 33]. Based on its conclusions,
three requests for additional information were issued to the applicant, mainly
related to plant modelling, which has to be consistent with those used for the
validation calculations.As future applications, the Brazilian regulatory body has already been
informed by the utility ETN of its intention to uprate 6% the Angra 2 power
together with a change in the fuel design, replacing it to a high thermal performance fuel with M5
fuel cladding. This will require the reanalysis of the LB-LOCA with uncertainty
quantification.Furthermore, for Angra 1 NPP steam-generators replacement, the utility
will submit a realistic evaluation model for the LB-LOCA, using the
Westinghouse methodology that encompasses the WCOBRA/TRAC code with the ASTRUM
methodology for uncertainty calculation. Additionally, the power will be
uprated 5% and a new fuel design will be used (16 next-generation fuel,
developed jointly by Westinghouse, Korea Nuclear Fuel (KNFC), and Indústrias
Nucleares do Brasil (INB)).


## 4. Regulatory Independent Angra 2 LB-LOCA Analysis Description

The independent calculation included the LB-LOCA calculation with
Relap5/Mod3.2.2 Gamma code and the uncertainty evaluation with the CIAU method
(code with capability of internal assessment of uncertainty) [34].In this application, the CIAU method used UMAE methodology for
uncertainty quantification that is based upon propagation of code output error
and does not rely on statistics. The inaccuracies are obtained by experimental calculation
comparison and are extrapolated to get uncertainty. The database for accuracy
extrapolation was derived from 32 experimental transients that were calculated
by Pisa University with Relap5/Mod3.2.2 Gamma code.The independent LB-LOCA calculation activities were planned with the
objective to consider the steps presented in a best-estimate analysis: a
qualified nodalization development (steady-state level and on-transient level),
transient reference-case calculation, uncertainty evaluation, and comparison
between the results obtained in the sensitivity studies and in the uncertainty
analysis.A “fictitious” 3D nodalization of the reactor pressure vessel
was adopted considering the experience in the analysis of the upper plenum test
facility experiments [35]. Two main nodalizations were established at the
beginning of the studies, characterized by:(i)
nonuniform upper
plenum behavior, pursuing the nodalization strategy of the utility ETN in the
FSAR analysis, top-down flow allowed only in the determined breakthrough
channels [36];(ii)
uniform upper
plenum behavior with top-down flow allowed in all channels except in the hot
assembly, with the worst conditions for core cooling inside the hydraulic hot assembly,
by “hydraulically separating” the hot fuel assembly from the average core
region.After defining a reference calculation and performing the sensitivity
study, the reference-case nodalization chosen was the one without cross-flow
simulation between the hot fuel assembly and the rest of the core (denominated
tr12), that might bring undue conservatism in the results. The one considering
this cross flow (denominated a2n04x) could be the reference case if
experimental data was available to establish the flow energy-loss coefficients.
Therefore, for the a2n04x run, these coefficients were established through
engineering judgment without an experimental basis. The use of S-RELAP5 code in
the Angra 2 FSAR LB-LOCA analysis considers implicit this cross flow through
the full two-dimensional treatment added to the hydrodynamic field equations.Figure2 shows a comparison of the reference calculation result to FSAR
result for the peak cladding temperature (PCT) for the “base case”. In the FSAR
analysis, this “base case” is defined in the adopted ETN methodology as the
nominal condition for the uncertainty analysis. This uncertainty analysis is
such that the 95% probability PCT was generated by using Monte Carlo
to combine uncertainties from the
three sources. The “base case” is the reference case for the determination of
the calculation-design matrix used to generate data for fitting the response
surfaces. Also, the “base case” is the reference case where the effects of the plant uncertainties are
determined.Figure 2
Cladding temperature of the hot rod.The comparison of the PCT from the “base case” and the “reference calculation”
indicates a discrepancy, with a higher value observed in independent
calculation result. In the case of “reference calculation”, it is shown that the
removal of conservatism of assuming no cross flow to the hot channel
substantially lowers the reported value. This outcome confirms the importance
of assessing, by using experimental data, the cross flow to the hot channel if
this is considered.In the independent regulatory calculation, automatic uncertainty bands
for primary-system pressure, mass inventory, and rod surface temperature at 2/3
of the core active height are generated by the CIAU method and constitute the
results of the application. Figure3 shows the result for PCT.Figure 3
Uncertainty bands for rod surface temperature at 2/3 of the core height-CIAU result.The number of experiments, which were used to derive code uncertainty
from CIAU, is limited. Therefore, a sensitivity study has been performed to
confirm the results obtained from this methodology. Additional objective was to
confirm that the impact of an assigned input parameter upon the results is
dependent on the nodalization.A
comprehensive-sensitivity study has been carried out including two series of
calculations. Starting from the two main nodalizations, single parameters are
varied in each code run. Six groups of input parameters are distinguished:
“fuel”, “nodalization”, “loop hydraulics”,
“PSA and ECCS”, “neutronics”, and “others”. The
number of performed runs was 112.Thefirstseries
aims at confirming the influence of selected input parameters upon the LB-LOCA
predicted scenario, and showing the importance of nodalization upon the same
prediction when an assigned input parameter is varied. Code runs with single
change of input parameters and with realistic variation ranges were used for
the envelope uncertainty evaluation. Examples of input parameters varied, at
one time, in the code run: fuel (gap thickness, UO2 conductivity, gap
conductance), loop hydraulics (critical flow model, spacer grid modelling,
reactor pressure-vessel bypass flow), nodalization (upper-plenum pressure drop,
counter current-flow limitation in the core), PSA and ECCS (loss of offsite
power delay, components actuation), and neutronics (moderator coefficient,
decay power). The result is shown in Figure4 where the envelope of all the
considered calculations is reported.Figure 4
Angra 2 NPP LBLOCA sensitivity study: upper and lower bounds from the rod surface temperature
Envelope uncertainty evaluation.The second series aims at
determining boundary values for PCT. Three input parameters, chosen among those
considered in the first series of calculations, are selected and varied
simultaneously in each run. Examples of chosen parameters are UO2 conductivity,
break-discharge coefficient, ECCS components actuation, decay power, and gap
conductance. The ranges of variations are maximized. These code runs are
adopted for the deterministic evaluation of the uncertainty (see Figure5).Figure 5
Angra-2 NPP LBLOCA sensitivity study, achievement of a deterministic value for ΔPCT
Labels XXX through VVV representing code runs based on combination of three variations of input parameters.The parameter ΔPCT is defined as the difference between the PCT
of the reference calculation and the PCT obtained from the generic sensitivity
run. The dispersion of results for ΔPCT obtained from the first series of code
runs provides an overall picture of the influence of nodalization upon
predictions, confirming the importance of the nodalization upon the predicted
scenario.The following valuable results were obtained.(i)
The upper and lower uncertainty bands from the envelope uncertainty evaluation
in Figure4 can be compared with the CIAU uncertainty bands in Figure 3.
Therefore, the uncertainty results obtained by CIAU are supported by the
outcome of the sensitivity study.(ii)
The
uncertainty ranges predicted by CIAU, resulting from the sensitivity study and
the ones reported in the FSAR, are comparable.The adopted noding scheme, that is, the nodalization, has been found as
the critical issue of the study. The nodalization features affect the
prediction of the safety relevant parameters, the interpretation of the
performed “sensitivity” runs, and the use of the outcomes from the uncertainty
method. Namely, the application of a 1D designed assessed code, having at the
basis a fictitious 3D model of the vessel, requires a number of engineering
choices. These choices have been proven to impact noticeably the results, and must
be adequately supported by a suitable experimental evidence.Results from a best-estimate code prediction are largely affected by the
nodalization features. Therefore, the full demonstration of the nodalization
quality at the “steady state” and at the “on-transient” level is needed to
derive meaningful conclusions about the safety performance of the concerned
NPP. Considering Angra 2 features, basically, the hot leg injection, a decisive
importance is revealed by the upper plenum and core outlet modeling.


## 5. Conclusions

As described in the previous sections, when using a realistic evaluation
model to analyze the LOCA, different approaches have been used in the licensing
arena to demonstrate the fulfillment of the ECCS acceptance criteria.Besides the different approaches, the regulators are aware of the
development in the uncertainty methodologies and, therefore, further actions should
be required even after a methodology has been accepted.The Brazilian regulatory body is monitoring these activities and it has
concluded that, in the current environment, the
independent regulatory
calculation is recognized once again as a relevant support for the
staff decision within the licensing framework of a realistic LB-LOCA analysis.In the case of Angra 2 LB-LOCA, the independent calculation complemented,
on a quantitative basis, the task of reviewing and assessing, and allowed to check
the completeness  and consistency of the submitted accident analysis. The use of
an uncertainty methodology (CIAU) that has a different approach compared to the designer approach (Siemens)
contributed to the understanding of the validity limits of the results
submitted by the licensee within the FSAR. Conclusions are provided in relation
to the acceptability of the actual safety margins of the Angra 2 NPP.In the case of Angra 1 LB-LOCA reanalysis for the steam-generators
replacement, to be submitted with Westinghouse methodology, the ASTRUM methodology
uses a nonparametric order-statistics
methodology to demonstrate that the criteria in 10 CFR 50.46(b) are satisfied.The different approaches
observed in the nuclear-power plants in Brazil increase the staff effort to deal
with the licensing process. For a small size regulatory body, this diversity of
methods, to demonstrate the fulfillment of the ECCS acceptance criteria, indicates a challenge to be faced with
technical support organizations providing worldwide recognized experts in the
use of best-estimate tools to contribute in the review and assessment process.



---

*Source: 312154-2007-12-03.xml*