NUREG/IA-0220 

 International 

 Agreement Report

 

Quantitative Code Assessment with 

Fast Fourier Transform Based Method 

Improved by Signal Mirroring 

Prepared by: 

A. Prošek, B. Mavko 

Jožef Stefan Institute 

Jamova cesta 39 

SI-1000 Ljubljana, Slovenia 

A. Calvo, NRC Project Manager 

Office of Nuclear Regulatory Research 

U.S. Nuclear Regulatory Commission 

Washington, DC 20555-0001 

December 2009 

 

Prepared as part of 

The Agreement on Research Participation and Technical Exchange 

Under the Thermal-Hydraulic Code Applications and Maintenance Program (CAMP) 

Published by 

U.S. Nuclear Regulatory Commission
AVAILABILITY OF REFERENCE MATERIALS 

IN NRC PUBLICATIONS

NRC Reference Material 

As of November 1999, you may electronically access 

NUREG-series publications and other NRC records at 

NRC’s Public Electronic Reading Room at 

http://www.nrc.gov/reading-rm.html. Publicly released 

records include, to name a few, NUREG-series 

publications; Federal Register notices; applicant, 

licensee, and vendor documents and correspondence; 

NRC correspondence and internal memoranda; 

bulletins and information notices; inspection and 

investigative reports; licensee event reports; and 

Commission papers and their attachments. 

NRC publications in the NUREG series, NRC 

regulations, and Title 10, Energy, in the Code of 

Federal Regulations may also be purchased from one 

of these two sources. 

1. The Superintendent of Documents 

 U.S. Government Printing Office 

 Mail Stop SSOP 

 Washington, DC 20402-0001 

 Internet: bookstore.gpo.gov 

 Telephone: 202-512-1800 

 Fax: 202-512-2250 

2. The National Technical Information Service 

 Springfield, VA 22161-0002 

 www.ntis.gov 

 1-800-553-6847 or, locally, 703-605-6000 

A single copy of each NRC draft report for comment is 

available free, to the extent of supply, upon written 

request as follows: 

Address: Office of the Chief Information Officer, 

 Reproduction and Distribution 

 Services Section 

 U.S. Nuclear Regulatory Commission 

 Washington, DC 20555-0001 

E-mail: DISTRIBUTION@nrc.gov 

Facsimile: 301-415-2289 

Some publications in the NUREG series that are 

posted at NRC’s Web site address 

http://www.nrc.gov/reading-rm/doc-collections/nuregs 

are updated periodically and may differ from the last 

printed version. Although references to material found 

on a Web site bear the date the material was accessed, 

the material available on the date cited may 

subsequently be removed from the site. 

Non-NRC Reference Material

Documents available from public and special technical 

libraries include all open literature items, such as 

books, journal articles, and transactions, Federal 

Register notices, Federal and State legislation, and 

congressional reports. Such documents as theses, 

dissertations, foreign reports and translations, and 

non-NRC conference proceedings may be purchased 

from their sponsoring organization. 

Copies of industry codes and standards used in a 

substantive manner in the NRC regulatory process are 

maintained at— 

The NRC Technical Library 

Two White Flint North 

11545 Rockville Pike 

Rockville, MD 20852-2738 

These standards are available in the library for 

reference use by the public. Codes and standards are 

usually copyrighted and may be purchased from the 

originating organization or, if they are American 

National Standards, from— 

American National Standards Institute 

11 West 42nd Street 

New York, NY 10036-8002 

www.ansi.org 

212-642-4900 

Legally binding regulatory requirements are stated only 

in laws; NRC regulations; licenses, including technical 

specifications; or orders, not in 

NUREG-series publications. The views expressed in 

contractor-prepared publications in this series are not 

necessarily those of the NRC. 

The NUREG series comprises (1) technical and 

administrative reports and books prepared by the staff 

(NUREG–XXXX) or agency contractors 

(NUREG/CR–XXXX), (2) proceedings of conferences 

(NUREG/CP–XXXX), (3) reports resulting from 

international agreements (NUREG/IA–XXXX), 

(4) brochures (NUREG/BR–XXXX), and 

(5) compilations of legal decisions and orders of the 

Commission and Atomic and Safety Licensing Boards 

and of Directors’ decisions under Section 2.206 of 

NRC’s regulations (NUREG–0750). 

DISCLAIMER: This report was prepared as an account of work sponsored by an agency of the U.S. Government. 

 Neither the U.S. Government nor any agency thereof, nor any employee, makes any warranty, expressed or 

implied, or assumes any legal liability or responsibility for any third party=s use, or the results of such use, of any 

information, apparatus, product, or process disclosed in this publication, or represents that its use by such third 

party would not infringe privately owned rights. 
NUREG/IA-0220

Quantitative Code Assessment with Fast 

Fourier Transform Based Method 

Improved by Signal Mirroring 

Manuscript Completed: November 2007 

Date Published: November 2009 

Prepared by: Andrej Prošek, Borut Mavko 

Jožef Stefan Institute 

Jamova cesta 39 

SI-1000 Ljubljana, Slovenia 

A. Calvo, NRC Project Manager 

Prepared as part of 

The Agreement on Research Participation and Technical Exchange 

Under the Thermal-Hydraulic Code Applications and Maintenance Program (CAMP)

Office 

U.S. Nuclear Regulatory Commission 

Washington, DC 20555-0001

iii 

ABSTRACT 

In recent years, the number of quantitative comparisons between experimental data and 

calculated data in the area of nuclear technology has increased. The fast Fourier transform 

based method (FFTBM) is the tool most widely used to quantify the accuracy of thermalhydraulic code calculations, which are continuing to improve. However, in preliminary 

applications of the original FFTBM to a severe accident test, the need for further optimization 

became evident. Analysts observed that FFTBM favors certain trends (e.g., monotonic 

increasing function). Thus, the purpose of the present study was to improve this aspect of 

FFTBM. Improvement was achieved by signal mirroring. Among other improvements, an index 

for the detection of the time shift between the compared signals was proposed. For the 

demonstration of improved FFTBM by signal mirroring, the analysis used the Loss of Fluid Test 

(LOFT) L2-5 test (large-break loss-of-coolant accident). Both qualitative and quantitative 

analyses were performed, and the conclusions about the quality of reference calculations were 

compared with those of the Best-Estimate Methods Uncertainty and Sensitivity Evaluation 

(BEMUSE). The results show that, with improved FFTBM, the analyst can achieve a true picture 

of the contribution of each discrepancy to the accuracy as judged by FFTBM. Analysis that 

considers the improved FFTBM tool as a “black box” obtains optimal information, which greatly 

aids the analyst in making a final judgment about the calculation. “Black box” here is meant that 

no information of the curve shape is input by the user to the FFTBM tool. The easy-to-use 

method is best suited to the automated comparison of several calculations of the same test. 

v 

CONTENTS

Page

ABSTRACT .................................................................................................................................iii

ACKNOWLEDGMENTS............................................................................................................viii

ABBREVIATIONS .......................................................................................................................ix

1. INTRODUCTION.....................................................................................................................1

2. COMPLEX REPRESENTATION OF SINUSOIDS..................................................................3

3. FOURIER TRANSFORM OF DISCRETELY SAMPLED DATA.............................................5

3.1 Discrete Fourier Transform ...............................................................................................6

3.2 Fast Fourier Transform for Computing Discrete Fourier Transform ..................................7

4. DESCRIPTION OF ORIGINAL FAST FOURIER TRANSFORM BASED METHOD..............9

4.1 Input Parameters for Fast Fourier Transform....................................................................9

4.2 Average Amplitude and Weighted Frequency.................................................................10

4.3 Accuracy of Code Calculation .........................................................................................10

4.4 Methodology for Quantifying Code Accuracy ..................................................................12

5. DESCRIPTION OF EXTENDED ORIGINAL FAST FOURIER TRANSFORM BASED 

METHOD...............................................................................................................................13

5.1 New Proposed Accuracy Measures ................................................................................13

5.2 Time-Dependent Accuracy Measures .............................................................................13

6. DESCRIPTION OF FAST FOURIER TRANSFORM BASED METHOD IMPROVED BY 

SIGNAL MIRRORING...........................................................................................................15

6.1 Signal Mirroring ...............................................................................................................15

6.2 Calculation of Average Amplitude by Signal Mirroring ....................................................16

7. DEMONSTRATION APPLICATION OF THE FAST FOURIER TRANSFORM BASED 

METHOD IMPROVED BY SIGNAL MIRRORING ................................................................19

7.1 Facility and Test Description ...........................................................................................19

7.2 Qualitative Assessment of the Calculations ....................................................................19

7.3 Quantitative Assessment of the Calculations ..................................................................21

7.4 Criterion for Using Moving Average ................................................................................54

7.5 Comparison of Fast Fourier Transform Based Method and Automated Code Assessment 

Program...........................................................................................................................56

7.6 Correction Factors...........................................................................................................57

7.7 Discussion of the Results................................................................................................58

8. CONCLUSIONS....................................................................................................................61

9. REFERENCES......................................................................................................................63
vi 

Appendices 

Page

A. Accuracy Measures for L2-5 Test in Time Interval 0–100 seconds..................................... A-1 

Figures

1. Representation of complex numbers........................................................................................3

2. Different signals composed from the original LOFT L2-5 test pressurizer pressure signal....15

3. Intact loop pressure in hot leg (P1) in two different time intervals..........................................28

4. Intact loop pressure in hot leg (P1)—(a) time trends, (b) AA, (c) AAm....................................29

5. SG pressure, secondary side (P2)—(a) time trends, (b) AA, (c) AAm ....................................30

6. Accumulator A pressure (P3)—(a) time trends, (b) AA, (c) AAm ............................................31

7. Pressurizer pressure (P4)—(a) time trends, (b) AA, (c) AAm..................................................32

8. Core inlet fluid temperature (P5)—(a) time trends, (b) AA, (c) AAm .......................................33

9. Core outlet fluid temperature (P6)—(a) time trends, (b) AA, (c) AAm .....................................34

10. Upper head fluid temperature (P7)—(a) time trends, (b) AA, (c) AAm...................................35

11. SG DC bottom fluid temperature (P8)—(a) time trends, (b) AA, (c) AAm ..............................36

12. Break flow rate in cold leg (P9)—(a) time trends, (b) AA, (c) AAm ........................................37

13. Integrated break flow rate in cold leg (P9int)—(a) time trends, (b) AA, (c) AAm....................38

14. Break flow rate in hot leg (P10)—(a) time trends, (b) AA, (c) AAm ........................................39

15. Integrated break flow rate in hot leg (P10int)—(a) time trends, (b) AA, (c) AAm ...................40

16. ECCS integral break flow rate (P11)—(a) time trends, (b) AA, (c) AAm ................................41

17. Primary side total mass (P12)—(a) time trends, (b) AA, (c) AAm ..........................................42

18. SG pressure drop—UT PS (P13)—(a) time trends, (b) AA, (c) AAm .....................................43

19. Primary pump pressure drop (P14)—(a) time trends, (b) AA, (c) AAm ..................................44

20. Rod surface temperature in hot rod in hot channel (zone 4) at 2/3 core...............................45

21. Rod surface temperature in average rod in average channel (zone 2) at bottom level (P18)—

(a) time trends, (b) AA, (c) AAm...................................................................................................46

22. Rod surface temperature in average rod in average channel (zone 2) at 2/3 core height 

(P19)—(a) time trends, (b) AA, (c) AAm ......................................................................................47

23. Rod surface temperature in average rod in average channel (zone 2) at top level (P20)—(a) 

time trends, (b) AA, (c) AAm ........................................................................................................48

24. Total accuracy trend obtained with original FFTBM..............................................................49

25. Total accuracy trend obtained with FFTBM improved by signal mirroring ............................49

26. Time shift of break flow rates ................................................................................................53

27. Accuracy trends for SG pressure drop—UT PS using moving average for the experimental 

signal...........................................................................................................................................55

28. Time trends for SG pressure drop—UT PS for initial 5 seconds...........................................55
vii 

Tables

Page

1. Weighting Factor Components for the Analyzed Quantities (Ref. 5)......................................12

2. The Sum of Amplitudes of Signals Shown in Figure 2 ...........................................................16

3. Participants Performing Calculations .....................................................................................22

4. Variables Used in the BEMUSE Qualitative Analysis.............................................................23

5. Additional Derived Variables Available for Quantitative Assessment.....................................23

6. Masses for Mass Balance Verification ...................................................................................26

7. New Accuracy Measures Obtained with Extended Original FFTBM ......................................51

8. New Accuracy Measures Obtained with FFTBM Improved....................................................51

9. Accuracy Contribution from Non-Rod Surface Temperature Variables and Contribution from 

Rod Surface Temperature Variables ..........................................................................................53

10. Average SG Pressure Drops in Time Interval 15–120 Seconds ...........................................54

11. Comparison of FFTBM and ACAP Figures of Merit for Pressurizer Pressure (P4) in Time 

Interval 0–119.5 Seconds ...........................................................................................................56

12. Comparison of FFTBM and ACAP Figures of Merit for Rod Surface Temperature in the Hot 

Rod in the Middle of the Core (P16) in Time Interval 0–119.5 Seconds.....................................57

13. Correction Factors for Accuracy Judgment by Original FFTBM............................................58
viii 

ACKNOWLEDGMENTS 

The authors acknowledge the support of the Ministry of Higher Education, Science and 

Technology of the Republic of Slovenia within the program P2-0026 and the research 

projects J2-6542 and J2-6565. We especially thank our colleague Dr. Matjaž Leskovar for his 

fruitful ideas and suggestions regarding development of the fast Fourier transform based 

method (FFTBM) improved by signal mirroring. Finally, we thank the University of Pisa for 

providing the data and inviting us to perform this analysis. 
ix 

ABBREVIATIONS 

AA average amplitude 

ACAP Automated Code Assessment Program 

ATHLET Analyses of Thermohydraulics in Leaks and Transients 

BEMUSE Best-Estimate Methods Uncertainty and Sensitivity Evaluation 

CATHARE Code for Analysis of THermalhydraulics during an Accident of Reactor and safety 

 Evaluation 

CEA Commissariat à l’Energie Atomique 

CSNI Committee on the Safety of Nuclear Installations 

CWT continuous wavelet transform 

DC downcomer 

DFFT D’Auria fast Fourier transform 

DFT discrete Fourier transform 

DP differential pressure 

ECCS emergency core cooling system 

EDO Experimental Design Office 

FFT fast Fourier transform 

FFTBM fast Fourier transform based method 

GID EDO “Gidropress” 

GRS Gesellschaft für Anlagen- und Reaktor Sicherheit (GRS) mbH 

HPIS high-pressure injection system 

I time shift indicator 

IAEA International Atomic Energy Agency 

IRSN Institut de Radioprotection et de Sûreté Nucléaire 

ISP International Standard Problem 

JNES Japan Nuclear Energy Safety 

JSI Jožef Stefan Institute 

K acceptability factor 

K kelvin 

KAERI Korea Atomic Energy Research Institute 

KFKI Központi Fizikai Kutató Intézet 

kg kilogram 
x 

KINS Korea Institute of Nuclear Safety 

kPa kilopascal 

LOCA loss-of-coolant accident 

LOFT loss-of-fluid test 

LPIS low-pressure injection system 

MARS Multi-dimensional Analysis of Reactor Safety 

MPa megapascal 

MSE mean square error 

MWt megawatt thermal 

N number of points 

ND number of discrepancies 

NRI Nuclear Research Institute 

PCT peak cladding temperature 

PS primary system 

PSI Paul Scherrer Institute 

PWR pressurized-water reactor 

RELAP Reactor Excursion and Leak Analysis Program 

RTA relevant thermal-hydraulic aspect 

s second 

SG steam generator 

SPE standard problem exercise 

TAEK Türkiye Atom Enerjisi Kurumu 

TECH EDO “Gidropress” code used for large break loss-of-coolant accidents 

TRACE TRAC/RELAP Advanced Computational Engine 

TRAC-P Transient Reactor Analysis Code - Pressurized 

UPC Universitat Politècnica de Catalunya 

UPI University of Pisa 

UT U-tube 

VA variable accuracy 

WF weighted frequency 

XCC cross-correlation coefficient 

Subscripts 

c critical 
xi 

cal calculated 

cut cut-off 

exp experimental 

fix fixed 

i ith variable 

m mirroring 

ma moving average 

max maximal, maximum 

min minimal 

mir mirrored 

norm normalization 

pr primary pressure 

s sampling 

saf safety 

tot total 

var variable 

Superscripts 

M magnitude information of the signal 

1 

1. INTRODUCTION

For years, the fast Fourier transform based method (FFTBM) has been used to quantify the 

accuracy of code calculations. The FFTBM shows the discrepancies between measurements 

and predictions (accuracy quantification) in the frequency domain. It assists in determining how 

to conduct objective comparison and identifying improvements needed in the input model. 

Recently, an initiative to reduce the uncertainties in severe accident safety issues was started. 

Generally, the error made in predicting plant behavior is called “uncertainty,” while the 

discrepancies between measured and calculated trends related to experimental facilities are 

termed “the accuracy of the prediction.” When FFTBM was applied to the severe accident 

International Standard Problem (ISP)-46 (Phebus FPT1) (Ref. 1), the need to optimize FFTBM 

was identified. Specifically, analysts observed that when calculating the accuracy trend, the 

accuracy changes greatly when the experimental signal sharply increases or decreases (e.g., 

triangular shape of cladding temperature). When the signal starts to return to its previous value, 

the accuracy also follows this trend. It is not logical that the still-present discrepancy decreases 

the accuracy instead of increasing it. This problem was not evident when applying FFTBM to a 

few time windows and/or time intervals. It was recognized only after the development of FFTBM 

with the capability to calculate time-dependent accuracy (Ref. 2). Also, analysts found that for 

monotonically increasing or decreasing functions, the original FFTBM is normally highly 

accurate. The purpose of this study was, therefore, to improve this aspect of the FFTBM, which 

is caused by the edge effect. Namely, if the values of the first and last data point differ, then a 

step function is present in the periodically extended time signal. This step function creates 

several harmonic components in the frequency domain, thus increasing the sum of the 

amplitudes. The problem of the edge effect was resolved by signal mirroring. 

Section 2 presents the complex representation of the sinusoids. The fast Fourier transform 

(FFT) is based on complex numbers. As the data are always discrete, the Fourier transform of 

discretely sampled data is described in Section 3. Readers familiar with Fourier transform can 

skip Sections 2 and 3, which are important to understanding how FFTBM works. Section 4 

describes the original FFTBM, which, as its name suggests, was based on FFT. The Jožef 

Stefan Institute (JSI) has used the FFTBM since 1994 and was also involved in its testing. JSI 

performed some later extensions of the original FFTBM, as described in Section 5. Section 6 

describes the FFTBM improved by signal mirroring, which eliminates the edge effect between 

the first and last data point. Finally, Section 7 describes the results of the demonstration 

application of FFTBM improved by signal mirroring to the loss-of-fluid test (LOFT) L2-5 test, and 

Section 8 presents the report’s conclusions. 

3 

2. COMPLEX REPRESENTATION OF SINUSOIDS 

As shown in Figure 1, a complex number C can be expressed in rectangular notation C = a + ib

or polar notation iθ C = Me . 

In rectangular notation, the complex numbers are represented by locations in a two-dimensional 

plane called the “complex plane.” As shown in Figure 1(a), the horizontal axis of the complex 

plane is the real part of the complex number, and the vertical axis is the imaginary part. The 

mathematical notation for separating a complex number into its real and imaginary parts uses 

the operators Re() and Im(). 

Complex numbers can also be expressed in polar notation as shown in Figure 1(b). The 

magnitude M is the length of the vector starting at the origin and ending at the complex point, 

while the phase angle θ is measured between the vector and the positive x-axis. The following 

equations can convert complex numbers between rectangular and polar notation: 

⎥

⎦

⎤ ⎢

⎣

⎡ = + = C

C ( C) ( C) , arctan 2 2

Re

Im M Re Im θ 1) 

ReC = Mcos(θ ), ImC = M sin(θ ). 2) 

a+ib

Imaginary axis

-a+ib

Real axis

-a-ib a-ib

-a a

-b

b a+ib

Imaginary axis

-a+ib

Real axis

-a-ib a-ib

-a a

-b

b

Imaginary axis

Real axis

arctan( / )

or

2 2

b a

M a b

a ib

=

= +

+

arctan( / ) θ

or

2 2

b a

M a b

a ib

= −

= +

− +

θ

arctan( / )

or

2 2

b a

M a b

a ib

= −

= +

−

θ

arctan( / )

or

2 2

b a

M a b

a ib

= − −

= +

− −

θ

θ

θ

θ

θ

M M

M M

-a a

-b

b

Imaginary axis

Real axis

arctan( / )

or

2 2

b a

M a b

a ib

=

= +

+

arctan( / ) θ

or

2 2

b a

M a b

a ib

= −

= +

− +

θ

arctan( / )

or

2 2

b a

M a b

a ib

= −

= +

−

θ

arctan( / )

or

2 2

b a

M a b

a ib

= − −

= +

− −

θ

θ

θ

θ

θ

M M

M M

-a a

-b

b

(a) Complex numbers in rectangular form (b) Complex numbers in polar form 

Figure 1 Representation of complex numbers 

Beginning with a proper complex number a + ib and applying Eq. 2 obtains the following: 

a + ib = M(cosθ + i sinθ ). 3) 
4 

The expression on the left is the proper rectangular description of the complex number, while 

the expression on the right is the proper polar description. One of the most important equations 

in complex mathematics is the Euler’s relation: 

e cos(x) i sin(x). ix = + 4) 

Rewriting Eq. 3 using the Euler’s relation results in the most common way of expressing a 

complex number in polar notation (i.e., with a complex exponent): 

iθ a + ib = Me 5) 

This exponential form has the advantage of being convenient for mathematical operations, since 

it is very simple to multiply and divide complex numbers written in such a form: 

, ( )

1 2 1 2

θ1 θ 2 θ1 +θ 2 = i i i M e M e M M e 6) 

. ( )

2

1

2

1 1 2

2

1

θ θ

θ

θ − = i

i

i

e

M

M

M e

M e 7) 

With complex numbers, sine and cosine waves can be represented in a compact way. The 

conventional way of representing a sinusoid is M cos(ωt −φ) in polar notation and 

Acos(ωt) + B sin(ωt) in rectangular notation, where A = M cos(φ) , B = M sin(φ), 

( )

1/ 2 2 2 M = A + B and φ = arctan( ) B A . Since two parameters are required to represent a single 

sinusoid (i.e., A and B, or M and φ ), the use of complex numbers to represent these important 

waveforms is a natural solution. Using substitution, the change from the conventional sinusoid 

representation to a complex number is straightforward. In rectangular form, the substitution is 

( ) ( )

cos( ) sin( ) ,

conventional representation complex number

A ωt + B ωt a + ib 8) 

where . In words, the amplitude of the cosine wave becomes the real part of the 

complex number, and the sine wave’s amplitude becomes the imaginary part. It is important to 

understand that this is not an equation, but merely a way of letting a complex number represent 

a sinusoid. This substitution also can be applied in polar form: 

( ) ( )

cos( ) ,

conventional representation complex number

i M t Me θ ω − φ 9) 

where and . In words, the polar notation substitution leaves the magnitude and the 

phase angle the same. 

M M φ θ

A a and B b
5 

3. FOURIER TRANSFORM OF DISCRETELY SAMPLED DATA

A physical process can be described either in the time domain as a function of time F(t) or in the 

frequency domain as a function of frequency ( ) ~

F f (generally a complex number). For many 

purposes, it is useful to think of F(t) and F(f ) as being two different representations of the same 

function, which indeed they are. One switches between these two representations by means of 

the Fourier transform equations: 

( ) . ~ ( )

( ) ( ) , ~

2

2

F t F f e df

F f F t e dt

i f t

i f t

∫

∫

∞

−∞

∞

−∞

−

=

=

π

π

10) 

If the angular frequency ω is used instead of the frequency f , Eq. 10 looks like: 

( ) , ~

2

1 ( )

( ) ( ) , ~

ω ω

π

ω

ω

ω

F t F e d

F F t e dt

i t

i t

∫

∫ ∞

−∞

∞

−∞

−

=

=

11) 

where the following relation was considered: 

2 ( ) ( ) . ω / 2π ω π ω = ≡ = f f H H f 12) 

There are fewer factors of 2 to remember if the Fourier transform is written as a function of π f , 

especially in the case of discretely sampled data. In the most common situation, the function 

F(t) is sampled at evenly spaced intervals in time. Suppose that there are N consecutive 

sampled values 

F ≡ F(t ), t ≡ k , k = 0,1,2,...,N −1, k k k τ 13) 

with the sampling interval τ. For any sampling interval τ, there is also a special frequency fc, 

called the Nyquist critical frequency, given by: 

. 2

1

τ

fc ≡ 14) 

The Nyquist critical frequency (Ref. 3) is important for two related, but distinct reasons: the 

sampling theorem and aliasing. The sampling theorem states the conditions under which the 

samples of a signal (e.g., a function of time) can be used to reconstruct the signal perfectly: 

“When sampling a bandlimited signal (e.g., through analog to digital conversion) the sampling 

frequency must be greater than twice the signal’s bandwidth in order to be able to reconstruct 

the original perfectly from the sampled version.” Aliasing is an effect that causes different 

continuous signals to become indistinguishable (or to become aliases of one another) when 

sampled. When this happens, the original signal cannot be uniquely reconstructed from the 

sampled signal. Critical sampling of a sine wave is two sample points per cycle. 
6 

3.1 Discrete Fourier Transform

The Fourier transform of a function can be estimated from a finite number of its sampled points. 

With N numbers of input, the method, evidently, should be able to produce no more than N

independent numbers of output. So, instead of estimating the Fourier transform ( ) ~

F f at 

continuous values of f in the range –fc to fc, estimates should be made only at discrete values: 

. 2 ,..., 2 , N Nn

N

n fn ≡ = − τ 15) 

As the two extreme values of n are equal, this reduces the count to N. The approximation of the 

integral in Eq. 10 by the discrete sum gives the following: 

( ) ( ) . ~ 1

0

1

0

2 2 2 /

∫ ∑ ∑

−

=

−

=

− − ∞

−∞

− = ≈ =

N

k

N

k

i k n N

k

i f t

k

i f t F fn F t e dt F e F e π n π n k π τ τ 16) 

The final summation in Eq. 16 is called the discrete Fourier transform (DFT) of the N points Fk. 

This is denoted by Fn

~ : 

. ~ 1

0

2 /

∑

−

=

− =

N

k

i k n N Fn Fke π 17) 

The relation in Eq. 16 between the DFT of a set of numbers and their continuous Fourier 

transform, when they are viewed as samples of a continuous function sampled at an interval τ , 

can be written as: 

. ~ ( ) ~ n Fn F f ≈ τ 18) 

Until now, it has been assumed that the index n in Eq. 17 varies from –N/2 to N/2. However, it is 

evident that Eq. 17 is periodic in n, with period N. Therefore, for 1,2,... ~ ~ F−n = FN −n n = . With 

this conversion in mind, n and Fn

~ are allowed to vary from 0 to N-1 (i.e., one complete period). 

Then n and k (in Fk) vary exactly over the same range, so the mapping of N numbers into N

numbers is manifest. When this convention is followed, the zero frequency corresponds to n = 0 

and positive frequencies c 0 correspond to values < f < f 1< n < N / 2 −1, while negative 

frequencies − fc < f < 0 correspond to N / 2 +1< n < N −1. The value n = N/2 corresponds to 

both c f = f and c f = −f . The following is the formula for the discrete inverse Fourier transform, 

which recovers the set of Fk’s exactly from the Fn

~ ’s: 

. 1 ~1

0

2 /

∑

−

=

=

N

n

i k n N

k Fne

N

F π 19) 

The only differences between Eq. 19 and Eq. 17 are the changed sign in the exponent and the 

division of the expression by N. This means that a code for calculating DFTs can also, with 

slight modification, calculate the inverse transforms. 
7 

The complex transform Fn

~ from Eq. 17 written as a real transform is the following: 

) cos(2 / ), ~ Re(

1

0

∑

−

=

=

N

k

Fn Fk πkn N 20) 

) sin(2 / ), ~ Im(

1

0

∑

−

=

= −

N

k

Fn Fk πkn N 21) 

where ) ~ Re(Fn and ) ~ Im(Fn are cosine and sine amplitudes in the following equation of the real 

inverse transform: 

∑[ ] ( ) ( ) −

=

= −

1

0

sin(2 / ). ~ cos(2 / ) Im ~ Re 2 N

n

k Fn kn N Fn kn N

N

F π π 22)

For the calculation of the figure of merit presented in the next subsections, the absolute values 

of Fourier transforms are needed; these are the amplitudes (also called magnitudes). Using 

Eq. 1 produces: 

( )( )) . ~ ) Im( ~ Re( ~ 2 2

Fn = Fn + Fn 23) 

To make the (discrete) Fourier transform and inverse (discrete) Fourier transform equations 

undo each other, a scaling factor must be placed on one or the other equation. In the complex 

case, the scaling factors are 1/N for the discrete case or 1/2π for the continuous case. Since the 

real transforms do not use the negative frequencies, the scaling factors are twice as large: 2/N

and 1/π. The above equations of the Fourier transform and its inverse may look different in 

other publications. The scaling factor multiplying the sum and the sign of the exponent are 

merely conventions and differ in some treatments. The only important things are that the 

forward and inverse DFTs have opposite-sign exponents and the product of their normalization 

factors is 1/N. This is the case in the original FFTBM algorithm described in Section 4, where 

the scaling factor 1/N for the complex transform is in front of the (discrete) Fourier transform 

equation. Using this convention, the zero frequency component represents the mean value of 

the time signal (in the opposite when not using the scaling factor 1/N in front, it would be the 

sum of the discrete points). Nevertheless, as the sums of the experiment and the difference 

signal amplitude spectra are divided, the scaling factor has no influence on the accuracy results 

but the amplitude spectrum changes with the scaling factor (the amplitude spectrum is the 

optional output of the FFTBM software, when it is needed for detailed analysis of the frequency 

spectrum). 

3.2 Fast Fourier Transform for Computing Discrete Fourier Transform

The DFT can be computed with an algorithm called the FFT, which rapidly computes the DFT. 

For information on how FFT works, the reader should refer to the literature (Ref. 3). 

9 

4. DESCRIPTION OF ORIGINAL FAST FOURIER TRANSFORM 

BASED METHOD 

The FFTBM was developed to identify discrepancies between measurements and predictions of 

thermal hydraulic codes. 

4.1 Input Parameters for Fast Fourier Transform

To apply FFT, the function must be identified by a number of values that is a power with the 

base equal to 2 (this was a requirement for older FFT algorithms, such as the one used in the 

original FFTBM). Thus, if the number of points defining the function in the time domain is 

2 , +1 = m N 24) 

where m = 0, 1, 2,..., the FFT algorithm gives the transformed function defined in the frequency 

domain by 1 2 + m values corresponding to the frequencies 

, ( 1,2,...,2 ), m

d

n n

T

N f = = 25) 

where Td is the transient time duration of the sampled signal. 

To use FFTBM, the number of points must be selected for the FFT calculation. This is the same 

as selecting the sampling frequency. Since the FFT algorithm requires that functions are 

identified by a number of values (equally spaced), which is a power of 2, an interpolation to 

satisfy this requirement is necessary. On the other hand, the comparison of the experimental 

and the calculated signal implies that they have the same time scale. Normally, experimental 

data are sampled at a different frequency than are the calculated data. The interpolation results 

in discretely sampled experimental and calculated data at evenly spaced intervals in time 

(sampling interval). In FFT, the sampling frequency of interpolated data is used; therefore, for 

FFT, the sampling theorem must be fulfilled. 

After selecting the number of points N=2m+1, the maximum frequency of transformed functions 

by FFT is given by: 

, 2

2

2

2 2

1

2

1

max

d

m

d

m

d

s

T T T

f N f = = = = =

+

τ 26) 

where Td is the transient time duration of the sampled signal. The relation in Eq. 26 shows that 

the number of selected points is strictly connected to the sampling frequency of interpolated 

data. In the FFTBM algorithm, the minimum number of points is limited to 512. It was decided 

that, instead of the number of points, the input parameter for FFTBM is the fixed frequency fix f . 

The number of points increases until fmax determined by Eq. 26 is greater than fix f or until the 

maximum number of points allowed in the algorithm is reached. If fix f for a given time interval of 

interpolated data gives a number of points lower than the number of experimental and/or 
10 

calculated data, some information could be lost. If fix f is selected so that the number of points is 

too high, little new information is introduced. 

The interpolation using a linear method changes the slope, but it was verified that this effect is 

negligible because these spurious frequencies are at higher frequencies having lower 

amplitudes than typical frequencies characterizing the signal. To filter this spurious contribution, 

the cut-off frequency (fcut) was introduced as the second input parameter. 

4.2 Average Amplitude and Weighted Frequency

The FFTBM shows the measurement-prediction discrepancies in the frequency domain. For the 

calculation of these discrepancies, the experimental signal (Fexp(t)) and the error function ΔF(t)

(difference signal) are needed. The error function in the time domain is defined as 

( ) , exp ΔF t F (t) F (t) = cal − 27) 

where Fcal(t) is the calculated signal. The code accuracy quantification for an individual 

calculated variable is based on amplitudes of the discrete experimental and error signal 

obtained by FFT at frequencies fn (see Eq. 25). These spectra of amplitudes are used for the 

calculation of the average amplitude (AA) that characterizes the code accuracy: 

.

~

~

2

0

exp

2

0

∑

∑

=

= = m

n

n

m

n

n

F (f )

ΔF(f )

AA 28) 

A weighted frequency (WF) is defined as the sum of frequencies multiplied (weighted) by error 

function amplitudes, normalized to the sum of error function amplitudes: 

∑

∑

=

=

⋅

= 2

0

2

0

~

~

WF m

n

n

m

n

n n

ΔF(f )

ΔF(f ) f

. 29) 

In the past, several applications calculated the values of WF (Ref. 4). However, no judgment 

was based on WF. As will be shown in Sections 5 and 6, only AA was used to derive the new 

accuracy measures. 

4.3 Accuracy of Code Calculation

The overall picture of the accuracy of a given code calculation is obtained by defining average 

performance indices (i.e., the total weighted AA (total accuracy)): 
11 

( )( )f i

1

tot i

var

AA AA w

N

i

⋅

=

= ∑ 30) 

and the total WF 

( )( )f i

1

tot i

var

WF WF w

N

i

⋅

=

= ∑ 31) 

with 

( ) 1, var

1

∑ f i

=

=

N

i

w 32) 

where Nvar is the number of the variables analyzed, and (AA)i, (WF)i and (wf)i are the AA, the 

WF, and the weighting factors for the ith analyzed variable, respectively. Each (wf)i accounts for 

the experimental accuracy, the safety relevance of particular variables, and its relevance with 

respect to pressure (Ref. 5). 

Experimental accuracy (wexp)i: Experimental trends of thermal-hydraulic variables are 

characterized by uncertainty because of intrinsic characteristics of the instruments, the 

measurement method, and different evaluation procedures used to compare experimental 

measures and the code predictions. 

Safety relevance (wsaf)i: Higher importance is attributed to the accuracy of those calculated 

variables that are relevant for safety and design (such as pressure and peak clad temperature). 

Primary pressure normalization (wnorm)i: This contribution is given by a factor that normalizes the 

AA value calculated for the selected variables with respect to the AA value calculated for the 

primary pressure. This factor has been introduced to consider the physical relations existing 

between different quantities (i.e., fluid temperature and pressure in case of saturated blowdown 

must be characterized by the same order of error). 

The weighting factor for the ith variable is therefore defined as: 

( ) ( ) ( )( )

∑( ) ( )( ) =

⋅ ⋅

⋅ ⋅ = var

1

exp

exp

f i N

i

saf i norm i i

saf i norm i i

w w w

w w w

w , 33) 

where wexp is the contribution related to the experimental accuracy, wsaf is the contribution that 

expresses the safety relevance, and wnorm is the contribution of primary pressure normalization. 

Table 1 shows the weighting factors. 

The definition of weighting factors introduces a degree of engineering judgment in the 

development of the FFTBM method. In the later applications of FFTBM, these weighting factors 

have been fixed. The weights must remain unchanged during each comparison between code 

results and experimental data concerning the same class of transient. 
12 

Table 1 Weighting Factor Components for the Analyzed Quantities (Ref. 5) 

Quantity wexp wsaf wnorm

Pressure drops 0.7 0.7 0.5 

Mass inventories 0.8 0.9 0.9 

Flow rates 0.5 0.8 0.5 

Primary pressure 1.0 1.0 1.0 

Secondary pressure 1.0 0.6 1.1 

Fluid temperatures 0.8 0.8 2.4 

Clad temperatures 0.9 1.0 1.2 

Collapsed levels 0.8 0.9 0.6 

Core power 0.8 0.8 0.5 

4.4 Methodology for Quantifying Code Accuracy

Given a qualified user and qualified nodalization scheme, the code assessment process 

involves three steps: (1) selection of an experiment from the Committee on the Safety of 

Nuclear Installations (CSNI) validation matrices (Ref. 6) (or a plant transient), (2) qualitative 

assessment, and (3) quantitative assessment. The steps in the methodology for quantifying 

code accuracy, described in more in detail in (Ref. 4), are subdivisions of the scenario into 

“phenomenological windows”; for each phenomenological window, the methodology requires 

specification of key phenomena that are distinctive for this class of transients (for example, 

break flow), identification of the relevant thermal-hydraulic aspects (RTAs) which are particular 

to each transient (these are events or phenomena consequent to the physical process, such as 

subcooled blowdown), and selection of variables characterizing the RTAs (for example. average 

break flow); qualitative assessment of obtained results can be done by visually comparing the 

experimental and calculated variables trends. 

The qualitative assessment gives the first indications about the accuracy of the calculated 

predictions. The qualitative assessment phase is a necessary prerequisite for a subsequent 

quantitative phase. It is meaningless to perform this last phase through the FFTBM if any RTA is 

not predicted. 

The quantitative assessment can be managed by applying the FFTBM. Normally. 20 to 

25 variables are selected for the accuracy analysis. The most suitable factor for the definition of 

an acceptability criterion is the total average amplitude, AAtot. With reference to the accuracy of 

a given calculation, the following acceptability criterion can be defined: 

AA K, tot < 34) 

where K is the acceptability factor valid for the whole transient and is set to K = 0.4. The 

previous studies showed the following: 

• AAtot≤0.3 characterizes very good code predictions. 

• 0.3<AAtot≤0.5 characterizes good code predictions. 

• 0.5<AAtot≤0.7 characterizes poor code predictions. 

• AAtot>0.7 characterizes very poor code predictions. 

In addition, the acceptability factor K = 0.1 has been fixed for the primary pressure, because of 

its importance.
13 

5. DESCRIPTION OF EXTENDED ORIGINAL FAST FOURIER 

TRANSFORM BASED METHOD 

Since the original FFTBM was developed, JSI has suggested a few improvements. These 

improvements are new proposed accuracy measures based on variable accuracy and number 

of discrepancies and the calculation of time-dependent accuracy measures. 

5.1 New Proposed Accuracy Measures

The new accuracy measures were tested on IAEA-SPE-4 data (Ref. 7). Let the accuracy of the 

ith variable be 

( ) , VAi AAi wf i Nvar = ⋅ ⋅ 35) 

which shows what the total accuracy would be if all variables contribute to AAtot to the same 

degree as the ith variable. By this definition, the criteria for AAtot, presented in Section 4.4, are 

also applicable to variable accuracy (VA), and they are even more appropriate than AA for 

variables, because with VA, the weights are already considered. Therefore, VA is applicable 

only in conjunction with the AAtot calculation. 

Based on VA, three new accuracy measures were proposed: the number of discrepancies (ND) 

in the calculation, the minimal variable accuracy VAmin (see Eq. 36), and the maximal variable 

accuracy VAmax (see Eq. 37). The accuracy is minimal when the value of the accuracy measure 

is maximal (AA = 0 indicates perfect agreement). The accuracy is maximal when the value of 

the accuracy measure is minimal. The ND indicates the number of variables with variable 

accuracy VAi above the acceptability limit K = 0.4. The minimal variable accuracy is defined as 

min { } 1 var VA = max VAi ; i = to N 36

and maximal variable accuracy as 

min{ }; 1 . max Nvar VA = VAi i = to 37

The minimal variable accuracy also represents the hypothetical total accuracy combined from 

variables all having the same value of AA = VAmin. In this way, the variable accuracy can be 

compared to acceptability limits for AAtot. If the value indicates a very poor prediction, the 

analyst must refer to the qualitative analysis and check whether the reason for the discrepancy 

is understood. If this value indicates a very good prediction, this means that all other selected 

variables are predicted to also be very good. When VAmin is below K = 0.4, then ND is 0, 

according to the definition. 

5.2 Time-Dependent Accuracy Measures

The FFTBM requires the qualitative assessment and the subdivision of the transient into 

phenomenological windows. Normally, the accuracy analysis is performed for time windows and 

time intervals, where each phenomenological window represents one time window, while time 
14 

intervals start at the beginning of the transient and end at each phenomenological window end 

time. 

Instead of a few phenomenological windows, a series of narrow windows (phases) is proposed 

(around 30 windows for a transient). This makes it possible to check the accuracy of each part 

of the transient and to measure the time dependency of accuracy. In the quantitative 

assessment with three to five phenomenological windows, only global trends are available. In 

the present analysis, the term “moving time window” means a set of equidistant narrow time 

windows as the transient progresses (like a moving chart strip). The term “increasing time 

interval” means a set of time intervals each increased for the duration of one narrow time 

window, where the last time interval is equal to the whole transient duration time. The moving 

time window shows instantaneous details of ΔF(t), and consequently, an overall judgment about 

accuracy cannot be made from it. Instead, the analyst has a picture of instantaneous 

discrepancies. An integral approach is needed to make an overall judgment about accuracy, 

and this is achieved by increasing the time interval, which also shows how the accuracy 

changes with time progression. These time-dependant accuracy measures clearly show when 

the largest total discrepancy occurs and its influence on total accuracy. They also show how the 

transient duration selected for the analysis influences the results. 
15 

6. DESCRIPTION OF FAST FOURIER TRANSFORM BASED 

METHOD IMPROVED BY SIGNAL MIRRORING 

To make FFTBM applicable for all variables, signal mirroring is proposed to eliminate the edge 

effect in calculating AA. 

6.1 Signal Mirroring

In the case of a function Td F( and t) where 0 ≤ t ≤ Td is the transient time duration, its mirrored 

function is defined as 0 Fmir (t) = F(−t), where −Td ≤ t ≤ . From these functions, a new function is 

composed which is symmetrical in regard to the y-axis: m d Td F (t), where −T ≤ t ≤ . By combining 

the original signal and its mirrored signal (signal mirroring), a signal without the edge between 

the first and the last data sample is obtained, which is called a “symmetrized signal.” To explain 

this, Figure 2 shows different signals: the original signal (the LOFT L2-5 test measured intact 

loop hot leg pressure) together with the shifted original signal (Figure 2(a)), the mirrored original 

signal (Figure 2(b)), two periods of the original signal (Figure 2(c)), and the symmetrized signal 

composed from one period of the mirrored signal and one period of the original signal (Figure 

2(d)). Only the symmetrized signal is without the edge when treating an aperiodic signal as one 

period of the periodic signal. The edge is not visible in the plotted signal when the signal is not 

shifted or not plotted as a periodic signal (see Figure 2(c)). However, in the performance of FFT, 

the aperiodic signal is treated as a periodic signal, and therefore the edge is part of the signal, 

which is not physical. 

0

4

8

12

16

0 40 80 120

Time (s)

p (MPa)

Original

Original shifted

(a)

edge edge

(not visible)

0

4

8

12

16

-120 -80 -40 0

Time (s)

p (MPa)

Original mirrored

(b)

edge

(not visible)

0

4

8

12

16

-120 -80 -40 0 40 80 120

Time (s)

p (MPa)

Two periods of

original

(c)

edge

edge

(not visible)

0

4

8

12

16

-120 -80 -40 0 40 80 120

Time (s)

p (MPa)

Symmetrized

(d)

Figure 2 Different signals composed from the original LOFT L2-5 test pressurizer 

pressure signal
16 

Table 2 shows that, in performance of the FFT, the sum of amplitudes of the original, the 

original with shift, and the mirrored signal are the same. The Fourier transform of two periods of 

the original signal is also the same. However, when making the Fourier transform of the 

symmetrized signal, the sum is less than in the case of the signals where the edge is present. In 

this way, the edge contribution to the sum can be seen. In this example, the difference in the 

sum of amplitudes is 9.585 ((25.871 - 16.286) = 9.585). This means that the sum of amplitudes 

of the experimental signal is 37 percent less when the edge effect is not considered, which 

increases the AA almost 59 percent. This means that all integral variables and variables 

dropping to zero value (power, primary pressure during a loss-of-coolant accident (LOCA), etc.) 

exhibit lower AA values because of the edge in the experimental signal. This also partly explains 

the generally very high accuracy of these variables compared to other variables in the past 

applications (Ref. 4). 

The edge effect has a similar influence on the difference signal, with the difference that, in this 

case, the sum is in the numerator for AA calculation and the larger sum of the amplitudes 

means a larger AA. 

Table 2 The Sum of Amplitudes of Signals Shown in Figure 2 

Type of Signal Sum of Amplitudes of the Fourier Transform 

original signal 25.871 

shifted original signal 25.871 

mirrored signal 25.871 

two periods of original signal* 25.871 

symmetrized signal* 16.286 

* The scaling factor 1/N is in the DFT equation in FFTBM, while in the case of the Microsoft Excel Fourier analysis, it is 

in the inverse Fourier analysis. Therefore, when using Microsoft Excel, analysts need to divide the values by the 

number of points used in order to obtain the same results as obtained by FFTBM. 

As the edge effect is eliminated in both the experimental signal amplitude spectrum and in the 

difference signal amplitude spectrum, the new values of AA may be larger or smaller than 

D’Auria AA (Eq. (28) applied to original signals with no signal mirroring), depending on how the 

numerator and the denominator change. Nevertheless, the next section will show that the 

original FFTBM acceptance criteria for AA at the moment can remain in use and that AA based 

on mirroring provides a more objective quantitative measure. For calculating the total AA, the 

primary normalization factors were determined using the AAs with the edge effect. It seems that 

the primary pressure normalization weight partly compensates for the edge effect. Several 

calculations are needed to verify this hypothesis, but this is not the objective of this study. 

Finally, when variables are not equally important or the measurement accuracy is different, the 

importance (safety relevance in the area of nuclear safety) weighting factors and experimental 

accuracy factors can further be used for the primary system (PS) calculations. 

6.2 Calculation of Average Amplitude by Signal Mirroring

For the calculation of the average amplitude by signal mirroring (AAm), Eq. (28) is used for the 

calculation of AA, except that, instead of the original signal, the symmetrized signal is used. The 

reason to symmetrize the signal was to exclude the edge from the signal. The signal is 

automatically symmetrized in the program for the FFTBM improved by signal mirroring as is 

described in (Ref. 1). 
17 

As already mentioned, the edge has no physical meaning, but it causes FFT to produce 

harmonic components. By mirroring, the shapes of the experimental and error signal are 

symmetric and their spectra are different from the original signals spectra, mainly because they 

are without nonphysical edge frequency components. Because of different spectra, the sum of 

the amplitudes changes in both the numerator and the denominator of Eq. (28). For further use 

in distinguishing between the error and experimental signal edge contribution, two new 

definitions are introduced for the AA of the error and experimental signal, related to the 

numerator and denominator of Eq. (28): 

, ~

2 1

1 2

0

∑ + =

=

m

n

err m n AA ΔF(f ) (38) 

. ~

2 1

1 2

0

exp ∑ exp + =

=

m

n m n AA F (f ) (39) 

When both the original and error signal are without the edge, in principle, different AAerr and 

AAexp may be obtained by the original FFTBM and the FFTBM improved by signal mirroring. 

Indeed, AA and AAm are slightly different measures if the signals are without an edge. The 

values obtained with the original FFTBM and improved FFTBM by signal mirroring are the same 

only for symmetrical original signals, but this is not really a deficiency of the proposed improved 

FFTBM, since it is important only that the method judges the accuracy realistically and that it is 

consistent within itself. 

19 

7. DEMONSTRATION APPLICATION OF THE FAST FOURIER 

TRANSFORM BASED METHOD IMPROVED BY SIGNAL 

MIRRORING 

The FFTBM improved by signal mirroring was applied to calculations of the ISP-13 test 

performed in the BEMUSE project. These calculations were selected because they were part of 

an international activity and the number of calculations was large enough that one can readily 

see the relationship between variables and quantitative results. The objective of the ISP-13 test 

(LOFT L2-5 test) was to simulate a LOCA caused by a double-ended, off-shear guillotine cold 

leg rupture, coupled with a loss of offsite power in the nuclear LOFT test facility. The analysis 

assumed delayed initiation of the high-pressure injection system (HPIS) and the low-pressure 

injection system (LPIS) emergency core cooling system (ECCS) (Ref. 9). 

7.1 Facility and Test Description

The LOFT Integral Test Facility is a scale model of a pressurized-water reactor (PWR). The 

facility is used to model the nuclear thermal-hydraulic phenomena that would take place in a 

PWR during a LOCA. The general philosophy in scaling coolant volumes and flow areas in 

LOFT was to use the ratio of the LOFT core (50 megawatts thermal (MWt)) to a typical light 

PWR core (3,000 MWt). For some components, this factor is not applied; however, it is used as 

extensively as practical. In general, components used in LOFT are similar in design to those of 

a PWR. Because of scaling and component design, the LOFT LOCA is expected to closely 

model a PWR LOCA. The LOFT ECCS, which simulates the ECCS of a commercial PWR, 

consists of two accumulators, an HPIS, and an LPIS.

The experiment was initiated by opening the quick-opening blowdown valves in the broken loop 

hot and cold legs. The reactor scrammed on low pressure at 0.24 ± 0.01 seconds. Following the 

reactor scram, the operators tripped the primary coolant pumps at 0.94 ± 0.01 seconds. 

Accumulator injection of ECCS to the intact loop cold leg began at 16.8 ± 0.1 seconds when the 

system pressure dropped below 4.2 megapascals (MPa). Delayed emergency core cooling 

injection from the HPIS and LPIS began at 23.90 ± 0.02 seconds and 37.32 ± 0.02 seconds, 

respectively. The fuel rod peak cladding temperature of 1,078 ± 13 kelvin (K) was attained at 

28.47 ± 0.02 seconds. The accumulator emptied at 49.6 ± 0.01 seconds. The cladding was 

quenched at 65 ± 2 seconds, following the core reflood. The LPIS injection was stopped at 

107.1 ± 0.4 seconds, at which point the experiment was considered complete. 

In Phase II of the BEMUSE project, which is the reanalysis of the ISP-13 exercise and posttest 

calculation of the LOFT L2-5 test, 14 participants provided their calculations. The University of 

Pisa, the lead organization in Phase II, provided these calculations, together with experimental 

data. JSI performed an independent quantitative analysis. 

7.2 Qualitative Assessment of the Calculations

All steps of qualitative analysis were performed in the framework of BEMUSE, Phase II (Ref. 

10), and are briefly described below. The first part of the qualitative analysis entailed visual 

comparisons between experimental and calculated time trends, with four subjective judgments 

assigned (excellent, reasonable, minimal, and unqualified). Table 12 in the BEMUSE Phase II 
20 

Report (Ref. 10) shows the list of selected variables. Each participant presented detailed 

comments concerning judgment of the time trends. The unqualified mark was assigned to the 

steam generator (SG) secondary pressure of the Experimental Design Office (EDO) 

“Gidropress” (GID) calculation and the core inlet liquid temperature of the Korea Institute of 

Nuclear Safety (KINS) calculation. 

Participants also concluded that a possible unqualified mark might be assigned for the core inlet 

liquid temperature calculation of Universitat Politècnica de Catalunya (UPC) because of the 

jump of about 100 K at about 40 seconds and the hot rod temperature calculation (zone 4, 

bottom level) of UPC because of the absence of dryout (not used in quantitative analysis). 

For the purpose of this analysis, the above statements were again verified. Figures 4 through 23 

show the plots for time trends. The SG secondary pressure of the GID calculation shown in 

Figure 5 oscillates significantly, which is not physical. Similarly, the core inlet liquid temperature 

of the KINS calculation shown in Figure 8 drops to values around 320 K (like the ECCS injection 

flow temperatures) after 20 seconds. 

Table 13 of the BEMUSE Phase II Report (Ref. 10) presents the list of RTAs. The following are 

the most important considerations identified in Table 13: 

• All participants overpredicted the integral break flow rate at 100 s; 

• The time of pressurizer emptying was well predicted by each participant; 

• All participants underpredicted the time of PCT except for GRS and 

KAERI; 

• In general, the value of PCT was well calculated. The maximum 

differences with respect to the experimental value of 1,078 K have been 

obtained by NRI-2 (1250 K) and by KAERI (980 K); 

• The upper plenum pressure behavior has been well predicted during all 

transients, except in correspondence of 20 s when GID, JNES, KINS, and 

TAEK underpredicted the value of about 1–2 MPa; 

• As a consequence of the previous point, GID, JNES, KINS, and TAEK 

had an early accumulator injection; 

• A good agreement has been obtained for the mass injected by the 

accumulator with the exception of KAERI which overpredicted by about 

500 kg; 

• The HPIS and LPIS behaviors (time, flow rates and injected masses) 

were well predicted by all participants, except for GRS for which, due to 

typing mistakes during the preparation of the input deck, the intervention 

times were imposed 5 s earlier than in the experiment; 

• It shall be noted that experimental values of the primary system mass 

were not available. 

This study verified the RTA considerations listed above through visual observation of Figures 4 

through 23. The finding that Japan Nuclear Energy Safety (JNES) break flow integrals were 

overpredicted could not be verified because the break flow integral calculated by JNES shows 
21 

overprediction (see Table 13 of the BEMUSE Phase II Report (Ref. 10), while the JNES break 

flows in cold and hot leg (Figure 12 and Figure 14) are underpredicted. Section 7.3 and Table 6, 

which presents results of simple mass balance, give further details. The finding regarding 

pressurizer emptying was not verified as the time trend of pressurizer level was not available for 

quantitative analysis. 

The consideration regarding PCT can be verified from Figure 20. The Nuclear Research 

Institute (NRI-M) (labeled NRI-2 in BEMUSE) calculated the highest peak cladding temperature 

(PCT), while the Korean Atomic Energy Research Institute (KAERI) and JNES calculated the 

two lowest values (980 K and 973 K, respectively). This means that, in addition to the NRI-M 

and KAERI calculations, the JNES calculation should be listed among the calculations that 

show maximum difference with respect to the experimental PCT value. 

The upper plenum pressure was not used in quantitative analysis; however, the authors of this 

report verified that the experimental trend of upper plenum pressure found in data for sensitivity 

analysis is the same as the experimental hot leg pressure found in quantitative analysis. An 

examination of hot leg pressure in Figure 4 can verify that GID (GIDROPRESS in BEMUSE), 

KINS, JNES, and Türkiye Atom Enerjisi Kurumu (TAEK) calculations underpredicted the 

pressure by about 1 to 2 MPa at 15 seconds. The KAERI calculation is similarly underpredicted. 

Consequently, all of these five calculations have early accumulator injection, which is evident 

from the accumulator pressures shown in Figure 6. However, with the knowledge that total HPIS 

and LPIS injected flow until quenching was less than 200 kilograms (kg) and that the 

accumulator injection time ranged from 16.8 seconds to 49.6 seconds (Ref. 10), the ECCS 

integral flow shown in Figure 16 can be used to verify accumulator injection. When the 

accumulator stops injecting, the plotted lines of the ECCS integral break. At that time, the 

KAERI calculation shows the largest difference (around 500 kg). The TAEK calculation shows a 

similar difference. However, it is clear that the data for TAEK should be corrected (the initial 

value of mass at time 0 should be subtracted). The BEMUSE report (Ref. 10) shows the ECCS 

integral for TAEK correctly, while the authors plotted the data as received. 

This study could not verify the qualitative considerations concerning HPIS and LPIS behavior. 

However, Figure 16 indicates that the injection by HPIS and LPIS 5 seconds earlier contributes 

little to the total ECCS mass injection as the accumulator was prevailing at that time. Figure 17 

shows the PS mass. Because the BEMUSE Phase II report (Ref. 10) mentions that 

experimental ECCS integral flow rate and the PS mass are derived from a calculated quantity 

(experimental data were not available), no further analysis was made. 

On the basis of the verification of qualitative evaluation of variables and RTA in BEMUSE 

Phase II, the quantitative analysis was performed. If strictly following the methodology, 

quantitative analysis should not be performed for two calculations with an unqualified mark for 

time trends. However, this report will show that FFTBM was able to detect such variables as the 

worst predictions. 

7.3 Quantitative Assessment of the Calculations

In BEMUSE Phase II, quantitative assessment was performed with original FFTBM. For this 

report, quantitative assessment was performed using both the original FFTBM and FFTBM 

improved by signal mirroring. Although the results obtained from the assessment are not part of 

the BEMUSE final report, some interesting results were found by using the original FFTBM and 

FFTBM improved by signal mirroring and time-dependent analysis. Table 3 shows information 
22 

about the participants in the BEMUSE Phase II study. In total, 13 organizations performed 

14 calculations using 6 different codes (9 different code versions). The code used most 

frequently was RELAP5/MOD3.3. 

The time-dependent quantitative analysis was performed for an increasing time interval for each 

5 seconds. For the accuracy quantification, all frequencies were used (fcut higher than the 

maximum frequency was selected). Table 4 shows the variables used in the BEMUSE 

qualitative analysis. The present study’s quantitative analysis used 18 out of 20 available 

variables to calculate total accuracy, in the same method used in BEMUSE Phase II (see 

Table 14 in the BEMUSE Phase II Report (Ref. 10)). Table 5 shows some additional variables. 

The quantitative analysis determined accuracy for the cold and hot leg integrated break flow. 

However, the calculation of total accuracy did not consider these variables. 

Figures 4 through 23 show the results of the accuracy analysis. On each figure is shown the 

variable (a), D’Auria AA (b), and AA by signal mirroring (c). All AAs are calculated as a function 

of time. In this way, the contribution of each discrepancy is evident. Also, the single value 

parameters from qualitative analysis can be verified (e.g., integral break flow rate at 

100 seconds). The general difference between the D’Auria AA and the AA of the symmetrized 

signal is that the symmetrized signal AAs are generally larger (because the edge effect is 

eliminated from the experimental signal) and are more monotonic increasing functions than 

D’Auria AAs (if the measure is an integral, it should be a monotonic increasing function in 

principle). On the other hand, eliminating the edge effect from the difference signal also 

decreases the value of AA. In this analysis, it very rarely happened that in the whole time 

interval the edge effect is significant (however, this is not the case for shorter time intervals). 

Therefore, when ranking participants by values of AA for each variable, the rank remains mostly 

unchanged for the whole transient duration time interval. 

Table 3 Participants Performing Calculations 

Organization Calculation ID Code Used 

Commissariat à l’Energie Atomique (CEA), France CEA CATHARE 2.5 

EDO “Gidropress” (GID), Russia GID TECH-M-97 

Gesellschaft für Anlagen- und Reaktorsicherheit mbH 

(GRS), Germany 

GRS ATHLET1.2C 

Institut de radioprotection et de sûreté nucléaire (IRSN), 

France 

IRSN CATHARE 2.5 

Japan Nuclear Energy Safety (JNES), Japan JNES TRAC-P 5.5.2 

Korea Atomic Energy Research Institute (KAERI), South 

Korea 

KAERI MARS 2.3 

Központi Fizikai Kutató Intézet (KFKI), Hungary KFKI ATHLET 2.0A 

Korean Institute of Nuclear Safety (KINS), South Korea KINS RELAP5/MOD3.3

Nuclear Research Institute (NRI), Czech Republic NRI-K (Kyncl) RELAP5/MOD3.3

Nuclear Research Institute (NRI), Czech Republic NRI-M 

(Macek) 

ATHLET 2.0A 

Paul Scherrer Institute (PSI), Switzerland PSI TRACE 4.05 

Türkiye Atom Enerjisi Kurumu (TAEK), Turkey TAEK RELAP5/MOD3.3

Universitat Politècnica de Catalunya (UPC), Spain UPC RELAP5/MOD3.3

University of Pisa (UPI), Italy UPI RELAP5/MOD3.2
23 

Table 4 Variables Used in the BEMUSE Qualitative Analysis

ID Description of variable 

P1 Intact loop pressure in hot leg 

P2 SG pressure—secondary side 

P3 Accumulator A pressure 

P4 Pressurizer pressure 

P5 Core inlet fluid temperature 

P6 Core outlet fluid temperature 

P7 Upper head fluid temperature 

P8 SG DC bottom fluid temperature 

P9 Break flow rate in cold leg 

P10 Break flow rate in hot leg 

P11 ECCS integral break flow rate 

P12 Primary side total mass 

P13 SG pressure drop—UT PS 

P14 Primary pump pressure drop 

P15* Rod surface temperature in hot rod in hot channel (zone 4)—bottom level 

P16 Rod surface temperature in hot rod in hot channel (zone 4)—2/3 core height 

P17* Rod surface temperature in hot rod in hot channel (zone 4)—top level 

P18 Rod surface temperature in average rod in average channel (zone 2)—bottom 

level 

P19 Rod surface temperature in average rod in average channel (zone 2)—2/3 core 

height 

P20 Rod surface temperature in average rod in average channel (zone 2)—top level 

* Not used in FFTBM analysis 

Table 5 Additional Derived Variables Available for Quantitative Assessment 

ID Description of variable 

intP9* Integrated break flow rate in cold leg 

intP10* Integrated break flow rate in hot leg 

PN3+ Total integrated break flow 

* Calculated by JSI from data flow rates and not used in FFTBM analysis 

+ Received from University of Pisa; not used in FFTBM analysis 

Nevertheless, in comparisons of the accuracy of variables as the transient progresses, their 

rank changes with time. Because of the elimination of the edge effect from the signals, the 

mean of the total AA of calculations increases from 0.27 to 0.37. Figures 4 through 23 also 

show that variables with AA values close to 1 and above are very inaccurate from the very 

beginning of the transient (examples are P9, P10, P13, and P14). These are mostly break flows 

and pressure drops where the gradient of changes is very large. For example, in integrations of 

the break flow, the AA is 2 to 3 times lower; however, the weights are 3 times higher, resulting in 

a similar accuracy of the variable. It is obvious that these weights favor the mass lost through 

the break or injected into the system rather than the exact prediction of flows. However, 

because of weighting, the AAs of different variables cannot be directly compared; therefore, VAs 

were proposed. This approach is acceptable for one class of transients, although the FFTBM 

becomes a less general method if the definition of weights is required for each class of 

transients. 
24 

Figure 4 shows the hot leg pressure at the beginning of the accident. FFTBM indicates that, in 

several calculations, the discrepancy is already evident in the first seconds. A detailed 

investigation revealed that the experimental pressure drop occurred in 0.1 seconds, while 

several calculations (those of CEA, KINS, IRSN, NRI-K, NRI-M, PSI, TAEK, and UPI) provided 

data with a time step of 0.5 seconds. This discrepancy has the most impact on accuracy. After 

the first time interval of 0–5 seconds, the accuracy of some calculations (those of GID, JNES, 

KINS, KAERI, and TAEK), further decreases. Qualitative analysis also identified these 

calculations. This is a good example of the ability of time-dependent accuracy analysis to give 

information useful in understanding the discrepancies. For example, FFTBM suggests a 

detailed investigation in the beginning of the transient; therefore, hot leg pressure was plotted in 

the time interval 0–1 second (see Figure 3(a)). After that, it become evident that more calculated 

data would be needed to describe the fast pressure drop. Finally, Figure 3(b) fully supports the 

BEMUSE qualitative conclusion regarding the pressure underprediction and quantitative results. 

Figure 5 shows that, in the case of the JNES calculation of SG pressure on the secondary side, 

the AA is reduced because of the larger edge effect in the difference signal than in the 

experimental signal. In other cases, the contribution of the elimination of the edge effect from 

the experimental signal is larger than the contribution from the difference signal. Note that the 

very large AA value of the GID calculation is the result of several oscillations present in the 

signal, which are not physical, and the variable was therefore judged as unqualified in the 

BEMUSE qualitative analysis. 

Figure 6(a) shows the accumulator pressure. In general, the injection occurs earlier in the 

calculations than in the experiment as the setpoint pressure is reached earlier (see Figure 3(a)). 

Discrepancies also appear in the second part of the transient when in some calculations (GID, 

UPC, NRI-K) the accumulator was isolated. Figures 6(b) and 6(c) show that the trend for 

D’Auria AA is not monotonic because of the large edge effect contribution during the pressure 

decrease. On the other hand, the AA obtained by mirroring clearly shows that the major reason 

for discrepancies is the earlier start of the pressure drop. It is also evident that accuracy in the 

second part of the transient decreases (the AA is larger) only for the calculations with increasing 

discrepancies. Finally, it can be seen that the elimination of the edge effect from the 

experimental signal almost doubles the value of AAm. 

Figure 7(a) shows the pressurizer pressure dropping to the containment pressure. The 

predicted pressure drop is faster than in the experiment. The trend of D’Auria AA (Figure 7(b)) 

and AAm (Figure 7(c)) obtained by signal mirroring is similar to the trend of accumulator 

pressure, while the pressurizer pressure prediction is slightly more accurate than the prediction 

of accumulator pressure. 

The visual agreement between the calculated and the experimental core inlet coolant 

temperature (see Figure 8(a)) for most calculations is good. The exception is the KINS 

calculation, which is also predicted by FFTBM. Figure 8(b) shows the large contribution of the 

edge effect to the difference signal in the case of the KINS calculation and, in part, the JNES 

calculation between 10 and 30 seconds. This contribution is eliminated in the case of signal 

mirroring, as shown in Figure 8(c). The BEMUSE qualitative analysis judged the KINS 

calculation of core inlet temperature as unqualified. FFTBM very clearly confirms this 

conclusion. The reason for the rather poor prediction in the case of the JNES calculation is the 

mismatch in the initial core inlet temperature, which is 50 K higher than the temperature in the 

experiment (hardly visible from the graph) and almost equal to the core outlet temperature, the 

earlier temperature drop, and the two heatups between 40 and 50 seconds. The detected wrong 
25 

initial temperature needs clarification. Finally, the elimination of the edge effect from the 

experimental signal increases the AAm values by approximately 50 percent. 

The visual observation of the core outlet temperature shown in Figure 9(a) reveals the largest 

discrepancies for the CEA, GRS, and NRI-M calculations. The quantitative results shown in 

Figures 9(b) and 9(c) confirm this observation. The BEMUSE qualitative analysis does not 

comment on these discrepancies. 

No calculation predicted the behavior of the upper head fluid temperature shown in Figure 

10(a). As shown, the calculations follow the trend of the core outlet fluid temperature. Because 

of this mismatch, the AAs shown in Figure 10(c) are larger than in the case of core inlet and 

outlet fluid temperatures. The BEMUSE qualitative analysis does not comment on this finding. 

The differences in the SG fluid temperatures in the bottom downcomer are rather small (see 

Figure 11(a)); therefore the accuracy is also high. As the edge effect is small, the differences 

between the D’Auria AA and the AA obtained by mirroring are also small. 

The experimental cold leg break flow shown in Figure 12(a) was largest in the initial 4 seconds, 

contributing almost half of the inventory lost. Therefore, the correct prediction during this period 

is the most important. One source of discrepancies is that, in the experiment, the break flow 

starts at 0.32 seconds, while in the calculations, it begins at 0 seconds. Second, in some 

calculations, the data are available only every 0.5 seconds, thereby slightly mitigating the effect 

of this delay (ramp function instead of step function). As shown in Figure 12(c), AAm is then 

practically constant, and the value slightly decreases as the zero amplitude contributing to AAm

decreases as the transient progresses. Nevertheless, in the integration of break flows and 

making plots (see Figure 13(a)), FFTBM correctly ranks the calculations regarding cold leg 

break prediction. 

The hot leg break flow (see Figure 14(a)) is smaller than the cold leg break flow. In the 

calculations, the exceptions were the UPC calculation (which shows just the opposite) and the 

JNES flow (in some periods negative). In spite of this, the GRS calculation was judged to be 

less accurate than the UPC and JNES calculations (Figures 14(b) and 14(c)) because of the 

initial flow spike, which contributed almost nothing to the mass lost through the break. An 

additional study of the mass balance in the PS revealed that the data for the GRS calculations 

of break flows are incorrect, as shown by Table 6. 

The total integrated flow PN3 calculated by GRS is different from the sum of discharged masses 

calculated from the flow rate (P9, P10) also provided by GRS, while the mass balance for GRS 

is correct when using the GRS calculation of total break flow. At the end of the transient, the 

remaining PS mass is equal to PS initial mass (P120) plus the mass injected by the ECCS (P11) 

minus the mass discharged through the break (PN3). The mass balance check (without mass 

error) shows that the JNES flow rate data are also wrong. In the UPC calculations, the PS mass 

is not consistent with the calculated injected and discharged flow. Also, the break flows are 

quite different from the experimental data. It is also strange that more PS mass was discharged 

through the hot leg side of the break (the opposite of the situation in the experiment and most of 

the calculations). The CEA, GRS, and KFKI calculations obtained perfect mass balance, when 

considering the data received for the integral of total break flow. Besides the UPC calculation, 

the KAERI calculation also obtained rather poor mass balance, as judged by FFTBM. Finally, 

Figure 14(c) shows that FFTBM clearly distinguishes the GRS, JNES, and UPC flow rates (all 

suspicious data) from other calculations. 
26 

Table 6 Masses for Mass Balance Verification 

 Mass (kg) 

 P11 P120 P12 intP9 intP10 cal1P12

cal1P12-

P12 PN3 cal2P12 

cal2P12-

P12 

EXP 2019 5330 1862 3833 1602 1914 53 NA NA NA 

CEA 2305 5639 1115 5177 1404 1364 248 6829 1116 0 

GID 1677 5568 791 5119 1425 702 -89 6558 687 -104 

GRS 1849 6019 1437 6480 1819 -431 -1868 6431 1437 0 

IRSN 2303 5865 1237 5136 1458 1573 336 6595 1573 336 

JNES 1840 5515 1127 2062 -105 5397 4270 6428 926 -201 

KAERI 2479 5645 1556 4843 892 2388 832 5823 2301 745 

KFKI 1859 5566 1147 4631 1562 1233 86 6278 1147 0 

KINS 2430 5283 842 5292 1702 720 -122 6429 1285 443 

NRI-K 1953 5675 1742 3836 1341 2451 709 5375 2253 512 

NRI-M 1960 5565 1709 4625 1500 1401 -308 5907 1618 -91 

PSI 1989 5388 957 4885 1753 739 -218 6873 505 -453 

TAEK 2602 5533 2497 3371 940 3824 1327 5349 2786 289 

UPC 2168 5290 1571 1974 5900 -416 -1986 7874 -416 -1986 

UPI 2169 5342 1666 4006 1662 1844 177 5634 1877 211 

Note: P120 = initial PS mass, cal1P12 = (P11+P120-intP9-intP10), cal2P12 = (P11+P120-PN3) 

The conclusions concerning accuracy, as judged with FFTBM for hot leg break flow, are 

different from the conclusions obtained for integrated hot leg break flows, shown in Figure 15. 

The reason is that the accuracy of break flows depends too much on the first few seconds (later 

the rank remains practically unchanged); therefore, the KAERI calculation was judged to be the 

most accurate. For the integrated break flow, the NRI-M calculation was the best. The NRI-M 

calculation was also the best for the ECCS integrated flow rate shown in Figure 16(a), as it 

resulted in the best prediction of the primary side total mass (see Figure 17(a)). For mass 

balance, the difference is also small (Table 6). Because of the definition of AA (normalization 

with experimental signal), the values of AA and AAm are very large (Figures 16(b) and 16(c)). 

This results from the occurrence of the calculated accumulator injection earlier than in the 

experiment. After the accumulator empties, the mass trend (injected mass) becomes similar to 

that in the experiment, thus slightly increasing the accuracy of the ECCS injected mass. The 

accuracy (Figures 17(b) and 17(c)) of the primary side total mass shown in Figure 17(a) seems 

reasonable. 

The pressure drop shown in Figure 18(a) is an oscillating variable. The AA is also oscillating 

(see Figures 18(b) and (c)), which is explained in more detail in Section 7.4. As the edge effect 

in the difference signal is overwhelmed by oscillations, the D’Auria AA and AA obtained by 

mirroring become rather similar. The AA values are close to or vary around value one because 

both AAerr and AAexp depend on the oscillatory behavior of the experimental signal, and the 

calculated values are close to 0. Depending on all contributions to the AA, this value may be 

smaller or larger than 1. Because of an inherent feature of FFTBM, which is sensitive to 

oscillations, the results of AA are very limited for quantitative assessment. Nevertheless, such 

time functions of accuracy show the analyst that such results do not consider integrally the 

discrepancies and, therefore, the results are not appropriate for the calculation of total accuracy. 

Similar conclusions apply to the variable primary pump pressure drop shown in Figure 19. 

Without further treatment, such signals do not give a fair figure of merit. The weighting factors 

(see Table 1) have rather small values; therefore, this inconsistency is less pronounced in past 
27 

applications of the original FFTBM. Finally, the original FFTBM study in the framework of 

BEMUSE did not consider the variable primary pump pressure drop in the qualitative analysis 

but only in the quantitative analysis. The study reported that participants used an AA value 

close to or larger than 1.0 for the primary pump pressure drop. That conclusion is correct. 

However, without describing the reason (as given above), such a statement could be misleading 

as the value is close to or larger than 1.0 throughout the transient because of the highly 

oscillating experimental signal and the inability of the original FFTBM to make a fair quantitative 

assessment of such signals. 

Figure 20(a) shows the rod surface temperature in the hot rod in the middle of the core. The 

largest mismatch occurs when core quench takes place earlier or later than in the experiment. 

The values of AA obtained by mirroring (Figure 20(c)) seem reasonable and are similar to 

D’Auria AA at the end of the transient (Figure 20(b)). However, in the uncertainty analysis, the 

time period before core quench is more important. The BEMUSE Phase III report (Ref. 11) cites 

the GRS, KAERI, KINS, and UPI calculations as good reference calculations of maximum 

cladding temperatures. Table 14 of the BEMUSE Phase II report (Ref. 10) seems to contradict 

the above statement, as the GRS prediction of hot rod temperature was judged to be the worst. 

The explanation for this is that the GRS calculation in the first 35 seconds was in good 

agreement with the experimental data, as Figure 20 shows for both trend and accuracy 

measures. This was probably the meaning intended in the Phase III BEMUSE report. 

Nevertheless, when considering just the result for the whole time interval, the reader may get 

the impression that the GRS calculation of rod surface temperature is the least accurate. To 

resolve such problems, use of the FFTBM with the capability to calculate accuracy as a function 

of time is advantageous. 

In general, the calculated rod surface temperatures in the average rod at the bottom 

(Figure 21(a)), middle (Figure 22(a)), and top (Figure 23(c)) overpredict the core heatup. There 

are a few exceptions, such as the calculations of KAERI, IRSN, and UPC in Figure 21(c); IRSN 

and KAERI in Figure 22(c); and CEA, KAERI, IRSN, UPC, and UPI (no heatup) in Figure 23(c). 

The BEMUSE qualitative analysis was oriented to the difference in PCT and the time of PCT, 

which is complementary information not directly obtainable from quantitative analysis. In the 

quantitative analysis, the most important factor is the time of core quench, as discrepancies in 

these values contribute much to AA. For this reason, the KAERI, NRI-M, and UPI calculations of 

rod surface temperature in the hot rod in the middle of the core were judged to be the best, 

while the BEMUSE qualitative analysis stated that the NRI-M and KAERI calculations show the 

maximum difference with respect to experimental PCT. This example illustrates why the 

quantitative analysis should not be separated from the qualitative analysis. 
28 

Parameter N° 1: PRESSURE Hot Leg (Intact Loop)

0

2

4

6

8

10

12

14

16

0.0 0.5 1.0

Time (s)

p (MPa)

CEA GID

GRS IRSN

JNES KAERI

KFKI KINS

NRI-K NRI-M

PSI TAEK

UPC UPI

EXP

(a)

Parameter N° 1: PRESSURE Hot Leg (Intact Loop)

0

2

4

6

8

10

12

14

16

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Time (s)

p (MPa)

CEA GID

GRS IRSN

JNES KAERI

KFKI KINS

NRI-K NRI-M

PSI TAEK

UPC UPI

EXP

(b)

Figure 3 Intact loop pressure in hot leg (P1) in two different time intervals 
29 

Parameter N° 1: PRESSURE Hot Leg (Intact Loop)

0

2

4

6

8

10

12

14

16

0 20 40 60 80 100 120

Time (s)

p (MPa)

CEA GID

GRS IRSN

JNES KAERI

KFKI KINS

NRI-K NRI-M

PSI TAEK

UPC UPI

EXP

(a)

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0 20 40 60 80 100 120

Time (s)

AA (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(b)

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0 20 40 60 80 100 120

Time (s)

AAm (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(c)

Figure 4 Intact loop pressure in hot leg (P1)—(a) time trends, (b) AA, (c) AAm
30 

Parameter N° 2: PRESSURE SG Pressure - Secondary Side

4.0

4.5

5.0

5.5

6.0

6.5

7.0

0 20 40 60 80 100 120

Time (s)

p (MPa)

CEA GID GRS

IRSN JNES KAERI

KFKI KINS NRI-K

NRI-M PSI TAEK

UPC UPI EXP

(a)

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0 20 40 60 80 100 120

Time (s)

AA (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(b)

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0 20 40 60 80 100 120

Time (s)

AAm (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(c)

Figure 5 SG pressure, secondary side (P2)—(a) time trends, (b) AA, (c) AAm
31 

Parameter N° 3: PRESSURE Accumulator A

0

1

2

3

4

5

0 20 40 60 80 100 120

Time (s)

p (MPa)

CEA GID

GRS IRSN

JNES KAERI

KFKI KINS

NRI-K NRI-M

PSI TAEK

UPC UPI

EXP

(a)

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0 20 40 60 80 100 120

Time (s)

AA (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(b)

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0 20 40 60 80 100 120

Time (s)

AAm (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(c)

Figure 6 Accumulator A pressure (P3)—(a) time trends, (b) AA, (c) AAm
32 

Parameter N° 4: PRESSURE Pressurizer

0

2

4

6

8

10

12

14

16

0 20 40 60 80 100 120

Time (s)

p (MPa)

CEA GID

GRS IRSN

JNES KAERI

KFKI KINS

NRI-K NRI-M

PSI TAEK

UPC UPI

EXP

(a)

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0 20 40 60 80 100 120

Time (s)

AA (-)

CEA GID GRS IRSN JNES KAERI KFKI

KINS NRI_K NRI_M PSI TAEK UPC UPI

(b)

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0 20 40 60 80 100 120

Time (s)

AAm (-)

CEA GID GRS IRSN JNES KAERI KFKI

KINS NRI_K NRI_M PSI TAEK UPC UPI

(c)

Figure 7 Pressurizer pressure (P4)—(a) time trends, (b) AA, (c) AAm
33 

Parameter N° 5: COOLANT TEMP. Core Inlet

300

400

500

600

700

0 20 40 60 80 100 120

Time (s)

T (K)

CEA GID

GRS IRSN

JNES KAERI

KFKI KINS

NRI-K NRI-M

PSI TAEK

UPC UPI

EXP

(a)

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0 20 40 60 80 100 120

Time (s)

AA (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(b)

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0 20 40 60 80 100 120

Time (s)

AAm (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(c)

Figure 8 Core inlet fluid temperature (P5)—(a) time trends, (b) AA, (c) AAm
34 

Parameter N° 6: COOLANT TEMP. Core Outlet

300

400

500

600

700

0 20 40 60 80 100 120

Time (s)

T (K)

CEA GID

GRS IRSN

JNES KAERI

KFKI KINS

NRI-K NRI-M

PSI TAEK

UPC UPI

EXP

(a)

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0 20 40 60 80 100 120

Time (s)

AA (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(b)

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0 20 40 60 80 100 120

Time (s)

AAm (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(c)

Figure 9 Core outlet fluid temperature (P6)—(a) time trends, (b) AA, (c) AAm
35 

Parameter N° 7: COOLANT TEMP. Upper Head

300

400

500

600

700

0 20 40 60 80 100 120

Time (s)

T (K)

CEA GID GRS

IRSN JNES KAERI

KFKI KINS NRI-K

NRI-M PSI TAEK

UPC UPI EXP

(a)

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0 20 40 60 80 100 120

Time (s)

AA (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(b)

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0 20 40 60 80 100 120

Time (s)

AAm (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(c)

Figure 10 Upper head fluid temperature (P7)—(a) time trends, (b) AA, (c) AAm
36 

Parameter N° 8: COOLANT TEMP. Steam Generator Downcomer bottom

500

510

520

530

540

550

0 20 40 60 80 100 120

Time (s)

T (K)

CEA GID GRS

IRSN JNES KAERI

KFKI KINS NRI-K

NRI-M PSI TAEK

UPC UPI EXP

(a)

0.00

0.03

0.05

0.08

0.10

0 20 40 60 80 100 120

Time (s)

AA (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(b)

0.00

0.03

0.05

0.08

0.10

0 20 40 60 80 100 120

Time (s)

AAm (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(c)

Figure 11 SG DC bottom fluid temperature (P8)—(a) time trends, (b) AA, (c) AAm
37 

Parameter N° 9: BREAK FLOW Cold leg

-200

0

200

400

600

800

0 20 40 60 80 100 120

Time (s)

Flowrate (kg/s)

CEA GID

GRS IRSN

JNES KAERI

KFKI KINS

NRI-K NRI-M

PSI TAEK

UPC UPI

EXP

(a)

0.00

0.20

0.40

0.60

0.80

1.00

1.20

0 20 40 60 80 100 120

Time (s)

AA (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(b)

0.00

0.20

0.40

0.60

0.80

1.00

1.20

0 20 40 60 80 100 120

Time (s)

AAm (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(c)

Figure 12 Break flow rate in cold leg (P9)—(a) time trends, (b) AA, (c) AAm
38 

Parameter: Integral of BREAK FLOW Cold leg

0

1000

2000

3000

4000

5000

6000

7000

0 20 40 60 80 100 120

Time (s)

Mass (kg)

CEA GID GRS

IRSN JNES KAERI

KFKI KINS NRI-K

NRI-M PSI TAEK

UPC UPI EXP

(a)

0.00

0.20

0.40

0.60

0.80

0 20 40 60 80 100 120

Time (s)

AA (-)

CEA GID GRS IRSN JNES KAERI KFKI

KINS NRI_K NRI_M PSI TAEK UPC UPI

(b)

0.00

0.20

0.40

0.60

0.80

0 20 40 60 80 100 120

Time (s)

AAm (-)

CEA GID GRS IRSN JNES KAERI KFKI

KINS NRI_K NRI_M PSI TAEK UPC UPI

(c)

Figure 13 Integrated break flow rate in cold leg (P9int)—(a) time trends, (b) AA, (c) AAm
39 

Parameter N° 10: BREAK FLOW Hot leg

-200

0

200

400

600

800

0 20 40 60 80 100 120

Time (s)

Flowrate (kg/s)

CEA GID

GRS IRSN

JNES KAERI

KFKI KINS

NRI-K NRI-M

PSI TAEK

UPC UPI

EXP

(a)

0.00

0.50

1.00

1.50

2.00

2.50

0 20 40 60 80 100 120

Time (s)

AA (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(b)

0.00

0.50

1.00

1.50

2.00

2.50

0 20 40 60 80 100 120

Time (s)

AAm (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(c)

Figure 14 Break flow rate in hot leg (P10)—(a) time trends, (b) AA, (c) AAm
40 

Parameter: Integral of BREAK FLOW Hot leg

-1000

0

1000

2000

0 20 40 60 80 100 120

Time (s)

Mass (kg)

CEA GID GRS

IRSN JNES KAERI

KFKI KINS NRI-K

NRI-M PSI TAEK

UPC UPI EXP

(a)

0.00

0.20

0.40

0.60

0.80

0 20 40 60 80 100 120

Time (s)

AA (-)

CEA GID GRS IRSN JNES KAERI KFKI

KINS NRI_K NRI_M PSI TAEK UPC UPI

(b)

0.00

0.20

0.40

0.60

0.80

0 20 40 60 80 100 120

Time (s)

AAm (-)

CEA GID GRS IRSN JNES KAERI KFKI

KINS NRI_K NRI_M PSI TAEK UPC UPI

(c)

Figure 15 Integrated break flow rate in hot leg (P10int)—(a) time trends, (b) AA, (c) AAm
41 

Parameter N° 11: LIQUID MASS ECCS Integral Flow Rate

0

500

1000

1500

2000

2500

3000

0 20 40 60 80 100 120

Time (s)

Mass (kg)

CEA GID

GRS IRSN

JNES KAERI

KFKI KINS

NRI-K NRI-M

PSI TAEK

UPC UPI

EXP

(a)

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0 20 40 60 80 100 120

Time (s)

AA (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(b)

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0 20 40 60 80 100 120

Time (s)

AAm (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(c)

Figure 16 ECCS integral break flow rate (P11)—(a) time trends, (b) AA, (c) AAm
42 

Parameter N° 12 : Primary Side Total Mass

0

1000

2000

3000

4000

5000

6000

0 20 40 60 80 100 120

Time (s)

Mass (kg)

CEA GID

GRS IRSN

JNES KAERI

KFKI KINS

NRI-K NRI-M

PSI TAEK

UPC UPI

EXP

(a)

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0 20 40 60 80 100 120

Time (s)

AA (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(b)

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0 20 40 60 80 100 120

Time (s)

AAm (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(c)

Figure 17 Primary side total mass (P12)—(a) time trends, (b) AA, (c) AAm
43 

Parameter N° 13: PRESSURE DROPS SG Pressure drop

-50

0

50

100

0 20 40 60 80 100 120

Time (s)

p (kPa)

CEA GID

GRS IRSN

JNES KAERI

KFKI KINS

NRI-K NRI-M

PSI TAEK

UPC UPI

EXP

(a)

0.00

0.50

1.00

1.50

0 20 40 60 80 100 120

Time (s)

AA (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(b)

0.00

0.50

1.00

1.50

0 20 40 60 80 100 120

Time (s)

AAm (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(c)

Figure 18 SG pressure drop—UT PS (P13)—(a) time trends, (b) AA, (c) AAm
44 

Parameter N° 14: PRESSURE DROPS Primary Pumps Pressure Drop

-100

-50

0

50

100

0 20 40 60 80 100 120

Time (s)

p (kPa)

CEA GID GRS

IRSN JNES KAERI

KFKI KINS NRI-K

NRI-M PSI TAEK

UPC UPI EXP

(a)

0.00

0.40

0.80

1.20

1.60

0 20 40 60 80 100 120

Time (s)

AA (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(b)

0.00

0.40

0.80

1.20

1.60

0 20 40 60 80 100 120

Time (s)

AAm (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(c)

Figure 19 Primary pump pressure drop (P14)—(a) time trends, (b) AA, (c) AAm
45 

Parameter N° 16: Zone 4 - 2/3 Core Height PCT MAX

400

600

800

1000

1200

0 20 40 60 80 100 120

Time (s)

T (K)

CEA GID

GRS IRSN

JNES KAERI

KFKI KINS

NRI-K NRI-M

PSI TAEK

UPC UPI

EXP

(a)

0.00

0.20

0.40

0.60

0.80

0 20 40 60 80 100 120

Time (s)

AA (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(b)

0.00

0.20

0.40

0.60

0.80

0 20 40 60 80 100 120

Time (s)

AAm (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(c)

Figure 20 Rod surface temperature in hot rod in hot channel (zone 4) at 2/3 core 

height (P16)—(a) time trends, (b) AA, (c) AAm
46 

Parameter N° 18: Zone 2 - Bottom Level PCT MAX

400

600

800

1000

0 20 40 60 80 100 120

Time (s)

T (K)

CEA GID

GRS IRSN

JNES KAERI

KFKI KINS

NRI-K NRI-M

PSI TAEK

UPC UPI

EXP

(a)

0.00

0.40

0.80

1.20

1.60

0 20 40 60 80 100 120

Time (s)

AA (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(b)

0.00

0.40

0.80

1.20

1.60

0 20 40 60 80 100 120

Time (s)

AAm (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(c)

Figure 21 Rod surface temperature in average rod in average channel (zone 2) at bottom 

level (P18)—(a) time trends, (b) AA, (c) AAm
47 

Parameter N° 19: Zone 2 - 2/3 Core Height PCT MAX

400

600

800

1000

0 20 40 60 80 100 120

Time (s)

T (K)

CEA GID

GRS IRSN

JNES KAERI

KFKI KINS

NRI-K NRI-M

PSI TAEK

UPC UPI

EXP

(a)

0.00

0.40

0.80

1.20

1.60

0 20 40 60 80 100 120

Time (s)

AA (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(b)

0.00

0.40

0.80

1.20

1.60

0 20 40 60 80 100 120

Time (s)

AAm (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(c)

Figure 22 Rod surface temperature in average rod in average channel (zone 2) at 2/3 core 

height (P19)—(a) time trends, (b) AA, (c) AAm
48 

Parameter N° 20: Zone 2 - Top Level PCT MAX

400

500

600

700

800

0 20 40 60 80 100 120

Time (s)

T (K)

CEA GID

GRS IRSN

JNES KAERI

KFKI KINS

NRI-K NRI-M

PSI TAEK

UPC UPI

EXP

(a)

0.00

0.20

0.40

0.60

0.80

0 20 40 60 80 100 120

Time (s)

AA (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(b)

0.00

0.20

0.40

0.60

0.80

0 20 40 60 80 100 1

Time (s)

AAm (-)

CEA GID GRS IRSN JNES KAERI KFKI KINS

NRI_K NRI_M PSI TAEK UPC UPI

(c)

Figure 23 Rod surface temperature in average rod in average channel (zone 2) at top 

level (P20)—(a) time trends, (b) AA, (c) AAm
49 

Figures 24 and 25, respectively, show the total accuracy time trends obtained by the original 

FFTBM and the FFTBM improved by signal mirroring. As shown, GID total accuracy is 

significantly oscillating. The reason for this oscillation is that variable P13 with very high values 

of AA contributes much to AAtot. Quantitative assessment does not find such high values 

acceptable. However, as already mentioned, variable P13 requires treatment with the moving 

average to give consistent results. Section 7.4 discusses this in more detail. 

0

0.1

0.2

0.3

0.4

0.5

0.6

0 20 40 60 80 100 120

Time (s)

AAtot (-)

CEA GID GRS IRSN JNES

KAERI KFKI KINS NRI_K NRI_M

PSI TAEK UPC UPI GID(P13)

Figure 24 Total accuracy trend obtained with original FFTBM 

0

0.1

0.2

0.3

0.4

0.5

0.6

0 20 40 60 80 100 120

Time (s)

AAtot (-)

CEA GID GRS IRSN JNES

KAERI KFKI KINS NRI_K NRI_M

PSI TAEK UPC UPI GID(P13)

Figure 25 Total accuracy trend obtained with FFTBM improved by signal mirroring
50 

To show that P13 contributes to oscillations, total accuracy without consideration of P13 was 

calculated (labeled GID(P13) in Figures 24 and 25). The figures show that the GID calculation is 

not still oscillating and that the accuracy is comparable to that of the lower ranked calculations. 

The second calculation judged to be poor is the GRS calculation. The quantitative assessment 

in Phase II of the BEMUSE program also reached this conclusion. The primary reason for this is 

the rod surface temperature calculation, even though the reference calculation of the GRS 

maximum temperature was considered to be good in the framework of BEMUSE Phase III. Not 

considering the rod surface temperatures in the total accuracy analysis, the GRS calculation is 

again comparable to the lower ranked calculations (e.g., those of JNES and TAEK). On the 

other hand, without the consideration of rod surface temperature in the total accuracy analysis, 

the KAERI calculation is comparable to the upper ranked calculations. This means that, in 

principle, the calculations are thermal-hydraulically very close to each other, and the curves are 

bundled together, with no calculation showing clear superiority or significantly deviating in 

accuracy. 

This finding indicates that the rod surface temperature variables are the major contributors to 

inaccuracy. However, these variables are also the most sensitive and uncertain. This makes it 

difficult to judge the uncertainty based on reference calculations, as the uncertainty bounds are 

probably different for different codes and users. In principle, the reference calculations with 

narrow uncertainty bounds should be more accurate, in order to bound the experimental data, 

than those calculations with larger uncertainty bounds. The case described above actually 

happened in the uncertainty analysis performed later in BEMUSE Phase III (Ref. 11). According 

to results from Table 10 of the BEMUSE Phase III report (Ref. 11), the GRS, PSI, and UPC 

have a narrow width of uncertainty band, and the reference calculations for PSI and UPC were 

underestimated with respect to maximum cladding temperature, while the GRS calculation was 

slightly underestimated before 30 seconds and overestimated after 30 seconds. The PSI and 

UPC calculations were the only ones out of 10 calculations that did not envelop the 

experimental data. This is another example of the usefulness of quantitative results in analyzing 

the calculated data. 

Although the accuracy trend is important, the final judgment of the code calculation accuracy is 

normally based on values for the whole transient time intervals. Tables 7 and 8, respectively, 

collect the accuracy measures based on D’Auria AA (extended original FFTBM) and AAm

obtained by mirroring. The tables show that the number of variables with discrepancies more or 

less corresponds to the ranking of calculations. The smallest contributors to inaccuracy in most 

cases were the SG downcomer bottom fluid temperatures. The largest contributors to 

inaccuracy were clad temperatures. The exception is the GID calculation by P13 because the 

signal was not properly prepared in the time domain. In the case of the KINS calculation, the 

largest contributor to inaccuracy is P5, as Figure 8 clearly shows. This variable has also been 

marked as unqualified in the BEMUSE qualitative analysis. In the KAERI calculation, the largest 

contributor to inaccuracy is the variable P7. The KAERI value of VAmin is the smallest among all 

calculations. 

A comparison of AAtot values in Tables 7 and 8 reveals that the absolute values of D’Auria AAtot

and AAtot obtained by mirroring are different. However, the ranking between the calculations 

remains unchanged. The results also show that the differences in accuracy between different 

calculations are small except for the KAERI, GID, and GRS calculations as mentioned above. 

The KAERI calculation is judged as better than the others because of the rod surface 

temperatures. The reason for selecting more surface temperatures in the quantitative analysis 
51 

was to emphasize the three-dimensional rod surface distribution. BEMUSE participants agreed 

on this selection. 

Table 7 New Accuracy Measures Obtained with Extended Original FFTBM

 Time Interval 0–119.5 s 

Calculations AAtot VAmax Variable with VAmax VAmin Variable with VAmin ND 

CEA 0.295 0.039 P8 0.854 P19 5 

GID 0.437 0.037 P4 1.389 P13 9 

GRS 0.398 0.030 P8 1.274 P19 7 

IRSN 0.239 0.033 P2 0.581 P16 3 

JNES 0.320 0.079 P11 0.988 P19 5 

KAERI 0.197 0.028 P2 0.374 P7 0 

KFKI 0.287 0.017 P8 1.149 P19 5 

KINS 0.269 0.037 P8 0.707 P5 5 

NRI-K 0.278 0.027 P11 1.028 P19 4 

NRI-M 0.282 0.019 P8 1.425 P19 4 

PSI 0.265 0.032 P11 0.724 P19 4 

TAEK 0.311 0.037 P2 1.175 P19 5 

UPC 0.277 0.026 P8 1.140 P19 3 

UPI 0.254 0.020 P2 0.940 P19 3 

Table 8 New Accuracy Measures Obtained with FFTBM Improved 

by Signal Mirroring 

 Time Interval 0–119.5 s 

Calculations AAtot VAmax Variable with VAmax VAmin Variable with VAmin ND 

CEA 0.391 0.043 P8 1.196 P19 6 

GID 0.560 0.084 P4 1.514 P13 10 

GRS 0.505 0.027 P8 1.847 P18 7 

IRSN 0.326 0.040 P8 0.661 P16 5 

JNES 0.432 0.073 P8 1.398 P19 7 

KAERI 0.261 0.033 P2 0.547 P7 3 

KFKI 0.384 0.022 P8 1.601 P19 5 

KINS 0.366 0.037 P8 0.981 P5 6 

NRI-K 0.378 0.041 P8 1.440 P19 5 

NRI-M 0.387 0.023 P8 1.948 P19 6 

PSI 0.358 0.049 P8 1.044 P19 5 

TAEK 0.420 0.041 P8 1.584 P19 5 

UPC 0.381 0.028 P8 1.657 P19 7 

UPI 0.341 0.027 P2 1.216 P19 4 

However, an examination of the non-rod temperature variable contribution to the total accuracy 

based on VA reveals that, as shown in Table 9, the KFKI, NRI-K, and UPI calculations are even 

better than the KAERI calculation, while those of NRI-M and PSI are comparable. If considering 

just the rod-temperature contribution, the KAERI calculation obtained the highest accuracy, and 

only the IRSN calculation is comparable. This also explains why the KAERI calculation was 

judged as the best and IRSN as the second best (both with D’Auria AAtot and AAtot obtained by 
52 

mirroring). Nevertheless, the selection of variables is based on subjective judgment, as is, at 

least in part, the selection of the weights (fixed in the analysis). For the uncertainty evaluation, it 

is very important that the reference calculations are correct, as the uncertainty bounds are 

added to the reference calculation. In this respect, the KAERI and IRSN calculations would be 

expected to have the best reference calculations of rod surface temperatures and are the best 

candidates to bound the experimental data when adding uncertainty bounds (Table 10 in the 

BEMUSE Phase III report (Ref. 11) confirms this for both, except that in the case of IRSN, the 

width of the uncertainty band is medium, and therefore, at around 50 seconds the IRSN upper 

bound is slightly lower than the experimental data). As Figure 17 shows, the opposite occurs in 

the case of the primary side total mass where the KAERI calculation is among the least 

accurate during rod heatup. However, the KAERI calculation accurately predicts a small heatup 

of average rod temperatures, while most participants predicted a significant core heatup (see 

Table 8). This establishes the superiority of the KAERI calculation, but it is less important in 

terms of the uncertainty analysis for the maximum rod surface temperature in the hot rod. 

The third parameter treated in the sensitivity analysis was the upper plenum pressure, which 

was not selected for the FFTBM analysis. Nevertheless, the pressure trend is practically the 

same as the trend for hot leg pressure (the data for upper plenum pressure was obtained 

separately for sensitivity analysis). In a comparison of the accuracy of the hot leg pressure, the 

KAERI calculation is comparable to the GID calculation (i.e., the best and the worst calculations 

produce comparable results). This example illustrates that, besides the total accuracy, the 

accuracies of variables selected for the uncertainty analysis are important. The appendix to this 

report shows the obtained values of accuracy measures for the time interval 0–100 seconds. 

The results differ slightly from the results for the whole time interval. However, as KAERI 

provided data for 100 seconds, these data are presented for direct comparison purposes. 

Appendix Tables I-1 through I-14 show accuracy measures obtained by the original FFTBM, 

while Tables II-1 through II-14 show results obtained with FFTBM improved by signal mirroring. 

Each table presents results for one calculation. Shown are the values of AA, VA, I for time shift 

detection, based on AAM (Ref. 1), AMMIN (AAM is AA based on magnitudes, not considering 

phase (Ref. 1)), fraction A0 (contribution of zero frequency component in percentage), AAexp

and AAdif. Dividing AAdif by AAexp produces the value for AA (see Eq. 28). The information for 

WF is not shown, as those data are not useful when all frequency components are used for the 

AA calculation (there is no frequency cut). The time shift was indicated by indexIonly in the case 

of break flow rate (P9) for three calculations (GRS, GID, and KAERI). Figure 26 shows the 

break flow predictions in the first 5 seconds. The experimental break flow is delayed 

0.32 seconds even though the break opens at 0 seconds. The time shift is small, but it has a 

large influence on the accuracy. For the other calculations, the time shift in the break flow is not 

detected because the data were given every 0.5 seconds, causing only a ramp increase of the 

flow instead of a step increase in the flow rate, which is only half of the experimental flow rate 

(JNES, UPC). This example shows the need for a sufficient number of data points to more 

precisely quantify the accuracy when some important phenomena occur rapidly. 
53 

Table 9 Accuracy Contribution from Non-Rod Surface Temperature Variables and 

Contribution from Rod Surface Temperature Variables 

 D’Auria Mirroring 

 sum VA average VA sum VA average VA 

 P1-P14 

P16, P18-

P20 P1-P14

P16, P18-

P20 P1-P14

P16, P18-

P20 P1-P14 

P16, P18-

P20 

CEA 2.93 1.99 0.209 0.497 3.88 2.65 0.277 0.662 

GID 4.34 2.17 0.310 0.543 5.33 3.00 0.380 0.751 

GRS 3.48 3.09 0.249 0.774 3.99 4.27 0.285 1.068 

IRSN 2.59 1.43 0.185 0.358 3.61 1.89 0.258 0.472 

JNES 3.15 2.19 0.225 0.548 4.21 3.00 0.300 0.749 

KAERI 2.29 1.06 0.163 0.265 3.05 1.38 0.218 0.346 

KFKI 1.99 2.66 0.142 0.665 2.61 3.61 0.186 0.903 

KINS 2.87 1.65 0.205 0.414 3.92 2.23 0.280 0.557 

NRI-K 1.91 2.60 0.137 0.650 2.62 3.51 0.187 0.877 

NRI-M 2.32 2.32 0.165 0.581 3.28 3.10 0.234 0.775 

PSI 2.32 2.05 0.166 0.513 3.13 2.77 0.224 0.693 

TAEK 2.45 2.64 0.175 0.661 3.37 3.51 0.241 0.878 

UPC 2.86 1.79 0.204 0.448 3.91 2.47 0.279 0.617 

UPI 2.11 2.06 0.151 0.515 2.97 2.66 0.212 0.666 

Parameter N° 9: BREAK FLOW Cold leg

0

200

400

600

800

012345

Time (s)

Flowrate (kg/s)

CEA GID

GRS IRSN

JNES KAERI

KFKI KINS

NRI-K NRI-M

PSI TAEK

UPC UPI

EXP

Figure 26 Time shift of break flow rates 
54 

7.4 Criterion for Using Moving Average

When trends oscillate greatly (e.g., the pressure drops in P13), special treatment is needed. To 

correctly reproduce the experimental signal by linear interpolation, many points are needed. 

This is achieved by increasing the maximum frequency component of the signal. However, it 

makes no sense to increase the number of points, as some cases have a sampling frequency 

30 times smaller than the calculated data. When many points are used, the main contribution to 

the amplitude spectrum comes from the oscillations (very often noise) in the experimental signal 

for which the calculated data have no information. The correct procedure is therefore to smooth 

the data. Smoothing data removes random variations and shows trends and cyclic components. 

The simplest way to smooth data is by taking averages. This is done by use of the moving 

average of the experimental signal. Mathematically, the moving average is an example of a 

convolution of the input signal with a rectangular pulse having an area of 1. 

Without use of the moving average, AA varies around a certain value. In the presented case for 

P13, the value of AA is close to 1 (see Figure 18) because the calculated values are much 

smaller than the experimental values. Variations in AA are the consequence of inappropriately 

prepared experimental data for the FFTBM analysis. The problem of the oscillatory signal was 

less significant in the past, because the original FFTBM limited the number of data points to 

1,000, and data reduction was needed when this value was exceeded. Thus, data reduction is 

another possibility for use in partially smoothing the signal and thereby increasing the accuracy 

by eliminating some noise. However, as shown by Figure 4(e) in (Ref. 12), the D’Auria AA still 

varies because the moving average was not used. The reason is that, by increasing the time 

interval and not increasing the number of points, the amplitude spectrum changes as the signal 

between two consecutive data points is not a monotonic function (it oscillates). This gives a 

different amplitude spectrum of the experimental and difference signal. When moving average 

was used in the case of the P13 experimental signal, the AA values no longer oscillate in phase 

because of AAexp, as shown in Figure 27. This suggests that the observation of oscillations 

being in phase in the calculated AAs indicates that moving average should be used. Figures 

27(a) and 27(b) show a sudden increase in AA in the CEA and JNES calculations. The reason 

for this increase is the pressure spikes clearly shown in Figure 18. Each spike significantly 

deteriorates the results. Finally, FFTBM was able to detect the deviation in the NRI-K calculation 

at the end of the transient. 

Another important finding is that the mismatch between the experimental data and the 

calculations for variable P13 is present from the very beginning of the transient, as shown in 

Figure 28. Only the GID calculation reproduced the frequency of oscillations in the first second. 

However, because the peaks were too high, the calculation was not very accurate. Use of 

moving average removes the large oscillations from the experimental signal (EXP(ma)), while in 

the GID(ma) calculation, the oscillations still remain in the beginning of the transient. Later (at 

approximately 15 seconds), the pressure drop stabilizes and the values oscillate around their 

mean values. This means that the transient related to the pressure drop has more or less 

ended. Table 10 shows that the mean values are well below the experimental value and justify 

an AA above 0.5, as shown in Figure 27. 

Table 10 Average SG Pressure Drops in Time Interval 15–120 Seconds 

Average Pressure Drop (kPa) 

EXP CEA GID GRS IRSN JNES KAERI KFKI KINS NRI-K NRI-M PSI TAEK UPC UPI 

13.07 2.65 1.46 2.61 2.25 2.93 0.65 3.44 0.37 1.05 4.34 4.33 1.75 0.67 3.23
55 

0.00

0.50

1.00

1.50

2.00

0 20 40 60 80 100 120

Time (s)

AA (-)

CEA GID(ma) GRS IRSN JNES KAERI KFKI

KINS NRI-K NRI-M PSI TAEK UPC UPI

(a)

0.00

0.50

1.00

1.50

2.00

0 20 40 60 80 100 120

Time (s)

AAm (-)

CEA GID(ma) GRS IRSN JNES KAERI KFKI

KINS NRI-K NRI-M PSI TAEK UPC UPI

(b)

Figure 27 Accuracy trends for SG pressure drop—UT PS using moving average for 

the experimental signal 

Parameter N° 13: PRESSURE DROPS SG Pressure drop

-50

50

150

250

0.0 1.0 2.0 3.0 4.0 5.0

Time (s)

p (kPa)

CEA GID(ma)

GRS IRSN

JNES KAERI

KFKI KINS

NRI-K NRI-M

PSI TAEK

UPC UPI

EXP(ma) EXP

Figure 28 Time trends for SG pressure drop—UT PS for initial 5 seconds 
56 

7.5 Comparison of Fast Fourier Transform Based Method and Automated Code 

Assessment Program

Tables 11 and 12 show the comparison of FFTBM and Automated Code Assessment Program 

(ACAP) (Ref. 13) figures of merit for the pressurizer pressure (P4) and the hot rod temperature 

(P16). This comparison was made for the independent assessment that FFTBM provides for 

consistent figures of merit. The calculations are sorted according to AAm in ascending manner. 

For pressurizer pressure, it can be seen that AAm, AA, mean square error (MSE), and crosscorrelation coefficient (XCC) figures of merit agree well. The only difference is that MSE and 

XCC indicate that all calculations of pressurizer pressure are very good, while FFTBM shows 

that some are not so accurate and some do not even fulfill the original FFTBM primary pressure 

criterion. As the pressure criterion was developed without consideration of the edge effect, care 

must be taken in its use, as indicated by the ACAP results. 

Finally, D’Auria fast Fourier transform (DFFT) and continuous wavelet transform (CWT) figures 

of merit do not help much in this case. For hot rod temperature, AAm, AA, MSE, and XCC 

figures of merit agree well. The XCC figure of merit is in especially good agreement with the 

AAm. When comparing the UPC and TAEK calculations, FFTBM slightly favors the UPC 

calculation, while ACAP gives comparable values. The qualitative analysis of dryout occurence 

reported in Table 13 of the BEMUSE Phase II Report (Ref. 10) showed, that the UPC 

calculation receives three excellent and one minimal mark, while the TAEK calculation receives 

two excellent, one reasonable, and one minimal mark. One parameter representing dryout 

occurrence is peak cladding temperature and for it the UPC calculation is qualitatively judged 

better than TAEK calculation. These BEMUSE results support the FFTBM judgments for 

cladding temperature. Examination of AAm in Figure 20(c) shows that, in the initial period of 40 

seconds, the UPC calculation is significantly better because of the TAEK calculation’s large 

overprediction of cladding temperature. 

Table 11 Comparison of FFTBM and ACAP Figures of Merit for Pressurizer Pressure (P4) 

in Time Interval 0–119.5 Seconds 

P4 FFTBM ACAP 

Calculation AAm AA DFFT MSE XCC CWT 

GID 0.076 0.034 0.194 1.000 1.000 0.154 

GRS (110 s) 0.079 0.032 0.132 1.000 0.999 0.008 

NRI-M 0.082 0.034 0.173 1.000 0.999 0.116 

KFKI 0.085 0.036 0.223 1.000 0.999 0.059 

KINS 0.111 0.047 0.173 0.999 0.999 0.148 

NRI-K 0.129 0.051 0.194 1.000 0.999 0.179 

KAERI (100 s) 0.159 0.062 0.168 0.999 0.999 0.008 

TAEK 0.167 0.070 0.134 0.998 0.997 0.126 

PSI 0.230 0.093 0.082 0.998 0.993 0.006 

CEA 0.237 0.096 0.129 0.998 0.994 0.140 

IRSN 0.244 0.097 0.128 0.998 0.995 0.220 

UPI 0.275 0.119 0.110 0.997 0.992 0.069 

JNES 0.305 0.123 0.089 0.996 0.983 0.091 

UPC 0.458 0.186 0.096 0.991 0.972 0.053 

AA—average amplitude, m—mirroring, DFFT—D’Auria fast Fourier transform, MSE—mean square error, CWT—

continuous wavelet transform, XCC—cross-correlation coefficient 
57 

Table 12 Comparison of FFTBM and ACAP Figures of Merit for Rod Surface Temperature 

in the Hot Rod in the Middle of the Core (P16) in Time Interval 0–119.5 Seconds 

P16 FFTBM ACAP 

Calculation AAm AA DFFT MSE XCC CWT 

KAERI (100 s) 0.313 0.285 0.245 0.989 0.992 0.020 

NRI-M 0.375 0.337 0.197 0.984 0.987 0.193 

UPI 0.388 0.347 0.228 0.988 0.973 0.171 

UPC 0.409 0.356 0.206 0.982 0.960 0.106 

KFKI 0.442 0.374 0.203 0.983 0.968 0.006 

KINS 0.451 0.396 0.182 0.980 0.972 0.010 

GID 0.452 0.391 0.208 0.982 0.962 0.055 

JNES 0.488 0.429 0.181 0.968 0.962 0.055 

TAEK 0.504 0.429 0.207 0.981 0.967 0.004 

IRSN 0.555 0.487 0.158 0.948 0.883 0.043 

NRI-K 0.578 0.511 0.150 0.938 0.853 0.049 

PSI 0.600 0.515 0.152 0.940 0.832 0.000 

CEA 0.616 0.544 0.151 0.929 0.841 0.055 

GRS (110 s) 0.708 0.620 0.149 0.901 0.780 0.000 

AA—average amplitude, m—mirroring, DFFT—D’Auria fast Fourier transform, MSE—mean square error, CWT—

continuous wavelet transform, XCC—cross-correlation coefficient

7.6 Correction Factors

The acceptability criteria for the proposed FFTBM improved by signal mirroring need to be 

defined based on several calculations. The obtained results suggest slightly higher acceptability 

limits than for the original FFTBM. Section 4.4 presents the criteria for the original FFTBM. The 

criteria for the improved FFTBM do not currently exist, but, for now, the following approach can 

be taken: correct the AA, calculated with the original FFTBM, using the correction factors 

obtained by comparing AA and AAm for the whole time interval (see Table 13), and use the 

traditional acceptability limits of the original FFTBM method. 

Using this method, no special and more restrictive criterion for the primary pressure would be 

needed (this criterion was difficult to satisfy for transients where the edge was not present). The 

restrictive pressure criterion may have been set based on pressure trends during small-break 

LOCAs in facilities simulating typical PWRs (high initial pressure and large pressure drop and, 

therefore, high edge). When tests of different facilities were simulated, there were difficulties in 

satisfying the primary pressure criterion. The first example is the accuracy quantification of four 

standard problem exercises (SPEs) organized by the International Atomic Energy Agency 

(IAEA) (Ref. 14). Among the conclusions of this study, which considers only the PS pressure, 

was that in the case of SPE-3, the calculation is clearly unacceptable (the AA was 0.31) and 

that more complex transients lead to worse results than does the simple transient. As the report 

on this IAEA study (Ref. 14) shows no plots, no further conclusion can be reached except that 

the pressure drop (edge) is smaller than in a typical PWR. In other words, the initial pressure in 

this test is lower than in the typical PWR test. By lowering the pressure edge, the values of AA 

increase. This is better illustrated in the recent application of FFTBM to heavy-water reactors. In 

the study (Ref. 12), all participants satisfied the acceptance criterion for the total accuracy 

K<0.4, while the primary pressure criterion was not met. In the blind accuracy calculation, the 

AA value for primary pressure was 0.117 in the best calculation. The header 7 pressure with 

initial pressure around 10 MPa was selected as a variable representing the primary pressure. In 

the open accuracy analysis, a representative from Italy proposed the header 6 pressure. The 
58 

initial value of this pressure was around 12 MPa. For most of the participants, the value of AA 

was below 0.1, primarily because of the higher pressure edge effect (the best AA was 0.074). 

Table 13 shows that the largest correction factors are for P1, P3, and P4 (all pressures) and 

P11 (flow integral). For all other variables, the difference was ±20 percent. 

Table 13 Correction Factors for Accuracy Judgment by Original FFTBM 

for LOFT L2-5 Test

Variable/ 

Calc. P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P16 P18 P19 P20

CEA 1.4 0.9 1.8 2.1 1.2 1.3 0.9 0.9 0.9 0.9 1.4 1.0 0.9 0.8 0.9 1.1 1.2 1.2 

GID 1.5 1.1 1.2 1.9 1.3 1.2 1.3 0.8 0.8 0.9 1.0 0.9 0.9 0.8 1.0 1.1 1.2 1.2 

GRSa 1.5 0.9 1.8 2.0 1.3 1.3 0.8 0.8 0.8 0.9 1.1 0.7 0.8 0.8 0.9 1.2 1.2 1.2 

IRSN 1.5 1.0 1.8 2.1 1.3 1.3 1.2 0.9 0.9 0.9 1.4 1.0 0.9 0.8 0.9 1.1 1.2 1.2 

JNES 1.6 0.5 1.8 2.1 1.1 1.3 1.2 0.6 0.9 0.9 1.5 1.0 0.9 0.8 0.9 1.1 1.2 1.2 

KAERIb 1.6 1.0 1.6 2.1 1.1 1.3 1.2 0.6 0.8 0.9 1.1 1.0 0.9 0.8 0.9 1.1 1.2 1.1 

KFKI 1.6 0.8 1.9 1.9 1.2 1.3 1.3 1.1 0.8 0.9 1.1 0.9 0.9 0.8 1.0 1.1 1.2 1.2 

KINS 1.5 1.0 1.6 2.0 1.2 1.2 1.2 0.8 0.8 0.9 1.2 1.0 0.9 0.8 0.9 1.1 1.2 1.2 

NRI-1 1.6 1.1 1.2 2.1 1.3 1.3 1.2 0.8 0.8 0.8 1.7 1.2 0.9 0.8 0.9 1.1 1.2 1.2 

NRI-2 1.4 0.9 1.9 2.0 1.3 1.3 1.2 1.0 0.8 0.9 1.6 1.2 0.9 0.8 0.9 1.1 1.1 1.2 

PSI 1.5 0.9 2.0 2.1 1.3 1.2 1.2 0.8 0.8 0.9 2.0 0.9 0.9 0.8 1.0 1.1 1.2 1.2 

TAEK 1.5 0.9 1.9 2.0 0.9 1.2 1.2 0.7 0.9 0.9 1.3 1.3 0.9 0.8 1.0 1.1 1.1 1.2 

UPC 1.5 0.9 1.2 2.0 1.2 1.3 1.2 0.9 0.9 0.8 1.5 1.4 0.8 0.8 1.0 1.1 1.2 1.2 

UPI 1.5 1.1 1.7 1.9 1.2 1.3 1.2 0.8 0.9 0.9 1.6 1.3 0.9 0.8 0.9 1.1 1.1 1.2 

average 1.5 0.9 1.7 2.0 1.2 1.3 1.2 0.8 0.8 0.9 1.4 1.1 0.9 0.8 0.9 1.1 1.2 1.2 

a For time interval 0–110 s 

b

 For time interval 0–100 s

7.7 Discussion of the Results

The FFTBM improved by signal mirroring was used in the LOFT L2-5 test to demonstrate the 

consistent judgment of code calculations. For the LOFT L2-5 test calculation, the original 

FFTBM and the FFTBM improved by signal mirroring reach similar conclusions. Nevertheless, 

this holds only for the time interval of the whole transient duration. A time-dependent analysis 

clearly shows the differences between AA and AAm. In the analysis using the original FFTBM, 

the edge effect has too much impact on the results, as it first increases and then decreases 

value (triangular shape), while in the analysis using the FFTBM improved by signal mirroring, 

the edge effect is eliminated. Generally, the AA increases when there are discrepancies. Once 

the calculated results begin to approach the experimental values, the AA slowly decreases. 

However, if the experimental signal approaches steady-state conditions while the calculated 

signal still changes its shape, the AA does not decrease. 

The quantitative analysis performed in this study supports most of the conclusions reached 

independently in the BEMUSE program. First, the BEMUSE qualitative conclusions are in 

agreement with the quantitative judgment. FFTBM judged both calculated variables designated 

as unqualified in the BEMUSE qualitative analysis (i.e., the SG secondary pressure of the GID 

calculation and the core inlet liquid temperature of the KINS calculation), to be very poor 

calculations. 
59 

The quantitative results also confirm and clarify the conclusions for RTA from the qualitative 

analysis in BEMUSE Phase II. A good example is the comparison of the UPC and TAEK 

calculations of hot rod temperature (P16). FFTBM slightly favors the UPC calculation, while 

ACAP shows comparable values. It also becomes clear that extending FFTBM with the 

capability to calculate accuracy as a function of time is indispensable in such studies. The 

analyst can see the contribution of each discrepancy to the AA. In the case of hot rod 

temperature, the most important contributor to AA is the time of core quench. When not 

calculating accuracy as a function of time, the reasons for discrepancies can be misleading if 

the analyst looks only at the results for the whole time interval. For example, the BEMUSE 

Phase III Report (Ref. 11) mentions that GRS, KAERI, KINS, and UNIPI had good reference 

calculations of maximum cladding temperatures. However, the quantitative analysis results of 

BEMUSE Phase II (Ref. 10] seem to contradict that statement, as the GRS prediction of hot rod 

temperature was judged to be the least accurate. However, the AAm for the GRS calculation in 

the present study shows that the GRS calculation of hot rod temperature was in good 

agreement in the time period before quench, while later agreement declined because the time of 

core quench was not accurately predicted. Also, in BEMUSE Phase III, GRS provided a 

calculation with improved prediction of core quench time. 

The results also show that the average contribution to AA is 55 percent for non-rod surface 

temperature variables and 45 percent for rod surface temperature variables. This finding 

indicates that the rod surface temperature variables are the major contributors to inaccuracy. 

However, these are also the most sensitive and uncertain variables. Therefore, it is difficult to 

judge the uncertainty based on reference calculations, as the uncertainty bounds may differ for 

different codes and users. In principle, the reference calculations with narrow uncertainty 

bounds should be more accurate (in order to bound the experimental data) than the calculations 

with larger uncertainty bounds. The results of BEMUSE Phase III confirm this hypothesis. 

Finally, the reason for selecting more surface temperatures in the quantitative analysis was to 

emphasize the three-dimensional rod surface distribution. BEMUSE participants agreed to this 

selection. The KAERI calculation, as well as those of GRS, KINS and UPI, for hot rod 

calculation were judged as good reference calculations in BEMUSE Phase III. The BEMUSE 

qualitative analysis even found that the maximum differences with respect to the experimental 

value of 1,078 K were obtained by NRI-M (1,250 K) and by KAERI (980 K). However, the KAERI 

calculation better predicted a small heatup of average rod temperatures than the calculations of 

most other participants, which predicted a significant core heatup (see Table 8). This 

established the superiority of the KAERI calculation, but it is less important in the uncertainty 

analysis for the maximum rod surface temperature in the hot rod only. As noted in Figure 17, for 

the primary-side total mass, the KAERI calculation was among the least accurate during rod 

heatup. This example illustrates that both qualitative and quantitative analyses are needed for 

objective judgment about the quality of a calculation. 

61 

8. CONCLUSIONS

Two decades ago, under the aegis of CSNI, work began on the development of methods for 

code accuracy quantification. In an attempt to develop the figure of merit to characterize the 

accuracy of code prediction and to compare two or more calculations, the original FFTBM was 

proposed. The FFTBM became widely used for code accuracy quantification of PS thermalhydraulic codes and was also included in the ACAP tool developed for U.S. Nuclear Regulatory 

Commission code consolidation purposes. However, a new figure of merit was proposed based 

on AA and WF. Recently, it was observed that the judged accuracy obtained by the original 

FFTBM was, in general, unreasonably high for monotonic variable trends (pressure blowdown, 

break flow integral) in comparison with other trends. Similarly, in calculating the time-dependent 

accuracy with the extended original FFTBM, the accuracy changed greatly during a fast 

increase or decrease in variables. The reason for this was identified as the nonphysical edge 

effect (influence of difference between the first and the last data point), which produces several 

harmonic amplitudes. Signal mirroring was proposed to eliminate the edge effect, which is a 

deficiency in the original FFTBM. 

The FFTBM improved by signal mirroring was applied to the large-break LOCA test LOFT L2-5. 

The results show that the improved FFTBM judges the accuracy of variables reliably and 

consistently. For the LOFT L2-5 test, the results for the entire transient time interval obtained by 

the original FFTBM and by the FFTBM improved by signal mirroring qualitatively agree except 

for monotonic variables. The results for time-dependent accuracy with the FFTBM improved by 

signal mirroring show that the analyst can now find accurate information on how the 

discrepancies influence the accuracy. This information is needed to verify and clarify the 

conclusions made in the BEMUSE program. 

The results obtained with FFTBM improved by signal mirroring have been useful in systematic 

evaluation of several code calculations. The method eliminates the nonphysical edge influence 

on accuracy, detects the time shifts between calculated and experimental data, detects the 

unqualified variables from the qualitative analysis, helps to independently verify the qualitative 

conclusions, and aids the analyst in judging the quality of code predictions. The method also 

contributes to uncertainty analysis, as a good reference calculation is as important as the 

determination of uncertainty bounds. The capability to calculate time-dependent accuracy helps 

the analyst to identify the variables that need further treatment in the time domain before the 

signal is quantitatively assessed. These findings lead to the conclusion that the FFTBM 

improved by signal mirroring realistically judges accuracy and is a generally applicable tool for 

quantitative assessment. 

63 

9. REFERENCES

1. Prošek, A., and Leskovar, M., “Application of FFTBM to Severe Accidents,” Int. Conf. 

Nuclear Energy for New Europe 2005, Bled, Slovenia, September 5–8, 2005. 

2. Prošek, A., and Mavko, B., “Quantitative assessment of time trends: Influence of time 

window selection,” Proc. 5th Int. Conf. on Nuclear Option in Countries with Small and 

Medium Electricity Grids, Dubrovnik, Croatia, May 16–20, 2004. 

3. Press, W.H., Teukolsky, S.A., Vetterling, W.T., and Flannery, B.P., Numerical Recipes in 

Fortran 77: The Art of Scientific Computing, Vol. 1, Chapter 12: “Fast Fourier transform,” 

Second Edition, 2001, ISBN 0-521-43064-X. 

4. Prošek, A., D’Auria, F., and Mavko, B., “Review of quantitative accuracy assessments 

with fast Fourier transform based method (FFTBM),” Nuclear Engineering and Design, 

217 (1–2), 179–206, 2002. 

5. D’Auria, F., Leonardi, M., and Pochard, R., “Methodology for the evaluation of thermalhydraulic codes accuracy,” Proc. Int. Conf. on New Trends in Nuclear System 

Thermohydraulics, Pisa, May 30–June 2, 1994. 

6. Aksan, S.N., Bessette, D., Brittain, I., D’Auria, F., Gruber, P., Holmström, H.L.O., 

Landry, R., Naff, S., Pochard, R., Preusche, G., Reocreux, M., Sandervag, O., 

Städtke, H., Wolfert, K., and Zuber, N., “CSNI Code Validation Matrix of ThermoHydraulic Codes for LWR LOCA and Transients,” CSNI Report No. 132, 1987. 

7. Prošek, A., “Accuracy assessment of IAEA-SPE-4 predictions with improved FFTBM,” 

presented at 2002 Winter Meeting of the American Nuclear Society, November 17–21, 

2002, Washington. Trans. Am. Nucl. Soc., Vol. 87, 199–201, 2002. 

8. Prošek, A., and Leskovar, M., “Improved FFTBM by signal mirroring,” IJS Report, Jožef 

Stefan Institute, IJS-DP-9336, Ljubljana, 2006. 

9. Organisation for Economic Co-operation and Development/Nuclear Energy Agency 

(OECD/NEA), “CSNI International Standard Problems (ISP), Brief descriptions (1975–

1999), NEA/CSNI/R(2000)5, OECD/NEA, Paris, France, 2000. 

10. BEMUSE Phase II Report, “Re-Analysis of the ISP-13 Exercise, Post Test Analysis of the 

LOFT L2-5 Test Calculation,” NEA/CSNI/R(2006)2, OECD/NEA, Paris, France, 2006, 

<http://www.nea.fr/html/nsd/docs/2006/csni-r2006-2.pdf>. 

11. BEMUSE Phase III Report, “Uncertainty and Sensitivity Analysis of the LOFT L2-5 Test,” 

NEA/CSNI/R(2007)4, OECD/NEA, Paris, France, 2007, 

<http://www.nea.fr/html/nsd/docs/2007/csni-r2007-4.pdf>. 

12. Prošek, A., D’Auria, F., Richards, D.J., and Mavko, B., “Quantitative assessment of 

thermal-hydraulic codes used for heavy water reactor calculations,” Nuclear Engineering 

and Design 236, 295–308, 2006. 

13. Kunz, R.F., Kasamala, G.F., Mahaffy, J.H., and Murray, C.J., “On the automated 

assessment of nuclear reactor systems code accuracy,” Nuclear Engineering and 

Design, 211, 245–272, 2002. 

14. D’Auria, F., Eramo, A., Frogheri, M., and Galassi, G.M., “Accuracy quantification in SPE1 to SPE-4,” organized by IAEA, Proc. Int. Conf. on Nuclear Engineering (ICONE-4), 

New Orleans, Louisiana, Vol. 3, 1996.

A-1 

APPENDIX A 

ACCURACY MEASURES FOR L2-5 TEST 

IN TIME INTERVAL 0–100 SECONDS 
A-2 

CONTENTS

I. ORIGINAL FAST FOURIER TRANSFORM BASED METHOD........................................... A-4 

II. FAST FOURIER TRANSFORM BASED METHOD IMPROVED BY SIGNAL 

 MIRRORING .....................................................................................................................A-11 

Tables

Table I-1 Accuracy Measures for CEA Calculation with Original FFTBM. ............................... A-4 

Table I-2 Accuracy Measures for GID Calculation with Original FFTBM. ................................ A-4 

Table I-3 Accuracy Measures for GRS Calculation with Original FFTBM................................ A-5 

Table I-4 Accuracy Measures for IRSN Calculation with Original FFTBM............................... A-5 

Table I-5 Accuracy Measures for JNES Calculation with Original FFTBM. ............................. A-6 

Table I-6 Accuracy Measures for KAERI Calculation with Original FFTBM............................. A-6 

Table I-7 Accuracy Measures for KFKI Calculation with Original FFTBM. .............................. A-7 

Table I-8 Accuracy Measures for KINS Calculation with Original FFTBM. .............................. A-7 

Table I-9 Accuracy Measures for NRI-K Calculation with Original FFTBM.............................. A-8 

Table I-10 Accuracy Measures for NRI-M Calculation with Original FFTBM. .......................... A-8 

Table I-11 Accuracy Measures for PSI Calculation with Original FFTBM................................ A-9 

Table I-12 Accuracy Measures for TAEK Calculation with Original FFTBM. ........................... A-9 

Table I-13 Accuracy Measures for UPC Calculation with Original FFTBM............................ A-10 

Table I-14 Accuracy Measures for UPI Calculation with Original FFTBM. ............................ A-10 

Table II-1 Accuracy Measures for CEA Calculation with Improved FFTBM........................... A-11 

Table II-2 Accuracy Measures for GID Calculation with Improved FFTBM............................ A-11 

Table II-3 Accuracy Measures for GRS Calculation with Improved FFTBM. ......................... A-12 

Table II-4 Accuracy Measures for IRSN Calculation with Improved FFTBM. ........................ A-12 

Table II-5 Accuracy Measures for JNES Calculation with Improved FFTBM......................... A-13 

Table II-6 Accuracy Measures for KAERI Calculation with Improved FFTBM. ...................... A-13 

Table II-7 Accuracy Measures for KFKI Calculation with Improved FFTBM.......................... A-14 

Table II-8 Accuracy Measures for KINS Calculation with Improved FFTBM.......................... A-14 

Table II-9 Accuracy Measures for NRI-K Calculation with Improved FFTBM. ....................... A-15 

Table II-10 Accuracy Measures for NRI-M Calculation with Improved FFTBM...................... A-15 

Table II-11 Accuracy Measures for PSI Calculation with Improved FFTBM. ......................... A-16 

Table II-12 Accuracy Measures for TAEK Calculation with Improved FFTBM....................... A-16 

Table II-13 Accuracy Measures for UPC Calculation with Improved FFTBM. ....................... A-17 

Table II-14 Accuracy Measures for UPI Calculation with Improved FFTBM.......................... A-17 
A-3 

ABBREVIATIONS 

A0 contribution of zero frequency amplitude component in percentage 

AA average amplitude 

AMMIN AA based on magnitudes, not considering phase 

CEA Commissariat à l’Energie Atomique 

EDO Experimental Design Office 

GID EDO “Gidropress” 

GRS Gesellschaft für Anlagen- und Reaktor Sicherheit (GRS) mbH 

IRSN Institut de Radioprotection et de Sûreté Nucléaire 

JNES Japan Nuclear Energy Safety 

KAERI Korea Atomic Energy Research Institute 

KFKI Központi Fizikai Kutató Intézet 

KINS Korea Institute of Nuclear Safety 

NRI Nuclear Research Institute 

PSI Paul Scherrer Institute 

TAEK Türkiye Atom Enerjisi Kurumu 

UPC Universitat Politècnica de Catalunya 

UPI University of Pisa 

VA variable accuracy 

Subscripts 

dif difference 

exp experimental 
A-4 

I. ORIGINAL FAST FOURIER TRANSFORM BASED METHOD 

Table I-1 Accuracy Measures for CEA Calculation with Original FFTBM 

Calculation Window 0–100 s 

CEA Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.125317 0.138302 0.391783 0.090041 0.011398 0.092367 0.011575

2 P2 0.076823 0.055957 0.223395 0.062795 0.265155 0.024468 0.00188

3 P3 0.146882 0.162102 0.344385 0.109256 0.294715 0.039266 0.005767

4 P4 0.093479 0.103165 0.451952 0.064381 0.127436 0.127714 0.011939

5 P5 0.063593 0.1078 0.508101 0.042168 0.018878 3.004864 0.191089

6 P6 0.210502 0.356834 0.511556 0.139262 0.105666 3.172731 0.667867

7 P7 0.352581 0.59768 0.172971 0.300588 0.188643 3.202652 1.129194

8 P8 0.024746 0.041948 0.654144 0.01496 0.37236 2.132043 0.052759

9 P9 0.403267 0.089011 0.496651 0.269446 0.030703 4.072386 1.64226

10 P10 0.865179 0.190965 0.900228 0.455303 0.000864 1.146323 0.991775

11 P11 0.196551 0.140562 0.089063 0.180477 0.256126 20.907 4.109296

12 P12 0.355552 0.254271 0.053528 0.337487 0.290154 30.52846 10.85444

13 P13 0.955056 0.258234 0.4864 0.642529 0.046794 0.810833 0.774391

14 P14 1.439387 0.389191 0.742958 0.82583 0.03381 0.827981 1.191786

15 P16 0.536187 0.639085 1.274555 0.235733 0.110448 7.176047 3.847703

16 P18 0.499013 0.594777 0.303958 0.382691 0.122759 3.222118 1.607879

17 P19 0.716008 0.853415 0.309828 0.546643 0.106172 3.241802 2.321158

18 P20 0.224976 0.26815 0.539994 0.146089 0.013018 3.235821 0.727982

 total 0.291192 0.203854 

Table I-2 Accuracy Measures for GID Calculation with Original FFTBM 

Calculation Window 0–100 s 

GID Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.082393 0.090931 0.42856 0.057676 0.099276 0.092367 0.00761

2 P2 0.587521 0.427943 0.142306 0.514329 0.060096 0.024468 0.014376

3 P3 0.179549 0.198153 0.193935 0.150384 0.078299 0.039266 0.00705

4 P4 0.032436 0.035797 0.183942 0.027397 0.133646 0.127714 0.004143

5 P5 0.076883 0.130329 0.470494 0.052284 0.074851 3.004864 0.231023

6 P6 0.197552 0.334882 0.610214 0.122687 0.030421 3.172731 0.626781

7 P7 0.28444 0.482171 0.728136 0.164594 0.0565 3.202652 0.910964

8 P8 NA NA NA NA NA NA NA 

9 P9 0.669672 0.147812 3.68698 0.142879 0.017727 4.072386 2.727164

10 P10 0.645398 0.142455 0.552942 0.415597 0.00064 1.146323 0.739835

11 P11 0.170088 0.121637 0.008555 0.168645 0.193647 20.907 3.556032

12 P12 0.51294 0.366826 0.085147 0.472692 0.266781 30.52846 15.65928

13 P13 4.705256 1.272237 0.508289 3.119599 0.012783 0.810833 3.815177

14 P14 0.903181 0.244208 0.957425 0.461413 0.025216 0.827981 0.747817

15 P16 0.387229 0.461541 0.615652 0.239674 0.052967 7.176047 2.778777

16 P18 0.550875 0.656592 0.478891 0.372492 0.057033 3.222118 1.774985

17 P19 0.681829 0.812676 0.327268 0.513708 0.129185 3.241802 2.210353

18 P20 0.541346 0.645234 0.389418 0.389621 0.16777 3.235821 1.7517 

 total 0.412993 0.347876 
A-5 

Table I-3 Accuracy Measures for GRS Calculation with Original FFTBM

Calculation Window 0–100 s 

GRS Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.052953 0.05844 0.662973 0.031843 0.009996 0.092367 0.004891

2 P2 0.050301 0.036639 0.22727 0.040986 0.19617 0.024468 0.001231

3 P3 0.046279 0.051074 0.298897 0.035629 0.088406 0.039266 0.001817

4 P4 0.033329 0.036782 0.50958 0.022078 0.184502 0.127714 0.004257

5 P5 0.08395 0.142308 0.393803 0.060231 0.164754 3.004864 0.252257

6 P6 0.311528 0.528089 0.593336 0.195519 0.072503 3.172731 0.988394

7 P7 0.494108 0.83759 0.375228 0.359292 0.117685 3.202652 1.582455

8 P8 0.014927 0.025303 0.76339 0.008465 0.11638 2.132043 0.031824

9 P9 1.166972 0.257578 2.166693 0.368514 0.024264 4.072386 4.752361

10 P10 2.360307 0.520975 0.512743 1.560283 0.007589 1.146323 2.705674

11 P11 0.061591 0.044046 0.024338 0.060128 0.250394 20.907 1.287683

12 P12 0.401236 0.286941 0.112048 0.360808 0.142937 30.52846 12.24911

13 P13 0.634571 0.171579 0.222375 0.51913 0.075166 0.810833 0.514531

14 P14 0.948934 0.256579 0.501136 0.632144 0.026327 0.827981 0.7857 

15 P16 0.603574 0.719404 0.98136 0.304626 0.103845 7.176047 4.331275

16 P18 1.100491 1.311682 0.284809 0.85654 0.228864 3.222118 3.54591

17 P19 1.050651 1.252278 0.256394 0.836243 0.287316 3.241802 3.406003

18 P20 0.347371 0.414034 1.213124 0.156959 0.106028 3.235821 1.124029

 total 0.386185 0.266193 

Table I-4 Accuracy Measures for IRSN Calculation with Original FFTBM 

Calculation Window 0–100 s 

IRSN Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.130025 0.143498 0.484573 0.087584 0.003989 0.092367 0.01201

2 P2 0.046897 0.034159 0.436979 0.032636 0.080019 0.024468 0.001147

3 P3 0.139209 0.153634 0.284899 0.108342 0.316628 0.039266 0.005466

4 P4 0.094489 0.10428 0.473322 0.064133 0.129615 0.127714 0.012068

5 P5 0.1215 0.205962 0.644493 0.073883 0.099181 3.004864 0.365091

6 P6 0.127369 0.21591 0.580425 0.080592 0.027535 3.172731 0.404107

7 P7 0.237121 0.401957 0.733562 0.136782 0.119917 3.202652 0.759415

8 P8 0.024462 0.041467 1.08553 0.011729 0.277444 2.132043 0.052154

9 P9 0.577665 0.127504 0.581245 0.365323 0.020215 4.072386 2.352474

10 P10 0.848922 0.187377 0.914811 0.443345 0.000672 1.146323 0.973139

11 P11 0.219163 0.156733 0.157437 0.189352 0.243622 20.907 4.582038

12 P12 0.375776 0.268734 0.102863 0.340727 0.219851 30.52846 11.47185

13 P13 0.815834 0.22059 0.284124 0.635323 0.058437 0.810833 0.661505

14 P14 1.314795 0.355503 1.040841 0.644242 0.007083 0.827981 1.088626

15 P16 0.483709 0.576536 1.423791 0.199567 0.096902 7.176047 3.471116

16 P18 0.319801 0.381172 0.576156 0.202899 0.00901 3.222118 1.030435

17 P19 0.367894 0.438495 0.677712 0.219283 0.025382 3.241802 1.19264

18 P20 0.245802 0.292973 0.6941 0.145093 0.001653 3.235821 0.795372

 total 0.239249 0.146775 
A-6 

Table I-5 Accuracy Measures for JNES Calculation with Original FFTBM 

Calculation Window 0–100 s 

JNES Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.122946 0.135685 0.48258 0.082927 0.085675 0.092367 0.011356

2 P2 0.335039 0.244038 0.030705 0.325058 0.235355 0.024468 0.008198

3 P3 0.167406 0.184752 0.383059 0.12104 0.218683 0.039266 0.006573

4 P4 0.119752 0.13216 0.484763 0.080654 0.135031 0.127714 0.015294

5 P5 0.198729 0.336876 0.519914 0.13075 0.003326 3.004864 0.597153

6 P6 0.187297 0.317497 0.559091 0.120132 0.039849 3.172731 0.594243

7 P7 0.236295 0.400556 0.569417 0.150562 0.094883 3.202652 0.75677

8 P8 0.054401 0.092218 0.241136 0.043832 0.438121 2.132043 0.115985

9 P9 0.775679 0.171211 0.399577 0.554224 0.020599 4.072386 3.158866

10 P10 1.702814 0.375851 1.312278 0.736422 0.030234 1.146323 1.951974

11 P11 0.105292 0.075299 0.256486 0.083799 0.015916 20.907 2.201348

12 P12 0.32024 0.229018 0.066186 0.300361 0.28745 30.52846 9.776445

13 P13 0.641787 0.173531 0.202941 0.533515 0.076865 0.810833 0.520383

14 P14 0.922596 0.249457 0.579576 0.584078 0.078393 0.827981 0.763893

15 P16 0.429385 0.511787 0.601596 0.268098 0.093347 7.176047 3.081285

16 P18 0.483976 0.576854 0.38722 0.348882 0.161953 3.222118 1.559428

17 P19 0.839236 1.000292 0.359334 0.617388 0.205426 3.241802 2.720638

18 P20 0.460023 0.548304 0.381705 0.332938 0.207316 3.235821 1.488551

 total 0.319744 0.224198 

Table I-6 Accuracy Measures for KAERI Calculation with Original FFTBM 

Calculation Window 0–100 s 

KAERI Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.087288 0.096332 0.391718 0.062719 0.08944 0.092367 0.008063

2 P2 0.037958 0.027648 0.448465 0.026205 0.033889 0.024468 0.000929

3 P3 0.106203 0.117207 0.324808 0.080165 0.169578 0.039266 0.00417

4 P4 0.062419 0.068887 0.334943 0.046758 0.193971 0.127714 0.007972

5 P5 0.090249 0.152986 0.440381 0.062656 0.135224 3.004864 0.271186

6 P6 0.104129 0.176514 0.531153 0.068007 0.092258 3.172731 0.330372

7 P7 0.220699 0.37412 0.652748 0.133535 0.160342 3.202652 0.706822

8 P8 0.035996 0.061019 0.823863 0.019736 0.082793 2.132043 0.076745

9 P9 0.660296 0.145743 2.780263 0.174669 0.015012 4.072386 2.688979

10 P10 0.275904 0.060899 0.428631 0.193125 0.064604 1.146323 0.316275

11 P11 0.283581 0.202801 0.102955 0.25711 0.149058 20.907 5.928834

12 P12 0.34792 0.248813 0.063313 0.327203 0.32659 30.52846 10.62145

13 P13 0.834978 0.225767 0.135456 0.735368 0.073896 0.810833 0.677028

14 P14 1.208797 0.326842 0.910195 0.632813 0.011395 0.827981 1.000861

15 P16 0.285422 0.340196 0.585273 0.180046 0.089465 7.176047 2.048199

16 P18 0.23898 0.284842 0.587302 0.150557 0.050674 3.222118 0.770021

17 P19 0.286482 0.34146 0.697671 0.16875 0.009565 3.241802 0.928718

18 P20 0.249246 0.297078 1.039658 0.1222 0.026631 3.235821 0.806516

 total 0.197175 3.549153 0.128862 
A-7 

Table I-7 Accuracy Measures for KFKI Calculation with Original FFTBM 

Calculation Window 0–100 s 

KFKI Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.057774 0.063761 0.57536 0.036674 0.037248 0.092367 0.005336

2 P2 0.056569 0.041204 0.16043 0.048748 0.157171 0.024468 0.001384

3 P3 0.061302 0.067654 0.358264 0.045132 0.138664 0.039266 0.002407

4 P4 0.036815 0.04063 0.434754 0.025659 0.21252 0.127714 0.004702

5 P5 0.086473 0.146585 0.310425 0.065988 0.190833 3.004864 0.259839

6 P6 0.106561 0.180638 0.669979 0.06381 0.057437 3.172731 0.33809

7 P7 0.2371 0.401922 0.752974 0.135256 0.133652 3.202652 0.75935

8 P8 0.010846 0.018386 0.440898 0.007527 0.102586 2.132043 0.023124

9 P9 0.626122 0.1382 2.024413 0.207023 0.009795 4.072386 2.549811

10 P10 0.505531 0.111583 0.551316 0.325872 0.006018 1.146323 0.579502

11 P11 0.077974 0.055763 0.012116 0.077041 0.235055 20.907 1.630206

12 P12 0.338304 0.241936 0.078292 0.31374 0.25799 30.52846 10.32789

13 P13 0.443754 0.119985 0.536681 0.288775 0.101078 0.810833 0.359811

14 P14 1.149578 0.31083 0.7396 0.660829 0.051934 0.827981 0.951829

15 P16 0.375135 0.447126 0.726839 0.217238 0.049237 7.176047 2.691984

16 P18 0.864375 1.030254 0.27805 0.676323 0.170749 3.222118 2.785118

17 P19 0.964233 1.149276 0.284851 0.750463 0.216703 3.241802 3.125853

18 P20 0.41048 0.489254 0.895625 0.216541 0.087653 3.235821 1.32824

 total 0.280832 0.193758 

Table I-8 Accuracy Measures for KINS Calculation with Original FFTBM 

Calculation Window 0–100 s 

KINS Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.168472 0.185928 0.476759 0.114082 0.056312 0.092367 0.015561

2 P2 0.062127 0.045252 0.94452 0.03195 0.042204 0.024468 0.00152

3 P3 0.114574 0.126446 0.40403 0.081604 0.203451 0.039266 0.004499

4 P4 0.045347 0.050046 0.313436 0.034525 0.21457 0.127714 0.005791

5 P5 0.404041 0.684913 0.343472 0.300744 0.250958 3.004864 1.214089

6 P6 0.107093 0.18154 0.519998 0.070456 0.084119 3.172731 0.339778

7 P7 0.221937 0.376218 0.544943 0.143654 0.155087 3.202652 0.710788

8 P8 0.020428 0.034629 0.448388 0.014104 0.1526 2.132043 0.043554

9 P9 0.356403 0.078666 0.319726 0.270058 0.037326 4.072386 1.451409

10 P10 0.991559 0.21886 0.770879 0.559925 0.009363 1.146323 1.136647

11 P11 0.225419 0.161207 0.111286 0.202846 0.177744 20.907 4.712845

12 P12 0.404191 0.289055 0.068836 0.37816 0.373997 30.52846 12.33933

13 P13 0.804497 0.217525 0.222652 0.657993 0.074057 0.810833 0.652313

14 P14 1.02036 0.275891 0.638386 0.622784 0.035174 0.827981 0.844839

15 P16 0.381236 0.454398 0.615083 0.236048 0.091216 7.176047 2.73577

16 P18 0.376898 0.449228 0.518477 0.248208 0.03789 3.222118 1.214411

17 P19 0.498874 0.594612 0.64717 0.302868 0.146456 3.241802 1.617252

18 P20 0.373166 0.444779 0.650309 0.226119 0.122622 3.235821 1.207497

 total 0.270511 0.184817 
A-8 

Table I-9 Accuracy Measures for NRI-K Calculation with Original FFTBM 

Calculation Window 0–100 s 

NRI-K Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.121585 0.134183 0.516363 0.080182 0.000641 0.092367 0.01123

2 P2 0.052365 0.038142 0.427627 0.036679 0.065181 0.024468 0.001281

3 P3 0.116611 0.128694 0.349754 0.086394 0.007648 0.039266 0.004579

4 P4 0.049788 0.054947 0.357369 0.03668 0.116409 0.127714 0.006359

5 P5 0.060709 0.102911 0.447292 0.041947 0.041214 3.004864 0.182422

6 P6 0.091976 0.155914 0.586473 0.057975 0.0087 3.172731 0.291816

7 P7 0.230096 0.390048 0.589636 0.144748 0.105925 3.202652 0.736917

8 P8 0.024516 0.041558 0.289768 0.019008 0.25237 2.132043 0.052269

9 P9 0.353881 0.07811 0.270098 0.278625 0.000131 4.072386 1.44114

10 P10 0.542266 0.119691 0.672785 0.324169 0.007685 1.146323 0.621612

11 P11 0.034956 0.024999 0.299831 0.026893 0.012409 20.907 0.730828

12 P12 0.20557 0.147013 0.043338 0.197031 0.241056 30.52846 6.27575

13 P13 0.861551 0.232952 0.20411 0.715508 0.062481 0.810833 0.698574

14 P14 1.023881 0.276843 0.479195 0.692188 0.041479 0.827981 0.847755

15 P16 0.508028 0.605521 1.478403 0.204982 0.094797 7.176047 3.64563

16 P18 0.718596 0.856499 0.180847 0.608543 0.148526 3.222118 2.315401

17 P19 0.854162 1.018081 0.163245 0.734292 0.204253 3.241802 2.769024

18 P20 0.490546 0.584685 0.3975 0.351017 0.186828 3.235821 1.587318

 total 0.277266 0.201947 

Table I-10 Accuracy Measures for NRI-M Calculation with Original FFTBM 

Calculation Window 0–100 s 

NRI-M Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.123309 0.136085 0.471286 0.08381 0.009574 0.092367 0.01139

2 P2 0.049085 0.035753 0.225015 0.040069 0.062695 0.024468 0.001201

3 P3 0.051749 0.057111 0.680975 0.030785 0.083493 0.039266 0.002032

4 P4 0.033993 0.037516 0.473792 0.023065 0.220267 0.127714 0.004341

5 P5 0.09961 0.168855 0.288013 0.077337 0.187218 3.004864 0.299316

6 P6 0.341229 0.578436 0.755438 0.194384 0.102755 3.172731 1.082627

7 P7 0.22555 0.382342 0.732847 0.130162 0.010833 3.202652 0.722358

8 P8 0.011393 0.019313 0.442575 0.007898 0.136854 2.132043 0.02429

9 P9 0.446399 0.098531 0.252345 0.35645 0.015976 4.072386 1.817908

10 P10 0.813306 0.179516 0.62854 0.499408 0.002026 1.146323 0.932312

11 P11 0.041246 0.029497 0.323663 0.031161 0.168476 20.907 0.862333

12 P12 0.248151 0.177463 0.071886 0.231508 0.273069 30.52846 7.575653

13 P13 0.839762 0.22706 0.152647 0.728552 0.052292 0.810833 0.680907

14 P14 1.005103 0.271766 0.579972 0.636153 0.036967 0.827981 0.832207

15 P16 0.337136 0.401834 0.517859 0.222113 0.092579 7.176047 2.419301

16 P18 0.476474 0.567913 0.327881 0.358823 0.084232 3.222118 1.535256

17 P19 1.162685 1.385812 0.222183 0.951318 0.221548 3.241802 3.769194

18 P20 0.303404 0.361629 1.0687 0.146664 0.077199 3.235821 0.98176

 total 0.284246 0.201256 
A-9 

Table I-11 Accuracy Measures for PSI Calculation with Original FFTBM 

Calculation Window 0–100 s 

PSI Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.14766 0.16296 0.542452 0.095731 0.03127 0.092367 0.013639

2 P2 0.126883 0.09242 0.220129 0.103992 0.380247 0.024468 0.003105

3 P3 0.068947 0.076091 0.248432 0.055227 0.039886 0.039266 0.002707

4 P4 0.08976 0.09906 0.397535 0.064227 0.091993 0.127714 0.011464

5 P5 0.093713 0.158858 0.312952 0.071376 0.065924 3.004864 0.281595

6 P6 0.12163 0.206182 0.376601 0.088355 0.059621 3.172731 0.385899

7 P7 0.22121 0.374986 0.570136 0.140886 0.081961 3.202652 0.708459

8 P8 0.032624 0.055303 0.304516 0.025009 0.257579 2.132043 0.069556

9 P9 0.489069 0.107949 0.403282 0.348518 0.019511 4.072386 1.991676

10 P10 0.85347 0.188381 0.71642 0.497239 0.010359 1.146323 0.978353

11 P11 0.056775 0.040602 0.280542 0.044336 0.025463 20.907 1.186987

12 P12 0.405337 0.289874 0.212755 0.334228 0.240104 30.52846 12.3743

13 P13 0.881032 0.238219 0.159404 0.759901 0.054513 0.810833 0.71437

14 P14 1.215099 0.328546 0.922855 0.631924 0.055339 0.827981 1.006079

15 P16 0.510844 0.608878 0.631375 0.313137 0.056652 7.176047 3.665839

16 P18 0.528446 0.629859 0.306697 0.404414 0.21699 3.222118 1.702716

17 P19 0.602311 0.717899 0.27882 0.47099 0.327835 3.241802 1.952574

18 P20 0.378195 0.450774 0.435791 0.263406 0.246335 3.235821 1.223772

 total 0.268158 0.19053 

Table I-12 Accuracy Measures for TAEK Calculation with Original FFTBM 

Calculation Window 0–100 s 

TAEK Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.179764 0.198391 0.476459 0.121754 0.082956 0.092367 0.016604

2 P2 0.049388 0.035973 0.642136 0.030075 0.050785 0.024468 0.001208

3 P3 0.096723 0.106745 0.43694 0.067311 0.112476 0.039266 0.003798

4 P4 0.068056 0.075108 0.402615 0.048521 0.236249 0.127714 0.008692

5 P5 0.175827 0.298055 0.089808 0.161338 0.255299 3.004864 0.528338

6 P6 0.136296 0.231043 0.548814 0.088 0.177602 3.172731 0.432431

7 P7 0.257374 0.436289 0.620384 0.158835 0.18729 3.202652 0.824279

8 P8 0.026232 0.044467 0.324649 0.019803 0.23564 2.132043 0.055927

9 P9 0.426358 0.094107 0.201193 0.354946 0.009488 4.072386 1.736294

10 P10 0.635359 0.140239 0.717669 0.369896 0.021957 1.146323 0.728327

11 P11 0.192755 0.137847 0.173013 0.164324 0.374535 20.907 4.029919

12 P12 0.192629 0.137758 0.267693 0.151953 0.241804 30.52846 5.880677

13 P13 0.812102 0.219581 0.166089 0.696432 0.06482 0.810833 0.658479

14 P14 1.011509 0.273498 0.585951 0.637793 0.048901 0.827981 0.837511

15 P16 0.421822 0.502773 0.90702 0.221194 0.023937 7.176047 3.027016

16 P18 0.880803 1.049835 0.222529 0.720477 0.127582 3.222118 2.838052

17 P19 0.950797 1.133262 0.166418 0.815143 0.23468 3.241802 3.082296

18 P20 0.342909 0.408716 0.762272 0.194584 0.050909 3.235821 1.109594

 total 0.306871 0.226801 
A-10 

Table I-13 Accuracy Measures for UPC Calculation with Original FFTBM 

Calculation Window 0–100 s 

UPC Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.087652 0.096734 0.345105 0.065163 0.107984 0.092367 0.008096

2 P2 0.087159 0.063486 0.324746 0.065793 0.319884 0.024468 0.002133

3 P3 0.226689 0.250178 0.276748 0.177552 0.04034 0.039266 0.008901

4 P4 0.182109 0.200978 0.408911 0.129255 0.155198 0.127714 0.023258

5 P5 0.137164 0.232514 0.369824 0.100133 0.170392 3.004864 0.412159

6 P6 0.101093 0.171369 0.510111 0.066944 0.11158 3.172731 0.320741

7 P7 0.230268 0.390341 0.611579 0.142884 0.160767 3.202652 0.73747

8 P8 0.015379 0.026069 0.484579 0.010359 0.092063 2.132043 0.032788

9 P9 0.752199 0.166028 0.514547 0.49665 0.023455 4.072386 3.063246

10 P10 2.266162 0.500195 0.084873 2.088874 0.065808 1.146323 2.597754

11 P11 0.1259 0.090037 0.298696 0.096943 0.264601 20.907 2.632188

12 P12 0.192475 0.137647 0.153581 0.16685 0.400921 30.52846 5.875961

13 P13 0.923047 0.249579 0.065873 0.866001 0.066354 0.810833 0.748437

14 P14 0.891563 0.241066 0.495698 0.596085 0.034481 0.827981 0.738198

15 P16 0.359062 0.427969 1.406281 0.149219 0.064802 7.176047 2.576649

16 P18 0.233764 0.278625 0.524202 0.153368 0.11897 3.222118 0.753215

17 P19 0.958668 1.142643 0.254569 0.764141 0.166386 3.241802 3.107811

18 P20 0.234832 0.279898 0.905545 0.123236 0.046878 3.235821 0.759874

 total 0.274742 0.198579 

Table I-14 Accuracy Measures for UPI Calculation with Original FFTBM 

Calculation Window 0–100 s 

UPI Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.120216 0.132673 0.431978 0.083951 0.016903 0.092367 0.011104

2 P2 0.029314 0.021352 0.517831 0.019313 0.006261 0.024468 0.000717

3 P3 0.070005 0.077258 0.550066 0.045162 0.014026 0.039266 0.002749

4 P4 0.115849 0.127853 0.394542 0.083073 0.132984 0.127714 0.014796

5 P5 0.094317 0.159883 0.348576 0.069938 0.103518 3.004864 0.283411

6 P6 0.091223 0.154638 0.617834 0.056386 0.034597 3.172731 0.289427

7 P7 0.231416 0.392287 0.713008 0.135093 0.12621 3.202652 0.741145

8 P8 0.024189 0.041004 0.310968 0.018451 0.251293 2.132043 0.051571

9 P9 0.480747 0.106112 0.28185 0.375042 0.003372 4.072386 1.957789

10 P10 0.893154 0.19714 0.80736 0.494176 0.007618 1.146323 1.023843

11 P11 0.090095 0.064431 0.367624 0.065877 0.031561 20.907 1.883613

12 P12 0.213088 0.152389 0.105864 0.192689 0.349853 30.52846 6.505258

13 P13 0.922732 0.249494 0.34446 0.686322 0.051861 0.810833 0.748182

14 P14 1.027638 0.277859 0.657101 0.620142 0.042326 0.827981 0.850865

15 P16 0.349301 0.416334 1.233852 0.156367 0.044769 7.176047 2.506601

16 P18 0.711032 0.847484 0.254864 0.566621 0.196289 3.222118 2.291029

17 P19 0.777408 0.926597 0.17537 0.661416 0.219534 3.241802 2.520202

18 P20 0.18975 0.226164 0.656954 0.114517 0.07239 3.235821 0.613996

 total 0.253942 0.179959 
A-11 

II. FFTBM IMPROVED BY SIGNAL MIRRORING 

Table II-1 Accuracy Measures for CEA Calculation with Improved FFTBM 

Calculation Window 0–100 s 

CEA Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.217078 0.239571 0.004409 0.216125 0.009828 0.030982 0.006725

2 P2 0.083258 0.060644 0.183735 0.070335 0.250487 0.011973 0.000997

3 P3 0.293369 0.323767 0.082189 0.271088 0.27867 0.010416 0.003056

4 P4 0.236101 0.260566 0.159377 0.203645 0.10192 0.031674 0.007478

5 P5 0.096816 0.164119 0.269931 0.076238 0.015208 1.227415 0.118834

6 P6 0.332419 0.563503 0.345903 0.246986 0.082218 1.293564 0.430006

7 P7 0.411512 0.697577 0.261212 0.326283 0.187823 1.380685 0.568168

8 P8 0.025042 0.04245 0.208794 0.020717 0.366545 1.072209 0.02685

9 P9 0.404493 0.089281 0.079871 0.374576 0.024919 2.506108 1.013704

10 P10 0.959294 0.211739 0.225665 0.782673 0.000668 0.669841 0.642574

11 P11 0.324033 0.23173 0.050846 0.308354 0.316725 5.137662 1.66477

12 P12 0.378875 0.27095 0.032752 0.36686 0.394402 10.55885 4.000486

13 P13 1.020489 0.275926 0.242973 0.821007 0.037314 0.47675 0.486518

14 P14 1.464666 0.396026 0.372618 1.067061 0.026536 0.519373 0.760708

15 P16 0.587065 0.699727 0.586843 0.369958 0.090263 4.017688 2.358644

16 P18 0.708245 0.844162 0.210746 0.584966 0.095495 1.462038 1.035481

17 P19 1.036154 1.234999 0.330601 0.778712 0.083392 1.42883 1.480488

18 P20 0.329562 0.392808 0.751826 0.188125 0.010205 1.411689 0.46524

 total 0.388864 0.304954 

Table II-2 Accuracy Measures for GID Calculation with Improved FFTBM

Calculation Window 0–100 s 

GID Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.152882 0.168723 0.009392 0.15146 0.079911 0.030982 0.004737

2 P2 0.756896 0.551314 0.075386 0.703837 0.047759 0.011973 0.009062

3 P3 0.259394 0.286272 0.138593 0.22782 0.102356 0.010416 0.002702

4 P4 0.076336 0.084246 0.108052 0.068892 0.114713 0.031674 0.002418

5 P5 0.124295 0.2107 0.263936 0.09834 0.056783 1.227415 0.152562

6 P6 0.299833 0.508264 0.502203 0.199596 0.024628 1.293564 0.387853

7 P7 0.427034 0.72389 0.355929 0.314938 0.043733 1.380685 0.589599

8 P8 NA NA NA NA NA NA NA 

9 P9 0.689477 0.152184 0.562348 0.441308 0.014017 2.506108 1.727904

10 P10 0.716096 0.158059 0.164501 0.614938 0.000495 0.669841 0.47967

11 P11 0.208714 0.149261 0.03474 0.201707 0.321718 5.137662 1.072302

12 P12 0.476698 0.340907 0.021141 0.466828 0.415799 10.55885 5.03338

13 P13 5.098523 1.378571 0.423321 3.582132 0.010052 0.47675 2.43072

14 P14 0.862333 0.233163 0.296508 0.665119 0.021092 0.519373 0.447872

15 P16 0.449621 0.535906 0.214428 0.370232 0.040818 4.017688 1.806435

16 P18 0.775937 0.924845 0.334348 0.58151 0.044704 1.462038 1.13445

17 P19 0.978951 1.166819 0.230327 0.795684 0.10227 1.42883 1.398755

18 P20 0.802917 0.957002 0.385336 0.579583 0.129891 1.411689 1.133469

 total 0.535556 0.465254 
A-12 

Table II-3 Accuracy Measures for GRS Calculation with Improved FFTBM

Calculation Window 0–100 s 

GRS Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.102707 0.113349 0.015792 0.10111 0.007698 0.030982 0.003182

2 P2 0.056376 0.041064 0.031934 0.054632 0.179199 0.011973 0.000675

3 P3 0.092033 0.101569 0.284971 0.071623 0.083956 0.010416 0.000959

4 P4 0.083324 0.091957 0.090306 0.076422 0.149076 0.031674 0.002639

5 P5 0.134754 0.228428 0.216275 0.110792 0.125882 1.227415 0.165398

6 P6 0.483227 0.819145 0.230291 0.392774 0.057433 1.293564 0.625084

7 P7 0.515158 0.873273 0.48003 0.348073 0.131169 1.380685 0.711271

8 P8 0.01547 0.026224 0.101652 0.014042 0.111864 1.072209 0.016587

9 P9 1.098199 0.242398 0.651727 0.664879 0.02099 2.506108 2.752205

10 P10 2.498109 0.551391 0.723391 1.449531 0.006148 0.669841 1.673335

11 P11 0.088859 0.063547 0.040381 0.08541 0.353818 5.137662 0.456529

12 P12 0.357334 0.255545 0.025687 0.348385 0.232473 10.55885 3.773037

13 P13 0.662126 0.17903 0.080028 0.613063 0.061379 0.47675 0.315668

14 P14 0.909275 0.245855 0.138616 0.798579 0.021943 0.519373 0.472253

15 P16 0.701668 0.836323 0.321031 0.531152 0.07993 4.017688 2.819085

16 P18 1.554102 1.852345 0.240389 1.252915 0.17893 1.462038 2.272157

17 P19 1.473331 1.756073 0.273298 1.157098 0.232884 1.42883 2.105139

18 P20 0.490393 0.584503 0.568713 0.312609 0.086244 1.411689 0.692283

 total 0.492335 0.378836 

Table II-4 Accuracy Measures for IRSN Calculation with Improved FFTBM 

Calculation Window 0–100 s 

IRSN Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.235697 0.260119 0.006346 0.234211 0.003287 0.030982 0.007302

2 P2 0.056081 0.040849 0.131183 0.049578 0.068507 0.011973 0.000671

3 P3 0.270075 0.298059 0.049104 0.257434 0.308223 0.010416 0.002813

4 P4 0.242685 0.267832 0.124584 0.2158 0.101941 0.031674 0.007687

5 P5 0.186169 0.315586 0.243579 0.149704 0.079387 1.227415 0.228507

6 P6 0.19956 0.338285 0.206586 0.165392 0.021594 1.293564 0.258143

7 P7 0.341547 0.578975 0.312638 0.260199 0.096746 1.380685 0.471568

8 P8 0.023827 0.04039 0.238122 0.019244 0.283747 1.072209 0.025547

9 P9 0.589887 0.130202 0.13881 0.517986 0.016115 2.506108 1.478321

10 P10 0.945894 0.208781 0.212455 0.780148 0.000517 0.669841 0.633598

11 P11 0.384426 0.27492 0.091785 0.352108 0.283148 5.137662 1.975051

12 P12 0.417615 0.298655 0.029504 0.405646 0.286539 10.55885 4.409531

13 P13 0.873574 0.236202 0.131077 0.772338 0.046499 0.47675 0.416476

14 P14 1.235288 0.334005 0.415539 0.872663 0.006021 0.519373 0.641575

15 P16 0.529863 0.631547 0.495309 0.35435 0.079155 4.017688 2.128823

16 P18 0.440464 0.524992 0.325993 0.332177 0.007222 1.462038 0.643976

17 P19 0.538057 0.641314 0.602165 0.335831 0.019726 1.42883 0.768792

18 P20 0.360991 0.430268 0.769491 0.204008 0.001292 1.411689 0.509607

 total 0.325055 0.253313 
A-13 

Table II-5 Accuracy Measures for JNES Calculation with Improved FFTBM 

Calculation Window 0–100 s 

JNES Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.234715 0.259036 0.007132 0.233053 0.067027 0.030982 0.007272

2 P2 0.203477 0.14821 0.008609 0.20174 0.396756 0.011973 0.002436

3 P3 0.361335 0.398776 0.530478 0.236093 0.191341 0.010416 0.003764

4 P4 0.306673 0.33845 0.20226 0.25508 0.106511 0.031674 0.009714

5 P5 0.278829 0.47266 0.398924 0.199317 0.002907 1.227415 0.342239

6 P6 0.289532 0.490801 0.410116 0.205325 0.031675 1.293564 0.374528

7 P7 0.363222 0.615718 0.397462 0.259916 0.07173 1.380685 0.501495

8 P8 0.043185 0.073206 0.092498 0.039529 0.54979 1.072209 0.046304

9 P9 0.82305 0.181667 0.132567 0.726712 0.015804 2.506108 2.062653

10 P10 1.70166 0.375596 1.153259 0.790272 0.025938 0.669841 1.139841

11 P11 0.206094 0.147387 0.121733 0.183729 0.016577 5.137662 1.058844

12 P12 0.337515 0.241372 0.038662 0.324952 0.395048 10.55885 3.563772

13 P13 0.683345 0.184767 0.142757 0.59798 0.061508 0.47675 0.325785

14 P14 0.904649 0.244605 0.253813 0.721519 0.063851 0.519373 0.46985

15 P16 0.493337 0.588012 0.234211 0.399718 0.072699 4.017688 1.982074

16 P18 0.667397 0.795474 0.310381 0.509315 0.129666 1.462038 0.975759

17 P19 1.215663 1.448957 0.323824 0.918296 0.161194 1.42883 1.736975

18 P20 0.672203 0.801203 0.529314 0.439545 0.162919 1.411689 0.948942

 total 0.433661 0.327613 

Table II-6 Accuracy Measures for KAERI Calculation with Improved FFTBM 

Calculation Window 0–100 s 

KAERI Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.165778 0.182956 0.010205 0.164104 0.070337 0.030982 0.005136

2 P2 0.045964 0.03348 0.053939 0.043612 0.028652 0.011973 0.00055

3 P3 0.204975 0.226214 0.256026 0.163194 0.165934 0.010416 0.002135

4 P4 0.158587 0.17502 0.058983 0.149754 0.15422 0.031674 0.005023

5 P5 0.123417 0.209211 0.196049 0.103187 0.121275 1.227415 0.151483

6 P6 0.159827 0.270931 0.259798 0.126867 0.073856 1.293564 0.206746

7 P7 0.322978 0.547498 0.217954 0.265181 0.127323 1.380685 0.445931

8 P8 0.026363 0.044689 0.369041 0.019257 0.112612 1.072209 0.028267

9 P9 0.660725 0.145837 0.734014 0.381038 0.012213 2.506108 1.655847

10 P10 0.291021 0.064235 0.052877 0.276405 0.05251 0.669841 0.194937

11 P11 0.384827 0.275207 0.046604 0.367691 0.223928 5.137662 1.97711

12 P12 0.430634 0.307965 0.027964 0.41892 0.382188 10.55885 4.547001

13 P13 0.889928 0.240624 0.111863 0.800393 0.059074 0.47675 0.424273

14 P14 1.200857 0.324695 0.600688 0.750213 0.009161 0.519373 0.623693

15 P16 0.312883 0.372928 0.313324 0.238238 0.073027 4.017688 1.257068

16 P18 0.315898 0.376521 0.649973 0.191457 0.042325 1.462038 0.461855

17 P19 0.411059 0.489944 0.454187 0.282673 0.007577 1.42883 0.587334

18 P20 0.343449 0.40936 0.487809 0.230842 0.022193 1.411689 0.484844

 total 0.260962 0.204529 
A-14 

Table II-7 Accuracy Measures for KFKI Calculation with Improved FFTBM 

Calculation Window 0–100 s 

KFKI Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.109811 0.12119 0.0156 0.108125 0.02927 0.030982 0.003402

2 P2 0.05727 0.041715 0.021288 0.056077 0.158941 0.011973 0.000686

3 P3 0.128502 0.141817 0.203915 0.106737 0.124928 0.010416 0.001338

4 P4 0.088517 0.097689 0.082414 0.081777 0.178546 0.031674 0.002804

5 P5 0.136813 0.23192 0.202886 0.113738 0.147929 1.227415 0.167927

6 P6 0.164992 0.279687 0.182619 0.139514 0.045582 1.293564 0.213428

7 P7 0.355601 0.602799 0.16443 0.305386 0.103556 1.380685 0.490973

8 P8 0.01322 0.02241 0.039803 0.012714 0.083843 1.072209 0.014174

9 P9 0.64922 0.143298 0.503312 0.43186 0.00769 2.506108 1.627014

10 P10 0.550214 0.121445 0.271876 0.4326 0.00474 0.669841 0.368556

11 P11 0.107047 0.076554 0.02518 0.104418 0.349051 5.137662 0.54997

12 P12 0.350334 0.25054 0.027476 0.340966 0.360853 10.55885 3.699129

13 P13 0.468138 0.126578 0.11993 0.418006 0.081636 0.47675 0.223185

14 P14 1.122835 0.303599 0.339769 0.838081 0.042465 0.519373 0.58317

15 P16 0.421431 0.502306 0.215487 0.346718 0.039217 4.017688 1.693177

16 P18 1.193997 1.423133 0.224253 0.975286 0.136476 1.462038 1.745669

17 P19 1.400923 1.66977 0.28939 1.0865 0.169533 1.42883 2.001681

18 P20 0.597556 0.71223 0.616824 0.369586 0.069142 1.411689 0.843563

 total 0.381593 0.305385 

Table II-8 Accuracy Measures for KINS Calculation with Improved FFTBM 

Calculation Window 0–100 s 

KINS Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.317655 0.350569 0.002546 0.316848 0.044607 0.030982 0.009842

2 P2 0.079056 0.057584 0.043129 0.075788 0.033956 0.011973 0.000947

3 P3 0.22585 0.249251 0.248442 0.180905 0.194921 0.010416 0.002352

4 P4 0.111122 0.122636 0.078417 0.103042 0.176877 0.031674 0.00352

5 P5 0.580557 0.984135 0.173003 0.494932 0.214205 1.227415 0.712585

6 P6 0.167504 0.283946 0.140649 0.14685 0.066083 1.293564 0.216677

7 P7 0.327375 0.554951 0.180274 0.277372 0.122177 1.380685 0.452001

8 P8 0.021824 0.036996 0.08172 0.020176 0.142292 1.072209 0.0234 

9 P9 0.352896 0.077892 0.078206 0.327299 0.030688 2.506108 0.884395

10 P10 1.085341 0.23956 0.176149 0.922792 0.007333 0.669841 0.727006

11 P11 0.320825 0.229436 0.099931 0.291677 0.254602 5.137662 1.648288

12 P12 0.462025 0.330414 0.0252 0.450668 0.473908 10.55885 4.878453

13 P13 0.856764 0.231657 0.093153 0.783755 0.05925 0.47675 0.408462

14 P14 1.023401 0.276714 0.200158 0.852722 0.028008 0.519373 0.531527

15 P16 0.436387 0.520132 0.184446 0.368431 0.071305 4.017688 1.753266

16 P18 0.510322 0.608256 0.334138 0.38251 0.030896 1.462038 0.74611

17 P19 0.714421 0.851524 0.571506 0.454609 0.116243 1.42883 1.020787

18 P20 0.544067 0.648477 0.736152 0.313375 0.096578 1.411689 0.768053

 total 0.369674 0.298348 
A-15 

Table II-9 Accuracy Measures for NRI-K Calculation with Improved FFTBM 

Calculation Window 0–100 s 

NRI-K Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.231934 0.255966 0.003611 0.231099 0.000502 0.030982 0.007186

2 P2 0.070055 0.051027 0.093947 0.064038 0.049882 0.011973 0.000839

3 P3 0.163319 0.180242 0.082049 0.150935 0.010313 0.010416 0.001701

4 P4 0.129155 0.142538 0.152692 0.112046 0.090647 0.031674 0.004091

5 P5 0.095342 0.16162 0.173595 0.081239 0.032186 1.227415 0.117024

6 P6 0.148113 0.251075 0.28586 0.115186 0.006639 1.293564 0.191594

7 P7 0.338836 0.57438 0.355519 0.249968 0.083589 1.380685 0.467826

8 P8 0.024549 0.041615 0.018799 0.024096 0.251061 1.072209 0.026322

9 P9 0.357502 0.078909 0.069394 0.334303 0.000105 2.506108 0.895938

10 P10 0.54551 0.120407 0.155233 0.472208 0.00655 0.669841 0.365405

11 P11 0.078908 0.056431 0.191458 0.066228 0.011207 5.137662 0.405404

12 P12 0.237139 0.169589 0.047789 0.226324 0.302676 10.55885 2.50392

13 P13 0.908611 0.245676 0.133656 0.801488 0.050479 0.47675 0.43318

14 P14 1.02472 0.27707 0.144958 0.894985 0.0331 0.519373 0.532212

15 P16 0.554021 0.660341 0.598595 0.346567 0.077782 4.017688 2.225882

16 P18 0.988613 1.178335 0.146279 0.862454 0.119195 1.462038 1.445391

17 P19 1.201202 1.431721 0.172229 1.024716 0.165088 1.42883 1.716314

18 P20 0.738962 0.880773 0.409639 0.524221 0.142416 1.411689 1.043184

 total 0.375429 0.306813 

Table II-10 Accuracy Measures for NRI-M Calculation with Improved FFTBM

Calculation Window 0–100 s 

NRI-M Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.221764 0.244743 0.003725 0.220941 0.007951 0.030982 0.006871

2 P2 0.051813 0.03774 0.032779 0.050169 0.060807 0.011973 0.00062

3 P3 0.10131 0.111807 0.203628 0.08417 0.080544 0.010416 0.001055

4 P4 0.081244 0.089663 0.074884 0.075584 0.186168 0.031674 0.002573

5 P5 0.137078 0.232369 0.113466 0.12311 0.166852 1.227415 0.168252

6 P6 0.536165 0.908884 0.363738 0.393158 0.080354 1.293564 0.693564

7 P7 0.332437 0.563532 0.328688 0.250199 0.008541 1.380685 0.45899

8 P8 0.014006 0.023743 0.112566 0.012589 0.110891 1.072209 0.015018

9 P9 0.46617 0.102895 0.061962 0.438971 0.012454 2.506108 1.168272

10 P10 0.837258 0.184802 0.271875 0.658286 0.001687 0.669841 0.560829

11 P11 0.092589 0.066215 0.111757 0.083282 0.153003 5.137662 0.475694

12 P12 0.30351 0.217053 0.034148 0.293488 0.323384 10.55885 3.204716

13 P13 0.882434 0.238598 0.106269 0.797667 0.0424 0.47675 0.4207 

14 P14 1.005855 0.271969 0.153257 0.872187 0.029502 0.519373 0.522414

15 P16 0.379748 0.452624 0.246143 0.304738 0.073544 4.017688 1.525707

16 P18 0.645308 0.769147 0.229737 0.524753 0.068667 1.462038 0.943465

17 P19 1.723486 2.054234 0.214571 1.419007 0.169881 1.42883 2.462568

18 P20 0.442652 0.5276 0.50875 0.29339 0.060762 1.411689 0.624887

 total 0.394312 0.320563 
A-16 

Table II-11 Accuracy Measures for PSI Calculation with Improved FFTBM

Calculation Window 0–100 s 

PSI Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.272856 0.301128 0.002682 0.272126 0.025274 0.030982 0.008454

2 P2 0.132016 0.096159 0.120966 0.11777 0.374163 0.011973 0.001581

3 P3 0.153491 0.169395 0.241797 0.123604 0.033836 0.010416 0.001599

4 P4 0.232175 0.256233 0.27091 0.182684 0.071841 0.031674 0.007354

5 P5 0.12594 0.213489 0.257063 0.100186 0.060163 1.227415 0.154581

6 P6 0.176784 0.299676 0.169029 0.151223 0.050404 1.293564 0.228681

7 P7 0.331512 0.561965 0.201261 0.27597 0.063554 1.380685 0.457714

8 P8 0.029767 0.050459 0.082643 0.027494 0.281223 1.072209 0.031916

9 P9 0.507266 0.111966 0.099348 0.461425 0.015314 2.506108 1.271264

10 P10 0.898958 0.198421 0.279003 0.702859 0.008432 0.669841 0.602159

11 P11 0.122245 0.087423 0.09553 0.111585 0.024109 5.137662 0.628052

12 P12 0.343884 0.245927 0.03413 0.332535 0.409928 10.55885 3.631019

13 P13 0.911855 0.246553 0.129562 0.807264 0.044877 0.47675 0.434727

14 P14 1.198659 0.324101 0.315725 0.911025 0.044803 0.519373 0.622551

15 P16 0.571915 0.68167 0.390346 0.411347 0.045279 4.017688 2.297776

16 P18 0.757719 0.90313 0.373082 0.551838 0.167083 1.462038 1.107814

17 P19 0.865622 1.031741 0.411218 0.613386 0.25928 1.42883 1.236827

18 P20 0.563088 0.671149 0.422234 0.395918 0.189988 1.411689 0.794906

 total 0.358366 0.28083 

Table II-12 Accuracy Measures for TAEK Calculation with Improved FFTBM 

Calculation Window 0–100 s 

TAEK Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.331574 0.365931 0.002442 0.330766 0.067173 0.030982 0.010273

2 P2 0.058826 0.042848 0.055888 0.055713 0.043652 0.011973 0.000704

3 P3 0.233364 0.257545 0.227311 0.190143 0.088041 0.010416 0.002431

4 P4 0.168228 0.185659 0.054797 0.159488 0.193061 0.031674 0.005328

5 P5 0.185449 0.314366 0.116911 0.166038 0.296866 1.227415 0.227623

6 P6 0.206096 0.349365 0.083765 0.190167 0.144319 1.293564 0.266599

7 P7 0.372284 0.631079 0.162655 0.320202 0.150465 1.380685 0.514007

8 P8 0.023816 0.040371 0.069191 0.022274 0.25855 1.072209 0.025535

9 P9 0.448123 0.098911 0.063146 0.421507 0.007349 2.506108 1.123044

10 P10 0.659074 0.145473 0.133414 0.581495 0.018147 0.669841 0.441475

11 P11 0.312652 0.223591 0.027238 0.304362 0.470736 5.137662 1.606301

12 P12 0.365564 0.261431 0.082948 0.337563 0.184556 10.55885 3.859931

13 P13 0.861712 0.232995 0.059747 0.81313 0.052049 0.47675 0.410821

14 P14 1.015 0.274442 0.170522 0.867134 0.03892 0.519373 0.527164

15 P16 0.493706 0.588452 0.259324 0.392041 0.0183 4.017688 1.983557

16 P18 1.215473 1.44873 0.197323 1.015159 0.102076 1.462038 1.777067

17 P19 1.354052 1.613904 0.286065 1.052865 0.187305 1.42883 1.934711

18 P20 0.4974 0.592854 0.396076 0.356284 0.040302 1.411689 0.702174

 total 0.425997 0.359303 
A-17 

Table II-13 Accuracy Measures for UPC Calculation with Improved FFTBM 

Calculation Window 0–100 s 

UPC Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.164587 0.181641 0.010221 0.162922 0.085892 0.030982 0.005099

2 P2 0.09564 0.069663 0.029089 0.092936 0.298459 0.011973 0.001145

3 P3 0.355136 0.391934 0.166209 0.304522 0.048631 0.010416 0.003699

4 P4 0.463952 0.512026 0.123148 0.413082 0.123054 0.031674 0.014695

5 P5 0.212457 0.360147 0.302364 0.163132 0.134917 1.227415 0.260772

6 P6 0.156388 0.265103 0.129517 0.138456 0.088627 1.293564 0.202298

7 P7 0.338232 0.573356 0.217379 0.277836 0.127188 1.380685 0.466992

8 P8 0.017124 0.029028 0.061436 0.016133 0.082363 1.072209 0.018361

9 P9 0.778733 0.171885 0.279822 0.608469 0.018443 2.506108 1.951588

10 P10 2.147637 0.474034 0.054002 2.037603 0.059533 0.669841 1.438575

11 P11 0.240529 0.172013 0.150553 0.209055 0.282353 5.137662 1.235757

12 P12 0.286608 0.204966 0.027572 0.278918 0.389985 10.55885 3.026252

13 P13 0.923863 0.2498 0.059923 0.871632 0.056486 0.47675 0.440451

14 P14 0.884488 0.239153 0.185736 0.74594 0.027758 0.519373 0.459379

15 P16 0.40384 0.48134 0.298248 0.311065 0.051555 4.017688 1.622503

16 P18 0.321711 0.383449 0.347969 0.238663 0.095444 1.462038 0.470353

17 P19 1.360614 1.621726 0.166391 1.166517 0.133251 1.42883 1.944087

18 P20 0.352381 0.420006 0.524959 0.231076 0.035873 1.411689 0.497453

 total 0.377848 0.318913 

Table II-14 Accuracy Measures for UPI Calculation with Improved FFTBM

Calculation Window 0–100 s 

UPI Variable AA VA I AMMIN fraction A0 AAexp AAdif

1 P1 0.218847 0.241523 0.003475 0.218089 0.013868 0.030982 0.00678

2 P2 0.037899 0.027605 0.052458 0.03601 0.004958 0.011973 0.000454

3 P3 0.130647 0.144184 0.125518 0.116077 0.014194 0.010416 0.001361

4 P4 0.275549 0.3041 0.188389 0.231867 0.112939 0.031674 0.008728

5 P5 0.133828 0.226859 0.200889 0.111441 0.089477 1.227415 0.164262

6 P6 0.142247 0.24113 0.106877 0.128512 0.027262 1.293564 0.184005

7 P7 0.341735 0.579294 0.188683 0.28749 0.099319 1.380685 0.471828

8 P8 0.023329 0.039547 0.022238 0.022822 0.259552 1.072209 0.025014

9 P9 0.500574 0.110488 0.080581 0.463245 0.002636 2.506108 1.254493

10 P10 0.956947 0.211221 0.211551 0.789853 0.006096 0.669841 0.641002

11 P11 0.187863 0.134349 0.056761 0.177773 0.030857 5.137662 0.965179

12 P12 0.281422 0.201257 0.039184 0.27081 0.3837 10.55885 2.971489

13 P13 0.968259 0.261804 0.209952 0.800246 0.04211 0.47675 0.461617

14 P14 1.028306 0.27804 0.187316 0.866076 0.033782 0.519373 0.534074

15 P16 0.376762 0.449065 0.488415 0.25313 0.03714 4.017688 1.513712

16 P18 0.98083 1.169058 0.320347 0.742858 0.157105 1.462038 1.434011

17 P19 1.005378 1.198317 0.237849 0.812197 0.192949 1.42883 1.436514

18 P20 0.271297 0.32336 0.137742 0.238452 0.05814 1.411689 0.382987

 total 0.341178 0.281724