Verification and Validation 

of Selected Fire Models for 

Nuclear Power Plant 

Applications 

Volume 7: 

Experimental Uncertainty 

January 2006 

U.S. Nuclear Regulatory Commission Electric Power Research Institute 

Office of Nuclear Regulatory Research 3412 Hillview Avenue 

Washington, DC 20555-0001 Palo Alto, CA 94303 

 

NUREG-1824 EPRI 1011999 

Draft for Comment Preliminary Report

Verification & Validation of Selected 

Fire Models for Nuclear Power Plant 

Applications 

Volume 7: Experimental Uncertainty 

NUREG-1824 EPRI 1011999 

January 2006 

U.S. Nuclear Regulatory Commission Electric Power Research Institute (EPRI) 

Office of Nuclear Regulatory Research (RES) 3412 Hillview Avenue 

Division of Risk Analysis and Applications Palo Alto, CA 94303 

Two White Flint North, 11545 Rockville Pike 

Rockville, MD 20852-2738 

U.S. NRC-RES Project Manager EPRI Project Manager 

M. H. Salley R. P. Kassawara 

DISCLAIMER OF WARRANTIES AND LIMITATION OF LIABILITIES 

THIS DOCUMENT WAS PREPARED BY THE ORGANIZATION(S) NAMED BELOW AS AN 

ACCOUNT OF WORK SPONSORED OR COSPONSORED BY THE ELECTRIC POWER RESEARCH 

INSTITUTE, INC. (EPRI). NEITHER EPRI NOR ANY MEMBER OF EPRI, ANY COSPONSOR, THE 

ORGANIZATION(S) BELOW, OR ANY PERSON ACTING ON BEHALF OF ANY OF THEM: 

(A) MAKES ANY WARRANTY OR REPRESENTATION WHATSOEVER, EXPRESS OR IMPLIED, 

(I) WITH RESPECT TO THE USE OF ANY INFORMATION, APPARATUS, METHOD, PROCESS, OR 

SIMILAR ITEM DISCLOSED IN THIS DOCUMENT, INCLUDING MERCHANTABILITY AND FITNESS 

FOR A PARTICULAR PURPOSE, OR (II) THAT SUCH USE DOES NOT INFRINGE ON OR 

INTERFERE WITH PRIVATELY OWNED RIGHTS, INCLUDING ANY PARTY’S INTELLECTUAL 

PROPERTY, OR (III) THAT THIS DOCUMENT IS SUITABLE TO ANY PARTICULAR USER’S 

CIRCUMSTANCE; OR 

(B) ASSUMES RESPONSIBILITY FOR ANY DAMAGES OR OTHER LIABILITY WHATSOEVER 

(INCLUDING ANY CONSEQUENTIAL DAMAGES, EVEN IF EPRI OR ANY EPRI REPRESENTATIVE 

HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES) RESULTING FROM YOUR 

SELECTION OR USE OF THIS DOCUMENT OR ANY INFORMATION, APPARATUS, METHOD, 

PROCESS, OR SIMILAR ITEM DISCLOSED IN THIS DOCUMENT. 

ORGANIZATION(S) THAT PREPARED THIS DOCUMENT: 

U.S. Nuclear Regulatory Commission, Office of Nuclear Regulatory Research 

Science Applications International Corporation 

National Institute of Standards and Technology 

ORDERING INFORMATION 

Requests for copies of this report should be directed to EPRI Orders and Conferences, 1355 Willow 

Way, Suite 278, Concord, CA 94520, (800) 313-3774, press 2 or internally x5379, (925) 609-9169, 

(925) 609-1310 (fax). 

Electric Power Research Institute and EPRI are registered service marks of the Electric Power 

Research Institute, Inc. EPRI. ELECTRIFY THE WORLD is a service mark of the Electric Power 

Research Institute, Inc. 

iii 

COMMENTS ON DRAFT NUREG-1824 REPORT 

This report is being published jointly by the U.S. Nuclear Regulatory Commission (NRC) and 

the Electric Power Research Institute (EPRI). Any interested party may submit comments on 

this report for consideration by the NRC and EPRI staffs. Comments may be accompanied by 

additional relevant information or supporting data. Please specify both the report number (Draft 

NUREG-1824) and the volume number in your comments, and send them by March 31, 2006, to 

the following address: 

Chief Rules Review and Directives Branch 

U.S. Nuclear Regulatory Commission 

Mail Stop T-6D59 

Washington, DC 20555-0001 

For any questions about the material in this report, please contact: 

Mark Henry Salley 

Mail Stop T-10E50 

U.S. Nuclear Regulatory Commission 

Washington, DC 20555-0001 

Phone: (301) 415-2840 

Email: MXS3@nrc.gov

If EPRI members also wish to provide comments to EPRI, they may send them to the following 

address: 

R.P. Kassawara 

Electric Power Research Institute 

3412 Hillview Avenue 

Palo Alto, CA 94304 

Phone: (650) 855-2775 

Email: RKASSAWA@epri.com

v 

CITATIONS 

This report was prepared by 

U.S. Nuclear Regulatory Commission, 

Office of Nuclear Regulatory Research (RES) 

Two White Flint North, 11545 Rockville Pike 

Rockville, MD 20852-2738 

Principal Investigators: 

K. Hill 

J. Dreisbach 

Electric Power Research Institute (EPRI) 

3412 Hillview Avenue 

Palo Alto, CA 94303 

Science Applications International Corp (SAIC)

4920 El Camino Real 

Los Altos, CA 94022 

Principal Investigators: 

F. Joglar 

B. Najafi 

National Institute of Standards and Technology 

Building Fire Research Laboratory (BFRL) 

100 Bureau Drive, Stop 8600 

Gaithersburg, MD 20899-8600 

Principal Investigators: 

K McGrattan 

R. Peacock 

A. Hamins 

Volume 1, Main Report: B. Najafi, M.H. Salley, F. Joglar, J. Dreisbach 

Volume 2, FDTS

: K. Hill, J. Dreisbach 

Volume 3, FIVE-REV. 1: F. Joglar 

Volume 4, CFAST: R. Peacock, J. Dreisbach, P. Reneke (NIST) 

Volume 5, MAGIC: F. Joglar, B. Guatier (EdF), L. Gay (EdF), J. Texeraud (EdF) 

Volume 6, FDS: K. McGrattan, J. Dreisbach 

Volume 7, Experimental Uncertainty: A. Hamins 

This report describes research sponsored jointly by U.S. Nuclear Regulatory Commission, Office 

of Nuclear Regulatory Research (RES) and Electric Power Research Institute (EPRI). 

The report is a corporate document that should be cited in the literature in the following manner: 

Verification and Validation of Selected Fire Models for Nuclear Power Plant Applications, 

Volume 7: Experimental Uncertainty, U.S. Nuclear Regulatory Commission, Office of Nuclear 

Regulatory Research (RES), Rockville, MD: 2005 and Electric Power Research Institute (EPRI), 

Palo Alto, CA. NUREG-1824 and EPRI 1011999.

vii 

ABSTRACT 

There is a movement to introduce risk- and performance-based analyses into fire protection 

engineering practice, both domestically and worldwide. This movement exists in the general fire 

protection community, as well as the nuclear power plant (NPP) fire protection community. 

In 2002, the National Fire Protection Association (NFPA) developed NFPA 805, PerformanceBased Standard for Fire Protection for Light-Water Reactor Electric Generating Plants, 2001 

Edition. In July 2004, the U.S. Nuclear Regulatory Commission (NRC) amended its fire 

protection requirements in Title 10, Section 50.48, of the Code of Federal Regulations (10 CFR 

50.48) to permit existing reactor licensees to voluntarily adopt fire protection requirements contained 

in NFPA 805 as an alternative to the existing deterministic fire protection requirements. In 

addition, the nuclear fire protection community wants to use risk-informed, performance-based 

(RI/PB) approaches and insights to support fire protection decision-making in general. 

One key tool needed to support RI/PB fire protection is the availability of verified and validated 

fire models that can reliably predict the consequences of fires. Section 2.4.1.2 of NFPA 805 

requires that only fire models acceptable to the Authority Having Jurisdiction (AHJ) shall be 

used in fire modeling calculations. Further, Sections 2.4.1.2.2 and 2.4.1.2.3 of NFPA 805 state 

that fire models shall only be applied within the limitations of the given model, and shall be 

verified and validated. 

This report is the first effort to document the verification and validation (V&V) of five fire models 

that are commonly used in NPP applications. The project was performed in accordance with the 

guidelines that the American Society for Testing and Materials (ASTM) set forth in Standard 

E1355-04, “Evaluating the Predictive Capability of Deterministic Fire Models.” The results of 

this V&V are reported in the form of ranges of accuracies for the fire model predictions. 

ix 

CONTENTS 

1 SELECTION OF DATA FOR MODEL EVALUATION ...........................................................1-1

1.1 Fire Experiments and Test Selection .........................................................................1-1

1.2 Measurement Selection..............................................................................................1-3

1.3 Fire Models.................................................................................................................1-6

1.4 Uncertainty and Sensitivity.........................................................................................1-6

1.4.1 Types of Uncertainty .................................................................................................1-6

1.4.2 Methodology for Evaluating the Models....................................................................1-8

1.5 Organization of this Volume .......................................................................................1-9

2 MODEL INPUT AND ITS UNCERTAINTY .............................................................................2-1

2.1 Heat Release Rate Measurement Uncertainty...........................................................2-1

2.2 Radiative Fraction ......................................................................................................2-2

2.3 Summary of Results from the Experiments................................................................2-2

3 TEST SERIES.........................................................................................................................3-1

3.1 FM/SNL Test Series...................................................................................................3-1

3.1.1 Heat Release Rate....................................................................................................3-2

3.1.2 Radiative Fraction .....................................................................................................3-2

3.1.3 Other Measurements ................................................................................................3-3

3.2 NBS Multi-Compartment Test Series .........................................................................3-4

3.2.1 Heat Release Rate....................................................................................................3-4

3.2.2 Radiative Fraction .....................................................................................................3-6

3.2.3 Other Measurements ................................................................................................3-6

3.3 ICFMP Benchmark exercise #2..................................................................................3-7

3.3.1 Supplementary Information.......................................................................................3-7

3.3.2 Heat Release Rate....................................................................................................3-7

3.3.2 Radiative Fraction .....................................................................................................3-8

3.4 ICFMP Benchmark Exercise #3 ...............................................................................3-12

3.4.1 Supplementary Information.....................................................................................3-12

3.4.2 Heat Release Rate..................................................................................................3-13

3.4.3 Radiative Fraction ...................................................................................................3-13

3.4.4 Other Measurements ..............................................................................................3-14

3.5 ICFMP Benchmark Exercise #4 ...............................................................................3-14
x 

3.5.1 Heat Release Rate..................................................................................................3-15

3.5.2 Radiative Fraction ...................................................................................................3-16

3.6 ICFMP Benchmark Exercise #5 ...............................................................................3-17

3.6.1 Heat Release Rate..................................................................................................3-17

3.6.2 Radiative Fraction ...................................................................................................3-17

4 MEASUREMENT UNCERTAINTY .........................................................................................4-1

4.1 Temperature and Hot Gas Layer Depth.....................................................................4-1

4.2 Gas Species Volume Fraction....................................................................................4-6

4.3 Smoke Concentration.................................................................................................4-7

4.4 Pressure.....................................................................................................................4-8

4.5 Heat Flux....................................................................................................................4-8

4.6 Surface/Target Temperature......................................................................................4-9

4.7 Summary..................................................................................................................4-10

5 SENSITIVITY OF MODEL RESULTS TO UNCERTAINTY IN MEASURED INPUT 

PARAMETERS..........................................................................................................................5-1

5.1 Hot Gas Layer and Ceiling Jet Temperatures............................................................5-1

5.2 Hot Gas Layer Depth..................................................................................................5-2

5.3 Plume Temperature....................................................................................................5-2

5.4 Flame Height..............................................................................................................5-2

5.5 Gas Concentration .....................................................................................................5-3

5.6 Smoke Concentration.................................................................................................5-4

5.7 Pressure.....................................................................................................................5-5

5.8 Heat Flux....................................................................................................................5-6

5.9 Wall/Target Surface Temperature..............................................................................5-6

5.10 Summary....................................................................................................................5-6

6 REPRESENTATIVE UNCERTAINTIES .................................................................................6-1

6.1 Summary of the Estimated Measurement Uncertainty and Subsequent Model 

Input Uncertainties ................................................................................................................6-1

6.2 Representative Uncertainties .....................................................................................6-4

6.3 Conclusions................................................................................................................6-5

7 REFERENCES .......................................................................................................................7-1

A DOCUMENTATION OF PERSONAL COMMUNICATIONS................................................. A-1
xi 

B TIME AVERAGING ............................................................................................................... B-1

xiii 

FIGURES 

Figure 1-1: Comparison of hypothetical measured and simulated temperature profiles. 

Uncertainties associated with the measurements and model inputs are also shown. .......1-9 

Figure 3-1: Prescribed and measured heat release rate as a function of time during Test 21 of 

the FM/SNL test series.......................................................................................................3-3 

Figure 3-2: Prescribed and measured heat release rate as a function of time during Test 100A 

of the NBS multi-room test series. .....................................................................................3-5 

Figure 3-3: Prescribed and measured heat release rate as a function of time during Test 100Z of 

the NBS multi-room test series. .........................................................................................3-5 

Figure 3-4: Prescribed heat release rate as a function of time during Case 1 of the BE #2 test 

series..................................................................................................................................3-9 

Figure 3-5: Prescribed heat release rate as a function of time during Case 2 of the BE #2 test 

series..................................................................................................................................3-9 

Figure 3-6: Prescribed heat release rate as a function of time during Case 3 of the BE #2 test 

series................................................................................................................................3-10 

Figure 3-7: Measured and prescribed heat release rate as a function of time during Test 3 of the 

ICFMP #3 test series........................................................................................................3-14 

Figure 3-8: The estimated HRR based on the mass loss rate as a function of time during Test 1 

of the BE #4 test series....................................................................................................3-16 

Figure 3-9: The heat release rate as a function of time during Test 4 of the BE #5 test series. 

Only the first 20 min of the test were used for model evaluation. ....................................3-18

xv 

TABLES 

Table 1-1: Overview of the Experiments Used for Model Evaluation.........................................1-2 

Table 1-2: Summary of the Test Series, Experiments, and Instrumentation Used for the Model 

Evaluation. .........................................................................................................................1-5 

Table 2-1: Summary of the Heat Release Rate and Radiative Fraction ....................................2-3 

Table 3-1: Mass Loss and Heat Release Rates for Case 1 of the BE #2 test series...............3-10 

Table 3-2: Mass Loss and Heat Release Rates for Case 2 of the BE #2 test series...............3-10 

Table 3-3: Mass Loss and Heat Release Rates for Case 3 of the BE #2 test series...............3-11 

Table 3-4: Relative Weighting of the Calculation of HGL.........................................................3-14 

Table 3-5: Measured Mass Loss Rate and Calculated ............................................................3-15 

Table 4-1: Expanded Measurement Uncertainty of a Bare Bead Thermocouple in the Hot Smoky 

Upper Layer of a Compartment Fire ..................................................................................4-4 

Table 4-2: The Relative Expanded Uncertainties (Ue) Associated with the Measured HGL Depth 

and Temperature Rise .......................................................................................................4-5 

Table 4-3: The Relative Expanded Uncertainties (Ue) Associated with the Measured Ceiling Jet 

Temperature Rise ..............................................................................................................4-5 

Table 4-4: The Relative Expanded Uncertainties (Ue) Associated with the Measured Plume 

Temperature Rise ..............................................................................................................4-6 

Table 4-5: Summary of the Relative Expanded Uncertainties Associated with the Oxygen and 

Carbon Dioxide Concentrations .........................................................................................4-7 

Table 4-6: The Relative Expanded Uncertainties (Ue) Associated with the Measured Smoke 

Concentration and the Compartment Pressure..................................................................4-7 

Table 4-7: Summary of the Relative Expanded Uncertainties Associated with the Measured 

Target Heat Flux and Target Temperatures.......................................................................4-9 

Table 4-8: Summary of the Relative Expanded Uncertainties Associated with the Surface Heat 

Flux and Temperatures......................................................................................................4-9 

Table 5-1: Summary of the model sensitivity, Um, to uncertainty in the HRR ............................5-7 

Table 6-1: Summary of the Relative Expanded Uncertainties Associated with the HGL Layer 

Depth and Temperature Rise.............................................................................................6-1 

Table 6-2: Summary of the Relative Expanded Uncertainties Associated with Ceiling Jet 

Temperatures.....................................................................................................................6-2 

Table 6-3: Summary of the Relative Expanded Uncertainties Associated with Plume 

Temperatures.....................................................................................................................6-2 

Table 6-4: Summary of the Relative Expanded Uncertainties Associated with Oxygen and 

Carbon Dioxide ..................................................................................................................6-3 

Table 6-5: Summary of the Relative Expanded Uncertainties Associated with the Smoke 

Concentration and the Compartment Pressure..................................................................6-3 

Table 6-6: Summary of the Relative Expanded Uncertainties Associated with the Target Heat 

Flux and Target Temperatures...........................................................................................6-3 

Table 6-7: Summary of the Relative Expanded Uncertainties Associated with the Surface Heat 

Flux and Surface Temperatures.........................................................................................6-4 

Table 6-8: The weighted combined expanded uncertainty, Ucw, determined from Eq. 5.1 and 

Tables 6-1 through 6-7.......................................................................................................6-4

xvii 

REPORT SUMMARY 

This report documents the verification and validation (V&V) of five selected fire models 

commonly used in support of risk-informed and performance-based (RI/PB) fire protection at 

nuclear power plants (NPPs). 

Background 

Over the past decade, there has been a considerable movement in the nuclear power industry to 

transition from prescriptive rules and practices towards the use of risk information to supplement 

decision-making. In the area of fire protection, this movement is evidenced by numerous 

initiatives by the U.S. Nuclear Regulatory Commission (NRC) and the nuclear community 

worldwide. In 2001, the National Fire Protection Association (NFPA) completed the 

development of NFPA Standard 805, “Performance-Based Standard for Fire Protection for Light 

Water Reactor Electric Generating Plants 2001 Edition.” Effective July, 16, 2004, the NRC 

amended its fire protection requirements in 10 CFR 50.48(c) to permit existing reactor licensees 

to voluntarily adopt fire protection requirements contained in NFPA 805 as an alternative to the 

existing deterministic fire protection requirements. RI/PB fire protection relies on fire modeling 

for determining the consequence of fires. NFPA 805 requires that the “fire models shall be 

verified and validated,” and “only fire models that are acceptable to the Authority Having 

Jurisdiction (AHJ) shall be used in fire modeling calculations.” 

Objectives 

The objective of this project is to examine the predictive capabilities of selected fire models. 

These models may be used to demonstrate compliance with the requirements of 10 CFR 50.48(c) 

and the referenced NFPA 805, or support other performance-based evaluations in NPP fire 

protection applications. In addition to NFPA 805 requiring that only verified and validated fire 

models acceptable to the AHJ be used, the standard also requires that fire models only be applied 

within their limitations. The V&V of specific models is important in establishing acceptable 

uses and limitations of fire models. Specific objectives of this project are: 

Perform V&V study of selected fire models using a consistent methodology (ASTM E1355) and 

issue a report to be prepared by U.S. Nuclear Regulatory Commission Office of Nuclear 

Regulatory Research (RES) and Electric Power Research Institute (EPRI). 

Investigate the specific fire modeling issues of interest to the NPP fire protection applications. 

Quantify fire model predictive capabilities to the extent that can be supported by comparison 

with selected and available experimental data. 

The following fire models were selected for this evaluation: (i) NRC’s NUREG-1805 Fire 

Dynamics Tools (FDTs), (ii) EPRI’s Fire-Induced Vulnerability Evaluation Revision 1 (FIVE-
xviii 

Rev. 1), (iii) National Institute of Standards and Technology’s (NIST) Consolidated Model of 

Fire Growth and Smoke Transport (CFAST), (iv) Electricite de France’s (EdF) MAGIC, and (v) 

NIST’s Fire Dynamics Simulator (FDS). 

Approach 

This program is based on the guidelines of the ASTM E1355, “Evaluating the Predictive 

Capability of Deterministic Fire Models,” for verification and validation of the selected fire 

models. The guide provides four areas of evaluation: 

Defining the model and scenarios for which the evaluation is to be conducted, 

Assessing the appropriateness of the theoretical basis and assumptions used in the model, 

Assessing the mathematical and numerical robustness of the model, and 

Validating a model by quantifying the accuracy of the model results in predicting the course of 

events for specific fire scenarios. 

Traditionally, a V&V study reports the comparison of model results with experimental data, and 

therefore, the V&V of the fire model is for the specific fire scenarios of the test series. While 

V&V studies for the selected fire models exist, it is necessary to ensure that technical issues 

specific to the use of these fire models in NPP applications are investigated. The approach 

below was followed to fulfill this objective. 

1. A set of fire scenarios were developed. These fire scenarios establish the “ranges of 

conditions” for which fire models will be applied in NPPs. 

2. The next step summarizes the same attributes or “range of conditions” of the “fire 

scenarios” in test series available for fire model benchmarking and validation exercises. 

3. Once the above two pieces of information were available, the validation test series, or 

tests within a series, that represent the “range of conditions” was mapped for the fire 

scenarios developed in Step 1. The range of uncertainties in the output variable of 

interest as predicted by the model for a specific “range of conditions” or “fire scenario” 

are calculated and reported. 

The scope of this V&V study is limited to the capabilities of the selected fire models. There are 

potential fire scenarios in NPP fire modeling applications that do not fall within the capabilities 

of these fire models and therefore are not covered by this V&V study. 

Results 

The results of this study are presented in the form of relative differences between fire model 

predictions and experimental data for fire modeling attributes important to NPP fire modeling 

applications, e.g., plume temperature. The relative differences sometimes show agreement, but 

may also show both under-prediction and over-prediction. These relative differences are 

affected by the capabilities of the models, the availability of accurate applicable experimental 

data, and the experimental uncertainty of this data. The relative differences were used, in 

combination with some engineering judgment as to the appropriateness of the model and the 

agreement between model and experiment, to produce a graded characterization of the fire 

model’s capability to predict attributes important to NPP fire modeling applications. 
xix 

This report does not provide relative differences for all known fire scenarios in NPP applications. 

This incompleteness is due to a combination of model capability and lack of relevant 

experimental data. The first can be addressed by improving the fire models while the second 

needs more applicable fire experiments. 

EPRI Perspective 

The use of fire models to support fire protection decision-making requires that their limitations 

and confidence in their predictive capability is well understood. While this report makes 

considerable progress towards that goal, it also points to ranges of accuracies in the predictive 

capability of these fire models that could limit their use in fire modeling applications. Use of 

these fire models present challenges that should be addressed if the fire protection community is 

to realize the full benefit of fire modeling and performance-based fire protection. This requires 

both short term and long term solutions. In the short term a methodology will be to educate the 

users on how the results of this work may affect known applications of fire modeling. This may 

be accomplished through pilot application of the findings of this report and documentation of the 

insights as they may influence decision-making. Note that the intent is not to describe how a 

decision is to be made, but rather to offer insights as to where and how these results may, or may 

not be used as the technical basis for a decision. In the long term, additional work on improving 

the models and performing additional experiments should be considered. 

Keywords 

Fire Fire Modeling Verification and Validation (V&V) 

Performance-based Risk-informed regulation Fire Hazard Analysis (FHA) 

Fire safety Fire protection Nuclear Power Plant 

Fire Probabilistic Risk Assessment (PRA) Fire Probabilistic Safety Assessment 

 (PSA) 

xxi 

PREFACE 

This report is presented in seven volumes. Volume 1, the Main Report, provides general 

background information, programmatic and technical overviews, and project insights and 

conclusions. Volumes 2 through 6 provide detailed discussions of the verification and validation 

(V&V) of the following five fire models: 

Volume 2 Fire Dynamics Tools (FDTs

) 

Volume 3 Fire-Induced Vulnerability Evaluation, Revision 1 (FIVE-Rev1) 

Volume 4 Consolidated Model of Fire Growth and Smoke Transport (CFAST) 

Volume 5 MAGIC 

Volume 6 Fire Dynamics Simulator (FDS) 

Finally, Volume 7 quantifies the uncertainty of the experiments used in the V&V study of these 

five fire models. 

xxiii 

FOREWORD 

Fire modeling and fire dynamics calculations are used in a number of fire hazards analysis (FHA) studies and 

documents, including fire risk analysis (FRA) calculations; compliance with, and exemptions to the regulatory 

requirements for fire protection in 10 CFR Part 50; the Significance Determination Process (SDP) used in the 

inspection program conducted by the U.S. Nuclear Regulatory Commission (NRC); and, most recently, the 

risk-informed performance-based (RI/PB) voluntary fire protection licensing basis established under 

10 CFR 50.48(c). The RI/PB method is based on the National Fire Protection Association (NFPA) Standard 

805, “Performance-Based Standard for Fire Protection for Light-Water Reactor Generating Plants.” 

The seven volumes of this NUREG-series report provide technical documentation concerning the predictive 

capabilities of a specific set of fire dynamics calculation tools and fire models for the analysis of fire hazards in 

nuclear power plant (NPP) scenarios. Under a joint memorandum of understanding (MOU), the NRC Office of 

Nuclear Regulatory Research (RES) and the Electric Power Research Institute (EPRI) agreed to develop this 

technical document for NPP application of these fire modeling tools. The objectives of this agreement include 

creating a library of typical NPP fire scenarios and providing information on the ability of specific fire models to 

predict the consequences of those typical NPP fire scenarios. To meet these objectives, RES and EPRI initiated 

this collaborative project to provide an evaluation, in the form of verification and validation (V&V), for a set of five 

commonly available fire modeling tools. 

The road map for this project was derived from NFPA 805 and the American Society for Testing and Materials 

(ASTM) Standard E1355-04, “Evaluating the Predictive Capability of Deterministic Fire Models.” These 

industry standards form the methodology and process used to perform this study. Technical review of fire 

models is also necessary to ensure that those using the models can accurately assess the adequacy of the scientific and 

technical bases for the models, select models that are appropriate for a desired use, and understand the levels 

of confidence that can be attributed to the results predicted by the models. This work was performed using 

state-of-the-art fire dynamics calculation methods/models and the most applicable fire test data. Future 

improvements in the fire dynamics calculation methods/models and additional fire test data may impact the results 

presented in the seven volumes of this report. 

This document does not constitute regulatory requirements, and RES participation in this study neither 

constitutes nor implies regulatory approval of applications based on the analysis contained in this text. The 

analyses documented in this report represent the combined efforts of individuals from RES and EPRI, both of 

which provided specialists in the use of fire models and other FHA tools. The results from this combined 

effort do not constitute either a regulatory position or regulatory guidance. Rather, these results are intended 

to provide technical analysis, and they may also help to identify areas where further research and analysis are 

needed. 

 _________________________________

 Carl J. Paperiello, Director 

 Office of Nuclear Regulatory Research 

 U.S. Nuclear Regulatory Commission

xxv 

ACKNOWLEDGMENTS 

The work documented in this report benefited from contributions and considerable technical support 

from several organizations. 

The verification and validation (V&V) studies for FDTs

 (Volume 2), CFAST (Volume 4), and 

FDS (Volume 6) were conducted in collaboration with the U.S. Department of Commerce, 

National Institute of Standards and Technology (NIST), Building and Fire Research Laboratory 

(BFRL). Since the inception of this project in 1999, the NRC has collaborated with NIST 

through an interagency memorandum of understanding (MOU) and conducted research to provide 

the necessary technical data and tools to support the use of fire models in nuclear power plant 

fire hazard analysis (FHA). 

We appreciate the efforts of Doug Carpenter and Rob Schmidt of Combustion Science 

Engineers, Inc. for their comments and contribution to Volume 2. 

In addition, we acknowledge and appreciate the extensive contributions of Electricité de France 

(EdF) in preparing Volume 5 for MAGIC. 

We also appreciate the efforts of organizations participating in the International Collaborative 

Fire Model Project (ICFMP) to Evaluate Fire Models for Nuclear Power Plant Applications, 

which provided experimental data, problem specifications, and insights and peer comment for 

the international fire model benchmarking and validation exercises, and jointly prepared the 

panel reports used and referred to in this study. We specifically appreciate the efforts of the 

Building Research Establishment (BRE) and the Nuclear Installations Inspectorate in the United 

Kingdom, which provided leadership for ICFMP Benchmark Exercise (BE) #2, as well as 

Gesellschaft fuer Anlagen-und Reaktorsicherheit (GRS) and Institut fuer Baustoffe, Massivbau 

und Brandschutz (iBMB) in Germany, which provided leadership and valuable experimental data 

for ICFMP BE #4 and BE #5. In particular, ICFMP BE #2 was led by Stewart Miles at BRE; 

ICFMP BE #4 was led by Walter Klein-Hessling and Marina Rowekamp at GRS, and R. 

Dobbernack and Olaf Riese at iBMB; and ICFMP BE #5 was led by Olaf Riese and D. Hosser at 

iBMB, and Marina Rowekamp at GRS. We acknowledge and sincerely appreciate all of their 

efforts. 

We greatly appreciate Paula Garrity, Technical Editor for the Office of Nuclear Regulatory 

Research, and Linda Stevenson, agency Publication Specialist, for providing editorial and 

publishing support for this report. We also greatly appreciate Dariusz Szwarc, Nuclear Safety 

Professional Development Program participant, for his assistance finalizing this report. 

xxvii 

LIST OF ACRONYMS 

AGA American Gas Association 

AHJ Authority Having Jurisdiction 

ASME American Society of Mechanical Engineers 

ASTM American Society for Testing and Materials 

BE Benchmark Exercise 

BFRL Building and Fire Research Laboratory 

BRE Building Research Establishment 

CFAST Consolidated Fire Growth and Smoke Transport Model 

CFR Code of Federal Regulations

EdF Electricité de France 

EPRI Electric Power Research Institute 

FDS Fire Dynamics Simulator 

FDTs Fire Dynamics Tools (NUREG-1805) 

FHA Fire Hazard Analysis 

FIVE-Rev1 Fire-Induced Vulnerability Evaluation, Revision 1 

FM-SNL Factory Mutual & Sandia National Laboratories 

FPA Foote, Pagni, and Alvares 

FRA Fire Risk Analysis 

GRS Gesellschaft fuer Anlagen-und Reaktorsicherheit (Germany) 

HRR Heat Release Rate 

IAFSS International Association of Fire Safety Science 

iBMB Institut für Baustoffe, Massivbau und Brandschutz 

ICFMP International Collaborative Fire Model Project 

IEEE Institute of Electrical and Electronics Engineers 

MCC Motor Control Center 

MQH McCaffrey, Quintiere, and Harkleroad 
xxviii 

MOU Memorandum of Understanding 

NBS National Bureau of Standards (now NIST) 

NFPA National Fire Protection Association 

NIST National Institute of Standards and Technology 

NPP Nuclear Power Plant 

NRC U.S. Nuclear Regulatory Commission 

NRR Office of Nuclear Reactor Regulation (NRC) 

RES Office of Nuclear Regulatory Research (NRC) 

RI/PB Risk-Informed, Performance-Based 

SDP Significance Determination Process 

SFPE Society of Fire Protection Engineers 

V&V Verification & Validation
1-1 

1 

SELECTION OF DATA FOR MODEL EVALUATION 

The methodology employed in this study follows the guidelines outlined in ASTM E 1355, 

Standard Guide for Evaluating the Predictive Capability of Deterministic Fire Models [Ref. 1]. 

One of the stated goals of such a study is to “Quantify the uncertainty and accuracy of the model 

results in predicting the course of events in similar fire scenarios.” The subject of model 

accuracy is addressed in this volume through evaluation of the models. To accomplish the 

evaluation, the discussion in this volume focuses on experimental uncertainty. 

The objective of this volume is not to provide a comprehensive description of the selected fire 

experiments. That information can be obtained in the original test reports, which are cited in the 

text. Rather, this volume serves as a link between the experiments and the models, especially 

with regard to experimental uncertainties, which are typically not reported. Also, certain 

parameters required as input by the fire models, like the radiation loss from the fire, are often not 

provided in the original test reports, because the information has not been measured. This 

document provides information that cannot be found in the original test reports for implementing 

the models and comparing the results to the experimental measurements. 

1.1 Fire Experiments and Test Selection 

This volume contains descriptions of the six sets of fire experiments that are being used in the 

evaluation of the five models considered in this report series. A number of the experiments were 

designed for the V&V studies, which were part of a series of Benchmark Exercises (BE) of the 

International Collaborative Fire Model Project (ICFMP) and a number were selected from the 

literature, based on the appropriateness of the data (FM/SNL, NBS). 

In general, the experiments established steady fires burning in simple compartment geometries. 

The decision to include or exclude a particular test from a particular experimental test series as 

part of this validation study was made for a variety of reasons. Table 1-1 summarizes the 

experiments selected for the validation study in terms of the number of tests and the number of 

measurement types, as well as aspects of the fire and the compartment, including the fire heat 

release rate (Q& ), and the compartment volume (V) and height (H). 
Selection of Data for Model Evaluation 

1-2 

Table 1-1: Overview of the Experiments Used for Model Evaluation 

Series 

Number of 

Tests 

Number of Types of 

Measurements 

Q&

(kW) 

V

(m3) 

H 

(m) 

FM/SNL 3 4 500 1400 6.1 

NBS 3 2 100 15 2.4 

ICFMP BE #2 3 2 1800-3600 5900 19 

ICFMP BE #3 15 8 400-2300 580 3.8 

ICFMP BE #4 1 3 3500 74 5.7 

ICFMP BE #5 1 4 400 73 5.6 

The six fire experiments are introduced below, approximately in chronological order: 

1. FM/SNL Test Series: 25 fire experiments conducted for the US NRC by Factory Mutual 

Research Corporation (FMRC) in 1985, under the direction of Sandia National Laboratories 

(SNL) [Ref. 2]. The primary purpose of these tests was to provide data with which to evaluate 

fire models used in hazard assessments of NPP enclosures. The results of three of these 

experiments have been used in the current validation study. 

The FM/SNL series involved a large number of measurements made during a long test series, 

but much of the data was never thoroughly analyzed. In particular, uncertainties were not 

provided. The three experiments selected were described in greater detail than the others, 

and these same three experiments have been used in a variety of prior validation studies. 

Estimates of uncertainty were determined from the test reports, the data itself, and supporting 

information, including conversations with one of the experimentalists. 

2. NBS Multi-Compartment Test Series: 45 fire experiments, representing 9 different sets of 

test conditions, with multiple replicates of each set, that were conducted in a three-room suite 

at the National Bureau of Standards in 1985 (NBS, now the National Institute of Standards and 

Technology or NIST) [Ref. 3]. The primary purpose of these experiments was to evaluate fire 

models under development at the NBS at that time, in particular, the zone model CFAST. The 

results of three of these experiments have been used in the current validation study. Estimates 

of uncertainty were determined from the test reports, the data itself, and supporting 

information. 

3. ICFMP Benchmark Exercise (BE) #2: A series of 8 fire experiments, representing 3 different 

sets of test conditions, that were conducted within a 19 m high test hall at VTT, the Finnish 

National Testing Laboratory during 1998 and 1999 [Ref. 4]. The test results were 

contributed to the International Collaborative Fire Model Project (ICFMP). All three cases, 

representing averaged results from the 8 tests, have been used in the current validation study. 

4. ICFMP BE #3: A series of 15 fire experiments were conducted at NIST in 2003, which were 

partially funded by the U.S. NRC and NIST as part of the ICFMP. The results of all the 

experiments have been used in the current V&V study. All of the BE #3 experiments were 
Selection of Data for Model Evaluation 

1-3 

used. Re-analysis of the heat release rate data led to a revised report [Ref. 5], which included 

an expanded explanation of the heat release rate uncertainty estimate. 

5. ICFMP BE #4: A series of small compartment kerosene pool fire experiments, conducted at the 

Institut für Baustoffe, Massivbau und Brandschutz (iBMB) of Braunschweig University of 

Technology in Germany in 2004 [Ref. 6]. The results of two of these experiments were 

contributed to the ICFMP, of which one has been used in the current V&V study. A 

malfunction in the measurement of the fuel mass loss rate was reported in Test 3, implying a 

level of uncertainty that is unacceptable for the current V&V study. The results from Test 1 

were considered here. 

6. ICFMP BE #5: A series of fire experiments in 2004 that involved realistically routed cable 

trays inside the same concrete enclosure at iBMB as BE #4 [Ref. 7]. The results of four tests 

were contributed to the ICFMP, of which one has been used in the current V&V study. 

BE #5 was conducted primarily for the evaluation of cable ignition and flame spread. The 

results were erratic, and no replicate experiments were performed. Given the primitive 

nature of the ignition and spread algorithms within the models, it was decided that only a 

qualitative analysis would be possible with the data from three of the four experiments. 

However, in one experiment, the first 20 min involved a fairly well-characterized ethanol 

pool fire burning on the opposite side of the compartment from the cable tray. This part of 

the experiment has been used as part of the model evaluation. 

1.2 Measurement Selection 

In addition to selecting individual experiments from the six test series, specific measurements 

were selected for use in the model evaluation. Section 2.4 of Volume 1 describes the rationale 

for the selection of these parameters, which was based on the need for fire safety in nuclear 

power plant applications. Because of the time and expense involved in conducting large-scale 

fire experiments, there is often a desire by the test laboratory to include as many measurements 

as possible. Unfortunately, much of the data goes unanalyzed, either because only a relatively 

small subset of the data is actually needed by the sponsoring organization, or because the budget 

for the project is exhausted in the preparation and execution of the experiments. In any event, 

there is a tremendous amount of fire test data that has never been thoroughly analyzed or used 

for V&V. Chapter 3 of this volume, which describes the experiments, provides some details 

about the decision to use, or exclude, a given measurement from the evaluation. This study 

evaluates the model calculations for the following 13 experimental parameters: 

• Hot gas layer temperature 

• Hot gas layer height 

• Ceiling jet temperature 

• Plume temperature 

• Flame height 

• Oxygen concentration 

• Smoke concentration 
Selection of Data for Model Evaluation 

1-4 

• Compartment pressure 

• Radiated heat flux to target 

• Total heat flux to target 

• Target temperature 

• Total heat flux to walls 

• Wall temperature 

Table 1-2, taken from Table 2-3 of Volume 1 of this report, summarizes the measurements from 

each set of experiments that were used for comparison with the models. About 45 of the 78 (or 

60 %) of the cells in Table 1-2 have entries that say, “no data,” which means that either the 

measurement was not conducted, or that the data was flawed or otherwise suspect. Some of the 

tests provided more information than others, but none of the tests provided information for all of 

the parameters of interest. The graphical comparisons of measured and calculated results for each 

of the five fire models are presented in Appendix A of Volumes 2 through 6. 
Selection of D

Table 1-2: Summary of the Test Series, Experiments, and Instrumentation Used for the Model Eval

Selected Test Series/Experiments/Instrumentation 

Fire Modeling FM/SNL NBS ICFMP BE #2 ICFMP BE #3 ICFMP B

Parameters Tests 4,5, & 21 100A, 100O, 100Z Part I, Cases 1, 2, 3 Tests 1-5, 7-10, 13-18 Test 1 

1. HGL temperature Vertical arrays of thermocouples Vertical arrays of thermocouples Vertical arrays of thermocouples Vertical arrays of thermocouples Vertical arra

thermocoup

2. HGL depth Vertical arrays of thermocouples Vertical arrays of thermocouples Vertical arrays of thermocouples Vertical arrays of thermocouples Vertical arra

thermocoup

3. Ceiling jet 

temperature Thermocouple No Data No Data Thermocouple No Data 

4. Plume 

temperature Thermocouple No Data Thermocouple No Data No Data 

5. Flame height No Data No Data Photos Photos No Data 6. Gas 

concentrations No Data No Data No Data Oxygen meter No Data 

7. Smoke 

concentration No Data No Data No Data Laser transmission No Data 

8. Compartment 

pressure 

No Data No Data No Data Differential transducer No Data 

9. Radiant heat flux 

to target No Data No Data No Data Radiometers No Data 

10. Total heat flux 

to targets No Data No Data No Data Heat Flux Gauges Heat Flux G

11. Target surface 

temperature No Data No Data No Data Thermocouples Thermocoup

12. Total heat flux 

to walls No Data No Data No Data Heat Flux Gauges No Data 

13. Wall surface 

temperature No Data No Data No Data Thermocouples Thermocoup
Selection of Data for Model Evaluation 

1-6 

1.3 Fire Models 

Section 2.2 of Volume 1 describes the selection of the five fire models considered in this study. 

These particular models were chosen based on the fact that most of them have been used fairly 

extensively to calculate fire conditions in NPP fire protection applications, or were developed by 

stakeholders within the nuclear industry for NPP fire protection applications. The models 

represent a wide range of capabilities and mathematical and computational sophistication. The 

models are: 

• Two libraries of engineering calculations: FDTs and FIVE-Rev1 

• Two two-zone models: CFAST and MAGIC 

• One field model: FDS 

Volumes 2 to 6 of this report describe these models in detail. Table 2-2 of Volume 1 lists the 

output provided by each of the fire models. While two of the models provide output for all of the 

13 parameters listed above, the other three models considered in this study do not. 

1.4 Uncertainty and Sensitivity 

According to ASTM E1355 [Ref. 1], “Uncertainties in model inputs can lead to corresponding 

uncertainties in the model outputs. Sensitivity analysis is used to quantify these uncertainties in 

the model outputs based upon known or estimated uncertainties in model inputs.” This section 

provides a discussion of the various forms of uncertainty that play a role in the model evaluation 

process. In the Appendices of Volumes 2 through 6, there are hundreds of graphs comparing 

time histories of experimental measurements and model predictions. Rarely do the curves lie 

exactly on top of each other, which raises the question as to how well did the models predict the 

experimental measurements. There are numerous methods of quantifying the “closeness” of the 

agreement. The degree to which the curves ought to match, however, depends on the application. 

In practice, each model user may have a different application. For example, knowing the gas 

temperature within a few degrees might be important if the application is detection, but an order 

of magnitude larger if the interest is the prediction of flashover. Fire protection engineers 

performing a hazard analysis are often content to demonstrate merely that the model is 

consistently “conservative”; that is, that a safety factor is implicit in the model formulation. 

Forensic experts, however, require the model to be as accurate as possible, with no built-in bias. 

In either case, model accuracy needs to be quantified. This means comparing model predictions 

to experimental measurements, as is done in the Appendices of Volumes 2 through 6, and then 

quantifying the differences between the two. The agreement between measurements and models 

is considered here in terms of the combined measurement and model input uncertainties. 

1.4.1 Types of Uncertainty 

The model evaluation process considers experimental uncertainty in two ways: (1) the sensitivity 

of the modeling results to uncertainty associated with input derived from experimental 

measurements, and (2) the uncertainty associated with the experimental measurement of the 

parameters of interest, that is, those parameters that are model output. 
Selection of Data for Model Evaluation 

1-7 

The former type of uncertainty is referred to here as model input uncertainty. This uncertainty 

does not represent model uncertainty in its totality. For example, the intrinsic uncertainty of a 

model is not accounted for by the model input uncertainty. That is, the physical and 

mathematical assumptions and methods that are an intrinsic part of the model formulation and its 

implementation are not part of the model input uncertainty. Examples of the intrinsic model 

uncertainty are the two-layer assumption in a zone fire model, the description of turbulence in a 

CFD fire model, or the grid size used in a CFD fire model. The model input uncertainty is related 

to the uncertainty of the input parameters to a model. Physical quantities like the thermal 

properties of the solid surfaces, chemical properties of the fuels, yields of the various products of 

combustion, and most importantly, the heat release rate of the fire, all have a certain degree of 

uncertainty. The uncertainty in the model output results are related to the uncertainty of each of 

the input parameters. 

Beyond the input uncertainty, uncertainty associated with the experimental measurements is also 

considered in the model evaluation process. Measurements by thermocouples, heat flux gauges 

and gas analyzers all have a certain degree of uncertainty related to their operation, calibration, 

etc. This is referred to as measurement uncertainty. A measurement result is fully documented 

only when accompanied by a quantitative statement of its uncertainty. The uncertainty of 

experimental measurement results are typically broken into two types of uncertainty: instrument 

uncertainty and repeatability [Refs. 8, 9]. When the components of the measurement uncertainty 

are quantified, they are pooled into a combined uncertainty value that is a better representation of 

the total measurement uncertainty. The uncertainty is often expressed in terms of an expanded 

uncertainty, in which the confidence level that the measurement result falls within the expanded 

bounds is high. For an expansion factor of two, the uncertainty is related to two standard 

deviations (2·σ) and the confidence level corresponds to 95 %. Refs. [8, 9] discuss the various 

types of measurement uncertainty and ways to quantify them. 

Typically, it is possible to provide rational estimates of the experimental measurement 

uncertainty and the experimental model input uncertainty. Both are related to measurements. 

The model intrinsic uncertainty, however, is far more difficult to quantify. Intrinsic uncertainty

may have many components that may be coupled to each other. The interplay of various 

algorithms within a fire model, especially a CFD fire model, make it virtually impossible to 

dissect the various pieces, assess their uncertainty, and then combine the result into a single 

number. In this report, the fire model is used to simulate the experiments, and then the effects of 

measurement uncertainty and the model sensitivity to model input uncertainty are considered as 

possible sources of the difference between the model calculation results and the measurements. 

As an example, Figure 1-1 shows the measured and the simulated temperature as a function of 

time for a hypothetical scenario. For simplicity, measurements and model results are compared at 

the time of their respective maximum values in this report. In the case shown in Figure 1, the 

maxima occur nearly 600 s after ignition, albeit at somewhat different times. The uncertainty 

bars in Figure 1-1 do not overlap near 600 s, and there are significant differences between the 

peak value (Mp) of the model prediction and the peak value of the experimental measurement 

(Ep). The measurement uncertainty and the sensitivity to model input uncertainty are also shown 

in Figure 1-1 as a function of time, and are represented as a set of circles. The purpose of this 

volume is to determine how much of the relative difference between the models and 

measurements can be attributed to a combination of the measurement and model input 

uncertainties. 
Selection of Data for Model Evaluation 

1-8 

1.4.2 Methodology for Evaluating the Models 

The process of deducing the predictive capability of the model from the quantitative results is 

documented in Volumes 1 to 6 of this report series. Appendix A in each volume contains 

detailed comparisons of model predictions and experimental measurements. Two criteria are 

used to characterize the predictive capability of the fire models. The criteria are described in 

Section 2.6.2 of Volume 1. Chapter 6 in Volumes 2 to 6 describes how the quantitative results 

were used to arrive at the characterization of predictive capability for each of the models under 

consideration. 

In the approach used in Volumes 2 to 6 of this report, model predictions and measurements are 

compared in terms of the relative difference between their maximum values: 

( ) ( )

( ) p o

p o p o

E E

M M E E

E

M E

−

− − − = ∆

∆ − ∆

ε = (1.1) 

where ∆M is the difference between the peak value of the model prediction (Mp) and its baseline 

value (Mo), and ∆E is the difference between the peak value of the experimental measurement 

(Ep) and its baseline value (Eo). The parameter ε represents the relative difference between 

model predictions and measurements. In this study, the significance of the value ofε is 

considered through comparison with the combined relative expanded (2σ) uncertainty of ε , 

denoted as Uε . The measurement and model results and their uncertainties are graphically 

represented in Fig. 1-1. The values of ∆M and ∆E in Fig. 1-1 are about 220 oC and 280 oC, 

respectively, while the relative expanded measurement and model input uncertainties, Ue and 

Um, are about ±10 % and ±20 %, about the respective peaks. 

Propagation of error for ε in Eq. 1.1 leads to an expression for the absolute combined expanded 

(2σ) uncertainty [Ref. 8], which is given by: 

Uε = (ε + 1) ( Ue

 2 + Um 2 ) 1⁄2 (1.2) 

where Ue

 is the relative expanded (2σ) measurement uncertainty and Um is the relative expanded 

(2σ) model input uncertainty. This formulation considers the functional form of Eq. 1.1. 

Because ε is typically small, the term (ε + 1) is approximately one, so that: 

Uε ≈ ( Ue

 2 + Um 2 ) 1⁄2 (1.3) 

Quantification of the measurement (Ue) and model input (Um) uncertainties is discussed in the 

chapters that follow. 

The statistical normal significance test considers the difference of the mean values with known 

and unequal variances to be in agreement at a 95 % confidence level [Ref.10] if: 

ε = ≤ Uε (1.4) 

where the distributions for Ue and Um (about the mean values, ∆E and ∆M, respectively) are 

assumed to be Gaussian. The examination associated with Eq. 1.4 is analogous to asking if the 

expanded uncertainty bars in Figure 1.1 overlap. If Eq. 1.4 does not hold, then the difference 

between the model and measurement are not within the bounds of the combined expanded 
Selection of Data for Model Evaluation 

1-9 

uncertainty,Uε . This may be due to a variety of reasons including issues not considered in the 

uncertainty analysis, such as measurement bias, uncharacterized experimental uncertainty, 

inappropriate application of a model, or intrinsic model uncertainty. 

The hypothetical data shown in Fig. 1-1 and the data from all but one of the experimental series 

was time-averaged over a 10 s interval for use in this V&V study. The data in the NBS data set 

was acquired every 10 s with a 6 s time-average. A nearly-uniform value for the time-averaging 

interval establishes consistency among the various experimental data sets. The value of this 

interval is selected based on a desire to reduce the impact of experimental noise on the 

determination of ε (see Eq. 1.1), and the need to minimize data smoothing without loss of 

significant information. Appendix B provides example calculations regarding this issue. 

0

50

100

150

200

250

300

350

0 200 400 600 800 1000 1200

Temperature (oC)

Time (s)

measurement

model

Figure 1-1: Comparison of hypothetical measured and simulated temperature profiles. 

Uncertainties associated with the measurements and model inputs are also shown. 

1.5 Organization of this Volume 

This volume is organized as follows: 

• Chapter 2 defines the heat release rate and discusses uncertainty in its measurement, 

which is important the determination of the model input uncertainty (Um, see Eq. 1.3), 

which is discussed in Chapter 5. 

• Chapter 3 discusses each of the six experimental series considered in this report (see 

Table 1-1). 
Selection of Data for Model Evaluation 

1-10 

• Chapter 4 addresses the uncertainty in the experimental measurements, which is 

important in model evaluation and the determination of the combined uncertainty (Ue, see 

Eq. 1.3) 

• Chapter 5 discusses the value of the model input uncertainty (Um) for each of the 

measurements. For each parameter of interest, a simple analytic description of the 

sensitivity of that parameter to the fire heat release rate is given. The model input 

uncertainty (Um) is estimated based on uncertainty in the experimental heat release rate, 

and a number of other parameters. The magnitude of this uncertainty plays an important 

role in the evaluation of model accuracy through consideration of Eq. 1.4 above. 

• Chapter 6 summarizes the experimental (Ue) and model input (Um) uncertainties taken 

from Chapters 4 and 5, respectively, and uses these values to calculate the combined 

uncertainty (see Eq. 1.3). The combined uncertainties are weighted and used to 

determine a representative uncertainty for comparison with ε , the relative difference 

between the measurements and the models. 

Chapter 7 presents a list of references. 

Appendix A provides documentation of personal communications cited in the text. 

Appendix B presents a series of calculations addressing the interval used to time-average the 

data 
2-1 

2 

MODEL INPUT AND ITS UNCERTAINTY 

This chapter discusses model input and its uncertainty. Attention focuses primarily on the heat 

release rate (HRR), its measurement, and its uncertainty. The uncertainty in the HRR turns out 

to be the most significant model input uncertainty. The fire HRR is the single most important 

parameter in terms of characterizing a fire. The HRR controls the thermal impact of a fire on its 

environment. The current generation of fire models cannot accurately predict the transient HRR 

of a fire and for the fire model evaluations considered in this report, the HRR is prescribed. The 

sensitivity of model output to the HRR uncertainty is an important part of the model evaluation 

in this report series and is discussed in detail in Chapter 5. 

The actual HRR of a fire (Q& ) is a function of the mass loss rate of fuel ( m& ), the heat of 

combustion (Hc), and the combustion efficiency (χA): 

Q& = χA m& Hc (2.1) 

In some fire models, the radiative fraction of a fire is a model input. Since the radiative fraction 

was not reported in the documentation of the experiments considered in this report, this chapter 

suggests values for use in the model evaluation considered in this report series. The radiative 

fraction provides information on how the energy from a fire is distributed between the sensible 

enthalpy convected by the plume and the heat transferred to the surroundings by radiation. The 

radiative fraction, χrad, is based on the idealized HRR of a fire (m& Hc) and is defined as: 

Qr

& = χrad m& Hc (2.2) 

whereQr

& is the radiative emission from a fire to its surroundings. As used in the fire literature 

(e.g., Ref. 11), the radiative fraction of a fire does not consider radiative exchange with walls or a 

hot upper layer. 

2.1 Heat Release Rate Measurement Uncertainty 

In the experiments considered in this report the heat release rate was determined by measuring 

the mass loss rate of fuel or by an oxygen consumption calorimetry measurement. 

Calorimetric measurements typically involve dozens of independent measurements [Ref. 11], 

and a combined uncertainty analysis can be quite elaborate. Measurement uncertainty depends 

on the exact instrument types, details of the flow and sampling hardware, experimental 

procedures, and application details. Ref. [11] provides a survey of the few studies that address 

uncertainty of the HRR measurement by oxygen consumption calorimetry. These studies 

considered completely different situations including various types of instrumentation, different 

levels of the HRR, and detail of the analysis, including several versions of the HRR equation. 

Ref. [12] estimated the HRR measurement uncertainty for the measurement as conducted in the 

Single Burning Item Test and the Room Corner Test (ISO 9705). The relative expanded 
Model Input and Its Uncertainty 

2-2 

uncertainty estimates ranged from 0.07 to 0.14, depending on the apparatus and the HRR. The 

uncertainty of the oxygen concentration measurement, followed by the heat of combustion factor 

and the mass flow rate measurement were identified as the major sources of uncertainty. Details 

on the uncertainty of the oxygen and the mass flow rate measurements were presented. The study 

notes that for larger oxygen deficits the combined uncertainty of the HRR measurement is less. 

Ref. [13] performed an analytical estimate of the HRR measurement uncertainty for the cone 

calorimeter. They estimated relative expanded uncertainty values from 0.10 to 0.12, depending 

on the HRR. The greatest sources of uncertainty were identified as the heat of combustion factor, 

the combustion expansion factor, and the oxygen measurement. Ref. [14] performed an 

uncertainty analysis for experiments conducted in a compartment with a controlled energy 

supply. The volume flow rate and oxygen measurement were identified as majors source of 

uncertainty, and a relative expanded uncertainty of 0.12 was reported under conditions when the 

exhaust volume flow rate was optimized for the fire size. The oxygen depletion measurement 

and the exhaust mass flow rate measurement have been consistently identified as major sources 

of uncertainty. Ref. [15] reviewed, however, the results of the HRR round-robin tests, and 

highlighted the uncertainty due to random effects, such as material and burning variability, 

environmental effects, operator error, and measurement bias between laboratories. 

For some of the experiments considered here, the HRR was estimated through measurement of 

the fuel flow rate. In these cases, the HRR was calculated based on the heat of combustion and 

an assumed combustion efficiency (Eq. 2.1). While the mass flow rate measurements typically 

have low uncertainties, the uncertainty in the combustion efficiency is not necessarily small. 

Inside a compartment, even less is known about combustion efficiency as the fire plume is 

partially engulfed in a hot upper layer and the oxygen volume fraction in the lower layer is 

vitiated. 

2.2 Radiative Fraction 

The radiative fraction is typically determined through single or multiple location measurements 

of the radiative flux, q&′′, at a significant distance from the fire in conjunction with a 

measurement of the mass burning rate, m& . Assuming symmetry, χrad can be determined either 

through single or multiple-location heat flux measurements at a suitable distance from the fire 

source. The radiative fraction of burning pools of hydrocarbons, such as those used in the 

experiments considered here, vary with fuel type [Ref. 16, 17] and scale of the fire [Ref. 18]. 

Measurement of the radiation from a fire should avoid reflection or re-radiation from nearby 

walls or partitions. 

2.3 Summary of Results from the Experiments 

The values of the HRR varied from experiment to experiment, depending on the nature of the 

experimental conditions. This information is available in the test reports [Refs. 2 through 6] and 

is summarized in Table 2-1. Table 2-1 also presents the values of the estimated uncertainty of 

the HRR for the six experiments, which varied from 15 % to 25 %. For two of the experiments 

considered here (BE #2 and BE #4), the HRR was estimated through the fuel flow rate 

measurement results. In these cases, the HRR was calculated based on Eq. 2.1 and data on the 

heat of combustion and the combustion efficiency. Whereas the mass flow rate measurement 

typically has low uncertainty, the uncertainty in the combustion efficiency is not necessarily 
Model Input and Its Uncertainty 

2-3 

small. Inside a compartment, even less is known about combustion efficiency as the fire plume 

may be partially engulfed in a hot upper layer and the oxygen volume fraction in the lower layer 

may be vitiated. The table also lists values of the radiative fraction, and its uncertainty. 

Discussion of the table entries is given in detail in Chapter 3 for each of the six experimental test 

series. 

Table 2-1: Summary of the Heat Release Rate and Radiative Fraction 

Series Test Fire Peak HRR (kW) Radiative Fraction

Test 4 510 ± 20 % 

Test 5 480 ± 20 % 

FM/SNL 

Ref. [2] 

Test 21 

Propylene 

0.9 m round sand burner 

470 ± 20% 

0.35 ± 20 % 

100A natural gas 110 ± 15 % 0.20 ± 20 % 

100O 110 ± 20% 0.30 ± 30 % 

NBS 

Ref.[3] 

100Z 

natural gas/acetylene 

gas burner 110 ± 20 % 0.30 ± 30 % 

Case 1 heptane on water 

 round 1.17 m pan 

Case 2 

BE #2 

Ref. [4] 

Case 3 

heptane on water 

round 1.60 m pan 

Mass loss 

Uncertainty: ± 15% 

0.35 ± 20 % 

BE #3 

Ref. [5] 

15 tests heptane & toluene (1 test) spray 

round, 1 m ± 0.1 m diameter pool 

400-2300; see Ref.[5] 

± 17 % uncertainty 

heptane: 0.44 ± 16 % 

toluene: 0.40 ± 23 % 

BE #4 

Ref. [6] 

Test 1 jet fuel Mass loss 

Uncertainty: ± 25% 

0.35 ± 20 % 

BE #5 propane gas burner 0.35 ± 20 %

Ref. [7]

Test 4 

ethanol pan fire 

see Fig. 3-9 

Uncertainty: ± 15% 0.20 ± 20 %

3-1 

3 

TEST SERIES 

In this chapter, each of the six experimental series is described and certain measurements are 

discussed, most notably, the heat release rate (HRR) and the radiative fraction. The HRR and the 

radiative fraction varied from experiment to experiment, as did the uncertainty of these 

parameters. This information is summarized in Table 2-1 in Chapter 2. 

Because measurement uncertainty was not documented for many of the experiments, engineering 

judgment is used, in this and the following chapter, to estimate its value. Measurements vary 

from experiment to experiment, and for each attribute being measured. The accurate 

determination of experimental uncertainty is challenging, and characterizing the uncertainty in 

experiments conducted by others is even more so. A good faith effort is made here to quantify 

measurement uncertainty, but the uncertainty determination provided in this document should be 

regarded as estimates and the uncertainty bounds presented here should be regarded as 

guidelines, to assist in the evaluation of the predictive capabilities of the models. Some factors 

that contribute to experimental uncertainty were not considered here, but may be important. For 

measurements, systematic error may have been present, but may have been unidentified. Human 

error is always present in the implementation of instrumentation and interpretation of 

measurement results. In this sense, it is recognized that the uncertainty values presented here are 

not necessarily all-inclusive or definitive. This highlights the importance of expert judgment in 

this study and in the interpretation of the agreement between measurements and models. 

3.1 FM/SNL Test Series 

The Factory Mutual and Sandia National Laboratories (FM/SNL) Test Series was a series of 25 fire 

tests conducted for the NRC by Factory Mutual Research Corporation (FMRC), under the 

direction of Sandia National Laboratories (SNL). The primary purpose of these tests was to 

provide data with which to validate computer models for various types of NPP compartments. The 

experiments were conducted in an enclosure measuring 60 ft long x 40 ft wide x 20 ft high (18 m x 

12 m x 6 m), constructed at the FMRC fire test facility in Rhode Island. All of the tests involved 

forced ventilation to simulate typical NPP installation practices. The fires consisted of a simple 

gas burner, heptane or methanol liquid pools, or a polymethylmethacrylate (PMMA) solid fire. 

Four of the tests were conducted with a full-scale control room mockup in place. Parameters 

varied during the experiments included fire intensity, enclosure ventilation rate, and fire location. 

The FM/SNL test series is described in detail, including the types and locations of measurement 

devices, as well as some results in Refs. [2, 19, 20]. 

This V&V study used data from only three of the experiments (Tests 4, 5, and 21). In these tests, 

the fire source was a propylene gas-fired burner with a diameter of approximately 0.9 m (36 in), 

with its rim located approximately 0.1 m (4 in) above the floor. For Tests 4 and 5, a round 0.3 m 

(1 ft) diameter burner was centered along the longitudinal axis centerline, 6.1 m (20 ft) laterally 

from the nearest wall. For Test 21, the fire source was placed within a simulated benchboard 

electrical cabinet. 
Test Series 

3-2 

3.1.1 Heat Release Rate 

The HRR was determined using oxygen consumption calorimetry in the exhaust stack with a 

correction applied for the CO2 in the upper layer of the compartment. The uncertainty of the fuel 

mass flow was not documented. All three tests selected for this study had the same target peak 

heat release rate (HRR) of 516 kW following a 4 min “t-squared” growth profile. The test report 

contains time histories of the measured HRR, for which the average sustained HRR following 

the ramp up for Tests 4, 5, and 21 have been estimated as 510 kW, 480 kW, and 470 kW, 

respectively. Once reached, the peak HRR was maintained essentially constant for another 6 min 

(Tests 4 and 5) and 16 min (Test 21). Figure 3-1 shows the prescribed and measured HRR as a 

function of time during Test 21 of the FM/SNL test series. Even though time-dependent curves 

are reported in Ref. [20], it was decided to use averaged values during the steady phase, because 

it is assumed that the fuel flow would have been held steady during this period, and that 

fluctuations in HRR are expected from calorimetry measurements. It was noted in the test report 

that during Tests 4, 5, and 21 there was a downward bias in the measured HRR due to 

“significant” loss of effluent from the exhaust hood. This bias was treated as an additional 

uncertainty, and the relative combined expanded uncertainty was assumed to equal ±20 %, which 

is somewhat larger than typical calorimetric measurement uncertainty. 

3.1.2 Radiative Fraction 

The radiative fraction (see definition in Chapter 2) was not measured for propylene, but it is 

assumed to equal 0.35 which is typical for smoky hydrocarbons [Ref. 17]. The expanded 

uncertainty in this value was taken as ± 20 %, a value typical of reported uncertainty [21, 22]. It 

was further assumed that the radiative fraction was about the same in Test 21 as the other tests, 

as fuel burning must have occurred outside of the electrical cabinet in which the burner was 

placed. 
Test Series 

3-3 

Heat Release Rate

FM/SNL, Test 21

Time (min)

0 5 10 15 20 25 30

Heat Release Rate (kW)

0

100

200

300

400

500

600

Prescribed

Calorimetry

Figure 3-1: Prescribed and measured heat release rate as a function of time during Test 21 

of the FM/SNL test series 

3.1.3 Other Measurements 

As seen in Table 1-2, four types of measurements were conducted during the FM/SNL test series 

and used in the evaluation conducted here, including the HGL temperature and depth, and the 

temperatures of the ceiling jet and the plume. Aspirated thermocouples were used to make all of 

these measurements. The aspirated thermocouple (TC) measurements presented in this study are 

preferable to bare-bead TC measurements, as radiative exchange measurement errors are 

reduced. For the relatively low temperatures observed (~100 o

C), however, the differences are 

expected to be small. 

Aspirated Thermocouples: Aspirated thermocouple measurements for the range of temperatures 

measured are typically accurate to about 10 °C (13 °F ); see the discussion of thermocouple 

uncertainty in Chapter 4. The temperatures were measured using the aspirated thermocouples in 

Sectors 1, 2 and 3, plus the near-ceiling TCs directly above the burner in Test 4 and 5. 

HGL Depth and Temperature: Data from all of the vertical thermocouple trees were used when 

reducing the HGL height and temperature. For the FM/SNL Tests 4 and 5, Sectors 1, 2 and 3 

were used, all weighted evenly. For Test 21, Sectors 1 and 3 were used, evenly weighted. 

Sector 2 was partially within the plume in Test 21. 
Test Series 

3-4 

3.2 NBS Multi-Compartment Test Series 

The National Bureau of Standards (NBS, which is now the National Institute of Standards and 

Technology, NIST) Multi-Compartment Test Series consists of 45 fire tests representing 9 

different sets of conditions, with multiple replicates of each set, which were conducted in a threeroom suite described in detail in Ref. [3]. The suite consisted of two relatively small rooms, 

connected via a relatively long corridor. The fire source, a gas burner, was located against the rear 

wall of one of the small compartments. Fire tests of 100 kW, 300 kW and 500 kW were 

conducted. For the current V&V study, only three 100 kW fire experiments have been used, 

including Test 100A from Set 1, Test 100O from Set 2, and Test 100Z from Set 4. The selected 

data are also available in Ref. [23]. 

For the NBS Multi-room series, Tests 100A, 100O and 100Z were used because they were 

constructively used in a previous EPRI study [Ref. 23]. The data in the NBS data set was 

acquired every 10 s with a 6 s time-average. This time-averaging interval was somewhat smaller 

than all of the other experimental series, which were time-averaged over a 10 s interval. 

3.2.1 Heat Release Rate 

Figures 3-2 and 3-3 show the experimentally measured HRR as a function of time during Tests 

100A and 100 Z, respectively, of the NBS multi-room test series. In these two tests, for which 

the door was open, the measured HRR via oxygen consumption calorimetry was taken to be 

110 kW ±17 kW (±15 %). The combined relative expanded (2σ) uncertainty in the calorimetric 

HRR is assigned a value of ±15 %, consistent with the replicate measurements made during the 

experimental series and the uncertainty typical of oxygen consumption calorimetry. This value is 

also consistent with the measurement variation seen in the figures. It was assumed that the 

closed door test (Test 100O) had the same HRR as the open door tests. 

The prescribed HRR is also shown in the figures. The prescribed HRR is nominally the value of 

the HRR in the compartment (unless the fire is underventilated, which was not the case in these 

experiments). The prescribed HRR differs from the measured value in the exhaust hood due to 

mixing in the upper layer of the compartment, which effectively adds a time-response function to 

the measurement. The prescribed HRR has been inferred from the calorimetric measurement 

made in the exhaust hood. As expected, the measured HRR increased slowly with time, and then 

approached the prescribed value, which was intended to be steady 

The mass flow of the fuel (natural gas in Test 100 A, or natural gas mixed with acetylene in 

Tests 100O and 100Z) was not metered; rather, the effluent was captured in a hood mounted 

above the open door in the corridor and the HRR was measured using oxygen consumption 

calorimetry. The manner by which the fuel flow was controlled is not documented. In Test 100, 

candles were used to increase smoke in the upper layer to allow visualization. Tests 100 O and 

100Z used acetylene (about 20 % by volume) to produce smoke. In those tests, the flow of 

natural gas and acetylene were adjusted to obtain approximately the same HRR as in Test 100A. 
Test Series 

3-5 

Heat Release Rate

NBS Multi-Room, Test 100A

Time (min)

0 5 10 15 20

Heat Release Rate (kW)

0

20

40

60

80

100

120

140

160

Calorimetry

Prescribed

Figure 3-2: Prescribed and measured heat release rate as a function of time during Test 

100A of the NBS multi-room test series. 

Heat Release Rate

NBS Multi-Room, Test 100Z

Time (min)

0 5 10 15 20

Heat Release Rate (kW)

0

20

40

60

80

100

120

140

160

Calorimetry

Prescribed

Figure 3-3: Prescribed and measured heat release rate as a function of time during Test 

100Z of the NBS multi-room test series. 

The addition of acetylene increased the radiative fraction of the fire, which is discussed in the 

next section. 
Test Series 

3-6 

3.2.2 Radiative Fraction 

For many reasons, natural gas supplied by large utility companies is often used in fire 

experiments. While its composition may vary from day to day, there is little change expected in 

the value of the radiative fraction. As mentioned above, natural gas was used as the fuel in Test 

100A. In Tests 100O and 100Z, acetylene was added to the natural gas to increase the smoke 

yield, and as a consequence, the radiative fraction increased. The radiative fraction of natural 

gas has been studied previously, whereas the radiative fraction of the acetylene/natural gas 

mixture has not been studied. The radiative fraction for the natural gas fire was assigned a value 

of 0.20, whereas a value of 0.30 was assigned for the natural gas/acetylene fires [Ref. 11]. 

The relative combined expanded (2σ) uncertainty in this parameter was assigned a value of 

±20 % in Test 100A and ±30 % in 100O and 100Z. The 20 % expanded deviation value is 

consistent with typical values of the deviation reported in the literature for the measured radiative 

fraction. The 100O and 100Z tests had a 50 % larger value assigned, because the effect on the 

radiative fraction of adding acetylene to the natural gas was not measured. 

3.2.3 Other Measurements 

Measurements made during the NBS test series included gas and surface temperature, pressure, 

smoke and major gas species concentration, and doorway gas velocity. As seen in Table 1-2, 

only two types of measurements conducted during the NBS test series were used in the 

evaluation considered here. These were the HGL temperature and depth, in which bare bead 

thermocouples were used to make these measurements. Single point measurements of 

temperature within the burn room were not used in the evaluation of plume or ceiling jet 

algorithms. This is because, in neither instance, was the geometry consistent with the 

assumptions used in the algorithms. Specifically, the burner was mounted against a wall, and the 

room width to height ratio was less than that assumed by the various ceiling jet correlations. 
Test Series 

3-7 

3.3 ICFMP Benchmark exercise #2 

The experiments are described in Ref. [4]. Benchmark Exercise #2 (BE #2) consisted of 8 

experiments, representing 3 sets of conditions, to study the movement of smoke in a large hall 

with a sloped ceiling. The results of the experiments were contributed to the International 

Collaborative Fire Model Project (ICFMP) for use in evaluating model predictions of fires in 

larger volumes representative of turbine halls in NPPs. The tests were conducted inside the VTT 

Fire Test Hall, which has dimensions of 19 m (62 ft) high by 27 m (89 ft) long by 14 m (46 ft) 

wide. Each case involved a single heptane pool fire, ranging from 2 MW to 4 MW. 

As seen in Table 1-2, four types of measurements conducted during the VTT test series (BE #2) 

were used in the evaluation considered here, including the HGL temperature and depth, average 

flame height and the plume temperature. Three vertical arrays of thermocouples, plus two plume 

thermocouples, were used. The HGL temperature and height were reduced from an average of 

the three TC trees using the standard algorithm. No ceiling jet evaluation was done because the 

ceiling in the test hall is not flat. 

3.3.1 Supplementary Information 

The VTT test report lacks some information that is needed to model the experiments, so some 

information was based on private communications with the principal investigator [Ref. 24]. 

Surface Materials: The walls and ceiling of the test hall consist of a 1 mm (0.039 in) thick layer 

of sheet metal on top of a 5 cm (2 in) layer of mineral wool. The floor was constructed of 

concrete. The report does not provide thermal properties of these materials. 

Natural Ventilation: In Cases 1 and 2, all doors were closed, and ventilation was restricted to 

infiltration through the building envelope. Precise information on air infiltration during these tests 

is not available. The scientists who conducted the experiments recommend a leakage area of 

about 2 m2 (20 ft2), distributed uniformly throughout the enclosure. By contrast, in Case 3, the 

doors located in each end wall (Doors 1 and 2, respectively) were open to the external ambient 

environment. These doors are each 0.8 m (2.6 ft) wide by 4 m (5 ft) high, and are located such that 

their centers are 9.3 m (30.5 ft) from the south wall. 

Mechanical Ventilation: The test hall has a single mechanical exhaust duct, located in the roof 

space, running along the center of the building. This duct has a circular section with a diameter of 

1 m (40 in), and opens horizontally to the hall at a distance of 12 m (39 ft) from the floor and 10.5 m 

(34.4 ft) from the west wall. Mechanical exhaust ventilation was operational for Case 3, with a 

constant volume flow rate of 11 m3

/s drawn through the 1 m (40 in) diameter exhaust duct. 

HGL height and Temperature: All of the vertical thermocouple trees were used to compute the 

HGL depth and temperature, with all of the trees evenly weighted. 

3.3.2 Heat Release Rate 

Each test used a single fire source with its center located 16 m (52 ft) from the west wall and 

7.4 m (24.3 ft) from the south wall. For all tests, the fuel was heptane in a circular steel pan that 

was partially filled with water. The pan had a diameter of 1.17 m (46.0 in) for Case 1 and 1.6 m 

(63 in) for Cases 2 and 3. In each case, the fuel surface was 1 m (40 in) above the floor. The 

trays were placed on load cells, and the HRR was calculated from the mass loss rate (see 
Test Series 

3-8 

definition in Chapter 3). For the three cases, the fuel mass loss rate was averaged from 

individual replicate tests. However, the HRR was not measured, and must be inferred from the 

mass loss data. Thus, the HRR in BE #2 was based on the mass burning rate measurement (see 

Chapter 2). 

In the HRR calculation, the heat of combustion for n-heptane was used (44.6 kJ/g). Hostikka 

[Ref. 4] suggests a value of 0.8 for the combustion efficiency, which is a compromise between 

0.7 [Ref.17] and various other estimates around 0.9. Bundy [Ref. 25] put the efficiency of a 

500 kW heptane fire as high as 0.97. The magnitude of the combustion efficiency is a 

complicated function of fire size and other effects. The combustion efficiency tends to decrease 

as the fire gets larger, and compartment effects are not well understood. In this report, a 

combustion efficiency of 0.85 ± 0.12 (or ± 14 %) is recommended, based on engineering 

judgment. Given the range of values associated with the value of χa, the uncertainty in the HRR 

is dominated by uncertainty in the combustion efficiency. Uncertainty in the mass loss rate 

measurement also contributes, so the uncertainty in the HRR was estimated as 15 %. Figures 3-4 

to 3-6 show the prescribed HRR as a function of time during Cases 1 to 3, respectively, during 

the ICFMP BE #2 test series. Tables 3-1 to 3-3 represent the mass loss and estimated HRR 

associated with Figures 3-4 to 3-6, respectively. 

3.3.2 Radiative Fraction 

The radiative fraction was assigned a value of 0.30, similar to many smoky hydrocarbons 

[Ref. 17]. The relative combined expanded (2σ) uncertainty in this parameter was assigned a 

value of ±20 %, typical of uncertainty reported in the literature for this parameter. 
Test Series 

3-9 

Heat Release Rate

ICFMP BE #2, Case 1

Time (min)

0 2 4 6 8 10

Heat Release Rate (kW)

0

1000

2000

3000

4000

5000

Figure 3-4: Prescribed heat release rate as a function of time during Case 1 of the BE #2 

test series. 

Heat Release Rate

ICFMP BE #2, Case 2

Time (min)

0 2 4 6 8 10

Heat Release Rate (kW)

0

1000

2000

3000

4000

5000

Figure 3-5: Prescribed heat release rate as a function of time during Case 2 of the BE #2 

test series. 
Test Series 

3-10 

Heat Release Rate

ICFMP BE #2, Case 3

Time (min)

0 2 4 6 8 10

Heat Release Rate (kW)

0

1000

2000

3000

4000

5000

Figure 3-6: Prescribed heat release rate as a function of time during Case 3 of the BE #2 

test series. 

Table 3-1: Mass Loss and Heat Release Rates for Case 1 of the BE #2 test series 

Time (s) MLR (kg/s) HRR (kW) 

0 0 0 

13 0.033 1250 

90 0.045 1706 

288 0.049 1857 

327 0.047 1782 

409 0.036 1364 

438 0 0 

Table 3-2: Mass Loss and Heat Release Rates for Case 2 of the BE #2 test series 

Time (s) MLR (kg/s) HRR (kW)

0 0 0 

14 0.057 2161 

30 0.067 2540 

91 0.081 3071 

193 0.086 3261 

282 0.083 3147 

340 0.072 2729 

372 0.060 2275 

395 0 0 
Test Series 

3-11 

Table 3-3: Mass Loss and Heat Release Rates for Case 3 of the BE #2 test series 

Time (s) MLR (kg/s) HRR (kW) 

0 0 0 

13 0.064 2426 

63 0.084 3184 

166 0.095 3601 

256 0.096 3639 

292 0.091 3449 

330 0.070 2654 

345 0 0 
Test Series 

3-12 

3.4 ICFMP Benchmark Exercise #3 

Benchmark Exercise #3, conducted as part of the International Collaborative Fire Model Project 

(ICFMP) and sponsored by the NRC, consisted of 15 large-scale experiments performed at NIST in 

June 2003. The experiments are documented in Ref. [5]. The fire sizes ranged from 350 kW to 

2.2 MW in a compartment with dimensions 21.7 m x 7.1 m x 3.8 m high, designed to represent a 

compartment in a NPP containing power and control cables. Walls and ceiling were covered 

with two layers of marinate boards, each layer 0.0125 m (1⁄2 in) thick. Thermophysical properties 

of the marinate and other materials are given in Ref. [5]. The floor was covered with one layer of 

0.0125 m (1⁄2 in) thick gypsum board on top of a 0.0183 m (23/32 in) layer of plywood. The room 

had one door and a mechanical air injection and extraction system. Ventilation conditions, the 

fire size, and fire location were varied, and the numerous experimental measurements included 

gas and surface temperatures, heat fluxes, and gas velocities. 

3.4.1 Supplementary Information 

Natural Ventilation: The compartment had a 2 m x 2 m door in the middle of the west wall. 

Some of the tests had a closed door (Tests 4, 10, 16, and 17), and in those tests compartment 

leakage was a very important parameter. Ref. [5] reports leakage area based on measurements 

performed prior to Tests 1, 2, 7, 8, and 13. For the closed door tests (Tests 4, 10, 16, and 17), the 

leakage areas ought to be based on the last available measurement. It should be noted that the 

chronological order of the tests differed from numerical order [Ref. 5]. For Test 4, it is 

recommended that the leakage area measured before Test 2 be used. For Tests 10 and 16, it is 

recommended that the leakage area measured before Test 7 be used. 

Mechanical Ventilation: Mechanical ventilation and exhaust was used during Tests 4, 5, 10, and 

16, providing about 5 air changes per hour. The supply duct was positioned on the south wall, 

about 2 m off the floor. An exhaust duct of equal area to the supply duct was positioned on the 

opposite wall at a comparable location. The flow rates through the supply and exhaust ducts 

were measured in detail during breaks in the testing, in the absence of a fire. 

During the tests, the flows were monitored with single bi-directional probes during the tests 

themselves. A bi-directional probe was positioned in the center of the exhaust duct, and its 

velocity was recorded under the column header “BP Exhaust Vent” in the BE #3 experimental 

data sets. Its value varied between 3 m/s (10 ft/s) and 4 m/s (13 ft/s) for the four ventilated tests. 

The supply and exhaust volume flow rates and other pertinent information can be found in 

Ref. [5]. Usually this is expressed as the vent area times an average velocity. Another bidirectional probe was positioned in the supply duct, 30 cm (1 ft) from the bottom of the duct 

during Tests 4 and 5, and 15 cm from the bottom of the duct for Tests 10 and 16. In the data 

sets, this measurement is listed under the column header “BP Supply Vent-16”. Its value was 

between 3 m/s (10 ft/s) and 4 m/s (13 ft/s) for Tests 4 and 5, and was as high as 10 m/s (33 ft/s) 

during Tests 10 and 16. 

The exhaust duct profile was relatively uniform, whereas the supply was not. Most of the air 

was blown out of the bottom third of the supply duct. The single point measurements during the 

fire tests indicated that the flow field changed from its ambient values. The measured supply 

volume flow rate of 1.06 m3

/s (37.4 ft3/s) pre-test decreased to 0.9 m3/s (31 ft3/s) during testing. 

For the exhaust, the measured volume flow rate of 1.03 m3

/s (36.4 ft3/s) pretest increased to 
Test Series 

3-13 

about 1.7 m3

/s (60 ft3/s) during testing. The uncertainties during the fires are substantially higher 

than the uncertainties in the ambient measurements (±0.2 m3

/s or 7 ft3/s). Doubling this value is 

appropriate. 

The ventilation system affected the compartment pressure, HGL temperature, and the surface 

temperature of various cable targets. The cable surface TCs were just outside of the direct path 

of the supply fan. In the absence of a fire, blowing was observed to flow upwards at about a 35° 

angle. 

Fuel Delivery: The fire was located at floor level in the center of the compartment for most of the 

tests (Tests 1 - 13, 16, and 17). In Test 14, the fire was centered 1.8 m (72 in) from the North 

wall. In Test 15, the fire was centered 1.25 m (50 in) from the South wall. In Test 18, the fire 

was centered 1.55 m (62 in) from the South wall. Physically, the fuel pan was 2 m (80 in) long x 

1 m (40 in) wide and 0.1 m (4 in) deep. A single nozzle was used to spray liquid hydrocarbon 

fuels onto a 1 m (40 in) by 2 m (80 in) fire pan that was about 0.02 m deep (1 in). The test plan 

originally called for the use of two nozzles to provide the fuel spray. Experimental observation 

suggested that the fire was less unsteady with the use of a single nozzle. In addition, it was 

observed that the actual extent of the liquid pool was well-approximated by a 1 m (40 in) circle 

in the center of the pan. The uncertainty in the location of the liquid fuel was about ±0.1 m (± 4 

in). For safety reasons, the fuel flow was terminated when the lower-layer oxygen concentration 

dropped to approximately 15% by volume. 

3.4.2 Heat Release Rate 

The fuel used in 14 of the tests was heptane, while toluene was used for one test (Test 17). The 

HRR was determined using oxygen consumption calorimetry. The uncertainty in the HRR 

measurement was documented in Ref. [5]. The recommended uncertainty values were 17 % for 

all of the tests. Figure 3-7 shows the measured and prescribed HRR as a function of time during 

Test 3. Ref. [5] discusses the shape of the prescribed HRR curve in detail. 

3.4.3 Radiative Fraction 

The radiative fraction was measured in an independent study for the same fuels using the same 

spray burner as used in the BE #3 test series [Ref. 21]. The value of the radiative fraction and its 

uncertainty were reported as 0.44 ± 16 % and 0.40 ± 23 % for heptane and toluene, respectively 

(also see Table 2-1.). 
Test Series 

3-14 

Heat Release Rate

ICFMP BE #3, Test 3

Time (min)

0 5 10 15 20 25 30

Heat Release Rate (kW)

0

300

600

900

1200

1500

Prescribed

Calorimetry

Figure 3-7: Measured and prescribed heat release rate as a function of time during Test 3 

of the ICFMP #3 test series. 

3.4.4 Other Measurements 

As seen in Table 1-2, measurements made during the NIST test series (BE #3) included all 13 

parameters considered in this study, except the plume temperature. 

The vertical thermocouple trees were used to determine the HGL depth and temperature, except 

Tree 4, which had a faulty TC. The thermocouple data was weighted based on the location of the 

thermocouple tree in the compartment, and the portion of the area that was represented by the 

data from each tree. Table 3-4 shows the relative weightings. Trees 1 and 7 were weighted 0.3 

each, Tree 2 was weighted 0.2, Tree 3 was weighted 0.1, and the others were weighted 0.05. 

Table 3-4: Relative Weighting of the Calculation of HGL 

Thermocouple Tree Relative Weight

1 0.3 

2 0.2 

3 0.1 

4 0 

5 0.05 

6 0.05 

7 0.3 

3.5 ICFMP Benchmark Exercise #4 

Benchmark Exercise (BE) #4 consisted of kerosene pool fire experiments conducted at the 

Institut für Baustoffe, Massivbau und Brandschutz (iBMB) of the Braunschweig University of 
Test Series 

3-15 

Technology in Germany. The results of two experiments were contributed to the International 

Collaborative Fire Model Project (ICFMP) and documented in the report, Evaluation of Fire Models 

for Nuclear Power Plant Applications: Fuel Pool Fire inside a Compartment [Ref. 6]. These 

experiments involved relatively large fires in a relatively small (3.6 m x 3.6 m x 5.7 m high or 12 

ft x 12 ft x 19 ft) concrete enclosure. Only Test 1 was selected for consideration in the present 

V&V study, because a significant amount of data was lost in Test 1, and the measured HRR 

during Test 3 exhibited significant amounts of fluctuation. As seen in Table 1-2, five types of 

measurements conducted during the BE #4 test series were used in the evaluation conducted 

here, including the HGL temperature and depth, temperature of targets and compartment 

surfaces, and heat flux. 

3.5.1 Heat Release Rate 

The fire in Test 1 was a 1 m x 1 m (40 in x 40 in) square pan of jet fuel, type A-1. Ref. [6] 

reports that the fuel is a mixture of hydrocarbons with a summary formula as C11.64H25.29, and 

that the thermophysical properties of the jet fuel were similar to dodecane [Ref. 6]. 

Figure 3-8 shows the heat release rate, which was estimated from the mass loss rate 

measurement. Table 3-5 lists the measured mass loss rate, as well as the calculated HRR, in 

which the heat of combustion and the combustion efficiency were taken as 42.8 MJ/kg and 1, 

respectively, as suggested by Ref. [6]. There were several reported difficulties in measuring the 

mass loss rate, including data loss due to an instrument malfunction and significant fluctuations 

in the measured mass loss rate. Due to the measurement issues and because the combustion 

efficiency was not well-characterized, the HRR uncertainty was assigned a relatively large 

expanded uncertainty of ±25 %. 

Table 3-5: Measured Mass Loss Rate and Calculated 

Time (s) Mass Loss Rate (kg/s) HRR (kW) 

0 0 0 

92 0.0028 119 

180 0.037 1583 

260 0.0613 2623 

600 0.0747 3197 

822 0.0783 3351 

870 0.079 3381 

1368 0.0822 3518 
Test Series 

3-16 

Heat Release Rate

ICFMP BE #4, Test 1

Time (min)

0 5 10 15 20 25 30

Heat Release Rate (kW)

0

1000

2000

3000

4000

Figure 3-8: The estimated HRR based on the mass loss rate as a function of time during 

Test 1 of the BE #4 test series. 

3.5.2 Radiative Fraction 

The radiative fraction of the jet fuel was taken as 0.35, similar to other smoky hydrocarbons 

[Ref. 17]. The relative combined expanded uncertainty in this parameter was assigned a value of 

±20 %, consistent with typical values reported in the literature for the measured radiative fraction 

[Refs. 19, 20]. 
Test Series 

3-17 

3.6 ICFMP Benchmark Exercise #5 

Benchmark Exercise (BE) #5, conducted under the International Collaborative Fire Model Project 

(ICFMP), comprised a series of four large-scale tests inside a concrete enclosure with realistically 

routed cable trays [Ref.7]. This test series was conducted at the Institut für Baustoffe, Massivbau 

und Brandschutz (iBMB) of the Braunschweig University of Technology in Germany. Test 4 of 

this test series was selected for the quantitative V&V study documented in this report. 

As seen in Table 1-2, six types of measurements conducted during the BE #5 test series were 

used in the evaluation conducted as part of this report series, including the HGL temperature and 

depth, oxygen gas concentration, the temperature of targets and compartment surfaces, and heat 

flux. 

3.6.1 Heat Release Rate 

The fire involved a large fire in a relatively small enclosure. The first part of the selected test 

consisted of preheating the cable trays in the room. A 1 m2

 (11 ft2) round pan on the floor filled 

with ethanol (ethyl alcohol) was used as the pre-heating source, and a propane gas burner was 

used as the fire source after pre-heating. 

Exhaust products were collected in an exhaust duct and the HRR was measured using the oxygen 

calorimetry. For the purpose of this V&V study, the measured HRR was used as direct input to 

the various fire models. The first 20 min of data were used for the model evaluation. After 20 

min, the HRR became relatively noisy. At 33 min, thermoplastic cables located in the 

compartment began to burn and contribute to the HRR. Figure 3-9 depicts the measured HRR 

profile. The relative combined expanded uncertainty in this parameter was assigned a value of 

±15 %, consistent with typical values of this parameter [Ref. 11]. 

3.6.2 Radiative Fraction 

The radiative fractions for the ethanol pool fire and the propane fire were taken as 0.20 and 0.35, 

respectively. The relative combined expanded uncertainty in this parameter was assigned a value 

of ±20 %, which is consistent with typical values reported for this parameter [Ref. 21, 22]. 
Test Series 

3-18 

Heat Release Rate

ICFMP BE #5, Test 4

Time (min)

0 15 30 45 60 75 90

Heat Release Rate (kW)

0

200

400

600

800

1000

Figure 3-9: The heat release rate as a function of time during Test 4 of the BE #5 test 

series. Only the first 20 min of the test were used for model evaluation. 
4-1 

4 

MEASUREMENT UNCERTAINTY 

For many of the experiments and most of the measurements considered in this report, the 

experimental uncertainty was not documented. In this chapter, estimates of measurement 

uncertainty based on engineering judgment are provided. Quantifying uncertainty in measured 

quantities provides a basis for evaluation of model results through consideration of Eqs. 1.1 to 

1.4. Documentation of the measurement uncertainty produces a consistent set of data for use in 

the evaluation of the models considered in this report. The uncertainty (Ue) of each of the 13 

measurements considered in the model evaluation is provided (see Section 1.2 for a list of 

parameters, which included the hot gas layer temperature and depth, the ceiling jet and plume 

temperature, etc.). 

The uncertainty estimates provided here are limited by knowledge of the details associated with 

each of the experiments. For this reason, the uncertainty values provided here should be thought 

of as rough estimates, rather than precise determinations of measurement uncertainty. Even for 

the experimentalists themselves, the accuracy of an uncertainty analysis is often limited by 

incomplete understanding. For example, in heat flux gauges, the uncertainty due to soot 

deposition on the face of the gauge is difficult to quantify. The amount of soot deposition 

depends on many parameters, such as the location of the gauge, the flow field and the 

temperature field near the gauge, the duration of the test, and the local soot volume fraction. 

Unexpected events or poor understanding limits the accuracy of an uncertainty analysis. 

In general, measurement uncertainty depends on many issues, including the exact type of 

instrumentation, the experimental procedure, and the details of the measurement scenario. The 

uncertainty in many of the experimental measurements is difficult to accurately estimate, 

because most of the test reports do not provide uncertainties for the individual measurements. 

For this reason, the values are inferred based upon engineering judgment and experience with 

similar instrumentation. 

4.1 Temperature and Hot Gas Layer Depth 

The gas phase temperature was measured in all six of the experiments considered in this report. 

Thermocouples were used to characterize the depth of the hot gas layer (HGL) and the local 

temperature. Measurements were typically made at several vertical locations at one or more 

horizontal locations. Bare bead thermocouples were used in all of the experiments considered in 

this report except the BE #3 and FM/SNL tests in which bare and aspirated thermocouples were 

used. The HGL temperatures and depths were determined using the reduction method described 

by Peacock and Babrauskas [Ref. 26]. 

The interpretation of a bare bead thermocouple signal must consider several possible sources of 

error. Thermocouple (TC) measurement error can occur due to breakdown of the TC insulation 

at high temperatures, corrosion from acid combustion byproducts, decalibration at high 

temperatures, inherent measurement accuracy limited by materials effects, and measurement 
Measurement Uncertainty 

4-2 

error due to radiative exchange effects. The latter requires attention for the experiments 

considered in this report. 

Blevins [Ref. 27] reported on the development of a model to compute TC error due to radiative 

exchange effects. Radiative exchange between the thermocouple and walls, flame gases, soot, 

and the ambient environment were considered. In the same document, report of soot 

accumulation on the TC bead or sensing junction and changing of its thermophysical properties 

may also contributing to error. Variability in the convective heat transfer between the sample 

gas and the TC junction is also cited as a source of error. 

There are several ways to handle radiative exchange effects on TC measurements. A correction 

may be applied to the signal to improve measurement accuracy. This is burdensome, because the 

radiative exchange correction may depend on the character of the thermocouple, such as its 

diameter, its material composition, its surface emissivity, as well as the details of its application, 

involving the local temperature, radiation, and flow fields. So, the correction depends on local 

conditions, and each TC should be independently analyzed. Typically, not enough experimental 

information is available to perform an accurate radiation correction. In such a case, it may be 

more practical to expand the uncertainty bounds for the measurement to encompass radiative 

exchange effects that are estimated based on simplified assumptions. This approach is used here 

and described below. 

A completely different measurement approach involves the use of an aspirated TC as the 

measurement device, in which radiative exchange effects are minimized. Aspirated TCs are 

constructed, such that the TC junction is within two concentric steel tubes. A mechanical pump 

draws gas extracted from the sampling location through a tube, engulfing the TC sensing 

junction with the sample gas, while the tubes protect the TC sensing junction from impinging 

thermal radiation. The error associated with the aspirated probe measurement itself depends on 

the application conditions and the results are a strong function of the aspiration velocity as well 

as a number of other parameters [Ref. 27, 28]. Aspirated thermocouples provide accurate 

temperature information, but typically are used sparingly, due to their relatively high cost 

compared to bare bead thermocouples. FM/SNL used aspirated TCs in the experiments 

considered in this report and BE #3 used aspirated TCs to assess the accuracy of the bare bead 

TC results. 

In a hot upper layer of a compartment with lots of soot, a TC reading may be fairly accurate and 

not need to be corrected for radiative exchange effects. This is because the environment in such 

a scenario is nearly optically thick, for which radiative exchange effects are minimized. Many 

reports, including Refs.[2, 29, 30], have examined the magnitude of the TC measurement error 

due to radiative exchange by comparing bare bead TC measurements to measurements by nearby 

aspirated thermocouples in well-developed hot (60 oC to 800 o

C; or 140 oF to 1500 oF) smoky 

upper layers of compartment fires. For the fires considered in this report, it is assumed that the 

upper layer in all of the tests had a high opacity at the time of the peak temperature readings. 

This is a reasonable assumption for the types of fuels being used in the experiments considered 

here with the exception of NBS Test 100A in which natural gas was used as a fuel. For the other 

tests, Table 2-1 shows that the fuels included smoky fuels such as toluene, heptane, propylene, 

jet fuel, and propane. Estimation of the uncertainty in the bare bead TC measurements involves 

determination of the uncertainty of the various components of the measurement and the effect 

that each component has on the overall uncertainty. Computation of the overall uncertainty 
Measurement Uncertainty 

4-3 

considers the measurement error due to radiative exchange, the inherent uncertainty associated 

with a bare-bead thermocouple, and the error associated with use of an aspirated thermocouple. 

The manufacturer reported accuracy for Type K TCs is 1.1 °C (2.0 o

F) for temperatures between 

0 °C (32 o

F) and 293 °C (560 oF), and 0.4 % between 293 °C (560 oF) and 1250 °C (2280 oF), 

interpreted here as the expanded uncertainty [Ref. 31]. These values are listed in Table 4-1 as 

Uw for a number of temperatures. The expanded uncertainty associated with the aspirated probe 

measurement (Ua) is taken as approximately 4 % for an aspiration velocity of 15 m/s to 20 m/s as 

used in BE #3 [Ref. 27]. The value of Ua, listed in Table 4-1, also includes the inherent 

thermocouple uncertainty. The contribution of the uncertainty due to radiative exchange on a 

bare bead TC is estimated from the results of Refs. [2, 29, 30]. In those experiments, the 

measurement results of bare bead TCs were within 3 o

C (5 oF) to 15 oC (27 oF) of nearby 

aspirated thermocouples. In the three FM/SNL tests [Ref. 2], for example, a bare bead 

thermocouple in the upper layer was within 5 o

C (9 oF) to 6 oC (11 oF) of a nearby aspirated 

thermocouple, for upper layer temperatures of about 60 o

C (140 oF). In a hot upper layer of a 

heptane fire [Ref. 29], bare bead thermocouples in the HGL were within 12 oC (22 oF) of nearby 

aspirated thermocouples, for various tests in which the upper layer temperatures ranged from 

400 o

C (750 oF) to 800 oC (1500 oF). Intermediate temperatures were estimated based on linear 

interpolation between the higher and lower temperature results. Contributions to the uncertainty 

associated with radiative exchange are listed as Ur in Table 4-1. The component expanded 

uncertainties were used to compute the combined expanded uncertainty for a bare bead TC as 

listed in Table 4-2. The largest contributor to the overall uncertainty at high temperatures was the 

uncertainty in the aspirated thermocouple measurement, while at low temperatures the 

contribution of uncertainty in the radiative exchange was relatively more important. 

In the lower layer of compartment fires with a smoky, high-opacity upper layer, radiative gain 

due to flux from the hot upper layer may lead to erroneously high thermocouple readings. This 

is not considered a significant problem for the model evaluation considered here, as the lower 

layer information is used only to determine the hot gas layer (HGL) depth. It is assumed that the 

uncertainty in the layer depth calculation is not significantly impacted by erroneously high lower 

layer TC readings, since the layer calculation looks for locations of temperature difference, 

rather than the value of that difference. A larger contributor to the uncertainty of the layer depth 

is the physical distance between thermocouples, which is the spatial resolution of the 

measurement. Consideration of the algorithm used to determine the HGL depth suggests that the 

uncertainty will be approximately one-half the distance between thermocouples at the location of 

the layer interface. 

Table 4-3 summarizes the estimates of the relative expanded measurement uncertainties, Ue, 

associated with the HGL layer depth and temperature rise for the experiments considered in this 

report. For the HGL depth, the distance between thermocouples differed for each of the tests, 

varying from 0.1 m (4 in) to 0.3 m (12 in). Repeatability of the depth determination was 

investigated for BE #3 by examining the results for the repeat tests. The difference in the 

calculated HGL depth between the repeat measurements was about 3 %, on average. It was 

assumed that the repeatability in the other tests was similar to that determined in BE #3. 

Measurement repeatability was considered in the determination of Ue. The HGL depths varied 

from about 1 m (40 in) to 3 m (120 in) for all of the experiments considered here, and the relative 

expanded uncertainties in the values of the HGL depth varied from about 6 % to 23 % as seen in 

Table 4-3. The value of Ue does not accurately account for radiative exchange in NBS Test 
Measurement Uncertainty 

4-4 

100A, in which natural gas was the fuel. Treating the HGL in this case, as optically thick is 

inappropriate, and the value of Ue is likely small. 

Ceiling jet measurements were conducted in BE #3 and the FM/SNL tests, in which the 

temperatures were measured using bare bead and aspirated thermocouples, respectively, and 

temperatures in the plume were conducted in BE #2 and the FM/SNL tests, in which the 

temperatures were measured using bare bead and aspirated thermocouples, respectively. Because 

the ceiling jet is located high in the hot smoky upper layer, radiative exchange effects on 

thermocouples should be minimal and the results are treated in the same way as the bare bead 

thermocouples in the upper layer. 

Table 4-1: Expanded Measurement Uncertainty of a Bare Bead Thermocouple in the Hot 

Smoky Upper Layer of a Compartment Fire 

Gas Temperature 

°C (o

F) 

Uw

°C (o

F) 

Ur

°C (o

F) 

Ua

°C (o

F) 

Ue, Combined Expanded 

Uncertainty °C (°F) 

0 (32) 2 (4) 0 (0) 2 (4) 3 (5) 

60 (140) 2 (4) 5 3 (5) 6 (11) 

120 (250) 2 (4) 6 (11)* 4 (7) 8 (14) 

150 (300) 2 (4) 6 (11)* 5 (9) 8 (14) 

300 (570) 2 (4) 9 (16)* 12 (22) 15 (27) 

500 (930) 4 (7) 12 (22) 20 (36) 24 (43) 

800 (1500) 6 (11) 12 (22) 32 (58) 35 (63) 

* based on linear interpolation between the higher and lower temperature results. 

Repeatability of the ceiling jet temperatures was investigated for BE #3 by examining the results 

of the repeat tests. The difference between the repeat measurements was about 8 %, on average. 

The difference in the repeatability of the FM/SNL tests was assumed to be small, since radiative 

exchange effects were negligible with the use of aspirated thermocouples. The values of Ue for 

the ceiling jet varied from 4 % to 12 % as seen in Table 4-4 for the two test series considered 

here. Assessment of the uncertainty of the plume temperature measurements is more 

problematic. In this case, the bare bead thermocouple results are treated in the same manner as 

the bare bead thermocouple results for measurements in an upper layer. The repeatability of the 

BE #2 measurements was assumed to equal 8 %, the same as the ceiling jet measurements in BE 

#3. Still, the value of Ue is probably too small in the lower part of the plume for BE #2, which 

was not in a well-developed smoky upper layer. The values of Ue for the plume varied from 4 % 

to 11 % as seen in Table 4-5 for the two tests considered here. 
Measurement Uncertainty 

4-5 

Table 4-2: The Relative Expanded Uncertainties (Ue) Associated 

 with the Measured HGL Depth and Temperature Rise 

Ue (%) 

Series HGL Depth HGL Temperature Rise

NBS 6 6 * 

FM/SNL 23 10 

BE #2 8 9 

BE #3 6 6 

BE #4 22 6 

BE #5 9 7 

* value of Ue does not accurately account for radiative exchange in Test 100A, in 

which natural gas was the fuel; its value is likely small; see text. 

Table 4-3: The Relative Expanded Uncertainties (Ue) Associated 

with the Measured Ceiling Jet Temperature Rise 

Ue (%) 

Series 

Ceiling Jet Temperature Rise

BE #3 12 

FM/SNL 4 
Measurement Uncertainty 

4-6 

Table 4-4: The Relative Expanded Uncertainties (Ue) Associated with the Measured Plume 

Temperature Rise 

Ue (%) 

Series 

Plume Temperature Rise (°C)

BE #2 upper 

BE #2 lower 

9 

11 * 

FM/SNL 4 

* the value of Ue is too small in the lower part of the plume in 

BE #2, which was not in a well-developed smoky upper layer. 

4.2 Gas Species Volume Fraction 

The volume fractions of the combustion products CO and CO2 were measured using gas 

sampling in conjunction with non-dispersive infrared analyzers, while the O2 volume fraction 

was typically measured using a paramagnetic analyzer. Gases were extracted through stainless 

steel or other types of lines and were pumped from the compartment and passed through the 

analyzers. For several reasons, water in the sample was typically filtered, so the reported results 

are denoted as “dry” and comparison with model results must be corrected. Analyzers were 

calibrated through the use of standard gas mixtures, with low relative uncertainties. Problems 

with the technique may involve instrument drift, analyzer response, incomplete and partial 

drying of sample gases, or in the case when drying is not used, undetermined amounts of water 

vapor in the oxygen cell resulting in inaccurate readings. 

Measurements of gas species volume fractions are considered for two of the experiments, namely 

BE #3 and BE #4. The species were measured in the upper and lower layer at a single location 

in BE #3. The relative expanded uncertainty in the measured values in BE #3 were about 3 % for 

both the oxygen depletion and the CO2 measurements. The largest contributor to the uncertainty 

was uncertainty in the composition of the calibration gas and the possibility of an undetermined 

amount of water vapor in the sample. Repeatability of the gas measurements was investigated 

for BE #3 by examining the results for the repeat tests. The difference between the repeat 

measurements was about 2 %, on average, for both the oxygen depletion and the carbon dioxide 

measurements. Combining the uncertainties, the relative expanded uncertainty was typically 4 % 

for both, the CO2 gain and the O2 depletion, measurements. It was assumed that the uncertainty 

in BE #5 was similar to that determined in BE #3. 
Measurement Uncertainty 

4-7 

Table 4-5: Summary of the Relative Expanded Uncertainties Associated with the Oxygen 

and Carbon Dioxide Concentrations 

Ue (%) 

Series HGL CO2 Concentration HGL O2 Concentration 

Decrease 

LGL O2 Concentration 

Decrease

BE #3 4 4 4 

BE #5 4 4 4 

4.3 Smoke Concentration 

The mass based smoke concentration was measured in one of the experiments, namely BE #3. In 

that experiment, smoke was measured using laser transmission at 632.8 nm. The reported mass 

concentration of smoke, Ms, was computed using the following expression: 

L

I I M

s

s φ

ln( / ) 0 = (4.1) 

where L is the path length, I and Io are the laser signal and reference signal and φs is the specific 

extinction coefficient, which has a nearly universal value of 8.7 m2

/g ± 5 % for hydrocarbons 

[Ref. 32]. Other uncertainties in the measurement were due to errors in the path length, L , and 

the light attenuation, I / I 0 . In BE #3, Ref. 5 reported the expanded uncertainty of the Ms

measurement as 18 %, with the dominant contribution to the uncertainty coming from drift in the 

laser measurement. Repeatability of the smoke measurement was investigated for BE #3 by 

examining the results for the repeat tests. The mean difference between the measurements was 

about 21 %. Therefore, a combined expanded experimental uncertainty of 28 % is suggested. 

 

Table 4-6: The Relative Expanded Uncertainties (Ue) Associated with the Measured Smoke 

Concentration and the Compartment Pressure 

Ue (%) 

Series 

Smoke Concentration Pressure

BE #3 28 27 
Measurement Uncertainty 

4-8 

4.4 Pressure 

The compartment pressure was only measured in one of the experiments, namely BE #3. The 

uncertainty in pressure measurements is typically small, but depends on the sensor type and its 

calibration. In BE #3, the differential pressure gauge used was temperature compensated, highlylinear, and very stable. A conservative estimate of the expanded measurement uncertainty led to 

a value of 1 %. Repeatability of the pressure measurement was investigated for BE #3 by 

examining the results for the repeat tests. The average difference between the repeat 

measurements was about 27 %. Compartment leakage is a likely explanation for the large 

difference between repeat tests. An expanded uncertainty value of 27 % is suggested in 

Table 4-7. 

4.5 Heat Flux 

Measurements of the heat flux are considered for three of the experiments, namely BE #3, 

BE #4, and BE #5. Heat flux gauges were used to measure the transport of radiant energy or the 

combination of radiation and convection. Several types of gauges were used and different types 

of instrumentation and procedures were used to calibrate the gauges. In BE #3, four types of 

gauges were used including those that measured total heat flux, or just radiative heat flux. In BE 

#4 and BE #5, total heat flux gauges were used. 

The uncertainty associated with a heat flux measurement depends on many factors including 

gauge characteristics, the calibration conditions and accuracy, as well as the incident flux modes 

(convective, radiative, conductive) and their magnitudes in the actual measurement situation 

[Ref. 33]. Ref. [34] performed two rounds of an international round-robin test of heat flux 

gauges in which five fire laboratories performed independent calibrations of two sets of Gardon 

and Schmidt-Boelter total heat flux gauges. The results showed that the calibrations agreed to 

within about 5 %. Typically, the reported expanded uncertainties of heat flux gauges varies from 

about 5 % to 10 %, with the measurement uncertainty typically dominated by uncertainty in the 

calibration and repeatability of the measurement. Ref. [33] suggests that much higher values may 

be more realistic, depending on the exact nature of the fire conditions and the type of gauge. 

Repeatability of the various heat flux measurements in BE #3 was determined by examining 

measurements by the same instruments for different pairs of repeat tests. The difference between 

the measurements was about 7 %, on average, for both the radiative flux measurements and the 

total flux measurements. In this report, an expanded uncertainty value of about 10 % is 

suggested based on the BE #3 measurement repeatability and calibration uncertainties. It is 

assumed that the measurement uncertainty in BE #4 and BE #5 was similar to BE #3 as listed in 

Table 4-8. 
Measurement Uncertainty 

4-9 

Table 4-7: Summary of the Relative Expanded Uncertainties Associated 

 with the Measured Target Heat Flux and Target Temperatures

Ue (%) 

Series 

Total Heat Flux to Targets Rise in Target Surface Temperature

BE #3 10 1 – 7 

BE #4 10 1 - 2 

BE #5 10 4 

4.6 Surface/Target Temperature 

Bare bead thermocouples were used to measure the temperature of targets and compartment 

surfaces. A typical method is to “peen” the thermocouple into surface, that is, to bend and 

effectively spring-load the metal thermocouple until physical contact with the surface or target 

occurs. Heating of the thermocouple can cause it to undergo a shape change, which can cause a 

“peened” thermocouple to have poor physical contact with a surface. Another method pulls on 

the leads of a thermocouple with a small gravity load, forcing it to make physical surface. 

The inherent expanded uncertainty associated with a type K thermocouple is approximately 2 °C 

(4 °F) for temperatures below 200 °C (390 °F) [Ref. 31]. Repeatability of the surface and cable 

temperature measurements was investigated in BE #3 by examining the measurements of the 

same instruments for different repeat tests. The difference between the measurements was about 

8 % and 7 %, on average, for the cable and wall surface measurements, respectively. Combining 

the inherent thermocouple uncertainty and the repeatability yields uncertainty on the order of 10 

% for the thermocouple surface measurements. It is assumed that the measurement uncertainty 

in BE #4 and BE #5 was similar to BE #3.

Table 4-8: Summary of the Relative Expanded Uncertainties Associated 

 with the Surface Heat Flux and Temperatures

Ue (%) 

Series 

Total Heat Flux Rise in Surface Temperature

BE #3 10 10 

BE #4 10 10 

BE #5 10 10 
Measurement Uncertainty 

4-10 

4.7 Summary 

In general, measurement uncertainty depends on many issues, including the exact type of 

instrumentation, the experimental procedure, and the details of the measurement scenario. 

Because uncertainty was not documented for most of the experiments considered in this report, 

engineering judgment was used to provide estimates of measurement uncertainty for each of the 

parameters of interest. This information on measurement uncertainty is combined with the 

model input uncertainty in Chapter 6 to provide a basis for the evaluation of the fire models, as 

described in Volumes 2-6 of this report. 
5-1 

5 

SENSITIVITY OF MODEL RESULTS TO UNCERTAINTY 

IN MEASURED INPUT PARAMETERS 

In this chapter, the sensitivity of model results to uncertainty in measured input parameters, 

referred to here as the model input uncertainty (Um), is quantified. There are a number of ways 

that this could be achieved. A sensitivity analysis for the models could have been performed by 

running many calculations and determining the variation of a calculated output parameter as a 

function of the change in one or more input parameters. This is a brute force approach, which 

provides relevant information, but is labor intensive and does not necessarily offer physical 

insight. In addition, such an approach would be model specific. Rather than a brute-force 

method, the approach presented here is based on correlations available in the fire literature. 

For each parameter of interest, a simple analytic description of the sensitivity of that parameter to 

the fire HRR is given. For example, the nature of the hot gas layer (HGL) is largely a function of 

the fire size, and the uncertainty in the HRR measurement for the various experiments ranged 

from 15 % to 25 %. The dependence has been quantified based on results documented in the 

technical literature, in which simple analytical relationships or correlations have been developed 

from measurements made in many compartment fire experiments performed over several 

decades. Using the empirical correlations, it is possible to estimate how the uncertainty in the 

specified HRR influences the various parameters of interest. The magnitude of this uncertainty 

(Um) plays an important role in the evaluation of model accuracy through consideration of 

Eq. 1.4 in Chapter 1. The remainder of this chapter discusses the sensitivity of model output 

parameters to the model input parameters. 

5.1 Hot Gas Layer and Ceiling Jet Temperatures 

According to an empirical correlation by McCaffrey, Quintiere and Harkleroad (MQH) 

[Ref. 35], the HGL temperature is proportional to the HRR raised to the two-thirds power: 

1/3

0 0

2

6.85 ⎟

⎟

⎠

⎞

⎜

⎜

⎝

⎛ − ∞ =

k T

g A H h A

Q T T & (5.1)

where Q& is the heat release rate in kW, A0 is the area of the opening, H0 is the height of the 

opening, w hk = k /δ is the heat transfer coefficient, k is the thermal conductivity (kW/m/K), δ w

is the wall thickness (m), and AT is the total compartment surface area (m2

). Although the MQH 

correlation has limitations, it encapsulates a set of observations that give insight into trends 

associated with HGL temperature. 

The uncertainty in the HRR measurement for the various experiments ranged from 15 % to 

25 %. The uncertainty in the HGL temperature prediction varies, therefore, by 2/3 from the 

uncertainty in the HRR measurement. 
Sensitivity of Model Results to Uncertainty in Measured Input Parameters 

5-2 

According to the empirical ceiling jet correlations by Alpert [Ref. 36], the temperature within the 

ceiling jet is also proportional to the HRR raised to the two-thirds power, thus the same 

sensitivity to the HRR should apply for the ceiling jet temperature as for HGL temperature. 

5.2 Hot Gas Layer Depth 

The location of the HGL is relatively insensitive to the HRR of the fire. According to the 

correlation of Heskestad and Delichatsios [Ref. 37], the layer height, z, is given by the function: 

⎟

⎟

⎠

⎞

⎜

⎜

⎝

⎛ = −

−

2

1/ 3 4 / 3

/

1.11 0.28 ln

A H

t Q H

H

z & (5.2) 

The rate of change of the height with time, according to this expression, is not a function of the 

HRR at all: 

t

H

dt

dz 0.28 = − (5.3) 

where H is the ceiling height. This equation shows that the rate of change with time becomes 

small for large values of t. Empirical correlations relating the sensitivity of the layer height to 

the HRR were not found in the literature. It should be noted that in fire scenarios with open 

doors or windows, the gas layer height is largely affected by the position of the soffit. The 

dependence of the HGL depth to the HRR was presumed to be weak. Yet, experimental 

observation suggests that the depth is a function of the door opening size and the relative size of 

the fire. 

To confirm the weak dependence of the HGL depth to the HRR, a sensitivity analysis was 

conducted using CFAST for Test 3 of BE #3. The HGL depth in CFAST is not a result of a 

simple correlation from the fire literature, but is a product of the solution of the energy and mass 

conservation equations. The CFAST results showed that there was about a ±2 % change in the 

HGL depth for a ±15 % change in the HRR. A 25 % uncertainty in the HRR would be expected 

to yield a 3 % uncertainty in the depth. The dependence of the HGL depth on the HRR is weak, 

but non-zero. The reasons for this likely involve a balance between the increasing HGL 

temperature and decreasing density and the balance of flows in and out of the compartment. 

5.3 Plume Temperature 

The plume temperature is mainly a function of the HRR. According to McCaffrey’s correlation 

of fire plume temperature and velocity [Ref. 38], the centerline temperature at heights between 

1.3 D* and 3.3 D* is proportional to the HRR to the two-fifths power, whereas at heights greater 

than 3.3 D*, the temperature rise is proportional to the two-thirds power, just like the HGL 

relationship. A similar argument can be made for the sensitivity of the fire plume temperature 

rise to HRR as that for the HGL temperature rise. 

5.4 Flame Height 

According to the empirical correlation by Heskestad [Refs.38, 39], flame height, L, is related to 

the HRR and the “diameter” of the fire via the expression: 
Sensitivity of Model Results to Uncertainty in Measured Input Parameters 

5-3 

2

*2/ 5 * 3.7 1.02 ; c T gDD

Q Q Q DL

∞ p ∞

= − = ρ

& & & (5.4) 

which is valid for values of * Q& , such that 0.12 < * Q& < 1.2·104

. Evaluating the constants, 

Heskestad [Ref. 38] gives an expression that is approximately valid for most types of 

hydrocarbon pool fires: 

L = -1.02 D + 0.235 Q& 2/5 (5.5) 

From this relationship, the sensitivity of the flame height to the HRR can be inferred. 

Considering a small change in L, denoted asδL , leads to: 

L

δL

 = 5

2 (1+ 1.02 D/L) Q

Q

&

& δ (5.6) 

For the experiments considered in this report, a representative relation between D and L is 

D ~ 3

1 L, so Eq. 5.6 is approximately: 

L

δL ~ 1⁄2 

Q

Q

&

& δ (5.7) 

and an uncertainty of 15 % or 25 % in the HRR, leads to an uncertainty of about 8 % or 13 %, 

respectively, in the flame height. 

5.5 Gas Concentration 

Most fire models assume that combustion product gases, once generated in the fire, are passively 

transported throughout the compartment. The major products of combustion, like carbon dioxide 

and water vapor, plus the major reactant, oxygen, are generated, or consumed, in direct 

proportion to the burning rate, which is directly proportional to the HRR. The mass fraction of 

any species in the HGL is related to its yield times the mass entrained into the upper layer 

[Ref. 37]: 

a c e

i

e

i

i H m

y Q

m

y m

Y &

&

&

&

χ = = (5.8) 

where m& is the fuel mass burning rate (equal to 

m aHc

Q

& χ

& ), and me & is the mass entrained into the 

upper layer, which is approximately equal to the air entrained into the upper layer [Ref. 37]): 

me & = 0.0059Q& Z/L (5.9) 

where Z is a vertical location above the surface of the fire below which air entrainment occurs 

and L is the flame height (for a fire burning in the open). Here, Z is taken as the location of the 

HGL, which can be approximated as the position the soffitt in a compartment fire with natural 

ventilation, and Z is always less than or equal to L by definition. 
Sensitivity of Model Results to Uncertainty in Measured Input Parameters 

5-4 

a c

i

i Z H

y L Y

0.0059 χ = (5.10) 

From this relationship, the sensitivity of Yi to the HRR can be inferred. The terms other than L in 

the above equation represent parameters that are assumed to be constants for the ventilated 

compartment fire experiments that represent most of the experiments considered in this report. 

In fact, the combustion efficiency and the values of yi are functions of fire conditions in the 

compartment, particularly when the fire is underventilated. Considering a small change in Yi, 

denoted asδYi , leads to: 

L

L

Y

Y

i

δ i δ

= ≈ 1⁄2 

Q

Q

&

& δ (5.11) 

and from Eq. 5.11, the uncertainty in the change of a gas species in the HGL is can be related to 

the uncertainty inQ& . An uncertainty of 15 % in the HRR, leads to a change of about 8 % in the 

CO2 and O2 volume fractions. 

5.6 Smoke Concentration 

Smoke, or soot, is a product of incomplete combustion. Smoke particulate is not a gas, but a 

complex solid whose form and concentration depend on the type of fuel and the ventilation 

conditions within the compartment. Once formed, it is transported with other combustion 

products. A simple assumption used in many zone and field fire models is that smoke is 

transported in the same way as gas products. The soot generation rate or soot yield per unit fuel 

mass, ,s y is difficult to predict and the fire models are subject to error due to uncertainty in the 

prescribed soot yield. The soot yield for BE #3 was reported as 1.5 ± 18 % for heptane [Ref.5]. 

This uncertainty, however, is a measurement uncertainty and omits changes in chemistry that 

may occur, for example, during an underventilated compartment fire. In the upper layer of a 

compartment fire, the mass of soot per unit volume, Ms, is equal to the product of the mass 

fraction of soot (Ys) and the density (ρ) of the HGL: 

Ms = ρ Ys (5.12) 

Considering a small change in Ms, denoted as δMs, and using the chain rule: 

s

s

M

δM

 = 

ρ

δρ + 

s

s

Y

δY (5.13) 

The relation between 

i

i

Y

δY and 

Q

Q

&

& δ is given by Eq. 5.11. A term for the uncertainty in the soot 

yield (

s

s

y

δy ), which arises from Eq. 5.10 must also be considered. The relation between the 

change in the density and the HRR is determined from the ideal gas law, 

ρ

δρ ~ (1/ )

(1/ )

T

δ T

 = - 

T

δT (5.14) 
Sensitivity of Model Results to Uncertainty in Measured Input Parameters 

5-5 

From Eq. 5.1, δT ≈ 2/3 Q& -1/3 δQ& , so 

T

δT = 2/3 (1- Tg

T∞ ) Q

Q

&

& δ (5.15)

Substituting Eqs. 5.11, 5.14, and 5.15 into 5.13 yields: 

s

s

M

δM = [ 2/3 

Tg

T∞ – 1/ 6] Q

Q

&

& δ

 + 

s

s

y

δy (5.16) 

An uncertainty in Q& leads to an uncertainty in Ms that is dependent on the ratio of the (absolute) 

ambient temperature to the gas temperature. For Tg equal to 200 o

C (473 K), an uncertainty of 

15 % in Q& leads to an uncertainty of about 4 %, in Ms. In this example, a value of 200 o

C was 

selected for Tg, because it represents a typical HGL temperature during the BE #3 tests. The 

uncertainty due to heat release rate is therefore small, and the dominant contributor is uncertainty 

in the soot yield. 

5.7 Pressure 

In a closed compartment, the average pressure, p0 , is governed by the equation 

( ) ( ) out in leak

0

loss

0

V

1 V V V p Q Q dt V

dp & − & − & − & + & − = γ γ (5.17) 

Here, γ is the ratio of specific heats (about 1.4), V is the volume of the enclosure, 

Qnet Q Qloss & & & = − is the net rate of energy heating up the gases in the compartment, and the V&

terms are the volume flow out of the compartment due to the exhaust, the volume flow into the 

compartment due to a fan, and the leakage, respectively. The volume flow rate due to leakage is 

given by: 

∞

− ∞ = ρ2 ( ) 0

leak leak

p p V& A (5.18) 

where A is the leakage area. The maximum compartment pressure is achieved when the pressure 

rise in Eq. 5.17 is set to zero. Rearranging terms, an estimate for the maximum pressure in the 

compartment can be made as follows: 

2

leak

net vent

0 max

( 1)

2 ( ) ⎟

⎟

⎠

⎞

⎜

⎜

⎝

⎛ − − − ≅

∞

∞ ∞ ∞ A p

Q p V

p p

γ

ρ γ γ & & (5.19) 

The term Vvent

& is the net ventilation volume flow. The expression for the maximum pressure can 

be used to estimate the uncertainty in the model prediction to changes in the measured HRR, net 

ventilation volume flow, and leakage area. 

The compartment pressure was considered using data from BE #3 only. The uncertainty in Aleak 

for the open-door experiments was very small, but for the closed door tests, an uncertainty of 
Sensitivity of Model Results to Uncertainty in Measured Input Parameters 

5-6 

10 % to 15 % is expected [Ref. 5]. The uncertainty in Qnet

& is dominated by uncertainty in Q& . In 

the absence of forced ventilation, an uncertainty of 15 % in the HRR and leads to an uncertainty 

of 30 % in pressure. For the closed door tests with ventilation, the uncertainty in Vvent

& is difficult 

to characterize because the mechanical flows were generally not well characterized in the 

experiments. It is estimated that an uncertainty of 30 % is not unreasonable, leading to an 

uncertainty of about 75 % in pressure for the closed-door vented condition. 

5.8 Heat Flux 

The radiative heat flux onto surfaces in the upper layer is nearly equal to εσT4 where ε is the 

effective emissivity of the upper layer, 11 σ = 5.67 ×10 kW/m2

/K4, and T is the upper layer gas 

temperature in Kelvin. Considering a small change in the flux, q&′′ , denoted as δ q&′′ , leads to: 

δ q&′′ = 4 T3 δT (5.20) 

Considering Eq. 5.20, the fractional change in q&′′ , (

q

q

&

&

′′

δ ′′

) is equal to: 

q

q

&

&

′′

δ ′′ ≈ 8/3 

Q

Q

&

& δ (1- Tg

T∞ ) (5.21) 

so that, an uncertainty of 15 % in the HRR, leads to an uncertainty of about 15 % in q&′′ for Tg

equal to 200 o

C (473 K), typical of the HGL temperature during the BE #3 tests. For BE #4, the 

temperature was much higher (~1000 K), and the 25 % uncertainty in the HRR leads to an 

uncertainty of about 50 % in q&′′ . 

5.9 Wall/Target Surface Temperature 

The surface temperature of the walls and targets is a function of the heat flux, the thermophysical 

and optical properties of the material, and its temperature. A simplistic approach to uncertainty 

considers a well-insulated wall or ceiling surface in the optically thick HGL of a compartment 

fire. After steady state is reached, uncertainty in the HRR leads to uncertainty in the HGL 

temperature, which translates into uncertainty of the surface or target temperature. An 

uncertainty in the HRR of 15 % corresponds to uncertainties of 10 % in the upper layer and wall 

temperatures. 

5.10 Summary 

The empirical correlations described in this chapter provide recipes to calculate the uncertainty 

in the model outputs resulting directly from the uncertainty in key input parameters. For 

example, a two-thirds power dependence of HGL temperature on the HRR means that the 

uncertainty in the predicted HGL temperature is about two-thirds the uncertainty in the measured 

HRR. This uncertainty does not include the measurement uncertainty. For example, it does not 

include whatever error is associated with the thermocouple itself. For many of the measured 

quantities under consideration, the model input uncertainty is greater than the measurement 

uncertainty as discussed in the next chapter. 
Sensitivity of Model Results to Uncertainty in Measured Input Parameters 

5-7 

Table 5-1 lists the measured quantities, along with the parameters that most influence them. The 

last column in the table lists the uncertainties in the given quantity based on the combined 

uncertainty of the key parameters. For illustrative purposes, the propagated model input 

uncertainty results, Um, shown in the Table is based on an expanded (2σ) uncertainty of 15 % for 

the HRR, although this value actually varied from 15 % to 25 % for the six test series (see Table 

2-1). The resulting values of the propagated uncertainty for the various parameters should be 

regarded as the expanded value. 

Table 5-1: Summary of the model sensitivity, Um, to uncertainty in the HRR 

Quantity Input Parameter Power Dependence Expanded Uncertainty, 

Um (%) 

HGL Temperature, Tg Q& 2 / 3 10 

HGL Depth, z Q&, A, H Eq (5.3) 2 

Ceiling Jet Temperature Q& 2 / 3 10 

Plume Temperature Q& 2 / 5 1.3 / 3.3 * < z D <

* 2 / 3 3.3 < z/ D

 6 

 10 

Flame Height Q& , D 1⁄2 8 

Gas Concentrations Q& 1⁄2 8 

Smoke Concentration Q& , s y Eq (5.16), 1 18 ** 

Pressure Q& , Aleak , Vvent

& 2, 2, 2 30 (no forced ventilation)

 75 (with ventilation) 

Heat Flux, q&′′ Q& Eq (5.21) 15 ** 

Surface/Target 

Temperature Q& 2/3 10 

* actual uncertainty in the HRR varied from 15 % to 25 % among the experiments, and there was also

some variation among the tests for a single experimental series (see Table 2-1 and text). 

** for Tg ≈ 200 o

C. 

6-1 

6 

REPRESENTATIVE UNCERTAINTIES 

6.1 Summary of the Estimated Measurement Uncertainty and Subsequent 

Model Input Uncertainties 

A summary of the estimated measurement and model input uncertainties are presented in this 

section. The uncertainties for each of the parameters of interest in each of the experimental series 

are listed below in Tables 6-1 to 6-7. Each table contains information on the expanded (2σ) 

relative measurement uncertainty (Ue), the expanded (2σ) relative model input uncertainty 

associated with model input uncertainty (Um), and the combined relative expanded (2σ) 

uncertainty (Uc), defined in Eq. 1.3 as: Uc = Uε = (Ue 2

 + Um 2 ) 1⁄2. The values of Ue are taken 

from Tables 3-2 to 3-9. The values of the HRR uncertainty for each experiment are taken from 

Table 2-1. 

Table 6-1: Summary of the Relative Expanded Uncertainties Associated 

 with the HGL Layer Depth and Temperature Rise 

Series HGL Depth HGL Temperature Rise 

Ue 

(%) 

Um

(%) 

Uc

(%) 

Ue 

(%) 

Um

(%) 

Uc 

(%) 

NBS 6 2 6 6 * 10 12 

FM/SNL 23 2 23 10 13 16 

BE #2 8 2 8 9 10 13 

BE #3 6 2 6 6 11 12 

BE #4 22 2 22 6 17 18 

BE #5 9 2 9 7 10 12 

* the value of Ue does not accurately account for radiative exchange in Test 100A, in which 

natural gas was the fuel; its value is likely small (see text in Section 3.2.3). 
Representative Uncertainties 

6-2 

Table 6-2: Summary of the Relative Expanded Uncertainties Associated with Ceiling Jet 

Temperatures 

Ceiling Jet Temperature Rise 

Series Ue 

(%) 

Um

(%) 

Uc

(%) 

BE #3 12 11 16 

FM/SNL 4 13 14 

Table 6-3: Summary of the Relative Expanded Uncertainties 

 Associated with Plume Temperatures 

Plume Temperature Rise (°C) 

Series Ue 

(%) 

Um

(%) 

Uc

(%) 

BE #2 upper 

BE #2 lower 

9 

11 * 

10 

10 

14 

15 

FM/SNL 4 13 14 

* the value of Ue does not accurately account for radiative exchange; 

its value is likely small (see text). 
Representative Uncertainties 

6-3 

Table 6-4: Summary of the Relative Expanded Uncertainties Associated 

 with Oxygen and Carbon Dioxide 

HGL CO2 Concentration HGL O2 Concentration 

Decrease 

LGL O2 Concentration 

Decrease 

Series 

Ue 

(%) 

Um

(%) 

Uc

(%) 

Ue 

(%) 

Um

(%) 

Uc

(%) 

Ue 

(%) 

Um

(%) 

Uc

(%) 

BE #3 4 8 9 4 8 9 4 8 9 

BE #5 4 8 9 4 8 9 4 8 9 

Table 6-5: Summary of the Relative Expanded Uncertainties Associated with the Smoke 

Concentration and the Compartment Pressure 

Smoke Concentration Pressure 

Series Ue 

(%) 

Um

(%) 

Uc

(%) 

Ue 

(%) 

Um

(%) 

Uc

(%) 

BE #3 28 20 33 27 30 (no vent) 

75 (vent) 

40 (no vent) 

80 (vent 

Table 6-6: Summary of the Relative Expanded Uncertainties Associated with the Target 

Heat Flux and Target Temperatures 

Total Heat Flux to Targets Rise in Target Surface 

Temperature 

Series 

Ue 

(%) 

Um

(%) 

Uc

(%) 

Ue 

(%) 

Um

(%) 

Uc

(%) 

BE #3 10 16 19 10 11 14 

BE #4 10 47 48 10 13 16 

BE #5 10 13 17 10 10 14 
Representative Uncertainties 

6-4 

Table 6-7: Summary of the Relative Expanded Uncertainties Associated with the Surface 

Heat Flux and Surface Temperatures 

Total Heat Flux Rise in Surface Temperature 

Series Ue 

(%) 

Um

(%) 

Uc

(%) 

Ue 

(%) 

Um

(%) 

Uc

(%) 

BE #3 10 16 19 10 11 14 

BE #4 10 47 48 10 13 16 

BE #5 10 13 17 10 10 14 

6.2 Representative Uncertainties 

Using engineering judgment, a weighted expanded combined uncertainty, Ucw, representative 

of the uncertainty for each of the parameters was estimated: 

i i c

i

i

cw U n

n

U ∑

∑ =

=

= 6

6 1

1

1

 (5.1)

where the weighted average used the combined expanded uncertainties (Uc listed in Tables 6-1 

through 6-6), and was based on the sum of the number of tests (ni) (see Table 1-1) for the 

parameters of interest for each of the six test series (i). A weighted average based on the number 

of experimental tests was used rather than some other weighting approach, because the model 

evaluation considered each of the 26 tests as independent trials (see Chapter 6, Volumes 2 to 6). 

The number of experiments (and the associated number of tests), and these varied for each 

parameter. The weighted expanded combined uncertainty, Ucw, and the total number of tests 

(∑=

6

i 1

ni ) for each of the parameters is shown in Table 6-8. Fifteen of the tests were from BE #3, 

and in this sense, the results from BE #3 strongly influenced the results. 

Table 6-8: The weighted combined expanded uncertainty, Ucw, determined from Eq. 5.1 and 

Tables 6-1 through 6-7 

Quantity Number of Tests Weighted Expanded Combined Uncertainty, 

Ucw (%) 

HGL Temperature rise, Tg

26 13 

HGL Depth, z 26 9 

Ceiling Jet Temperature 18 16 

Plume Temperature 6 14 
Representative Uncertainties 

6-5 

Quantity Number of Tests Weighted Expanded Combined Uncertainty, 

Ucw (%) 

Gas Concentrations 16 9 

Smoke Concentration 15 33 

Pressure 15 40 (no forced ventilation) 

80 (with forced ventilation) 

Heat Flux, q&′′ 17 20 

Surface/Target Temperature 17 14 

6.3 Conclusions 

In this evaluation, fire experiments are used as a way of establishing confidence in model 

predictions. As part of the evaluation considered in this report series, the effect of experimental 

uncertainty on both models and measurements is considered. A literature search was conducted 

to select experiments that would allow evaluation of key aspects of fire models. In general, the 

report of experiments did not adequately address measurement uncertainty. 

There may be many reasons for this, but an important reason is that the expense of performing a 

comprehensive uncertainty analysis is often larger than the cost of the measurement itself. 

Without such analysis, however, the fire measurements are unlikely to experience improvement 

in accuracy. Experimental design and execution must carefully address measurement uncertainty, 

and in particular, attempt to reduce the uncertainty in key experimental parameters, most notably, 

the fire HRR. The ability to accurately and precisely measure HRR requires a significant 

institutional investment, as a facility must be well-instrumented, its performance characterized, 

and the uncertainties analyzed and documented. 

In this study, measurement uncertainty was estimated for the experiments of interest using 

engineering judgment. The importance of the uncertainty in the HRR was emphasized in this 

study, because this parameter drives the thermal environment in the model calculation results, 

and the calculation results are sensitive to the uncertainty of this parameter. Quantifying the 

uncertainty in the HRR provides a basis for understanding model sensitivity to this parameter, 

taken here as a representation of model input uncertainty. To date, there have not been many fire 

model verification and validation studies based on ASTM 1355. There have been few that have 

tried to document experimental uncertainty as part of the analysis. The uncertainty in model 

output was assessed through consideration of the uncertainty in measured quantities that are used 

as input for the models. Both measurement uncertainty and model input uncertainty are found to 

be important and both contribute to the combined uncertainty. The information from this volume 

is used as a basis for the model evaluation as discussed in Volumes 2-6 of this report series.

7-1 

7 

REFERENCES 

1. ASTM Standard Guide for Evaluating the Predictive Capability of Deterministic Fire 

Models, ASTM E1355-05, American Society for Testing and Materials, West 

Conshohocken, PA 2005. 

2. Nowlen, S.P., Enclosure Environment Characterization Testing for the Base line Validation 

of Computer Fire Simulation Codes, NUREG/CR-4681, SAND86-1296, Sandia National 

Laboratories, Albuquerque, NM, March 1987. 

3. Peacock, R. D.; Davis, S.; Lee, B. T. Experimental Data Set for the Accuracy Assessment of 

Room Fire Model, National Bureau of Standards, Report NBSIR 88-3752; 120 p., April 

1988. 

4. Hostikka, S., Kokkala, M., Vaari, J., “Experimental Study of the Localized Room Fires,” 

NFDC2 Test Series, VTT Research Notes 2104, 2001. 

5. Hamins, A., Maranghides, A., Johnsson, E., Donnelly, M., Yang, J., Mulholland, G., and 

Anleitner, R., Report of Experimental Results for the International Fire Model Benchmarking 

and Validation Exercise #3, NIST Special Publication 1013-1, National Institute of Standards 

and Technology, Gaithersburg, MD, 2005. 

6. Klein-Heßling, W. and M.Röwenkamp, Evaluation of Fire Models for Nuclear Power Plant 

Applications: Fuel Pool Fire inside a Compartment. May 2005. 

7. Hosser, D., Riese, O., and Klingenberg, M., Performing of Recent Real Scale Cable Fire 

Experiments and Presentation of the Results in the Frame of the International Collaborative 

Fire Modeling Project ICFMP, June 2004. 

8. International Organization for Standardization (ISO), “Guide to the Expression of 

Uncertainty in Measurement,” Geneva, Switzerland, 1993. 

9. Taylor, B.N., and Kuyatt, C.E., Guidelines for Evaluating and Expressing the Uncertainty of 

NIST Measurement Results, National Institute of Standards and Technology (NIST) 

Technical Note 1297, 1994. 

10. Hogg, R.V. and Tanis, E.A., Probability and Statistical Inference, 2nd Ed., MacMillan, 1983. 

11. Bryant, R., Ohlemiller, T., Johnsson, E., Hamins, A., Grove, B., Guthrie, W.F., Maringhides, 

A., and Mulholland, G., The NIST 3 Megawatt Quantitative Heat Release Rate Facility - 

Procedures and Guidance, NIST Special Publication 1007, National Institute of Standards 

and Technology. Gaithersburg, MD, December 2003. 

12. Axelsson, J., Andersson, P., Lonnermark, A., VanHees, P. and Wetterlund, I., Uncertainties 

in Measuring Heat and Smoke Release Rates in the Room/Corner Test and the SBI, SP 

Report 2001:04 SP Swedish National Testing and Research Institute, Boras, Sweden, (2001). 
References 

7-2 

13. Enright, P.A., and Fleischmann, C.M., Uncertainty of Heat Release Rate Calculation of the 

ISO5660-1 Cone Calorimeter Standard Test Method, Fire Technology 35, 153-169 (1999). 

14. Yeager, R.W., Uncertainty Analysis of Energy-Release Rate Measurement for Room Fires, J. 

Fire Sciences 4, 276-296 (1986). 

15. Janssens, M.L., Variability in Oxygen Consumption Calorimetry Tests, (1427), American 

Society for Testing and Materials (ASTM), Dallas, TX, 147-162 (2002). 

16. Tewarson, A., Prediction of Fire Properties of Materials, Part 1., National Bureau of 

Standards, NBS-GCR-86-521, December 1986. 

17. Twearson, A., Generation of Heat and and Chemical Compounds in Fires, The SFPE 

Handbook of Fire Protection Engineering, 3rd Ed. (Section 3, Chap 4), National Fire 

Protection Association and The Society of Fire Protection Engineers, Quincy, MA, (Eds.: P. 

J. DiNenno, D. Drysdale, C. L. Beyler, and W. D. Walton, eds.), 2003. 

18. Barbrauskas, V., Estimating Large Pool Fire Burning Rates, Fire Technology, 19, 251-261, 

1983. 

19. NUREG/CR-5384, A Summary of Nuclear Power Plant Fire Safety Research at Sandia 

National Laboratories, 1975–1987, December 1989. 

20. Nowlen, S.P., Enclosure Environment Characterization Testing for the Base line Validation 

of Computer Fire Simulation Codes, NUREG/CR-4681, SAND86-1296, Sandia National 

Laboratories, Alburquerque, NM, March 1987. 

21. Hamins, A., Maranghides, M., and Mulholland, G., The Global Combustion Behavior of 

1 MW to 3 MW Hydrocarbon Spray Fires Burning in an Open Environment, National 

Institute of Standards and Technology. Gaithersburg, MD, Internal Report NISTIR 7013, 

June 2003. 

22. Hamins, A., Klassen, M., Gore, J., and Kashiwagi, T., Estimate of Flame Radiance via a 

Single Location Measurement in Liquid Pool Fires, Combust. Flame, 86, 223-228 (1991). 

23. Fire Modeling Code Comparison (TR-108875), Electric Power Research Institute (EPRI), 

Palo Alto CA. 

24. Hostikka, S., VTT, Helsinki, Finland, personal communication, 2005. 

25. Bundy, M., NIST, Gaithersburg, MD, personal communication, 2005. 

26. Peacock, R.D., and Babrauskas, V., Analysis of Large-Scale Fire Test Data, Fire Safety 

Journal, 17, 387-414, 1991. 

27. Blevins, L.G., Behavior of Bare and Aspirated Thermocouples in Compartment Fires, 

Proceedings of the 33rd National Heat Transfer Conference, Albuquerque, NM., August 15-

17, 1999. 

28. Pitts, W. M., Braun, E., Peacock, R. D., Mitler, H. E., Johnsson, E. L., Reneke, P. A., and 

Blevins, L. G., Temperature Uncertainties for Bare-Bead and Aspirated Thermocouple 

Measurements in Fire Environments, National Institute of Standards and Technology (NIST), 

Internal Report NISTIR 6242, October 1998. 

29. Hamins, A., Maranghides, A., Johnsson, E., Donnelly, M., Yang, J., Mulholland, G., and 

Anleitner, R., Report of Experimental Results for the International Fire Model Benchmarking 
References 

7-3 

and Validation Exercise #3, NIST Special Publication 1013-1, National Institute of Standards 

and Technology, Gaithersburg, MD, 2005. 

30. Hamins, A., Maranghides, A., McGrattan, Ohlemiller, T., and Anleitner, R., Federal Building 

and Fire Safety Investigation of the World Trade Center Disaster: Experiments and Modeling 

of Multiple Workstations Burning in a Compartment, NIST Special Publication NCSTAR 1-

5E. National Institute of Standards and Technology. Gaithersburg, MD, June 2005. 

31. OMEGA Engineering Inc., Appendix B, Thermocouple Characteristics in The Temperature 

Handbook, 2000. 

32. Mulholland, G.W., and Croakin, C., Specific Extinction Coefficient of Flame Generated 

Smoke, Fire and Materials, 24, 227-230, 2000. 

33. Bryant, R., Womeldorf, C., Johnsson, R., and Ohlemiller, T., Radiative heat flux 

measurement Uncertainty, Fire and Materials, 27:209–222 (2003). 

34. Pitts, W.M., Murthy, A.V., de Ris, J.L., Filtz, J.-R., Nygård, K., Smith, D., Wetterland, I., 

Round Robin Study of Total Heat Flux Gauge Calibration at Fire Laboratories, National 

Institute of Standards and Technology (NIST), Special Publication 1031, October 2004. 

35. Walton, W.D. and Thomas, P.H., Estimating Temperatures in Compartment Fires, The SFPE 

Handbook of Fire Protection Engineering, 3rd Ed., (Section 3, Chap 6), National Fire 

Protection Association and The Society of Fire Protection Engineers, Quincy, MA, (Eds.: P. 

J. DiNenno, D. Drysdale, C. L. Beyler, and W. D. Walton), 2003. 

36. Alpert, R.L., Ceiling Jet Flows, The SFPE Handbook of Fire Protection Engineering, 3rd 

Ed., (Section 2, Chapter 2), National Fire Protection Association and The Society of Fire 

Protection Engineers, Quincy, MA, (Eds.: P. J. DiNenno, D. Drysdale, C. L. Beyler, and W. 

D. Walton), 2003. 

37. Milke, J.A., Smoke Management in Covered Malls and Atria, The SFPE Handbook of Fire 

Protection Engineering, 3rd Ed. (Section 4, Chapter 13), National Fire Protection Association 

and The Society of Fire Protection Engineers, Quincy, MA, (Eds.: P. J. DiNenno, D. 

Drysdale, C. L. Beyler, and W. D. Walton), 2003. 

38. Heskestad, G., Fire Plumes, Flame Height and Air Entrainment, The SFPE Handbook of Fire 

Protection Engineering, 3rd Ed. (Section 2, Chap 1), National Fire Protection Association 

and The Society of Fire Protection Engineers, Quincy, MA, (Eds.: P. J. DiNenno, D. 

Drysdale, C. L. Beyler, and W. D. Walton, eds.), 2003. 

39. Heskestad, G., Luminous Heights of Turbulent-Diffusion Flames, Fire Safety J. 5, 103-108 

(1983). 

A-1 

A DOCUMENTATION OF PERSONAL 

COMMUNICATIONS 

Bundy, Matthew, NIST, Gaitherburg, MD. Conversation at NIST during May, 2005 on 

determination of the combustion efficiency and the heat release rate in heptane spray flames in 

the NIST Large Fire Laboratory. Considering stoichiometry of the combustion and 

measurements in the exhaust hood, a combustion efficiency of 0.97 was determined for a 500 

kW heptane spray fire. 

Hostikka, Simo, VTT, Helsinki, Finland. Telephone conversation on May 2005 discussing the 

surface materials used during BE #2.

B-1 

B TIME AVERAGING 

Figure B-1 shows various periods used to time-average heat flux data acquired at 1 Hz for Gauge 

9 during Test 13 of BE #3. The time response of the measurement itself is on the order of 1 s. 

The data used in this report were averaged over a 10 s period. Time averaging the data over too 

long a period may cause the loss of legitimate data and valuable information. No time-averaging 

at all, causes signal noise to play a role in the determination of peak values. This issue would 

have less significance if there had been many report experiments, and a statistically large number 

of peak values had been determined for one set of experimental conditions, but this was not the 

case. The selection of a 10 s averaging period is a compromise, but appears to be reasonable for 

the types of measurements considered in this report. 

0

2

4

6

8

10

12

14

16

0 100 200 300 400 500 600 700 800 900 1000

Time (s)

Flux (kW/m2

)

Test 13 Gauge 9

10 sec averaging

20 sec averaging

30 sec averaging

Figure B- 1: Various periods used to time average heat flux data acquired at 1 Hz for 

Gauge 9 during Test 13 of BE #3.