NISTIR 88-3878

Hazard I - Results of a

User Evaluation of the

Prototype Software

R. W. Bukowski

U.S. DEPARTMENT OF COMMERCE

National Institute of Standards and Technology

(Formerly National Bureau of Standards)

National Engineering Laboratory

Center for Fire Research

Gaithersburg, MD 20899

September1988

Issued November 1988

75 Sdfimitatino Amenca* Progress

NISTIR 88-3878

Hazard I - Results of a

User Evaluation of the

Prototype Software

R. W. Bukowski

U.S. DEPARTMENT OF COMMERCE

National Institute of Standards and Technology

(Formerly National Bureau of Standards)

National Engineering Laboratory

Center for Fire Research

Gaithersburg, MD 20899

September 1988

Issued November 1988

National Bureau of Standards became the

National Institute of Standards and Technology

on August 23, 1988, when the Omnibus Trade and

Competitiveness Act was signed. NIST retains

all NBS functions. Its new programs will encourage

improved use of technology by U.S. industry.

U.S. DEPARTMENT OF COMMERCE

C. William Verity, Secretary

NATIONAL INSTITUTE OF STANDARDS

AND TECHNOLOGY

Ernest Ambler, Director
:t‘

',.' - '^. y: - V

'

= 3 n

'I Jcftsyfs"^' '5*'^^

^' '','rvA

"5

-,W'',_ , :S i r | 4

'

"ii-- ""'v l« ^ \

"' '

V'

V'

' ':! j,; /pnr.;,;' ’>

' '

>-» '*jV** f»t •• -ft • V 'V Tb'>J^#Fr “??>'':• ;0^ ''A'^

.uj4:; *; '.;V X •i^.5’V5C '• 0- 'j»(tV». s^C*t

' <•. .;, T' 3'IV! ,". ;• T ' A: • %-' ' '

:

. *

I ' '

' "," ' ' '' '•’ 'vS'*'' .3S^fiijri?K00 "'sgi'waMTRAi^ff MAi,.

iatp'#;,i»#ii»';&^^r •_,.

\' •,.

f~/{', -‘ -« i„ .^^i'V' it r ,. ‘g •

.
Table of Contents

Table of Contents iii

Abstract 1

1.

Background 2

2.

Goals of the Beta Test

3. Selecting Participants 4

4. Requirements for Participation 5

5. Results 7

5.1 Participant Response 7

5.2 Hardware Compatibility 7

5.3 Software CompatibiHty 8

5.4 User Compatibility 9

6. Specific Programs, Comments and Fixes 11

6.1 PRODUCT.ONE 11

6.2 HREDATA 12

6.3 MLTFUEL 12

6.4 HNPUT 12

6.5 FAST 13

6.6 FASTPLOT 13

6.7 DETACT 14

6.8 EXITT 14

6.9 TENAB 15

7. User Needs 16

7.1 Breadth (flexibility) 16

7.2 Ease of Use 17

7.3 Execution Time 17

7.4 Technical Depth 18

8. Applications 19

9. Conclusions 22

APPENDIX A 23

HAZARD I - Results of a User Evaluation

of the Prototype Software

Richard W. Bukowski, P.E., Head

Fire Hazard Analysis

Center for Fire Research

Abstract

After five years of development, the prototype of a personal computer based, fire

hazard analysis method was distributed to 93 volunteers representing all aspects of

the fire community. These persons agreed to evaluate the software and documentation, and attempt to apply it to a problem of their own choosing. Written

comments were to be returned, which would be used to establish priorities for future

changes, and where possible, be incorporated into the general release version of the

product.

Written responses were received from 47 participants, most of which dealt with

suggestions for improvements to the user interface (rather than any technical

shortcomings). Based on the responses received, it has been concluded that: the

software will be of substantial, broad benefit; with the identified improvements, the

user interface is comparable to commercial software in ease of use; the data base

is particularly useful, but needs to contain many more entries; and priority

enhancements need to be made in the areas of combustion modeling and pictorial

graphics.

Key Words: computer models; computer programs; evaluation; fire models; hazard

models

Page 1
1. Background

In July of 1983, the Center for Fire Research (CFR) made the commitment to produce a practical fire hazard assessment method within three to five years. In July of 1987, the first

embodiment of this method, called HAZARD.!, was approved for release to and evaluation

by a limited group representing all facets of the eventual user community.

The organized evaluation by users of computer software is such a common practice that it has a name - beta testing. Specifically, the alpha test version of software is that evaluated

within the developing organization, but by persons not directly involved in its creation. The

beta test version is then made available (often sold at a price significantly below the

introductory retail price) to users. Beta testers are encouraged to report problems and

provide comments on the software so that the release version is both free of "bugs", and

meets the expectations of the user audience. Often, beta testers are provided a copy of

the final release version of the software at reduced price (or free).

The details of the beta test plan for HAZARD I were developed to provide information

on the expectations of the fire community with respect to the design and implementation

of such software in order to:

• assure that their experience with hazard analysis software is not so negative

as to adversely impact people’s willingness to use it,

• identify the degree to which the package meets the needs of the range of

potential users,

• document the expectations of the fire community as to the level of "user

friendliness" in the software and documentation, and

• identify the level of thoroughness required in the science for the applications

of interest to our constituency.

Since the fire community is made up of a number of disciplines each of which might have

different needs and requirements, the test program was designed to encourage a relatively

balanced representation from each. Also, since CFR staff have little experience in most

of these areas, surveys or other forced responses which might inhibit users from freely

reacting to their experience were avoided. Thus the general philosophy was to treat each

participant like the purchaser of a commercial software package and deal with their

complaints and comments as they wish to make them.

Page 2
2. Goals of the Beta Test

The beta test was intended to provide feedback on:

• Usefulness

• User friendliness

• Application-specific improvements/modifications

• Hardware compatibility

• Level of interest

As was discussed above, the process was intended to provide specific feedback on the

HAZARD I package, and general guidance on the needs and expectations of the fire

community for analytical software. Information on the computer skill level of the "typical"

user, equipment available to them, appropriate technical level for the software and

documentation, and the degree to which judgement can be assumed, all were to be explored.

From the comments and suggestions received, priorities for improvements have been

assigned. In addition, several issues of concern with the package were identified by CFR

staff during its development and review. There was a desire to determine if the points

which bother us also cause concern to our "customers." With regard to these specific issues,

if no specific responses were received, they could be explored by personal contact with one

or more users who would be knowledgeable in that area. Likewise, comments or complaints

that we did not expect or which show very strong feelings were followed up by telephone.

Another important topic to be addressed was that of hardware compatibility - not only with

foreign hardware, but also with systems such as UNIX computers which run DOS as a task.

We need to know if there are common system configurations which are incompatible with

the way in which the package is installed (e.g., the use of external drives for system and

user software).

A final aspect to the process is the need to assess the level of interest by the fire

community in a hazard analysis method which can be used to assist them in their work.

To give verbal support to such a concept is easy. To be willing to invest time in selflearning one demonstrates a real commitment to it. If there is insufficient commitment, the

required investment may be considered too large and require a change to our research

priorities. But if the commitment is high, we can expect that a viable method will be

incorporated into the fabric of fire protection practice almost as fast as we can deliver it.

One issue which the beta test was not designed to address is accuracy assessment. While

we certainly expected to hear about any cases where the models gave unusual results, the

fact that they did or did not is not proof of accuracy or a lack thereof. Such assessments

can only be made through a statistically-designed program of validation studies.

Page 3
3. Selecting Participants

Participants were secured by invitations, sent initially to persons selected from various CFR

mailing lists, and later from inquiries received in response to a press release or to

recommendations from other participants. The package of material provided with the

invitation letter included a description of the software capabilities, assumptions, and

limitations; and the "Getting Started" booklet. This booklet is a verbatim copy of the use

of the software on an example problem, intended to show clearly the amount of work

involved in a case. In addition, the letter estimated the need for about 40 hours of selfstudy before the user is prepared to work a case of his or her own. The primary intent

of all this was to make sure that all participants understood the extent of their commitment.

Respondents were asked to classify their interests into one or more of eight user categories

which were used to insure a relatively even representation. In responding to unsolicited

inquiries, the choice of sending only an information package or a package and invitation

to participate was used to effect this control. One category (fire protection engineering)

had to be closed to additional participation due to too many respondents. In general, there

was a limit of one package per company or organization; although users were free to copy

the software and documentation, and several did.

A total of 217 invitation packages were sent out, resulting in 93 registered beta test

participants. The representation by application area is as presented below (each

organization selected multiple application areas). Several of the participating organizations

with multiple offices distributed copies to each office, but counted as only one participant.

In addition, a few copies of the software were provided to organizations who were not

official participants and from whom a response was not required.

Of the 93 registered participants, 72 were within the United States, and the remainder

distributed as follows: Canada (1), England (2), Sweden (3), Australia (3), New Zealand (1),

Norway (1), Germany (3), Spain (1), and Japan (6). In each category, the organizational

sizes varied from one person businesses to large, multiple office operations.

The eight user categories along with their final distribution of participants were:

Fire investigation/reconstruction - 37

Fire Protection engineering (design or analysis) - 37

Architectural design - 15

Code administration/enforcement - 24

Product development/manufacture/marketing - 18

Fire services - 14

Fire research/testing - 37

Public fire safety education - 14

Page 4
4. Requirements for Participation

Each recipient was required to respond in writing in order to participate in the program.

This response was to include:

• the name, address, and telephone number of the person actually working

with the software, so that we could maintain direct contact through a series

of newsletters and by telephone;

• a detailed description of the computer hardware on which the software was

installed, to document any hardware compatibility problems;

• the commitment to learn the system and then apply it to a problem of their

choosing, which they hope the software would properly address; and

• respond in writing with observations, comments, and suggestions on their successes or failures.

In some cases, participants commented on the problems which they planned to tackle with

the system. Examples of these excerpted from their acceptance letters include:

• A producer of a fire-resistant product is comparing the performance of his

product to the traditional alternative to demonstrate the benefits of its additional cost, and another is developing specifications for new products.

• A code official is evaluating the impact of a proposed code change for use

as supporting data in his presentation to the legislative body.

• A fire investigator is testing the ability to reconstruct a residential fire

incident by predicting in advance the result of a test in which an abandoned

house will be burned.

• One fire department is evaluating proposals for alternative means of

compliance with the code, another for firefighter training, and a third for

educating the public on the benefits of residential detectors and sprinklers.

• A government agency is evaluating emergency evacuation procedures for fires

in underground mines.

• A university is using the package in conjunction with a course on fire

dynamics and predictive methods.

Page 5
A fire researcher is evaluating fire safety on large ferry boats, and another

on offshore oil platforms; both are in conjunction with projects which include

small- and large-scale testing.

Several consultants are conducting analyses for use in civil and criminal

litigations. Topics include the relative contribution of one burning item to

the overall outcome, the potential impact of a detector which was not

present or not operating, and in supporting a finding of arson. A testing laboratory is reducing the number of tests of varying geometric

arrangements by using the models to reproduce the tested configuration and

then predict others.
5. Results

5.1 Participant Response

As of the date of this writing, 47 participants have provided written comments on their

thoughts and experiences. This represents a response rate of 50%. These collected

thoughts are summarized in the following sections.

5.2 Hardware Compatibility

In their original acceptance letter, each participant provided details of the computer on

which the software was operated, allowing us to verify the compatibility of the software with

a broad range of computers, and to assist in troubleshooting user problems. A number of

the larger users identified multiple machines on which the software would be installed.

The user hardware represented the equipment of 27 manufacturers, including U.S.,

Japanese, and European origin. The distribution of processor types was; 54 (8088), 35

(80286), and 6 (80386) machines. All three of the supported graphics systems (CGA, EGA,

and Hercules) were represented by a myriad of card and monitor suppliers.

Hardware incompatibilities were surprisingly few. Two users of AT&T machines obtained

a peculiar run-time error message while the software was trying to write to the screen. The

associated error code is described by the compiler producer as of unknown origin, and is obtained during input or output operations on some hardware (it was reported only by

these two AT&T users). They go on to say that not only do they not know what causes

it, but they have no intention of fixing it. A problem with Japanese machines is related to their typical 5 1/4 in. floppy drives being

650 kB density, so they cannot read our 360 kB disks. This drove most users in Japan to

buy a U.S. made computer or at least install a U.S. drive. Late in the program, one U.S.

user bought a PS/2 machine which had only a 3 1/2 in. drive and found that he could not

transfer the software to it. We supplied him with a copy on 3 1/2 in. format which he

installed with no problem.

Page 7
5.3 Software Compatibility

There were two reported problems related to "bugs" in DOS 2.X. The first involved the

batch file (HAZARD.BAT^ which calls the programs from the main menu. The batch file

branches to a labeled line in response to the keyboard input. The number of characters

in the labels varied, which is allowed in DOS 2.X as long as the first eight characters are unique. In fact what DOS 2.X does is to truncate the label at eight characters, resulting

in a mismatch and a "label not found" error for any label longer than eight characters. This

"bug" does not appear in DOS 3.X, so it was not found earlier. The fix was simply to

shorten the labels in the batch file to no more than eight characters. This could be done

easily by the user, or we supplied a corrected file that could be downloaded from the CFR

bulletin board (CFRBBS).

The second DOS 2.X "bug" was more subtle. Users would randomly get the error message

"File Allocation Table Bad on Drive requiring a re-boot of the system. We do not

know what causes it, but it has never appeared on a system running DOS 3.X. One user who was having this problem consistently solved it by upgrading his DOS version.

An incompatibility about which we warned users from the start is the fact that the graphics

drivers that we supply are not compatible with ANSI.SYS - a utility supplied with DOS to

allow custom configuration of your screen (to display information like time, date, directory,

etc. at any location). There were a few problems associated with users initially not seeing

the instruction to remove the call to ANSI.SYS from their CONFIG.SYS file before trying

to use the HAZARD I graphics. A reminder in the first newsletter took care of most of

these problems.

While we supplied printer drivers for the CGA and Hercules graphics, we did not supply

one for the EGA since it operates in color, requiring significantly more coding to map the

colors to patterns for output to a monochrome device. Instead, we recommended a commercial printer interface software package such as PIZZAZ, which is inexpensive and

does this for you. When triggered, PIZZAZ pauses the task while it transfers the screen

image to the printer. The problem is that after the copy is made, PIZZAZ fails to resume

the task and the computer must be re-booted.

There was also a problem noted with the CGA printer driver. While the CGA graphics

driver properly displays the graph in monochrome on the screen, the printer driver

alternates the curves between the foreground and background colors. This results in the

curves 2, 4, 6, and 8 disappearing on the printer output. While this might seem to be an

easy fix, it was not; so we chose to address this and several other problems with a single

change which will be discussed later.

Page 8
A minor problem was encountered with BASIC. The batch file calls BASICA (the IBM

version), but many clones use BASIC, GWBASIC, or even N88BASIC (on a NEC machine

in Japan). This required a simple correction to the batch file. Once this change was made,

only N88BASIC had problems running the three BASIC programs provided in HAZARD I.

There were no reports of conflicts with any resident programs, even though the graphics

driver is resident - such conflicts usually involve two programs vying for the same computer

resources. Since this is a fairly common problem with resident software, we were surprised

that such did not occur.

5.4 User Compatibility

This addresses the degree to which the users were able to cope with the software and

supporting documentation. This is where the majority of the problems were encountered.

If there is one thing we have learned from the beta test, it is that we must make the

installation and set-up of the software as automatic as possible. There are a number of

users who run applications software on their computer with "(little or) no working

knowledge of DOS or the PC itself’, as one respondent pointed out. This created problems

such as when BASIC was not on the machine or the PATH was not defined such that it could be found. Likewise, several users had trouble with the question about what graphics

board was present in their machine. Several participants wrote that they did not understand

directories and default drives - even though these were handled automatically.

Based on these observations, the use of a carefully-written installation routine is crucial to

the success of a software product for general use. With HAZARD I, we were assured that

all of the software was installed in a uniform manner, making troubleshooting easier when

people called with problems. All of the needed files were sure to be copied to the correct

place, and properly named. The remaining problems generally involved the incorrect

selection of the graphics system or errors in the path name to DBASE.

What was somewhat surprising was the large assortment of disk storage arrangements found

to be in use. Many systems are shared among staff who have their own removable platter

for an external drive. Common use software is then on an internal drive or second external

platter (the latter being non-bootable). This means that the software needed to run the

program can be scattered over multiple drives, leading to complex path statements. This

also means that the install program must allow full freedom of drive and file path for the

source and target drives, and system files.

To minimize these problems, the software should be made independent of files in other

directories. This means that everything needs to be compiled. Compiling all BASIC

modules will also correct problems associated with variations in BASIC interpreters and

Page 9
even the name of the interpreter (e.g., BASICA vs GWBASIC vs N88BASIC in Japan).

By compiling the DBASE program files we can also eliminate the need to access the

DBASE interpreter. The plan for HAZARD II is to create our own data base without a proprietary database software system.

Several users requested an "escape button" to stop a run at any time from any module, and

return to DOS. We simply need to point out that such exists in DOS (Ctrl C). Others

commented on the need for overall consistency in the way in which data is input. Two

examples are (1) with all but some inputs in FIREDATA, a <RETURN> is required after

any response and (2) the default entry for (Y or N?) responses vary. In the latter case,

this was done intentionally so that the default is the normally-expected response to that

question.

Also noted was that "bootleg" copies often did not work because they were

COPY ’d rather than DISKCOPY ’d - which did not copy the LABEL used by INSTALL

to verify the correct disk.

Page 10
6. Specific Programs, Comments and Fixes

In the following sections, detailed user comments on each of the programs in HAZARD

I will be discussed, along with a description of the changes which have been made to the

general release version (referred to as HAZARD I vl.O) to address these problems. A

block diagram of HAZARD I is presented in figure 1.

HAZARD I SOFTWARE

Figure 1

6.1 PRQDUCrONE

The only complaint was that many users did not understand the purpose of this module.

Most ignored it; it could be eliminated if desired.

Page 11
6.2

FIREDATA

The main complaints were that the database should be connected to the input program so

that data could be transferred automatically, and that it needs to contain much more data.

The first will be done with thermophysical data in the new input program (FAST_IN) to

be a part of HAZARD I vl.O. All database files will be directly integrated in HAZARD

II, without the use of any proprietary software. Several positive comments were received

on the advantages of having an organized fire database in which the user’s data could be

incorporated as well. One person (not a participant in the beta test) requested a copy of

FIREDATA only as a way to store his own data.

6.3

MLTFUEL

The few comments received on this module centered on the suggestion that it incorporate

spread from item to item and the establishment of the time line now done manually. It should then be integrated into the input program. Both will be done for HAZARD I vl.O.

6.4

FTNPUT

A few "bugs" were identified and have already been fixed. Other comments received

addressed the scrolling display and help text which sometimes caused confusion. The total

replacement of FINPUT by FAST_IN will resolve all of these comments and then some.

File lists will be supported, and even the "quick analysis" mode should help address

complaints on the slowness of FAST.

Positive comments were made concerning the error checking and units conversion features.

These make the program much more "friendly", and avoid simple errors. The capability to

change units sometimes results in the user forgetting what the current units are. FAST_IN

will always display the current units, avoiding this trap.

One comment was received on the prohibition of overwriting an existing file. This was

done intentionally, to prevent an inadvertent loss of a file; but this user would like to

Page 12
replace a corrected file without going back to DOS for a delete and rename. We will go

to a "replace existing file (Y/N) ?" message in FAST_IN.

6.5 FAST

Since FAST is run in a batch mode, there is not much to say about the software interface.

Comments were limited to: "too slow", "how do I know that it is still running?", and "how

do I stop it?".

There is little that we can do about the execution speed. Frequent users can invest in

faster hardware - a PS/2 Model 80 will run FAST 25 times faster than an XT. We have

implemented a "quick analysis" mode within FAST_IN which gives lower accuracy but much

faster results for the impatient.

For those who want reassurance that FAST is running, and in fact how far it has

progressed and when it may finish, HAZARD I vl.O will support run-time graphics. This

will allow the user to select a few, key variables and watch them evolve. This will also

show when the desired results are not being obtained so that the user can choose to abort

the run.

6.6 FASTPLOT

The primary problems reported were associated with it not producing plots at all (they

loaded the wrong graphics driver in INSTALL) and difficulties in obtaining printer copies

of the plots as was discussed in section 5.3. Both problems have been corrected in

HAZARD I vl.O by the change to a universal graphics and printer driver from a commercial vendor. This single driver supports CGA, EGA, Hercules, and VGA; is not

memory-resident, and will not conflict with ANSI.SYS. Thus this one change solves a range

of problems.

Other minor comments about the gap in the curves for their label always appearing in the

most important part of the curve and the labels themselves (variable number rather than

room/layer) have been corrected in HAZARD I vl.O. The gap has been removed and

provision made for eight line patterns (that correspond with the eight colors in EGA) and

a legend window that identifies room and layer.

Page 13
Other improvements such as:

• autoscaling,

• X and Y offset by a constant,

• clearer variable naming,

• more appropriate units for species,

• the ability to input and display variables from multiple files, and

• provisions to allow plotting of tenability data

have been implemented to make the program much more useful Requests to provide the

ability to create ASCII files in a format compatible with other graphics programs (LOTUS,

CHART, etc.) and support for HP plotters have also been implemented, the latter through

an HPGL file format option.

It is interesting to note that several users commented on the low resolution of the plots in

CGA mode. Of course, this is totally hardware-controlled and not within our ability to

change; although the ASCII interface to LOTUS or CHART allows output to a pen-plotter.

6.7 DETACT

From the general lack of comment on this program we surmise that few people used it. In HAZARD II the plan is to integrate it within FAST.

6.8 Exnr

Comments included:

• too many questions in the beginning,

• use of numbers for both the nodes and for the people is confusing,

• the process to enter your own building is tedious,

• node location is arbitrary and will result in variations in escape times,

• manual entry of smoke data should be replaced by transfer from a file, and

• some behaviors did not make sense (e.g., if a family has more young children

than adults, some get left in the house even though there is enough time

to rescue).

Page 14
To address these points, a number of changes have been made:

• Initial questions have been reduced,

• to avoid confusion letters are now used to represent people and numbers

for nodes, and

• many inconsistent behaviors have been identified and corrected.

When EXITT is compiled, we will add the ability to transfer the required smoke data

directly from the FAST dump file. Unfortunately, it is not feasible to make improvements

on the method of entering a new building and establishing the node map, although these

data will be read from a file. Full graphic input will have to await the Computer Aided

Design (CAD) type interface planned for BAZARD II or III.

6.9 TENAB

There were no negative comments about TENAB. On the positive side, comments

included: straightforward, easy to use, and good data presentation. For HAZARD I vl.O

we will include a second calculation of Fractional Effective Dose (FED) using a refined

model for incapacitation. We will also update the original to include reduced oxygen

effects.

To enhance the presentation of output, TENAB will be modified to produce an output file

compatible with FASTPLOT. This will allow plots to be made of the tenability parameters

against time, for each occupant.

Page 15
7. User Needs

Based on the comments received and numerous discussions with participants, an opinion

was formed of the needs and desires of the user community with respect to fire hazard

analysis software in general These identified issues will be used to help shape future

versions of HAZARD and other software products under developmento

7,1 Breadth (flexibility)

Clearly the greatest desire among users is for the ability to address a broad range of

problems and situations. The most frequently heard comment started out "It’s really great,

but I wish it could Examples from the user wish list include:

• spreading the fire from item to item,

• spreading the fire from room to room,

• burn through of a partition,

• wall burning,

• transport and evaluation of the toxic effect of species related to the product

in which they are interested e.g., HCi, HBr, TDI, etc,

• enhanced burning due to radiation feedback to the fuel surface,

• ventilation controlled burning,

• forced convection,

• additional occupancies (especially industrial),

• high-rise buildings,

• suppression by sprinkler systems,

• vertical openings,

• long corridors,

• more data in the database on ... (fill in your product).

It is not that every user wishes the method to address all of these items, but rather that

each user has specific applications in which one of these are involved. It is also important

to understand that the users are asking for the models to address these phenomena, without

giving specific thought to the technical level at which they should be included. This is a factor which was explored further with the users and is discussed in section 7.4.

Page 16
7.2 Ease of Use

The second most important factor for users is the ease with which the system can be used

for their analyses. This refers to more than just the "user friendliness" of the software. It includes the usefulness of the documentation in answering their questions (and how easy

it is to find something), consistency in the commands employed for a given purpose,

compatibility with other software systems (e.g., CAD or spreadsheets), and the ability to

display results in a presentation quality.

Some of these issues can be a two-edged sword. For example, documentation which is thorough enough for a fire researcher can be too voluminous for a user doing public fire

safety education. Likewise, building in flexibility to interface with other software can result

in too many choices among which the users must decide. Finally, there is a competition

for resources between improvements in the science-related aspects and those which are purely computer programming. Each of these competing forces must be considered in the

development of a package for use on a broad range of problems.

7.3 Execution Time

Many users expressed frustration in the amount of time needed to do an analysis. Part of

this frustration comes from the large quantity of information needed to "feed" all of the

programs. But these data needs are at least partly related to the large number of

phenomena included, and will increase as the wish list in section 7.1 is implemented. One

potential answer is the development of versions of HAZARD which are customized for

specific applications. The first such package is currently under development in cooperation

with the Consumer Product Safety Commission (CPSC).

The other part of the frustration comes from the long execution time required by FAST.

The most efficient approach to shortening this time is for the user to buy a faster computer

- the range in speed for current PC hardware is a factor of 25! We also feel that a part

of the frustration is that the user must stare at a blinking cursor for many hours while

FAST is running, wondering if it has crashed, or what may be happening. The run-time

graphics with HAZARD I vl.O will at least relieve this.

Page 17
7.4 Technical Depth

One of the key findings of this evaluation is that the majority of users are relatively

unconcerned with the technical depth of the models and calculational procedures included

in HAZARD I. This is borne out by discussions with users and the fact that only two

respondents raised issues related to the underlying science.

There are two explanations for this attitude among the user group. First, for most

applications any answer that does not defy reason is an improvement to the status quo

(expert judgement); and second, most trust that a scientific method produced by CFR will

be at least technically credible, if not the best that can be done. The latter was probably

more of a factor among the beta testers than might be expected from the universe of

potential users since the participants are mostly persons with whom we have had prior

professional interactions. But in general, users trying to answer a question would rather

have an educated guess (so labeled) than to be told that it is beyond the scope of the

method.

Another possible factor, although not expressed as such, is the tendency of people to accept

computer output without question ("Garbage in; gospel out!"). This syndrome could have

been exacerbated by the fact that most users did not find the time to exercise the software

to the extent originally planned, so they probably did not have the time to examine the

underlying science in detail.

Page 18
8. Applications

In section 4, a list of applications proposed in, and excerpted from, letters of acceptance

sent by beta test participants is presented. Unfortunately, as of the date of this writing,

only one user application has been documented in writing to us. This is a reconstruction

of a full-scale fire experiment conducted in 1982 in which an upholstered chair was burned

in the living room of a residential house. The only available details of the building

dimensions and construction, and of the materials used in the chair and other combustibles

in the room were those included in the published report.

For the first model run, items (drapes and a chair) which appeared to be similar to those

described in the experimental report were selected from FIREDATA and used to model

the fire. The ignition of the drapes by the burning chair was modeled using the procedure

provided in the HAZARD I documentation. Examination of the results as shown in figure

2 revealed that the agreement was of the same order of magnitude, but appeared to be

shifted in time.

WRONZ HOUSE FIRE

HAZARD ANALYSIS

WP0N25J»tTto CAW40C(2t/VM)

Figure 2
A comparison (figure 3) was then made between the rate of heat release of the items

obtained from the HAZARD I database, and that estimated from the observations published in the test report. This showed that the burning rate of the fuel in the experiment

was faster than for the database items selected.

WRONZ HOUSE FIRE

HAZARD ANALYSIS

»»OW2l i .PtT2 CA WAOC(JV3/a8)

Figure 3

After an adjustment in the assumed burning rate based on the comparison in figure 3, a second FAST run was made. This resulted in an excellent match between the measured

temperature near the ceiling and the upper layer temperature predicted by the model (as

shown in figure 4), up to the time at which a window broke in the experiment. (Modeling

of a vent opening during the run is a feature which could not be addressed in the beta test

version, but will be included in the HAZARD I vl.O release).

Page 20
WRONZ HOUSE FIRE

HAZARD ANALYSIS

vKONZn ^lti CANACC (21/3/S8)

Figure 4

It should be stated that, although the results obtained by this user were excellent, such

results may not be reflective of an inherent accuracy in the models or software but may be

the result of errors which cancelled in this particular analysis. The quantitative accuracy

of these methods are the subject of ongoing study, which will be published as the work

progresses.

Page 21
9. Conclusions

The overall reaction of the program participants was highly positive. Most participants

commented that the software would provide a valuable tool in their area(s) of interest.

Most respondents made suggestions which they felt would improve the ability of the

software to meet their needs; with most of these in the general category of user interface

issues (rather than science issues).

There was a general consensus that the data base was very useful. (One person outside

of the beta test program requested a copy of the data base by itself as a means of storing

and retrieving his own data.) Most, however, expressed the need for a significant expansion

of the list of burning items. One manufacturer’s association is contracting with a testing

laboratory to conduct Furniture Calorimeter tests on a range of products representative of

those currently being marketed by their membership, to support their own use of HAZARD

I.

Several of the respondents expressed a strong desire for the pictorial graphics capabilities

which we have demonstrated in the past, but are not a part of the current implementation.

This was particularly the case in applications where results are presented to persons outside

of the fire community; such as for marketing presentations or for public fire safety

education. We hope to be able to deliver this capability with HAZARD II.

Perhaps the biggest success of the evaluation program was the fact that we were able to

address almost all of the participant comments with changes to the software. Thus, the

time and effort invested by the users will be rewarded in the release version, along with

some technical improvements that will greatly enhance the capabilities of the software

package. These include the influence of reduced oxygen levels on the combustion and the

ability to open and/or close interior or exterior openings at will.

The greatest disappointment of the program was the failure of the respondents to conduct

and/or document the applications to which they committed. While their intentions were

good, the press of business prevented most from completing this important task. To fill this

remaining gap, we intend to conduct and publish a series of applications (as our funding

permits) using the release version, and to encourage users of the release version to share

their results (perhaps by holding a HAZARD I user’s conference).

In summary, the beta test program has provided us with considerable insight into the needs

and desires of the fire community for the software implementation of predictive tools. The

lessons learned in this program will have a long lasting impact on the work of CFR and

other organizations involved in the development of these tools, and on the degree to which

they become integrated into the application of fire protection principles in the future.

Page 22
APPENDIX A

SELECTED EXCERPTS FROM PARTICIPANT RESPONSES

Each indented paragraph is a direct quote from a participant response letter. The statement

following is a comment on the change implemented in the release version of HAZARD I to respond to that point.

T believe that the concept of bringing together past and current research

done by your organization and others in this very useful way is a good one."

We agree! A hazard assessment must integrate all facets of fire research into a single

analytical tool. HAZARD I represents the first time that the complete package has been

so assembled.

"Your recommendations suggest that the next release of HA21ARD I will

require DOS 3.0 or higher. I would suggest that you make this clear in the

next release, as many users not only have no working knowledge of DOS

or their PC, but will not know what version of DOS they are running."

Several "bugs" in the beta test were traced to problems in DOS 2, and which were

corrected by upgrading to (or never showed up with) DOS 3. Since DOS is less than $100,

this should not be a hardship.

"HAZARD I appears to be an extremely good tool for assisting us in

evaluating whether or not our analysis of a fire occurrence is correct,

feasible and supportable."

This points out that in fire reconstruction, the theories will likely be posited by the

investigators, and then tested using HAZARD I. It is a way of obtaining a corroboration

without having to hire a second expert.

Page 23
"We are delighted with HAZARD I and feel that a hearty pat on the back

and thanks should go to all involved in its conception and writing. You

have created a landmark in the annals of Gre safety!"

Thank you!

"It should always be clear to the user when a decimal point is required. Use

"Don't forget the decimal point" in appropriate places."

This refers to the fact that computers demand a decimal point for the entry of a "real

number", even if it is a whole number. The input program (FINPUT) and the new user

interface program (FAST_IN) take care of this for you. Thus, you only use a decimal

point when you enter a number with a decimal portion.

"Could not find the default units for thermal K ..."

The new user interface (FAST_IN) displays the current units for all entries.

"The installation was completed without any problems and all communication

with the programs was as expected."

This was the general observation. Where there were installation problems, they were

normally associated with answering one of the INSTALL questions wrong. The release

version has eliminated all of the INSTALL questions.

"The biggest problem I have found thus far is in finding a good source of

data for the type of objects that I would like to simulate..."

This was a common problem. The need for appropriate data will be the largest impediment to the rapid adoption of these predictive methods. We are working on ways to

address these needs on a national or international basis. But they all involve the

willingness of the material and product producers to collect and share data on their

products.

Page 24
"More data on product performance is needed. Possibly additional data disks

made available as data becomes available. NBS could act as clearing house

for data that is made available by outside labs, companies, etc."

Several distribution methods are being explored including a subscription update service and

a dial-up master data base from which data can be downloaded.

"Our ability to utilize it in our own product evaluation was quite limited as we did not have data available from your data base on products close to our

own designs."

For applications to specific products, it is clear that the product producer will need to

supply the data. It is already possible to have such data taken by testing laboratories such

as Underwriters Laboratories or Factory Mutual.

"The test example for the FAST program was executed without problems,

but the run time felt rather long compared with the time of simulation."

The required execution time is typically beyond our control. As discussed in the report, for a given problem the run time varies by a factor of 25 from the older PC to the 386/25

machines. Within a year, the high end PC will run at 33 MHz, and a year after that at 50

MHz. As long as the computational complexity stays relatively constant, the run times will

continue to decrease - as long as the user is willing to upgrade to new machines. Here,

the cost of the equipment will be weighed against the benefits of the method.

"Add a message to the software that indicates to the user that the software

is running properly. I did not know if the software was running or hung

up."

This has been addressed by the implementation of run-time graphics.

"In terms of experience (for fire reconstruction), I have had moderate

success with the program. One run was exceedingly good. There have been

several intermediate runs that were of mixed success. The last one was a

complete bomb. I am currently reviewing the input data to make sure I have not entered erroneous data. It may also be that the fire scene is beyond the limits of the program."

Page 25
Users will always need to ask themselves if the model results make sense. We are examining automatic ways to provide uncertainty estimates as the case is entered.

"It would be desireable to be able to input the fire description using rate of

heat release information."

This change has been implemented in the release version.

"When water is defined as one of the species to be tracked by FASX the

water concentrations computed for the various compartments over time

include only the water produced fi’om the fire chemistry, "^ically, the

ambient humidity is of the same order of magnitude and should therefore

be included if the true water concentration is desired."

This has also been implemented. The relative (ambient) humidity is set by the user, and

is converted to a mass (of water vapor). Water produced by the combustion is added and

the resulting total mass concentration is tracked.

"We have noticed one theoretical disadvantage - the program can not

simulate ventilation controlled fires."

This feature has also been added with the incorporation of a vitiated combustion algorithm.

"Output data expressing heat flux at floor level would be useful information

in judging the effects of fire on a compartment"

This too has been added. The incident heat flux at the floor is available in FASTPLOT

as the variable "ON TARGET’.

"The ability to open a vent during the simulation period either at a specified

time or a predicted time (or both?) [is suggested]."

Once again, this feature has been implemented in the release version. With the parameter

CVENT, interior or exterior vents can be opened or closed at any time (or multiple times)

during the simulation.

Page 26
"Less critical but still important problem [is] the assumption of a constant

heat of combustion (our data often contradict this assumption)."

With the addition of the ability to enter rate of heat release, the heat of combustion is no

longer necessarily a constant. By specifying the heat release rate and the mass loss rate,

heat of combustion can be varied at will.

"A comparative plotting capability, Le., plotting the same variable found in

different [files is suggested]."

The ability to make comparative plots from different files has been implemented in

FASTPLOT Moreover, it is simplified in that once you select the desired variables from

the first file, you simply select a new file and use the command "AGAIN" to duplicate the

variable list from the new file(s).

"It is very time consuming to get hard copies of the graphs firom

FASTPLOT Using PIZZAZ, the program hangs up after each graph, thus

requiring a reboot and another dump file search for the next variable(s) to

be plotted."

This problem has been solved with entirely new graphics and printer drivers that are universal (works with all displays from CGA to VGA automatically) and are not resident, so do not conflict with ANSLSYS. By also adding dash/dot line patterns to FASTPLOT,

color is not required to differentiate the curves. Thus, you can see it in color on the

monitor and then send it to the printer in monochrome.

"Would like results output to ASCII files."

We have added an option to FASTPLOT to save variables in ASCII files as columns of

data, which should be compatible with most commercial spreadsheet and graphics software.

Page 27
"EXl l'l": The inclusion of an allegedly deterministic model of human

behavior implies the same confidence level in its algorithm as say the model

in FAST This is, of course, not the case. The concept of such a behavioral

model is fundamentally flawed. Its inclusion in HAZARD I cheapens the

entire package. In addition, the implementation is poor and the sources for

the decision algorithm undefined. I strongly recommend that you reconsider

releasing EXl'lT as part of HAZARD L"

One person was quite opposed to EXITT on the basis that behavior is a largely stoichastic

process. However, we feel that occupant hazard cannot be assessed without a measure of

the time needed. In family settings (and certain other situations such as health care), there

is a strong altruistic component to evacuation which can actually dominate this process.

Thus, until something better comes along, EXITT will remain a part of the package.

"EXITT is also an excellent program with excellent graphics. I quite enjoy

its game-like aura. Providing that the assumptions are sound, the program

should be quite good for exit problems. Somehow, I think the assumptions

need to be refined. One example is where 4 residents are in the ranch

house. The father got out of the house first, leaving the mother to take

care of the infant and the 5 year-old to escape by himself. One must

wonder if the father had nothing to do for two minutes, or the program was

written by a biased, prejudiced, and sexist programmer."

Yet another user recognizes the need for EXITT, but questions some of the behavioral

rules. As the program develops, we hope that these inconsistencies will be corrected.

"The manuals are of a very generous length. However, this does mean that

a lot of effort is required to become conversant with the package. I think

many people would not have the patience to read the manuals in much

detail A short user-guide would be useful"

A new format will be used for the release version documentation. The software use

information will be separate from the supporting technical reports. The latter will then

serve as a reference base to be consulted only when needed.

Page 28
NDd>ll4A IRtV. 4-ovi U.S. DEPT. OF COMM.

BIBLIOGRAPHIC DATA

SHEET (See instructions)

1. PUBLICATION OR

Rj?.pruaT--wn

NISTIR-88/3878

2. Performing Organ. Report No. 3. Publication Date

November 1988

4. TITLE and subtitle

HAZARD I - Results of a User Evaluation

of the Prototype Software

5. AUTHOR(S)

R.W. Bukowski , P.E.

6. PERFORMING ORGANIZATION (If joint or other than NBS, see in struction s) National Institute of Standards and

Technology

U.S. Department of Commerce

C-aithersburg , MD 20899

7. Contract/Grant No.

8. Type of Report & Period Covered

9. SPONSORING ORGANIZATION NAME AND COMPLETE ADDRESS (Street, City. State. ZIP)

10.

supplementary notes

Document describes a computer program; SF-185, FlPS Software Summary, is attached.

11.

ABSTRACT (A 200-word or less factual summary of most si gnificant information. If document includes a significant

bi bliography or literature survey, mention it here)

After five years of development, the prototype of a personal computer based,

fire hazard analysis method was distributed to 93 volunteers representing all

aspects of the fire community. These persons agreed to evaluate the software

and documentation, and attempt to apply it to a problem of their own choosing.

Uritten comments were to be returned, which would be used to establish

priorities for future changes, and where possible, be incorporated into the

general release version of the product.

Uritten responses were received from 47 participants, most of which dealt with

suggestions for improvements to the user interface (rather than any technical

shortcomings). Based on the responses received, it has been concluded that:

the software will be of substantial, broad benefit; with the identified

improvements, the user interface is comparable to commercial software in ease

of use; the data base is particularly useful, but needs to contain many more

entries; and priority enhancements need to be made in the areas of combustion

modeling and pictorial graphics.

12.

KEY WORDS fS/x to twelve entries; alph abeti cal order; capital! ze only proper names; and separate key words by semi colon s) computer models; computer programs; evaluation; fire models; hazard models

13.

availability

u n 1 united

' For Official Distribution. Do Not Release to NTIS

Order From Suoerintendent of Documents, U.S. Government Printing Office, Washington, D.C.

20402.

Order From National Technical Information Service (NTIS), Springfield, VA. 22161

14. NO. OF

PRINTED PAGES

32

15. Price

$11.95

USCOMM-DC 6043-P80
' iT —I- — — -.^-..j—jj 1 . _ utj- T i Ii.-i' -

, 1 -I i.i Ti- r- Ii-j-"- i .fi i M««» i ii nir- | , -1H.. I I ‘ - "

| ^J^—

~.| | ..—««.^ ti 7

:‘“"

--.i.lt.Hvr; 1m ..'3 ms}^

>v-> fa, I?,5^' r incite ^vr?5f^;t%

,

' ' • Ti • Vj

*ViS- -; -V/®^ ?;:*'> f

'! ' V ?* '® J, >' •' .',4

n.’ ,,t (, •., V, 1

.V. -tr- ,F.

'

r '> ^<1

- 1

' S' ? £

4 i-4 SiA

;'-'3 j 1 , :-i- * { vmm .

'. ?!v,,

t-^ • -

If

. 'i.

r.

. 4 't T;,^

f‘*Mi *}'.>'t«S‘>^ *

*

• V

-r^- -«|4

V, ^S/

««K,- -,-s ' .”-* -fr *-s -^- ‘V’si"

''' —

,;j-- 't ,.-,,tt,. r.^'V •^ V-

'.%, '; 2>x ;',-^ •

=; ^5 .'. CA

••’-

S U p',, - ,

l.-. ,i..U ’ vC'. » _ J-

,

- f

' '^ - :'v.

-, -j * r- .- be* ; ' ••• ••-=-' ’.

,. , V. . ... -; I - ^ : „• f'-r

v>5

7 . •.. . 4 ‘i-':; U‘r ms '^iC

- '

' , . • “ ^ V\ y

.

V V f

, -.. v: 7 , ; M,s

• .•- q

-: I <a\/-^-iv- Xj^ -. .;. -- •^’/ 1 4 ;

. ^ . ,i . s Xp%

V S:n ' -'7 '' : ii'ir -‘- ' '

:*v.

7fV - ii'

'I, . f

'j n V:'

» -*< *- t*

ij- '

' -;..

. •-

.,,,f.^ xru:L: t->'

. .=

' p: i '£ 4 a <* -- 2 - o *: . £ i&2

'

i'. '.^ ?!':. V ' '- ...'.^V^^

z *^^ '* ^ S4r\iji-'-v

r

"• 'fT :.h.

$. ij' •“ .?3 "t'l ©•'

t-.r..\-.c .. ) : 'i'7:* •

rj...

'.V.- f.yvi/’®

",. ^s£-"

4: *-j-';^;' -f ^sr' vl. ..

t • ^ 'h'j CiHi-nC'.. ,{r ,r . “•

*•

y;.SrV»'. *;

rf’ ,''‘.iii\ i-V

^ tfh

W^'""

< !

•. I

' 747" I „ ,ut..o.-,j45oi~, V «,. .1 WfvJ*-'.iW.'»»»»4»««i» ,:iv'f®*

" i;.:.’>c':. »:>iV ‘ .0'3»:j.!liS'/‘.-;*«a.ag0-iT5

j^«.“ - .. . . .

; I

, • '

jiL. .’'i «i —ii ~Twi*i~ •

i "r 'T~^

- 4 .C-v^ -J

. I *!«•'

,^"2

^ 2 iTt®4 c4 ;<x,v» r/C

.MA V^t spki74v'^ jr,fi<>^.''f«»v4D .*?nai^vjoO

'

, .

• ^. ., - ,..si’. -I;

MIS -.A^/- rsi;rwm9o^rt,|

'-'A,; ‘A''

% - *.4 f) t: c *»^