. Department

bommeree

lonal Bureau

tandards

Computer Science

and Technology

NBS Special Publication 500-73

Computer Model

Documentation

Guide
NATIONAL BUREAU OF STANDARDS

The National Bureau of Standards' was established by an act of Congress on March 3, 1901. The Bureau's overall goal is to strengthen and advance the Nation's science and technology

and facilitate their effective application for public benefit. To this end, the Bureau conducts

research and provides: (1) a basis for the Nation's physical measurement system, (2) scientific and technological services for industry and government, (3) a technical basis for equity in trade, and (4) technical services to promote public safety. The Bureau's technical work is per- formed by the National Measurement Laboratory, the National Engineering Laboratory, and

the Institute for Computer Sciences and Technology.

THE NATIONAL MEASUREMENT LABORATORY provides the national system of

physical and chemical and materials measurement; coordinates the system with measurement

systems of other nations and furnishes essential services leading to accurate and uniform

physical and chemical measurement throughout the Nation's scientific community, industry,

and commerce; conducts materials research leading to improved methods of measurement,

standards, and data on the properties of materials needed by industry, commerce, educational

institutions, and Government; provides advisory and research services to other Government

agencies; develops, produces, and distributes Standard Reference Materials; and provides

calibration services. The Laboratory consists of the following centers:

Absolute Physical Quantities- — Radiation Research — Thermodynamics and

Molecular Science — Analytical Chemistry — Materials Science. THE NATIONAL ENGINEERING LABORATORY provides technology and technical ser- vices to the public and private sectors to address national needs and to solve national problems; conducts research in engineering and applied science in support of these efforts; builds and maintains competence in the necessary disciplines required to carry out this research and technical service; develops engineering data and measurement capabilities;

provides engineering measurement traceability services; develops test methods and proposes

engineering standards and code changes; develops and proposes new engineering practices;

and develops and improves mechanisms to transfer results of its research to the ultimate user. The Laboratory consists of the following centers:

Applied Mathematics — Electronics and Electrical Engineering^ — Mechanical

Engineering and Process Technology- — Building Technology — Fire Research —

Consumer Product Technology — Field Methods.

THE INSTITUTE FOR COMPUTER SCIENCES AND TECHNOLOGY conducts

research and provides scientific and technical services to aid Federal agencies in the selection, acquisition, application, and use of computer technology to improve effectiveness and

economy in Government operations in accordance with Public Law 89-306 (40 U.S.C. 759), relevant Executive Orders, and other directives; carries out this mission by managing the Federal Information Processing Standards Program, developing Federal ADP standards

guidelines, and managing Federal participation in ADP voluntary standardization activities; provides scientific and technological advisory services and assistance to Federal agencies; and

provides the technical foundation for computer-related policies of the Federal Government.

The institute consists of the following centers:

Programming Science and Technology — Computer Systems Engineering.

'Headquarters and Laboratories at Gaithersburg, M D, unless otherwise noted; mailing address Washington, DC 20234. Some divisions within the center are located at Boulder, CO 80303.
Computer Science

juN 1 s 1981 and Technology

not ft" - NBS Special Publication 500-73

Computer Model

Documentation

Guide

Prepared by:

Federal Computer Performance Evaluation

and Simulation Center

Washington, DC 20330

for the

Center for Programming Science and Technology

Institute for Computer Sciences and Technology

National Bureau of Standards

Washington, DC 20234

U.S. DEPARTMENT OF COMMERCE

Philip IVI. Klutznicit, Secretary

Jordan J. Baruch, Assistant Secretary for Productivity,

Technology and Innovation

National Bureau of Standards

Ernest Ambler, Director

Issued January 1981
Reports on Computer Science and Technology

The National Bureau of Standards has a special responsibility within the Federal

Government for computer science and technology activities. The programs of the

NBS Institute for Computer Sciences and Technology are designed to provide ADP

standards, guidelines, and technical advisory services to improve the effectiveness

of computer utilization in the Federal sector, and to perform appropriate research and

development efforts as foundation for such activities and programs. This publication

series will report these NBS efforts to the Federal computer community as well as to interested specialists in the academic and private sectors. Those wishing to receive

notices of publications in this series should complete and return the form at the end of this publication.

National Bureau of Standards Special Publication 500-73

Nat. Bur. Stand, (U.S.), Spec. Publ, 500-73, 56 pages (Jan. 1981)

CODEN: XNBSAV

Library of Congress Catalog Card Number: 80-600190

U.S. GOVERNMENT PRINTING OFFICE

WASHINGTON: 1980

For sale by the Superintendent of Documents, U.S. Government Printing Office, Washington, D.C. 20402

Price $3.75

(Add 25 percent for other than U.S. mailing)
PREFACE

This document was prepared for the Institute for Computer Sciences and Technology, National Bureau of Standards,

by the Federal Computer Performance Evaluation and Simulation Center (FEDSIM). It is based on a similar document

originally prepared for use by the U.S. Air Force in support

of military analyses. That document has been rearranged,

examples have been changed, and the document has been made

more generally applicable so that it may be used throughout

the Federal simulation community. Recommendations for improvements of these guidelines are solicited. It is intended that after thorough reviews and trial use this document

will be reissued as a Federal Guideline in the series of

Federal Information Processing Standards (FIPS). All comments should be directed to:

Institute for Computer Sciences and Technology

Programming Science Division

Washington, D. C. 20234

r

iii
TABLE OF CONTENTS ! Page

PREFACE i i i TABLE OF CONTENTS TV

TABLES V

FIGURES Vi

ABSTRACT 1 Section

I. INTRODUCTION 1

II. GUIDELINES FOR PREPARING A MANAGEMENT SUMMARY MANUAL.. 4

1. Introduction 4

2. Model Description 4

3. Model Development and Experimentation 7

4. Current and Additional Applications 8

APPENDICES 8 | III. GUIDELINES FOR PREPARING A USER'S MANUAL 9

1. Introduction 9

'

2. Description of the Model 9

i 3. Model Input Data 17

r 4. Model Output Data 20

5 5. Run Preparation Instructions 21 |

6. Sample Model Run 21

7. Trouble-Shooting Guide 22

APPENDICES 22

IV. GUIDELINES FOR PREPARING A PROGRAMMER'S MANUAL 23

1. Introduction 23 I

-.1

2. Model Specifications 23

iv
3. Model Description 25

4. Description of Routines 31

5. Data Base Description 32

6. Source Listing 34

7. Error Messages 34

APPENDICES 34

V. GUIDELINES FOR PREPARING AN ANALYST'S MANUAL 35

1. Introduction 3 5

2. Functional Description of the Model 35

3. Model Input and Output Data 40

4. Model Verification and Validation 42

APPENDICES 43

VI. MODEL SUMMARY 45

VII. SUMMARY 4 7 BIBLIOGRAPHY 48

TABLES

TABLE

IV-1 Cross-Ref erenced Data Structure List Sample 28

IV-2 Sample List of Routines by Overlay Segment 30

IV-3 Cross-Ref erenced Routine List Example 33

V
FIGURES

II- l Recommended Table of Contents for a Management

Summary Manual 5

III-l Recommended Table of Contents for a User's Manual 10

III-2 Physical System Highlights Example 12

III-3 Input/Output Schematic 13

III-4 Model Schematic Sample #1 15

III-5 Model Schematic Sample #2 16

III-5 Special Procedures Example 19

IV-1 Recommended Table of Contents for a Programmer's

Manual 24

IV-2 Example of a Model Overview Block Diagram 26

IV-3 Sample Overlay Structure Diagram 29

V-1 Recommended Table of Contents for an Analyst's

Manual 36

V-2 Modeled System Highlights Example 37

V-3 Input/Output Schematic 39

V-4 Model Functional Flow Example 41

V-5 Comparison of Actual and Simulated Results 44

vi
COMPUTER MODEL DOCUMENTATION GUIDE

This document provides guidelines for preparing documentation for computer models. Recommended structures for four types of manuals providing

model information for four different classes of

audiences (managers, users, analysts, and programmers) is presented. This document specifies the

content of sections and subsections for each type

of manual. Manuals prepared using these guidelines will enable persons interested in a model to

understand the capabilities and limitations of

that model.

Key words: documentation; manuals; models;

8 imulation

.

I. INTRODUCTION

This draft document provides guidelines for preparing

documentation for computer models, as well as complete submodels delivered separately. The primary goal of model documentation is to communicate effectively the details of

model design and operation to persons with varying interests

in a model. Since a model's developers are frequently not

the model's ultimate users, complete, concise documentation

is essential for effective model use. Documentation should

inform analysts familiar with the phenomena being modeled,

or the modeling techniques employed, of the essential

features and assumptions of a new model. Throughout its

life cycle, a model may be used and modified by various people, making accurate and current documentation of the underlying computer program essential for proper, correct use and

maintenance of the model. Ultimately, model results may be

used in a decision-making environment by individuals who are

unfamiliar with the details of modeling and the associated

benefits, risks, and costs. In such situations, model documentation should describe, in non- technical terms, the environment in which a model can be useful, limitations on its

use, and the manpower, time, and dollar costs required by

its use. These guidelines recommend structures and some

conventions for preparing model documentation in the form of

manuals for users, analysts, programmers, and managers.

Each type of manual should provide clear, concise documentation that is directed toward an audience with a particular

interest in a model.

-1-
These guidelines devote a section to each type of manual, i.e.. The Manager's Manual, The User's Manual, The

Programmer's Manual, The Analyst's Manual. Each section begins with a table of contents that lists recommended topics

of interest to users of that manual. Items may be added to

or deleted from this table of contents, however, according

to individual requirements. The discussion for each manual

enlarges upon the items required in each of the sections and

subsections, as recommended in the table of contents for

that manual. Terms used in each manual should be those

directed toward that manual's audience.

These guidelines are for models that are used chiefly

in a decision-making environment. Thus, the main goal of a Manager's Manual is to assist managers to make decisions.

To accomplish this, the Manual must describe the model and

its application to managers (including the management that

sponsored the model) who may be interested in using a developed capability. The Manual should provide managers

with sufficient information to permit them to accurately assess model input requirements (including time, money, and

other resources), available outputs, and the accuracy and

precision of the results. Managers can use this Manual in

justifying the employment of the model and in evaluating

subsequent results.

A user is assumed to be interested mainly in deriving

results from a model for specific applications. The guidelines recommend that the User's Manual be organized into a section for the user and a section for the data technicians

who will set-up and run the model. To use the model intelligently, a user must be aware of its logical structure, the

general simulation approach, and any assumptions and limitations affecting the model's applicability. A user need not

be interested in details of programming or analysis beyond

the preparation of input data and the interpretation of

model results.

Programmers are interested primarily in maintaining and

modifying a model. A programmer must correct any errors

discovered during model usage that are not attributable to

user-entered data. Programmers, especially those required

to convert a model to another computer system, need to

understand features of a model that are installation unique.

Thus, the Programmer's Manual must provide all the details

necessary to understand the operation of a model: to debug

it, to maintain and modify it, and to convert the model to

other computer systems.

-2-
These guidelines assume an analyst to be Interested

primarily In the analytical techniques and algorithms used

in a model. An analyst Is concerned with the equations used

in a model and the methods used for model verification and

validation. An analyst does not need to know user details

such as input and output formats, or programming details involving language syntax.

Decisions about which of these manuals are actually required, whether or not they should be prepared in separate

volumes, etc., should be made on a case-by-case basis*

Also, a plan should be developed for documentation updates

and maintenance, so that these manuals remain current. Such

issues as these should be dealt with early during the model

planning and development phases, so that documentation requirements actually become part of the development plan,

rather than an afterthought. Further, applicable documentation produced using programming conventions should be used

in conjunction with these guidelines.

Other guidelines prepared specifically to support computer software documentation are available which may be used

in conjunction with this guideline. These documents are:

FIPS PUB 30, Software Summary for Describing Computer Programs and Automated Data Systems. It is used to announce

computer programs which are transferable, and have broad applicability. A standard software summary form is defined

(SF-185), which permits description of the program for identification, reference and dissemination. This form is used

by the General Services Administration for registry of programs and for publication of program abstracts in the

Federal Software Exchange Catalog.

FIPS PUB 38, Guidelines for Documentation of Computer Programs and Automated Data Systems. It provides guidance to

documentation content for the development phase, including

requirements documentation, system specifications, user,

operations and maintenance manuals, and test documentation.

FIPS PUB 64, Guidelines for Documentation of Computer Programs and Automated Data Systems for the Initiation Phase.

This document provides guidance for project requests, feasibility studies, and cost benefit analyses.

-3-
II. GUIDELINES FOR PREPARING A MANAGER'S MANUAL

This section provides a recommended structure for the

Manager's Manual and describes the contents of each section

and subsection. A manual prepared using these guidelines

will provide descriptions of model capabilities and requirements such that model strengths and limitations will be communicated to decision makers and potential users. The

Manager's Manual will provide managers with sufficient Information to permit them to accurately assess model Input

requirements (Including time, money, and other resources),

available outputs, and the accuracy and precision of the

results. Managers can use the Manager's Manual In justifying the employment of the model and In evaluating subsequent

results. Figure II-l Is a recommended table of contents for

preparing a Manager's Manual. The sections and subsections

Included In that figure list suggested topics that are of

Interest to managers. Items may be added to or deleted from

this table of contents, however, according to Individual requirements .

1. Introduction

The Introduction should Identify the sponsoring organization, provide the background of the project, state the

purpose of the model, and present an overview of the remaining sections in the manual. A common Introduction used for

other manuals prepared for a model may be used only if that

introduction is void of specialized terms. The specific

purpose of the Manager's Manual should be Included in the

introduction in a statement of the form:

"The purpose of this manual is to communicate to

management the capabilities and limitations of (model

name )

.

2. Model Description

This section should provide a summary of model capabilities and limitations. Use high-level block diagrams to

clarify the narrative, as needed.

-4-
1. Introduction

2. Model Description

2.1 Capabilities

2.2 Input/Output Classes

2.3 Assumptions and Limitations

3. Model Development and Experimentation

3.1 Development History

3.2 Verification and Validation

3.3 Model Experiments

3.4 Costs and Resource Requirements

4. Current and Additional Applications

4.1 Current Use

4.2 Additional Applications

APPENDICES

A. Project Documentation

B. Bibliography

RECOMMENDED TABLE OF CONTENTS FOR A

MANAGEMENT SUMMARY MANUAL

FIGURE II-l

-5-
2.1 Capabilities

This subsection should briefly summarize the capabilities of the model. Include highlights of mathematical and

engineering concepts (but not equations) used as the basis

of the model. Include a statement of the model's primary

purpose. For example, "the model can be used to determine

the daily number of machines in a job shop required to process the daily orders." Provide an overview of functional

details that explains how the model accomplishes its stated

purpose. Discuss the general areas of the model's applicability, including the decision making environments. For example, describe the types of systems and situations that can

be simulated by the model (possibly with minor changes), including the number and kinds of subsystems that can be simulated. For example, if a job shop model includes order processing, machine repair, or distribution subsystems, then

their descriptions should be provided. Also include the relationship of this model to any other models (i.e., another

model may prepare input data for this model).

2.2 Input/Output Classes

Provide a short discussion on the different classes of

input data required to drive the model and of output data

generated by the model. For example, a job shop model might

require entering the number of production centers, the

number of machines per production center, the service rates

of the machines, and the routing of the jobs (orders). Examples of model output include statistics that show the

utilizations (percent busy time) of the production centers

and the job turnaround (total processing) times. Identify

any special preprocessing required for input data, as well

as all post- processing required on model results.

2.3 Assumptions and Limitations

List assumptions and limitations concerning the applicability of the model. Identify any restrictions on model

usage caused by accuracy limitations of input data and output quantities. Provide comments on levels of detail in the

model that affect the model's applicability. For example,

an analytical representation rather than a detailed simulation of a system component could affect model application.

Also describe any use of random parameters that may affect

the accuracy and use of model output.

-6-
3. Model Development and Experimentation

This section should describe significant model experiments already run and should provide details on model verification and validation procedures used. Include Information

on the model's development history resource costs and requirements, and use.

3.1 Development History

This subsection should provide pertinent details of the

history of model development. Include comments on any alternative methods to computer simulation that were considered. Provide information on any "lessons learned" during model development, such as cost overruns, model development delays, user dissatisfaction with model results, insufficient workload data to support current and future model

applications, inadequate model documentation, poorly defined

problems , e tc

.

3.2 Verification/Validation

This subsection should describe any verification and

validation procedures performed on the model. Include any

analyses performed on the sensitivity of model output data

to variations in model input data.

3.3 Model Experiments

Describe significant model experiments performed and

their results. Briefly describe the purpose of each experiment and the extent to which each experiment's goals were

realized. Discuss the management decisions affected by each

experiment. Discussion of major model experiments may be

included in separate subsections (e.g., 3.3.1, 3.3.2).

3.4 Costs and Resource Requirements

This section should provide details on the costs and

resource requirements of the model. Include the cost (in

time and money) of collecting and validating input data.

For example, long and costly data collection efforts may be

necessary. Provide comments on model maintenance and experiment costs. Discuss job turnaround times (including typical run times) and peculiar model requirements such as abnormally large core requirements or long run times. Include

comments on model portability and security requirements, as

needed

.
4, Current and Additional Applications

This section should summarize benefits already derived

from the model and recommend other applications for the

model.

A . 1 Current Use

This subsection should briefly describe how the model

has been used by management in its decision-making process.

Provide details of recommendations and conclusions derived

using the model.

4,2 Additional Applications

This subsection should provide details of any additional applications and uses of the model beyond the current

usage. Discuss in general terms any extensions and enhancements to the model which are feasible and could improve its

utility. Identify any extensions which have been scheduled

or planned

.

APPENDICES

Two appendices should be provided as required. Appendix A should reference all other project documentation (including the User's Manual, Analyst's Manual, and

Programmer's Manual), including references to the organization and person responsible for maintaining the document.

Include references to any documentation of experiments performed using the model. Appendix B should list all applicable documents (excluding project documentation previously

included in Appendix A), including cited and uncited references.

-8-
III. GUIDELINES FOR PREPARING A USER'S MANUAL

This section presents a recommended User's Manual organizational structure and discusses the contents of sections and subsections to be included therein. A User's

Manual prepared using these guidelines will enable a nonprogramming model user to understand the model's logical structure, the input data requirements, the results produced by

the model, and the use of model results. Figure III-l

presents a recommended table of contents for a User's Manual. The sections and subsections contained in the figure

cover the general needs of a user interested in a model. In

documenting a particular model, however, sections and subsections may be added to improve clarity, and some subsections may be omitted for simple models. Note that there is

a certain amount of redundance among the various sections of

a User's Manual prepared according to these guidelines.

Nevertheless, the progressively increasing level of detail

dictated by this structure is desirable to satisfy different

levels of user interest in the manual.

1. Introduction

The User's Manual introduction should contain the background of the project, the purpose of the model, and an

overview of the remaining sections in the manual. A common

introduction may be used for all the manuals prepared for a model, but the specific purpose of the User's Manual should

be included in a statement of the form:

"The purpose of this manual is to provide nonprogramming users of (model name) with the information necessary to use the model effectively."

2. Description of the Model

This section should contain a well-structured presentation of the logical details of the model. The material here

should be descriptive and include block diagrams and tables

and charts where needed; it should not give details needed

by a data technician to run the model.

2 . 1 Overview

This subsection should provide sufficient general information about the model to assist a user in determining

the applicability of the model for specific needs.

-9-
1. Introduction

2. Description of the Model

2.1 Overview

2.1.1 Model Identification

2.1.2 Physical System Highlights

2.1.3 Model Applicability

2.1.4 Input and Output

2.2 Methodology

2.2.1 Physical System Details

2.2.2 Model Logic and Data Flow

2.3 Assumptions and Limitations

2.3.1 System-Related Assumptions and Limit.ation6

2.3.2 Model Parameters

2.3.3 Output Limitations

2.3.4 Restrictions on Model Dse

3. Model Input Data

3.1 General Description

3.2 Detailed Descriptions

3.2.1 Data Set Name (First Data Set)

3.2.1.1 Number of Inputs

3.2.1.2 Other Related Data Sets

3.2.1.3 Description of Data Items

3.2.1.4 Sample Input

3.3 Data Collection and Maintenance

3.3.1 Data Sources

3.3.2 Collection Procedures

3.3.3 Updating Procedures

4. Model Output Data

4.1 General Description

4.2 Detailed Description

4.2.1 Data Set Name (First Data Set)

4.2.1.1 I>escription of Items

4.2.1.2 Interpretation

4.2.1.3 Sample Output

5. Run Preparation Instructions

5.1 Run-Stream Description

5.2 Resource Requirements

^.3 Re start /Recovery Procedures

€. Sample Model Run

7. Trouble-shooting Guide

APPENDICES

A. List of Abbreviations and Acronyms

B. Glossary

C. Bibliography

RECOMMENDED TABLE OF CONTENTS FOR A

USER'S MANUAL

FIGURE III-l

-10-
2.1.1 Model Identificatio n The Identification should contain

the name of the physical system being simulated, name of the

model (acronym and expansion), programming language(s) used

to Implement the model, computer(s) on which the model may

be run, and relationships. If any, to other models.

2.1.2 Physica l System Highlights . Include a block diagram

that shows the physical system or phenomenon being simulated. Discuss, at a macro level, the major system elements

shown In the diagram, their relation to each other, and the

flow of control, information, data, and activity between

them, as appropriate. In the case of complex models, provide in this subsection a first-level block diagram that

shows the major subsystems and their interactions, and postpone the details of each of the subsystems until Subsection

2.2.1, Figure III-2 is an example of physical system

highlights depicting the operations of a typical shop model,

2.1.3 Mode 1 Applicability . Discuss the general magnitude of

model applicability. The types of systems or situations

that can be simulated by the model (possibly with minor

changes) and the number of subsystems (e.g., production

centers and machines per production center in a job shop

simulation) that can be handled are examples of material to

be included in this subsection.

2.1.4 Input and Output . Provide a general statement of the

different kinds of input data needed to drive the model, the

output data generated by the model, and uses of model output. For example, a job shop model might require entering

the number of production centers, the number of machines per

production center, the service rates of the machines, and

the routing of the jobs (orders). Examples of model output

include statistics that show the utilizations (percent busy

time) of the production centers and the job turnaround (total processing) times. The principal model use could be to

determine the number of machines in a job shop required to

process the daily orders.

Highlight any special data collection procedures (e.g.,

run other models or computer programs, extract data from documents or listings, conduct sampling experiments) required

to produce model input data. List any unique data sources

or other organizations that might have to be contacted to

gather data. Figure III-3 is an example of an input/output

schematic

.

-11-
JOB

ARRIVALS

ROUTE JOBS

TO

PRODUCTION

CENTER

PRODUCTION CENTER 1

QUEUE 1 MACHINE GROUP 1 O O

ooo

QUEUE 2

PR(X)UCTION CENTER 2 MACHINE GROUP 2

PRODUCTION CENTER N

QUEUE N

^C^CHINE GROUP N

A

A A

JOB.

COMPLETION.'

PHYSICAL SYSTEM HIGHLIGHTS EXAMPLE

FIGURE III-2

-12-
SIMULATION

MODEL

MACHINE

tSlARACTERISTiqS

JOB

CJIARACTERISTK S

JOB TURNAROUND TIMES

JOB

QUEUING

TIMES

MACHINE

QUEUING

STATISTICS

MACHINE

UTILIZATION

STATISTICS

INPUT/OUTPUT SCHEMATIC

FIGURE III-3
2.2 Methodology

This subsection should provide the user with a detailed

understanding of how the model works.

2.2.1 Physical System Det ails . This subsection is an elaboration of Subsection 2.1.2. The operations that take

place in each of the blocks in the Physical System Block Diagram should be discussed in detail. Detailed block diagrams of subsystems should be provided for a complex model.

The level of detail used in the simulation (e.g., the

smallest meaningful time increment for event-type models,

the way in which complex system interactions are simplified

in the model) should be clearly indicated.

2.2.2 Model Logic an d Data Flow . This subsection should

describe the logical flow of data through the model, from

the entry of input data to the generation of output data.

Include a schematic that indicates the major model software

elements, the data flow between model elements, and model

inputs and outputs. Figures III-4 and III-5 are two types

of schematics for the same model. Either of these two

types, or any other type of schematic that clearly depicts

model logic and data flow, may be used. Accompanying discussion should relate model elements and data flow to physical system elements and data flow described in Section

2.1.2. For complex models, include a table that relates

physical system names to the program segments that simulate

them.

2.3 Assumptions and Limitations

All the system-related assumptions, assumptions on

model parameters (e.g., hard-coded values), limitations on

output accuracy, and any restrictions on the use of the

model should be discussed in detail.

2.3.1 Sy s tem - Re lated As sumpt i ons and Limltat ion s » List any

assumptions that limit or describe the kinds of systems or

phenomena that are treated in the model. For example, a description of a job shop model should define the model's

boundaries (i.e., the subsystems that the model includes),

the kinds of activities simulated (e.g., machine failures

and repairs), etc.

2.3.2 Model Parameters . List the valid ranges for principal

model input parameters (e.g., the maximum and minimum

number of subsystems). Also list values for any parameters

that are included in the model software and cannot be modified by the use r .

-14-
^^lOD. CKisrrER

AND MACHINE

GROUP DATA

JOB

CHAl^CTERISTIC

DATA

INITIALIZE

PROD. CENTER

AND MACHINE

G3R0UPS

INITIALIZE

JOB

(CHARACTERISTIC

ROUTE

JOB TO

PRODUCTION

CENTER

PROD. CENTER

MACHINE

JOB

\CTERISTIClS

REPORT

ALLOCATE

MACHINE

PROCESS

JOB

PRINT

REPORTS

JOB

STATISTICS

MACHINE

GROUP

STATISTICS

MODEL SCHEMATIC SAMPLE #1

FIGURE III-4

-15-
r^POD. CENTER

A^D MACHINE

GROUP DATA

ERISTIC

DATA

READ PRODUCTION CENTER

AND MACHINE GROUP DATA

READ JOB CHARACTERISTIC

DATA

ROUTE JOB TO

PRODUCTION CENTER

PROD. CENTER

AND MACHINE

GROUP REPQl

JOB

CHARACTERISTiq

REPORT

4. ALLOCATE MACHINE

5. PROCESS JOB

6. PRINT REPORTS >

JOB

STATISTICS

MACHINE

STATISTICS

MODEL SCHEMATIC SAMPLE

FIGURE III-5

#2
2,3v3 Output Limitations . List any limitations on output

usage caused by Inaccuracy of the output data. For example,

all digits in an output data field may not be significant

because the input data are estimates, and high precision in

the output data is either unobtainable or inappropriate.

2.3.4 Restrictions on Mode 1 Use . Enumerate all restrictions

on model usage. For example, a job shop model restriction

might be that only flrst-in, first-out queuing disciplines

are modeled for the production centers.

3. Model Input Data

This section should describe in detail all the input

data needed to run the model. The material in this and the

four subsequent sections should serve as a reference for

both the user and the data technician who runs the model.

3.1 General Description

This subsection should describe the overall input data

structure and the data media (tape, cards, disk data sets,

etc.). Include a table that shows input data set names,

their media, and any general data limitations. Also,

describe the Interdependence, if any, of input data sets.

(Detailed descriptions of Individual data items within the

input data sets should be left for Section 3.2.)

3.2 Detailed Descriptions

Input data items are normally organized In related

groups, such as machine performance characteristics, job

processing requirements, etc., or as the data items that are

entered on one punch card. These related groups of data establish and define a data set and should be described together. The input data sets and the items within each data

set should be discussed in the order of their appearance in

the run stream. For each input data set, provide the following Information (each data set description should begin

on a new page).

3.2.1 Data Set Name .

In this subsection, give an overview of the data set's contents and its purpose.

3.2.1.1 Numbe r o f Inputs . Indicate the number of data sets

of this type and the maximum number of data items in the

data set that may (or must) be used in the simulation. Discuss any factors that influence the total number of inputs

from this data set.

-17-
3.2.1.2 Other Re lated Data Sets . List any data sets whose

contents depend on or dictate the input values for this data

set. Discuss the relationships between data items In the

data sets.

3.2.1.3 Description of Data I terns . In this subsection, provide general comments on the format of the data items (e.g.,

free form. Integer in card columns 8-11, NAMELIST) followed

by the description of each of the data items. For each

item, the following should be given: name, type, format (if

fixed), permissible range or fixed value, unit of measurement, default value (value assumed by the program when the

item is omitted), definition of the item describing how it

is used in the model, relationship to other data items.

Tables should be used where appropriate.

3.2.1.4 Sample Input .

A format layout should be provided for the data set to provide the user a visual reference for preparing the input

data

.

3.3 Data Collection and Maintenance

An Important part of model application is data collection. Therefore, it is necessary to include appropriate instructions on data collection and maintenance. Specific

responsibilities need to be assigned to analysts and users

for these functions.

3.3.1 Data Sources . Discuss the data sources for each input

data set. The discussion should identify the form in which

raw data are available, other organizational elements from

which the data must be collected, if appropriate, and the

time required to collect the data.

3.3.2 Collect ion Procedures . Describe any special statistical techniques or experiments for obtaining the data. Identify any other computer programs or models that must be used

to collect or process data, and list or reference instructions for their use. Where appropriate. Include a flowchart

that illustrates the major data collection steps and their

sequence. Figure III-6 is an example of special procedures

to be used in obtaining data for a model. In this example,

the type and frequency of orders are analyzed along with

production center performance data to produce a statistical

data base. This, data base is then used as input to a model

-18-
1

STATISTICALLY

ANALYZE

OFDER

DATA

STATISTICALLY

ANALYZE

PRODUCTION

CENTER DATA

SPECIAL PROCEDURES EXAMPLE

FIGURE III-6

-19-
of the order handling process. The order model produces

data that profiles the arrival patterns, routing distributions, and processing requirements of the jobs. These data

are. In turn, used as Input to the job shop model.

3.3.3 Updating Procedures . Give step-by-step procedures for

maintaining the data sets and preparing them for new experiments. Identify any other computer programs that must be

used to update the data sets, and list or reference Instructions for their use. Where appropriate. Include a flowchart

that Illustrates the major update procedures and their sequence.

A. Model Output Data

This section should describe In detail all the output

data produced by the model and should Indicate their meanings and uses.

4.1 General Description

Discuss the overall output structure In this section.

Indicate the number and types of output data sets, output

media, correlation between outputs, quantity of output (optional and mandatory), and postprocessing, If any, that

should be performed on the output data.

4.2 Detailed Description

For each output data set (or major group of logically connected data Items), provide the following Information (each

output data set should begin on a new page).

4.2.1 Data Se t Name . Give the full name or acronym of the

output data set or group of data under this subheading.

Give an overview of the data set's contents, its purpose,

and its relation to other model results.

4.2.1.1 De script i on of Items . Each output item should be

Included in a table that shows its name, a brief description, and gives information to use in validity checking, if

appropriate. Accompanying discussion should expand on each

Item's description and should show how the items are derived

or calculated. Include mathematical formulae where appropriate . 4.2.1.2 Interpre tat ion . Explain how the data items can be

used, and describe actions to be taken for any subsequent

runs based on the output.

-20-
^.2.1.3 Sample Output . Include a sample of the output data

set. A sample format is satisfactory where it is not practical to provide an actual sample.

5. Run Preparation Instructions

This section of the User's Manual should describe procedures for organizing the input data to submit computer

runs as discussed in Section 3.

5.1 Run-Stream Description

This subsection should give a pictorial (or tabular)

representation of the deck constituting the run-stream that

shows all the control cards and the data cards in proper sequence. Mandatory and optional cards should be discussed.

If the model is interactive, include comments on any special

techniques used for interactive submission of jobs.

5.2 Resource Requirements

This subsection should describe the computer resources

required by the model. These include main memory, mass

storage, number of tape units, execution time, numbers of

punched cards, and printed lines expected as output. If the

computer resources vary depending on input data, provide

aids to estimate them.

5.3 Restart/Recovery Procedures

For models that require large amounts of computer

resources it is important to recover from abnormal terminations and to restart the job. If any such provisions are

made in the model design, they should be discussed in this

subsect ion

.

6. Sample Model Run

Include a sample run that illustrates the complete input scenario and the resulting output to assist a beginning

user in making a test run and verifying correctness of procedures .

-21-
7. Trouble-Shooting Guide

Tabulate user input error-messages produced by the

model software, and describe the required corrective action.

Since other errors should be handled by programmers, those

errors should be discussed in the programmer's manual.

APPENDICES

Three appendices should be provided as required. Appendix A should provide an alphabetical listing of all abbreviations and acronyms used in the User's Manual. Appendix B should list all specialized User's Manual terms and

their definitions. All applicable documents, including cited and uncited references, should be provided in Appendix C.

-22-
IV. GUIDELINES FOR PREPARING A PROGRAMMER'S MANUAL

This section provides a recommended organization for a Programmer's Manual and describes the contents of each section and subsection proposed for that manual. A

Programmer's Manual written using these guidelines will enable a programmer to maintain and modify a model. The

guidelines will provide all the details necessary for a programmer to understand the operation of the model and to

trace through it for debugging, for making modifications,

and for determining if and how the model can be converted to

other computer systems. Figure IV-1 is a recommended table

of contents for a Programmer's Manual. The sections and

subsections included in the figure cover the general needs

of a programmer interested in a model. In documenting a particular model, however, sections and subsections may be

added to improve clarity, and some subsections may be omitted for simple models. Any appropriate documentation produced using a program documentation language could be used

to satisfy the guidelines contained herein.

1. Introduction

The introduction to the Programmer's Manual should contain the background of the project, the purpose of the

model, and an overview of the remaining sections in the

manual. A common introduction may be used for all the manuals prepared for a model, but the specific purpose of a Programmer's Manual should be included in a statement of the

form:

"The purpose of this manual is to provide programmer

personnel of (model name) with the information necessary to effectively maintain and modify the model."

2. Model Specifications

This section should provide a summary of the model's

specifications, including capabilities (i.e., problems addressed and methods of solution), a description of the host

computer system, and the processing requirements (i.e.,

memory, peripherals, languages) placed by the model on that

host system. The details should be presented in tabular

form (supplemented by narrative description, as appropriate), whereby one table describes the complete modeling system and additional tables describe major submodels or programs as needed for clarity.

-23-
1. Introduction

2. Model Specifications

3. Model Description

3.1 Processing

3.1.1 Overview

3.1.2 Major Components

3.1.3 Model Initialization and Wrap-up

p. 2 Data Structures

3.2.1 Local Data Structures

* 3.2.2 Global Data Structures

t 3.2.3 Special Data Structures

3.3 Overlays

3.4 Model Modifications

3.4.1 Planned Maintenance

3.4.2 Other Changes

4. Description of Routines

4.1 Routine Name (First Routine)

1. 1 Purpose

1. 2 Type

1. 3 Calling Sequence

1. 4 Argument Definition

1. 5 Calling Routines

1. 6 Called Routines

1. 7 Files

1. 8 Error Messages

1. 9 Narrative

1. 10 Block Diagrams

1. 11 Sample Test Run

5. Data Base Description

5.1 File Name (First File)

5.1.1 Purpose

5.1.2 Format

5.1.3 Routines

5.1.4 Updating

6. Source Listing

7. Error Messages

APPENDICES

A. Glossary

B. Bibliography

C. Index

D. Model Test Results

RECOMMENDED TABLE OF CONTENTS FOR A

PROGRAMMER'S MANUAL

FIGURE IV-1

-24-
3. Model Description

This section should contain a we 1

1 - s t r uc t ured presentation with emphasis on the operational details of the model.

The discussion should be written in an easy-to-understand

manner that cross-references special model language terms

with modeled system features whenever possible. This section should be divided into four subsections.

3 . 1 Processing

This subsection should provide details on model operations for programmers who need to understand the processing

techniques used in the model. The discussion should be at

the Macro level, with a discussion of internal routine details postponed until Section 4 of this manual. Details on

I/O formats and default input data values should be reserved

for the User's Manual. Block diagrams should be used as

necessary to supplement the narrative.

3.1.1 Overview . This subsection should present, in modeled

system terminology, an overview of the problem solved by the

model. Include a discussion of the basic tasks modeled.

Figure IV-2 is an example of a block diagram that could supplement a narrative description in this subsection.

3.1.2 Ma jor Compo nents . This subsection should describe the

flow of data or control information through the model at the

major routine, or routine group, level. Include detailed

block diagrams that depict paths among the modeled tasks,

highlighting major decision points in the logic flow. This

subsection may contain as many levels of discussion as are

necessary to clearly describe model operation.

3.1.3 Model Init ializat io n and Wr ap

- Up . This subsection

should note any differences in the performance of model

tasks accomplished during model initialization and wrap-up

and those same tasks when performed during normal processing .

3 . 2 Data Structur es

.

This subsection should provide information on all data

structures internal to the model. Include descriptions of

local and global variables, arrays, and data sets, as well

as any special data structures, such as the set-entity relationships in SIMSCRIPT. If required for understanding,

separate descriptions of each array index should be provided .

-25-
MAIN

SIMULATES A

JOB SHOP

OPERATION

W

TIMING. RDOTINE

DETERMINES MOST

IMMINENT EVENT

JOB. ARRIVAL

REPRESENTS JOB

ARRIVING AT THE

JOB SHOP

END. OF. PROCESSING

REPRESENTS PROCESS

COMPLETION AT A

PRODUCTION CENTER;

RELEASES MACHINE

REPORT

PRINTS SIMULATION

STATISTICS

ROUTE. JOB

DETEPMINGS NEXT

PRODUCTION CENTER

TO PROCESS JOB

ALLOCATE.MACHINE

QUEUES JOB OR

ALLOCATES MACHINE

FDR JOB PROCESSING;

SCHEDULES END. OF.

PROCESSING

END. OF. JOB

DETEFMINES IF

JOB IS COMPLETED;

IF NOT, CALIFS

ROUTE. JOB

T

EXAMPLE OF A MODEL OVERVIEW BLOCK DIAGRAM

FIGURE IV-2

-26-
3.2.1 Local Data St ructure

s

. This subsection should contain

the meaning and purpose of all local variables, arrays, and

data sets (local data structures have their values defined

only within particular routines). To improve clarity, local

data structures should be associated with the routines in

which they appear.

3.2.2 Global Data St ructures . This subsection should contain the meaning and purpose of all global variables, arrays, and data sets (global data structures are defined

throughout the model). Include an alphabetized list of global data structures (including special data structures),

cross-referenced by the routines in which they appear, and

the source code li ne numbers (Table IV— 1). Source code line

numbers can be obtained from the source listing in Section 6

of this manual. Examples of global data structures that

should be Included in this subsection are the COMMON blocks

of FORTRAN.

3.2.3 Special Data St ructure s . Any special data structures,

both local and global, should be listed and described in

this subsection. For example, a job shop model implemented

in SIMSCRIPT might represent the jobs with temporary entities, the job processing requirements with entity attributes, and the sequence of production centers required to

process the job with a set (owned by the job with production

centers as numbers). A GPSS implementation, however, might

represent the jobs with transactions, the processing requirements with transaction parameters, and the route with a row in a matrix save value (the columns contain the sequence

of production centers).

3.3 Overlays

If the model is overlayed, this subsection should provide details of the overlay design decisions that determined

the overlay strategy. Included should be a narrative and a block diagram description of the control flow of the overlays and their interactions. Figure lV-3 contains a sample

overlay structure with a main program, four primary overlay

segments, and five secondary overlay segments. Routines

residing in each overlay, and their memory requirements,

should be listed (Table IV-2). References should be made

to the discussion of model processing in this manual (Subsection 3.1) to reinforce or clarify the overlay discussion.

-27-
NAME SOURCE CODE

NUMBER ITEM ROUTINE LINE NUMBER

1 AMGO (V)^ MAIN 15

2 AOPS (A)"^ FASTER 52

3 BVBC (V) MAIN 12

TFBAI 163

PRNTREP 210

212

2 90

4 CMCODE (V) MAIN 10

ACSUMT 150

PRNTREP 211

286

5 LEVDAT (DS) MAIN 13

AUXSUM 12 0

THBAB 184

THBAI 195

PRNTREP 250

6 TCTV (V) MAIN 14

ABFSTR 106

108

PRNTREP 210

253

Item is a variable.

Item is an array.

Item is a data set.

CROSS-REFERENCED DATA STRUCTURE LIST SAMPLE

TABLE IV-1

-28-
<<

CM

00

in

D

EhU

EhW<>Ou

CO I

ZD

-29-
OVERLAY

SEGMENT NUMBER

ROUTINE

NAME

MEMORY

REQUIREMENTS

(KBYTES)

1 MAIN 50

ROUTl 10

ROUT 2 20

ROUTS 20

2 PROCl 100

PR0C2 25

PR0C3 25

3 COMPi 75

C0MP2 50

4 SOLVl 75

S0LV2 50

5 LSIGNl 100

• LSIGN2 20

LSIGN3 15

6 COMPIO 140

7 FINDl 70

FIND2 48

8 SUMIO 140

9 WRITER 137

10 REPORT 128

SAMPLE LIST OF ROUTINES BY OVERLAY SEGMENT

TABLE IV-2

-30-
3. A Model Modifications

This subsection should include information concerning

changes in model software and data bases. Include a description of any programming conventions used in the model

(e.g., all variables referencing one data base may begin

with a specific character). In addition, this subsection

should provide procedures needed by programmers during the

model compilation, recompilation , and execution stages. Include a sample control card setup that illustrates each of

those states, including mandatory and optional cards. If

the model is interactive, include comments on interactive

procedures.

3.4.1 Plan ned Ma inten a nee . This subsection should identify

all planned periodic maintenance on the model and its data

bases (e.g., periodic data base updates).

3.4.2 Othe r Changes . This subsection should identify procedures for making all modifications to the model other than

planned periodic maintenance. Provide details for making

changes necessitated by programming errors discovered during

model usage, as well as changes to the model required by

changes in the host modeling language. Include directions

for implementing software changes to produce a new version

of the model (e.g., changes in model applicability).

A. Description of Routines

This section should provide a detailed description of

principal model routines. Include a discussion of all types

of routines that comprise the model (i.e., event,

subroutine, function, etc). Provide an alphabetized listing

of all routine names along with calling routines and called

routines (Table IV-3) or a block diagram showing routine

linkages, as needed. Each routine should be described in a separate subsection. For each routine, provide the following information.

A.l Routine Name (First Routine)

A. 1.1 Pur po se . Briefly state the purpose of the routine

(e.g., routine ALLOCATE computes the time a job is scheduled

to complete its processing at a production center).

A. 1,2 Type . Specify the type of routine (i.e., function,

subroutine). A description of all routine types in the

model should be contained in the introductory comments of

this sect ion

.

-31-
^•1«3 Calling Sequence . List all variables, arrays,

pointers in the routine calling sequence.

4.1.4 Argument De f ini t ion . Define all routine arguments.

4.1.5 Calling Rout ine s . List all routines that call this

rout ine

.

4.1.6 Called Rout ines . List all routines called by this

routine

.

4.1.7 Files . List all files this routine creates or uses.

4.1.8 Error Me s sage s . Itemize all error messages which can

originate in this routine.

4.1.9 Narrati ve . Include a narrative description as necessary, to amplify and highlight subtleties included in the

code. As a minimum, include any equations and formulae

referenced from the Analyst's Manual.

4.1.10 Block Diagrams . Use block diagrams or other documentation aids (such as program documentation languages), as

required, to clearly depict operation of the routine.

4.1.11 Sample Tes t Run . Provide the results of test runs,

along with values of input data, for each complex routine to

assist in verifying changes to those routines.

5. Data Base Description

This section should discuss all mass storage files used

or created by the model. Each file should be described in a separate subsection and should contain the following information (each file description should begin on a new page).

5.1 File Name (First File)

Provide the full name or acronym of all the model

files.

5.1.1 Pur po se . Briefly state the purpose of the file (e.g.,

contains preprocessed destination data).

5.1.2 Format . Explain the format of the file (i.e., block,

size, record size, data item identification, and field

sizes)

.

-32-
NUMBER CALLING ROUTINE CALLED ROUTINE

1 ABFSTR NONE

2 ACSUMT TFBAI

TKBAB

AUXSuM ACSDMT

4 FASTER ABFSTR

AUXSUM

5 MAIN FASTER

6 PRNTREP NONE

7 TFBAI PRNTREP

8 THBAB THBAI

9 THBAI PRNTREP

CROSS-REFERENCED ROUTINE LIST EXAMPLE

TABLE IV-3

-33-
5.1.3 Rout 1 ne

s

. Identify all routines that use or create

the file.

5.1.4 Updat ing . Include Instructions for file maintenance

and updating as appropriate.

6. Source Listing

This section should contain the source code of the

model. If the source listing is large, it should be bound

separately and made available upon request. Also, source

listings with line numbers can be referenced from Subsection

3.2.2 as a cross-reference for model variables.

7. Error Messages

All program-generated error messages, the names of the

routines in which they are generated, and suggested corrective actions should be listed in this section. Each error

message may be described in a separate subsection.

APPENDICES

Four appendices to this manual should be provided as

required. Appendix A should define all terms in the

Programmer's Manual not defined elsewhere in the document.

A list of applicable documents, including cited and uncited

references, should be provided in Appendix B. Appendix C

should provide an alphabetized index that gives the page on

which each subject contained in the Programmer's Manual may

be found. If the Programmer's Manual is divided into more

than one volume, the index in the first volume should be the

index of the volumes. The index in each of the remaining

volumes should reference only those subjects within that

volume. Appendix D should provide a listing of model test

results along with values entered into the model that produced those results. Include any interim model outputs

necessary to understand the final outputs. Provide analyses

of model results as necessary.

-34-
GUIDELINES FOR PREPARING AN ANALYST'S MANUAL

This section presents a recommended organization for an

Analyst's Manual and describes the contents of each section

and subsection to be Included In that manual. An Analyst's

Manual prepared using these guidelines will enable an

analyst to understand a model's functional structure, the

algorithms used in the model, and techniques employed for

model verification and validation. Figure V-1 contains a recommended table of contents for an Analyst's Manual. The

sections and subsections Included cover the general needs of

an analyst Interested in a model. In documenting a particular model, however, sections and subsections may be added to

improve clarity, and some subsections may be omitted for

s Imple mode Is .

1. Introduction

The introduction to the Analyst's Manual should contain

the background of the project, the purpose of the model, and

an overview of the remaining sections In the manual. A common introduction may be used for all the manuals prepared

for a model, but the specific purpose of the Analyst's Manual should be included in a statement of the form:

"The purpose of this manual is to provide nonprogramming analysts of (model name) with the details of the

algorithms used in the model and the techniques employed for model verification and validation."

2. Functional Description of the Model

This section should contain a well-structured presentation with emphasis on the functional details of the model.

The discussion should be written In an easy-to-understand

manner that, whenever possible, avoids the use of highly

specialized terms. The section should be divided into four

subsections.

2.1 Overview

This subsection should provide a functional description

of the model in sufficient detail for an analyst to understand the salient system features that were modeled. Functional flow charts and other graphics should be used to

enhance the narrative. Include a statement of the kind of

model (i.e., discrete-event model that simulates jobs entering a job shop at arbitrary points in time, being routed

through a predetermined sequence of production centers, and

being processed at the production centers) and the degree to

which the model portrays the real world system. Figure V-2

is an example of modeled system highlights. Included should

-35-
1. Introduction

2. Functional Description of the Model

2.1 Overview

2.2 Detailed Methodology

2.3 Assumptions and Limitations

2.3.1 Stochastic Assumptions

2.3.2 Magnitude Limitations

2.3.3 Critical Values

2.4 Model Flexibility

3. Model Input and Output Data

3.1 Input Data

3.2 Output Data

4. Model Verification and Validation

4.1 Verification Techniques

4.2 Validation Considerations

APPENDICES

A. Glossary

B. Bibliography

RECOMMENDED TABLE OF CONTENTS

FOR AN ANALYST'S MANUAL

FIGURE V-1

-36-
JOB

ARRIVALS

PRODUCTION CEWrER 1

QUEUE 1

ROUTE JOBS

TO

PRODUCTION

CENTER

MACHINE GROUP 1 oo

ooo

QUEUE 2

PRODUCTION CEINTTHR 2 MACTINE GROUP 2

PRODUCTION CENTER N

QUEUE N

MACHINE GROUP N

A

A A

JOB

COMPLETIONS

MODELED SYSTEM HIGHLIGHTS EXAMPLE

FIGURE V-2

-37-
be the set of model responses (output) produced by a given

set of model input data. Figure V-3 provides an example of

the types of model input and output. For additional details

on model description, the analyst should be directed to the

appropriate section of the User's Manual for this model.

2.2 Detailed Methodology

This subsection should provide the functional details

for analysts to understand the algorithms and equations used

in the model. Well-known mathematical equations (and formulae) should be clearly identified and references should be

cited for their derivation. For example, in a job shop model

the queuing discipline simulated for each production center

should be stated. Include the derivation for extensions of

known results or for the development of new analytical techniques. Special complicating details, such as the use of

precalculated data for job arrival times should be stated.

The description must be detailed enough to demonstrate how

the model uses the input data to calculate output information. Functional flow charts and graphs should be used to

enhance the narrative descriptions of each algorithm. Figure V-4 is an example of a model functional flow chart.

This section should include a subsection for each major algorithm or set of equations.

2.3 Assumptions and Limitations

This subsection should list all model assumptions and

all factors that affect or limit model output use. The following items should be included as appropriate.

2.3.1 Stochasti c Assumption s . In this subsection, itemize

all stochastic assumptions that affect model output accuracy. For example, the treatment of certain random variables

in a simplistic manner, by using only their mean values and

not sampling from a statistical distribution, should be

described in this subsection. Each stochastic item should

be described in a separate subsection (e.g., 2.3.1.1,

2.3.1.2).

2.3.2 Magni tude Limlta t ions . This subsection should include

all limitations on the size of the problems the model can

-38-
SIMULATION

MODEL

INPUT/OUTPUT SCHEMATIC

FIGURE V-3

-39-
address. For example, the current dimensions of certain arrays in a model may limit the number of activities that can

be represented in that model. Each limitation should be

described in a separate subsection.

2.3.3 Criti c al Values . This subsection should identify

critical input data values to which model outputs are sensitive. Many elements that have a range of values will have

one value that is particularly significant to the analyst.

This may be a breakpoint, a minimum stock level, or a critical job rate, etc. Each critical value should be described

in a separate subsection (e.g., 2.3.3.1, 2.3.3.2, etc.).

2.4 Model Flexibility

This subsection should address the capability of adapting the model to changing requirements, such as anticipated

physical system operational changes, interacting with new or

improved models, and planned periodic changes. An example

of a flexible design is one that facilitates the addition of

a machine failure and repair subsystem to a job shop model.

Model components and procedures designed to be flexible

shall be clearly identified. Factors that affect model

flexibility are the familiarity of the analyst with the

model, the model's size, its complexity, and its data structures. Subsections should be used as required.

3. Model Input and Output Data

This section should discuss the categories of input

data and the accuracy of model output data. The material

contained in the next two subsections will enable the

analyst to assure the existence of the data necessary to execute the model and to ascertain the accuracy of the data

generated by the model.

3 . 1 Input Data

Identify all categories of input data and any special

analytical techniques required to obtain those data. If the

sources of input data include output from other models, provide sufficient details to enable an analyst to assess the

appropriateness of those data in solving his problem. For

example, if the arrival rate of jobs is provided by a separate model of the order handling process, the analyst

needs to determine that the simulated ordering process

-40-
JOB

ARRIVAL

ROUTE JOB

TO NEXT

PRODUCTION

CENTER

MACHINE

IDLE?

YES—

IK)

QUEUE

JOB FDR

PRODUCTlOM

CENTER

SCHEDULE

END OF

SERVICE

REMOVE

FIRST JOB

FROM QUEUE

MODEL - FUNCTIONAL FLOW EXAMPLE

FIGURE V-4

-41-
corresponds to the one existing in his job shop. Details on

input data types and formats should be reserved for the

User's Manual

.

3.2 Output Data

This subsection should provide the analyst with a methodology for assessing the accuracy of model output data.

Since the accuracy of the output values will be judged in

relating to the method used to derive them, a review of the

algorithms used to compute those output values may be necessary at this point. Describe in detail any corrective actions to be taken by an analyst in case of inaccurate output

values, (i.e.. Should the analyst contact a programmer for a program change or have a user modify the input data deck to

correct the problem?). Subsections may be used as required.

4. Model Verification and Validation

This section of the Analyst's Manual should describe

the methodology used to verify and validate the model.

Model verification (sometimes referred to as software validation) is concerned with the compatibility of the model's

programmed structure to the analyst's design and with model

debugging. Model validation provides the analyst, and user,

with the confidence that the model provides a good representation of the modeled system.

4.1 Verification Techniques

This section should provide an analyst with concise

procedures by which the model was verified. Each equation

included in Section 2.2 of the Analyst's Manual should be

verified and cross-referenced to the Programmer's Manual for

this model. Include all other verification techniques used.

4.2 Validation Considerations

This section should provide an analyst with the

description of any procedures that were used to ensure that

the model is an "accurate" abstraction of the real system.

Any methodology used to determine how well the model

represents the real system should be presented in this section. While complete confidence in a model may be impossible, a good validation procedure can increase the amount of

confidence an analyst has in a model. Figure V-5 is an example of a graph that could be used In a model validation

procedure

.

-42-
APPENDICES

Two appendices to this manual should be provided as required. Appendix A should define all terms In the Analyst's

Manual not defined elsewhere In the document. Appendix B

should provide a list of applicable documents and a bibliography designed for use by system analysts.

-43-
FREQUENCY DISTRIBUTION OF JOB TURNAROUND TIMES

COMPARISON OF ACTUAL AND SIMULATED RESULTS

ACTUAL

SIMULATED

TIME (MINUTES)

FIGURE V-5

-44-
VI. MODEL SUMMARY

It is suggested that documentation for each model include the information outlined below, as part of the model

documentation package. This information provides general information about the model and facilitates its possible use

by others.

A. Basic Description.

1. Name or title of Model.

2 . De ve lope r ( s ) . 3. Agency or company.

4. Sponsor; purpose or objective of sponsor.

5. When developed?

6. Where developed? . 7. Development time and cost?

8. Developed separately or as part of larger

study?

B. Subject Matter of Model.

1. Major purpose of model.

2 . Scope of model

.

3. State basic description or theory underlying

the model

.

4. State specific disc ipl ine ( s) required for model

use , if required . 5. How does model differ from other similar

mode Is?

Modeling Technique.

(e.g.

)?

mode 1

?

Describe type of model.

Does model use any standard packages

linear programming, statistical, etc

Was the model developed from another

If yes, describe process.

Is its structure clear? Its variables?

Describe data requirements of model.

Does model receive any data from other models?

What constraints are affecting the model?

Computer Aspects of Model.

1. In what computer language is the model written?

2. What machine(s) is it programmed for?

3. How much time does it take to run?

4. Size of model (lines of code, core to run etc.)

5. How many parameters does model require?

-45-
E. Validation of model.

1. Has model been validated? How?

2. Has model been documented? How well?

3. Has model been critiqued or appraised? By whom?

At what point?

4. Has there been a sensitivity analysis performed

on the model? By whom?

5. Can the model by used from current documentation?

Has it been used?

F. Model Use.

1. If asked, how would you demonstrate the utility

of the model? Have you demonstrated it?

2. With whom should one get in touch to discuss

use of the model?

3. How much would it cost to transfer the model?

4. Are the model relationships or parameters easy

to use for the user?

5. Have there been any papers given or written on

the model? Cite references.

6< Is the output of the model special or is it

designed for a general audience?

-46-
VII. SUMMARY

The guidelines on the previous pages should assist in

the preparation of documentation for computer models. Such

documentation is primarily a tool for human communication.

Varying information needs of different types of readers such

as managers, model users, programmers, and analysts are

accommodated. While managers are often in need of a broad

spectrum of general information required for decisionmaking,

users are primarily interested in practical aspects of the

model and its application to the user's specific problems.

Programmers require technical details which are needed for

maintenance and modification of the model, while analysts are

interested in the processing aspects and the underlying

analytical methods and algorithms. By providing sections

which have been specifically tailored to the viewpoints of

diverse readers, human communications are enhanced and

differing information needs are satisfied.

Users of these guidelines are requested to provide feedback on their use of this document to the authors. It would

be of interest how well the document has served the user's

needs, what parts have been especially useful, what parts were

not used and why, and what changes or additions are suggested

by the readers and users of the document. Such comments would

be used in future revisions. Use of this user experience

would be of great value to the Federal Modeling Community to

which this paper is addressed.

-47-
BIBLIOGRAPHY

1. Balen, N. E. "An Air Force Guide to Contracting for

Software Acquisition," Bedford, MA: Mitre Corporation,

January 1976 (NTIS No. AD-A020-444 )

.

2. Comptroller General of the United States. "Ways to Improve Management of Federally Funded Computerized Models,"

Washington, DC: National Bureau of Standards, Department of

Commerce, General Services Administration, August 23, 1976

(LCD-75-111 )

.

3. Connally, J. T. "Software Acquisition Management Guidebook: Regulations, Specifications and Standards," Bedford,

MA: Mitre Corporation, October 1975 (NTIS No. AD-AOl 6-401 )

.

5. Control Analysis Corporation. "Draft Report TAC Warrior

Analysts Manual," Palo Alto, CA : Control Analysis Corporation, 1978.

5. Cooley, B. "Documenting Simulation Models for Management Use," Winter Simulation Conference, Gaithersburg , MD:

National Bureau of Standards, December 1977.

6. Gass, Saul I. "Computer Model Documentation," Winter

Simulation Conference, Gaithersburg, MD: National Bureau of

Standards, December 1978.

7. Gass, Saul I. "Computer Science and Technology: Computer Model Documentation: A Review and an Approach," Washington, DC: National Bureau of Standards, Department of Commerce, February 1979 (NBS Pub. No. 500-39).

8. Hagan, S. R., et al. "An Air Force Guide for Monitoring

and Reporting Software Development Status," Bedford, MA:

Mitre Corporation, September 1 975 (NTIS No. AD-AO 1 6-4 88 ) .

9. Jet Propulsion Laboratory. "Software Design and Documentation Language," Pasadena, CA: National Aeronautics and

Space Administration, July 1977.

10. Katzan, Harry, Jr., "Systems Design and Documentation,

An Introduction to the HIPO Method," New York, NY: Van Nostrand Company, 1976.

11. Mihran, G, Arthur, "Simulation Statistical Foundations

and Methodology," New York, NY: Academic Press, 1973.

12. , "Study for Assessing Ways to Improve

the Utility of Large-Scale Models," Palo Alto, CA: Control

Analysis Corporation, 1977.
13. Mulford, James 0. and Jarze, Lyle R. "Critique of the

TAC Turner and TAG Warrior Documentation," Washington, DC:

FEDSIM, April 1978.

14. Nance, Richard E. "The Feasibility of and Methodology

for Developing Federal Documentation Standards for Simulation Models," Blacksburg, VA : Virginia Polytechnic Institute, June 1977.

15. Newton, Otis L. and Weatherbee, James E. "Guidelines

for Documenting Computer Simulation Models," Washington, DC:

FEDSIM, October 1979.

16. . "Update 2 to Computer Program Documentation Guidelines ."Washington, DC: FEDSIM, March 1979.

17. . "Update 3 to Computer Program Documentation Guidelines," Washington, DC: FEDSIM, June 1979.

18. . "Update 4 to Computer Program Documentation Guidelines," Washington, DC: FEDSIM, June 1979.

19. Ramapriyan, H. K. , Mulford, James 0., and Jarze, Lyle

R. " Evaluation of Computer Program Documentation Guidelines," Washington, DC: FEDSIM, June 1978.

20. Ramapriyan, H. K. Weatherbee, James E., and Newton,

Otis L. "Update 1 to Computer Program Documentation Guidelines," Washington, DC: FEDSIM, January 1979.

21. Schoeffel, W. L. "An Air Force Guide to Software Documentation Requirements," Bedford, MA: Mitre Corporation,

June 1976 (NTIS No. AD-A02 7 -05 1 ) .

-49-
NBS-n4A IREV. 2-ec) U.S. DEPT. OF COMM* 2« Performing Or^sn* Report No«

RiRi innRAPHir data REPORT NO. SHEET (See instructions) SP 500-73 January 1981

4. TITLE AND SUBTITLE

Computer Science and Technology:

Computer Model Documentation Guide

5. AUTHOR(S)

Prepared by the Federal Computer Performance Evaluation and Simulation Center

6. PERFORMING ORGANIZATION (If joint or other thon NBS. see instructions;

Federal Computer Performance Evaluation

and Simulation Center

Washington, DC 20330

7. Contract/Grant No.

I. Type of Report & Period Covered

Final

9, SPONSORING ORGANIZATION NAME AND COMPLETE ADDRESS (Street, City. State, ZIP)

Programming Science Division

Center for Programming Science and Technology

Institute for Computer Sciences and Technology

National Bureau of Standards, Washington, DC 20234

10. SUPPLEMENTARY NOTES

Library of Congress Catalog Card Number: 80-600190

Document describes a computer program; SF-185, FlPS Software Summary, Is attached.

11. ABSTRACT (A 200-word or less factual summary of most sigr\ificant informatiort. If rfocument includes a significant

bibliography or literature survey, mention it here)

This document provides guidelines for preparing documentation for computer

models. Recommended structures for four types of manuals providing model

information for four different classes of audiences (managers, users,

analysts, and programmers) is presented. This document specifies the

content of sections and subsections for each type of manual. Manuals

prepared using these guidelines will enable persons interested in a model to understand the capabilities and limitations of that model.

12. KEY WORDS (Six to twelve entries; alphabetical order; capitalize only proper names; and separate key words by semicolons)

Documentation; manuals; models; simulation.

13. AVAILABILITY

Unlimited

1

1 For Official Distribution. Do Not Release to NTIS

r% Order From Superintendent of Documents, U.S. Government Printing Office, Washington, D.C.

20402.

Order From National Technical Information Service (NTIS), Springfield, VA. 22161

14. NO. OF

PRINTED PAGES

56

15. Price

$3.75

USCOMM-OC e043-P80
ANNOUNCEMENT OF NEW PUBLICATIONS ON

COMPUTER SCIENCE & TECHNOLOGY

Superintendent of Documents,

Government Printing Office,

Washington, D. C. 20402

Dear Sir:

Please add my name to the announcement list of new publications to be issued in the series: National Bureau of Standards Special Publication S00-.

Name

Company

Address

City State Zip Code

(Notincation key N-S03)

it us. GOVERNMENT PRINTING OmCE : 1981 O—340-997 (342)

There's

anew

look

to...

/ A ^

t^X? 1 / ' ' ' monthly

y/ /y/ magazine of the Nation- ^ ^^^^^^"^y^ of Standards.

>/ Still featured are special ar- ^^^^0^^^^/ tides of general interest on

^•^^^^^^ y/ current topics such as consum-

\^>^ y / er product safety and building ^^^

/ technology. In addition, new sections are designed to . . . PROVIDE

SCIENTISTS with illustrated discussions

of recent technical developments and

work in progress . . . INFORM INDUSTRIAL

MANAGERS of technology transfer activities in

Federal and private labs. . . DESCRIBE TO MANUFACTURERS advances in the field of voluntary and

mandatory standards. The new DIMENSIONS/NBS also

carries complete listings of upcoming conferences to be

held at NBS and reports on all the latest NBS publications,

with information on how to order. Finally, each issue carries a page of News Briefs, aimed at keeping scientist and consumalike up to date on major developments at the Nation's physical sciences and measurement laboratory.

(please detach here) BSCRIPTION ORDER FORM

;r my Subscription To DIMENSIONS/NBS at S11.00. Add $2.75 for foreign mailing. No additional

tage is required for mailing within the United States or its possessions. Domestic remittances

ji uld be made either by postal money order, express money order, or check. Foreign remittances

I uld be made either by international money order, draft on an American bank, or by UNESCO

jl pons.

I «d Subscription to:

I NAME-FIRST, LAST III I I II 1 I I II II I I

Remittance Enclosed (Make checks payable

to Superintendent of Documents)

O Charge to my Deposit Account No.

LI 1 II 1

1 1

COMPANY NAM/: OR ADDITIONAL ADDRESS LINE

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

LI 1 1 1 1 1 1

STREET ADDRESS

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

LI 1 1 1 1

CITY

1 1

1 1 1 1 1 1 1 1

STATE

1

ZIP CODE

1 1 II

:ASE PRINT

MAIL ORDER FORM TO:

Superintendent of Documents

Government Printing Office

Washington, D.C. 20402

NBS TECHNICAL PUBLICATIONS

PERIODICALS

JOURNAL OF RESEARCH—The Journal of Research of the National Bureau of Standards reports NBS research and develop- ment in those disciplines of the physical and engineering sciences in which the Bureau is active. These include physics, chemistry,

engineering, mathematics, and computer sciences. Papers cover a broad range of subjects, with major emphasis on measurement

methodology and the basic technology underlying standardization. Also included from time to time are survey articles on topics

closely related to the Bureau's technical and scientific programs. As a special service to subscribers each issue contains complete

citations to all recent Bureau publications in both NBS and non- NBS media. Issued six times a year. Annual subscription: domestic

S13; foreign $16.25. Single copy, $3 domestic; $3.75 foreign. NOTE: The Journal was formerly published in two sections: Section A "Physics and Chemistry" and Section B "Mathematical

Sciences." DIMENSIONS/NBS—This monthly magazine is published to in- form scientists, engineers, business and industry leaders, teachers,

students, and consumers of the latest advances in science and

technology, with primary emphasis on work at NBS. The magazine

highlights and reviews such issues as energy research, fire protection, building technology, metric conversion, pollution abatement,

health and safety, and consumer product performance. In addition, it reports the results of Bureau programs in measurement

standards and techniques, properties of matter and materials, engineering standards and services, instrumentation, and

automatic data processing. Annual subscription: domestic $11;

: foreign $13.75.

! NONPERIODICALS

Monographs—Major contributions to the technical literature on

various subjects related to the Bureau's scientific and technical activities. Handbooks—Recommended codes of engineering and industrial

' practice (including safety codes) developed in cooperation with inI terested industries, professional organizations, and regulatory

bodies. Special Publications—Include proceedings of conferences spont sored by NBS, NBS annual reports, and other special publications

appropriate to this grouping such as wall charts, pocket cards, and

bibliographies. Applied Mathematics Series—Mathematical tables, manuals, and

studies of special interest to physicists, engineers, chemists,

I biologists, mathematicians, computer programmers, and others

' engaged in scientific and technical work.

National Standard Reference Data Series—Provides quantitative

(i data on the physical and chemical properties of materials, comI .piled from the world's literature and critically evaluated.

|l Developed under a worldwide program coordinated by NBS under

jl the authority of the National Standard Data Act (Public Law

I 90-396).

NOTE: The principal publication outlet for the foregoing data is the Journal of Physical and Chemical Reference Data (JPCRD)

published quarterly for NBS by the American Chemical Society

(ACS) and the American Institute of Physics (AIP). Subscriptions,

reprints, and supplements available from ACS, 1 155 Sixteenth St., NW, Washington, DC 20056.

Building Science Series—Disseminates technical information developed at the Bureau on building materials, components,

systems, and whole structures. The series presents research results,

test methods, and performance criteria related to the structural and

environmental functions and the durability and safety characeristics of building elements and systems.

Technical Notes—Studies or reports which are complete in themselves but restrictive in their treatment of a subject. Analogous to monographs but not so comprehensive in scope or definitive in treatment of the subject area. Often serve as a vehicle for final reports of work performed at NBS under the sponsorship of other government agencies.

Voluntary Product Standards—Developed under procedures

published by the Department of Commerce in Part 10, Title 15, of

the Code of Federal Regulations. The standards establish nationally recognized requirements for products, and provide all concerned interests with a basis for common understanding of the

characteristics of the products. NBS administers this program as a supplement to the activities of the private sector standardizing

organizations. Consumer Information Series—Practical information, based on NBS research and experience, covering areas of interest to the con- sumer. Easily understandable language and illustrations provide

useful background knowledge for shopping in today's technological marketplace. Order the above NBS publications from: Superintendent of Documents, Government Printing Office, Washington, DC 20402. Order the following NBS publications—F/PS and NBSIR's—from

the National Technical Information Services, Springfield, VA 22161.

Federal Information Processing Standards Publications (FIPS

PUB)—Publications in this series collectively constitute the Federal Information Processing Standards Register. The Register

serves as the official source of information in the Federal Govern- ment regarding standards issued by NBS pursuant to the Federal Property and Administrative Services Act of 1949 as amendec

Public Law 89-306 (79 Stat. 1127), and as implemented by E>

ecutive Order 11717(38 FR 12315, dated May 11, 1973) and Par

of Title 15 CFR (Code of Federal Regulations).

NBS Interagency Reports (NBSIR)—A special series of interim or

final reports on work performed by NBS for outside sponsors

(both government and non-government). In general, initial dis- tribution is handled by the sponsor; public distribution is by the National Technical Information Services, Springfield, VA 22161,

in paper copy or microfiche form.
U.S. DEPARTMENT OF COMMERCE

National Bureau of Standards

Washington,

D C. 20234

OFFICIAL BUSINESS

Penalty for Private Use. $300

POSTAGE AND FEES PAID

U.S. DEPARTMENT OF COMMERCE

COM-2 1 S

SPECIAL FOURTH-CLASS RATE

BOOK