NATL INST. OF STAND & TECH R.I.C.
'

.
NATIONAL INSTITUTE OF STANDARDS &

TECHNOLOGY

Research Information Center

Gaithersburg, MD 20890

NISTIR 88-3826

Evaluating Emergency Management

Models and Data Bases: A

Suggested Approach

Robert E. Chapman, Saul I. Gass and James J. Filliben

U.S. DEPARTMENT OF COMMERCE

National Institute of Standards and Technology

(Formerly National Bureau of Standards)

Center for Computing and Applied Mathematics

National Engineering Laboratory

Gaithersburg, MD 20899

and

Carl M. Harris

Department of Operations Research

and Applied Statistics

George Mason University

Fairfax, Va 22030

March 1 989

Sponsored by:

Federal Emergency Management Agency

500 C Street, SW

Washington, DC 20472

and

U.S. DEPARTMENT OF COMMERCE

National Institute of Standards and Technology

Center for Computing and applied Mathematics

National Engineering Laboratory

Gaithersburg, MD 20899

NISTIR 88-3826

Evaluating Emergency Management

Models and Data Bases: A

Suggested Approach

Robert E. Chapman, Saul I. Gass and James J. Filliben

U.S. DEPARTMENT OF COMMERCE

National Institute of Standards and Technology

(Formerly National Bureau of Standards)

Center for Computing and Applied Mathematics

National Engineering Laboratory

Gaithersburg, MD 20899

and

Carl M. Harris

Department of Operations Research

and Applied Statistics

George Mason University

Fairfax, Va 22030

March 1 989

National Bureau of Standards became the

National Institute of Standards and Technology

on August 23, 1988, when the Omnibus Trade and

Competitiveness Act was signed. NIST retains

all NBS functions. Its new programs will encourage

improved use of technology by U.S. industry.

Sponsored by:

Federal Emergency Management Agency

500 C Street, SW

Washington, DC 20472

and

U.S. DEPARTMENT OF COMMERCE

National Institute of Standards and Technology

Center for Computing and Applied Mathematics

National Engineering Laboratory

Gaithersburg, MD 20899

U.S. DEPARTMENT OF COMMERCE

Robert A. Mosbacher, Secretary

Ernest Ambler, Acting Under Secretary

for Technology

NATIONAL INSTITUTE OF STANDARDS

AND TECHNOLOGY

Raymond G. Kammer, Acting Director

A NOTE TO THE READER

On August 23, 1988, the Omnibus Trade and Competitiveness Act (Public Law

100-418) was signed into law. The Act created the National Institute of

Standards and Technology (NIST) from the National Bureau of Standards (NBS).

Since 1901, NBS has served as the Nation’s central laboratory for developing

and disseminating measurement standards and scientific data.

Central to the competitiveness portion of the Act was the expanded mission of

NIST. The Institute’s new responsibilities augment and build on the technical

expertise of NBS, the only Federal laboratory with a specific mission to

support U.S. industry.

NIST will maintain the traditional functions of NBS and will continue to offer

the full array of measurement and quality assurance services that were

provided by NBS. These services include calibration services, standard

reference materials, standard reference data, and measurement assurance

p rograms

.

iii
PREFACE

The Center for Computing and Applied Mathematics, National Engineering

Laboratory, National Institute of Standards and Technology (NIST) is

conducting a research effort under the sponsorship of the Federal Emergency

Management Agency (FEMA) to develop a set of guidelines which promotes both a critical review and unbiased evaluation of FEMA models and data bases.

Large-scale models and data bases are key informational resources for

FEMA. The type and nature of FEMA models and data bases, however, are perhaps

unique within the Federal government in that they are often concerned with

severe changes or disruptions ranging from limited effects to extremes. In

order to carry out its emergency missions, it is necessary for FEMA to

determine which models, modeling techniques and data bases are appropriate for

what purposes and which ones need modification, updating and maintenance. The

development of evaluation guidelines is therefore of direct benefit to FEMA in

discharging its emergency management duties.

FEMA's Dynamic General Equilibrium Model (DGEM) was subjected to a critical review and evaluation of its capabilities as a military mobilization

model to test and illustrate the approach recommended for evaluating emergency

management models and data bases. DGEM is ideally suited for this purpose

because it was designed to deal with economic and policy impacts of energy and

resource supply (disruptions), catastrophic losses of resources and recovery

from enemy attack, and demand and production surges and resource constraints

during a military mobilization. It is hoped that these evaluation guidelines

and their application to DGEM will prove useful to emergency managers and

decision makers both in the application of these guidelines to other emergency

management models and data bases and in their understanding and appreciation

of the capabilities provided in DGEM.

iv
EXECUTIVE SUMMARY

The purpose of this report is twofold. First, it is designed to provide

the reader with a generic set of guidelines which can be used to evaluate

large-scale, computer-based models and data bases. Second, the guidelines are

illustrated through a critical evaluation of the Dynamic General Equilibrium

Model (DGEM). DGEM is currently being used by the Federal Emergency

Management Agency (FEMA) to analyze a variety of emergency management

problems. The evaluation of DGEM serves both as a step-by-step procedure for

conducting an in-depth model evaluation and as an introduction to a non-proprietary model which has broad applicability to the analysis of

macroecomic issues.

Many of the models currently in use within government and the private

sector are by definition complex and large-scale. Consequently, the ability

to produce accurate and realistic estimates of alternative courses of action

has become a most challenging task. Furthermore, many models are dependent on

data bases that are generated and maintained by diverse organizations, with

each data base having a raison d’etre that may or may not be relevant to the

problem under review. It should then be clear that the acceptance of model

outputs as inputs to the decision-making process without investigating the

validity of their generation is a luxary that the community of model users can

no longer afford.

Over the past 15 years a body of research has been developed that

addresses the concern of model users who, upon being presented with the

results of a computer-based model analysis, must decide whether the results

are accurate enough or appropriate for the problem under review. This

research - termed model evaluation - has contributed greatly to our ability to

understand better the role of modeling in policy-oriented decision making.

Model evaluation may take on a variety of forms and levels of complexity.

However, any evaluation procedure has at least the following essential phases:

1. Obtain a clear and comprehensive statement of user requirements and

objectives pertaining to the application of the model;

2. Generate appropriate information about the model design and performance pertaining to user requirements; and

3 . Evaluate model attributes and properties according to predetermined

criteria of performance required by the user.

These phases are characterized by the following activities:

Determine user requirements and objectives.

0 Based on user requirements, develop questions to be answered about

model performance and identify problems to be resolved or analyses to

be performed.

v
0 Ascertain from the questions any problems for which specialized techniques are to be used to generate information needed about model

design or performance relevant to the intended application of the

model by the user.

° Perform the necessary tasks to generate the needed information for

the evaluation.

0 Establish criteria for the model evaluation.

0 Conduct a formal evaluation to judge the integrity of the model

relative to its intended use.

The first five chapters of this report focus on the development of a generic evaluation procedure. Additional material on the general topic of

model evaluation is presented in Appendices A, B and C. The remainder of the

report illustrates how the procedure was applied to DGEM

.

DGEM was selected for evaluation primarily for its unique approach to

modeling emergency situations. The availability of documentation was also a major factor. We wish to alert the reader to the importance of documentation

in any model evaluation effort because it is a means through which performance

against user requirements can be measured.

From a review of the DGEM documentation, it became clear that much care

went into the design, development and testing of the model. Furthermore, the

model's documentation is quite complete and should be sufficient to: (1) set

it up on the host system; (2) execute a base-case simulation; (3) interpret

the results of the base-case simulation; and (4) create, run, and interpret

user-specified simulations.

The analysis of the U.S. economy during a military mobilization is used

to illustrate the generic evaluation procedure. This approach was taken for

two reasons. First, a military mobilization represents a major perturbation

about DGEM ' s "business as usual" base-case simulation. Consequently, any

weaknesses of the model (e.g., instabilities, implausible results, etc.)

should be revealed by such a perturbation. Second, a military mobilization

requires an explicit treatment of the joint interactions of several factors

(e.g., international trade, the business cycle, available capacity,

investment, etc.).

The formulation of the mobilization see

DGEM was a two-stage process. In the first,

constructed which made explicit the perturba

values in the DGEM data base. The mobilizat

of reference for how the joint interactions

path of the national economy. In the second

combination subject to an experimental desig

detail cetain patterns which were uncovered

nario used in the evaluation of

a mobilization baseline was

tion about the business as usua

ion baseline also provided a po

of the perturbation affected th

, six key variables were varied

n. This enabled us to explore

in the mobilization baseline.

1int

e in

in

For the case at hand, DGEM was found to produce solutions which were both

realistic and internally consistent from an economic perspective. This is

especially important in light of the results of the structured sensitivity

vi
analysis because the model was sufficiently robust to produce mea

estimtaes of key economic variables even though many of the "inpu

were significantly different from their business as usual values.

DGEM T s analytical approach and detailed reporting format enabled

illustrate how techniques used in a model evaluation can be appli

studies with policy implications.

ningful

t" variables

Finally,

us to

ed to major

Vll
TABLE OF CONTENTS

Page

A NOTE TO THE READER iii

PREFACE . . . iv

EXECUTIVE SUMMARY v

LIST OF FIGURES ..... x

LIST OF TABLES xi

ACKNOWLEDGEMENTS xiii

1. INTRODUCTION...... 1

i . 1 Background 1

1 . 2 Purpose 3

1.3 Scope and Approach....... 3

1.4 A Primer on Emergency Management: Problems, Policies

and Models ....................... 6

PART I GUIDELINES FOR EVALUATING EMERGENCY MANAGEMENT

MODELS AND DATA BASES 13

2. DESCRIPTIVE ANALYSIS 14

2.1 Motivation for the Model 14

2.2 Organization of the Subject.... 15

2.3 Theoretical Underpinnings 16

2.4 Model Development Context 19

3. PROGRAM VERIFICATION AND ANALYSIS 20

3.1 In-Depth Review of Documentation 20

3.2 Review of Source Code 23

3.3 Implementation of the Model on the Host System 24

3.4 Documentation of Evaluator's Hands-On Experiences 25

4. DATA AUDIT 27

4.1 In-Depth Review of Documentation.. 27

4.2 Analysis of File Computerization and Accessibility 29

4.3 Specialized Techniques: An Overview... 30

5. SENSITIVITY ANALYSIS. 33

5.1 A Note on Methodology....... 33

5.2 The Method of Model Sampling... 36

viii
PART II SELECTED RESULTS FROM AN EVALUATION OF THE

DYNAMIC GENERAL EQUILIBRIUM MODEL ( DGEM ) 40

6. ESTABLISHING USER REQUIREMENTS FOR A MOBILIZATION APPLICATION 41

7. THE DGEM ANALYSIS FRAMEWORK . 44

7.1 The DGEM Documentation 44

7.2 Software Considerations 49

7.2.1 Code Review and Modification 49

7.2.2 The Mobilization TUTOR 61

7.3 DGEM From the User’s Point of View 64

8. THE DGEM DATA BASE 68

9. THE MOBILIZATION SCENARIO 77

9.1 Specification of the Mobilization Baseline and the Structured

Sensitivity Analysis 77

9.2 Impacts of the Mobilization Scenario on the

National Economy 83

9.2.1 Analysis of the Mobilization Baseline 83

9.2.2 Results of the Structured Sensitivity Analysis 85

10. SUMMARY OF MAJOR FINDINGS 116

REFERENCES 126

APPENDIX A CRITICAL STAGES IN THE DEVELOPMENT OF LARGE-SCALE

COMPUTER-BASED SYSTEMS 128

APPENDIX B SPECIALIZED TECHNIQUES FOR PERFORMING A DATA AUDIT 130

APPENDIX C THE EVALUATION OF MULTI-MODEL SYSTEMS 137

APPENDIX D THE POTENTIAL ROLE OF DGEM IN AN INTEGRATED

SYSTEM FOR MILITARY MOBILIZATION 140

APPENDIX E DICTIONARY OF TERMS USED IN DGEM 157

E.l Endogenous Variables 157

E.2 Exogenous Variables 160

APPENDIX F DESCRIPTION OF SUBROUTINES 163

IX
LIST OF FIGURES

1.1 A Suggested Approach for Evaluating Emergency Management

Models and Data Bases 5

1.2 Conceptual Framework for Emergency Mobilization Preparedness 11

2.1 Interrelationships Between Model Evaluation Act ivi ties ...........

.

18

7.1 Flowchart of the DGEM Computer Program............................ 51

% 9.1 Values of Real GNP for the Base-Case Simulation, the

Mobilization BAseline and the Mobilization Maxima 87

9.2 Box Plots of Mobilzation Values for Real GNP........ 88

9.3 Box Plots of Base-Case Deviations for Real GNP 91

9.4 Box Plots of Annual Rates of Change for Real GNP... 92

9.5 Values of Real PCE for the Base-Case Simulation, the

Mobilization Baseline and the Mobilization Maxima 94

9.6 Box Plots of Mobilization Values for Real PCE 95

9.7 Box Plots of Base-Case Deviations for Real PCE.. 97

9.8 Box Plots of Annual Rates of Change for Real PCE 98

9.9 Values of Real Investment for the Base-Case Simulation,

the Mobilization Baseline and the Mobilization Maxima 100

9.10 Box Plots of Mobilization Values for Real Investment 101

9.11 Box Plots of Base-Case Deviations for Real Investment 103

9.12 Box Plots of Annual Rates of Change for Real Investment 104

9.13 Values of Capital Services for the Base-Case Simulation,

the Mobilization Baseline and the Mobilization Maxima 105

9.14 Box Plots of Mobilization Values for Capital Services 106

9.15 Box Plots of Base-Case Deviations for Capital Services 108

9.16 Box Plots of Annual Rates of Change for Capital Services 109

9.17 Values of Total Energy for the Base-Case Simulation, the

Mobilization Baseline and the Mobilization Maxima 112

x
9.18 Box Plots of Mobilization Values for Total Energy 113

9.19 Box Plots of Base-Case Deviations for Total Energy 114

9.20 Box Plots of Annual Rates of Change for Total Energy 115

D.l Scenario Development and Analysis 142

D.2 Analysis of Production Requirements 146

D.3 Analysis of Capacity..... 147

LIST OF TABLES

1.1 General Measures Used as a Function of the Policy Issue....,,,,,... 9

1.2 Mobilization Categories........ 10

5.1 A Typical Randomly Generated Latin Hypercube Sampling Scheme 39

7.1 The DGEM Sectoring Scheme..................... 47

7.2 DGEM COMMON Blocks 57

7.3 User/DBUG Options for DGEM. 59

8.1 Goodness-of-Fi t Measures for Three NIPA Components

During the Historical Period..... 70

8.2 Structure of Total Domestic Output... 72

8.3 Average Annual Percentage Growth Rates for Constant

Dollar Variables... 73

8.4 The Business Cycle: Annual Percentage Growth Rate,

1982-2000..../... 74

8.5 Composition of Final Demand as a Percent of GNP

,

1982-2000 74

8.6 Industry Growth Rates..... 76

9.1 Base-Case Values 81

9.2 Mobilization Baseline Values 81

9.3 Extreme Values Used in the Structural Sensitivity Analysis 82

xi
9.4 Summary Statistics for Real GNP 90

9.5 Summary Statistics for Real PCE 96

9.6 Summary Statistics for Real Investment 102

9.7 Summary Statistics for Capital Services 107

9.8 Summary Statistics for Total Energy Ill

xii
ACKNOWLEDGEMENTS

The material presented in this report benefitted from numerous dialogues;

both the encouragement and technical discussions throughout the project, as

well as, the critical reviews of the completed manuscript proved invaluable.

Special appreciation is due to Mr. Robert R. Wilson, the FEMA Project Officer,

both for his managerial skills and his understanding of the many technical

issues which were employed to produce a set of guidelines which were

appropriate for emergency management models and data bases.

Drs. Edward A. Hudson and Richard J. Goettle of Dale Jorgenson and

Associates deserve special praise for their cooperation in the evaluation of

the Dynamic General Equilibrium Model (DGEM). As model developers and

principal architects of the current version of DGEM, they exhibited the

highest standards of professionalism during the entire period of review,

modification, and evaluation of their model. Their cooperation proved that

model evaluation can be a two-way street in which all parties, especially the

user community, benefit. Prof. Ernst R. Berndt of MIT provided many valuable

insights. His view of model evaluation was particularly useful in

establishing some of the user requirements given in Chapter 6.

Messrs. E. Lawrence Salkin and 0. Cleveland Laird of FEMA and David K.

Henry and Albert Walderhaug of the Department of Commerce lent their skill and

experience in modeling military mobilizations to the project. The many

discussions with these individuals were crucial to the formulation of the

military mobilization presented in Chapter 9 which was used to evaluate DGEM

and, at the same time, illustrate the guidelines described in Part I of this

report

.

Special appreciation is extended to Mr. A. Jeffrey Shibe of the Center

for Fire Research at the National Institute of Standards and Technology for

his critical review of this report. His comments enabled us to both simplify

and clarify the presentation of several key points. We also wish to

acknowledge the assistance provided by our colleagues, especially Dr. Javier

Bernal and Mr. Robert G. Hendrickson. Dr. Bernal played a major role in the

conversion of DGEM from a machine-dependent version of FORTRAN to the current

version which complies with the ANSI X3. 9-1978 standard. Mr. Hendrickson's

involvement during the early stages of this project was helpful in

establishing the evaluation guidelines described in Part I of this report.

xiii
'

.

.

-
1 . INTRODUCTION

1.1 BACKGROUND

The subject of model and data base evaluation has grown rapidly in

importance during the last 15 years. This trend is due primarily to a shift in

the type of models being developed by members of the operations research and

management science communities. During the period immediately following World

War II, numerous large-scale models were developed and used by management as

aids to decision making. In many cases these models dealt with

production-oriented problems. Model evaluation was thus fairly straightforward,

in that a recommended solution could be implemented and the response measured

against predictions. However, as one moves from engineering-based,

production-oriented problems to the realm of policy analysis, the problem

becomes more complex. No decision maker faced with implementing a major policy

is in a position to control all of the variables to attain the desired response.

Consequently, to compensate for this situation, the types of models used to

provide information to decision makers have become both large and complex.

Although many traditional mathematical tools are used in such models to make the

analysis tractable, the implications of their use must be carefully stated if

the model's results are to be useful, usable and used by the decision maker.

The problems associated with the use of large-scale models in the policy

arena have led a number of researchers to advocate the development of

well-defined sets of model and data base evaluation guidelines. One of the best

known positions, advocated by Saul I. Gass,-*- stresses the role of evaluation as

an integral part of the building and use of any large-scale, policy-oriented

model. Gass provides three reasons for advocating this form of evaluation:

(1) in many cases, the ultimate decision maker is far removed from the

modeling process and needs a basis for deciding when to accept the

model's results;

(2) users of a model developed for others must be able to obtain a clear

statement of the applicability of the model to their problem area;

and

(3) it is difficult to assess the impact of a model's assumptions, data

availability, and other elements on the model's structure and

results without a formal, independent evaluation.-

^Gass, S.I. "Decision-Aiding Models, Validation, Assessment, and Related Issues

for Policy Analysis," Operations Research , Vol. 31, No. 4, 1983, pp . 603-631 .

“It is important to recognize that model evaluation is not model certification.

Certification commonly refers to a guarantee that the model yields outputs and

results that are suitably accurate for a particular application. This is an

unattainable goal for the policy-oriented models that are of concern here.

Evaluation, on the other hand, acknowledges this limitation, and seeks to

improve the model's usefulness by identifying as explicitly as possible, its

strengths, weaknesses, and appropriate uses.

1
Since this report focuses on the subject of model and data base

evaluation, it is first useful to introduce and define a few key terms. The

literature on model and data base evaluation refers to the following key

procedures: (1) documentation; (2) verification; and (3) validation. Since

many authors are writing on this subject area, ^ there is often disagreement on

the exact definition of each term. A definition of each term is therefore

given here in order to promote an understanding of the topics which follow.

Documentation refers to the written description of the model. In

practice a variety of reports comprise the documentation for the model. These

reports include such information as:

(1) precise statements of what the model is supposed to do;

(2) the mathematical/ logical definitions, assumptions, and formulation

of the problem; and

(3) a complete set of operating instructions for the user, including

current inputs, outputs, test cases, options and limitations.

Among the major purposes of documentation are: (1) to promote the

intelligent use of the model; (2) to provide a technical knowledge base; and

(3) to broaden the class of users. Although documentation is not designed for

model evaluation per se , it is the cornerstone of any evaluation effort.

Without documentation a model can, at best, be considered a black box.

In most cases the documentation describes what the model should do; but

may not necessarily describe what it really does. The truth of the previous

statement rests on the recognition that the documentation describes both the

abstract model and the operational model. Since the operational model is

embodied in a set of computer code, referred to hereafter as the program, the

relationship described in the documentation may be imperfect. It is therefore

necessary to ensure that the program:

(1) describes accurately the model as designed;

(2) is properly mechanized on the computer; and

(3) runs as intended.

The process of documenting the relationship between the abstract and

operational model is referred to as verification.

^-For a review of the literature on model and data base evaluation

see: Chapman, R.E., R.G. Hendrickson and S.F. Weber. A Survey of Techniques

for Evaluating Emergency Planning Models and Data Bases . Gaithersburg, MD:

National Bureau of Standards, NBSIR 84-2963, 1984.

?
The third term, validation , refers to the correspondence between the

outputs of the model and the real-world situation it was designed to simulate.

Given the complex nature of policy-oriented problems it is useful to

differentiate between two forms of validation. The first, objective

validation, is primarily concerned with model-subject fidelity and the degree

to which the model represents reality. The second, subjective validation, is

based on the intended use of the model and the extent to which the model

satisfies the needs of the user. For cases in which models are used to assess

the consequences of alternative policies, an objective validation is usually

impossible. This is because constraints on available data and computer

resources may require substantial simplification of the model design, so that

the model focuses only on certain aspects of the situation which are of

crucial importance to the decision maker. When this view is taken, the

relevance of subjective validation to the policy arena becomes evident.

1.2 PURPOSE

The purpose of this document is twofold. First, it describes four major

classes of analytical techniques which are capable of addressing the

quantitative issues of model and data base evaluation. These techniques are

designed to generate information which can be used to test the degree to which

a particular model or data base satisfies a specific set of user requirements.

Second, this document focuses on how user requirements are determined and how

a test against these requirements is performed. This approach is taken to

illustrate how the four classes of analytical techniques provide the

information required to perform an in-depth evaluation.

1.3 SCOPE AND APPROACH

Model evaluation may take a variety of forms and levels of complexity.

Irrespective of the form and level, however, a thorough evaluation procedure

is based on three essential phases:

(1) obtaining a clear and comprehensive statement of user requirements

and objectives pertaining to the application of the model;

(2) generating appropriate information about the model design and

performance pertaining to user requirements; and

(3) evaluating model attributes and properties according to predetermined

criteria of performance required by the user.

The last phase includes an assessment of the model's qualifications as a tool

in the desired policy analysis. These three phases may be formally

characterized by the following activities:

3
(1) determine user requirements and objectives;

(2) based on the statement of user requirements, develop questions to be

answered about model performance and identify problems to be resolved

or analyses to be performed;

(3) ascertain from the questions any problems for which analytical

techniques are to be used to generate information needed about model

design or performance relevant to the intended application of the

model by the user;

(4) perform the necessary tasks to generate the needed information for

the evaluation;

(5) establish criteria for the model evaluation; and

(6) conduct a formal evaluation to. judge the integrity of the model

relative to its intended use.

These activities emphasize the importance of a thorough and comprehensive

statement of user requirements to the process and to the success or failure of

the evaluation. The way in which these activities are integrated into an

evaluation procedure is summarized in Figure 1.1. It is important to note

that the three major components (user requirements, information development

and generation, and evaluation of test results) shown in Figure 1.1, comprise

a framework for the NBS model/data base evaluation guidelines.

This report focuses on the central role played by a comprehensive set of

analytical techniques in the development and generation component of the

evaluation procedure which appears at the center of Figure 1.1. Specific

analytical techniques are described in Part I, Chapters 2 through 5. Part II,

Chapters 6 through 10, serves to relate these techniques to the framework of

structural information and analysis which supports an in-depth model/data base

evaluation

.

Chapter 2, Descriptive Analysis, focuses on the development and

organization of background information about the model, which includes the

motivation for the model creation, its design and theory, and the conditions

under which it was developed. The chapter also develops a framework through

which the real world problem may be related to the conceptual model used to

formulate the mathematical model and the resulting operational model.

Chapter 3, Program Verification and Analysis, treats the importance of

the clarity, completeness, and adequacy of the model’s documentation. The

details are realized from three activities: (1) an in-depth review of

documentation format, structure, and content; (2) a review of the source code;

and (3) an assessment of experiences in using the model.

Chapter 4, Data Audit, discusses the evaluation requirements as they

pertain to: (1) review and clarification of data definitions; (2) analysis of

the nature and structure of the data base; (3) analysis of the file

computerization and accessibility; (4) determination of data sources; and

(5) analysis of the quality of selected, important data by estimating

uncertainties formulated as bounds on the data variability.

4
ch for Evaluating Emergency Management

Figure 1.1 A Suggested Approa

Models and Data Bases

5
Chapter 5, Sensitivity Analysis, describes the role of sensitivity

analysis in model evaluation and its contribution to quantitative and

qualitative information development relevant to the needs of the model

assessment process.

Chapter 6 establishes a set of four user requirements which provide the

means for evaluating DGEM . Requirement 1 focuses on model usage (i.e., the

operational model). Requirement 2 addresses the clarity and completeness of

the documentation for the conceptual and mathematical models. Requirement 3

focuses on the relationships between the National Income and Product Accounts

(NIPA) and the model’s output. Requirement 4 is concerned with a variety of

economic-technical attributes.

Chapter 7 addresses the issues raised in Chapters 2 and 3. Background

informatin, which serves to define the motivation for the model and its

theoretical under pinnings, is presented first. The focus is then shifted to

the DGEM software, including: (1) a review of the DGEM source code; (2) the

implementation of the model on the NBS computer system; (3) enhancements which

were made to the basic code; and (4) a report on the evaluator's hands-on

experiences

.

Chapter 8 focuses on an audit of the DGEM data base. Material is

presented which documents the extensive testing that was performed on the DGEM

data base by its developers (e.g., comparisons with realized values,

assumptions involved in projections up through the year 2000, etc.) as well as

independent studies (e.g., concerns about substitution elasticity estimates,

the treatment of capital aggregation, etc.).

Chapter 9 provides information on the mobilization scenario used in the

evaluation of DGEM. The mobilization scenario analyzed with DGEM is patterned

after an unclassified scenario approved by DoD. Basically, the scenario

indicates a conventional war of three years duration in Europe, the Persian

Gulf and Korea. The period of the mobilization follows that of the

unclassified DoD scenario and is assumed to be 1983 through 1986. The first

year represents a period of buildup prior to the onset of hostilities. For

each year of the mobilization, values for key economic variables were

specified. Once the "mobilization baseline" had been established, a

structured sensitivity analysis was performed.

Chapter 10 provides a summary of major findings. Material is presented

which compares critically information generated in the model evaluation

process against user requirements. This comparison enables us to conclude

that DGEM is both an appropriate and useful model for analyzing the types of

problems anticipated during a military mobilization.

1 . 4 A PRIM ER ON EMERGENCY MANAGEMENT: PROBLEMS, POLICIES AND MODEL

S

The ability of public officials to respond correctly to an emergency

situation is conditioned by their being able to recognize that a particular

type of emergency has occurred, and to initiate predetermined and ad hoc

actions designed to alleviate the situation. Some localities have established

contingency plans in the area of public safety. Many metropolitan areas, and

6
local councils of governments, have centralized the process by which an

emergency is declared and resources positioned for action and response (e.g.,

during a hurricane or blizzard). Some localities have functioning command

centers that enable them to pool, coordinate and allocate emergency resources.

In general, however, the management of the response to an emergency tends to

be reactive instead of proactive. Other aspects of emergency response are:

(1) the need to determine appropriate guidelines for response actions and

allocations of resources under changing conditions; (2) the training of

emergency managers; and (3) the development of computer-based decision aids

for use by emergency managers.

From a national perspective, the Federal Emergency Management Agency

(FEMA) has an important and direct role in ensuring that our citizens and

their property are given the best protection under all emergencies. FEMA’s

major activity areas center around four policy issues. The first, mitigation,

consists of those activities which eliminate or reduce the probability of

occurrence of an emergency. It also includes long-term activities which

reduce the effects of unavoidable emergencies. The second, preparedness,

involves activities by governments, organizations and individuals to develop

plans to save lives and minimize damage. Preparedness measures also seek to

enhance response operations. The third, response, includes activities to

provide warning, population protection, and emergency assistance. They also

seek to reduce the probability of secondary damage and to speed recovery

operations. The fourth, recovery, consists of both short-term and long-term

activities. Short-term recovery returns vital life-support systems to minimum

operating standards. Long-term recovery may continue for a number of years

after a disaster. Their purpose is to return life to normal, or improved

levels

.

Associated with each policy issue is a series of general measures which

cut across all emergencies and a series of emergency-specific measures. The

general measures associated with each policy issue are summarized in Table

1 . 1 .

Emergencies may be classified according to type and size. The types of

emergencies include: (1) natural disasters; (2) technological emergencies;

(3) resource shortages; (4) disorders; and (5) war. The various types of

emergencies can be ranked by magnitude into local, regional, national, and

international emergencies. It should be recognized, however, that there are

variations in individual emergencies and that local emergencies may be of

national or international interest and/or impact.

A framework for mobilizing the resources which may be needed is outlined

in National Security Decision Directive 47 and Executive Order 11490 in order

to address the types of emergencies just mentioned. Seven categories serve to

define the context of emergency mobilization. These categories are described

in Table 1.2.

National Security Decision Directive 47 states that it is the policy of

the United States to have an emergency mobilization capability that will

ensure that government, at all levels, in partnership with the private sector

and the American people, can respond decisively and effectively to any major

national emergency.

7
Executive Order 11490, Assignment of Emergency Preparedness Functions to

Federal Departments and Agencies, requires departments and agencies to prepare

national plans and programs and to attain an appropriate level of readiness

with regard to the functions assigned. It also requires the Director of FEMA

to establish Federal policies for and coordinate all emergency preparedness

activities and functions of the Federal Government, and be responsible for the

preparation of guidance to Federal departments and agencies to assist them in

performing their assigned emergency functions.

The final consideration in emergency mobilization preparedness is the

timing of management actions. Four phases have been established for planning

purposes. In a real emergency the phases may not exist in clearly discernible

stages, but they do provide a useful framework for planning for the full range

of management actions. The relationship between the policy issues, the scale

of the emergency and the time frame is illustrated in Figure 1.2.

There exists a diversity of activities and related problems that FEMA

must address in carrying out its mission. FEMA's activities include

day-to-day data gathering, responding to the immediate needs of an emergency,

and strategic planning and analysis. It is not our purpose to be complete

here, but instead to emphasize the decision making requirements faced by FEMA

and related organizations and individuals.

Central to all of FEMA’s activities is a viable and current information

support system that is directed toward the needs of emergency managers. Such

a system is defined to include not only data and descriptive information, but

also analysis and interpretive programs. The establishment and maintenance of

this support system, although a problem in itself, cannot be accomplished

without recognition of the decision problems and other related problems of

emergency managers. These include the development and implementation of

training procedures for emergency managers at all levels, the requirement of

emergency managers at all levels to be able to find and interpret current

emergency response rules and regulations, the need of FEMA to be able to

evaluate proposed emergency guidelines and legislation, and the basic

management procedural requirements for coordinating multi-regional responses

and allocations of resources. Thus, to be effective, an information support

system must be designed to reflect the needs of these varied problems. In

particular, one must be cognizant of the many confounding and limiting aspects

inherent in the broad spectrum of emergency operations. These include the

varied nature of emergencies and the extremely wide range of possible

responses requiring formal evaluation procedures.

The time taken to respond to an emergency can also be critical. Public

safety organizations (e.g., police, fire, ambulance service) are always on

duty and take pride in their short response times. But for broader emergency

situations, like toxic and explosive spills or hurricanes that require

decision structures and response systems that are basically dormant, the

ability to mount a proper and timely response is difficult. Here, emergency

managers must be able to couple their static personnel and resource base with

sporadic incoming information to make a dynamic set of responses. This can

8
Table 1.1 General Measures Used as a Function of th Policy Issue

Issue Measure

Mitigation Building codes

Zoning and land use management

Compliance and enforcement

Public education

Preparedness Preparedness plans

Emergency exercises/ training

Emergency communications systems

Evacuation plans and training

Response Activate public warning

Mobilize emergency personnel/equipment

Emergency medical assistance

Man emergency operations centers

Recovery Damage insurance/loans and grants

Temporary housing

Disaster unemployment insurance

Economic impact studies

9
Table 1.2 Mobilization Categories

Category Definition

Military Mobilization

Industrial Mobilization

Economic Mobilization

The act of preparing for war or other

emergencies through assembling and organizing

military resources.

The process of marshalling the industrial

sector to produce goods and services required

to support military operations and the needs

of the civil sector during domestic or

national security emergencies.

The process of marshalling the money, credit,

and taxes needed: (1) to finance the management of the emergency; (2) to maintain a stable economy; and (3) to stimulate key

sectors of the economy.

Infrastructure Mobilization The process of marshalling the output of

infrastructure systems to support the entire

mobilization.

Human Resources Mobilization The process of marshalling people to provide

needed labor.

Government Mobilization The process of marshalling resources of

Federal, State and local governments to carry

out the tasks required to manage emergencies

Civil Preparedness Mobilization The process of marshalling resources to

provide protection for the people, industry,

and institutions of the United States against

the effects of the spectrum of emergencies.

10
Figure 1.2 Conceptual Framework for Emergency Mobilization Preparedness

TIME

FRAME

POLICY ISSUES
only be effective if a full range of contingency plans has been developed and

tested by the emergency management team. Such contingency planning is not a priority item at most levels of emergency management. It is costly, it

requires experienced personnel, but it is a tactical activity that must be

placed on a solid base.

The analysis of emergencies and emergency responses is backed up with

some theory on the physics of particular emergencies, but little theory on the

behavioral aspects of most emergencies. There are fire spread models, nuclear

blast and fallout models, network flow models, epidemiological models, some of

which can be of assistance in contingency planning, as well as in an

operational situation. A key question is how available theory can be utilized

in an emergency information support system, especially at the local level.

One is quick to recognize that the major concerns of emergency management

operations, like all complex decision situations, have both technological and

human components. And, as the problem is of national importance and something

must be done to understand better all aspects of emergency management, there

is a danger of rushing to apply technical methodologies. An integrated

emergency information-communications system will be ineffective if the

personnel and behavioral aspects of who is to use it, how will the users be

trained, and what the decision and resource needs of the users are ignored.

From a general emergency perspective, the individuals, and groups that

concern themselves with an emergency (i.e., the stakeholders and actors)

include persons at the Federal, State and local levels, as well as persons in

private sector organizations. Thus, emergency management operations and

supporting systems must be responsive to a wide diversity of needs and

priorities

.

12
PART I

GUIDELINES FOR EVALUATING EMERGENCY

MANAGEMENT MODELS AND

DATA BASES

13
2. DESCRIPTIVE ANALYSIS

A descriptive analysis focuses on providing background information on the

model. Such information includes the motivation for the model, its

organization and theoretical underpinnings, and the conditions under which it

was developed.

As a first step, the descriptive analysis should document the policy

context and the real world conditions for which predictions are desired. This

information should provide a background for evaluating the appropriateness and

relevancy of the model. Information on the model's organization promotes a more

thorough understanding of the model's structure, its inputs and outputs and

general flow of information. A major component of the descriptive analysis is a review and summary of key publications describing the model's objectives,

structure and principal results. The literature review is also helpful in

evaluating the appropriateness of the model's structure for dealing with policy

issues and the plausibility of its results. Finally, it is useful to know the

conditions under which the model was developed. This information is intended to , alert the users or evaluators to special circumstances of development likely to

make the model either more or less suitable for the proposed application,

2.1 MOTIVATION FOR THE MODEL

This portion of the evaluation consists of four components. The first

component focuses on a description of what is to be described by the model and

why. The second discusses how to apply the model to the problem(s) of interest.

The third describes how the various parts of the model fit together. The final

component provides a means for documenting which, if any, previous models may

have influenced the development of the current model.

To facilitate the discussion which follows, we may note that there is a real world problem which is of interest. This problem provides the motivation

for model development, because a model provides a method of analysis. However,

in order to develop a model, it is necessary to make certain assumptions and

simplifications. These notions may be quite general and hence serve only as an

outline of how to build the model; they may be thought of as constituting a conceptual model. The four components of this portion of the evaluation define

the linkage between the real-world problem and the conceptual model.

As a first step we must document what information is available on the

real-world problem which must be analyzed and the policy and/or other options

which may be applied to affect an outcome. This step serves to document the

policy context in which the conceptual model is to operate. At the most basic

level, the conceptual model would focus on one or more of the four policy issues

with which emergency managers are concerned. Specifying the basic policy focus

defines which issues are important and how they relate to the real-world

problem.

The identification of specific types of policies is the objective of the

second component. The way in which specific policies are used to address the

problem of interest provides insight and guidance into how one would apply the

conceptual model. The reviews in this area seek to identify the basic analysis

strategy and restrictions which may affect the model's range of application.

This review provides a basis for identifying those resources which may be used

14
or are required to manage the real-world problem. Special emphasis should be

placed on relating these resources to the seven mobilization categories used by

FEMA.

The third component of the analysis provides the basis for a description of the

overall system. Here the evaluator documents the general decision problem and

outlines how the various parts of the model fit together. For example, the

general problem may fit into one of two broad areas, physical or policy.

Additional refinement is accomplished by identifying the analytical/ technical

approach used in the model. Researchers in applied mathematics usually refer to

the possible approach as either descriptive or prescriptive. In a descriptive

approach, the solution provided by the model aims at explaining what is

happening not why it is happening. In essence, the model provides a snapshot.

In a prescriptive approach, the solution provided by the model prescribes the

best way to utilize existing resources. Such models explain what should be

done; they do not necessarily provide a great deal of insight into what is

happening. An example of a descriptive approach is a discrete event simulation

where certain components in a network are being modeled during an emergency

situation. As the emergency unfolds, certain components of the network function

at reduced levels of performance. Performance degradation may cause bottlenecks

to develop in the network; if serious enough, they could compromise the entire

operation. A simulation model enables the analyst to identify the likely

bottlenecks prior to an actual emergency. The basic design of the network may

be changed to incorporate this information. Although the design of the new

network may be a distinct improvement over the old one, it need not be the

optimal configuration. To design an optimal network it would be necessary to

make use of specialized optimization techniques. Optimization techniques which

are in wide use include linear, non-linear, and dynamic programming.

The final component provides a means for documenting how the model under

analysis may have benefitted from previous research. The objective of this step

is to see how experience gained from other modeling efforts may have led to the

choice of particular algorithms or structural properties. This step serves to

sharpen the focus of the evaluation by highlighting the strengths or weaknesses

of alternative model specifications brought out in prior studies.

2.2 ORGANIZATION OF THE SUBJECT

This portion of the analysis seeks to develop an outline of the conceptual

model. This is done by first cross referencing the purpose of the model with

the basic policy issues (mitigation, preparedness, response and recovery) and

various emergency-specific measures. Once the relevant policy issues and

emergencies have been identified, it is necessary to define the level of detail

of the model’s output. This entails a critical review of the relevant policy

considerations, as well as empirical considerations and mode 1 ing/ sys tern

constraints

.

^-It is possible for some models to be hybrids, having major components within

each of the two areas.

15
There are three additional steps in the analysis. These steps focus on:

(1) software configuration; (2) variable selection; and (3) causal and dynamic ' relationships. Since these steps are closely related to a number of activities

which are carried out under the in-depth review of the documentation, they will

be discussed here only briefly. Readers wishing a more detailed discussion are

referred to Chapter 3.

In developing a configuration for the software, it is necessary to identify

the major elements of the programming approach. This requires the development

of a gross-level flowchart which shows all major modules and their relationships

to one another. The next step is to determine what information is required to

study the problem and why. This would involve both intra- and inter-module

transfers of information. More specifically, the criteria for selecting

variables should be stated. Variables should also be classified by type (e.g.,

exogeneous, endogenous, controlled)* The final step is to identify the

determinants of logic flow and information processing. Since causal and dynamic

relationships may drive calculations both within a model or serve to define a hierarchy among modules, it is important that both the criteria used to specify

these relationships and their effect on the solutions provided (e.g.,

convergence) be discussed as an integral part of the conceptual model.

It should be clear from the previous discussion that the conceptual model

will exert a substantial influence on both the mathematical model and the

operational model. This influence is discussed in some detail in the section

which follows.

2.3 THEORETICAL UNDERPINNINGS

This portion of the evaluation has two components. The first is an

analysis of the economic and other technical foundations of the model. The

second deals with the mathematical characterization of the model, including such

properties of the solution algorithm as uniqueness, convergence, and the

behavior of successive iterations as the solution is approached.

The first step in the analysis includes the process by which the precise

mathematical set of equations is deduced from the conceptual model. An in-depth

analysis is required, including a determination of the theoretical basis and

structure of the model. A study of basic technological and behavioral

assumptions made or implied is also required. The analysis must also consider

whether the level of aggregation of activities is appropriate, whether

activities are insufficiently modeled, or whether activities that have only

marginal significance introduce unnecessary cost and/or complexity.

The mathematical characterization portion of the analysis focuses on

developing a thorough understanding of the theoretical foundations of the

problem; including references to the open literature where appropriate. It

should define the problems solved, describe the mathematical model employed, and

document the computational algorithms and numerical techniques implemented in

the code. For example, cases where locally convergent procedures are used

should be carefully documented. This is because locally convergent procedures

are only designed to find a solution near some initial vector of values used to

start the iteration procedure. It is important here to recognize that local

convergence is being sought and that the initial values of the input vector are

crucial to the practical success of the algorithm.

16
This portion of the analysis should also include a comprehensive

description both of the problem solved and of the data processing functions

performed. The description of the physical theory in terms of a mathematical

model should be reasonably self contained. Sources for the model and the

mathematical formulations should be referenced. Sufficient detail is needed to

judge the suitability of the model for application to a particular situation.

Assumptions should be noted and information given about limitations.

The relationships between the two major components of this activity and the

activities described in Sections 2.1 and 2.2 can best be understood through

reference to Figure 2.1. This figure shows that there are three distinct types

of models which are derived from and related to the real-world problem. These

model types are: (1) the conceptual model; (2) the mathematical model; and (3)

the operational model. As indicated before, the real-world problem and

associated policy issues result in the formulation of a conceptual model. The

purpose of the conceptual model is to describe what is to be studied and why.

The conceptual model may be based in part on a comparative analysis of the

strengths and weaknesses of other models in this problem area or on certain

historical information. The mathematical model makes explicit all of the

assumptions of the conceptual model; it also specifies the data and domain of

definition, statistical properties, and equation structure. The operational

model is the version of the mathematical model which resides on the computer

system; it is the version of the model which is executed on the computer to

address specific policy and/or related issues.

We can now relate the activities of the descriptive analysis portion of the

evaluation to Figure 2.1. The methodology described in Section 2.1, motivation

for the model, is concerned with the relationship of the real-world problem to

the conceptual model. The methodology described in Section 2.2, organization of

the subject, serves to define the conceptual model. The economic/ technical

foundations portion of the analysis documents the correspondence between the

conceptual model and the mathematical model. The mathematical characterization

portion of the analysis serves to define the mathematical model (e.g.,

assumptions, data, algorithms, equation structure). The activities which

comprise the descriptive analysis portion of the evaluation begin at the top of

Figure 2.1, move through the conceptual model to the box labeled mathematical

model

.

Figure 2.1 also relates the techniques used in a descriptive analysis to

the terms verification and validation introduced in Chapter 1. Through

reference to the figure we can see that verification is the process of

documenting the relationship between the mathematical model and the operational

model. Descriptive analysis sets the stage for what follows by outlining the

linkage between the real-world problem, the conceptual model and the

mathematical model. This linkage should be carefully explained in the model's

documentation reports. Both types of validation, subjective and objective, are

shown on the figure. Since the conceptual model is an abstraction of the

real-world problem, subjective validity implies that the model satisfies both

its intended use (as set forth in the conceptual model) and a particular set of

user requirements. The ideal case, objective validity, implies that the model

closely tracks the real-world problem. Both types of validation are shown as

dashed lines to highlight the complexities inherent in "validating" emergency

management models.

17
Figure 2.1 Interrelationships Between Model Evaluation Activities

L 8
2.4 MODEL DEVELOPMENT CONTEXT

The final design of a model is often influenced by the practical

constraints of resources and schedules. It is useful, therefore, to examine the

administrative conditions under which a model was developed and to use this

information to alert the user or evaluator to special features of the model

which impact on its application or its role in a system of models. The

principal constraints which are of interest are resource limitations (project

scheduling and the availability of staff) and operational limitations.

Resource limitations must be evaluated in terras of how they impacted the

development of the model, especially the time available for theoretical

considerations. Resource limitations may result in design freezes, and

concessions, and compromises or trade-offs by the model developers on its

complexity. The development of a knowledge base for a model evaluation should

identify these shortcomings.

Operational limitations which may affect the design of the model include

the computer configuration, software support, program language, portability

(i.e. the ability to use the program on computers having compatible languages),

operational features, turnaround time, and model-to-computer coordination.

The major sources of information about constraints on model development are

from documentation. Ideally, the documentation would include descriptions that

portray the accommodations made by the developers of the model due to resource

or operational constraints. Departures from specifications defined in the

contract for the original development and from the description and execution of

the acceptance test of the model, which was used to prove compliance with

specifications and to test the model's performance and general integrity, should

also be documented. Documentation is usually the main model component that is

impacted by resource limitations.

The approach for assessing the features of model development, as they

relate to the evaluation of the model in the context of user requirements,

consists of extracting from the information sources described above, the

assumptions, compromises, or trade-offs which resulted from resource or

operational constraints, and bringing this information into the evaluation

process as described in the guidelines for the model evaluation procedure.

The information presented in this section touches on all phases of the

software life cycle. Although model evaluation is carried out with regard to a fixed frame of reference, recommendations are available for the planning and

development of new models. One such set of recommendations is summarized in

Appendix A.

19
3. PROGRAM VERIFICATION AND ANALYSIS

This component of the evaluation guidelines focuses on the clarity,

completeness and adequacy of the model's documentation. This involves three

interdependent activities: (1) an in-depth review of the documentation; (2) a review of the source code; and (3) an analysis of experiences in executing the

model

.

The in-depth review of the documentation provides the insights which are

necessary to perform the remaining portions of the evaluation. Beyond the use

of the documentation as a base of technical knowledge is the issue of

completeness.^ This issue is related to the recommendations for program

documentation specified in the Federal Information Processing Standards (FIPS)

Publication 38 ^ and the NBS Special Publication 500 - 73 .

The next activity is to review the source code itself. Any discrepancies

between the documentation and the source code should be recorded, especially

with regard to their effect on the structure of the program. This review should

use the latest techniques of software analysis and code fault isolation.

The analysis of experiences in executing the model consists of several

activities. First, any problems in setting up the model on the host system are

documented. This step involves interviewing model users and operating system

support personnel. Second, the users' or evaluators' experiences in executing

the benchmark problem^ provided by the model developers are recorded. The

benchmark problem helps the user/evaluator to determine if the input data are

properly communicated throughout the various logical paths of the program.

Finally, it is essential that the evaluators document their hands-on experiences

with the model. This step documents instances where proper input data are used

but errors are introduced through improper data processing. This step also

documents experiences in reproducing published results and the ease with which

special purpose modules can be used or generated.

3 . 1 IN-DEPTH REVIEW OF DOCUMENTATION

It is generally accepted that the evaluator needs more to work with than

the code which defines the operational model. Current consensus among

authorities states that proper documentation is an absolutely essential

ingredient for the acceptability and use of any complex computer model.

*-Gass”7 S.I~ "Documenting a Computer-Based Model," Interfaces , Vol. 14, No. 3,

1984, pp. 84-93.

^Guidelines for Documentation of Computer Programs and Automated Data Systems . Gaithersburg, MD: National Bureau of Standards, FIPS PUB 38, 1976.

^ Computer Model Documentation Guide . Gaithersburg, MD: National Bureau of

Standards, Special Publication 500-73, 1981.

4 A benchmark should be a well-defined problem and solution chosen by the

developer to exercise a large portion of the model’s logic. This problem

should serve as an important vehicle for determining if the model is running

properly on the host system.

20
A substantial body of documentation is required if a model evaluation is to

be effective. Researchers at the National Bureau of Standards and elsewhere

have identified four basic types of documentation reports which are essential

for efficient use of a model. These reports are: (1) Management Summary Manual;

(2) User’s Manual; (3) Programmer’s Manual; and (4) Analyst’s Manual. These

reports are extremely important in any comprehensive model evaluation. A brief

description of each report is therefore given in the text which follows to

promote a more complete understanding of the activities involved in an in-depth

review of the documentation.

The first manual is designed as a management tool. It provides the

information necessary to assess the model’s input requirements (including time,

money, and other resources) and the usefulness of the model's results. The

Management Summary Manual focuses on how the model can facilitate the decision

making process rather than the specifics of how to set up and run the model.

The second manual is designed as a reference document for a nonprogramming

model user. Information contained in this manual is similar to the first but

with increased emphasis on detail. In-depth discussions of the following topics

are included: the model's logical structure; the input data requirements; the

results produced by the model; and the use of the model results.

The third and fourth documents are designed for use by programmers and

analysts, respectively. The third manual provides guidelines for maintaining

and modifying the model. These guidelines should be of sufficient detail to

enable the programmer to understand the operation of the model and to trace

through it for debugging, for making modifications, and for determining if and

how the model can be converted to other computer systems. The fourth manual

differs from the third in that its emphasis is on the model's functional

structure, types of algorithms used, and the techniques employed for model

verification and validation.

The first step in this activity is to review the documentation both for its

completeness and its clarity. This step greatly facilitates the comparison of

the model's purpose and purported capabilities with the user's requirements.

Specific tasks within this activity are a determination of what the model is to

do and its intended range of application. Issues at stake within this part of

the review are the motivation and objectives of the model, its scope and

approach, and its assumptions and restrictions.

The document or documents which describe the theory should detail specific

information about the model being evaluated. The main goal here is to detail

the mathematical methods used and their relationships to the real-world problem.

The algorithms used to obtain numerical solutions from a set of relationships

(e.g., equations) should be described with references to particular algorithms

and numerical techniques provided.

The precision of results obtained by important algorithms and any known

dependence on particular types of computers or operating systems should be

described. For iterative solutions, the use and interpretation of convergence

tests and recommended values of convergence criteria should be included. For

probabilistic solutions, the precision of results should be discussed.

21
A related task is to develop a precise understanding of how the model

works. This entails an analysis of the logical flow of data through the model

from the entry of input data to the generation of the output. Steps should also

be taken to relate the model elements and data flow to the real-world system

(e.g., physical elements and information flows). Although particular process

equations may be described, documentation providing the rationale for selecting

a particular model input or parameter may be missing. Because major

uncertainties can arise from uncertainties in specifying the input data, the

process for selecting the input must be carefully documented and understood.

Although an analysis of the input data is the subject of Chapter 4 (Data

Audit), it is important at the earliest stage to characterize the overall input

data structure and the data media. Specifically, for each input variable one

should give: (1) the variable name; (2) a description and definition; (3) the

source; (4) the procedure through which the data was collected or generated; (3)

dimensional units of the variable; and (6) formats for reading the information.

It is also desirable to provide information on if or how specific data entries

can be updated or, if appropriate, ranges and/or default values which may be set

internally.

The interface between the model and the relevant data files must also be

covered in the documentation. Details which specify the names, usage (input,

output, scratch) and structure of external and temporary data files should be

provided. This includes a discussion of program procedures related to the use

and maintenance of data files.

As the review of the documentation progresses, the evaluator should focus

on the development of a complete technical description of the model with

particular emphasis on its operational details. The documentation reports which

focus on a description of the source program should provide the basis for this

activity. These reports should include: (1) the identification of the source

language (e.g., FORTRAN 77); (2) references to any relevant software standards

(e.g., ANSI X3. 9-1978); and (3) a flow chart showing the overall program

structure and logic, and, where appropriate, detailed flow charts.

Most complex models contain certain programming considerations which are

important for successful implementation. At least four are of sufficient

importance that they should be checked against the existing documentation.

First, the system control commands required to execute the program (including

options) must be clearly explained. Second, the storage allocation and data

management procedures should be described. Special attention should be paid to

any requirements for storage and/or data manipulations which are problem

dependent. In such cases, alternatives which affect data storage and use should

be discussed. Third, the overlay or segmentation scheme for the model should be

documented. Finally, any restart, recovery or successive case capabilities

should be discussed. This issue could be of critial importance where models are

used in the emergency response mode which may entail operation in a hostile

environment. Provisions for graceful degradation of service and manual

operation and/or restart should therefore be carefully explained and related to

the real-world problem.

22
Many of the issues raised in the previous discussion touch upon the topics

of usage and transportability. This part of the documentation review seeks to

describe how the model is to be used and what steps are necessary to run it on

another computer system. Critical factors which affect both the ease with

which a model can be used on one system or set up on another are: (1) a run-stream description; (2) a sample model run; (3) a discussion of options; (4)

an interpretation of the model's output; (5) a guide to troubleshooting when

executing with user-generated data; and (6) a set of provisions for software

exchange. At a minimum, the first four factors should be addressed in the

documentation, if the model is to be either usable or transportable.

The operating system control commands (either as generic statements or as

card images) required to execute the program should be given with an indication

of any interdependencies with either input options or data files. The program

output should be described with relationships shown. Any normalization of

results should be described and associated dimensional units listed. Whenever

the model is transferred from one system to another, the relevant standards

governing information transfer should be followed. This would include

computer-readable material written on magnetic tape which lists as separate

files: (1) the source code; (2) any auxiliary code; (3) a benchmark data set;

and (4) the output from running the benchmark data set.

3.2 REVIEW OF SOURCE CODE

This portion of the evaluation determines whether the operational model

runs as intended by the developer. More precisely, we determine if the source

code checks against the mathematical model described in the documentation

reports. Deficiencies can arise, both in the translation from statement of

purpose to model formulation, and in the translation from the mathematically

stated model to the source code. Determining the equivalence of the two, or

documenting the differences, is an important part of the evaluation process.

The source code is checked carefully against the model specification, as

determined from existing documentation and augmented where necessary with

supplemental information provided by the model evaluator. Critical issues which

are analyzed include: (1) the relationship between assumptions as stated in the

documentation reports and as they are written into the code; (2) the

identification of critical sets of calculations; (3) the identification of

hard-wired parameters or sections of logic which are by-passed; and (4) the

effect of default settings.

The input data must also be shown to be properly communicated throughout

the various logical paths of the computer program. It is possible for the

equations to be properly programmed and for the input to be read correctly, but

for subsequent difficulties to be introduced through code default values or

improper data processing.

It is important to point out that the review of the source code should be

selective. The earlier review of the model's documentation should provide

insights as to where important information is read, transferred or output. It

is these areas where a line-by-line review of the code should be focused. This

review may make use of debugging options which allow for tracing or for checking

the values of arguments in an array.

23
A thorough review of the source code should produce several tangible

benefits. First, it should classify the role and function of the MAIN program

and each subprogram. Second, for each subprogram, it should indicate those

subprograms which call it and, in turn, those subprograms it calls. Third, it

should provide an understanding of how problem variables and constants relate to

program mnemonics. Fourth, it should critically examine the relationship

between the actual model outputs and those described to the users in the

documentation reports. This review may focus on a variety of topics ranging

from a check on formats used to create output, to the effect of options and

print switches on the form and substance of the output. Fifth, it should

identify all shared storage assignments (e.g., COMMON statements in FORTRAN).

Sixth, it should identify the series and the level of the operating system,

language processors, and associated subroutine libraries invoked by the model.

Finally, it should describe known deviations from the manufacturer's supported

software that are required by the model (e.g., local mathematical and utility

routines, and other installation-dependent software).

There is a major benefit to the model user associated with an independent

review of the source code. The evaluator, by cooperating with the model user

and developer, can provide some documentation, if necessary, rather than

pointing out that certain key components are incomplete or missing. Naturally,

the documentation provided by the evaluator is not always fully consistent with

the views of the modeler. However, if the evaluator helps to provide the

necessary documentation about the condition of the model and resists the

temptation to improve or replace specific features without the concurrence of

the modeler, then they make a contribution that outweighs any potential

disadvantage

.

3.3 IMPLEMENTATION OF THE MODEL ON THE HOST SYSTEM

This portion of the evaluation focuses on three activities. The first

activity aims at developing information which documents the model's

operating characteristics. The second activity seeks to determine what special

attributes of the host system were built into the model. The third activity

documents information on the user's perception of the model.

Only the first activity is traditionally included in the documentation for

the model. Unfortunately, the statistics which are generated on run times,

execution costs and core requirements for the host system should be viewed as

machine dependent. ^ However, if the cases which are analyzed and used to

generate these statistics are carefully chosen, they can teach the user a great

deal about the relative cost and efficiency of running the program on the host

system.

In the previous section, we discussed how a critical review of the source

code could help to determine what special attributes of the host system were

built into the model. Another source of information on this subject is the

current set of users on the host system. Interviews with the users may produce

^Unless the model has been run on different hardware/sof tware configurations,

these statistics may provide only a rough idea of what it would be like to run

the model on another system.

24
information on their experiences in replicating the benchmark problem or of

specific tests which they may have run to test their hypotheses about how the

program works. It may also be possible to review memos written while the model

was being built which document time or funding constraints which may have had an

adverse effect on the final product (see Section 2.4).

The last activity seeks to evaluate the user’s perception of the model.

Information collected here is of particular value in determining if the model is

useful, if it is usable on the host system, and if it is being used as a decision-aiding tool. To obtain this information it is necessary for the

evaluator to interview the users of the model. There are several issues which

need to be addressed in order to provide a basis for a more detailed evaluation

of model usability. First, the evaluator must determine how the user’s

understanding of the model affects their perception. For example, casual users

may believe the model is too "data intensive" or "output oriented," whereas more

frequent users may have concerns about how to combine options to address a specific policy issue. Second, whenever someone works with a large-scale model

over an extended period of time, they encounter problems. The source(s) of the

problem(s) may be due to user error or to faults within the program. If it is

possible to document some of the more common problems (either real or

perceived), they may provide valuable information regarding the usability of the

program. Finally, the user’s assessment of the flexibility of the model must be

carefully documented. It is important to qualify how the term flexibility is

being used because there are two notions at work. The first notion of

flexibility deals with the nature of emergency management. Emergencies by

definition are low-probability and/or high-loss events. The model should be

designed so that certain variables (including coefficients) can experience large

changes in value and the model will still converge.-*- The second notion of

flexibility deals with the way in which specific policy issues are addressed.

For example, the model may require the values of certain inputs to be preset.

The question would then be how to relate a specific policy to a particular input

or set of inputs. Is the relationship clear? Would someone else select the

same variable or set of variables?

3.4 DOCUMENTATION OF EVALUATOR’ S HANDS-ON EXPERIENCES

• A thorough evaluation requires hands-on experience with the model in

question to fully understand the requirements for its effective use. Hence,

this task must include the facility and organizational arrangements that permit

access to the model either remotely or by conversion and operation on the

evaluator’s computer system. Regardless of whether the model is run on the

host system or on the evaluator's computer system, it is essential that the

computer code be frozen at some point in time so that the evaluation can be

conducted with respect to a fixed frame of reference.

The first activity which must be documented is the relative ease with which

the model can be used. This includes an estimate of the time it takes to learn

enough about the model to perform a meaningful analysis. The precise

specification of the model, as executed in terms of documentation, data and

control statements, should be given so that processing considerations can be

-*-No assertion is being made that the value to which the model converges is

correct. We only require here that the model produces a solution.

25
evaluated. The experiences of the evaluation team can serve to further document

the model’s operating characteristics. For example, statistics on run time,

execution costs and core requirements can be compared to any which may have been

reported in the documentation reports.

A related topic is the evaluator's experience in obtaining solutions and

replicating published results. The main activity here is to identify those

model attributes which promote or complicate its use by third parties. For

example, in performing a sensitivity analysis, it may be necessary to construct

a coordinated set of runs. If the method of model sampling is used (see Chapter

5), the model is run according to a prespecified experimental design. This may

require special purpose routines. It is therefore necessary to determine

whether such special purpose routines are easily generated. Specific issues

which must be addressed include: (1) modifications to the model's basic logic

flow; and (2) any input and/or output considerations.

A major benefit of the evaluator's hands-on experiences with the model is

an assessment of the adequacy of the model's maintenance plan. Large models

which are used as inputs to policy analysis are constantly being changed. Three

issues are of crucial importance on a model's software maintenance plan; they

are: (1) unscheduled updating; (2) periodic updating; and (3) review.

Unfortunately, unscheduled updating is the most common; it is also the least

conducive to adequate software maintenance. Unscheduled updating occurs when

errors are found in the program which entail code changes. Since the code

changes are often made with a view towards what is happening locally, large

sections of logic in other parts of the program may inadvertently get altered or

turned off. Similarly, new program applications may be identified causing the

model to be pressed into a service for which it was not originally designed. On

the other hand, periodic updating is usually associated with cases in which the

entire data base or key portions of the data base are revised. It is also

possible that empirical relationships will be reestimated, producing new values

for key parameters. These activities may result in a new set of documentation

reports or an abridged version which documents the changes which were made. For

example, all major input-output tables which are based on the Department of

Commerce benchmark are on a five year updating cycle. The fact that model

documentation may be adequate at one point in time does not imply that it will

be adequate at some future date. In order to insure that the software is

maintained properly and the documentation is kept up-to-date, it is necessary to

have a formal review procedure. This procedure should document both the

strengths and weaknesses of the model. The end result of the review is whether

the model should continue in use as is, be modified, or not be used. It should

be clear that once a model has been subjected to a critical evaluation, a procedure which calls for periodic reviews would be a logical followup.

26
4. DATA AUDIT

The confidence which a user or decision maker has in a model, and therefore

its utility, are increased significantly if one can document the sources of the

data. It is also desirable to show that the projected impacts of the input data

uncertainties on the calculated responses of interest are sufficiently small

that they do not invalidate the intended uses of the model. The focus of the

data audit is therefore on the accuracy and internal consistency of the data

base

.

The data audit portion of the evaluation requires: (1) review and

clarification of data definitions; (2) analysis of the nature and structure of

the data base; (3) determination of data sources; (4) analysis of the file

computerization and accessibility and (5) analysis of the quality of selected

important data by estimating uncertainties as bounds.

Specific techniques which should be applied as an integral part of the data

audit are based both on exploratory data analysis and classical statistics. Due

to their interactive graphical analysis capabilities, software packages such as

DATAPLOT^ are especially attractive for performing a data audit. These

techniques are essential to any thorough evaluation because they facilitate the

analysis of descriptive, relational and structural properties of the data base.

This information is particularly useful in analyzing the interactions between a particular model and data base. Special attention should be placed on

identifying and determining the extent of any da ta-imposed constraints on model

formulation.

4.1 IN-DEPTH REVIEW OF DOCUMENTATION

Many of the items associated with an in-depth review of the model's data

files have already been discussed in Section 3.1. Since the model and its data

files are closely related, the information used to perform a data audit are

discussed in all four documentation reports. From a practical standpoint,

however, one would expect that more emphasis would be placed on data issues in

the User's Manual and Programmer's Manual. This is because these reports are

concerned with the everyday running of the model and its maintenance procedures,

respectively.

At the most basic level, these reports should describe the overall input

data structure and the data media (e.g., tape, cards, disks). They should

include a table that shows the input data file names, their media, and any

general data limitations. The table(s) should also describe the

interdependencies, if any, among input data files.

Specific data inputs are normally organized in related groups or as data

records that are entered on a card image. These related groups of data

establish and define a data file and should be described together. The input

data files and the items within each data file should be discussed in the order

of their appearance in the run stream.

^Filliben, J.J. DATAP L OT - I ntroduction and Overview. Gaithersburg, MD:

National Bureau of Standards, Special Publication 667, 1984.

27
Any detailed description of a data file should include four items: (1) the

file's name; ( 2 ) the number of inputs; (3) a list of related data files; and (4)

a description of data records. The first item should give an overview of the

data file's contents and its purpose. The second item should include any

factors which influence the total number of inputs from the data file, as well

as the maximum number of records that may (or must) be used to execute the

model. The third item lists any data files whose contents depend on or dictate

the input values of the file under analysis. The fourth item provides general

comments on the format of the data records followed by a description of each of

the data records.

The preferred approach for summarizing the information on the four items

just described is through the use of a data dictionary. A comprehensive data

dictionary consists of two parts: (1) a dictionary of input variables; and (2) a dictionary of internal variables. Both dictionaries should be arranged in

alphabetical order by variable name.

The dictionary of input variables provides an in-depth summary of all input

terms. Each array dimension should be defined and, where applicable, references

should be made to the effects caused by variable settings in other parts of the

input deck.. If it is necessary to refer to a specific element within an array,

then a labeling convention should be adopted and used throughout the dictionary.

A mnemonic which helps associate the variable with its purpose should also be

given. For example, the key letters in the related name may be capitalized

and/or underlined. The specification through which the variable is read into

the program (e.g., variable type in a FORMAT statement) should then be given.

It is also necessary to describe where within the model the variable is read.

If the variable has a range, the range should be given. If the input data are

constructed through a series of worksheets, then the variable should be

cross-referenced to the appropriate worksheet. A complete definition of the

input variable should then be given; any additional information should also be

noted at this time.

The dictionary of internal variables focuses on those variables which are

used for intermediate calculations and hence are not directly under the control

of the user. Ideally, four types of variables should be included in this

dictionary: (1) any variable which appears in an output report but is not an

input; (2) any variable which appears in a COMMON statement; (3) any variable

which is passed as an argument in a call to a subprogram; and (4) any variable

which appears in a DIMENSION statement but is not otherwise covered. The

discussion in the dictionary of internal variables should parallel that of the

dictionary of input variables. The major purpose of this dictionary, however,

is to provide enough information to enable programmers to make changes to the

source code dictated by either user needs or operating system characteristics.

An important part of model usability is data collection. Explicit

instructions for data collection and maintenance should therefore be included in

the appropriate model documentation reports. These instructions should include

the identification of the parties responsible for data collection and

ma intena nee

.

At the most basic level, the documentation reports should discuss the data

sources for each data file. The discussion should identify the form in which

the raw data are available, and if appropriate, other organizational units from

28
which the data may be collected. Any special statistical or other techniques

used to obtain or generate the data should be carefully explained. A flow chart

that illustrates the major data collection steps and their sequence is highly

desirable. Updating procedures should also be explained. The documentation

reports should provide step-by-step instructions for maintaining the data files.

If other programs are used to update the data files, they should be identified

and instructions for their use given. As before, a flow chart that illustrates

the major updating procedures and their sequence should be included.

4.2 ANALYSIS OF FILE COMPUTERIZATION AND ACCESSIBILITY

A variety of items should be checked to ensure that the data files have

been properly mechanized. Some of the information on these items will be

uncovered during the review of the model’s documentation reports and some will

be uncovered during the review of the source code. Information uncovered in the

documentation review which affects file mechanization should include: (1) an

outline of the general contents and organization of the data files; (2)

references to other programs which create, modify or edit these files; and (3)

the relationship of the data files to the execution of the program. Additional

information governing special input techniques and requirements, the handling of

consecutive cases and the general conventions governing default values should

also be a natural by-product of the documentation review. Information uncovered

in the review of the source code focuses on such items as: (1) the dimensions

of data arrays; (2) the dependence of data storage requirements on problem input

parameters; (3) the restrictions on the range of values of variables; (4) the

program's restart and recovery procedures; and (5) the program's error messages

and their causes.

A critical component in the analysis of file computerization is a description of all data structures internal to the model. This description

should focus on both local and global variables, arrays and data records.

Local data structures should contain the meaning and purpose of all local

variables, arrays and data records (local data structures have their values

defined only within particular routines). To promote a better understanding of

the file, local data structures should be associated with the subprograms in

which they appear.

Global data structures should contain the meaning and purpose of all

global variables, arrays, and data records (global data structures are defined

throughout the model).

The relationship between local data structures, global data structures

and specific data files can be understood better by adopting the following

convention: (1) general data are summarized in tables; (2) organizational

characteristics are summarized in figures; and (3) the contents of a specific

file are listed in exhibits. The tables focus on generic information only

(e.g., variable name, format, purpose, range, etc.). Cross-referencing between

exhibits, figures and tables should be sufficient to enable the programmer

and/or evaluator to understand how the model approaches an application problem.

Global data structures must be made available to the various subprograms of

the model. The most frequently used method is COMMON storage. By using the

COMMON storage accessing method, frequently used data may be referenced by the

29
same mnemonic throughout the program. The variables in the COMMON storage areas

may be the user's input, or program defined data which may be character, integer

or real. No particular ordering is used in assigning variables to the areas.

However, those areas used for user's input are allocated in the same order as

the input records. The documentation reports should contain a description of

COMMON storage areas. These descriptions should present a synopsis of the type

of variables stored in each area as well as a listing of variables in each area.

A series of tables listing the variable name (including dimensions for arrays),

its contents, ranges on values, and data type (e.g., character, real, integer)

should be sufficient for cross-referencing.

Most large-scale models produce two basic types of outputs, normal outputs

and error messages® The normal outputs have been discussed under several

headings in Chapter 3. Error messages were mentioned earlier; they are used to

address the everyday problems of incorrect formatting, recording and sequencing

errors in user-generated data. They are also useful in identifying

inconsistencies within a data file. If the model is at all complex, it should

contain a system for edit-checking the values of key input variables. If an

error is encountered, a message should be printed out to the user to help locate

the source of the error and then correct it. The model's documentation should

thus include a description of the error messages output with suitable

cross-referencing to the input deck and/or data file. The description of the

error message should include: (1) the diagnostic (i.e., the error message

output to the user); (2) the action taken (e.g., program terminates or said

variable is set to its maximum or minimum allowable value); (3) the data output

(e.g., the value read, the maximum value allowed, or the column number of an

array undergoing a transformation); and (4) the remedy (e.g., check the settings

on read loops, check the sequencing of card images, check for missing data).

Care should be taken so that the description offers sufficient detail to enable

the user to locate and correct the error based on the prescription given in the

remedy. Any instances where error messages are suppressed due to settings on

print switches should also be given to alert the user that it may be necessary

to look for second-order effects.

4.3 SPECIALIZED TECHNIQUES: AN OVERVIEW

Specialized techniques for a data audit may focus on two distinct types of

data analysis: (1) internal; and (2) external. In an internal data analysis

quantitative comparisons may be made across records for a given variable, across

variables for a given record, or across time for a given variable. In an

external data analysis quantitative comparisons are made to comparable data in

other data bases or in published documents. The focus of this section is on an

internal data analysis.

An internal data analysis has three basic objectives. The first, and most

basic, is to produce a frozen, archived and documented data base ready for

efficient computer analysis. The reasoning here is the same as in the case of

program verification and analysis, namely, the data audit must be conducted with

respect to a fixed frame of reference. The second objective is to produce a general understanding of the data file(s). This includes: (1) a general

understanding of the key variables; (2) a rough cut at response meaningfulness

in terms of agreement to definition and consistency; and (3) a qualification of

the types of error present. The third objective is a quantification and summary

of the probable errors and/or error patterns present in the data files.

30
A number of researchers have stressed the potential for performing two

fundamentally different forms of quantitative data analysis. These methods are

(1) the exploratory data analysis techniques of Tukey; and (2) classical

statistical techniques.

Exploratory data analysis is concerned with isolating patterns and features

of the data and with revealing them clearly to the analyst. Ideally,

exploratory data analysis provides the first contact with the data, preceding

any firm choice of model for either deterministic or stochastic components.

Exploratory data analysis is particularly useful in uncovering unexpected

departures from familiar models or the assumptions upon which these models rest.

An important element of the exploratory approach is flexibility, both in

tailoring the analysis to the structure of the data and in responding to

patterns that successive steps of the analysis uncover.

Specific techniques in exploratory data analysis are based on four major

themes: (1) resistance; (2) residuals; (3) re-expression; and (4) display.

Resistance is a matter of insensitivity to "misbehavior" in data. More

formally, an analysis or summary is resistant if an arbitrary change in any

small part of the data produces only a small change in the analysis or summary.

For example, in summarizing the central tendency of a sample, the median (the

5Uth percentile) is highly resistant. By contrast, the mean is highly

non-resistant. Consequently, a number of exploratory techniques for structured

forms of data provide resistance because they are based on the median.

Residuals are what remains of the data after a summary or fitted model has

been subtracted out. The attitude of exploratory data analysis is that an

analysis of a set of data is not complete without a careful examination of the

residuals. This emphasis reflects the tendency of resistant analyses to provide

a clear separation between dominant behavior and unusual behavior in the data.

When the bulk of the data follows a consistent pattern, that pattern determines

a resistant fit. The residuals then contain any drastic departures from the

pattern, as well as the customary chance fluctuations. Unusual residuals

suggest a need to check on the circumstances surrounding these observations.

Re-expression involves the question of what scale would help to simplify

the analysis of the data. Exploratory data analysis emphasizes the benefits of

considering, at an early stage, whether the scale in which the data are

originally expressed is satisfactory. The re-expressions most often used in

exploratory data analysis come from the family of functions known as power

transformations together with the logarithmic transformation.

Displays meet the analyst's need to see behavior (of the data, of fits, of

diagnostic measures, and of residuals) and thus to grasp the familiar

regularities

.

Graphical analysis packages such as DATAPLOT incorporate all four themes of

exploratory data analysis. DATAPLOT is a FORTRAN-based , interactive high-level

31
language for data analysis and graphics. There are six reasons why packages

like DATAPLOT are recommended for use in a data audit. First, the use of

graphics takes full advantage of human pattern recognition capabilities.

Second, graphical techniques make use of a minimal number of assumptions, which

has as an important consequence that the conclusions are less likely to be

approach dependent. Third, from a communications point of view, graphics are

generally a much more understandable and efficient way of conveying information

to others than is a set of summary statistics. Fourth, graphics allow the

analyst to see and use all of the data. In forming a summary statistic, this is

generally not true, since information latent in the entire data set is mapped

into a single number. In general, the statistic is only sensitive to one

particular analysis aspect of the data. Fifth, graphics permit the analyst to

check many different aspects of the data simultaneously. Information will

therefore be relayed not only about what is being investigated, but also about

unsuspected anomalies in the data. Finally, graphics complement classical

statistical analysis techniques. This point will become more clear as specific

techniques are described. Three types of exploratory data analysis techniques

which can be performed with DATAPLOT are related to measures of: (1) central

tendency; (2) distributional form; and (3) relational properties.

The use of classical statistical analysis in a data audit serves to focus

on the same broad areas as exploratory techniques, namely, central tendency,

distributional form, and relational properties. Measures of central tendency

usually reported in statistical analysis packages include the mean, the median

and the mode of the sample. Measures of dispersion include the variance and the

standard deviation of the sample. Measures of distributional form include such

statistics as skewness and kurtosis based on the moments of the distribution

function as well as histograms and goodness-of -f it tests. Classical techniques

for analyzing relational properties are correlation and multiple regression.

Box-Jenkins techniques are also of great value in promoting a more complete

understanding of data having a time dimension.

^A detailed description of a wide variety of statistical techniques for

performing a data audit is provided in Appendix B.

32
5. SENSITIVITY ANALYSIS

Sensitivity analysis deals essentially with the response of a model to

perturbations (changes) among its state and data variables. All of the methods

of sensitivity analysis address this underlying relationship. The techniques,

however, differ in the way they analyze perturbations, depending on the nature

of the perturbation and its manifestation in the model formulation and design.

Conventional wisdom dictates that models may only be validated within given

constraints for a specific application. From a different perspective, however,

it may be possible to validate a model in a universal sense if knowledge of the

subject is known beyond scientific doubt or is automatically acceptable as a representation of a process. The techniques used in sensitivity analyses are

themselves models of the distributional behavior of error, and are, therefore,

subject to the same scrutiny afforded other models. On the other hand, the

collective evidence of information gained from the methodology provides a powerful component to model evaluation procedures.

5.1 A NOTE ON METHODOLOGY

Sensitivity analysis is usually essential in developing information

necessary for a comprehensive assessment of model performance. It serves not

only in the generation of information, but contributes to understanding the

model’s competence and its range of application to policy issues.

Since sensitivity analysis deals, in a broad sense, with model response to

some condition or perturbation, an analysis of the response depends on: (1) the

type of model under evaluation; (2) the nature of the condition or perturbation

in effect; and (3) the way the response is measured and described. These three

factors form the basis for a wide variety of analytical techniques which

collectively constitute the methodology of sensitivity analysis.

^For an in-depth review of the theoretical and empirical considerations in the

literature on sensitivity analysis, the interested reader is referred to:

Hendrickson, R.G. A Survey of Sensitivity Analysis Methodology . Gaithersburg,

MD: National Bureau of Standards, NBSIR 84-2814, 1984.

O^Two advanced topics in the theory of sensitivity analysis which have important

applications in the evaluation of policy-oriented models are summarized in:

Harris, C .M . Issues in Sensitivity and Statistical Analysis of Large-Scale,

Computer-Based Models . Gaithersburg, MD: National Bureau of Standards,

NBS GCR 84-466", 1984.

33
In order to evaluate a model it may be necessary to develop information

on model performance in one or more of the following problem areas from which

error or uncertainty may originate: (I) errors associated with the

mathematical formulation of the subject; (2) importance analysis, which is an

assessment of weak or non-contributing data, representations, or levels of

aggregation in the design of the model; (3) statistical properties and bounds

of the model response under stress or test scenarios; (4) stability, the

model's behavior to certain rare or unexpected conditions; and (5) robustness,

the performance of the model when some or all of the assumptions on which the

model design was based are ignored. Each of these areas of potential interest

embrace a variety of forms the problem may take. Consequently, the evaluation

procedures which are invoked to develop performance information in these five

areas are themselves generally grouped along appropriate lines of formal

mathematical and statistical analysis.

The subject of error analysis covers a wide variety of types of errors

which may be encountered during the evaluation of a model or in a database

audit. The variety includes numerical errors in data, significance of values in

data sets, functional error analysis traditionally measured by means and

variances of functions and represented by sensitivity coefficients (partial

derivatives), propagation of errors through estimated computations in

subroutines or algorithms, and sensitivities as measured by Lagrange

multipliers

.

The objective of importance analysis is to identify those variables, or

subsections of a model, which do not appreciably affect the application of the

results of the model. The determination of the relative weakness of variables,

data, or model components is generally a subjective matter to be decided by the

evaluator. Identifying these non-contributing elements, however, may lead to

model simplification, modification of databases, or to the elimination of

unnecessary complexity in the functional representation, of activities.

Statistical analyses are the most widely used methodology of sensitivity

analysis. They include the calculation of mean, variance, confidence or

tolerance limits, regression and polynomial fitting, hypothesis testing,

goodness-of-f i t , covariance, estimates of statistical parameters, and

distribution-free techniques. The information about model performance gained

from statistical computations may be used to analyze non-s ta tionarity

,

distributional properties, variation of output, and other measures of

sensitivity of a stochastic nature.

Model stability encompasses three different manifestations of response:

(1) the relation of domain to range; (2) transition points in model behavior as

a function of parametric trajectories; and (3) the existence of cusps, or

catastrophes, within the model design. If the model is viewed as a mapping of

input data (domain) into an output set (range), then the evaluator is interested

in any anomalies in this operation which introduce abnormal results or mappings

which imply an increase in error or decrease in the confidence of the output.

34
Transition points are defined as those points at which the expected character of

the model behavior undergoes a significnt alteration. They indicate to the

evaluator that the output has taken a new direction, but are regarded to be less

critical than the cusp or catastrophic transition. The transition which is a catastrophe is one in which a critical state variable of the model has suddenly

undergone an abrupt change of value. It usually represents either an adjustment

of the model to stress or it is a rapid minimizing response to unstable

conditions. The points of transition are directly related to values of

parameters which pursue a path in control space (parameter trajectory). The

path, if it moves into an unstable region may produce either smooth transition

of a state variable or it may trigger a cusp or catastrophe response, which is a discontinuous jump in the state variable. Parameter tracking becomes,

therefore, an important activity in the study of model stability.

Robustness in statistics is the attribute associated with a method which is

successful although the underlying assumptions for its application are violated.

In a similar way, a model may be said to be robust if it can be used for

scenarios it was not designed to address, or that it may be used successfully

when its use violates underlying assumptions inherent in its design or

formulation. Errors which arise in this context should be studied directly in

terms of the assumptions themselves, and also in terms of the model behavior to

the direct or indirect impact of these assumptions on the formulation of the

subject

.

If a model displays robustness and can therefore be applied to policy

problems beyond those for which it was originally designed, it is desirable to

determine, as part of the evaluation, its range of application , and to use this

information to establish and understand the model’s relationship and power to

similar models of its own class, and to identify the scope of its multiple

capabilities with respect to models assigned to other classes of performance.

In addition to the numerical accuracy and significance of input data it is

necessary to recognize two additional sources of error: system noise and the

error of observation. If the data are not a time series then the analytical

techniques associated with the classical theory of errors are adequate tools of

evalua t

i

examined

the data

regressi

f ormulat

non-s tat

on. If, however, the data set is a time series, then the errors are

according to the model of the series and the intrinsic properties of

. The range of models for time-series studies is quite large:

on at the lowest level, through polynomial and autoregressive

ions to autoregressive, integrated, moving average (ARMIA) models for

ionary time series.

35
5.2. THE METHOD OF MODEL SAMPLING

Recent work by McKay, Conover and Beckman^ and by Harris^ >3 >4 has produced

techniques which permit the effects of uncertainty to be rigorously analyzed.

Their approach makes use of the method of model sampling. The method of model

sampling is a procedure for sampling from a stochastic process to determine,

through multiple trials, the nature and effects of a probability distribution

that would be difficult or impossible to determine by standard statistical

means. The method of model sampling, or distribution sampling as it is also

called, has a long history of use by statisticians to derive distributions

empirically that are difficult or impossible to derive by other means.

Although the method of model sampling is described as an analytical

technique for performing a sensitivity analysis, it is also one of the most

appropriate tools for generating answers to "what if" type questions for policy

analyses

.

The method of model sampling is ideally suited for carrying out a

structured sensitivity analysis. In this case, information on the probability

distribution for each variable of interest, , F

1 (x^), is available from the

data audit. Uncertainty about the values of the x^ can then be handled by

treating them as random variables. In turn, this information may be used to

make inferences about the probability distribution of the output. The mechanics

of the method of model sampling involves a "numerical" experiment which

characterizes the distribution of the model's output. In essence, the

experiment searches for an unknown transformation, h(X) , of the vector, X, each

of whose components has known probability distributions, F

1 (x ][). Clearly,

several sets of values of X, say X^ , ..., X^, must be selected as successive

sets of inputs in order to determine the probability distribution of the

output

.

^-McKay, M.D., W. J. Conover and R. J. Beckman, "A Comparison of Three Methods

Tor Selecting Values of Input Variables in the Analysis of Output from a Computer Code," Technometrics , Vol . 21, No. 2, 1979, pp. 239-245.

^Harris, C. M. "An Assessment of Climatological Uncertainties Using Monte-Carlo

Analysis," in Selected Assessment Strategies Applied to Short-Term Energy

Models . Gaithersburg, MD: National Bureau of Standards, NBSIR 83-2672, pp

.

30-54, 1983.

^Harris, C. M. Issues in Sensitivity and Statistical Analysis of Large-Scale , Computer-Based Models . Gaithersburg, MD: National Bureau of Standards, NBS

GCR 84-466, 1984.

^Harris, C. M. Computer Generation of Latin Hypercube Sampling Plans . Gaithersburg, MD: National Bureau of Standards, NBS GCR 84-476, 1984.

36
There are a variety of methods available for selecting the values of the

vectors. The three most popular methods are: (1) random sampling; (2)

stratified sampling; and (3) Latin hypercube sampling. Random sampling has

intuitive appeal in that an entire body of statistical literature may be used to

make inferences about the distribution of the output. In a numerical experiment

based on the method of random sampling, the values X]_ , ..., are a random

sample from the individual distributions, F

1

(x-j_). Although this method produces

unbiased estimators of the parameters of the distribution, it does not guarantee

that the observations are reasonably spaced over the full range of the

probability distribution. This problem may be quite serious if low-probability

occurrences (i.e., those in the tails of the distribution) are of importance. In

addition, there is no guarantee that values are not duplicated, which could

waste sampling resources.

Some of the difficulties with the method of random sampling just mentioned

have been used to justify the use of another method, stratified sampling. In

this method, all areas of the sample space S of X are represented by input

values. Let the sample space S of X be partitioned into I disjoint strata, S^,

of size

Pi = P r (X e Si)

where

I

1 Pi “ 1 i=l

Let X|j , j=l, ...ni, be a random sample from stratum Si, where

l H = N

i=l

As was the case when the method of random sampling was used, stratified sampling

produces an unbiased estimator of the mean of the output distribution. However,

the variance of the output is less when stratified sampling is used than when

random sampling is used. If the probability sizes, Pi , of the strata and the

sample sizes, ni , are chosen so that

n t = PiN

then a proportional allocation is achieved. Tocher^ has shown that if

stratified sampling with proportional allocation is used, then the variance of

the output distribution is reduced further. This implies that any stratified

plan which employs subsampling, ni > 1 , can be improved by further

stratification. This leads to a special case, proportional allocation with one

sample per stratum. As we shall see shortly, there is a relationship between

this case and the third method, Latin hypercube sampling.

•^Tocher , D. The Art of Simulation . Princeton, NJ : D. Van Nostrand, 1963, pp.

106-107

.

37
In stratified sampling the range space S of X can be arbitrarilypartitioned to form strata. In Latin hypercube sampling the partitions are

constructed in a specific manner using partitions of the ranges of each

component of X. Consider the case where k components of X are of interest. For

simplicity, we may assume they are the first k components. Let the ranges of

each of the k components of X be partitioned into N intervals of probability

size 1/N. The Cartesian product of these intervals partitions S into cells

each of probability N

-

^. Each cell can be labeled by a set of k cell

coordinates

Mi = ( m± j

1 s •••» raijk)

where m^j^ is the interval number of component x^ represented in cell i. A

Latin hypercube sample of size N is obtained from a random selection of N of the

cells M ]_ , M^, with the condition that for each jk the set { mi i=l ,N is a permutation of the integers 1, N. The columns of permutations, jk, of the

integers 1, . .., N, must also be created in such a way that there is no

duplication in any row. One random observation is made in each cell. The

method so obtained can therefore be considered a k-dimensional extension of

Latin square sampling.

McKay, et al , were able to show that Latin hypercube sampling schemes

provide the most efficient means for empirically estimating the distribution

function of the output as well as the parameters of the distribution for a

limited number of trials. One additional advantage of Latin hypercube sampling

results when the model’s output is dominated by only a few components of X.

This is because the Latin hypercube sampling method ensures that each of those

components is represented in a fully stratified manner, no matter which

components might turn out to be important.

Whereas the construction of nonrandom Latin hypercube sampling schemes is

not a hard problem, the construction of random Latin hypercube sampling schemes

is a nontrivial exercise. Recent work by Harris has resulted in a procedure for

generating large-scale random Latin hypercube sampling schemes via a computer

code. This code uses as arguments the values of N and k. For example, if the

distribution of each of four random variables was divided into ten equally

probable increments (i.e., N=10, k=4), then the procedure would generate a sampling scheme such as is shown in Table 5.1. For this simple case, ten

simulations would be performed in the "numerical" experiment. The probability

distribution for each variable F^-(xj_), F^(x2 ), F^(x 3 ) and F^(x4 ) is divided into

0.10 increments. Since N is even, the median of each increment is used (i.e.,

the 5th, 15th, ..., 95th percentiles) for the value of the random variable x-j_.

Through reference to Table 5.1 we see that the first simulation would have

variable xy set to the 65th percentile of its distribution, X 2 to the 15th

percentile, X 3 to the 45th percentile, and X4 to the 55th percentile.

38
Table 5.1 A Typical Randomly Generated Latin Hypercube Sampling Scheme

Simulation

Variable

xi x 3 x4

1 7 2 5 6

2 9 6 7 3

3 4 10 3 9

4 8 3 9 1

5 1 4 8 5

6 3 5 10 4

7 10 7 2 8

8 6 8 1 7

9 2 9 4 10

10 5 1 6 2

39
PART II

SELECTED RESULTS FROM AN EVALUATION

OF THE DYNAMIC GENERAL EQUILIBRIUM

MODEL (DGEM

)

40
6 . ESTABLISHING USER REQUIREMENTS FOR A MOBILIZATION APPLICATION!

The Dynamic General Equilibrium Model (DGEM) is an annual model for

analyzing the structure and growth of the U.S. economy. DGEM incorporates a methodology for contingency planning so that quantitative analyses of the impact

of economic policies and disruptions of the U.S. economy may be conducted. DGEM

provides a detailed analysis of supply and demand factors for simulations up

through the year 2000, as well as a framework for determining sector-specific

demands and supplies and relating these developments to other sectors of the

economy. DGEM was developed by Dale W. Jorgenson and Associates; it is an

expanded version of the Long Term Interindustry Transactions Model^ also known

as the Hudson-Jorgenson Model. ^ >4 The model was designed to deal with three

major types of emergency situations: (1) energy-economic interactions

associated with supply interruptions or strategic policies; (2) the economic

impacts of a loss of resources due to an enemy attack on the United States with

particular emphasis on which policies could stimulate recovery; and (3) the

economic impacts of demand surges and resource constraints typical during a period of military mobilization. The focus of the material presented in Part II

of this report is on a military mobilization application.

A military mobilization is a complex series of events which requires

careful specification and in general, a variety of models and databases. ^ One of

the key components of any such analysis is a macroeconomic model. The purpose

of such models in a military mobilization is to specify how the transition from

a peace-time to a war-time economy takes place. Once this information is

available to the analyst, it becomes possible to design and evaluate alternative

policies which should, at least in theory, stimulate or regulate certain sectors

of the economy which could otherwise produce shortfalls or bottlenecks.

The material presented in this chapter has benefited substantially from

information provided by: Prof. Ernst R. Berndt, Massachusetts Institute of

Technology; Mr. David K. Henry, U. S. Department of Commerce; and Messrs. 0.

Cleveland Laird, E. Laurence Salkin, and Robert R. Wilson, Federal Emergency

Management Agency.

O E.A. Hudson and D.W. Jorgenson, The Long Term Interindustry Transactions

Model: A Simulation Model for Energy and Economic Analysis , Washington, DC:

Federal Preparedness Agency, 1979.

^ E.A. Hudson and D.W. Jorgenson, "U.S. Energy Policy and Economic Growth,

1975-2000,” The Bell Journal of Economics and Management Science , Vol. 5,

No. 2, Autumn 1974, pp . 461-514.

^ E.A. Hudson and D.W. Jorgenson, "Assessment and Selection of Models for Energy

and Economic Analysis," in Validation and Assessment Issues of Energy Models , S.I. Gass (Ed.), Gaithersburg, MD: National Bureau of Standards, NBS

Special Publication 569, 1980, pp. 431-444.

5 A multi-model approach to military mobilization is outlined in Appendix D.

41
Due to the importance of macroeconomic models in a mobilization, it is

useful to establish a set of requirements which the model should possess. Four

general requirements are presented in the text which follows. In Chapters 7

through 9, these requirements are expanded to incorporate the type of

information described in Chapters 2 through 5. Chapter 10 concludes with an

assessment of how well DGEM measures up to these requirements.

Requirement 1

The model must have documentation sufficient to enable an analyst to:

(a) set it up on the host system;

(b) execute a base-case simulation;

(c) interpret the results of the base-case simulation; and

(d) create, run, and interpret user-specified simulations.

Requirement 2

The structure of the model should be clearly documented. This requirement

includes information on the model’s:

(a) policy context;

(b) objectives;

(c) exogenous and endogenous variables;

(d) equation structure; and

(e) database(s).

To the extent feasible, the user should be assured that the forecast values

of the base-case simulation have been compared to historically achieved

values. Any major discrepancies should be documented and, if possible,

explained.

Requirement 3

The model must be cross referenced to the National Income and Product

Accounts (NIPA) . This minimum requirement provides a set of control totals,

which may be used to disaggregate the model’s output to address sector specific

issues. Furthermore, it is desirable for the model to provide information on

those endogenous and exogenous variables likely to be of interest to decision

makers. This information may be useful either in gaining insights regarding the

"path" of the economy to a war-time footing or in preparing customized reports.

Requirement 4

The model must incorporate the following economic-technical attributes:

(a) the increasing importance of international trade over the past

decade

;

(b) the business cycle concept;

(c) the changing composition of the Gross National Product (GNP);

(d) the concepts of investment, capital services, depreciation, and

emergency capacity;

42
(e) wage and price variables;

(f) both supply (e.g., capital and labor services available) and

demand (e.g., military requirements) concepts;

(g) an explicit treatment of both fiscal and monetary policy;

(h) dynamic characteristics whereby production and consumption

decisions in one period affect the economy in future periods; and

(i) bridges to or from key variables (e.g., from the NIPA components

to specific sectors as defined by the Commerce Department's

4-digit Standard Industrial Classification (SIC)).

Items (a) through (i) are at the heart of most mobilization modeling

problems. Consequently, it is essential that the model provide a means through

which the user can address each item both individually and in combination. Only

in this manner can the importance of individual items to the overall problem be

measured. For example, a model which does not incorporate business cycles may

be unable to measure the impact on wages and prices due to a mobilization which

•begins in a recession versus one which begins during a period of full

employment. Similarly, defense expenditures are going to exert a different

impact on certain sectors of the national economy than are other government

expenditures. Finally, the model should provide sufficient sectoral detail to

identify areas where in-depth studies may be worthwhile.

43
7. THE DGEM ANALYSIS FRAMEWORK

This chapter addresses the issues discussed in Chapters 2 and 3.

Background information is presented first; this information serves to define the

motivation for the model and its theoretical underpinnings. The focus is then

shifted to the DGEM software, including: (1) a review of the DGEM source code;

(2) the implementation of the model on the NBS computer system; and (3)

enhancements which were made to the basic code. The section concludes with a

report on NBS* experiences, as well as those of several other Federal agencies,

who used the model to reproduce the base-case simulation and then performed

certain mobiliza tion=oriented studies.

7.1 THE DGEM DOCUMENTATION

The material which follows is based on a thorough review of the DGEM

documentation reports. In some cases, the material presented incorporates

information provided to the NBS staff by the model developers. Most of this

information was prompted by the need for NBS to bring the DGEM code into

compliance with the ANSI X3 e 9-1978 standard for FORTRAN 77. Although this

information delves deeper into the documentation than would be required for an

analyst to understand and use the model, it helps to integrate material from

several different documentation reports into an overall picture of how the model

operates

.

From a review of the documentation, it is clear that much care went into

the design, development and testing of the model. The documentation is quite

complete and should be sufficient for an analyst to setup and run the model on

the host system and interpret its output.

The

consists

documentation reports were prepared in two stages,

of a set of reports on:

The first stage

(1) household-*- and producer^ behavior;

(2) comparisons of the DGEM predictions against realized values up

through 1974; 3 and

(3) results from a series of tests on the convergence properties of the

model for extreme changes in certain key variables.^

^ The 36 Sector Model of Household Behavior , Lexington, MA: Data Resources,

Inc . , 1979.

“ The 36 Sector Model of Producer Behavior , Lexington, MA: Data Resources,

Inc., 1979.

^ Simulations of the 36 DGEM Over the Historical Period , Lexington, MA: Data

Resources, Inc., 1980.

^ Test and Sensitivity Simulations of the 36 Sector Dynamic General Equilibrium

M odel , Lexington, MA: Data Resources, Inc., 1980.

44
The second stage consists of a set of reports giving:

(1) explicit instructions for applying the model to a wide variety of

emergency situations;

1

(2) descriptions of key variables, comparisons of how the model performed

outside the estimation period, and details of the base-case projection

through the year 2000 and

(3) a mathematical description of the model, including key equations, and

the solution algorithm.

The approach used in DGEM is based on the application of econometric

modeling to input-output analysis. Where input-output analysis assumes fixed

input-output coefficients at any point in time, DGEM provides for flexible

input-output coefficients induced by price variations in primary inputs which

are associated with economic policies or anticipated contingencies. The

complete model consists of an inter-industry model incorporating the flexible

input-output methodology and a macroeconometric model that integrates demand and

supply conditions for consumption, investment, capital and labor.

The macroeconometric model divides economic activity into four types of

goods and services: the output of consumption and investment goods; and the

inputs of capital and labor. A production function relates the output of

consumption and investment goods to the inputs of capital and labor services,

for a given level of technical efficiency.

The inter-industry model determines inter-industry transactions for 36

domestic sectors (see Table 7.1), the demand for primary inputs, the allocation

of GNP as final demand among the sectors, and the total sector outputs. The

technology of each producing sector is represented by a price possibility

frontier that determines the supply price of output as a function of the prices

of primary and intermediate inputs and the level of technical efficiency.

Each of the 36 sectors in the domestic economy is represented by a submodel

of producer behavior. These submodels are based on the translog price

possibility frontier. This frontier is a function relating the price of output

charged by a sector to the prices that the sector pays for its inputs. Output

price equals average cost, with profit, in the form of return to capital, being

included in this cost. Also, technical change is included in the price

frontiers

.

^ R.J. Goettle and E.A. Hudson, User’s Guide to the 36 DGEM Simulation Model , Cambridge, MA: Dale Jorgenson and Associates, 1984.

O R.J. Goettle and E.A. Hudson, Final Report on the Dynamic General Equilibrium

M odel , Cambridge, MA: Dale Jorgenson and Associates, 1984.

^ R.J. Goettle and E.A. Hudson, 36 DGEM : The Dynamic General Equilibrium

Simulation Model , Cambridge, MA: Dale Jorgenson and Associates, 1984.

45
The household sector is also explicitly analyzed, not only in terms of its

demand for the output of the producing sectors but also in terms of the supply

of labor and the volume of saving. Part of the model is organized within an

interindustry transactions framework. This permits balance and consistency

between input and output patterns to be achieved over all intermediate and final

goods markets. Another part of the model covers the supply of primary inputs,

in particular of capital and labor, the demand for these inputs and the

adjustment of activity patterns so that the input markets are in balance. The

model also covers the growth of the economy over time with explicit attention

given to savings and investment mechanisms and expansion of productive

capacities through increases in capital and labor input and improvements in

technical efficiencies.

The analytical framework in DGEM incorporates several key factors. The

consistency of the framework ensures that the quantity and the value of flows in

each market in the economy are simultaneously in balance. Both price and

quantity aspects of economic activity are explicitly included in the model.

Behavior by producers and consumers is considered in both price and quantity

terms. For producers, the formation of output prices is considered as well as

the selection of those input patterns that are appropriate in the face of these

prevailing input and output prices (i.e., the determination of the input-output

coefficients for each producer as a function of technological information and of

prevailing prices). For the household sector, prices enter the determination of

labor supply, of total consumption expenditure, and of saving. In addition, the

quantities of household demands for produced goods and services are functions of

preference parameters and the prevailing price.

The flow of inputs to and outputs from production is handled within an

interindustry transaction framework. In this framework the transactions are

organized in a matrix with each column representing inputs to an industry and

each row representing sales or output from an industry. Each row corresponds to

supply from a sector; each column represents purchases by a sector. There are

36 producing sectors. There are three further sources of supply: (1) capital

services; (2) labor services; and (3) imports. Also, there are four more

purchasing sectors, the final demand activities. These are: (1) personal

consumption; (2) investment; (3) government purchases; and (4) exports.

To summarize, DGEM is comprised of several components.

1. Submodels of producer behavior, one for each of the 36 domestic

producing sectors.

2. A model of consumer behavior.

3. Balance equations covering physical flows through the interindustry

system equating demand and supply quantities of each good or service

transacted

.

4. Market balance equations equating value of expenditure and receipts

for each good or service transacted.

5. Financial identities aggregating value flows into aggregate income,

financial and economic accounts.

6. Government and rest of the world accounts.

46
Table 7.1 The DGEM Sectoring Scheme

Sector Number Sector Name

1 Agriculture, Forestry and Fisheries

2 Metal Mining

3 Coal Mining

4 Crude Petroleum and Natural Gas

5 Nonmetallic Mining and Quarrying, except Fuel

6 Construction

7 Food and Kindred Products

8 Tobacco Manufactures

9 Textile Mill Products

10 Apparel and Other Fabricated Textile Products

11 Lumber and Wood Products

12 Furniture and Fixtures

13 Paper and Allied Products

14 Printing, Publishing and Allied Industries

15 Chemicals and Allied Products

16 Petroleum Refining

17 Rubber and Miscellaneous Plastic Products

18 Leather and Leather Products

19 Stone, Clay, and Glass Products

20 Primary Metal Industries

21 Fabricated Metal Products

22 Machinery, except Electrical

23 Electrical Machinery

47
Table 7.1 The DGEM Sectoring Scheme (continued)

Sector Number Sector Name

24 Motor Vehicles and Motor Vehicles Equipment

25 Transportation Equipment & Ordnance, except Motor Vehicles

26 Instruments %

27 Miscellaneous Manufacturing Industries

28 Transportation

29 Communications

30 Electric Utilities (including Federal, state, and local

)

31 Gas Utilities

32 Trade

33 Finance, Insurance, and Real Estate

34 Services (including water and sanitary services)

35 Government Enterprises (excluding electric utilities)

36 Miscellaneous

48
7.2 SOFTWARE CONSIDERATIONS

The current version of DGEM is written in FORTRAN 77 and designed for

execution on any mainframe or mini which supports the ANSI X3. 9-1978 standard

for FORTRAN. During the review of the source code, which was provided to NBS by

FEMA, it became obvious that several modifications would be required if the

model were to run on equipment other than the UNIVAC 1100 series of computers,

the original target machine configuration. The review of the source code did,

however, demonstrate that the description of the mathematical model given in the

36 DGElT : The Dynamic General Equilibrium Simulation Model report is an adequate

description of the operational model.

7.2.1 Code Review and Modification

The discussion which follows first focuses on a brief description of the

solution procedure. This discussion parallels that given in the 36 DGEM : The

Dynamic General Equilibrium Simulation Model report. Emphasis is then placed on

documenting the changes in the DGEM source code resulting from its implementation on selected mini and mainframe computer systems. A one page description of

each subroutine, which identifies specific code modifications, is given in

Appendix F.

The basic approach of the DGEM model is to generate a numerical solution

for a set of simultaneous equations. The simulation program provides a framework for this solution and for the handling of the large amounts of data and

information generated. The components of the program are subroutines that

handle data input, subroutines that incorporate the equations of the model,

procedures for numerical solution of equation systems, and subroutines for the

handling of solution information. These are organized into a consistent framework by the MAIN program.

The logical sequence of the simulation is as follows. First, control information such as the simulation period and the nature of any alterations to the

system is inserted. Then, the database is assembled from the data file. This

database comprises the values of the endogenous and exogenous variables over the

simulation period and the values of the coefficients or parameters of the model

system.-*- Any user-specified changes in data or assumptions are also introduced.

Next, the equations of the model are introduced. Then, sequentially, for each

year of the simulation period, the equation system is solved, given the values

of the exogenous variables and the coefficients. For each year's solution, the

numerical values of the exogenous variables, the lagged endogenous variables,

and the coefficients are inserted into the equations. This generates a system

of N equations involving N variables, the N endogenous variables. A numerical

solution algorithm is then used to ascertain these solution values for the

endogenous variables. The values of the endogenous variables are stored for

use in the next year’s simulation, as lagged endogenous variables. Also, the

1 The data file, from which the database for the simulation is developed,

contains information covering the years 1958 through 2000. The post 1982

values in the data file are projections, based on data available in 1983,

which include: (1) trends in real economic growth and inflation; (2) the

business cycle; (3) aggregate sources of growth; (4) trends in final demand;

(5) energy use; and (6) industry developments.

49
solution values are saved for display at the conclusion of the simulation. This

solution sequence is then repeated for each year of the simulation period.

Finally, the solution information for the entire simulation period is displayed

in the form and for those variables requested by the user.

The structure of the DGEM simulation program is illustrated in Figure 6.1.

Figure 6.1 shows the principal information flows in the program and indicates,

in capitals, the principal subroutines involved at each step. The program first

collects control information through subroutines CHROPT and CNTRLR. These

introduce run control options,-*- the initial and final simulation years, whether

the simulation is static or dynamic, ^ the number and specification of endogenous

variable constraints to be imposed on the model, the number and specification of

data changes to be introduced, and a series of print controls. Subroutine DATA

is then called to read the data for the endogenous variables, the exogenous

variables, the coefficients, the values for each constraint variable, and any

adjustments in the exogenous variables. The values of the lagged endogenous

variables for this simulation year are then assembled by subroutine SAVLAG,

drawing on the previous year f s data values (for the first year) of the

endogenous variables. Under some output specifications, actual values of

user~=selected variables will be saved here so that they can be compared, after

solution, with the simulated values of these variables; subroutine SAVOUT is

concerned with this data storage. At this point, all the information is in

place for the initiation of the simulation. As the model equations are solved

numerically, an initial guess of the values of the endogenous variables is

required. Specifically, only N(N>40) endogenous variables are solved for

numerically in the reduced dimension system, so initial guesses on these N basis

variables are required. 3 These guesses are selected by subroutine GUESS,

drawing on the actual values of the endogenous variables. The solution

procedure is now invoked; this is handled by subroutine NEWTM , which begins at

statement label C. The data (exogenous variables, lagged endogenous variables,

and coefficients) are inserted into the model equations, contained in

subroutines FCR, FCRVAL, SHPRA, FCRA, FCRB, and SHPRB. The equations are

evaluated given the current guess of the values of the endogenous variables.

*- Examples of run control options are: (I) output the National Income and

Product Accounts for each year; (J) print a table of energy aggregates for

each year; (M ) create a new data file; and (R) suppress the standard output

whenever reports on special variables are desired.

O^ In a dynamic simulation the solution values for one year form the initial

conditions for the following year.

3 The value of N is at least 40. In this case, the first variable is IVT, the

next 36 variables correspond to the vector PFM(i), the last three variables

are PKD, PLM , and RNW, respectively. In cases where N is greater than 40, the

first 40 variables are as defined above and the basis includes one additional

variable for each equality constraint. The variable corresponding to each

equation is the instrument nominated by the user.

50
Figure 7.1 Flowchart of the DGEM Computer Program

C/D > CO

CE II

J

§

D

o

51
Figure 7.1 Flowchart of the DGEM Computer Progarm (continued)

®

Construct a Basis

Substitute

Basis Variables

into Equations

Form Primary Input

Prices and Estimates

of Output Prices

JL

Solve for Producers

Output Prices

Calculate Input

Costs, input Shares,

Production Costs

and Residuals

NEWTM Solve for the Price of

Consumer Goods

and Services

NEWTMB

\ t FCRB

FCR Calculate SHPRB

FCRVAL

NEWTMA

FCRA

SHPRA

Consumption

Expenditure Patterns

and Residuals

v

Calculate Final

Demands, InputOutput Coefficients,

Real Output and

Transactions

DECSOL

52
Figure 7.1 Flowchart of the DGEM Computer Program (continued)

53
This evaluation is partly a matter of recursive equation solution but also

involves the numerical solution of three blocks of equations (i.e., there are

three separate numerical solutions within the outside numerical solution loop).

One block is solved for the output prices using a Newton’s method procedure in

NEWTMA that, in turn, employs FCRA and SHPRA; the next block involves solving

for the pattern of household sector purchases which uses subroutines NEWTMB,

FCRB, and SHPRB; and the third block involves the interindustry transactions

balances and is solved by the linear equation solver DECSOL. If the equations

hold exactly (i.e., the N residuals corresponding to the basis variables are

simultaneously zero) the current endogenous variable values are solution values

and no further processing is required (i.e., passes out through statement label

E). If the residuals are not zero, the outside loop Newton's method algorithm

commences a systematic adjustment of their values (i.e, returns to statement

label C). After each adjustment, the entire model is re-evaluated and the

residuals computed. When the residuals are simultaneously zero, model solution

has been achieved and the adjustment procedure is halted. The solved values are

saved, by SAVLAG, to provide the lagged values for the next year's simulation.

Also, the solved values are stored, for display or analysis after the simulation

has been completed. Subroutine SAVOUT saves specific information for OUTPUT;

0UU10D organizes and prints blocks of variables. The program now moves to the

next year and repeats the solution process. This continues until the system has

been solved for every year of the specified simulation period. Finally, the

output information requested by the user is displayed from subroutine OUTPUT.

The method of evaluating the equations of DGEM is itself complicated. This

solution is handled by the Newton's method procedure in subroutine NEWTT1 which

calls subroutine FCR to evaluate the equations. The equations are set up as a sequence of equation blocks that can be solved conditionally on the values of N

basis variables. The purpose of FCR is to evaluate N residual equations

corresponding to these basis variables; NEWTM adjusts the values of these

variables until the residuals are simultaneously zero. Within FCR, the entire

system of equations must, be evaluated. FCR inserts the values of the basis

variables into these variables by name then calls subroutine FCRVAL. FCRVAL

calculates the primary input prices and makes initial guesses on the output

prices before calling NEWTMA to solve numerically for the output prices. NEWTMA

calls subroutine FCRA (which, in turn, uses SHPRA) to calculate input costs,

input shares, production costs and, from these, the residuals for the output

prices for flexible share sectors. It also calculates average price residual

for fixed coefficient sector (i.e., sector 36). Once the output prices have

been determined, FCRVAL proceeds to construct initial estimates of the

expenditure weighted average price facing households, PCC. NEWTMB is then used

to numerically solve for this consumption price. The equations used in this

solution are in FCRB and SHPRB and cover the determination of consumption

expenditure patterns. With these consumption prices and consumption patterns,

FCRVAL proceeds to calculate all production and final demand prices, to find all

final demands, and to compute the input-output coefficients. Next, the

input-output system is solved numerically using DECSOL, a linear equation

solver. From the resulting sectoral outputs the entire matrix of real

transactions is calculated. Next, the financial, government and foreign trade

54
variables are calculated. At this point, all equations and variables in the

model have been evaluated except for the N equations corresponding to the basis

variables. The program now returns to FCR where these N equations are

evaluated, yielding the residuals used by NEWTM in solving for the N basis

variables

.

Shortly after our initial review of the DGEM documentation and source code

was completed, NBS installed a new computer system. The new system consists of

a CDC Cyber 205 (622 series) supercomputer coupled with a CDC Cyber 180/855

mainframe computer. The Cyber 855 is used as the front-end system for the

supercomputer. For a brief period during the summer of 1985, both the original

NBS Sperry UNIVAC 1100/82 system and the new Cyber 205-855 were operating

concurrently. This enabled the NBS team to perform extensive testing and code

revisions simultaneously on both machines. Side-by-side comparisons of the

model's output were then performed to catch errors introduced by the code

revisions or to isolate second-order effects. This approach was taken to ensure

that changes in the model's logic were not inadvertently introduced into the

source code. Copies of the revised source code and a sequential version of the

data file were provided to the developers for their review. Several

enhancements were subsequently incorporated by the developers to facilitate the

use of the Mobilization TUTOR described in Section 7.2.2. The discussion

which follows incorporates these enhancements.

The revised code now satisfies the ANSI X3. 9-1978 standard for FORTRAN and

is operational on the NBS Cyber 205-855 system and the Center for Computing and

Applied Mathematics VAX/VMS 11-785 minicomputer. Major revisions to the DCEM

source code were of three types. All three types of revisions were due to the

use of Sperry UNIVAC extensions to the FORTRAN standard. The first set of code

revisions is related to the use of INCLUDE statements throughout the model. The

second set of revisions involved four blocks of subroutines; these blocks are

associated with the following subroutines: (1) DECSOL; (2) FCRA; (3) FCRB; and

(4) GETOUT. The third set of revisions involved character string

manipulations, most notably within the original GETOUT block of subroutines

referred to hereafter as the "parser." Minor revisions to the DGEM source code

included: (1) file definition statements; (2) the initialization of certain key

variables within subroutines; and (3) settings for the variables used to test

for convergence. Brief descriptions of each type of revision are given in

the text which follows. Subroutine-by-subroutine descriptions of code revisions

are given in Appendix F.

The INCLUDE statement is a Sperry UNIVAC extension to the ANSI X3 . 9-1978

standard for FORTRAN; it inserts an externally defined set of FORTRAN statements

into the program being compiled. The Sperry UNIVAC Procedure Definition

Processor (PDP) is used to create a FORTRAN PROC. This is the set of FORTRAN

statements which can be inserted into the source language with an INCLUDE

statement. The PDP accepts the source language statements defining FORTRAN

procedures and builds an element in the user-defined program file. By using

INCLUDE statements, these procedures may be referenced subsequently in a compilation without redefinition. This approach saves both time in

recompilation and space in the program listing.

55
A normal application for INCLUDE procedures is one or more INCLUDE elements

containing a set of data declarations shared between different portions of a

larger user program. In the case of DGEM , INCLUDE statements were used to refer

to named COMMON blocks and the PARAMETER block. This convention prompted a replacement of the INCLUDE statements with explicit COMMON block statements and,

where appropriate, with the PARAMETER block statements. In addition, it was

necessary to replace the FORTRAN procedures with a BLOCK DATA subprogram. Under

the ANSI X3. 9-1978 standard, BLOCK DATA subprograms are used to specify

parameters and to provide initial values for variables and elements in named

COMMON blocks. Brief descriptions of all DGEM named COMMON blocks are given in

Table 7.2.

A subprogram is a program unit that has a FUNCTION, SUBROUTINE, or

BLOCK DATA statement as its first statement. A subprogram whose first statement

is a SUBROUTINE statement is called a subroutine subprogram.

Under the Sperry UNIVAC implementation of FORTRAN, a subroutine subprogram

may be referred to either as external or internal. A subroutine is external if

it appears as the first program unit in the source input, or its SUBROUTINE

statement is immediately proceeded by an END statement. Otherwise the

subroutine is internal. The Sperry UNIVAC implementation of FORTRAN permits an

external subroutine subprogram to contain several internal subroutines (i.e.,

several SUBROUTINE statements). As indicated earlier, there were four cases in

which an external subroutine subprogram contained more than one SUBROUTINE

statement .

1 In the original source code, subroutine OUTPUT contained two FUNCTION

statements. Subroutine OUTPUT was revised in the same way as subroutines

DECSOL, FCRA, FCRB, and GETOUT (see the text which follows). However, the

changes to subroutine OUTPUT were relatively straight forward.

56
Table 7.2 DGEM COMMON Blocks

Name Purpose

B

CHRSET3

CNTRL

COEFF

DIMSIZ 3

‘ ENDOG

EXOG

ICOB

LAG

OPTb

OPTN3

OTPTNM

OUTPTA

OUTPTB

TARINF

TIERH

TIERP

Variable names and coefficient names

Host’s FORTRAN character set

Control Parameters

Coefficients

Dimensions for special output variables

Endogenous variables

Exogenous variables

Cobb-Douglas specifications

Lagged variables

Defines DEBUG

Defines DEBUG

Information on special

variables for which

the user wants reports

Information on user-controlled variables

Indices for the household model

Indices for the producer models

3 New COMMON block

b No longer used

57
Under the Sperry UNIVAC implementation, entities that may be referenced

from both external and internal program units are referred to as global.

Entities which may be referenced only within a particular internal subprogram

are referred to as local (i.e., local to that subprogram). Entities used in an

external program unit can not be referenced by another external program unit

unless they are in COMMON blocks or passed as arguments. Variables, arrays,

parameter variables, and so forth, which are declared or used in an external

program unit can be referenced by any of its internal subprograms.

Consequently, the names of certain entities need not be used as arguments in

calling an internal subroutine.

The code revisions prompted by the Sperry UNIVAC extensions to the FORTRAN

standard were of two basic types. First, each subroutine was made external.

This was done by ensuring that an END statement was the last line of each

subroutine. Second, the calling sequence for each subroutine which was formerly

internal now includes, as explicit arguments, the names of entities which are

referenced within it and which were previously local. For example, the calling

sequence for subroutine OFFSET within subroutine GETOUT was changed from

CALL OFFSET(IVAR)

to

CALL OFFSET( IVAR , SUB 1

, SUB2 , SUB3 , SUB4 , POINT , INREC , KEYVAR , LOCVAR)

because variables SUB1 through LOCVAR were previously treated as local

variables

.

The purpose of the parser is to read the run control information. This

includes the specification of User/DEBUG options (see Table 7.3), the first and

last years of the simulation, whether the simulation is static or dynamic and

the numbers of target instrument pairs, data changes and outputs. The names and

values of the variables are also read.

The need to revise the parser was due to the use of word-length dependent

software designed for execution on Sperry UNIVAC equipment. This software was

very flexible in that it enabled the analyst to operate on bit strings.

Unfortunately, the Sperry UNIVAC operates on a 36-bit word. Thus, word bytes

and OCTAL field codes which operate on Sperry UNIVAC equipment will either not

work or may produce misleading results on other machines. For example, the FLD

function used in subroutine PACK operates on the Fieldata code set rather than

the ASCII code set. Although all Fieldata characters have corresponding

representations in ASCII, characters are four to a word rather than six and will

have different internal representations. A similar problem results from using

INTEGER declarations to store 6-character strings or multiples thereof.

58
Table 7.3 User/DEBUG Options for DGEM

OPTION FUNCTION

B Suppress all Newton's method output.

C In subroutine FCRVAL, suppress the calls to the FCRA and FCRB

subroutines (i.e., compute only those variables listed in FCRVAL).

D Suppress all lagged endogenous variable output.

E Call subroutine OUTMOD after the first time through FCRVAL.

F List the residuals from NEWTMA, the numerical solution for PO.

G List the residuals and the expenditure share from NEWTMB, the

numerical solution for PCC.

H Compute only the financial variables in subroutine FCRVAL, omitting

all other equations and suppressing the calls to the FCRA and FCRB

subroutines

.

I Output National Income and Product Accounts for each year.

J Print table of energy aggregates for each year.

K List PO every time subroutine FCRVAL is called.

L List household prices every time through subroutine FCRVAL.

M Overwrite the file database with the database created in this run.

(Use this option with caution — the previous database will be lost.)

R Suppress standard output whenever NUMOUT is greater than zero.

S Print the interindustry transaction results (specifically, the P, XT,

YF, and ZX matrices in constant and current dollars).

T Print the input-output coefficients and import shares.

U Print all endogenous variables.

V Print all exogenous and lagged variables.

W Print all coefficients.

X Diagnostic messages in subroutine GETOUT.

Z Writes the simulated values of NUMOUT variables to logical unit 12

(the values are written in card image format).

59
To get around the first problem, a new subroutine, CHROPT, was developed.

Subroutine CHROPT initializes an array, FLDTA, in terms of the host

installation’s FORTRAN character set. To get around the second problem, an

explicit CHARACTER statement is used rather than word bytes (i.e., INTEGER

declarations)

.

The minor revisions to the DGEM source code will now be described. The

DGEM data file consists of 19,327 entries for each of 43 years (i.e., 1958

through the year 2000). Consequently, it was necessary to properly define the

file so that individual entries could be retrieved and/or stored. This was

accomplished through the use of OPEN statements in the MAIN program for the

Cyber version and in the DATA and SETDAT subroutines for the VAX version. One

reason for the difference in where the OPEN statements are placed has to do with

whether the operating; system can handle large direct-access files.

Several subroutines in the original DGEM source code used DATA statements

to initialize key variables. For example, in subroutine NIPABR the variables

TEMPX and TEMPM are used to calculate for each year of the simulation the value

of exports and imports, respectively. If the values are initially set to zero

with a DATA statement, then with each subsequent call to subroutine NIPABR they

will contain the cumrnulative values for all previous years of exports and

imports. Consequently, it was necessary to replace all such DATA statements

with explicit statements, such as

TEMPX=0.0

TEMPM =0.0 ,

in each of the affected subroutines.

Based on discussions with the model developers, it was decided to change

the settings for the variables used to test for convergence. These changes were

all within the Newton’s method subroutines (i.e., NEWTM , NEWTMA, NEWTMB); in

each case the stringency of the convergence criterion was relaxed somewhat. The

effect of these changes was to significantly speed up convergence, because fewer

iterations were required, without significantly changing the precision of the

solution values.

Several other changes which should be noted involve the insertion of

explicit COMMENT statements and the addition of two new subroutines. The first

change was performed to replace within line comments with full line comments.

Where appropriate, additional comment lines were inserted to provide greater

detail on what calculations were being performed. The two new subroutines were

required to accommodate the new capabilities provided through the use of the

Mobilization TUTOR. The two new subroutines are CONVT72 and CONVT82, which

convert government purchases and exports to 1972 and 1982 dollars,

respectively.

60
7.2.2 The Mobilization TUTOR

In an actual application, it is necessary for the analyst to first construct

a runstream file. This file defines the years of the simulation, the variables

and their values which require modification, and the types of reports desired.

The runstream file consists of two parts. The first is a set of job

control language to set up the file assignments and initiate execution. The

second is a set of control instructions.

The control instructions consist of two parts. The first part of the

control instructions contains the following information:

1.

The options selected by the analyst for specialized outputs and file

manipulations

;

2.

The period of the simulation;

3. Whether the simulation is static

4. The number of variables to be sea

5. The number of

6. The number of

instrument pa

7. The number of

The second part

information:

variables to be set

endogenous variable

irs) that the analys

reports for variabl

of the control inst

or dynamic;

led under instructions from the analyst;

to specified values by the analyst;

equality constraints (i.e., target/

t is going to specify; and

es that the analyst is going to specify.

ructions contains the following

1. Names of exogenous variables or coefficients to be scaled;

2. Scale factors for each of the named exogenous variables or coefficients;

3. Names of exogenous variables or coefficients to be set;

4. Numerical values, for each year of the simulations, for each of the named

exogenous variables or coefficients to be set equal to;

5. Pairs of variable names, one pair for each endogenous variable to be

targeted

;

6. Numerical values for the targets, for each year of the simulations, for

each of the endogenous variables to be constrained; and

7. Names of variables for which detailed reports are to be generated.

The previous discussion provides an indication of both the complexities and

subtleties inherent in the construction of the runstream file. Consequently, in

order to promote the use of DGEM by a wide class of analysts, an on-line TUTOR

was developed.

61
The TUTOR, through a nested set of menus, prompts the analyst to provide

information which captures the economic issues inherent in major application

areas such as military mobilization. Furthermore, the TUTOR provides direct,

on-line access to the data file. The analyst in constructing the runstream can

therefore examine the base-case values (i.e., both the realized and projected)

contained in the data file to provide guidance in selecting the values of key

variables for the event to be analyzed. In cases where endogenous variables are

to be set to a specified level (i.e., targeted), the TUTOR recommends to the

analyst one or more instruments which will affect the value of the targeted

variable. The TUTOR stores all intermediate results in a scratch file. Once the

analyst is satisfied that answers to all of the major questions have been

provided, the TUTOR constructs and formats a runstream file. The TUTOR-genera ted

runstream file can be used to execute DGEM or further refined by the analyst.

Initially, the TUTOR presents the analyst with several areas or options each

characterizing some aspect of the event. This is the start of the detailed

specification of the event. Indicators of the industry, structure and duration

of the event (e.g., military mobilization) are selected or specified here.

Policy responses are also selected here, as are reports on the economic effects

of the event.

The definition of the event is accomplished through a tree structure. Each

area of economic or policy specification leads, according to which instruction

the analyst provides, to more detailed areas. In this way, the analyst can

provide a comprehensive definition of the event in areas of greatest concern,

while providing only the most general definition in other areas. In other words,

the analyst can control the definition of the event to be analyzed.

There are four broad areas that provide the starting points for the

definition of the event.

1 . Defense and other government purchases

2. Resources and foreign trade.

3. Government policies

.

4. Reports

.

Defense and Other Government Purchases

Government purchases are separated into three types of government activity:

Federal defense; Federal nondefense; and state and local government. In turn,

each type of government activity involves purchases, spread over a number of

supplying industries. For Federal defense purchases, a rudimentary bridge table,

which distributes defense purchases from the 55 DoD procurement categories to the

36 producing sectors, is included in the TUTOR. The analyst can therefore

control defense spending in a variety of ways within this structure. Overall

levels of defense spending can be altered, leaving the composition unchanged, or

the composition and level can be changed. Specific defense procurement

categories can be altered, year by year. Federal nondefense purchases and state

and local government purchases are spread over the 36 producing sectors. Federal

nondefense purchases can also be altered in total or industry by industry. State

and local purchases can be altered in total, although the composition remains the

same

.

62
Resources and Foreign Trade

This area covers critical inputs to production. Generally, labor and

capital are the most important inputs. Labor covers overall population and labor

conditions as well as the availability of labor to specific industries.

Similarly, overall capital stock and effective capital input are covered, as well

as capital availability to specific industries. Foreign trade covers imports and

exports, each of which can be affected by an event.

A detailed list of indicators describing population and labor input is

provided should the analyst choose to go through the labor branches of the

specification tree. Initial population, population growth rates, labor force

participation patterns, work hours and vacation patterns, and labor efficiency

are all included. Also, the analyst can specify how labor is used (i.e., the

allocation of labor between industries and the amount of labor available to

particular industries).

Similar possibilities are included for capital. The initial operational

capital stock, the efficiency or intensity of use of the capital stock, and

depreciation rates describe the general features of capital. Finally, capital

availability by industry can be specified. For example, if it were known that

certain industries had capacity reserves which could be called upon in times of

emergency, these net additions to the capital stock could be modeled explicitly

by the analyst.

Imports can be specified industry by industry and the analyst also has the

choice of tariffs or quantitative controls for limiting imports. Exports can be

altered industry by industry or the overall volume of exports can be changed,

keeping the composition unaltered. Also, there are considerations for the

balance of trade. This can be unconstrained or it can be subject to limits,

reflecting the constraints that usually exist on the balance of trade.

Government Policies

A wide range of policy measures are available to the U.S. government for use

during or after a major event such as military mobilization. Some of these

policies involve purchases of goods and services; some policies control the

allocation and use of available resources, in particular labor and capital; some

policies control production activities, the level of operation of specific

industries; some policies involve monetary and credit conditions, affecting the

level of borrowing and spending; and some policies affect fiscal conditions, such

as government spending, taxation, deficits and borrowing. These types of

policies are included and can be specified by the analyst.

For example, fiscal policy involves government revenues and expenditures,

and their difference, the government deficit. Government revenues, primarily

taxes, reduce the level of private disposable income and spending. Government

purchases are themselves an element of spending and directly add to private

spending in demand for goods and services. The government deficit affects the

capital markets, since the deficit is financed by government borrowing. The

larger the deficit, the more private capital is preempted (i.e., crowded out),

reducing the availability of funds for private spending, and showing up

particularly in its effect on private investment. Each of these policy measures

is available to the government and each can be adjusted by the analyst in

designing the analysis of the event.

63
Reports

The analyst can specify what information to retrieve from the simulation.

While all of the previous considerations referred to defining the event to be

investigated, this area covers the data to be reported after the simulation has

been completed. A range of reports is available and the analyst can select from

these to tailor the computer reports to the types of indicators and analysis

desired

.

The first set of reports covers the Gross National Product accounts. These

can be reported in current dollars, in constant dollars, or in terms of price

indices. They can also be reported in terms of different types of spending

(i.e., consumption, investment, government, exports and imports). At the

aggregate level, the Gross National Product indicators summarize spending,

production and prices.

Government purchases are covered in the next set of reports. Then, there

are reports on labor. These cover population, labor supply, labor prices (or

compensation rates), and labor input by industry. Similarly, for capital, there

are reports on capital stock, capital input, rates of return and capital input by

industry. At the industry level, there are reports on production levels and on

market demands. Prices can be reported at the overall level, by good or service,

and by industry. Interest rates can also be requested. All of these choices are

menu-driven. In addition, there is opportunity for the analyst to request a report on any variable in the model by specifying the name of the variable.

7.3 DGEM FROM THE USER’S POINT OF VIEW

Over the past year, DGEM has been used extensively by the NBS project team.

The model is also currently being used by at least five Federal agencies and four

non-Federal installations. Whereas the work carried out at NBS has encompassed a wide variety of modeling techniques, the activities of the other users have had a narrower focus. In particular, much of the non-NBS work has focused on gaining a better understanding of the model through interactive sessions on the TUTOR.

The installations to which NBS has supplied copies of the DGEM software

system-*- include:

(1) Economic Research Service

U.S. Department of Agriculture;

(2) Office of Business Analysis

U.S. Department of Commerce;

(3) Office of Industrial Base Assessment

U.S. Department of Defense;

(4) Office of Industrial Mobilization Programs

General Research Corporation;

1 Whenever the term DGEM software system is used, it is meant to include the

source code for both DGEM and the TUTOR, the DGEM database, and a set of one or

more test problems and solutions. Software documentation is also provided.

64
(5) Systems Analysis Department

Swedish National Defense Research Institute;

(6) Department of Operations Research and Applied Statistics

George Mason University.

All three of the non-Federal installations were provided with VAX versions

of the model. A brief description of the activities carried out at each of these

installations is given in the text which follows. The three Federal

installations will then be discussed.

The DGEM software system was provided to the General Research Corporation as

a potential candidate for inclusion in a software package being developed for the

Army Material Command (AMC). Since the target delivery system for the AMC

software package was a PC-AT class machine, the role of DGEM was quite limited in

this application.^

The Swedish National Defense Research Institute is currently pursuing a research project on the economic impacts of military mobilization. Their

interest in DGEM is primarily pedagogical because DGEM is calibrated with U.S.

data. However, because DGEM explicitly allows for major swings in both labor

services and international trade, a DGEM-like model calibrated with Swedish data

may provide an analytical tool which is currently unavailable to Swedish defense

planners

.

The activities carried out at George Mason University focused on how to

perform a structured sensitivity analysis with the model. Previous work by one

of the authors (Harris), produced a set of stand alone software for creating

random Latin hypercube sampling schemes. Input from experts on military

mobilization was then used to identify a set of key decision variables which

would strongly affect the path of the economy over the mobilization period.

These variables were then run through the stand alone software package to

establish an experimental design for executing DGEM. In the interim, the DGEM

software system was setup and tested on the George Mason University VAX. The

TUTOR was first used to construct a base-case simulation runstream for the period

of interest; a mobilization baseline (see Section 9.2.1) was then constructed.

Both runstreams were executed and compared to results produced on the NBS

computer systems. Individual runstream files for the structured sensitivity

analysis were then constructed via the VAX editor. This approach was taken

because it was considerably faster to copy and then edit the runstream files to

comply with the experimental design than to create them directly via the TUTOR.

The results of these and other activities are summarized in Chapters 9 and 10.

The three Federal agencies to which NBS provided copies of DGEM are applying

it in two distinct modes of operation. The Economic Research Service is

indicative of the first mode of operation; they view DGEM as a tool which is

complementary to certain key models used by or within their agency. For example,

forecasts produced by DGEM can be compared to those of proprietary macroeconomic

models to identify areas where refinements to the problem under analysis may be

called for. Similarly, the forecast values produced by DGEM may be used as

1 Subsequent work by FEMA has produced a PC-based version of DGEM.
inputs for certain key variables in large-scale models such as the Economic

Research Service's Food and Agriculture Policy Simulator (FAPSIM). The Office of

Business Analysis and the Office of Industrial Base Assessment are indicative of

the second mode of operation; their focus is on major emergencies (e.g., military

mobilization) and their impacts on national security. In this mode, DGEM is part

of a larger system which integrates information from other models and databases.

Although the applications envisioned by the Office of Business Analysis and the

Office of Industrial Base Assessment are in accordance with the stated "analysis

strategy" of DGEM, the degree of detail implied in these applications is much

finer than currently exists within DGEM. This leads one to conclude that if DGEM

is to be better integrated into these "analytical systems" it will first be

necessary to construct several bridge tables which disaggregate the forecast

values of selected DGEM variables to agree with those used elsewhere in the

system.

All three Federal agencies have found the TUTOR to be of great assistance in

understanding how the model operates, how the "scenario" of interest may be

defined, and what to expect from the model in terms of output. Because the

analyst can "callup" the values of key variables via the TUTOR, it provides

significant insights into the assumptions underlying the base-case simulation.

Furthermore, since one of the most complicated parts of the software transfer is

the creation of the DGEM database as a mass storage file from the sequential file

on the tape provided, the relatively straight forward programming style within

the TUTOR facilitates the process of properly defining the structure of the DGEM

database on the host system.

The previous discussion does not reflect the considerable use made of DGEM

by FE^IA, its developer, Dale Jorgenson and Associates (DJA) , and by several other

organizations. DGEM has been used by FEMA to simulate a number of major

emergency situations. One subject which is of great importance to FEMA is that

DGEM is the only macroeconomic model with demonstrated capability to simulate the

U.S. economy following a nuclear attack. In addition, FEMA's experience with

DGEM indicates that there appears to be no identifiable limit to the percentage

increase or decrease in economic activity that DGEM can simulate if handled in

stepwise increments, an attribute thought to be unique among macroeconomic

models. DGEM has been employed extensively by DJA on behalf of FEMA, the

Department of Energy, the Congress and other organizations. DGEM is currently in

use at the Oak Ridge National Laboratory and was used extensively by the Brookhaven National Laboratory in support of the Department of Energy. DGEM was

recently employed as a macroeconomic driver of the Transportation Express Model

used by the Department of Transportation in the training of National Defense

Executive Reservists.

By and large, the NBS experience with DGEM has been quite positive. The

relative completeness of the software documentation simplified our initial

efforts at setting up the model on the original UNIVAC 1100-82 computer system.

The code conversion which followed immediately thereafter and the development of

the TUTOR produced a deeper understanding of the model's capabilities and

limitations. The success in moving between computer systems and being able to

replicate results was also quite encouraging.

66
Based on our work with the model and feedback from other parties, it is

clear that DGEM can produce information which provides significant insights into

a wide-variety of policy issues. However, one should recognize that some of

these advantages may be adversely impacted if nothing is done to remedy two

deficiencies. These deficiencies, as perceived both by the NBS project team and

other informed parties, are the lack of a software maintenance plan and the

highly aggregated nature of the DGEM solution.

The need for a software maintenance plan has become increasingly important

over the past year. This need stems from three interdependent sources;

(1) the development of the TUTOR;

% (2) the incorporation of monetary policy into the model; and

(3) the requirement to revise portions of the database to accommodate the

previously-mentioned enhancements

.

To a certain extent the current "clearinghouse" role of NBS has enabled

users to obtain the most up-to-date information on the model. 'However, this role

needs to be formalized to ensure that all users have the benefit of the latest

software enhancements, as well as revisions to the database and documentation.

The need for increased sectoral detail in DGEM has been recognized by a number of modelers for sometime. Although this is a serious limitation of the

model for some applications, it may cause little or no problem for others. In

areas where a lack of sectoral detail is a problem, it may be possible to employ

bridge tables as a solution. Two areas where this approach appears feasible are:

(1) a bridge between the various military procurement categories and the DGEM

vector of final demands; and (2) a bridge between the DGEM solution and the

sectors defined by the Commerce Department’s 4-digit Standard Industrial

Classification (SIC). Whereas the construction of the first bridge table is

straightforward,-*- the second may require the redefinition of some internal

variables and entail a restructuring of the source code. Both sets of bridge

tables should be studied by a DGEM user group to define their use and assess

their technical merits.

1 Such a bridge table could be incorporated directly into the TUTOR. Data for

constructing the bridge table exists both within the Defense Department and the

Commerce Department (see Appendix D).

67
8 THE DGEM DATA BASE

The DGEM database consists of 19,327 entries for each of 43 years (i.e.,

1958 through the year 2000). Of the 19,327 entries for each year 5,903 are

endogenous variables, 1,643 are exogenous variables, and 11,781 are coefficients.

The estimation of the model over the historical period (i.e., 1958 to 1974) used

a database constructed by Jack Faucett and Associates. ^ The Faucett database

contains annual price and quantity data on 11 types of energy and over 30

intermediate inputs for 35 producing sectors of the U.S. economy over the years

1958 through 1974.

Since work began on DGEM in the mid 1970’s, several studies have examined

the Faucett database. Two studies which are particularly noteworthy for their

thoroughness and methodological approach are due to Berndt, Fuss and Waverman^

and Hazilla and Kopp.^ The latter study provides many insights into the Faucett

database because it examines the nature of factor adjustment, the treatment of

capital aggregation, alternative capital service price formulations, the

aggregation of energy types, the specification of technological change, and the

choice of scale economies. One conclusion from the Hazilla and Kopp study which

is of importance to the DGEM database is that substitution elasticity estimates

are quite sensitive to the capital service price formulation. In a subsequent

review of the Hazilla and Kopp study, Berndt points out that the rate of return

used in DGEM is based on realized profits and taxes, and thus is an ex

post rather than an ex ante rate of return. ^,5 This poses a methodological

problem because the choice of the capital service price formulation confuses ex

ante with ex post costs of capital. Berndt asserts that if DGEM were estimated

using the theoretically appropriate ex ante cost of capital (e.g., the Moody Baa

bond yield which captures average risk), that the new parameter and elasticity

estimates would differ considerably from those now found in the DGEM database.

As a part of the original task of building and testing DGEM, the developers

prepared a number of base-case simulations. Three sets of base-case simulations

are reviewed in this section; they are:

Jack Faucett and Associates, Development of 35 Order Input-Output Tables , 1958-1974, Final Report , Washington, DC: Federal Preparedness Agency, 1977.

^ Berndt, E.R., M. A. Fuss and L. Waverman, Empirical Analysis of Dynamic

Adjustment Models of the Demand for Energy in U.S. Manufacturing Industries , 1 947-1974 , Palo Alto, CA: Electric Power Research Institute, 1980.

^ Hazilla, M., and R. J. Kopp, Industrial Energy Substitution: Econometric

Analysis of U.S. Data, 1958-1974 , Palo Alto, CA: Electric Power Research

Institute, 1984.

^ Berndt, E. R. , "Summary/Critique of Industrial Energy Substitution:

Econometric Analysis of U.S. Data, 1958-1974," mimeo, 1985.

Since the ex post rate of return is used within DGEM as the ex ante cost of

capital, one would conclude that the firm is always in equilibrium.

68
(1) model performance over the historical period (i.e., simulations

between 1959 and 1974);

(2) model performance between 1975 and 1981, when published data on

certain key variables was available; and

(3) the base-case projections (i.e., simulations between 1982 and the

year 2000).

The historical simulations were of two types, static and dynamic. Both

types of simulation were compared to the realized values of personal consumption

expenditures, gross private domestic investment and GNP from the National Income

and Product Accounts (NIPA).^- All comparisons used as a goodness-of-f it measure

the root-mean-squared error (RMSE) as a percent of the actual mean.

The goodness-of-f it measures associated with the three NIPA components for

both the static and dynamic simulations are recorded in Table 8.1. For the

static simulation, the maximum RMSE is 8.4 percent for nominal investment and

11.9 percent for real investment (calculated in 1972 dollars). The maximum and

minimum RMSE figures for the sectoral prices are 14.7 percent for Tobacco

Manufacturers and 2.0 percent for Gas Utilities. The corresponding figures for

sectoral quantities are 49.5 percent for Tobacco Manufacturers and 3.0 percent

for Transportation Equipment and Ordnance, except Motor Vehicles.

For the dynamic simulation, the maximum RMSE is 12.2 percent for nominal

investment and 24.0 percent for real investment. The maximum and minimum RMSE

figures for the sectoral prices are 58.7 percent for Tobacco Manufacturers and

3.7 percent for Primary Metal Industries. The maximum and minimum RMSE for the

sectoral quantities is 455.2 percent for Tobacco Manufacturers and 3.1 percent

for Transportation Equipment and Ordnance, except Motor Vehicles. Other dynamic

simulations were performed within the historical period which did not include the

years prior to I960. These simulations produced significantly better agreement

with published data.

In evaluating DGEM's performance between 1975 and 1981, one must recognize

that while many variables were aligned to observed values, many were not. In

most cases, the simulated values for these compare favorably with information

developed from published sources. The discussion which follows provides several

specific examples from which the reader can gauge the model’s performance.

^ Simulations of the 36 DGEM Over the Historical Period , Lexington, MA: Data

Resources, Inc., 198U.

69
Table 8.1 Goodness-of-Fi t Measures for Three NIPA Components During the

Historical Period-*-

NIPA

Static Dynamic

Component

Nomi nal Real^ Nomi nal Real ^

Personal Consumption 1.8 2.7 6.2 9.2

Investment 'Cf- .

00 11.9 12.2 24.0

GNP 4.0 4.1 3.2 10.7

Between 1975 and 1981, actual inflation exceeded that predicted by the

model. The model also underestimates inflation in export and import prices, but

it accurately tracks the relative price trend which is important for considering

exchange rate movements.

The underestimate of ouput inflation has its origins with projected growth

in input prices. The price of labor received by households, PLB, was used as the

instrument for nominal consumption, CNIA. Simulated growth in PLB averaged 8.3

percent annually from 1974 through 1981. This compares with a 9.0 percent annual

rate for hourly labor compensation in the nonfarm business sector over the same

period. With broadly, though not exactly, similar coverage of the labor market,

the model predicts somewhat slower growth in labor costs which has a corresponding impact on output prices. Growth in capital prices is in close

agreement with past trends.

The model slightly overestimates income growth. Over the 1974 through 1981

period, national income (in current dollars) grew at an average annual rate of

10. b percent. The nominal income variable in DGEM , Y, rose moderately faster, at

an average rate of 11.6 percent annually. This combined with lower inflation

implies more rapid growth in real incomes within DGEM than did actually occur.

Since spending is aligned to observed history, the balance is saved, adding to

private wealth. The mechanisms that promote this additional saving are the

after-tax rates of return on capital and wealth.

-*- All entries are the RMSE as a percent of the actual mean.

^ Calculations are based on 1972 dollars.

70
A final area for comparison concerns economic structure. Periodically, the

U.S. Department of Labor publishes time-series data on domestic output for all

input-output industries. These data can be aggregated to conform to the

interindustry structure represented in DGEM . Table 8.2 compares the Labor and

DGEM data for the years 1974 and 1980. Data from the model for 1974 are as

developed by Jack Faucett and Associates, while the data for 1980 are the results

of simulations. Reference to Table 8.2 shows close agreement in the relative

importance of each industry and the changing nature of that importance over time.

For example, the shift away from construction and the so-called "smokestack”

industries (e.g., primary metals, motor vehicles) and toward communications,

trade, finance, and services is captured in the simulation.

The base-case projection represents a simulated path of economic growth and

structural change from 1982 to the year 2000.-*- The projection reflects both the

assumed values of the exogenous variables and the structure of DGEM itself. The

exogenous variables values reflect best estimates incorporating information

available in 1983. Correspondingly, the base-case projection might be viewed as

a "best estimate" of the economic future, as seen from the perspective of 1983.

The long term trends in real economic growth over the 18-year period are

recorded in Table 8.3. Economic growth is projected to average just over 3

percent a year. This represents continued growth at rates less rapid than during

the World War II to 1973 period but more rapid than during the 1973 to 1983

decade. ^ There are two reasons for this. First, the 1982 starting point was in

a recession, so the 1980's growth reflects both recovery and underlying growth.

Second, the rate of labor growth is considerably less rapid in the 1990 's than in

the 1980's. All principal uses of economic production (e.g., personal

consumption expenditures, investment, and government purchases) show sustained

growth. However, the growth rate for the volume of government purchases is, by

assumption, less rapid than for the economy as a whole, permitting private

spending to grow somewhat more rapidly than GNP.

* The description of the base-case projection follows closely that given in R.J.

Goettle and E.A. Hudson, Final Report on the Dynamic General Equilibrium Model , Cambridge, MA: Dale Jorgenson and Associates, 1984.

The base-case projection assumes a slight acceleration in inflation from recent

rates, averaging 5.3 percent over the 18-year period.

71
Table 8.2 Structure Of Total Domestic Output 1 2

DGEM Sector

Number

1974 1980

Labor DGEM Labor DGEM

1 4.1 4.5 4.5 4.7

2 0.2 0.2 0.1 0.2

3 0.3 0.3 0.3 0.2

4 0.8 0.7 0.7 0.6

5 0.2 0.2 0.2 0.2

6 7.5 7.3 6.5 6.1

7 6.2 6.1 6.2 6.0

8 0.5 0.5 0.4 0.4

9 1.1 1.2 1.2 1.3

. 10 1.7 1.9 1.6 1.7

11 lei 1.1 1.0 1.0

12 0.5 0.6 0.5 0.6

13 1.6 1 .

6

1.5 1.5

14 1.5 1.6 1 .

6

1.7

15 2.6 2.8 2.6 2.8

16 1.7 1 .

6

1.7 1.5

17 1.7 1.9 1.7 1.9

18 0.3 0.3 0.2 0.2

19 1.1 1.1 1.0 1.0

20 3.7 3.6 2.5 2.5

21 2.5 2.8 2.2 2.5

22 4.1 4.2 4.0 4.3

23 3.0 3.2 3.4 3.6

24 3.2 3.3 2.7 2.8

25 2.0 2.1 2.2 2.1

26 0.9 0.9 0.9 1.0

27 0.6 0.7 0 .

6

0.6

28 4.3 4.5 3.8 3.7

29 2.0 2.1 2.8 3.1

30 1.7 1.7 1.9 1.9

31 1.0 0.9 0.8 0.8

32 13.5 11.2 13.7 11.9

33 9.3 9.1 10.7 10.4

34 12.5 13.1 1.0 0.9

35 1.0 1.1 1.0 0.9

1 R.J. Goettle and E.A. Hudson, Final Report on the Dynamic General Equilibrium

Model , Cambridge, MA: Dale Jorgenson and Associates, 1984.

^ Percent of total real output across 35 industries.

72
Table 8.3 Average Annual Percentage Growth Rates for Constant Dollar Variables

Variable

Time Period

1982-1990 1990-2000 1982-2000

Consumption 3.5 3.1 3.3

Investment 3.7 3.9 3.8

Government Purchases 2.0 1.5 1.7

Real GNP 3.2 3.0 3.1

DGEM explicitly recognizes that economic growth tends to be cyclical rather

than steady. Hence, there will be cycles around the growth trends given in Table

8.3. Annual growth rates for the same four variables recorded in Table 8.3,

which incorporate these cycles, are recorded in Table 8.4. The different types

of final demand (e.g. , personal consumption expenditures, investment, and

government purchases) display very different patterns over the business cycle.

Government purchases are the most stable since they reflect political rather than

market forces. Although government purchases may show uneven growth rates, these

are not related to the business cycle and generally even work to dampen the

magnitude of the cycle. Personal consumption is intermediate. Service and

nondurables purchases are very stable, durable purchases are cyclical, so that

overall consumption volumes show only a mild cycle. Private investment is the

most volatile of the main spending components. In recession years, investment

growth is zero or even negative; in the recovery years, investment volumes grow

much more rapidly than the rest of the economy.

The composition of final demand between personal consumption expenditures,

private investment and government purchases (i.e., how the output of the economy

is used) is projected to show substantive change. Table 8.5 shows the

composition of final demand in both current and constant dollar terms.

Perhaps the most striking feature is the decline in the share of the final

output of the economy devoted to government purchases. ^ This feature is an

assumption of the forecasts, it is not an output of the model. The declining

share of government leaves a greater percentage of output available for the

private sector. Both consumption and investment share in this increase.

1 This refers to Federal, state and local government purchases of goods and

services. It does not include government transfers, a major component of

government expenditures. These figures do not necessarily imply a decline in

the ratio of government expenditures to GNP.

73
Table 8.4 The Business Cycle: Annual Percentage Growth Rates, 1982-2000

Year

Variable

Personal

Consumption

Private

Investment

Government

Purchases

Real

GNP

1983 2.8 5.5 2.1 2.2

1984 4.5 7.3 2.5 4.4

1985 4.9 5.0 2.3 4.5

1986 4.0 -0.7 1.9 2.9

1987 2.6 1.3 1.7 2.6

1988 3.5 7.5 1.8 3.8

1989 2.4 0.6 1.8 2.2

1990 3.0 3.8 1.8 2.9

1991 2.6 5.7 1 e 6 3.0

1992 2.5 4.4 1.6 2.6

1993 2.0 0.9 1.3 1.9

1994 3.9 4.2 1.4 3.5

1995 4.1 6.9 1.4 4.1

1996 4.1 4.4 1.4 3.7

1997 2.7 0.3 1.4 2.3

1998 2.8 2.6 1.5 2.5

1999 2.9 3.0 1.4 2.8

2000 3.8 6.2 1.4 3.7

Table 8.5 Composition of Final Demand as a Percent of GNP, 1982 i NJ O o o

1

—

Variable

Current Dollars Constant Dollars

Name

1982 1990 2000 1982 1990 2000

Personal Consumption 64.5 65.5 65.7 64.8 66.3 66.9

Private Investment 13.8 15*2 16.4 13.4 14.0 14.8

Government Purchases 21.2 18.9 17.5 19.7 18.0 15.7

1 Net exports, a component of GNP , are not shown separately in this table.

74
However, it is projected that the greater part of the increase goes to

consumption. Personal consumption is projected to rise from around 64 percent of

real GNP to over 66 percent. Investment rises too but a substantial part of this

rise reflects cyclical features, recovery from the low investment share in the

1982 starting point. Aside from cyclical features, the investment share of real

GNP increases by about one percentage point.

DGEM explicitly models 36 producing industries, the output of each, the

disposition of this output, and the inputs into each. In fact, the macroeconomic

picture is just an aggregation of these industry level developments. Table 8.6

ranks the industries in terms of volume growth rates over the 1980's (i.e.

average annual growth in the volume or quantity of sales). These industries

range from communications, the fastest growing, at almost 8 percent, all the way

down to tobacco products, zero growth, and crude petroleum and gas, negative

growth. It is difficult to generalize a pattern as to which industries show

rapid growth and which show slow growth since performance depends not only on

demand considerations (such as the demand response to population and income

increase) but also on supply considerations (covering input mix and productivity,

operating through their effect on prices). However, a rough generalization is as

follows. The fast growing industries are high technology (communications,

instruments, plastics, chemicals, electrical machinery) and services

( f inance/insurance/real estate, services, trade). The slow growing industries

are extractive and resource processing (crude petroleum and gas, gas utilities,

petroleum refining, nonmetallic mining, metal mining, coal mining) and

population-related (tobacco products, leather, agriculture, food products). The

medium growing industries are the established manufacturing industries (apparel,

textiles, motor vehicles, primary metals, wood products).

The differences in growth rates are substantial. An industry such as

communications, growing at 7.8 percent, compared to the overall average growth

rate of 3.2 percent, will more than double its relative size over the 1982-2000

period; for example, the industry provided 3 percent of total gross output in

1982, but by the year 2000 its share will be 7.5 percent. Or services, growing

at 4 percent, will increase their relative size by 15 percent, expanding from 14

percent to 16 percent of total gross output. At the other end of the range, an

industry, such as tobacco products, showing zero growth would find its relative

size cut by over 40 percent. Or, agriculture-forestry-fisheries, growing at 1.7

percent, would decline in size from 4.5 percent of gross output in 1982 to just

over 3 percent in 2000.

These considerations lead to two general conclusions. First, there will be

major changes in the industrial mix by the year 2000. The high technology and

service industries will become more important, the resource-based industries will

be relatively smaller than in 1982. Second, even though major restructuring

will occur, the structure of the American economy in the year 2000 will not, in

the very broadest sense, be fundamentally different from the way it was in 1982.

75
Table 8.6 Industry Growth Rates^

Industry Rate Rank

Agriculture, Forestry, Fisheries 1.7 32

Metal Mining 2.7 26

Coal Mining 2.6 27

Crude Petroleum, Natural Gas -0.7 36

Nonmetallic Mining and Quarrying 2.0 30

Construction 2.8 25

Food-Kindered Products 2.1 29

Tobacco Manufactures 0.0 35

Textile Mill Products 3.0 21

Apparel 2.9 24

Lumber and Wood Products 3.0 22

Furniture and Fixtures 4.4 8

Paper and Allied Products 3.9 12

Printing and Publishing 3.7 14

Chemi cals 5.4 4

Petroleum Refining 1.2 32

Rubber and Miscellaneous Plastic Products 6.3 3

Leather 1.0 33

Stone, Clay, and Glass 3.2 19

Primary Metal Industries 3.0 20

Fabricated Metal Products 4.0 11

Machinery, Except Electrical 4.0 10

Electrical Machinery 4 .

6

7

Motor Vehicles and Equipment

Transportation Equipment and Ordanance,

3.0 23

Except Motor Vehicles 3.7 15

Instruments 7.7 2

Miscellaneous Manufacturing 4.9 5

Transportation 3.7 13

Communications 7.8 1 Electrical Utilities 3.5 17

Gas Utilities 0.3 34

Trade 3.6 16

Finance, Insurance, and Real Estate 4.8 6

Services 4.0 9

Government Enterprises 3.3 18

Miscellaneous 2.2 28

1 Average annual percentage growth in sales quantity, 1982 to 1990.

76
9. THE MOBILIZATION SCENARIO

The mobilization scenario analyzed with DGEM is patterned after an

unclassified scenario approved by F . C. Ikle, the DoD Under Secretary for

Policy. Basically, the scenario indicates a conventional war of three years

duration in Europe, the Persian Gulf, and Korea. For purposes of the current • study, a one year warning period is assumed. The scenario also provides the

analyst with considerable latitude in evaluating how changes in combinations of

numerical values for key variables, thus describing wars of greater or lesser

intensity, affect the U.S. industrial infrastructure.

The analysis of the mobilization scenario was carried out in two stages. In

the first stage, a mobilization baseline was constructed which documented the

departures for key variables from the base-case values (i.e. , those contained in

the data file) due to the period of mobilization and conflict. The results from

the first stage of the analysis provided information on gross-level changes in

the NIPA components, as well as measures of the economic impacts of mobilization

on a sector by sector basis.

In the second stage, six variables were varied in combination according to

an experimental design. The six variables which were the subject of the

structured sensitivity analysis and their names, as they appear in the tables

which follow, and in Appendix E are:

1. Federal defense purchases, GZ ( 1 )

;

2. Federal nondefense purchases, GZ ( 2 )

;

3. The Federal deficit, DG;

4. The efficiency with which capital is employed, AKD;

5. The rate of depreciation of capital, U; and

6. The effective supply of labor services, LB.

The structured sensitivity analysis was based on Monte Carlo techniques. The

objective here was to evaluate how uncertainty in the values of the six input

variables just mentioned translated into changes in the level and composition of

the NIPA components, capital services and energy consumption.

9.1 SPECIFICATION OF THE MOBILIZATION BASELINE AND THE STRUCTURED SENSITIVITY

ANALYSIS

The period of the mobilization follows that of the unclassified DoD scenario

and is assumed to be 1983 through 1986. The first year represents a period of

buildup prior to the onset of hostilities. For each year of the mobilization,

values for key economic variables were specified. Once the mobilization baseline

had been established, a structured sensitivity analysis was performed. The

values of the six variables which were the subject of the structured sensitivity

analysis, their base-case values and their values in the mobilization baseline

are compared at the end of this section. The specification of the military

mobilization scenario presented in this chapter is as follows:

1. Government purchases;

2. Labor services;

3. Capital services;

77
4. Foreign trade;

5. Fiscal policy;

6. Structured sensitivity analysis; and

7. Tabular summary.

Government Purchases

Defense purchases are central to any mobilization effort. Through the

TUTOR, it is possible to control both the level and composition of defense

purchases. However, for the case at hand, only the level is increased so that

all defense purchases are increased in the same proportion. Defense purchases

are increased by 50 percent in 1983, prior to the onset of hostilities,

and by 80 percent in 1984, 120 percent in 1985, and 150 percent in 1986.

These increases are relative to the base case, or the economic conditions

in the absence of mobilization. Federal nondefense purchases are increased by

25 percent in the first year and by 50 percent for the three following years.

Labor Services

During the mobilization, many additional people will be recruited or

conscripted into the armed forces. This is directly taken into account through

the increase in defense purchases. However, it is assumed that mobilization

measures will also be applied to the civilian population and the civilian labor

force. Additional people are assumed to be induced or pressed into the labor

force. To reflect this, the labor force participation rate is increased by 4

percent in every year. Workers are also assumed to work longer hours, through

overtime or just through an extension of the standard workweek. Average hours

per week are raised by 13 percent. But, these changes are likely to adversely

affect labor efficiency. Consequently, the average efficiency of each hour of

labor is reduced by 7 percent. These three factors taken together produce a 9

percent increase in the effective supply of labor services.

Capital Services

A forced increase in capital use is assumed. This corresponds to the more

intensive use of existing capital, as plant and equipment is used more hours per

week, and the inclusion of emergency capacity. The overall increase .in the

efficiency with which capital is employed is taken to be 5 percent. This figure

is a very conservative estimate of the potential for increasing the efficiency of

the capital stock, since estimates of emergency capacity which can be brought on

line within one year frequently exceed peacetime capacity levels by 25 percent.

However, even this modest increase, is likely to accelerate the wear and tear on

capital. Accordingly, the rate of depreciation of capital is increased by 10

percent

.

Foreign Trade

It is assumed that all imports will be threatened,

reduction of 20 percent in imports is planned. Also, in

domestic requirements are satisfied first, an across the

percent in exports is planned.

An across the board

order to ensure that

board reduction of 50

78
Fiscal Policy

The major emphasis here is the Federal deficit, since government purchases

have already been addressed. The government deficit is constrained to exceed the

base-case levels by fixed percentages. Tax rates are increased in order to

secure the additional revenue that is required to maintain the targeted budget

deficit in the face of higher federal spending. This means that the increase in

government expenditure is paid for in part by an increase in taxes and in part by

increased deficits. This mixed strategy was designed to strike a balance between

increased investment and inflationary pressures. For example, if the

mobilization were financed by taxes, it would not come at the expense of

investment, since the government is not claiming an additional share of the

capital market in order to finance defense purchases, but would come from

consumption, since it is households and consumers who carry the main burden of

taxes. If, in contrast, the mobilization were deficit financed, then the

government would be claiming a larger share of investible funds so investment

rather than consumption would likely carry a greater burden of the mobilization

(i.e., private investment is crowded out). For the case at hand, the Federal

deficit is allowed to increase (over the base-case values) by 50 percent in 1983

and by 80 percent in 1984 through 1986.

Structured Sensitivity Analysis

Because the values of many key variables in a mobilization scenario are not

known with certainty, it is advisable to select a small set, whose impact is

likely to be substantial, and subject them to a structured sensitivity analysis.

Variations in the values of these input variables translate into variations in

the values of the output or endogenous variables in such a manner that the

economic impacts of shocks to the system can be measured quantitatively.

The approach selected for this study makes use of recent work by McKay,

Conover, and Beckman^- and by Harris.- Their work is based on the method of

M.D. McKay, W.H. Conover, and R.J. Beckman, "A Comparison of Three Methods for

Selecting Values of Input Variables in the Analysis of Output from a Computer

Code," Technometrics , Vol. 21, No. 2, 1979, pp. 239-245.

^ C.M. Harris, "An Assessment of Climatological Uncertainties Using Monte Carlo

Analysis," in Selected Assessment Strategies Applied to Short-Term Energy

Models , Gaithersburg, MD: National Bureau of Standards, NBSIR 83-2672,

1983, pp. 30-54.

J C.M. Harris, Issues in Sensitivity and Statistical Analysis of Large-Scale , Computer-Based Models , Gaithersburg, MD: National Bureau of Standards,

GCR 84-466, 1984.

^ C. M. Harris, Computer Generation of Latin Hypercube Sampling Plans , Gaithersburg, MD: National Bureau of- Standards, NBS GCR 84-476, 1985.

79
model sampling. The method of model sampling is a procedure for sampling from a stochastic process to determine, through multiple trials, the nature and effects

of a probability distribution. The method of model sampling, or distribution

sampling as it is also called, has a long history of use by statisticians to

derive distributions empirically that are difficult or impossible to derive by

other means. It permits the effects of uncertainty to be rigorously analyzed.

The procedure employed in this study is known as the Latin Hypercube

sampling scheme. The procedure, as it ? s name implies, is patterned after the

classical Latin square. Latin squares consist of a set of permutations such that

a given character or value appears only once in each row and each column. A

Latin hypercube is similar to a Latin square with the important exception that it

contains more rows than columns. For example, if each column is thought of as a

variable and each row is a simulation number, then entries in the cells contain

the values of a set of equally spaced percentiles from the parent cumulative

distribution function (CDF) of the variable of interest. For the case at hand,

with six variables of interest and ten simulations, the entries in the cells are

the 5th, 15th, ..., 95th percentiles of the parent CDF.

In reality, the exact nature of the parent CDF (e.g., measures of central

tendency and dispersion) is unknown. Estimates of the parameters (e.g., mean and

variance) of the CDF can be made and uncertainty can be reduced by investigation

and research. However, uncertainty can never be eliminated completely because

new sources of uncertainty are arising all the time. The true specification of

the CDF can only be known when mobilization is actually underway.

So that the flexibility of the procedure could be illustrated without undue

attention on the characterization of the CDF, it was decided to focus on the

triangular distribution. The triangular distribution was chosen because it may

be specified by three values, low, median, and high, which correspond to the Oth,

50th, and 100th percentiles.

Individual values within each of the six triangular distributions were

chosen aaccording to the Latin hypercube sampling scheme (i.e., the percentiles

recorded in the cells). These values were then recorded in a runstream file.

Each of the runstream files were then executed and the results of the simulation

were stored on-line for further analysis.

Tabular Summary

The data which were the focus of the structured sensitivity analysis are

summarized in Tables 9.1, 9.2 and 9.3. Table 9.1 records the base-case values of

the six variables (i.e., as they appear in the DGEM data file). Table 9.2

records the values of the same six variables for the mobilization baseline case.

These values are also the median values used in fitting triangular distributions

to each variable. The minimum and maximum values for each of the six variables

are recorded in Table 9.3. The information in Tables 9.2 and 9.3 served to

define the CDF for each variable. Values were then selected from each CDF

according to the Latin hypercube sampling scheme.

80
Table 9.1 Base-Case Values

Variable

Name

Values by Year

1983 1984 1985 1986

GZ ( 1

)

207.3 220.3 235.7 233.2

GZ ( 2) 68.3 72.3 86.9 67.62

AKD 0.146 0.159 0.160 0.156

U 0.062 0.062 0.062 0.062

LB 1037.7 1058.3 1050.9 1068.9

DG 174.0 167.3 211.9 192.5

Table 9.2 Mobilization Baseline Values

Variable

Name

Values by Year

1983

0 1984 1985 1986

GZ ( 1

)

311.0 396.5 5.8.5 583.0

GZ ( 2 ) 85.4 108.5 130.4 101.4

AKD 0.154 0.167 0.168 0.164

U 0.068 0.068 0.068 0.068

LB 1131.1 1153.6 1145.5 1165.1

DG 261 .

1

301.1 381.5 346.5

81
Table 9.3 Extreme Values Used in the Structured Sensitivity Analysis

Variable

Name

Values by Year

1983 1984 1985 1986

GZ(1) MIN 290.2 374.5 471.4 513.0

MAX 331.7 418.6 565.7 653.0

GZ ( 2 ) MIN 82.0 101.2 121.7 94.6

MAX 88.8 115.7 139.0 108.2

AKD MIN 0.150 0.164 0.165 0.161

MAX 0.156 0.170 0.171 0.167

U MIN 0.065 0.065 0.065 0.065

MAX 0.071 0.071 0.071 0.071

LB MIN 1110.3 1132.4 1124.5 1143.7

MAX 1151.8 1174.7 1166.5 1186.5

DG MIN 226.2 284.4 339.0 308.0

MAX 295.8 317.9 423.8 385.0

82
9.2 IMPACTS OF THE MOBILIZATION SCENARIO ON THE NATIONAL ECONOMY

9.2.1 Analysis of the Mobilization Baseline

The volume of economic output increases as a result of mobilization. Demand

increases due to high government purchases, while the effective volume of

available resources also rises, as a result of the increase in capital and labor

services. Thus, demand is pulling production higher while resources and capacity

permit an accompanying increase in production.

Total Production

Real GNP changes in 1983 are estimated to be:

1. Base case real GNP $3277.7 (10^);

2. Real GNP with mobilization $3634.4;

3. Increase in real GNP $356.7; and

4. Percentage increase in real GNP 10.9

The higher level of real GNP is continued from 1984 through 1986 where the

relative increases are 11.5, 11.3, and 10.8 percent.

Thus, the mobilization stimulates economic activity. Production responds;

the entire level of economic activity is increased. Real GNP increases by an

average of 11.1 percent compared to the base case economic situation. How is

this increase in production sustained? At an aggregate level, the increase can

be separated into three components:

1. Additional input of labor services;

2. Additional input of capital services; and

3. Increases in productivity.

Use of Output

This gain in production is used in the form of higher levels of final

demand. But, which types of spending benefit from the higher level of economic

activity? Clearly, government purchases will increase substantially, since the

rise in defense spending is the initiating factor in all these adjustments. But,

do consumption and investment share in the greater level of production? The

average increases in final demand during the period of mobilization are:

1

.

Personal consumption expenditures 6.2%

2. Gross private domestic investment 13.1%

3. Government purchaes 38.0%

4

.

Exports -50.0%

5. Imports -20.0%

83
Consumption and investment rise and government purchases rise

Thus, the gains from the shift to mobilization do not accrue to bu

Instead, the gains go to the government. Mobilization generates a economic structure, away from business and investment and consumpt

government activity.

most of all.

si nesses

.

shift in

ion and towards

Economic Growth

The initial effect of the mobilization is to raise the level of economic

activity. Real GNP in 1983 is some 10.9 percent higher than in the base case.

But, does this mean that the mobilization generates more rapid economic growth?

In fact, the mobilization does not increase economic growth. Certainly, the

growth in real GNP from 1983 is accelerated. But, this corresponds to a one-time

jump to a higher level of economic activity. Once production is at this new,

higher level, growth rates revert to their former pattern. Thus, mobilization

generates a single shift to more production, as both final demand and available

resources are boosted. Mobilization does not generate a continuing gain in

achievable rates of economic growth.

This is an important point, for it means that the early gains under

mobilization will not be repeated. A higher level of production is sustainable,

at least for some time, but large continuing gains are not realistic.

Prices

The mobilization does affect the level of prices. It might be expected that

inflation will accelerate (i.e., that the rate of increase in the general level

of prices will rise). This is exactly what happens. The GNP price deflator

shows an acceleration, over the GNP price deflator for the base case, of 2.3

percent in 1983, 4.5 percent in 1984, 9.7 percent in 1985 and 13.8 percent

in 1986. The estimated inflationary impact is not immediate. It is not until

the third year of the mobilization that the inflationary effects become

striking.

Industry Effects

Different industries are affected

These impacts reflect several forces,

change in input patterns accompanying a

to different degrees

The rise in defense

ny restructuring in

by the mobiliza

purchases; and a relative prices.

t ion.

Overall production rises, on the average, by 10.1 percent compared to the

base-case conditions. The percentage change in individual industries ranges from

a decline of 11.5 percent for metal mining to a rise of 32.1 percent for

transportation equipment and ordnance.

Several industries have increases in output volumes. These involve crude

petroleum and gas, petroleum refining, gas utilities, electric utilities,

construction, services, and, with the largest gain, military transport equipment

and ordnance. Some of these are expected as they are the materials directly

purchased for defense purposes. Others, however, reflect the indirect effects,

as requirements work their way through the industrial system. For example,

military transportation equipment and ordnance and petroleum are directly

related to defense purchases, whereas most of the remaining industries are

indirectly related.

84
In contrast, some industries are estimated to experience a reduction in

volume. Even though overall economic activity is higher, these industries are

less needed than in the base case.

Information on specific industries can be used in several ways. As a projection, it estimates what would happen under mobilization conditions. As a prescription, this indicates which industries are most important to the

mobilization effort. Industries with a greater than average increase in output

are of greater than average importance. Some of these are obvious, given their

direct role in defense purchases. Others, however, are less obvious and the

ability to reveal such industries is an important benefit from an analysis such

as this. By indicating the relative importance of each industry, this analysis

can help mobilization planning if policies were to be implemented to affect the

allocation of resources (i.e., the essence of such allocations is to

differentiate higher from lower priority claimants on available resources).

A related application of this analysis involves the identification of

bottlenecks. The increases in industry outputs are the increases required to

sustain the mobilization. These increases might be compared with the expansion

capability of an industry based on detailed knowledge of the industry and of

constraints specific to the industry. If the required expansion exceeds the

maximum likely expansion, or surge capacity, then the analyst has identified a potential bottleneck. Contingency plans can then be developed to ease such

bottlenecks in the event of a mobilization.

9.2.2 Results of the Structured Sensitivity Analysis

The general trends described in the previous subsection were reinforced in

the structured sensitivity analysis. Consequently, the focus here is on

documenting how perturbations about the values of the six key input variables

given in Table 9.2 translated into changes in five major output variables.

These output variables and their names are:

(1) Real Gross National Product, GNPQ;

(2) Real Personal Consumption Expenditures; CNIAQ;

(3) Real Gross Private Domestic Investment, INIAQ;

(4) Real Private Domestic Capital Services, KD; and

(5) Quantity of Energy Delivered, QED.

The first three variables are associated with the National Income and

Product Accounts (NIPA); they are referred to in the text, tables and figures

which follow as real GNP , real PCE and real investment, respectively. The fourth

variable, KD, registers the joint effects of variations in the efficiency with

which capital is employed, AKD, and capital accumulation over the mobilization

period. It is important to note that capital accumulation incorporates the

effects of investment and another key input variable in Table 9.2, U, the rate

of depreciation of capital. For each year in the mobilization, KD is equal to

the product of AKD and KL, the total real private domestic capital stock at the

beginning of the year. The fifth variable measures the amount of energy

delivered to the domestic economy in quadrillion BTU's (Quads). This amount is

equal to the sum total of Quads delivered for electricity, coal, natural gas and

petroleum.

85
The results obtained from the structured sensitivity analysis on each of the

five major output variables are described through reference to a series of tables

and figures. These tables and figures summarize three basic types of information

for each variable:

(1) The equilibrium levels experienced as a result of mobilization;

(2) The percentage change (increase or decrease) over the base-case

simulation as a result of mobilization; and

(3) The annual percentage change in the equilibrium levels during the period

of the mobilization.

These three types of information are hereafter referred to as mobilization

values, base-case deviations, and annual rates of change, respectively.

Most of the figures presented in this section make use of a graphical

analysis technique known as the box plot.^- Box plots provide a convenient means

for summarizing graphically an entire data set. Box plots are essentially a

graphical analysis of variance. One advantage of box plots for the structured

sensitivity analysis is that they permit multiple years worth of data to be

compared in a single diagram.

The box plot consists of a box with its "hinges" drawn at the 25th and 75th

percentiles, respectively. The width of the box is proportional to the number of

observations. The median (i.e., the 50th percentile) is marked within the box.

The extreme values are marked and connected to the hinges of the box with

"whiskers." The box plot thus graphically summarizes both the central tendency

of the data as well as its dispersion. Non-overlapping boxes on the same plot

are a strong indication that the two sets of data are significantly different.

All Box plots presented in this section were generated through application of the

statistical analysis package DATAPLOT.^

Growth in Total Production

As was shown earlier, the output of the economy increases during the

mobilization. Figure 9.1 provides a graphical summary of the economic impacts of

mobilization for three sets of values for real GNP between 1982 and 1986. The

first year, 1982, is prior to the mobilization period and is included as a reference point. The first set of values is for the base-case simulation; these

values are traced out by a solid line. These values provide a basis for

comparing the output of the peace-time economy to outputs anticipated during the

mobilization. The second set of values is for the mobilization baseline; these

values are traced out by a dotted line. The third set of values is for the

mobilization maxima (i.e., the set of highest values for each year across all

simulations); these values are traced out with triangles and dashed lines.

^ JT w! Tukey, Exploratory Data Analysis , New York: Addison-Wesley , 1977.

2 J. J. Filliben, DATAPLQT: Introduction and Overview , Gaithersburg, MD:

National Bureau of Standards, Special Publication 667, 1984.

86
Figure 9.1 Values of Real GNP for the Base-Case Simulation, the Mobilization

Baseline and the Mobilization Maxima

YEAR OF MOBILIZATION

BASE CASE - SOLID LINE MOBILIZATION = DOTTED LINE

MOBILIZATION MAXIMA * TRIANGLES ST DASHED LINE

87
m

&

>

r*

t*

o

a

Figure 9.2 Box Plots of Mobilization Values for Real GNP

YEAR OF MOBILIZATION

88
Through reference to Figure 9.1, it can be seen that real GNP increased from

just under $3200 billion in 1982 to nearly $4100 billion in 1986. The figure

documents the rapid increases in real GNP in 1983 and 1984. In 1985 and 1986,

the economy gives indications of a return to its long-run growth trend, since the

traces associated with the mobilization parallel that of the base-case scenario.

The figure also shows that mobilization has resulted in a net increase of

approximately $400 billion in the level of real GNP over the base-case simulation

for 1984 through 1986.

Figure 9.2 is a sequence of box plots showing the range of the mobilization

values for real GNP. The figure demonstrates that significant increases in the

output of the economy are experienced in each year of the mobilization. A

tabular summary of the data in Figure 9.2 is given in Part A of Table 9.4. From

Part A of Table 9.4, it can be seen that the range (i.e., the difference between

the maximum and minimum values) increases from approximately $55 billion in 1983

to nearly $80 billion in 1986. Although the standard deviation of the simulated

values increases in each year, it averages about 0.5 percent of the mean.

Consequently, the increases in real GNP noted earlier are statistically

significant

.

A quick comparison between Figures 9.1 and 9.3 documents the magnitude of

the change in the output of the economy due to mobilization. Figure 9.3

indicates that the war-time economy averages an increase of approximately 11

percent over the base-case simulation. Note that the base-case deviations reach

a peak in 1984 and then decline slightly through 1986. This observation is

consistent with the statement made earlier that the economy returned to its

long-run growth trend once the initial shocks of the mobilization worked their

way through the model. Part B of Table 9.4 provides a summary of the data in

Figure 9.3. These data show the significance of the initial surges, followed by

the resumption of a trend.

Comparisons between Figures 9.1, 9.2, and 9.4 and the data for 1983 in

Figure 9.3 illustrate how the output of the economy has grown over the

mobilization period. The initial surge in government purchases produces a rapid

increase in output. Reference to Figure 9.4 shows that the annual rate of change

in real GNP is nearly 15 percent in 1983.1 p n 1934 ^ the annual rate of change

has slowed to approximately 7.5 percent. By 1985, real GNP has returned to

growth rates consistent with a peace-time economy. Part C of Table 9.4 provides

a summary of the data in Figure 9.4. These data show how the initial surge

produces both a high rate of change and a high variability in that rate. By

1984, although the rate of change is still high, the variability has been reduced

substantially. The variability in the growth rate then hovers around 0.1 percent

through 1986. These observations lend support to the claim made earlier that

mobilization produces a one-time jump to a higher level of economic activity

rather than a continuing gain in achievable rates of economic growth.

1 The annual rate of change for 1983 is computed using the 1982 value of the

base-case simulation. Consequently, the annual rate of change of nearly

15 percent incorporates an underlying growth trend plus a deviation from the

base-case simulation. It is for this reason that the annual rate of change

for 1983 exceeds the base-case deviation for 1983.

89
Table 9.4 Summary Statistics for Real GNP

PART A: MOBILIZATION VALUES

YEAR MEAN MINIMUM MAXIMUM STD. DEV.

1983 3629.6 3606.4 3659 .

7

15.97

1984 3893.5 3862.4 3932.0 19.58

1985 3970.9 3936.7 4010.0 20.96

1986 4069.1 4035.0 4113.1 23.44

PART B: BASE-CASE DEVIATIONS

YEAR MEAN MINIMUM MAXIMUM STD. DEV.

1983 10.74 10.03 11.66 0.487

1984 11.50 10.61 12.60 0.561

1985 11.23 10.27 12.32 0.587

1986 10.77 9.84 11.97 0.638

PART C: ANNUAL RATES OF CHANGE

YEAR MEAN MINIMUM MAXIMUM STD. DEV.

1983 14.64 13.91 15.60 0.505

1984 7.27 7.10 7.44 0.107

1585 1.99 1.90 2.13 0.078

1986 2.47 2.25 2.61 0.100

90
WQ&!>

wn

M

Q

>

8-3 «

M

O

£3M

*Td

Figure 9.3 Box Plots of Base-Case Deviations for Real GNP

12 -

1 1 -

10 -

9 —

82 83 84 85 86

YEAR OF MOBILIZATION

91
Figure 9.4 Box Plots of Annual Rates of Change for Real GNP

YEAR OF MOBILIZATION

92
Use of Output

The focus of this portion of the analysis is on changes in real PCE and real

investment. The other NIPA components, government purchases, exports and

imports, follow the same patterns as in the mobilization baseline. Exports are

reduced by 50 percent and imports by 20 percent in all simulations. The levels

of government purchases were selected according to the Latin hypercube sampling

scheme described earlier. The average increase in government purchases was 38

percent over the period of the mobilization.

The values of real PCE experienced during the mobilization are plotted in

Figure 9.5. From the figure, it can be seen that the mobilization causes real

PCE to increase substantially in 1983. The increases in real PCE for 1984

through 1986, however, are lower than in the base-case simulation.

The way in which real PCE changes during mobilization period is clearly

illustrated in Figure 9.6. For example, real PCE for 1984 is significantly

higher than in 1983. However, once the initial surge in government purchases has

worked its way through the model, real PCE remains nearly constant. These

observations are summarized in Part A of Table 9.5, where the mean value of real

PCE changes by less than $20 billion between 1984 and 1986.

Comparisons between Figures 9.5 and 9.7 show that the mobilization values of

real PCE are trending toward those of the base-case simulation. Furthermore, the

trend seems to be significant during the latter years of the mobilization.

Reference to Part B of Table 9.5 shows that the average base-case deviation is

6.2 percent. The value for 1983 is 9.6 percent whereas it is only 2.4 percent

for 1986.

Figure 9.8 provides some additional insights into how real PCE changes over

the mobilization period. After an initial surge, the annual rate of change in

real PCE is well under 5 percent. Part C of Table 9.5 shows that the growth

rate for real PCE is slightly negative for 1985 and is essentially flat for 1986.

Real Investment

The mobilization values of real investment follow cyclical rather than a secular trend. These values plotted in Figure 9.9, reflect the underlying

business cycle. Investment increases sharply in 1983 and 1984, turns down in

1985 and experiences a rate of increase approximately equal to that of the base

case simulation in 1986.

The box plots of the mobilization values of real investment in Figure 9.10

indicate that all changes, both increases and decreases, are likely to be highly

significant. These observations are borne out through reference to Part A of

Table 9.6.

93
pq^

hh o

£5m

ON

POJJ<a5w

Figure 9.5 Values of Real PCE for the Base-Case Simulation, the Mobilization

Baseline and the Mobilization Maxima

YEAR OF MOBILIZATION

BASE CASE « SOLID LINE MOBILIZATION » DOTTED LINE

MOBILIZATION MAXIMA = TRIANGLES & DASHED LINE

94
m

£d>

t“*

t*O

O

Q

m

iz§ Q

»-e f

t*

mW

Figure 9.6 Box Plots of Mobilization Values for Real PCE

82 83 84 85 86 87

YEAR OF MOBILIZATION

95
Table 9.5 Summary Statistics for Real PCE

PART A: MOBILIZATION VALUES

YEAR MEAN MINIMUM MAXIMUM STD. DEV.

1983 2351.6 2327.7 2383.6 14.86

1984 2423.0 2395.1 2453.1 16.82

1985 2422.8 2393.0 2470.8 22.59

1986 2442.6 2406.9 2499.5 28.32

PART B: BASE-CASE DEVIATIONS

YEAR MEAN MINIMUM MAXIMUM STD. DEV.

1983 9.58 8.47 11.08 0.692

1984 8.18 6.93 9.52 0.751

1985 4.75 3.46 6.82 0.976

1986 2.38 0.89 4.77 1.187

PART C: ANNUAL RATES OF CHANGE

YEAR MEAN MINIMUM MAXIMUM STD. DEV.

1983 14.67 13.51 16.23 0.725

1984 3.04 2.79 3.47 0.190

1985 -0.01 -0.75 0.73 0.418

1986 0.81 0.33 1.38 0.309

96
MQfeS>WO

Figure 9.7 Box Plots of Base-Case Deviations for Real PCE

YEAR OF MOBILIZATION

97
WO

53

^

WQ^>

wn

O

M

»-3 >

Figure 9.8 Box Plots of Annual Rates of Change for Real PCE

YEAR OF MOBILIZATION

98
As was the case for real PCE , the early gains in investment due to

mobilization are narrowing by 1985. Figure 9.11 differs from Figure 9.7 in that

base-case deviations for real investment appear to stabilize at around 7 percent.

Figure 9.12 illustrates the importance of the business cycle. Due to the

combined effects of the accelerator principle-*- and the potential for crowding out

due to deficit financing, the rate of change in real investment is quite

volatile. Part C of Table 9.6 shows how volatile investment spending is during

the period of mobilization. After increasing by almost 41 percent in 1983, the

rate of change in real investment drops to slightly under 21 percent in 1984; it

then turns sharply negative to -8 percent in 1985. Investment spending then

increases in 1986, although the level is still less than at its peak in 1984.

* Capital Services

The wide swings in investment experienced during the mobilization lead one

to conjecture about their impact on capital services. Recall that capital

services are a function of the level of the capital stock at the beginning of

each year and the "efficiency" with which that stock can be utilized.

Figure 9.13 illustrates that, even though investment is quite volatile,

capital services increase steadily. By 1985, capital services appear to be

changing according to the same long-term trend that underlies the base-case

simulation.

The pattern of tapering off shown in Figure 9.13 is more pronounced in

Figure 9.14. In particular, after significant increases in capital services

experienced between 1983 and 1985, the change is quite modest in 1986. Reference

to Part A of Table 9.7 demonstrates the degree of overlap for capital services

in 1985 and 1986.

From Figure 9.13, it was clear that capital services exceeded those under

the base-case simulation. It is now important to determine if these increases

are significant. Reference to Figure 9.15 and Part B of Table 9.7 indicates

that except for 1983, the mobilization level of capital services is significantly

higher than in the base-case simulation. Thus investment volatility in the

war-time economy and increased shifts, which produce increased wear and tear on

the entire stock, is not sufficient to reduce the "productive flow" of services

from the capital stock.

1 The accelerator principle asserts that net investment is a function of the rate

of change in final output rather than the absolute level of output.
m

>

IT*

r*O

U

MSSOHft^Ha

Figure 9.9 Values of Real Investment for the Base-Case Simulation, the Mobilization Baseline and the Mobilization Maxima

YEAR OF MOBILIZATION

BASE CASE - SOLID LINE MOBILIZATION - DOTTED LINE

MOBILIZATION MAXIMA - TRIANGLES Sf DASHED LINE

100
c/3

ttf >

IT" f

O

O

03 ^

O

«-• ir* f

•-*

td

Figure 9.10 Box Plots of Mobilization Values for Real Investment

YEAR OF MOBILIZATION

101
Table 9.6 Summary Statistics for Real Investment

PART A: MOBILIZATION VALUES

YEAR MEAN MINIMUM MAXIMUM STD. DEV.

1983 629.8 617.6 645.1 7.62

1984 761.4 753.4 775.0 5.91

1983 700.1 687.1 720.1 9.83

1986 739.1 725.3 759.9 11.95

PART B: BASE-CASE DEVIATIONS

YEAR MEAN MINIMUM MAXIMUM STD. DEV.

1983 25.12 22.68 28.15 1.515

1984 15.14 13.92 17.20 0.894

1985 7.87 5.88 10.95 1.514

1986 6.75 4.75 9.74 1.726

PART C: ANNUAL RATES OF CHANGE

YEAR MEAN MINIMUM MAXIMUM STD. DEV.

1983 40.81 38.07 44.23 1.705

1984 20.90 20.02 22.13 0.740

1985 -8.06 -8.79 -7.09 0.629

1986 5.58 4.67 6.39 0.489

102
bdQ^>

Wn

M

Q

>

1-3

5z5 M

o

$d

W

^

Figure 9.11 Box Plots of Base-Case Deviations for Real Investment

YEAR OF MOBILIZATION

1 U3
Figure 9.12 Box Plots of Annual Rates of Change for Real Investment

YEAR OF MOBILIZATION

L04
m

5d>

ir* t

1O

O

Figure 9.13 Values of Capital Services for the Base-Case Simulation, the Mobilization Baseline and the Mobilization Maxima

YEAR OF MOBILIZATION

BASE CASE =» SOLID LINE MOBILIZATION = DOTTED LINE

MOBILIZATION MAXIMA - TRIANGLES & DASHED LINE

105
pqM

^

Mo

525 w

O

Pm

Figure 9.14 Box Plots of Mobilization Values for Capital Services

82 83 84 85 86 87

YEAR OF MOBILIZATION

106
Table 9.7 Summary Statistics for Capital Services

PART A: MOBILIZATION VALUES

YEAR MEAN MINIM UM MAXIM UM STD. DEV.

1983 651.9 642.3 659.3 5.41

1984 737.0 727.2 746.9 5.64

1985 777.4 766.1 788.1 6.47

1986 785.4 770.3 797.6 8.09

PART B:- BASE-CASE DEVIATIONS

YEAR MEAN MINIMUM MAXIMUM STD. DEV.

1983 0.95 LO 0 . 1 2.09 0.838

1984 2.93 1.56 4.31 0.788

1985 4.39 2.87 5.83 0.869

1986 4.41 2.39 6.03 0.108

PART C: ANNUAL RATES OF CHANGE

YEAR MEAN MINIMUM MAXIMUM STD. DEV.

1983 9.79 8.16 11.03 0.911

1984 13.05 12.49 13.51 0.389

1985 5.48 5.17 5.93 0.191

1986 1.04 0.24 1.33 0.344

107
MQ!2!>WO

MQ>^^WOJdM»D

Figure 9.15 Box Plots of Base-Case Deviations for Capital Services

7

6 -

5 -

4 -

3 -

2 -

1 -

0 ~

-1

82 83 87

YEAH OF MOBILIZATION
H

!2j W

O

5dM

^

tzj

»-•

M

Q

Sj

>

W

O

O

M

»-3 >

$d

Figure 9.16 Box Plots of Annual Rates of Change for Capital Services

YEAR OF MOBILIZATION

109
Since capital services are based on the quantity of capital stock available

at the beginning of the year, the largest annual rate of change is experienced in

1984.-*- The rate of change in capital services then drops markedly due in part to

investment volatility and the high rates of depreciation included to reflect wear

and tear on the capital stock. Part C of Table 9.7 provides a tabular summary

of the data shown in the box plots.

Quantity of Energy Delivered

Mobilization results in a significant increase in the demand for energy.

The results of the structured sensitivity analysis indicated major increases for

each type of energy (coal, electricity, natural gas, petroleum) treated by the

model. Consequently, it was decided to focus on a summary measure, the sum total

quantity of energy delivered, total energy.

Figure 9.17 illustrates how the quantity of total energy increases from just

under 50 Quads to in excess of 70 Quads. After the initial surge, total energy

delivered increases only slightly.

Due to the reduced range on the vertical axis in Figure 9.18, it is easier

to see the trend in the quantity of energy delivered. As was the case for real

GNP, total energy delivered increases sharply between 1983 and 1984 and then

slackens. This trend is summarized in tabular form in Part A Table 9.8.

The base-case deviations for total energy, shown in Figure 9.19, follow a similar trend. Part B of Table 9.8 indicates that the trend of increasing

quantities delivered in the first three years is likely to be significant while

that in the final year only borders on significance.

The annual rate of change in total energy delivered, shown in Figure 9.20,

follows the trend seen in Figure 9.17. After an initial surge, where the rate of

increase approached 50 percent, the annual rate of increase settles into a tight

band which is less than 5 percent. Part C of Table 9.8 documents the more than

10-fold decrease in the annual rate of change between 1983 and 1984 through 1986.

T Recall that the largest rate of change for investment was in 1983.

Consequently, capital put in place through investment in 1983 is included in

the quantity of capital stock available at the beginning of 1984.
Table 9.8 Summary Statistics for Total Energy

PART A: MOBILIZATION VALUES

YEAR MEAN MINIMUM MAXIMUM STD. DEV.

1983 71.89 71.62 72.18 0.183

1984 75.00 74.65 75.39 0.223

1985 75.57 75.16 75.94 0.258

1986 76.90 76.48 77.33 0.312

PART B: BASE-CASE DEVIATIONS

YEAR MEAN MINIMUM MAXIMUM STD. DEV.

1983 42.46 41.93 43.02 0.363

1984 44.13 43.47 44.88 0.428

1985 46.62 45.83 47.33 0.500

1986 47.38 46.57 48.21 0.600

PART C: ANNUAL RATES OF CHANGE

YEAR MEAN MINIMUM MAXIMUM STD. DEV.

1983 46.57 46.03 47.15 0.373

1984 4.32 4.23 4.45 0.068

1985 0.76 0.63 0.88 0.076

1986 1.77 1 . 64 1.92 0.092

111
Figure 9.17 Values of Total Energy for the Base-Case Simulation, the Mobilization

Baseline and the Mobilization Maxima

YEAR OF MOBILIZATION

BASE CASE - SOLID LINE MOBILIZATION = DOTTED LINE

MOBILIZATION MAXIMA = TRIANGLES & DASHED LINE

112
dtn?)M<H^WO

W

U

>

q«€>

Figure 9.18 Box Plots of Mobilization Values for Total Energy

78

77 -

76 -

75 -

74 -

73 -

72 —

71 —

82

X

X

X

83 84 85 86

YEAR OF MOBILIZATION

87

113
M

Q

>

h3

W

o

$d

W

^

Figure 9.19 Box Plots of Base-Case Deviations for Total Energy

49

48 -

47 -

46 -

45

44

43 -

42

41

82 84

~T~

85 86 87

YEAR OF MOBILIZATION

114
52$^

bd»-3>5d

Figure 9.20 Box Plots of Annual Rates of Change for Total Energy

115
10 SUMMARY OF MAJOR FINDINGS

This report developed a set of generic guidelines which may be used to

evaluate emergency management models and data bases. In order to provide both

an explicit statement of the evaluation methodology and a step-by-step procedure

for implementing it, the report was divided into two parts. The basic problem

was stated in Chapter 1 where a three-phased procedure was outlined which

focused on:

(1) obtaining a clear and comprehensive statement of user requirements

and objectives pertaining to the application of the model;

(2) generating appropriate information about the model design and

performance pertaining to user requirements; and

(3) evaluating model attributes and properties according to predetermined

criteria of performance required by the user.

The "analytical techniques" associated with the second phase of the

procedure form the core of the generic model/data base evaluation guidelines;

they are described in Part I of the report. The stages of descriptive analysis,

program verification and analysis, data audit, and sensitivity analysis provide

the pool of information which drives the evaluation process. If one views the

guidelines as having a front end where test results are evaluated, then the

analytical techniques provide the critical linkage.

The previous point may be bett

determine what the user requirement

scenario, it is necessary to unders

documentation which purports to des

system. These topics are the subje

One must also recognize that in the

of the evaluation, it is necessary

and which instruments are to be use

data, control variables, parameters

evaluator deeper into the topics di

and analysis.

er understood by noting that in order t s are and how they translate into a tes

tand the purpose of the model and revie

cribe the model, as implemented on the

ct of Chapter 2 and Sections 3.1 and 4.

information development and generation

to state which questions are to be stud

d to address these questions (e.g., spe

). Answers to these questions take the

scussed in Chapter 3, program verificat

0

tw the

host

1 .phase

ied

cial

ion

The data audit, Chapter 4, serves to firm up the test scenario by studying

the model-data interface and the analysis of file computerization and

accessibility. Specialized statistical techniques are also used to characterize

the variability of the input data. This step, in conjunction with a review of

the documentation for the data base, provides guidance in designing the

sensitivity analysis.

A structured sensitivity analysis is the culmination of the earlier stages

in the evaluation process. The specialized techniques for performing a sensitivity analysis discussed in Section 5.2 seek to develop relationships

between input uncertainties and the variability of the model's output, to find

areas where the model's output changes drastically (e.g., due to instabilities),

or to exploit unique capabilities of the model. A major goal in performing a sensitivity analysis is to generate the critical pieces of information for those

acceptance tests which focus on quantitative characteristics.

i 16
How the process of information development and generation is incorporated

into a generic model/data base evaluation procedure is illustrated in Part 11 of

the report. The Dynamic General Equilibrium Model (DGEM) is used as a test

vehicle for the generic evaluation guidelines. DGEM was selected for evaluation

primarily for its unique approach to modeling emergency situations. The

availability of documentation was also a major factor. We wish to alert the

reader to the importance of documentation in any model evaluation effort because

it is the principal means through which performance against user requirements

can be measured.

The analysis of the U.S. economy during a military mobilization is used to

illustrate how the evaluation procedure may be applied to DGEM . This approach

was taken for two reasons. First, a mobilization represents a major

perturbation about the "business as usual" base-case simulation. Consequently,

any weaknesses of the model (e.g., instabilities, implausible results, etc.)

should be revealed by such a perturbation. Second, a mobilization requires an

explicit treatment of the joint interactions of several factors (e.g.,

international trade, the business cycle, available capacity, investment, etc.).

The formulation of the mobilization scenario used in the evaluation of DGEM

was a two-stage process. In the first, a mobilization baseline was constructed

which made explicit the perturbations about the business as usual values in the

DGEM data base. The mobilization baseline also provided a point of reference

for how the joint interactions of these perturbations affected the path of the

national economy. In the second, six key variables were varied in combination

subject to an experimental design. This enabled us to explore in detail certain

patterns which were uncovered in the mobilization baseline.

The previous discussion and material presented in Chapter 9 demonstrates

that there are at least four reasons why DGEM can be a useful tool for analyzing

the economic impacts of a military mobilization. First and foremost, DGEM was

designed to deal with the demand surges and resource constraints which are

expected during the transition from a peace-time to a war-time economy.

Specifically, DGEM explicitly addresses factors which affect both demand and

supply on an annual basis up through the year 2000. Second, the TUTOR provides

the user with a set of step-by-step instructions for incorporating information

on Defense and other government expenditures as well as economic and technical

considerations which specify how the economy mobilizes. Third, from a review of

the documentation, it became clear that much care went into the design,

development and testing of the model. Furthermore, the model's documentation is

quite complete and should be sufficient to: (1) set it up on the host system;

(2) execute a base-case simulation; (3) interpret the results of the base-case

simulation; (4) create, run, and interpret user-specified simulations. Finally,

the DGEM source code has been made available to a number of users, enabling them

to explore the major linkages within the model as well as trace the flow of

calculations for particular variables of interest.

117
In Chapter 6, a set of four user requirements were postulated. For

ease of reference, each requirement is summarized briefly; a point-by-point

review is then given. Requirement 1 focuses on model usage (i.e., the

operational model). Requirement 2 addresses the clarity and completeness of the

documentation for the conceptual and mathematical models. Requirement 3 focuses

on the relationships between the National Income and Product Accounts (NIPA) and

the model’s output. Requirement 4 is concerned with a variety of

economic-technical attributes.

In the discussion which follows, material is presented which compares

critically the information generated in the model evaluation process against

user requirements. This comparison enables us to conclude that DGEM is both an

appropriate and useful model for analyzing the types of problems anticipated

during a military mobilization.

Requirement 1

must have documentation sufficient to enable an analyst to:

set it up on the host system;

execute a base-case simulation;

interpret the results of the base-case simulation; and

create, run, and interpret user-specified simulations.

Narrative

Item a : The documentation for DGEM is quite complete and should be sufficient

for an analyst to set it up on the host system.

Item b : The TUTOR provides a convenient and essentially "fool proof" means for

specifying and executing a base-case simulation.

Item c : Several parts of the DGEM documentation provide detailed descriptions

of the base-case simulation. Furthermore, these descriptions provide valuable

insights into the DGEM analysis framework, which should be helpful to users in

specifying their own simulations.

Item d : The DGEM documentation and the structure of the TUTOR are sufficient to

enable an analyst to create, run, and interpret a user-specified simulation.

The TUTOR, through its nested set of menus, facilitates the process of

specifying a simulation. In many cases, questions posed to the analyst by the

TUTOR provide reference to the values of key variables contained in the

base-case simulation. Consequently, the analyst may specify major portions of

the desired simulation via a set of "perturbations" about the base-case

simulation.

The model

(a)

(b)

(c)

(d)

118
Requirement 2

The structure of the model should be clearly documented,

includes information on the model’s:

This requirement

(a) policy context;

(b) objectives;

(c) exogenous and endogenous variables;

(d) equation structure; and

(e) data base(s).

To the extent feasible, the user should be assured that the forecast values

of the "base-case" simulation have been compared to historically achieved

values. Any major discrepancies should be documented and, if possible,

explained

.

Narra tive

Item a : DGEM provides the potential user with significant latitude for

specifying policy options and for evaluating their economic merits. The TUTOR

is especially helpful in defining how the model can address mobilizationoriented policies.

Item b: It is anticipated that model users will exhibit a wide range of

objectives. The documentation (especially the User’s Guide) and the TUTOR

should be sufficient for defining the user's analytical objectives and for

translating them into a runstream for executing the model. At an aggregate

level, the TUTOR alone may be sufficient to specify an approach for evaluating

the economic impacts of both the increased defense purchases and resource

constraints (e.g., supply interruptions) anticipated under a military

mobilization. Some sectoral information is also available. For example, for

any of the 36 sectors treated within the DGEM analysis framework, the analyst

can constrain the inputs to or outputs from that sector. Such an analytical

approach can be useful in identifying bottlenecks or assigning priorities to

certain industries for a particular set of inputs.

Item c : Information on the model's exogenous and endogenous variables is

presented in 36 DGEM: The Dynamic General Equilibrium Simulation Model . This

information has been expanded, reflecting the code changes described earlier as

well as the incorporation of the TUTOR, and is recorded in Appendix E.

Item d : The equation structure of DGEM is well documented. The report, 36DGEM : The Dynamic General Equilibrium Simulation Model , includes explicit mathematical

descriptions of: (1) submodels of producer behavior, one for each of the 36

domestic producing sectors; (2) a model of consumer behavior; (3) balance

equations covering physical flows through the interindustry system equating

demand and supply quantities of each good or service transacted; (4) market

balance equations equating value of expenditure and receipts for each good or

service transacted; (5) financial identities aggregating value flows into

aggregate income, financial and economic accounts; and (6) government and rest

of the world accounts.

119
Item e : The DGEM database consists of 19,327 entries for each of 43 years

(i.e., 1958 through the year 2000). Of the 19,327 entries for each year 5,903

are endogenous variables, 1,643 are exogenous variables, and 11,781 are

coefficients. The estimation of the model over the historical period (i.e.,

1958 to 1974) used a database constructed by Jack Faucett and Associates. In

Chapter 8, three sets of base-case simulations were reviewed; they were: (1)

model performance over the historical period (i.e., simulations between 1959 and

1974); (2) model performance between 1975 and 1981, when published data on

certain key variables was available; and (3) the base-case projections (i.e.,

simulations between 1982 and year 2000). In evaluating DGEM’s performance

between 1975 and 1981, one must recognize that while many variables were aligned

to observed values, many were not. In most cases, the simulated values for

these compare favorably with information developed from published sources. For

example, comparisons between figures published by the Department of Labor and

values from the database for 1974 and 1980 showed close agreement in the

relative importance of each industry and the changing nature of that importance

over time.

Requirement 3

The model must be cross referenced to the National Income and Product Accounts

(NIPA). This minimum requirement provides a set of control totals, which may be

used to disaggregate the model's output to address sector specific issues.

Furthermore, it is desirable for the model to provide information on

those endogenous and exogenous variables likely to be of interest to decision

makers. This information may be useful either in gaining insights regarding the

"path” of the economy to a war-time footing or in preparing customized reports.

Narrative

Some of the final demand and expenditure accounts in DGEM are based on an

accounting system developed by Christensen and Jorgenson. This system was

specifically designed to identify and handle consistently variables involved in

economic growth. Because of this, the system differs slightly from the National

Income and Product Accounts (NIPA) which are the commonly used framework for

economic accounting. The principal difference between the DGEM and NIPA

accounting systems arises in the treatment of household sector capital. This

capital comprises residential property, automobiles, and other consumer

durables. Household capital is explicitly treated as capital in the household

sector accounts. Purchases of consumer durables are included in investment, not

in consumption, and the imputed value of services from this capital is

calculated and included in consumption. This yields a system of accounts in

which investment includes all additions to capital stock and in which

consumption comprises only current goods and services. The NIPA system differs

from DGEM in that it includes in consumption an imputation for the services from

owner-occupied residential structures but not those from other forms of consumer

durables. Consequently, a bridge is used to map between the two accounting

systems. The bridge comprises a set of stochastic equations and identities that

permit the DGEM final demand aggregates to be expressed in NIPA terms. The

stochastic equations map model variables such as real consumption, real

investment, price of consumption, and price of investment into the corresponding

NIPA variables. The identities then aggregate expenditure and output into gross

national product (GNP), real GNP , and the remaining aggregates in the standard

national income accounts.

120
Requirement 4

The model must incorporate the following economic-technical attributes:

(a) the increasing importance of international trade over the past decade;

(b) the business cycle concept;

(c) the changing composition of the Gross National Product (GNP);

(d) the concepts of investment, capital services, depreciation, and

emergency capacity;

(e) wage and price variables;

(f) both supply (e.g., capital and labor services available) and demand

(e.g., military requirements) concepts;

(g) an explicit treatment of both fiscal and monetary policy;

(h) dynamic characteristics whereby production and consumption decisions

in one period affect the economy in future periods; and

(i) bridges to or from key variables (e.g., from the NIPA components to

specific sectors as defined by the Commerce Department’s 4-digit

Standardized Industrial Classification (SIC)).

Narrative

Item a : DGEM addresses foreign trade both as a part of final demand (exports)

and as a source of goods and services (imports). Via the TUTOR, imports can be

specified on an industry-by-industry basis; tariffs or quantitative controls are

available for limiting imports. Exports can be altered industry-by-industry or

the overall volume of exports can be changed by keeping the composition

unaltered. Also, there are considerations for the balance of trade. The

balance of trade can be unconstrained or it can be subject to limits, reflecting

the constraints that normally exist.

Item b: Between 1982 and the year 2000 economic growth is projected to average

just over 3 percent per year. This represents continued growth at rates less

rapid than during the World War II to 1973 period but more rapid than during the

1973-1983 decade. This growth is slightly more rapid in the 1980’s than in the

1990 's. Economic growth tends to be uneven or cyclical rather than regular.

Hence, there are cycles around the overall growth trend. A feature of many

previous cycles has been the influence of federal policies, with expansionary

policy in the run up to an election producing strong growth then the

post-election tightening of policy leading to a recession. The correlation is

not perfect but it is certainly present. Therefore, an election-related cycle

is assumed to continue over the DGEM forecast period.

Item c : The different types of final demand (e.g., personal consumption

expenditures, investment, and government purchases) display very different

patterns over the business cycle. Government purchases are the most stable

since they reflect political rather than market forces. Although government

purchases may show uneven growth rates, these are not related to the business

cycle and generally even work to dampen the magnitude of the cycle. Personal

consumption is intermediate. Service and nondurables purchases are very stable,

durable purchases are cylical, so that overall consumption volumes show only a mild cycle. Private investment is the most volatile of the main spending

components. In recession years, investment growth is zero or even negative; in

the recovery years, investment volumes grow much more rapidly than the rest of

the economy. The composition of final demand between personal consumption

expenditures, private investment and government purchases (i.e., how the output

121
of the economy is used) is projected to show substantive change (see Table 8.5

where the composition of final demand in both current and constant dollar terms

is shown).

Item d : Capital markets are modeled explicitly by DGEM . The model of capital

markets within DGEM includes private saving, private investment, government

borrowing, and foreign borrowing. The government deficit plays an important

role in capital markets as modeled within DGEM because it represents a "firm"

demand for funds in those markets. In cases where the government deficit is

financed, the government claims a larger share of investible funds and tends to

"crowd out" private investment. DGEM provides a model of investment which

incorporates the business cycle concept with the potential for crowding out due

to deficit financing. Since net investment is related to the rate of change in

final output rather than the absolute level of output, the rate of change in

real investment within DGEM is quite volatile. Capital services are a function

of the level of the capital stock at the beginning of each year and the

efficiency with which that stock can be utilized. Consequently, even though

investment is quite volatile, capital services increase steadily over the

mobilization period. Both the efficiency with which the capital stock can be

utilized and the rate of depreciation of that stock are explicit in DGEM. This

is essential for any mobilization application because it enables the model to

capture cases where the existing capital stock is used more intensively (e.g.,

going to multiple shifts or increased utilization for existing multi-shift

facilities) and consequently is subjected to greater wear and tear. Emergency

capacity can be modeled indirectly by adjusting the efficiency with which the

overall stock of capital is utilized. This approach results in an increased

flow of capital services which supports the higher levels of output required in

a mobilization situation but ignores the net additions to the overall stock of

capital. In this important aspect DGEM differs from ROCS, another emergency

management model currently used by FEMA and DoD (see Appendix D) , which uses

explicit sector-by-sector estimates of emergency capacity. Two other concerns

regarding DGEM ' s modeling of capital markets should also be noted. ^ First, DGEM

assumes all inputs (including durable plant and equipment, land and working

capital) are instantaneously adjustable among firms and sectors. Consequently,

there is no notion of industry-specific capital plant and equipment. This has

as an implication that the production estimates from DGEM may be somewhat

optimistic. Second, if the capital service price formulation used in DGEM were

estimated using the theoretically appropriate ex ante cost of capital, then the

new parameter and elasticity estimates could differ considerably from those now

found in the DGEM data base.

Item e : Both wage and price variables are treated in the DGEM analysis

framework. Household behavior is modeled through spending-saving and

work-leisure decisions. Prices of each industry’s output are endogenous,

reflecting input prices, productivity, and input patterns. The price setting

procedure of DGEM is based on the following assumptions: (1) all sectors are

characterized by constant returns to scale; (2) there is perfect competition in

the input and output markets; and (3) all inputs are instantaneously adjustable

among firms and sectors. Given the above assumptions, it follows that economic

profits are zero at each point in time for each sector and that price equals

average and marginal cost. In summary, price setting takes place by setting

^ Berndt , E~! r7, op cit . , pp 11-6.

122
price equal to average cost (which in turn equals marginal cost). Prices are

therefore independent of the level of demand for the product. Once these

prices are set, absolute demands for outputs and inputs can then be obtained.

In view of the previously stated assumptions and a non-substitution theorem put

forward by Samuelson, all industry 1-0 coefficients are independent of the

structure and distribution of final demand. 1 On the labor side, labor supply is

the outcome of consumer's optimizations, while labor demand is the sum of

demands from optimizations in each of the 36 producing sectors. For any given

real wage rate, labor supply is determined endogenously. The number of

unemployed workers (determined exogenously) is then subtracted from the labor

supply to get a "net" labor supply. Real wage rates are then adjusted so that

the leisure-work choice decision minus exogenous unemployment results in a net

labor supply that just equals total labor demand. Since unemployment is modeled

exogenously, the wage rates in DGEM are not those that clear labor markets but

rather those that clear markets given exogenously set unemployment rates. One

may therefore conclude that while the model has very appealing general

equilibrium features, the labor market is classical in that unemployment cannot

be explained.

Item f : The DGEM analysis framework is based on the application of econometric

modeling to input-output analysis. Where input-output analysis assumes fixed

input-output coefficients at any point in time DGEM provides for flexible

input-output coefficients induced by price variations in primary inputs which

are associated with economic policies or anticipated contingencies. The

complete model consists of an interindustry model and macroeconomic model that

integrates demand and supply conditions for consumption, investment, capital and

labor. The macroeconomic model separates economic activity into four types of

goods and services: the output of consumption and investment goods; and the

inputs of capital and labor. The demand and supply of each commodity is

calculated as well as market clearing prices and quantities. A household

submodel determines the demand for consumption goods, supply of savings, and the

supply of labor. A production function relates the output of consumption and

investment goods to the inputs of capital and labor services, given the level of

technical efficiency. The two other components of GNP , government expenditures

and exports, are exogenous to the model. The interindustry model determines

interindustry transactions for 36 sectors, the demand for primary inputs (labor,

capital, and competitive imports), the allocation of GNP as final demand among

the sectors, all total sector outputs. The technology of each producing sector

is represented by a price possibility frontier that determines the supply price

of output as a function of the prices of primary and intermediate inputs and the

level of technical efficiency.

Item g : The version of DGEM tested in this study did not include a monetary

policy model. ^ Via the TUTOR, a wide range of fiscal policy measures which are

available to the U.S. government for use during or after a major event, such as

military mobilization, may be modeled. Some of these policies involve purchases

of goods and services; some policies control the allocation and use of available

resources, in particular labor and capital; some policies control production

activities, the level of operation of specific industries; and some policies

affect fiscal conditions, such as government spending, taxation, and deficits.

Berndt, e7 r7, op cit , pp 11-6.

o A newer version of DGEM is currently available which incorporates both

monetary and fiscal policy.

123
These types of policies are included and can be specified by the analyst. For

example, fiscal policy involves government revenues and expenditures, and their

difference, the government deficit. Government revenues, primarily taxes,

reduce the level of private disposable income and spending. Government

purchases are themselves an element of spending and directly add to private

spending in demand for goods and services. The government deficit affects the

capital markets, since the deficit is financed by government borrowing. The

larger the deficit, the more private capital is preempted (i.e., crowded out),

reducing the availability of funds for private spending, and showing up

particularly in its effect on private investment. Each of these policy measures

is available to the government and each can be adjusted by the analyst in

designing the analysis of the event.

Item h : DGEM models the U.S. economy as a sequence of annual equilibria that

determine demand, supply, and the relative prices for all commodities. In each

period the supply of capital is fixed initially by past investments and

depreciation. Variations in demand for capital services by producing sectors

and households affect the price, but not the quantity of capital services.

However, present investment plans will alter the future level of the capital

stock. Similarly, in each period the available labor time is fixed by past

demographic developments, but variations in the demand for labor time by the

producing sectors and by the households for leisure consumption affect the price

of labor and the allocation of labor time between work and leisure. In

addition, the supply of saving is simultaneously determined in the household

model and must be in equilibrium with respect to investment demand. In each new

period, an equilibrium is determined for all factor and product markets by

integrating a macroeconomic model with an interindustry model (see the

discussion under Item f).

Item i : At the present time, DGEM consists of 36 producing industries. More

precisely, the DGEM analysis framework includes: (1) inputs to each industry

from each of the 36 producing industries and from capital, labor and competitive

imports; and (2) outputs from each industry to each of the 36 producing

industries and to final demand —• personal consumption, investment, government

purchases (specifically, federal defense, federal nondefense, and state and

local government), and exports. Bridge tables, either within DGEM or between

the TUTOR and DGEM, are of crucial importance for most military applications.

The reasoning behind the previous statement is that bottlenecks are more likely

to occur as the sectoral level of detail is increased. Conversely, the higher

the level of aggregation, the less likely is the analyst to be able to identify

a potential bottleneck. Based on the previous discussion, it should be clear

that bottlenecks within DGEM ' s 36 sector framework can be identified, analyzed

and to a certain extent alleviated. The question thus becomes one of sectoral

detail. The discussion in Appendix D, where a multimodel approach to military

mobilization is reviewed, expands upon this subject. It is also worth noting

that the sectoral level of detail of DGEM differs significantly from two other

military mobilization models, ROCS (115 sectors) and DEIMS (400 sectors), as

well as the BEA benchmark input-output table (537 sectors). This lack of

sectoral detail is probably the most serious deficiency of DGEM with regard to

military mobilization applications.

Based on our review of DGEM, both the model’s source code and documentation and

the TUTOR, it is clear that the addition of two bridge tables would

substantially enhance the ability of DGEM to provide a detailed analytical

framework for a wide variety of military mobilization applications. The first

124
is a "pre-processor" which incorporates a bridge into the TUTOR between the 55

defense procurement categories used in DEIMS and DGEM ' s 36 producing sectors.

Alternatively, the "pre-processor" bridge table could be between DoC's defense

procurement categories list and DGEM ' s 36 producing sectors. The discussion in

Appendix D addresses some of the pros and cons of both DoD's and DoC’s bridge

tables between their respective defense procurement categories and the economy's

producing sectors. The second is a "post-processor" which translates the DGEM

output into a form which is consistent with DoC's 4-digit Standard Industrial

Classification (SIC) level. If this were done, the output of DGEM would be

consistent with the BEA benchmark input-output table. We note that the

"pre-processor" bridge table discussed earlier may be incorporated in a

relatively straight-forward manner. This is because much of the logic has

already been incorporated into the TUTOR. Recall that government purchases are

separated into three types of government activity: (1) federal defense; (2)

federal nondefense; and (3) state and local government. In turn, each type of

government activity involves purchases, spread over a number of supplying

industries. For federal defense purchases, a rudimentary bridge table, which

distributes defense purchases from the 55 DoD procurement categories to the 36

producing sectors, is included in the TUTOR. The analyst can therefore control

defense spending in a variety of ways within this structure. Overall levels of

defense spending can be altered, leaving the composition unchanged, or the

composition and level can be changed. Specific defense procurement categories

can be altered, year-by-year. Federal nondefense purchases and state and local

government purchases are spread over the 36 producing sectors. Federal

nondefense purchases can also be altered in total or industry-by-industry.

State and local purchases can be altered in total, although the composition

remains the same. The development of a "post-processor" bridge table is

somewhat more complex, but, in principle, most of the data exists to go from the

NIPA components to the 4-digit SIC level of detail.

^ The discussion in Appendix D outlines one way in which this activity might be

accomplished

.

125
REFERENCES

Alsmiller, J.R., et al * Adjoint Sensitivity Theory and Its Application to Leap

Model 22C . Oak Ridge, TN : Oak Ridge National Laboratory, ORNL/TM-7 789 , 1981.

Chapman, R.E., R.G. Hendrickson and S.F. Weber. A Survey of Techniques for

Evaluating Emergency Planning Models and Data Bases . Gaithersburg, MD:

National Bureau of Standards, NBSIR 84-2963, 1984.

Comprehensive Emergency Management: A Review Leader’s Guide . Washington, DC:

National Governors’ Association, 1981.

Computer Model Documentation Guide . Gaithersburg, MD: National Bureau of

Standards, Special Publication 500-73, 1981.

Filliben, J.J. DATAPLOT - Introduction and Overview . Gaithersburg, MD:

National Bureau of Standards, Special Publication 667, 1984.

Filliben, J.J. "Testing Basic Assumptions in the Measurement Process," in

Validation of the Measurement Process . New York: American Chemical Society,

ACS Symposium Series No. 63, pp . 30-113, 1977.

Gass, S.I. "Documenting a Computer-Based Model," Interfaces , Vol. 14, No. 3,

1984, pp. 84-93.

Gass, Sd. "Decision-Aiding Models, Validation, Assessment, and Related Issues

for Policy Analysis," Operations Research , Vol. 31, No. 4, 1983, pp . 603-631.

Gass, S.I. (ed.). Utility and Use of Large-Scale Mathematical Models . Gaithersburg, MD: National Bureau of Standards, Special Publication 534, 1976.

Gass, S.I. and S. Parikh, "Credible Baseline Analysis for Multi-Model Public

Policy Studies," in Energy Model and Studies , B. Lev, (ed.), North-Holland , 1983.

Gass, S.I. and S. Parikh, "On the Determinants of Credible Analysis," in

Mathematical Modeling in Science and Technology , X.J.R. Avula (ed.), Pergamon

Press, 1984.

Guidelines for Documentation of Computer Programs and Automated Data Systems . Gaithersburg, MD: National Bureau of Standards, FIPS PUB 38, 1976.

Harris, C.M. Issues in Sensitivity and Statistical Analysis of Large-Scale , Computer-Based Models . Gaithersburg, MD: National Bureau of Standards, NBS GCR

84-466, 1984.

Harris, C.M. Computer Generation of Latin Hypercube Sampling Plans . Gaithersburg, MD: National Bureau of Standards, NBS GCR 84-476, 1984.

Harris, C.M. "An Assessment of Climatological Uncertainties Using Monte-Carlo

Analysis," in Selected Assessment Strategies Applied to Short-Term Energy

M odels . Gaithersburg, MD: National Bureau of Standards, NBSIR 83-2672, pp.

30-54, 1983.

126
Hendrickson, R.G. A Survey of Sensitivity Analysis Methodology . Gaithersburg,

MD: National Bureau of Standards, NBSIR 84-2814, 1984.

Hudson, E.A. and Jorgenson, D.W. The Long Term Interindustry Transactions

Model: A Simulation Model for Energy and Economic Analysis , Washington, DC:

Federal Preparedness Agency, 1979.

Hudson, E.A. and Jorgenson, D.W. "U.S. Energy Policy and Economic Growth,

1975-2000, "The Bell Journal of Economics and Management Science , Vol. 5,

No. 2, Autumn 1974, pp. 461-514.

Hudson, E.A. and Jorgenson, D.W. "Assessment and Selection of Models for Energy

and Economic Analysis," in Validation and Assessment Issues of Energy Models , S.I. Gass (Ed.), Gaithersburg, MD: National Bureau of Standards, NBS

Special Publication 569, 1980, pp. 431-444.

McKay, M.D., W.J. Conover, and R.J. Beckman, "A Comparison of Three Methods for

Selecting Values of Input Variables in the Analysis of Output from a Computer

Code," Technometrics , Vol. 21, No. 2, 1979, pp . 239-245.

Simulations of the 36 DGEM Over the Historical Period , Lexington, MA: Data

Resources, Inc., 1980.

Test and Sensitivity Simulations of the 36 Sector Dynamic General Equilibrium

Model , Lexington, MA: Data Resources, Inc., 1980.

The 36 Sector Model of Household Behavior , Lexington, MA: Data Resources,

Inc., 1979.

The 36 Sector Model of Producer Behavior , Lexington, MA: Data Resources,

Inc., 1979.

Tocher, K.D. The Art of Simulation . Princeton, NJ : D. Van Nostrand, 1963.

Weisbin, C.R., R.W. Peele, and A.S. Loebl . An Approach to Evaluating Energy- Economy Models . Oak Ridge, TN : Oak Ridge National Laboratory, 0RNL-5742, 1981.

127
APPENDIX A: CRITICAL STAGES IN THE DEVELOPMENT OF LARGE-SCALE

COMPUTER-BASED SYSTSEMS 1

The purpose of this appendix is to describe those factors to consider when

managing large-scale, computer-based model development activities. The

five-phased approach for the factors presented in this appendix can greatly

improve the management of these activities and make large-scale, computer-based

models more responsive to user needs. The more responsive these models are to

user needs the more effective and efficient they should be in providing

information to the decision-making process.

The following factors are the product of an analysis of management

weaknesses which are inherent in modeling development problems. These factors

also suggest procedures which are intended to serve as a reference document

for managing the development of large-scale, computer-based models.

Recommendations for the management of large-scale, computer-based model

development efforts are divided into five separate phases: problem definition;

preliminary design; detailed design; evaluation; and maintenance. In each phase

we describe suggested specific duties and responsibilities of the user and the

developer. Requirements for documentation should be met as necessary in each of

the five phases.

The problem definition phase is primarily intended to describe those tasks

that should be carried out by the user before agreeing to a model design.

Additionally, the phase also describes design considerations and when

appropriate, the contractual relationships which the user must determine before,

deciding whether a preliminary design effort should begin. During this phase,

the user should be acquiring a clear definition of the problem. If the

information being acquired during this phase indicates development should be

stopped before the end of the phase, the user should be prepared to terminate

development at this point. To determine whether to continue into the next

phase, the user should consolidate and thoroughly review work completed during

this phase. At this point the user should be confident that his needs will be

met by the model without making changes to decisions that have been made. Once

the decision is made to proceed to the preliminary design phase, a report should

be prepared which provides specific guidelines to be followed in the remaining

phases. These guidelines represent (contractual) responsibilities for the

development of the model.

The preliminary design phase includes specification of the information

content, general programming logic, and model algorithms necessary to develop a

useful model. Preliminary design of the model should be conducted by the

developer with information input and direction provided by the user. At

completion of this phase, the user should consolidate and thoroughly review work

completed on these factors. At this point the user should be confident that all

of the specifications necessary to develop a useful model have been identified

^Portions of this appendix are excerpted from: Gass, S. I. (editor),

Utility and Use of Large-Scale Mathematical Models . Gaithersburg, MD:

National Bureau of Standards, Special Publication 534, 1976.

128
and agreed to, and will not need to be changed during the following phases. The

user must then determine the contract or work agreement process to be followed.

The contract or work agreement should include specifications for the model and

the control, documentation, and report requirements of the following two or

three phases, as applicable (detailed design, evaluation, and maintenance).

In the detailed design phase the developer designs the model logic and

prepares detailed programs. Briefings should be held periodically between the

the developer and the user. One purpose of these briefings is to provide the

user with the knowledge and confidence necessary to apply the model. During

this phase, the user should continuously reevaluate the design being implemented

and, if necessary, should recommend minor changes within the scope of the

contract or work agreement. The user should always be available to the

developer to answer questions and provide needed information. Before continuing

into the next phase, the user should consolidate and thoroughly review the work

completed in this phase. This review should provide the software manager with

sufficient updated information to determine the adequacy and responsiveness of

the development effort at this point. If the model has been adequately

developed to meet the user's needs, procedures for evaluation of the model

should be established, documented and carried out in the next phase.

The evaluation phase provides for the final check of the model as a whole.

This operational testing supplements the evaluation of individual model programs

conducted during earlier phases. The user has ultimate responsibility for

evaluating the adequacy of the developed model. Actual evaluation of the model,

however, may be done by the user, both user and developer, and/or an independent

third party. Evaluation of the model is done according to the evaluation

criteria and test plan established in the problem definition phase. Evaluation

includes the determination of compliance with previously established agreements.

At the completion of this phase, the user should prepare a report based on the

evaluation work. This report should include the user’s overall satisfaction or

dissatisfaction with the modeling effort and the final model design. A decision

should then be made as to whether or not the model is usable.

In the maintenance phase the Federal agency that sponsored model

development establishes procedures for updating the model and for obtaining from

the users their comments on the adequacy of the model and whatever changes they

made to the model. The developer should be available for assisting the user

after completion of model development in accordance with the agreement

established during the preliminary design phase. Agency management should

obtain from the user an abstract of the model application to provide information

to others. Periodic reports should be prepared showing any changes made to the

model and indicating the current status of the model. When the model can no

longer meet user needs, its maintenance should be stopped and the status should

show it is not usable.

In summary, the factors represent some suggested procedures for model

development. They are intended to illustrate at least one method of enhancing

the user’s perspective of modeling and reducing the chance of failure during

model development. They are presented in a form that: (1) distinguishes five

separate phases of model development; (2) promotes a more thorough early

investigation of the nature of the problem and of possible solution methods; and

(3) provides a method of controlling the commitment to a modeling effort during

the model’s development period.

129
APPENDIX B: SPECIALIZED TECHNIQUES FOR PERFORMING A DATA AUDIT*

As discussed in Chapter 4, the statistical techniques used in a data audit

fall into two major classes: (1) exploratory data analysis techniques; and (2)

classical statistical techniques. Although the approach taken for each class of

techniques differs significantly, both classes focus on the same key measures.

These measures are: (1) central tendency: (2) distributional form; and (3)

relational properties. In the text which follows, emphasis will first be

placed on exploratory data analysis techniques <> The discussion of classical

statistical techniques follows immediately.

An important summary of the central tendency of a data set is a box and

whisker plot. This graphical display highlights five important characteristics

of the data set. The box, usually aligned vertically, encloses the

interquantile range of the data, with the lower line identifying the 25th

percentile and the upper line the 75th percentile. A line sectioning the box

displays the location of the median within the interquantile range. The

whiskers (line segments) at either end may extend to the extreme values or, for

large data sets, to the 10th or 90th percentiles or 5th and 95th percentiles.

Box plots are particularly useful for comparing the measures of central tendency

for two or more data sets or different segments of the same data set.

Two techniques which are particularly useful for measuring the central

tendency of a data set as well as providing a summary of distributional form are

histograms and stem and leaf diagrams. Both techniques sort the data from

smallest to largest and assign them to a group. Generally, the analyst must

select the interval widths or the number of groups prior to the analysis. Once

this has been done, a histogram becomes a graphical representation of a frequency distribution. Histograms typically use the heights of bars to exhibit

the relative frequency of occurrences within each group of values within the

data set. The main uses of a histogram in a data audit are: (1) to condense a set of data for easy visual comprehension of general characteristics such as

typical values, dispersion, and shape; (2) to suggest probability models or

transformations for subsequent analysis; and (3) to detect unexpected behavior

or unusual values in the data.

An important variant of the histogram is the stem and leaf diagram. A stem

and leaf diagram has the appearance of a histogram placed on its side. Each

group defines a stem and each piece of information (item) on a stem is a leaf.

The stem appears on the left side of a vertical line and the leaves appear on

the right side of the vertical line. The leaves may be labeled so that the

diagram preserves information distribution within groups. An additional feature

of these diagrams is a summary column which represents the cumulative number of

observations from each end of the data distribution (termed depths) up to the

group containing the median. The major advantages of stem and leaf diagrams are

that they show: (1) clearer separation into groups; (2) unsymmetric trailing

off; (3) unexpectedly popular and/or unpopular values; (4) where the data are

centered; and (5) how widely the data are spread.

* Por tions of the text which follows are abstracted from: Filliben, J. J.

"Testing Basic Assumptions in the Measurement Process," in Validation of the

Measurement Process , New York: American Chemical Society, ACS Symposium Series

No. 63, pp. 30-113, 1977.

130
Although histograms and stem and leaf diagrams provide some insight into

the underlying distribution of the data, probability plots permit specific

hypotheses about the distributional properties of the data to be tested. A

probability plot is a graphical tool for assessing the goodness of fit of some

hypothesized distribution to an observed data set. A probability plot is (in

general) simply a plot of the observed ordered (smallest to largest)

observations , Y-^, on the vertical axis versus the corresponding typical ordered

observations , M-j_, on the horizontal axis based on whatever distribution is being

hypothesized. Thus, for example, if one were forming a normal probability plot,

the following n coordinate plot points would be formed i (Yj, M^) (Y£,

M2)»**(Yn , M n ) where Y^ is the observed smallest data point, and M]_ is the

theoretical "expected" value of the smallest data point from a sample of size n normally distributed points.

Similarly, Y2 would be the second smallest observed value and M2 would be

the "expected value" of the second smallest observation in a sample of size

n from a normal distribution. Thus, in forming a normal probability plot,

the vertical axis values depend only on the observed data, while the

horizontal axis values are generated independently of the observed data and

depend only on the theoretical distribution being tested or hypothesized

(normality in this case) and also the value of the sample size n. A

probability plot is thus in simplest terms a plot of the observed versus

the theoretical or "expected."

The crux of the probability plot is that the i^ ordered observation in a sample of size n from some distribution is itself a random variable which has a distribution unto itself. This distribution of the i

c ^ ordered observation can

be theoretically derived and summarized, as can any other random variable. One

can then pose the relevant question as to what single number best typifies the

distribution associated with a given ordered observation in a sample of size n. In view of the resistance of the median to misbehavior in the data, the Mp on

the horizontal axis of the probability plot is taken to be the median of the

distribution of the ith ordered observation in a sample of size n from whatever

underlying distribution is being tested.

It is noted that the set of M^ as a whole will change from one hypothesized

distribution to another — and therein lies the distributional sensitivity of

the probability plot technique. For example, if the hypothesized distribution

is uniform, then a uniform probability plot would be formed and the M-^ will be

approximately equi-spaced to reflect the flat nature of the uniform probability

density function. On the other hand, if the hypothesized distribution is

normal, then the M-j_ will have a rather sparse spacing for the first few (M]_, M2,

M3...) and last few („.., M n_2» M n-i , M n ) values but will become more densely

spaced as one proceeds toward the middle of the set (..., M n_i/2> M n/2>

M n+i/2>«*»)» Such behavior for the M is of course reflecting the bell-shape of

the normal probability density function.

How does one use and interpret probability plots? In light of the above,

it is seen that if in fact the observed data do have a distribution that the

analyst has hypothesized, then (except for an unimportant location and scale

factor which can be determined after the fact) the Y^ and M-j_ will be

near-identical for all i, that is, over the entire set. Consequently, the plot

of Y-l versus Mj_ will be near-linear . This linearity is the dominant feature to

be checked for in any probability plot. A linear probability plot indicates

131
that the hypothesized distribution, D

Q , gives a good distributional fit to the

observed data set. This combination of simplicity of use along with

distributional sensitivity makes the probability plot an extremely powerful tool

for data analysis.

Graphical analysis techniques are particularly helpful at analyzing relational

properties. Techniques such as scatter diagrams, run sequence plots and lag -1

autocorrelation plots are important tools for the analyst. A usual first step

in analyzing the relationship between two variables is to construct a scatter

diagram, a collection of plotted points representing the pairs (x^, y-^),

i — 1

, .

. , n •

Scatter diagrams are also a useful precursor to regression analyses. The

objective of regression analyses is to develop a- mathematical relationship

between a measured response or dependent variable, y, and two or more predictor

or independent variables, xj , X2,....Xp. A usual initial step is to plot y

versus each of the x variables individually and to plot each x versus each of

the other x’s. This results in a total of p + p(p - 1 ) /

2

plots. Plots of y

versus x-j_, enable one to identify the x-^ variables which appear to have large

effects, to assess the form of a relationship between the y and x-^ variables,

and to determine whether any unusual data points are present. Plots of Xj_ vs.

Xj , i t j, help to identify strong correlations that may exist among the

predictor variables.

The run sequence plot is the next tool for analyzing relational properties.

Although the collection of data points may or may not have been equispaced in

time, the ordering of the data in time (i.e., the run sequence) is usually

well-defined. In cases where the data acquisition rate is such that there is an

equal time-spacing between collected data points, the run sequence has a natural

analogue to a possibly relevant factor (time). In other cases, when the data

acquisition rate is variable or random, no such analogue exists -- yet the run

sequence "factor" is still frequently of interest. The run sequence plot

(defined as a plot of versus i) is the simplest possible data plot and yet is

almost invariably informative. The run sequence plot is the recommended first

step in assessing whether such basic assumptions as fixed location, fixed

variation and the implicit corollary assumptions hold-up, namely that the data

set is outlier-free. With regard to economic data, the run sequence plot may

help to identify seasonal and/or other trends.

The lag -1 autocorrelation plot is defined as a plot of Yj_ versus Y^_i over

the entire data set; that is, the following n-1 points are plotted: (Y2> Y]_),

(Y3, Y2), (Y4, Y3),...(Yn , Yn_i). The lag -1 autocorrelation plot is sensitive

to randomness assumptions governing the data. If the data are random, then

adjacent observations will be uncorrelated and the plot of Yj_ versus Y^_^ will

appear as a data cloud with no apparent structure. However, if the data are

not random and if adjacent observations do have some autocorrelation this

structure will frequently manifest itself in the autocorrelation plot.

Part of the motivation of the previous discussion was to demonstrate how

techniques based on exploratory data analysis complement classical statistical

analysis techniques. In the discussion which follows, we shall introduce some

of the most important classical tehniques with particular emphasis being placed

on how these techniques can draw on information produced through an exploratory

examination of the data.

132
As a first cut, it may be helpful to determine the minimum and maximum

values of the variables within the data set. The range is then the difference

between the maximum and minimum values. It is also helpful to determine the

mode, or most popular value, as well as the median. The mean is the most common

measure of central tendency. Often referred to as the average, it is merely the

sum of the individual values for each case divided by the number of cases. The

variance of the sample, s^, is a measure of the dispersion of the data about the

mean. This statistic is one way of measuring how closely the individual values

of the variable cluster around the mean. Mathematically, it is the average

squared deviation from the mean. Squaring the deviations from the mean takes

into account all differences from the mean, including negative differences, and

gives additional weight to extreme cases. The standard deviation is another

measure of dispersion about the mean; it is the square root of the variance.

The advantage of the standard deviation is that it has a more intuitive

interpretation, being based on the same units as the original variable. If an

infinite number of equal-sized samples were drawn from a given population, the

mean of each sample would be an estimate of the true population mean, but not

all of them would be identical. The pattern of these means would actually

constitute a normal distribution and would have a standard deviation. The

standard deviation of this distribution is the standard error. Thus, the

standard error helps to determine the potential degree of discrepancy between

the sample mean and the usually unknown population mean. Skewness is a statistic needed to determine the degree to which a distribution of cases

approximates a normal curve, since it measures deviations from symmetry. The

measure of skewness is sometimes called the third moment and will take on a value of zero when the distribution is completely symmetric. A positive value

indicates that the cases are clustered more to the left of the mean with most of

the extreme values to the right. A negative value indicates clustering to the

right. Kurtosis is a measure of the relative peakedness or flatness of the

curve defined by the distribution of cases. A normal distribution will have a kurtosis of zero. If the kurtosis is positive, then the distribution is more

peaked (narrow) than would be true for a normal distribution, while a negative

value means that it is flatter. Kurtosis is sometimes called the fourth

moment

.

Partial correlation provides the analyst with a single measure of

association describing the relationship between two variables while adjusting

for toe effects of one or more additional variables. In essence, partial

correlation enables the analyst to remove the effect of the control variable

from the relationship between the independent and dependent variables without

physically manipulating the raw data. In partial correlation the effect of the

control variable(s) is assumed to be linear throughout its range, and it is this

linear assumption that makes partial correlation possible.

Partial correlation can be used in a wide variety of ways to aid the

researcher in understanding and clarifying relationships between three or more

variables. When properly employed, partial correlation becomes an excellent

technique for uncovering spurious relationships, locating intervening variables,

and can even be used to help the researcher make certain types of causal

inferences. Partial correlation can be a very helpful tool for enabling the

researcher to locate spurious relationships. A spurious correlation is defined

in a relationship between two variables, A and B for example, in which A

f s

133
correlation with B is solely the result of the fact that A varies along with

some other variable, C for example, which is indeed the true predictor of B. In

this case, when the effects of C are controlled, held constant, etc., B no

longer varies with A.

Another important feature of partial correlation lies in its ability to aid

the analyst in a search for intervening linking variables. While there is no

statistical difference between the computation of partials employed to locate

spurious relationships and those used to determine intervening variables, the

conceptual issues are different enough to merit separate treatment. The search

for intervening variables is highly related to the issue of causality insofar as

the researcher wishes to make statements of the sort: A leads to B which in

turn leads to C. While partial correlation can be of great assistance in such

problems, theoretical considerations become much more important in these types

of situations.

Multiple regression is a general statistical technique through which one

can analyze the relationship between a dependent variable and a set of

independent or predictor variables. Multiple regression may be viewed either as

a descriptive tool by which the linear dependence of one variable on others is

summarized and decomposed, or as an inferential tool by which the relationships

in the population are evaluated from the examination of sample data. Although

these two aspects of the statistical technique are closely related, it is

convenient to treat each separately, at least on a conceptual level. Since the

method (as a descriptive tool or inferential tool) can be used for a variety of

related purposes, we will illustrate only a few of its most common applications.

The most important uses of the technique as a descriptive tool are: (1) to find

the best linear prediction equation and evaluate its prediction accuracy; (2) to

control for other confounding factors in order to evaluate the contribution of a specific variable or set of variables; and (3) to find structural relations and

provide explanations for seemingly complex multivariate relationships.

For every use of regression as a descriptive tool, there is usua

corresponding question of statistical inference — whether one can ge

the results of the sample observation to the universe. The problems

statistical inference can be conveniently grouped into two general ca

estimation and hypothesis testing. The purpose of estimation is to f

most likely population parameters from the examination of sample obse

The main focus here is in delineating a particular value or values fo

population. The analyst may, on the other hand, focus on evaluating

hypotheses about the population. That is, instead of asking what val

population parameter is likely to have, one may simply test the null

that its value is zero against the alternative hypothesis that its va

greater or less than zero.

lly a neralize

of

tegories

ind the

rva tions

.

r the

various

ue a

hypothesis

lue is

As mentioned in the previous subsection, any thorough statistical analysis

should focus on an analysis of residuals. If a fitting procedure, such as

multiple regression is used, then the residuals which remain are assumed to be

random. The runs test is therefore of particular importance since it is

specifically used to test for randomness. The underlying theory behind the runs

test is that if the data are random and if the sample size is known (say n=50),

the number of runs up of length 1, of length 2, etc .

,

may be considered as

random variables whose expected values and standard deviations can be calculated

134
from theoretical considerations and these calculations will not depend on the

(unknown) distribution of the data but only on its assumed randomness. Having

computed such theoretical values, the final step in the test is to compute from

the data the observed number of runs (up) of length 1, of length 2, etc . , and

then determine how many theoretical standard deviations that this observed

statistic falls from the theoretically expected value. This is most easily done

by formation of the standardized variable:

[N± - E(Ni )]/SD(Ni )

where N-j_ is the observed number of runs (up) of length i, E(N-j_) is the

theoretical expected number of runs up of length i and SD(N-[) is the theoretical

standard deviation of the number of runs of length i. For random data, one

would expect values of say, ±1, ±2, ±3, i.e., the observed number of runs of

length 1 should be only a few (at most) standard deviations away from the

theoretical expected value for the number of runs of length 1. For nonrandom

data, the deviations from the expected values will, of course, be much larger

and this is the crux of the runs test.

A great deal of data in business, economics and engineering occur in the

form of time series, where observations are dependent and where the nature of

the dependence is of interest in itself. The body of techniques available for

the analysis of such series of dependent observations is called time series

analysis. Box-Jenkins techniques are concerned with the building of models for

discrete time series. In particular, time series are often best represented by

nonstationary models in which trends and other pseudo-systematic

characteristics which can change with time are treated as stochastic rather than

deterministic phenomena. Furthermore, economic time series often possess marked

seasonal or periodic components themselves, capable of change and needing

(possibly nonstationary) seasonal models for their description. Much of the

motivation behind the Box-Jenkins approach can be seen through reference to

Wold’s decomposition theorem. This theorem demonstrates that any covariance

stationary process can be decomposed into two parts, r) t and where r) t is a purely deterministic part and c

3

t

is the purely stochastic part. A corollary of

Wold’s theorem is that ?

t can be expressed as a one-sided moving average process

of infinite order

£t ” a t ^l a t-l v^2a t-2 " ••• or

00

" t = 'l ~ °j a t-j

j=o

where {a t } is a sequence of uncorrelated random variables and

E(a t ) = 0 for all t

E(a t )2 = for all t

E(a|-a s

) = 0 if t£s

135
The Box-Jenkins approach seeks to induce stationarity into the stochastic

process {Y t } by designing filters such that

dz d| X

Z t = A s A ( (

Y

t +u ) -l)/\

is a covariance stationary process where

A = Yt-Yt-i

A s = Y t -Y t . s X

((Y t +u) -l)/\ = the Box-Cox power transformation*

The mechanics of fitting such a model (i.e., obtaining estimates for dj , d 2 and

X) involve an analysis of the behavior of the sample autocorrelation function

and the sample partial autocorrelation function. With respect to Wold's

decomposition theorem, the Box-Jenkins approach is based on two assumptions.

d 2 di

The first assumption is that the differencing operators, A s A , and the power

transformation force r\ t to zero. The second assumption is that

£ t =

<l>l E t-l + ••• -N>pe t -p+a t“01 a t-l “ ••• ” Qqa t-q

or

4>(B)e t = Q(B)a t

P

where ( B ) = l-f})|B - ... - 4>qB

q

9(B) = 1-9 iB - ... - 9

q

B

B = the backshift operator

Both assumptions permit the infinite number of parameters in the original model

to be reduced to p+q+1 parameters, p autoregressive parameters, q moving average

parameters, and the variance of a t , a a

. The previous statement shows that the

types of models produced by Box-Jenkins techniques have a desirable property:

they are parsimonious. Box-Jenkins models are parsimonious because they employ

the smallest possible number of parameters for adequate representation.

136
APPENDIX C: THE EVALUATION OF MULTI-MODEL SYSTEMS 1

In the previous sections of the report, the focus has been on describing

evaluation considerations as they apply to a single, self-contained model.

However, due to limitations on time and funds, many problems encountered by FEMA

can only be studied in an expedient manner by combining existing models into a multi-model system in which the separate models receive data from and transmit

data to one another. For such systems, the analytic aspects of evaluation,

sensitivity analysis, data audits, and so on, are confounded and more complex

than for a single model. The situation is further complicated as the individual

models that are combined usually have evaluative and validation problems of

their own, and the applicability of the individual model is stretched to meet

the requirements of the multi-model area. In such situations, validation

testing of the total multi-model system is not possible; such testing is usually

ignored. The model analysts must, however, provide some evidence that the

multi-model system is credible.

The main criterion of credibility is the notion of consistency. In any

multi-model study, there is a requirement to establish a set of initial

conditions — baseline assumptions — about which the analysis takes place.

This requirement exists as the problem setting must be delineated; the models

cannot be operated without fixing the many input parameters and initial variable

settings that define the problem area. A complete statement of the baseline

assumptions is essential in a multi-model analysis. It is termed the baseline

scenario. Such a statement represents the major control mechanism by which the

models and analysts are coordinated and consistency maintained throughout the

analysis. That is, a proper baseline scenario ensures that the models (and

analysts) are addressing the same problem, are using the same assumptions, and

are starting with the same initial data.

A scenario, in general, represents a description of the problem area being

analyzed; it is based on the model builders’ and users’ perception of the

problem and the requirements of the models. Further, a scenario describes the

factual and/or postulated settings of the situation, including, in model-based

terms, descriptions of the future in which the analysis takes place. Any

model-based analysis, especially a multi-model analysis, depends upon the

construction, definition, interpretation and acceptance of the baseline and

other scenarios by which the multi-model system is coordinated and comparative

analyses takes place.

Although a scenario represents a statement about the future, it cannot be

interpreted as a prediction, but only as a plausible future whose implications

are to be analyzed by the modeling exercise. Scenario-based analyses are

conducted by varying assumptions and/or data to determine how sensitive the

results are to uncertain estimates of the future, for example, measuring the

impact of uncertain estimates of inflation on GNP. Such variations usually are

conducted about a reference or baseline scenario. The baseline scenario may be

considered to be a description of the status quo and its most plausible

projection into the future. The baseline scenario thus provides the reference

"

1 Portions of this appendix are excerpted from: S. I. Gass, and S. Parikh,

"Credible Baseline Analysis for Multi-Model Public Policy Studies,” in Ene rgy

Models and Studies B. Lev (ed.), Nor th-Hol land , 1983.

137
conditions that are input to the multi-model exercise. It enables the analysts

to produce a scenario impact analysis that is the result of the analysts

translating the baseline assumptions into corresponding model inputs, the

running of the models, and output analysis, including judgmental adjustments of

model results. Taken together, the baseline scenario and the resulting impact

analysis form the reference baseline analysis.

Some uncertainty usually surrounds many of the assumptions and data

specifications of a baseline scenario. There is a need to measure the impact of

this uncertainty by performing standard sensitivity studies on key assumptions

and/or data of the baseline scenario. This can be an expensive and

time-consuming activity and, thus, it is often limited to just a few alterations

of the baseline scenario. The results of analyzing these alternate baseline

scenarios represent, in a sense, the range of possible outcomes that can occur

for the assumed baseline future. For example, high and low GNP estimates may be

run and compared to the baseline (average) GNP, while keeping all other input

elements constant. In many studies, the above type of analysis is the extent to

which a multi-model system is employed.

By varying the assumptions and conditions of the baseli

the views of the plausible future, policy impacts can be ana

of possible future outcomes and compared with the reference

Then, depending on their assessment of the most likely cours

analysts and study sponsors can account for the uncertainty

results. The impacts will often change with the scenario as

consistent and controlled means of comparing policy implicat

futures is provided by a systematic use of the baseline and

scenarios. A comparative analysis, based on the results of

plausible scenarios, puts the decision maker in a better pos

which policies to implement.

ne scenario, that is

lyzed over a range

baseline analysis,

e of events, the

in the exercise

sumptions, but a ions of different

alternate baseline

a number of

ition to decide

In the above type of study, in which a baseline scenario is varied to

reflect uncertainty, the set of choices (policy options) available to the

multi-model system's decision process is not changed. Results change only

because a different choice is made under the alternate baseline scenarios.

Another and probably more important form of analysis occurs when policy options

are proposed that modify the baseline policy specifications. The analysis then

must be made using a set of choices that are different from the one available in

the baseline analysis. Alternate policy scenarios are then used to produce

policy analyses that can be compared to the baseline analysis to measure the

impact of allowing new policy options. For example, a FEMA baseline scenario

may stipulate that all steel production is to be given to the defense tier, with

the level of steel production being an uncertain parameter about which

sensitivity studies are made. But an alternate policy scenario may call for

138
only 50% of the steel output going to the defense tier. Another example could

have the baseline scenario requiring mobilization to be paid for by taxation,

while an alternate policy scenario states that mobilization would be financed by

deficit spending. The alternate policy scenarios are subject to the same

uncera tinties as the baseline scenario. To conduct a more complete policy

study, the alternate policy should be varied and processed in the same

manner as the alternate baseline scenarios. However, as the running of say,

high, average, and low GNP variations for each scenario is costly and time

consuming, a study is usually limited to just the processing and comparing of

the average baseline and policy scenarios. We note that any differences in the

two analyses are due to the model’s decision logic that must select from the

different policy options allowed by the scenarios.

Credibility of a multi-model system can be further enhanced if the reasons

why the particular set of models was selected are discussed. For example, a model that has been used in the past with favorable results and does not need

to be changed contributes positively to the notion of credibility; while a newly

built model or one that must undergo extensive reprogramming will not inspire

much credibility. No matter what the reasons are, they need to be made

explicit. Further credibility enhancing elements include the active involvement

of the user in specifying the scenarios and interpreting the results, and the

use of experts in selecting the models and in reviewing the results.

139
APPENDIX D: THE POTENTIAL ROLE OF DGEM IN AN INTEGRATED SYSTEM

FOR MILITARY M06ILIZATI0N1

There are several agencies within the Federal government responsible for

estimating resource requirements in the event of a military emergency. Should

an actual military emergency occur, these same agencies would also be

responsible for allocating available resources among competing ends. The most

obvious is the Department of Defense (DoD) which would be responsible for

estimating the structure of the military forces to fulfill the requirements of

winning or preventing a war. This would include estimates of dollar outlays to

fulfill the force structure. The Department of Commerce (DoC) has the

responsibility to evaluate the industrial base of the country and estimate the

output of goods and services necessary to fulfill not only the requirements of

the military, but also the requirements of the civilian economy. This task also

includes estimates of production required for industrial investment in

structures and equipment, net exports, state and local government expenditures,

Federal government non-defense expenditures and personal consumption. In the

production of output for investment and trade, it is very likely that the

required output will compete with production for national defense. Closely tied

to the Commerce Department estimates of output by industry are estimates of the

labor force (employment and occupations). This task is performed by the

Department of Labor (DoL).

Within the triangle of Federal agencies of DoD, DoC, and DoL is what might

be considered the core necessary to evaluate the industrial production and

employment effects of a military emergency. Within each of these agencies is

the ability to estimate total production and .employment needs for an emergency.

However, the responsibility and the corresponding capabilities within each

separate agency to analyze its resources argues for cooperation in mobilization

analysis. As a result, these three agencies signed a Memorandum of

Understanding in 1983 for cooperation in mobilization planning.

In addition to the resources monitored by DoD, DoC, and DoL, there are

other Federal agencies responsible for monitoring and allocating resources in

the event of a crisis. These include: (1) energy requirements monitored by the

Department of Energy (DoE); (2) strategic and critical materials monitored by

the Department of the Interior (Dol); (3) transportation requirements monitored

by the Department of Transportation (DoT); and (4) food and agricultural

requirements monitored by the Department of Agriculture (DoA).

This appendix outlines how these executive agencies can work together, and

have worked in the past, to estimate the resources necessary to meet the

requirements of a military emergency.

Central to the discussion in this appendix is the role of macroeconomic

models in interagency mobilization analyses. Special emphasis is placed on how

DGEM can complement proprietary models in such an analytical framework.

-*-This appendix was prepared by David K. Henry and Albert Walderhaug of the

Office of Business Analysis, Office of the Under Secretary for Economic Affairs,

U. S. Department of Commerce. Several contributions by Robert E. Chapman,

Center for Computing and Applied Mathematics, National Institute of Standards

and Technology are also included.

140
This appendix does not attempt to concord all of the technical parameters

of each agency's models and data. To concord each of these major models is a considerable task and would, and probably should as in past efforts, be

accomplished while performing an interagency mobilization analysis. At that

time, details such as the industry sectoring classification, constant dollar

series, industry to commodity conversion, price valuation, and national

accounting conventions should be accommodated and made consistent among the

models and supporting data used by the Federal agencies participating in the

analyses. The constant updating and change in the technical parameters of the

models that can be used in interagency analysis is another reason for not

attempting to concord the technical parameters in this appendix.

The most recent experience of coordinating a multi-Federal agency effort

analyzing resources to meet the requirements of a military emergency was an

exercise in 1984-85 that was coordinated by the Federal Emergency Management

Agency (FEMA) and was called the Federal Resource Assessment System (FRAS). FRAS

was set up under the Military Mobilization Working Group of the Emergency

Mobilization Preparedness Board (EMPB). The Department of Commerce, Office of

Business Analysis (OBA) was responsible for the technical coordination of all

the participating Agencies. The effort resulted in a report to EMPB in 1985

which described the interagency effort and provided preliminary estimates of the

resources. The EMPB dissolved shortly thereafter. Since then, a Senior

Interagency Group (SIG) for National Security Emergency Planning has been

established under the National Security Council. The Mobilization Planning

Group under the SIG has been an attempt to continue an interagency governmental

responsibility in mobilization planning.

Also in 1984-85, the National Security Council led an effort to evaluate

the quantities of strategic and critical materials that should be maintained in

the U. S. government's stockpiles in the event of a military emergency. The

stockpile study used a similar approach to the FRAS analysis, since industrial

output requirements during a military emergency needed to be estimated before

the estimate of critical materials could be made. The stockpile review was also

an interagency effort and a good portion of its military emergency assumptions

were incorporated into the FRAS analysis.

In addition to the FRAS and the Stockpile interagency efforts to analyze

the requirements of a military emergency, there was an interagency effort to

evaluate the effects of imports on the machine tool industry for a Section 232

investigation under the 1962 Trade Expansion Act for Import Vulnerability. The

results of the study led to the decision to apply Voluntary Restraint Agreements

on the imports of machine tools into the United States.

Each of these studies required an evaluation of the resources of the entire

economy. Military and civilian resource requirements estimates were made for

products common within each industry and decisions were made about the

industry's ability to meet these requirements. In each case, industrial output

was estimated to fulfill the production of goods and services that were

generated in an economy described by a level and composition of Gross National

Product (GNP). The economy as described by its GNP was used as the first step

in the analysis.

141
Figure D.l Scenario Development and Analysis

142
The Economy Under A Mobilization

Implicit in any military mobilization preparedness study is a well-defined

set of models, databases, and analysis strategy. Figure D.l illustrates the

process used to describe the domestic economy during an emergency.

The first stage of any such study is the specification of a scenario. A

scenario is defined as an account or a synopsis of a projected course of actions

or events. The military scenario describes a course of actions or events which

includes an enemy threat, a basic strategy to meet the threat, the size and

deployment of U.S. forces that fulfills the military strategy and the form and

intensity of the combat expected. In turn, the size and deployment of U.S.

forces requires a defense budget that accommodates or allows their support.

In addition to, or as a result of, the military scenario, there is an

economic scenario which describes how the domestic economy reponds to the

military emergency. In developing the economic scenario, policy decisions

affect the way the economy reacts to meet the threat. Economic assumptions

reflect, for example, the monetary and fiscal policies chosen to finance the

war, the world energy supply and corresponding prices, and projections for the

expected rate of growth of the economy and the Federal budget. Once the economy

has been described, in quantitative terms, it is possible to estimate the

production of goods and services to meet the requirements of the economy.

The need for macroeconomic (economy wide) analysis requires that models of

the economy be used to determine the economy's size and composition. There are

a number of -proprietary models such as those marketed by Wharton, Chase and DRI

available to be used by government agencies. In addition, there are some models

developed within the government for special purpose analyses, which incorporate

economy wide analysis as a front-end to their special analyses: the Defense

Economic Impact Modeling System (DEIMS), used by DoD to assess the market share

of output for defense; the Economic Growth System (EGS) of the Bureau of Labor

Statistics (BLS), used to analyze future needs for labor and skills as the

economy grows and changes; as well as DGEM

.

The proprietary models are, in general, peacetime models based on peacetime

trends and relationships. Their suitability for mobilization analysis depends

on the extent to which the structural relationships in the model can accommodate

policy relevant variables dictated by the mobilization scenario and the economic

objectives which are subject to the set of policy options. The options relate

to how the economy will be managed to achieve the mobilization objectives and

will involve: (1) tax policy and how it relates to persons and business; (2)

fiscal policy as it determines how the emergency is financed; (3) monetary

policy affecting the growth of money supply and its impact on interest rates;

and (4) government actions affecting labor force participation rates and

producitivity improvement incentives.

Since econometric models are based on historical relationships and trends,

it is important that they incorporate and reflect up-to-date information to the

maximum extent possible. The suitability of models from this aspect needs to be

assessed. Furthermore, the release by the Bureau of Economic Analysis of the

Department of Commerce of the 1977 input-output tables and rebenchmarking of the

National Accounts to 1982 prices, which form the basis of the macroeconomic

models and derivative models, may cause some problems of price compatibility and

industry classification and structural relationships.

143
Recently, several Federal agencies have expressed interest in using DGEM to

complement their modeling efforts. For example, DGEM may be used to forecast

the values of key macroeconomic variables (e.g., GNP , Personal Consumption

Expenditures, Investment, as well as other components of the National Income and

Product Accounts (NIPA)) which may be needed as inputs for a particular model.

Clearly, experiences gained by these Federal agencies would be beneficial in

performing similar analyses with DGEM for a war-time economy.

1

DGEM may be useful in this role for four other reasons. First and

foremost, DGEM was designed to deal with the demand surges and resource

constraints which are expected during the transition from a peace-time to a war-time economy. Specifically, DGEM explicitly addresses factors which affect

both demand and supply on an annual basis up through the year 2000. Second, the

TUTOR provides the user with a set of step-by-step instructions for

incorporating information on Defense and other government expenditures as well

as economic and technical considerations which specify how the economy

mobilizes. Third, from a review of the documentation, it became clear that much

care went into the design, development and testing of the model. Furthermore,

the model's documentation is quite complete and should be sufficient to: (1)

set it up on the host system; (2) execute a "base-case" simulation; (3)

interpret the results of the "base-case’ simulation; and (4) create, run, and

interpret user-specified simulations. Finally, the DGEM source code has been

made available to a number of users, enabling them to explore the major linkages

within the model as well as trace the flow of calculations for particular (e.g.,

mobilization oriented) variables of interest.

These "strengths" must be weighed against the lack of sector-specific

detail in DGEM. At the present time, DGEM consists of 36 producing industries.

More precisely, the DGEM analysis framework includes: (1) inputs to each

industry from each of the 36 producing industries and from capital, labor and

competitive imports; and (2) outputs from each industry to each of the 36

producing industries and to final demand — personal consumption, investment,

government purchases (specifically, Federal defense, Federal nondefense, and

state and local government), and exports. Bridge tables, either within DGEM or

between the TUTOR and DGEM, are of crucial importance for most military

applications. The reasoning behind the previous statement is that bottlenecks

are more likely to occur as the sectoral level of detail is increased.

Conversely, the higher the level of aggregation, the less likely is the analyst

to be able to identify a potential bottleneck. Based on the previous

discussion, it should be clear that bottlenecks within DGEM's 36 sector

framework can be identified, analyzed and to a certain extent alleviated. The

question thus becomes one of sectoral detail. It is also worth noting that the

sectoral level of detail of DGEM differs significantly from DEIMS (400 sectors)

and the BEA benchmark input-output table (537 sectors). This lack of sectoral

detail is probably the most serious deficiency of DGEM with regard to military

mobilization applications.

Based on our review of DGEM , both the model's source code and documentation

and the TUTOR, it is clear that the addition of two bridge tables would

substantially enhance the ability of DGEM to provide a detailed analytical

1 For a more detailed discussion of DGEM, the TUTOR, the DGEM analysis strategy,

and other related topics, the interested reader is referred to Chapters 6

through 9.

144
framework for a wide variety of military mobilization applications. The first

is a "pre-processor” which incorporates a bridge into the TUTOR between the 55

defense procurement categories used in DEIMS and DGEM 1 s 36 producing sectors.

Alternatively, the "pre-processor" bridge table could be between DoC's defense

procurement categories list and DGEM ' s 36 producing sectors. In the text which

follows some of the pros and cons of both DoD ' s and DoC's bridge tables between

their respective defense procurement categories and the economy's producing

sectors are discussed. The second is a "post-processor" which translates the

DGEM output into a form which is consistent with DoC's 4-digit Standard

Industrial Classification (SIC) level. If this were done, the output of DGEM

would be consistent with the BEA benchmark input-output table. We note that the

"pre-processor" bridge table discussed earlier may be incorporated in a relatively straight-forward manner. This is because much of the logic has

already been incorporated into the TUTOR. Recall that government purchases are

separated into three types' of government activity: (1) Federal defense; (2)

Federal nondefense; and (3) state and local government. In turn, each type of

government activity involves purchases, spread over a number of supplying

industries. For Federal defense purchases, a rudimentary bridge table, which

distributes defense purchases from the 55 DoD procurement categories to the 36

producing sectors, is included in the TUTOR. The analyst can therefore control

defense spending in a variety of ways within this structure. Overall levels of

defense spending can be altered, leaving the composition unchanged, or the

composition and level can be changed. Specific defense procurement categories

can be altered, year by year. Federal nondefense purchases and state and local

government purchases are spread over the 36 producing sectors. Federal

nondefense purchases can also be altered in total or industry by industry.

State and local purchases can be altered in total, although the composition

remains the same. The development of a "post-processor" bridge table is

somewhat more complex, but, in principle, most of the data exists to go from the

NIPA components to the 4-digit SIC level of detail. If these steps were taken,

an analysis framework similar to that outlined in Figures D.l through D.3 and

described in the text which follows would result.

To summarize, the key outputs required from the macroeconomic model are the

"snapshots" of the U.S. economy for the years in which the economy is distorted

by a mobilization or a military emergency. The description of the economy

needed in order to evaluate industrial requirements and supply conditions are

the levels of real GNP and its composition.

Estimating Industrial Production and Supply

Figures D.2 and D.3 illustrate the process generally followed in analyzing

the industrial impacts of major government programs, or responses to emergency

situations. Within the parameters of GNP and its components, the output of

goods and services implied by the scenario are estimated as well as the

capability of the existing economy to fulfill the requirements.

^It is important to point out tha

on DGEM as complementary to those

It is neither reasonable nor desi

supplant a loosely-coupled collec

but rather to use such a model to

feedback within the overall analy

t we view such an analysis framework based

based on collections of models and database

rable to have a single model, such as DGEf

,

tion of models from several Federal agencies

encourage consistency and an avenue for

tical framework.

145

s

.

*
Figure D.2 Analysis of Production Requirements

L 46
Figure D.3 Analysis of Capacity

147
OBA has developed bridge tables to be used in translating the final demand

components of GNP into their industrial composition. These estimates show the

direct impact of the final demand for goods and services on the producing

industries of the economy. However, before the bridge tables can be used, a set

of "control" totals must be established that correspond to the detail of the

bridge tables. A macroeconometric model is used to establish the levels for

most of the totals (composition of GNP).! However, as appropriate, in all of

these mobilization exercises the level and composition of national defense

spending deserves special attention.

Military Requirements

One of the most sensitive issues to be addressed by mobilization planners

is the size and the composition of the military budget to meet the threat.

Although the total national defense budget is a key exogenous variable in the

macroeconomic analysis of the economy, the composition of national defense is

necessary to allocate industrial final demand to the defense expenditures. For

the FRAS , Stockpile Review and the Section 232 Machine Tool studies, the detail

of the military budget estimates by budget category have been supplied by the

Defense Department.

An alternative way to translate a military scenario into defense outlays by

budget category has been through a model developed by The Analytical Sciences

Corporation (TASC) for FBI A called TASCMAIN, or Technique for Assessing the

Capability to Mobilize American Industry. Although TASCMAIN was designed to

evaluate industrial supply and demand during a mobilization, its uniqueness for

an interagency exercise is its ability to translate a military scenario into a force structure which is, in turn, translated into DoD budget accounts in

dollars. These budget accounts are necessary for DoC to calculate military

demand on industry.

Industrial Final Demands for Military Requirements

The emphasis of the work conducted in OBA to translate the GNP components

of final demand into their industrial composition has been on Federal government

expenditures for national defense. OBA has developed 34 bridge tables which

translate DoD budget expenditures into their industrial composition. These

bridge tables are keyed to defense budget categories: military personnel;

operations and maintenance; procurement; research, development, testing and

evaluation; military construction; family housing and non-DoD defense programs.

An alternative to the OBA translation of the defense budget to industrial

final demands is by the use of the defense budget translators maintained by the

Department of Defense’s Program, Analysis and Evaluation Division. The

translators are a part of the Defense Economic Impact Modeling System (DEIMS)

and in a great many respects are similar to the translators or bridge tables

maintained by the Department of Commerce.

1-0 BA is currently exploring a number of ways in which DGEM may be used to

generate the control totals required in defense-related studies.

148
The Civilian Economy

Estimates of components of GNP resulting from the macroeconomic analysis,

in addition to Federal government expenditures for national defense, include

expenditures for personal consumption, gross private capital formation, Federal

expenditures for non-defense, state and local government expenditures, exports

and imports. During peacetime, the percentage of GNP accounted for by national

defense has ranged from 5 to 6 percent in recent years. For each of the

civilian components of GNP, bridge tables were developed that describe their

composition of industrial final demand. The expenditure-to-commodity relationships developed in the bridge tables reflect historical relationships.

Therefore, the greater the GNP expenditure detail available from the

macroeconomic model, the more flexibility there can be in using bridge tables to

specify a "bill-of-goods" to describe the industrial composition of non-defense

GNP or final demand.

Sub-Tier Production

Final demand does not reflect the intermediate goods and services which are

consumed in making end products that are purchased for national defense or the

rest of the economy. A further step is necessary to estimate the output of

industries that is required to supply those indirect products, as well as the

products entering final demand directly. This is accomplished using

input-output techniques in which the bills-of-goods (aircraft, missiles, tanks,

ships, etc.) are applied to an input-output inverse matrix which generates the

prime contractor and sub-tier contractor requirements for output by industry to

support the specified level and detail of GNP. The levels of total output,

direct and indirect, by industry resulting from these operations form the basis

for subsequent evaluations of the adequacy of resources to meet the

requirements

.

The input-output models of the U.S. industrial economy available for use in

estimating the indirect output by industry to fulfill the final demand for goods

and services required by the emergency are derived mainly from the 537-sector

interindustry tables produced by DoC's Bureau of Economic Analysis (BEA). These

derivative tables are condensed and in some cases updated versions of the BEA

tables with industry classifications designed for specific purposes. The latest

version of the BEA input-output tables are for 1977. The next revision of the

benchmark table will be for 1982, although revisions for 1981 and 1982 of the

more aggregated 80-industry order table have been produced. Other interindustry

input-output tables, derived from the BEA benchmark table, include: the DRI

interindustry model (432 sectors), the BLS Economic Growth System (156 sectors),

the FEMA model called the Revised Growth for Industrial Potential, REGRIP,

(115 sectors), and DoE's Energy Disaggregated Input-Output Model (130 sectors).

Industrial Capacity

Estimates of industrial capacity by which to assess the feasibility of

producing the output as required during the emergency has and continues to be

the weakest link in industrial analysis. The Bureau of the Census report

entitled Survey of Plant Capacity lists capacity utilization rates for most

manufacturing industries at the 4-digit Standard Industrial Classification (SIC)

level. There are some considerable problems in utilizing the Census data for

mobilization planning. The utilization rates are for fourth quarter activity

only and are not average annual activity. If there was only one shift operating

149
during the fourth quarter, estimated capacity is based on that one shift.

During a military emergency, double and triple shifts could occur and the

industry's capacity would increase accordingly. This increase would be offset

somewhat by the faster rate of capital stock depletion.

FEMA recognized the problems in using industrial capacity data for

mobilization planning. Consequently, they contracted ORI, Inc. to provide

estimates of emergency capacity for manufacturing industries. The estimates for

emergency capacity for mining, manufacturing, and utility industries include

labor and capital shift factors and lead times for "greenfield" investments.

The emergency capacity data developed by ORI, Inc. can be used in conjunction

with the normal sources of capacity data from the Census and the Federal Reserve

Board.

In the past, DoC conducted Industry Evaluation Board (IEB) studies to

determine the capacities of defense-related industries. These capacity studies

were done at a very high level of detail. The IEB capacity studies used

considerable investigative resources and took a long time to complete. Survey

forms and mailing lists were prepared and surveys mailed. Follow-ups were made

to non-responders and the responses were reviewed and analyzed.

More recently, OBA, at the request of the National Security Council and the

Office of Management and Budget, developed a methodology to assess the investment requirements of certain manufacturing industries to meet the industrial

outputs projected for military mobilization.

The methodology to estimate new and replacement investment of the

industries evaluated depend on OBA's data base on capital stocks of

manufacturing industries. The OBA capital stock data base consists of measures

of investment, capital stocks and related statistics for manufacturing

industries. Twenty-one measures of capital stocks for structures and equipment

are available.

In OBA's method for estimating new and replacement investment, ratios of

capital stocks per unit of output are developed for each of the industries.

Based on the requirements estimates, a determination can be made from these

ratios as to the level of capital stocks necessary to support the industrial

output requirements during the emergency. Comparing existing capital stocks to

the emergency capital stocks requirements, and factoring the physical

depreciation (discards and wear and tear) of the existing capital stocks into

the emergency period, the amount of new investment can be calculated.

If the industrial output projection for defense related industry exceeds

existing capacity, a decision must be made to provide or not provide for the new

investment. If not provided for, the industry that needed the investment would

have to be identified as a bottleneck industry — an industry unable to supply

the estimated military and civilian requirements and an alternative policy

decision on allocation of available supply would have to be made.

The investment in the industries needed to meet the emergency production

levels is then distributed to the types of equipment and structures that are

necessary to provide the facilities. For example, machine tools and testing and

measuring equipment are important in the production of weapon systems. OBA

maintains a capital flow matrix that identifies the types of equipment and

structures in an industry's investment pattern.

150
These calculations of investment requirements really redefine the previous

estimates of Gross Private Fixed Investment which have been estimated in a less refined fashion. In other words, the GNP is redefined and industrial

requirements of all industries are restructured and additional output will be

required from an undefined set of industries to meet this investment. Thus, a second or even third review of industrial capacity may be necessary.

Recently, FEMA developed a model to identify capacity shortfalls for

industrial mobilization and the options to resolve them. The model will be used

by the Army Material Command to analyze industrial preparedness. The model was

named ROCS (Resolution of Capacity Shortfalls). The primary advantages of ROCS

are: (1) that it can be implemented on a Personal Computer (PC); (2) that it

calculates the cutbacks in civilian consumption necessary to meet military

requirements; and (3) offers alternatives to fulfill supply requirements.

The model utilizes most of the data used for the REGRIP model except that,

while REGRIP differentiates only between civilian consumption and military

requirements, ROCS deals with more detailed categories of civilian consumption:

(1) personal consumption expenditures; (2) investment; (3) exports; (4)

non-military Federal government purchases; and (5) state and local government

purchases. ROCS also differs from REGRIP in that an involved linear programming

solution is not required. Another feature of ROCS is the inclusion of data on

the sources of imports from 18 regions of the world; the user may supply their

own estimates of import percentages and reliability for each region.

Estimating Employment Requirements and Availability

DoL is responsible for estimating and evaluating all aspects of the U.S.

labor force for a military emergency. This includes estimating both defense and

civilian labor requirements by specific industry and occupation, projecting the

labor force available, and determining the feasibility of mobilization scenarios

in terms of labor resources.

The Economic Growth System (EGS) of the Bureau of Labor Statistics (BLS) is

designed to project the military and civilian labor force by age and sex, labor

productivity and labor requirements for 378 industries and 550 occupations.

This system is used to estimate labor requirements that are consistent with

macroeconomic assumptions and conditions and industry production levels

estimated by DoC. The EGS model overlaps the macroeconomic models used to

predict the economy and production estimates made by DoC. Its uniqueness in the

interagency framework is the employment and occupations matrices incorporated

into the EGS model.

The EGS contains the Labor Force Model which projects the labor force based

on projections made by the Bureau of the Census of "total population including

Armed Forces overseas." The population estimates are grouped by age, sex, and

race. Labor force participation rates are projected by BLS for each of 54

demographic groups and applied to the population data to estimate the aggregate

labor. This technique provides, in effect, an estimate of capacity in terms of

labor resources against which to evaluate the demand for labor as determined

from estimates of output by industry that will be required in the mobilization

effort. Undoubtedly, in developing the scenario for the emergency, personnel

requirements for the armed forces would have an impact on projection of the

151
civilian labor forces and on projections of potential GNP when formulating the

macroeconomic model. Similarly, the demographic detail of the population and

labor force participation rates will give guidance in policy decisions dealing

with the perceived labor shortages in the resource evaluation procedure.

The BLS Economic Growth System contains an input-output model that is used

to convert estimates of the level of GNP and its components into estimates of

gross output for 156 industries. The estimated industry outputs are used

to project employment by industry using a Labor Demand Model. However, for

consistency in the interagency program, the EGS can accept estimates of gross

output by industry for the overall resource assessment system which would be in

greater industry detail. When aggregated to the classification system

consistent with the Labor Demand Model, gross output by industry can be

converted into related employment by industry and then aggregated to assess

total requirements for civilian employment.

The Economic Growth System also contains an Occupational Demand Model

which uses industry occupational staffing patterns to estimate the skill

requirements of industries for 550 occupations which, when aggregated, will

yield estimates of total employment by occupation.

Estimating Infrastructure Requirements

DoE is the resource agency charged with energy resource management

functions. In providing analytical capability in support of these functions,

the Energy Information Administration maintains a complete hierarchy of models

and modeling systems to evaluate alternative aspects of energy supply and demand

to provide the basis for policy decisions within the DoE. This analytical

capability can be called upon in support of an interagency resource assessment

plan to examine the energy-related aspects of a military mobilization.

The Energy Disaggregated Input-Output (EDIO) model is an energy oriented

input-output system designed to expand the Department’s capability to analyze

the potential industrial economic impacts of energy policies. In presenting a more systematic representation of the basic energy interrelationships in the

economy, the EDIO model permits a much more detailed analysis of the energy

requirements within an 1-0 framework.

The sectoring plan in EDIO is important since it defines the industry and

commodity detail. While the industry classification used in EDIO can be related

to the other 1-0 systems available for mobilization analysis, the basic focus of

the EDIO table is energy, and therefore, the EDIO industry classification

emphasizes energy products. The model disaggregates energy producing

industries, industries consuming large amounts of energy per unit of output from

other sectors of the economy, thus enabling the analyst to identify explicitly

those industries most likely to be affected by energy policy. These features

provide a model capable of identifying the sectoral economic impacts for

different energy and macroeconomic scenarios. On the other hand, in other areas

of the economy where energy is not particularly important, the industries have

been combined to yield a smaller, more manageable system but one with

considerably more energy detail.

There are two approaches that can be taken in utilizing the EDIO model in

an interagency resource assessment system. The model itself is essentially self

contained requiring as inputs components of gross national product from a 152
macroeconomic model. This requirement would be satisfied in the general plan of

the overall assessment system which employs such a model in developing the

mobilization scenario. The EDIO system has a submodel which disaggregates the

GNP components into their industrial composition in order to compile a vector of

final demand in the industrial detail compatible with the EDIO models i.e., a bill-of-goods representing sales to users by each of the 117 sectors of the

model. Total gross output by industry to satisfy a given bill-of-goods is

derived through multiplication of the EDIO inverse matrix and the bill-of-goods.

Another approach to the assessment of energy requirements during the

mobilization recognizes that a central and coordinating element to the resource

assessment system is the step which estimates the gross output by industry

required to support the mobilization effort. This calculation is carried out

using an 1-0 model with much greater industrial detail (i.e., the 500-order

indus try/commodity tables produced at DoC by BEA) . The industry detail of gross

output from these estimates can be aggregated to the level of industrial detail

consistent with that maintained at DoE. Energy output coefficients from the

EDIO model or from the National Energy Accounts (NEA) system can then be applied

to determine energy requirements.

In the end, this may be a more flexible approach since the energy inputs

are perhaps some of the more variable inputs to industry and, within limits,

alternative configurations of energy sources and allocations can be evaluated

without significant feedback effects that would alter the output required of the

rest of the producing industries.

TRANSPORTATION

DoT has developed and utilizes a demand/capacity model which translates a given level of the U.S. economy into an estimated demand on the domestic

transportation sector. 1 This model has the capability to address both the

overall mobilization-related transportation demand/capacity requirements and

also the potential impact on the transportation infrastructure caused by

physical denial and/or labor stoppages.

The input to operate the DoT model would be available generally from the

resource assessment process. To estimate transportation requirements related to

the production and use of commodities, the DoT model converts dollar measures of

commodity output by industry into quantities transported. The estimates of

industrial output required by an emergency is part of the overall resource

assessment system and would be supplied in common to all participants in the

exercise. A consideration in coordinating the transportation analysis would be

the need to accommodate differing classification systems and price levels in the

related models and to formulate the data on other than annual (e.g., a quarterly

basis). These problems can be anticipated and should cause no difficulty in the

planning process.

^In a recent training program for National Defense Executive Reservists, DoT

employed DGEM as the macroeconomic driver for the Transportation Express Model.

153
The DoT transportation industry submodel apportions the industrial output

of commodities, in tonnages, among the rail, truck, domestic water and domestic

air modes in a way which is consistent with data on historical patterns of the

distribution of commodity shipments among the modes. Then, with information on

the average lengths of haul, it estimates the ton-mile demand by mode on the

transportation system.

The surge requirements for commercial transportation resources to move

defense personnel and material to initial deployment and resupply centers are

estimated and supplied to DoT by DoD’s Military Traffic Management Command in

the form of tonnages and personnel to be transferred between designated origin

and destination points in the continental U.S. and by mode of transportation.

The combination of the military and nonmilitary requirements form the

estimates of the demand to be placed on the commercial transportation system

during the emergency period. These demands are assessed by DoT in terms of

their estimates of the capacity of the transportation system to handle the load

and to determine limits to the level of freight services that can be attained

realistically under both short term and extended emergency conditions. The

emergency capacity concept assumes adequate supplies of supporting resources and

is based on a realistic assessment of vehicle utilization. Thus, the current

and projected inventory of heavy trucks is rated at recently achieved high

average annual miles per vehicle and average annual payload tons to estimate

the intercity ton mile capacity of the heavy truck fleet.

Similarly, the estimated rail freight ton-mile capacity is based on recent

peak utilization factors and is the product of the standing weight capacity of

the rail car fleet, the maximum average trips per car per year, the maximum

practical load factor and average haul.

The model implies that the major component of transportation activity

continues to be production and consumption in the civilian sector and that

commodity distribution patterns and seasonality of demand for various vehicles

categories will remain essentially unchanged, though possibly scaled in

proportion to overall economic activity.

AGRICULTURE

In fulfilling its responsibility to assure high levels of performance by

the food and fibre system of the United States, Agriculture has developed a comprehensive econometric model of the U.S. agricultural sector. The model has

two main objectives. One is to enhance and support the agency’s ability to

provide economic intelligence and make forecasts for the intermediate term. The

other is to provide a means to evaluate and quantify the impacts of alternative

policies or legislative proposals on the agricultural sector. The model

entitled Food and Agricultural Policy Simulator (FAPSIM) is designed to estimate

simultaneously a price-quantity equilibrium solution for a set of livestock and

crop products. It also determines farm production expenses, cash receipts, net

farm income, government deficiency and reserve storage payments, consumer price

indexes for food products and farmer participation in government commodity

programs

.

154
The FAPSIM model includes all the major livestock and crop commodities that

would be significant for support of the population and the economy during a mobilization period. It is a more detailed and dynamic approach to simulating

the agricultural sector than is contained in the standard input-output models

and, therefore, would be a more flexible tool to use in dealing with

mobilization or emergency situations. Its policy analysis capability enables it

to deal explicitly with problems likely to arise in an emergency. However, the

FAPSIM model requires the input of substantial information of a different

character and detail than is usual in or derived from the other models in the

resource assessment system. For instance, one major subcategory of exogenous

variables are government policy variables such as individual crop loan rates,

target prices, national program yields and acreages, and diversion and set aside

rates. These are questions that need to be addressed and resolved in developing

a scenario for the mobilization period. Another major subcategory of. exogenous

variables are the macroeconomic variables such as population, disposable

personal income, food processing wage rates, petroleum prices, nonfood consumer

price index, etc. These are the variables that generally form part of the

macroeconomic model that will be used to describe and establish the size and

shape of the economy in the emergency. In all, the FAPSIM model requires input

of 265 exogenous variables many of these variables are of an economic nature.

Consequently, the requirements for running FAPSIM must be considered in

developing a scenario and the variables most likely to be affected in the

emergency must be stipulated in a manner consistent with the model.

INTERIOR

The use of critical and strategic materials by industry is an important

concern in mobilization planning. In fact, a stockpile of these materials is

maintaianed by the Federal government in anticipation that excessive quantities

of these commodities will be required during an emergency. The quantities of

these materials necessary to produce the military, basic industrial and

essential civilian needs are based on estimates of material usage per unit of

output of an industry, estimates of outputs by industries, and estimated supply

availability of these products.

Interior's, Bureau of the Mines maintains sufficient worldwide supply data

for the stockpiled materials. The Department of Commerce has estimated the

material usage per unit of output by end-using industry from information from

the Bureau of Mines, the Census of Manufacturers, and from the industry.

Projections of material usage per unit of output by industry (materials

consumption ratios) are made and substitution factors are introduced to estimate

what the usage of material will be during the crisis. These ratios are applied

to the emergency industry outputs and each material requirement is summed across

all industries. Currently, there is no model maintained by either Interior or

Commerce to do this analysis. In the past, FEMA has been responsible for the

material estimates. However, this responsibility has recently been given to the

Department of Defense and the models to be used have not been selected.

1 Recently , the DoA acquired a copy of the DGEM source code to determine ways in

which DGEM could generate estimated values of critical "economic variables"

required by FAPSIM.

155
Conclusion

Sufficient analytical resources exist within the major administrative

agencies to support planning for resources needed during a military

mobilization. Further, this work can be provided for in an interactive,

interagency setting that allows each of the agencies to estimate resources for

which that agency is responsible during a military mobilization using the same

set of assumptions and using the same economic and military scenarios.

Macroeconomic models, such as DGEM , occupy a central position in this analytical

framework. The potential benefits from the use of DGEM in this setting are due

in part to its availability to all members as a non-proprietary software system

and in part to its intended use as an emergency management model. In addtion,

the use of DGEM may encourage greater consistency among the other models and

provide an avenue for feedback within the overall analytical framework.

156
APPENDIX E: DICTIONARY OF TERMS USED IN DGEM

The purpose of this appendix is twofold. First, it is used to expand upon

some of the information presented in Chapters 6 through 9. Second, it should

facilitate any cross referencing between the endogenous variables, included in

the ENDOG COMMON block, and the exogenous variables, included in the EXOG COMMON

block, with the program summaries given in Appendix F.

E.l ENDOGENOUS VARIABLES

The text which follows provides a brief description of each of DGEM’s

endogenous variables.

cc Real consumption of goods and services.

GDI Values of household purchases of capital goods.

CDIM Value of services from household capital.

CDIMQ Real services from household capital.

CDIQ Real purchases of household capital.

CNIA Personal consumption expenditures (uses NIPA definition).

CNIAP Personal consumption expenditure price deflator, NIPA.

CNIAQ Real personal consumption expenditure, NIPA.

CX(i) Proportion of domestic supply in total real supply of good or

service i.

D Value of depreciation of private domestic capital.

DG Value of government deficit (excluding social insurance funds).

DLDA(i) Rate of change in overall factor productivity in producing

sector i.

DR Value of the rest of the world deficit with the U.S.

E Value of government purchases of goods and services.

GD Real private net claims on the government.

GNIA Government purchases of goods and services, NIPA.

GNIAP Government purchases of goods and services price deflator, NIPA.

GNIAQ Real government purchases of goods and services, NIPA.

GNP Gross national product, NIPA.

157
GNPP Gross national product price deflator, NIPA.

GNPQ Real gross national product, NIPA.

INIA Gross private domestic investment, NIPA.

INIAP Gross private domestic investment price deflator, NIPA.

INIAQ Real gross private domestic investment, NIPA.

IVT Real private domestic investment including purchases of household

capital

.

K Total real private domestic capital stock, end of year.

KD Total real private domestic capital services.

LB Quantity of labor services supplied by the household sector.

LH Household -sector time endowment.

LP Quantity of time consumed as leisure by the household sector.

MNIA Imports, NIPA.

MNIAP Imports price deflator, NIPA.

MNIAQ Real imports, NIPA.

P(i,j) Price index for purchases of good or service i by sector j.

PCC Price index of consumption goods and services.

PCGXCC Value of household expenditure on consumption of goods and

services

PDOt'l ( m) Price index for domestic oil and natural gas.

PFM ( i

)

Scale factor for relative purchase prices of good or service i.

PFXF Value of household expenditure of consumption of goods,

services and leisure.

PI Price of investment goods.

PK Price of private domestic capital stock.

PKD Price of capital services.

PKLG Price of capital stock using the previous year's weights in

aggregation.

158
PLB

PLM

P0( i )

POP

PZ(i)

QE(k)

QED

QEE

QEF

QEG(k)

QEH

QEI(k)

RD

RK

RL

RNT

RNW

RP

RW

RX

S

V

W

XNIA

XNIPA

Net price of labor services received by households.

Scale factor for relative purchase prices of labor services.

Price, received by the producer, for domestic production of

good or service i.

Population over 16 years of age.

Supply price of good or service i.

Quantity of delivered energy of fuel type k.

Quantity of total delivered energy.

Quantity of electricity generated.

Quantity of energy input into electricity generation.

Quantity of energy input of fuel type k into electricity

generation.

Total U.S. primary energy input.

Quantity of primary energy input of fuel type k.

Real net claims by the U.S. on the rest of the world.

Government revenue from taxes on capital income.

Government revenue from taxes of labor income.

Rate of return on private domestic capital.

Rate of return on private wealth.

Government revenue from taxes on property.

Government revenue from taxes on wealth.

Government revenue from sales and excise taxes, less subsidies.

Value of gross private saving.

Value of capital gains on private wealth.

Private national wealth.

Exports, NIPA.

Exports price deflator, NIPA.

159
XNIAQ Real exports, NIPA.

XT(i) Total real output from sector i.

Y Value of gross private national income.

YF(i) Real final demand for good or service i.

ZT(i) Total real demand for good or service i.

ZX(i,j) Real transactions of good or service i into sector j.

E.2 EXOGENOUS VARIABLES

The text which follows provides a brief description of each of DGEM '

s

exogenous variables.

CC7 2 Real consumption of goods and services, 1972.

CYQ Aggregation variable between components and total real personal

consumption expenditure.

El Net interest paid by the government to the private domestic

sector

.

EIR Net interest paid by the government to rest of world.

EJ Investment income paid by the government to social insurance

funds

.

EL Government transfers, other than from social insurance funds,

to persons.

ER Government transfers to foreigners.

ET Government net foreign investment.

GYQ Aggregation variable between components and total real

government pruchases.

GZ ( 1

)

Federal defense purchases.

GZ ( 2) Federal nondefense purchases.

GZ ( 3 ) State and local government purchases.

HR Personal transfer payments to foreigners.

IYQ Aggregation variable between components and total gross

private domestic investment.

160
LH Total time available to the household sector.

NRE Net reinvested earnings, rest of world.

PCC72 Price index of consumption goods and services, 1972.

PGD Price of private net claims on the government.

PLB Price, received by households, of labor services.

PMR Price of imported goods and services

PRD Price of net claims on the rest of the world.

RE Current surplus of government enterprises.

RI Excise and sales tax accruals allocated to investment goods.

RLU Rate of civilian unemployment.

RM Aggregation variable between components and total government

revenue

.

RR Customs duties.

RT Personal non-tax payments to the government.

RV Transfers from social insurance funds to the government.

RZ Total real exports of goods and services.

SDR Allocation of special drawing rights.

T(i) Rate of Sales and excise tax, less subsidy, an output from

sector i.

TIME Time or year index, 1972 = 0.

TK Rate of tax on capital income.

TLB Rate of tax on labor income.

TP Rate of tax on property value.

TT(i,j) Rate of excise tax on sales of i to j.

TV Rate of tax on income from net claims on the government and the

rest of the world.

TW Rate of tax on the value of private wealth.

161
u Rate of depreciation of capital.

VR Net interest and corporate profits from the rest of the world.

VRQ Real net interest and corporate profits from the rest of the

world

.

XYQ Aggregation variable between components and total net exports.

162
APPENDIX F: DESCRIPTION OF SUBROUTINES

This appendix provides a brief description of all subroutines in the model

For the MAIN program, a more detailed description by functional block is given

in Chapter 7. The appendix is arranged in alphabetical order by subroutine

name. Each subroutine is described on a single program summary sheet. This

summary sheet includes: (a) the name of the subroutine; (b) the call statement

(c) a narrative description; (d) the calling routines; (e) the called routines;

(f) the COMMON blocks referenced; and (g) any modifications which were made to

bring the code up to the ANSI X3. 9-1978 standard for FORTRAN. The information

provided on the summary sheets in conjunction with the model flowchart shown in

Chapter 7 is designed to complement the DGEM documentation reports and to

facilitate the task of effectively maintaining the model. The interactions

among subroutines which are explicitly stated on the summary sheets should also

assist programmers in making any modifications to the source code dictated by

user needs or pecularities of the host operating system.

163
PROGRAM SUMMARY SHEET

Subroutine Name: ARLAb

Call Statement:— ARLAB (ARRAY, NUM ROW, NUM COL, LABEL)

Description:

This subroutine produces as an output to a file or a line printer a label and the values contained in a single-precision , floating-point

array.

Called By: NEWTMA.NEWTM , NEWTMB , AR3LAB , outmod

Calls:

Commons Referenced:

Program Modifications:

Uses FORTRAN CHARACTER statements rather than word bytes.

164
PROGRAM SUMMARY SHEET

Subroutine Name: arlist

Call Statement: arlist(array,nimrow,nijmcol)

Description:

This subroutine produces as an output to a file or a line printer the

values contained in a single-precision, floating-point array.

Called By: NEWTM

Calls:

Commons Referenced:

Program Modifications:

165
PROGRAM SUMMARY SHEET

Subroutine Name: AR3LAB

Call Statement: AR3 LAB (ARRAY , INDEX 1

, INDEX2 , INDEX3 ,NAME)

Description:

This subroutine constructs a subarray from a 3-dimensional

Called By:

OUTM OD , AR4 LAB

Calls: ARLAB

Commons Referenced:

Program Modifications:

array.

Uses FORTRAN CHARACTER statements rather than word bytes.

166
PROGRAM SUMMARY SHEET

Subroutine Name: AR4LAB

Call Statement: ar4lab(ariiay,itier,ilevel,irow,icol,name)

Description

This subroutine constructs a subarray from a 4-dimensional array,

Called By: OUTMOD

CallS: AR3LAB

Commons Referenced:

Program Modifications:

Uses FORTRAN CHARACTER statements rather than word bytes

167
PROGRAM SUMMARY SHEET

Subroutine Name: chropt

Call Statement: chropt

Description:

This routine places state values into the FLDTA array for special

characters according to the character set; it also reads option

information and sets the DEBUG flag accordingly*

Called By: MAIN

Calls:

Commons Referenced: CHRSET ,OPTN

Program Modifications:

Replaces old OPTION routine. Initializes FLDTA in terms of host system

FORTRAN character set rather than OCTAL field codes. Replaces part of

the original parser routines.

168
PROGRAM SUMMARY SHEET

Subroutine Name: CNTRLR

Call Statement: CNTRLR

Description:

This subroutine reads the run control information. For example, the

first and last years of the simulations, whether the simulation is

static or dynamic and the numbers of target-instrument pairs, data

changes and outputs. The names and values of the variables are also

read.

Called By: MAIN

Calls: GETOUT

Commons Referenced: CNTRL , OPTN , OUTPTA , TARINF

Program Modifications:

Replaced non standard free-format read with a standard free-format

read. Inserted OPEN statements. Replaced INCLUDE statements with

COMMON and PARAMETER statements.

169
PROGRAM SUMMARY SHEET

Subroutine Name: convrt

Call Statement: convrt

Description:

This subroutine takes logs of lagged variables®

Called By: main

Calls:

Commons Referenced: cntrl,coeff,endog,exog,lag,

OPTN.TARINF

Program Modifications:

Replaced INCLUDE statements with COMMON and PARAMETER statements.

170
PROGRAM SUMMARY SHEET

Subroutine Name: convt72

Call Statement:

Description:

Converts government purchases and exports to 1972 dollars.

Called By: MAIN

Calls: 9

Commons Referenced: CNTRL , COEFF , ENDOG , EXOG , LAG

,

OPTN,TARINF

Program Modifications:

New routine.

171
PROGRAM SUMMARY SHEET

Subroutine Name: convt82

Call Statement: Convt82

Description:

Converts government purchases and exports to 1982 dollars*

Called By: MAIN

Calls: 0

Commons Referenced: CNTRL , COEFF , ENDOG , EXOG , LAG

,

OPTN s TARINF

Program Modifications:

New routine.

172
PROGRAM SUMMARY SHEET

Subroutine Name: data

Call Statement: datacjyear, lag)

Description:

This subroutine reads data for a given year. LAG is one if retrieving

last year’s data; zero otherwise*

Called By:

MAIN

Calls:

Commons Referenced: CNTRL , COEFF , ENDOG , EXOG , OTPTNM , OUTPTA, TARINF

Program Modifications:

Replaced INLUCDE statements with COMMON and PARAMETER statements. File

read on unit 8 is now defined and opened within the MAIN program.

173
PROGRAM SUMMARY SHEET

Subroutine Name: DECSOL

Call Statement: DECSOL(N,A,F,X,NDIM)

Description:

This subroutine solves a linear system using Gaussian elimination with

partial pivoting, followed by back substitution. It is used in the

Newton’s method procedures and also to solve the linear system of

interindustry transaction balances.

Called By: newTM ,fcrval,newTMa

Calls: SINGE

Commons Referenced:

Program Modifications:

No longer includes subroutine SINGE

PARAMETER statement.

Replaced INCLUDE statement with

174
PROGRAM SUMMARY SHEET

Subroutine Name: ENERGY

Call Statement: ENERGY

Description;

This subroutine calculates the aggregate energy consumption figures

for each fuel and in total for a given year.

Called By: FCRVAL

Calls: o

Commons Referenced: COEFF , ENDOG , EXOG

Program Modifications:

Replaced INCLUDE statements with COMMON and PARAMETER statements

175
PROGRAM SUMMARY SHEET

Subroutine Name: err

Call Statement: err(code,ivar,inrec)

Description:

Prints an error message according to a table of error codes for an

incorrect target-instrument combination®

Called By: parse, numbrs, offset, pack, search

Calls:

Commons Referenced:

Program Modifications:

No longer included within the GETOUT routine® Calling sequence made

explicit. Uses FORTRAN CHARACTER statement rather than word bytes.

176
PROGRAM SUMMARY SHEET

Subroutine Name:

Call Statement: fcr(xn,resid,n)

Description:

This subroutine evaluates the residuals corresponding to the 40

variables in the basis. The first residual corresponds to the variable

IVT, the next 36 residuals correspond to the vector PFM(i), the next

residuals are for PKD, PLM and RNW, respectively. In addition, each

equality constraint imposed on the model introduces a new equation;

these equations are included in the block of residuals. The variable

corresponding to each of these equations is the instrument nominated

by the user.

Called By: NEWTM

Calls: FCRVAL

Commons Referenced: cntrl ,coeff , endog , exog , lag , optn , lAKiiNr , i £j£\.n.

, unttr

Program Modifications:

Replaced INCLUDE statements with COMMON and PARAMETER statements.

177
PROGRAM SUMMARY SHEET

Subroutine Name: fcra

Call Statement: FC ra(res)

Description:

Upon receiving values for the output prices, PO(i), from NEWTMA, FCRA

evaluates the KLEM shares (capital, labor, energy, material) and price

residuals implied by the price frontiers®

Called By: NEWTMA

Calls: SHPRA

Commons Referenced: CNTRL , COEFF , ENDOG , EXOG , ICOB , LAG , OPTN

,

— = TIERH ,TIERP

Program Modifications:

No longer included within the NEWTMA routine. Replaced INCLUDE

statements with COMMON and PARAMETER statements.

178
PROGRAM SUMMARY SHEET

Subroutine Name: FCRB

Call Statement: fcrb(res)

Description:

This subroutine calculates residuals for the household submodel.

Called By: NEWTMB

Calls: SHPRB

Commons Referenced: CNTRL , COEFF , ENDOG , EXOG , LAG , OPTN

,

TIERH ,TIERP

Program Modifications:

No longer included within the NEWTMB routine. Replaced INCLUDE

statements with COMMON and PARAMETER statements.

179
PROGRAM SUMMARY SHEET

Subroutine Name: FCRVAL

Call Statement: FCRVAL

Description:

Forms primary input prices and estimates of output prices. This

subroutine contains most of the equations of the model. The output

prices are calculated numerically by a Newton's method procedure in

NEWTMA. Consumption patterns are calculated numerically by NEWTMB.

Called By: FCR, NEWTMA

Calls: DECSOL , ENERGY , NEWTMA , NEWTM B , NIPABR

Commons Referenced: CNTRL , COEFF , ENDOG , EXOG , LAG , OPTN

,

TARINF ,TIERH ,TIERP

Program Modifications:

Replaced INCLUDE statements with CCMMON and PARAMETER statements.

180
PROGRAM SUMMARY SHEET

Subroutine Name: GETOUT

Call Statement -

: getout (numvar,locvar,keyvar,savnam,strtnm , NAM F IN , NM FLAG

)

Description:

Parses target inputs and calculates pointers to the endogenous and

exogenous variable common blocks.

Called By: CNTRLR

Calls: OFFSET, PARSE, SEARCH

Commons Referenced: b,chrset,dimsiz,optn

Program Modifications:

No longer includes Subroutines: ERR; PARSE; PACK; OFFSET; NUMBRS; and

SEARCH. Uses FORTRAN CHARACTER statements rather than word bytes.

Changes in COMMON blocks and initialization for parsing variable names

and subscripts.

181
PROGRAM SUMMARY SHEET

Subroutine Name: guess

Call Statement: GUESS (XN,NVAR)

Description:

This subroutine reads the base case values of the endogenous variables

(the ENDOG COMMON block) and extracts the values of the basis variables

used in the numerical solution of the model. When targets are being

enforced, the actual values of the nominated instruments are also

extracted. These values are inserted into the vector XN and serve as

the initial guesses of the variables in the numerical solution of the

model

.

Called By: main

Calls:

Commons Referenced: coeff, endog, exog,optn,tarinf

Program Modifications:

Replaced INCLUDE statements with COMMON and PARAMETER statements.

182
PROGRAM SUMMARY SHEET

Subroutine Name: NEWB1

Call Statement: newtm(x,n,*)

Description:

This subroutine contains a first order Newton's method algorithm for the

solution of an equation system; it solves for values of the N basis

variables. Equilibrium is reached when all N residuals are

simultaneously zero.

Called By: MAIN

Calls: ARLAB , ARL 1ST , DECSOL , FCR , FCRVAL , 0U1M OD

Commons Ref eren ced: cntrl , coeff , endog , exog , lag , optn , tarinf

Program Modifications:

Replaced INCLUDE statements with COMMON and PARAMETER statements.

Changed value of test variable, CONV , for the sum squared of the

residuals

.

183
PROGRAM SUMMARY SHEET

Subroutine Name: NEWTMA

Call Statement: NEWTMA

Description:

This subroutine is a simplified Newton's method algorithm for the

solution of a system of equations. It is set up specifically for the

solution of the price possibility frontiers for the values of the

sectoral output prices, PO(i). The model equations used in this

solution are in subroutines FCRA and SHPRA.

Called By: FCRVAL

Calls: ARLAB , DEC SOL , FCRA

Commons Pel@fep Qgd: cntrl , coeff , endog , exog , lag , optn ,

i 1 rLKtl , I ir,Kr

Program Modifications:

No longer includes subroutines FCRA and SHPRA. Replaced INCLUDE

statements with COMMON and PARAMETER statements. Changed value on

test variable, CONVY, for the sum squared of the residuals.

184
PROGRAM SUMMARY SHEET

Subroutine Name: NEWIMB

Call Statement: NEWIMB

Description:

This is a simplified Newton's method algorithm used to solve for the

expenditure weighted consumption price PCC. The equations corresponding

to these prices are in subroutines FCRB and SHPRB

Called By: FCRVAL

Calls: ARLAB , FCRB

Commons Referenced: CNTRL , COEFF , ENDOG , EXOG , LAG , OPTN

,

TIERH , TIERP

Program Modifications:

No longer includes subroutines FCRB and SHPRB. Replaced INCLUDE

statements with COMMON and PARAMETER statements. Changed value on

test variable, CONVY , for the sum squared of the residuals.

185
PROGRAM SUMMARY SHEET

Subroutine Name: NIPABR

Call Statement: NIPABR

Description:

This subroutine calculates the values of the components of the National

Income and Product Accounts (GNP, consumption, investment, government,

exports, imports ) . These aggregates are constructed from the solved

values of the endogenous variables and the exogenous conditions that

determine them.

Called By: FCRVAL

Calls:

Commons Referenced: COEFF , ENDOG , EXOG , LAG

Program Modifications: Changed initialization procedure for aggregation of GNP components

186
PROGRAM SUMMARY SHEET

Subroutine Name: numbrs

Call Statement: NUMBRS(RESULT,NUMCHR, IVAR, ITRING, INREC)

Description:

Packs 3 digits from vector ITRING into RESULT.

Called By: PARSE

Calls:

ERR

Commons Referenced:

Program Modifications:

No longer included within GETOUT routine. Uses FORTRAN CHARACTER

statements rather than word bytes.

187
PROGRAM SUMMARY SHEET

Subroutine Name: offset

Cal! Statement: OFFSET( IVAR , SUBl , SUB2 , SUB3 , SUB4 , POINT , inrec

,

KEYVAR,LOCVAR)

Description:

Calculates the location of a variable in a COMMON block*

Called By: getout

Calls. krr

Commons Referenced: DIMSIZ , OPTN

Program Modifications:

No longer included within the GETOUT routine*

188
PROGRAM SUMMARY SHEET

Subroutine Name: outbtu

Call Statement: OUTBTU( JYEAR)

Description:

This subroutine prints out the national energy transactions in

quadrillion BTU.

Called By: main

Calls:

Commons Referenced: coeff.endog

Program Modifications:

Replaced INCLUDE statements with COMMON and PARAMETER statements. Uses

FORTRAN CHARACTER statements rather than word bytes.

189
PROGRAM SUMMARY SHEET

Subroutine Name: OUTMOD

Call Statement: OUTMOD( JYEAR)

Description:

This subroutine prints out the values of the endogenous variables

exogenous variables and coefficients.

Called By: MAIN ,NEWTM

Calls: ARLAB , AR3 LAB , AR4 LAB

Commons p@f@f@n C0d: B , COEFF , ENDOG , EXOG , LAG , OPTN , TARINF

Program Modifications:

Replaced INCLUDE statements with COMMON and PARAMETER statements. Uses

FORTRAN CHARACTER statements rather than word bytes. Modified CALL

statement for subroutines, ARLAB, AR3LAB and AR4LAB.
PROGRAM SUMMARY SHEET

Subroutine Name: outnia

Call Statement: outnia(jyear)

Description:

This subroutine prints out the National Income and Produce Accounts for

a given year.

Called By: main

Calls:

Commons Referenced: ENDOG , EXOG , OPTN

Program Modifications:

Replaced INCLUDE statements with COMMON and PARAMETER statements. Uses

FORTRAN CHARACTER statements rather than word bytes.

191
PROGRAM SUMMARY SHEET

Subroutine Name: OUTPUT

Call Statement: OUTPUT( YEARl ,YEAR2)

Description:

This subroutine prints an output report for each variable requested by

the user. The simulated values are compared to the basecase values

in each report.

Called By: MAIN

Calls:

Commons Referenced: OPTN , OTPTNM , OUTPTA , OUTPTB

Program Modifications:

Replaced INCLUDE statements with COMMON and PARAMETER statements. No

longer includes functions DIVCHK and CHK. Conditional OPEN statement

for storing simulated values of special variables.

192
PROGRAM SUMMARY SHEET

Subroutine Name: PACK

Call Statement: pack( name, string, inrec,numchr,ivar)

Description:

Packs 6 characters from the vector STRING into NAME

Called By; PARSE

Calls; ERR

Commons Referenced:

Program Modifications:

No longer included within the GETOUT routine

statements rather than word bytes.

Uses FORTRAN CHARACTER

193
PROGRAM SUMMARY SHEET

Subroutine Name: parse

Call Statement:

PARSE ( ITHRU , IVAR , INREC , I ICHAR , NM FLAG , SAVNAM , NAMFIN.STRTNM, STRING, ITRING, SUB1 ,SUB2 ,SUB3,SUB4,

NAME.NUMCHR)

Description:

Parses card image into variable name and subscripts if appropriate.

This subroutine expects variable names to be valid FORTRAN variable

names. Presence of parentheses implies one, two, or three subscripts,

separated by a comma.

Called By: GETOUT

• Calls: ERR, NUMBRS, PACK

Commons Referenced:

CHRSET,OPTN

Program Modifications:

No longer included within the GETOUT routine. This is the main routine

where FORTRAN CHARACTER statements replaced word bytes.

194
PROGRAM SUMMARY SHEET

Subroutine Name: savlag

Call Statement: savlag

Description:

This subroutine extracts the variables that need to be saved for the

next year's simulation. The first year's variables enter through the

ENDOG, EXOG, and COEFF COMMON blocks; those variables needed for the

next year's simulation are stored in the LAG COMMON block.

Called By: MAIN

Calls:

Commons Referenced:

COEFF , ENDOG , EXOG , LAG

Program Modifications:

Replaced INCLUDE statements with COMMON and PARAMETER statements.

195
PROGRAM SUMMARY SHEET

Subroutine Name: savout

Call Statement: savoutcitime.iflag)

This subroutine saves the values for each variable for which an output

report was requested by the user, These values are used for

comparison and for calculating rates of change*

Called By: MAIN

Calls:

Commons Referenced: CNTRL , COEFF , ENDOG , EXOG , OPTN , OTPTNM

,

OUTPTA, OUTPTB

n Program Modifications:

Replaced INCLUDE statements with COMMON and PARAMETER statements.

196
PROGRAM SUMMARY SHEET

Subroutine Name: search

Call Statement: search(name,inrec,ivar,keyvar, point)

Description:

Finds pointers to variable names in the list of variable names.

Called By: GETOUT

Calls: ERR

Commons Referenced: b

Program Modifications:

No longer included within the GETOUT routine. Uses FORTRAN CHARACTER

statements rather than word bytes.

197
PROGRAM SUMMARY SHEET

Subroutine Name: setdat

Call Statement: SETDAT (JYEAR)

Description:

This subroutine writes data for a given year to a new file; it is

activated by the M option* Since it causes a new database to be

created, the M option should be used with caution.

Called By: MAIN

Calls:

Commons Referenced:

CNTRL , COEFF , ENDOG , EXOG

Program Modifications:

Replaced INCLUDE statements with COMMON and PARAMETER statements. File

written on unit 3 is now defined and opened within the MAIN program.

198
PROGRAM SUMMARY SHEET

Subroutine Name: shpra

Call Statement: shpra(index,js,icd)

Description:

This subroutine calculates the energy and materials shares and the

corresponding price aggregates.

Called By: FCRA

Calls:

Commons Referenced: CNTRL , COEFF , ENDOG , EXOG , ICOB , LAG , OPTN

,

TIERH,TIERP

Program Modifications:

No longer included within the NEWTMA routine. Replaced INCLUDE

statements with COMMON and PARAMETER statements.

199
PROGRAM SUMMARY SHEET

Subroutine Name: shprb

. » 1 1 1 1 ' —— " ' —

Call Statement: S hprb( index)

Description:

Determines consumption expenditure patterns.

Called By: fcrb

Calls:

Commons Referenced: cntrl,coeff,endog,exog,lag,optn, “ ‘ TIERH.TIERP

Program Modifications:

No longer included within the NEWTMB routine. Replaced INCLUDE

statements with COMMON and PARAMETER statements.

200
PROGRAM SUMMARY SHEET

Subroutine Name: singe

Call Statement, singe (loop)

Description:

This subroutine prints the message that a singularity was encountered

in DECSOL for a given loop number.

Called By: decsol

Calls :

Commons Referenced:

Program Modifications:

No longer included in the DECSOL routine.

201
PROGRAM SUMMARY SHEET

Subroutine Name: VARY

Call Statement: vary

Description:

Determines the value for any endogenous or exogenous variable or

coefficient which is being scaled or preset by the user.

Called By: MAIN

Calls:

Commons Referenced: CNTRL , COEFF , ENDOG , EXOG , OPTN , TARINF

Program Modifications:

Replaced INCLUDE statements with COMMON and PARAMETER statements.

202
FEDERAL INFORMATION PROCESSING STANDARD SOFTWARE SUMMARY

01. Summary date 02. Summary prepared by (Name and Phone)

Robert E. Chapman (301)975-2723

03. Summary action New Replacement Deletion

E j Previous Internal Software ID

Yr. Mo. Day

8 .9 0 1 0 6 05. Software title Dynamic General Equilibrium Model (DGEM) 04. Software date

Yr. Mo. Day

8 9 0 1 0 6 07. Internal Software ID

06. Short title

08. Software type

Automated Data

| | System

m Computer Program

Subroutine/ Module

09. Processing mode

| | Interactive H Batch

[X] Combination

10. General

Computer Systems

Support/Utility g

[y] Scientific/Engineering Q

J Bibliographic/Textual

Application are

Management/

Business

Process Control Other

3 cSpecific

Macroeconomic Modeling

11. Submitting organization and address

Center for Computing and Applied Mathematics

National Institute of Standards & Technology

Gaithersburg, MD 20899

12. Technical eontact(s) and phone

% Robert E. Chapman

(301)975-2723

i

13. Narrative

DGEM is an annual model for analyzing the structure and growth of the U.S. economy.

DGEM provides a detailed analysis of supply and demand factors for simulations up

through the year 2000. The model was designed to deal with three major types of

emergency situations? (1) energy-economic interactions associated with supply

interruptions or strategic policies; (2) the economic impacts of a loss of resources

due to an enemy attack on the United States with particular emphasis on which policies

could stimulate recovery; and (3) the economic impacts of demand surges and resource

constraints typical during a period of military mobilization.

14. Keywords

economic analysis; emergency management; mathematical model; policy analysis

I

15. Computer manuf'r and model 16. Computer operating system 17. Programing language(s) 18. Number of source program state- ments

DEC VAX 11-780 series VMS FORTRAN 77 APPX 10000

19. Computer memory requirements 20. Tape drives 9 track

21. Disk/Drum units 22. Terminals

1

1 M Byte 1600 or 6250 FPI 1

23. Other operational requirements

24. Software availability 25. Documentation availability

Available Limited In-house only Available Inadequate

3 m

26. FOR SUBMITTING ORGANIZATION USE

In-house only

I

l

j

|

j

Standard Form 185

1974 July

U.S. Dept, of Commerce—NBS

(FIPS. Pub. 30)

185-101
FORM NBS-1 14A (REV 1 1 84

U.S. DEPT. OF COMM. 1. PUBLICATION OR 2. Performing Organ. Report No. 3. Publication Date

BIBLIOGRAPHIC DATA

REPORT NO.

SHEET (See instructions) NISTIR 88-3826 MARCH 1989

4. TITLE AND SU BTITLE

Evaluating Emergency Management Models and Data Bases: A Suggested Approach

5. AUTHOR(S)

Robert E. Chapman, Saul I. Gass, James J. Filliben and Carl M. Harris

6. PERFORMING ORGANIZATION (If joint or other than N BS , see in struction s) 7. Contract/Grant No.

NATIONAL BUREAU OF STANDARDS

U.S. DEPARTMENT OF COMMERCE 8 . Type of Report & Period Covered

GAITHERSBURG, MD 20899 4

9„ SPONSORING ORGANIZATION NAME AND COMPLETE ADDRESS (Street. C/ty. State, ZIP)

Federal Emergency Management Agency

500 C Street, SW

Washington, DC 20472

10.

SUPPLEMENTARY NOTES

Document describes a computer program; SF-185, FlPS Software Summary, is attached,,

11.

ABSTRACT I A 200-word or less factual summary of most significant in formation . If document includes a significant

bi bliograohy or literature survey , mention it here)

The evaluation of large-scale mathematical models requires a set of analytical

techniques which focus on quantitative rather than qualitative issues. The four

major classes of analytical techniques described in this document are based on a thorough review of the technical literature in model and data base evaluation,

practical experience, and original research on model information development. The

analytical techniques discussed in this document include: (1) the theoretical

considerations upon which a data base or the model's formulation rests; (2) program

documentation and verification; (3) data base audits; and (4) the technical

considerations associated with an in-depth sensitivity analysis. The individual

techniques are designed in such a manner that they generate a framework of

structural information and analysis which supports an in-depth model/data base

evaluation.

12.

KEY WORDS (Six to twelve entries; alphabetical order; capitalize only proper names; and separate key words by semicolon s) Assessment; data base; data quality; documentation; emergency planning; evaluation;

mathematical model; model credibility; policy models; sensitivity analysis

13, AVAILABILITY 14. NO. OF

PRINTED PAGES

X Unl imited

For Official Distribution, Do Not Release to NTIS 217

Order From Superintendent of Documents, U.S. Government Printing Office, Washington, DC

~ 20402. 15. Price

Order From National Technical Information Service (NTIS), Springfield, VA 22161

1 1 $25.95

USCQMM-DC 85-6006