ARI Research Note 86-14

N

TRAINING EXTENSION COURSE COST AND TRAINING

CEFFECTIVENESS ANALYSIS METHODOLOGY

IPeter G. Sassone, James Bercos

Mellonics Systems Development Division

Litton Systems, Inc.

and

John E. Holmgren

ARI Fort Benning Field Unit

ILECTE

for S-D

ARI Field Unit at Fort Benning, Georgia

TRAINING RESEARCH LABORATORY

Seward Smith, Acting Director

U. S. Army

Research Institute for the Behavioral and Social Sciences

M" January 1986

'86 6 2 050

Approved for Public release; distribution unlimited.

I.
U. S. ARMY RESEARCH INSTITUTE

FOR THE BEHAVIORAL AND SOCIAL SCIENCES

A Field Operating Agency under the Jurisdiction of the

Deputy Chief of Staff for Personnel

W. DARRYL HENDERSON

EDGAR M. JOHNSON COL, IN

Technical I~irector Commanding

IThis report. as submitted by the contractor, has been cleared for release to Defense Technical Information Center

(OTIC) to comply with regulatory requirements. It has been given no primary distribution other than to DTIC

and will be available only through OTIC or other reference services such as the National Technical Information

Service (NTIS). The vicws. opiniens, and/or findings contained in this repor mare those of the authorislI and

should not be construeJ as an officia; Depattmemi of the Army position, policy. or decision. unless so designated

by other official documentation.

5A,
IM'A RT FT1fl

SECURITY CLASSIFICATION OF THIS PAGE (When Date Entered)

REPORT DOCUMENTATION PAGE BRE C0ST1PTIO1. REPORT NUMBER 2. GOVT ACCESSION NO 3. RECIPIENT'S CATALOG NUMBER

ARI Research Note 86-14 __ _ _ _ _

4. TITLE (md Subtitle) S. TYPE OF REPORT & PERIOD COVERED

TRAINING EXTENSION COURSE COST AND TRAINING Interim Report

EFFECTIVENESS ANALYSIS METHODOLOGY Dec. 1976 - Aug. 1979

6. PERFORMING ORG. REPORT NUMBER

7. AUTHOR(&i) I. CONTRACT OR GRANT NUMBER(&)

Peter G. Sassone, James Bercos, DAHC 19-77-C-0011

and John E. Holmgren

9. PERFORMING ORGANIZATION NAME AND ADDRESS 10. PROGRAM ELEMENT. PROJECT. TASK

Mellonics Systems Development Division AREA & WORK UNIT NUMBERS

Litton Systems, Inc., P.O. Box 2498 2Q763731A770

Fort Benning, Georgia 31905

I. CONTROLLING OFFICE NAME AND ADDRESS 12. REPORT DATE

ARI Field Unit at Fort Benning, Georgia January 1986

P.O. Box 2086, 13. NUMBER OF PAGES

Fort B enning, GA 31905 59

14. MONITORING AGENCY NAME& ADDRESS(if different i Controlling Oic) 1S. SECURITY CLASS. (of this report)

U.S. Army Research Institute for the Behavioral

and Social Sciences, 5001 Eisenhower Avenue, Unclassified

Alexandria, VA 22333-5600 ISa. DECLASSIFICATION/DOWNGRADING

SCHEDULE

IS. DISTRIBUTION STATEMENT (of thi. Report)

Approved for public release; distribution unlimited.

17. DISTRIBUTION STATEMENT (of the abstract entered In Block 20. II different frem Report)

IS. SUPPLEMENTARY NOTES

Seward Smith, contracting officer's representative.

19. KEY WORDS (Continue on reverse side It necessary and identify by block nimber)

Training Extension Course (TEC) ATEC Lessons,

Training Extension Material TEC Manager

Training Effectiveness Analysis TEC Validation-...

Training Methodology

2(L ABTRACT' (Cinthe - res s ift noeeery a" Idefilt by block ntmber)

• -This report addresses the development of a methodology for Training Extension Course (TEC) Cost Training Effectiveness Analysis (CTEA), where effectiveness is a function of unit performance as measured by the Army Training and

Evaluation Program (ARTEP). This report deals specifically with the theoretical

development of the methodology. .

DD F 14'0 EDITION OF f NOV £5 IS OBSOLETEU

DE ,T 7CL1473 UNCLASSIFIED

i SECURITY CL.ASSIFICATION OF TpjS PAGE ('lF~mDaro Entered)
FOREWORD

This task report is one of several provided by the Mellonics

Systems Development Division of Litton Systems, Inc., to the Army

Research Institute for the Behavioral and Social Sciences '(ARI)

under Contract Number DAHC 19-77-C-0011.

Under the contract, a part of the Mellonics effort concerns

support to the ARI evaluation of the utilization, acceptance, costs , and effectiveness of the Army Training Extension Course (TEC) Program.

Related earlier reports focused on the cost and training effectiveness

of TEC where effectiveness was a function of individual performances

on selected hands-on performance tests, and the relation of TEC usage

to individuals performances on their SQT. Results showed TEC effective

in training individuals and a positive relationship between TEC usage

and SQT scores.

The first TEC reportI in this current series documnts the conduct and findings of a training, cost and effectiveness literature

search and review of selected literature. It serves as the introduction to other reports in this series. The second report 2 provides

guidance for the collection of individual and unit training data

necessary to support the TEC Cost Training Effectiveness Analysis

(CTEA) Methodology. This report addresses the development of a TEC

CTEA Methodology, where effectiveness is a function of unit performance as measured by the Army Training and Evaluation Program (ARTEP)

evaluation. This report is the theoretical development of the Methodology. A planned subsequent report will deal with the practical

application of the Methodology.

_____________________-

iSassone, P. G. Literature Review - Cost and Training Effectiveness. 0

Litton-Mellonics, Training Extension Course Research Task Report:

July 1978.

2Bercos, J. Extension of Training Extension Course Cost and Training

Effectiveness Analysis Data Collection. Litton-Mellonics, Training

Extension Course Research Task Report: June 1979. lodes

iA t~d /or

.. S 0 0 0 0 * S 9° 0 Di~t PLcial

• " .

4 " ' .%4' ,'4 ' ~ , '',", 4" .'4"

A L L i..
TRAINING EXTENSION COURSE COST AND TRAINING EFFECTIVENESS ANALYSIS

METHODOLOGY

BRIEF

Requirement:

To develop a Cost and Training Effectiveness Analysis (CTEA)

Methodology that will identify the contributions of Training Extension Course (TEC) training to a unit performance as measured by the

Army Training and Evaluation Program.

Procedure and Findings:

The development of the Methodology required four areas of specification. First, a measure of unit effectiveness was assumed to be

derivable from the pass/fail (satisfactory/unsatisfactory) scoring

criteria currently in use. Secondly, the assumption was made that

costs of field, garrison, and TEC training can be determined. Thirdly,

the resource allocation model was developed from which can be inferred

the economic value of TEC. The effectiveness production function

assumes some specifiable relationship between inputs and outputs of

- the training process which operates under specific budget constraints.

Lastly, the use of the valuation methodology in a present value

analysis to derive an ultimate dollar value of TEC was outlined.

Utilization:

This report describes a CTEA Methodology which integrates implicit

knowledge of the training process and training budget constraints to

evaluate the effectiveness of TEC training. The unit is the object

of the analysis which is evaluated by means of ARTEP procedures.

Since TEC usage can change both effectiveness and costs, the valuation methodology was developed from a cost-benefit point of view.

However, this valuation methodology can be applied to any training

program which serves a complementary, rather than as a substitute,

training program.

V

3 ' 4'r W W_ W 0 W 0 M

5% % ,w ~ 6 2 %~ V ~ '~US~ ~2-'-~.~ %U hl ; .. - U ~'-U.
* i,

TABLE OF CONTENTS

Page

INTRODUCTION 1

The TEC Program 1

TEC Related Research 1

Purpose 3

Approach 4

Organization of the Report 6

TEC CTEA METHODOLOGY 7

Effectiveness Production Function 7

Cost of Training 9

Economic Theories 9

Army Training Adaptation 10

Development of the Valuation Methodology 13

Algebraic Example of the Valuation Methodology 40

Present Value Analyses 47

SUMMARY AND FURTHER CONSIDERATIONS 51

FIGURES

Figure 1. Cross-sectional view of the Effectiveness 16

Production Function: Effectiveness Versus

Training.

Figure 2. Cross-sectional view of the Effectiveness 18

Production Function: Training Versus Training.

p Figure 3. Three Dimensional View of the Effectiveness 20

Production Function.

Figure 4. Illustration of Solution to the Training 25

Resource Allocation Problem.

Figure 5. A shift in the Per Unit Cost of One Type 27

of Training.

Figure 6. The Training Demand Curve. 27

vii

... * 'V w 'w w w w " U S V w e W W-,

-' .I," ' .'i. . . %.%'% . .".,-"".''.''.'DP. 'V..:. . ..".''.' .,, , "..u" '-'. %'..', ,-V- '.- ; %1.!'. ,"-'' 'I 1' - _ . . ,b ' .'- ,'.,%\-L''.' ..,-'. "-.'-,'-"..L','-', *.
FIGURES (CONTINUED) Page

Figure 7. An Increase in Budget Causes an Increase 29

in T1.

Figure 8. An Increase in Budget Causes a Decrease 29 I

in Tl.

Figure 9. An Increase in Budget Causes No Change 29

in T 1.

Figure 10. First Geometric interpretation of Methodology. 36

Figue 1. Inerpetaton SeondGeoetri f Mthodlog. 3

Figure 12. Third Geometric Interpretation of Methodology. 39

TABLES

Table 1 Comparison of Aggregation Methods 49

vii

xi-.

%~
TRAINING EXTENSION COURSE (TEC) COST AI!D TRAINING EFFECTIVENESS A

ANALYSIS (CTEA) METHODOLOGY

INTRODUCTION

THE TRAINING EXTENSION COURSE (TEC) PROGRA-M

TEC is a program that has been designed to put into the hands

of trainers, both in units and in institutions, high quality performance-oriented multimedia training packages. It is designed to

provide soldiers with immediate access to self-paced instruction

especially designed to assist them in acquiring and maintaining

skills critical to their performance in combat. TEC lessons are

designed for use on an individual basis; however, they may be used

by small groups, under the supervision of an NCO.

TEC RELATED RESEARCH

The advent of the TEC program inaugurated a multi-faceted TEC

research effort conducted under the sponsorship of the U. S. Army

Training Support Center (USATSC) by the U. S. Army Research Institute for the Behavioral and Social Sciences (ARI) and the Mellonics

Systems Development Division of Litton Systems, Inc. The research

comprises investigations concerning current and projected usage of

TEC in the Active and Reserve Components, training effectiveness and

retention of TEC instruction, current and projected costs of the TEC

program, cost-effectiveness of TEC, and the development of a TEC

Manager's Guidebook.

o The effectiveness and retention 3 investication evaluated

TEC lessons relative to conventional instruction in the

Active and Reserve Components. The expericments included

five subject areas, one common to all combat arms soldiers

and four specific to each of the four cornzat arms. The

investigation determined that, averaged across the five

subject areas, the TEC trained soldiers performed better

than the conventionally trained soldiers on both the inib tial and retention (8 to 9 weeks after the initial) handson performance tests.

3 Holmgren, J. E., Hilligoss, R. E., Swezey, R. W. and Eakins, R. C.

The Training Effectiveness and Retention of TEC Instruction in the

Combat Arms. U. S. Army Research Institute for the Behavioral and

Social Sciences, Research Report Draft: "May 197.3.

%~ % .

r . . 4.

.,% *1..*

hu~ mu a.~ hw~ jl u~~ I~uh h Mf%~ A%' %
o An effectiveness and retention related investigation4

concerned the relationship of performance on the Skill

Qualification Test (SQT) ard the amount of TEC use.

The investigation was based on TEC usage data (TEC

usage investigation above) and SQT data obtained from 7

USATSC for the same personnel (3 Armor and 3 Infantry

battalions) to whom the usage data applied. Correlation analyses showed a consistent positive relationship

between the use of TEC lessons and performances on the

relevant SQT.

o The investigation of TEC program costs5 produced cost

information for use in a TEC cost and Training Effectiveness Analysis (CTEA). The information was developed

in terms of sunk costs (monies expended or obligated in

the period prior to FY 1978) and future costs (estimates

of annual costs to be incurred in continuing a given TEC

program during the period FY 1978 through FY 1987). Both '

costs, sunk and future, include contract costs and other

government costs. From the sunk costs and other information, estimates were developed for the cost of TEC to a

battalion.

o An adjunct TEC cost investigation6 concerned cost savings

or cost avoidance to the U. S. Army resulting from the

use of already available TEC materials to replace or

supplement conventional instruction at the TRADOC Branch

Schools. Data for FY 1977 collected from the TRADOC

schools showed real and potential annual cost savings

from the direct use of TEC materials in instructional

training and cost avoidance (no need to develop materials)

from TEC substitution for conventional instruction in new

or self-paced training.

4 Strasel, H. C., Holmgren, J. E., Bercos, J., Shafer, J. C., and

Eakins, R. C. Training Extension Courses (TEC): Cost and Training

Effectiveness. U. S. Army Research Institute for the Behavioral and

Social Sciences, Research Report Draft, November 1977.

~~55 Ibid.

6 1bid.

.J. .. %% . . % . j ,% , % %%. - .- 2

71"

% V,.

-g * 6 *. 0 0 0 S S S S %
o A second adjunct cost investigation concerned the quantification (and expression in dollars) of TEC benefits.

The benefits were estimated as potential savings of

-pItraining resources (time and azmunition) resulting from

TEC trained individuals' better performance on selected

tasks than conventionally trained individuals for a

period of combat/intensive training. The investigation

showed the potential is real to achieve a given level

of soldier proficiency with a reduced period of combat/

intensive training or to exceed the given level of

soldier proficiency in the normal period.

o The cost-effectiveness analyses7 integrated the data and

results from the above outlined research, and also used

the data and results to assess the relative worth of TEC.

For the relative worth analysis, an estimated cost of a

TEC impression (a single use of a TEC lesson by one individual) was developed, as was a model to estimate indi- *.-

vidual soldier proficiency in relation to training on

specific tasks conventionally and with TEC. The relative

worth analysis indicated that, at the current usage level,

TEC training is equivalently cost-effective as conventional \ % 

training, and that at higher usage levels, TEC can be more

cost effective.

The research outlined above mainly addressed the cost and training

effectiveness of TEC instruction relative to the individual combat ..

arms soldier. Obviously, the same or extended analyses are needed for

units. In this connection, the USATSC has defined a requirement for

Cost and Training Effectiveness Analysis (CTEA) of TEC in relation to

unit performance as demonstrated in an Army Training and Evaluation

Program (ARTEP) evaluation as discussed in the preceding TEC report.

Because of TEC's unique complementary role, it does not fit the usual

training program mold, and analytical techniques available in current

training literature are not completely suitable for a TEC evaluation.

General guidance for a TEC CTEA is available; nonetheless, it appears

a TEC-specific evaluation methodology must be developed.

PURPOSE

The purpose of this research is to develop a TEC CTEA Methodology

for determining the value of TEC in terms of unit performance on the

ARTEP evaluation.

7 Ibid

3

Ww. w W IV w - W W 0 V lipU U

% * -.

*5...........-"
APPROACH

General. The overall research was envisioned as an extension

of the CY77 TEC CTEA (outlined above) to include investigation of

the relations among TEC use and both unit and individual training

costs and effectiveness. The research would involve development

of a TEC CTEA methodology, data collection, and application of the S -

methodology to an analysis of the data.

A data collection plan - including the development of forms,

instructions for their use, determination of the types and numbers

of units required to provide data, and procedures for the implementation of data collection - has been drawn. Commencement of data

collection, however, pends the identification of specific units to

participate. Accordingly, the application of the methodology to

4. an analysis of empirical data is pending. As an interim offset,

therefore, the methodology was applied using hypothetical data.

The development of the methodology for determining the value of

TEC commenced with a review of literature 8 concerning the current

state of the art in CTEA methods. Although the review disclosed

no readily available CTEA methodology suitable for TEC, it provided

an essential overview of CTEA requirements.

TEC CTEA Requirements. CTEA in general must involve a comparison of the effectiveness and costs of alternative training systems.

Since any decision regarding a modification of any part of the training system must be made in consideration of the effects of that modification on the effectiveness of the force structure, a CTEA must '

relate a (proposed or actual) change in the training system to its

ultimate impact on combat effectiveness. This means the CTEA must

be performance-oriented. It also means that any proposed alterna- *

tives must be compared with the baseline system, which is the best

currently available training system. The comparison can be of three-*

different types: equal costs, unequal effectiveness; equal effectiveness, unequal costs; or unequal costs and unequal effectiveness.

The TEC program is primarily a complementary, rather than a substitute, training program. That it, it does not substitute for a

different training program (although it can be used, selectively, to

substitute for parts of the existant training program). It is intended to be used in conjunction with other existing programs. This

8 Sassone, Literature Review -Cost and Training Effectiveness.

4

W. V4P 4 W 4

4J.

N'..".'*

li "2."

.- -'.
complementary nature is a critical point, because it serves as a

basis for evaluating TEC as an addition to existing training

programs.

Since TEC is intended neither as strictly a cost-substitute

program to increase effectiveness at the same level of costs, nor

as strictly an effectiveness-substitute program to decrease costs

while maintaining effectiveness, the traditional cost-effectiveness

approach to training program evaluation is not applicable. That is,

it is not appropriate to evaluate TEC as though it were strictly :.

intended to minimize costs while maintaining effectiveness or to

maximize effectiveness while maintaining costs. Rather, it must be

explicitly recognized that TEC could simultaneously change both

effectiveness and costs. The issue then becomes: Is the change in

effectiveness worth the change in costs? This question demands a

cost-benefit9 , rather than a cost-effectiveness, point of view.

Accordingly, with account of the usual CTEA considerations1 0 and

Va TEC CTEA's kinship to a cost-benefit analysis, five methodological

4 .' requirements for a TEC CTEA were identified. t.

o the baseline training system must be identified

and its parameters quantified.

o the training system inclusive of TEC must be defined

and its parameters quantified.

o An input cost function must be developed which maps

any training system configuration into its dollar

cost.

o an approach must be devised which objectively transa. lates changes in effectiveness into corresponding

dollar values.

'The development of a TEC evaluation methodology to meet the requirements is susceptive to the methods of microeconomics and welfare

economies. The evaluation procedures are objective, and result in

ultimate dollar valuation of the TEC training concept.

9 The cost-benefit approach is sometimes adopted in private sector studies,

but these benefits are estimated as increases in wages. This approach is

A clearly inappropriate for military applications.

1 0 U.S. Army TRADOC Systems Analysis Activity. Cost and Training Effectiveness (CTEA) Handbook. White Sands Missile Range, New Mexico. Author, 1976.

5 

%

,,.- %

°*:, %
Technical Approach. The approach adopted in the development of

this CTEA methodology is an extension of a model and several concepts

from economic theory. The model used here to guide the choice among

various types of training and to assess the value of TEC is an extension of the microeconomic theory of the consumer. Basically, we

substitute a training effectiveness production function for the traditional utility function, where the former has much the same

mathematical structure as the latter. The key concepts enjoyed here

in developing dollar values for the TEC program are the compensatory

budget variation and the equivalent budget variation. These concepts

are central to theoretical welfare economics, where they provide

measures of the welfare effects of price changes on consumers. We

extend these concepts to provide dollar measures of the value of

training programs. The TEC CTEA methodology section report explains

the model and concepts just mentioned.

ORGANIZATION OF THE REPORT

The report comprises three major sections:

o This introduction is the first section. It presents ~ ~ the background, states the objective, outlines the

scope, and generally describes the overall approach

of the investigative effort.

o The second section outlines the TEC CTEA methodology.

It discusses the construction of effectiveness

production functions and procedures to determine

the cost of various types of training. This

section discusses the basis of the methodology

in economic theory, describes its adaption to an

Army Training situation, explicates the development of the valuation part of the methodology,

presents an algebraic example, and shows the use

of the valuation methodology in a present value

analysis to derive an ultimate dollar value of *.-

TEC.

o The last section summarizes the investigative

effort and presents some further consideration.

6

W~W V W 1W 9 W W W W W W M W

S ~ 4, . ~ S~. ~:~ . *. ** ~ .* ~ '4.
197

TEC CTEA METHODOLOGY

The objectives of this section are to develop and explain a

methodology which can be used to provide an objective economic

evaluation of the TEC Program, and very likely, other military

training programs as well. As mentioned above, the methodology

is based on modern welfare economics, that branch of economics

which deals with the normative evaluation of alternative economic

states. The methodology involves four steps:

o Specification and estimation of an Effectiveness

Production Function.

o Estimation of the "per unit" costs of the different

types of training contributing to overall effectiveness, as well as determination of the magnitude of

the training budget.

o Calculation of the current value of the TEC Program.

o Calculation of the present value of the TEC Program.

Each of these steps involves a number of (sometimes technically complex) substeps. The four steps are discussed in this section; however, Step 3, the Valuation Methodology, is the heart of the CTEA

Methodology, and this section of the report deals primarily with the

development of that step.

W~

EFFECTIVENESS PRODUCTION FUNCTION

The first step of the methodology is to determine the effectiveness production function. A valid measure of unit combat effectiveness must exist in order to carry out the valuation methodology. It

is assumed that a unit's performance on selected missions during the

Army Training and Evaluation Program (ARTEP) evaluation accurately

reflects the unit's capability to successfully perform these missions

in combat. It is also assumed that reliable measures of effectiveness v

can be construed for each ARTEP mission; however, it is recognized

that an adequate measure of effectiveness is not apparent from a

41 normal ARTEP evaluation. The ARTEP evaluation results in units beingL

judged satisfactory or unsatisfactory on selected missions. This

7
gross measurement of unit performance lacks the se'nsitivity necessary to adequately discriminate among the various types of individual

and collective training received by the unit. Accordingly, some

procedure must be established to convert the ARTEP ratings of satisfactory/unsatisfactory to a numerical measure of effectiveness.

Currently, a scale of zero to one is being utilized for the measure

of effectiveness.

A precise measure of relative unit performance is further complicated by the subjective nature of ARTEP evaluation. Even though

* many standards appear to facilitate an objective evaluation (time,

distance, numbers, etc.), most pass/fail decisions include considerations for visibility, climatic conditions, preceding unit activity,

tactical situation, etc., as well as the individual perspective of

the evaluation. Recognizing the inherent problems of obtaining

reliable unit effectiveness data, experience has proven that competent evaluators, properly briefed, will provide valid data and a

relatively consistent measure of unit effectiveness.

In addition to a precise measure of unit performance, there is

a requirement to determine the contribution of various TEC lessons

-to unit performance. Current TEC lessons are designed to primarily

teach individual tasks. This necessitates concentration on ARTEP

tasks and standards, rather than overall mission performance.

The difference in ARTEP design will require that an effectiveness

production function be constructed for each type of unit undergoing

evaluation. Two procedures for measuring unit performance are suggested. Each ARTEP task or standard may be equated to a numerical

effectiveness measure of zero to 10 points 

-a seven may be equal to

a satisfactory performance while a higher number indicates something

better than the performance standard (e.g. a five-minute standard

completed in four minutes) and a number below seven would allow for

some distinction among units judged unsatisfactory. The procedure

used in the development of hypothetical data to exercise this model * -

was to assign a numerical value to each standard or portion of a

standard and the unit performance (output) for each task was derived

from the number of satisfactory and unsatisfactory standards. The

vidual 

evaluation) 

inputs to 

personnel 

the 

and 

function 

the 

data 

collection 

(all 

are obtained 

personnel 

of detailed 

through 

assigned 

data 

the 

to 

collection 

which 

the unit 

reflects 

undergoing

of indiallF

individual and collective training activities of the unit.

IV 1W 'V 1W 'V V V W W W go
W1a , IO

COST OF' TRAINING

The second step of the methodology involves determining the

costs of the different types of training contributing to overall

unit effectiveness. It would be a simole task to determine the

annual tru.~ning budget for an Army Division; however, a squad's

share of t-it budget for a specific period of instruction is not

readily av ilable.

For L'~e development of this methodology, training was divided

into three categories: TEC, garrison, and field. Data forms were

developed to record all individual and collective training for an

eight week period prior to the ARTEP evaluation. The collected

data provides information concerning the time expenditure of all

personnel by rank, the type and amount of training each individual

has attended, the type of facility utilized, the type of vehicles

used and mileage for each, and the type and amount of all ammunitioni. With the above data, using current cost information, the

cost of each type of training was determined for use in the hypothetical data. This provides comparative costs for a given amount

of TEC, garrison, and field training.Z

No absolute training budget constraint figure was used for the

hypothetical units, but the budget constraint was treated as a

variable and was a consideration in all cost computations.

ECONOMIC THEORIES

Before commencing a technical development of the methodology,

it may be useful to briefly summarize the economic theories on

which it is based, which are the theory of consumer behavior and

the theory of economic welfare. The theory of consumer behavior

attempts to explain the behavior of rational individuals in the

context of a market. Individuals are faced with a large number of

available goods, each with fixed price, and have a fixed and

finite income with which to purchase some "market basket" of goods.

The individual is assumed to possess a set of preferences over all

V possible market baskets. These preferences satisfy four axioms:

greed, completeness, transitivity, and diminishing marginal rate

of substitution. These axioms are the basis for the utility

function, which is a convenient and analytically tractable representation of a consumer's preferences. The theory of economic

V)'Z

4 9
welfare starts from the preceding theory and asks: under what circumstances (particularly regarding price and income changes) is the

consumer made better or worse of f? An answer to this question is

formulated using the concepts of compensating and equivalent vari- S.

ations in income. At first blush, these economic theories seem to

bear little relation to the problem of evaluating military training

programs. Nonetheless, the areas are sufficiently formally parallel

that the economic theories can be successfully adapted to the military

context. In essence, the two different problem areas are found to

have similar mathematical structure, and the methodology to be described exploits that similarity.

ARMY TRAINING ADAPTATION

In the context of military training, we set up our model as

follows. An individual or unit (squad, platoon, company, battalion,

etc.) has a number of missions for which it is responsible. There

are a number of different types of training activities in which the

unit can engage, and each type of training activity makes some contribution to the ability of the unit to carry out each mission. To

be sure, certain types of training may enhance abilijty in one mission

area more than in another. The amount of each training activity

undertaken by the military unit is measurable in appropriate units

such as hours or trials. The overall ability and proficiency with

which the unit carries out its assigned missions is referred to as

the unit's effectiveness, and we assume a valid measure of group

effectiveness exists or can be constructed. The costs of carrying

out each type of training are also assumed to be known, and the unit's

training budget is assumed known and fixed (for the current training

* period).

Suppose a new training concept is developed and is considered

for introduction into this milieu. A procedure is needed to determine

whether, and to what extent, the new training concept is economically

viable. The basic approach is to determine whether the present value.

of the sum of the net values of the new training to each military

unit utilizing that training over the "lifetime" of the training

program exceeds the development cost of the training program. The

key to implementing the approach is the determination of the net

value of the new training to the unit using it. This can be accomplished using either one of two concepts from welfare economics.

10

VW AV W 0 W W V W M W
The first is the compensating budget variation. Here one attempts

to determine the maximum training budget reduction a unit can suffer

in the presence of the new training program which still permits the

previous level of effectiveness - the level reached with the initial

budget and no new training - to be achieved. The second concept is

the equivalent budget variation. Here one attempts to determine the

minimum training budget increase which a unit must receive to achieve%

the same increase in effectiveness without the new training program

as can be achieved with the initial budget and with the new training

program. Under certain conditions, the compensating budget variation

and the equivalent budget variation are numerically equivalent. More

generally, however, they have different values)-1 Which of the two

concepts provides the more appropriate measure of the value of the

new training program depends on the objective of the organization.

If the objective is to minimize the cost of achieving the current

level of effectiveness, the compensating budget variation is more

appropriate. If the objective is to increase effectiveness while

maintaining the initial budget level, the equivalent budget variation is more appropriate. To the extent that the organization's

objective is uncertain, or to the extent it combines both mentioned

objectives, a weighted average of the two variations can be used.

The numerical values of both the compensating and equivalent

budget variations can be derived from the effectiveness production

function and the unit's training budget constraint. 'The effectiveness production function is the relationship between the inputs and

output of the training process. The inputs are the various types

and amounts of training the unit receives, along with the relevant

attributes of the unit itself (such as intelligence and prior training) and the output is the assumed objective and measurable characteristic effectiveness. The training budget constraint is the relationship among the dollar value of the training budget, the unit

costs of the various types of available training, and the quantities

of each type of training which can be afforded.

The training budget constraint is a linear function and is conceptually straightforward. The effectiveness production function,

on the other hand, is (in general) non-linear and is rich in possible

specifications. In considering the impact on the production function '

of the introduction of a new training program, there appear to be four

distinctly different ways in which a new program can affect effectiveness. First, the new program cart influence effectiveness in the same

1 1The reason for this asymmetry, which was first pointed out by

A. Henderson ("Consumers Surplus and the Compensating Variation,"

Review of Economic Studies, 1941, is somewhat technical and unimportant for present purposes.

qP Ar I W M W 11 W W W 0 

W~' 'U 'U 'U U U U V U U V0U
way that every other program does, i.e., it is simply another among

many training programs. Second, the new program can enhance or

augment the effect of one or several of other training programs,

i.e., it can make it seem like more of some other training is being

4 received than is actually the case. Third, the new program can

4 uniformly augment all the other training, i.e., it can raise the

entire effectiveness production function. Finally, the new training program can increase the substitutability of one type of training for another, permitting some of a less costly type of training

to be substituted for some of a more costly type of training while

maintaining the same level of effectiveness.

The effectiveness production function must be empirically determined. Multiple regression analysis is the logical starting point

for this estimation process. The observations (data points) may be

cross-sectional or longitudinal, or perhaps a pooled sample. The

N" regressand is the effectiveness measure achieved by the unit. The

regressors would be the amounts of each type of training, as well as

such standardizing variables as intelligence and prior training,

relevant to each effectiveness observation.

The training budget constraint, unlike the effectiveness production function, need not be empirically estimated, but can be

constructed using available data.

12

~a~ MW W ' WW W W W A
DEVELOPMENT OF THE VALUATION PIETHODOLOGY

Assumptions. At this point we begin a somewhat rigorous development of the Valuation Methodology, which is the third of the four

steps comprising this CTEA Methodology. We start with the following

assumptions.

0 Military training is carried out as a rational resource allocation process. The trainer behaves

as if he knows the effectiveness production function12 (the relation between inputs and outputs of .-

i the training process), he knows the costs 13 of the

various types of training, he knows his training

budget, and he attempts to maximize the effectiveness

he can achieve in his charges subject to his cost and

" budget constraints.

0 There exists a measure of unit effectiveness.

J 5-.

12 The trainer may have an implicit knowledge of the effectiveness

function. That is, while he may not be able to state the mathematical form of the function, he behaves - in his resource allocation

choices - as though he knows it. This is similar to the pocket

*4 billiards expert who may have no explicit knowledge of the principles

of mechanics, but surely behaves as though he does. Step I of the

,- CTEA Methodology involves the explicit estimation of the effectiveness production function. -N

13 As in the previous footnote, it is sufficient to assume the trainer

has an implicit, or behavioral, knowledge of these costs. Step 2

of the CTEA Methodology involves the explicit determination of the

unit costs of each type of training.

13

* rl7 %s % .. '. .

015%

'g WP 'P g' W ' • • W W .
0 The effectiveness production function is a known, continuous, differentiable, strictly quasi-concave function

with domain and range the nonnegative real numbers.

0 The unit costs of each type of training are known, as

is the total amount of the training budget.

0 There are I (greater than one) different types of training available, not including the new training which is

the subject of the evaluation.

Of these Five assumptions, only the third demands elucidation at this

point, the others being sufficiently self-explanatory.

We begin with a definition. The effectiveness production function

(EPF) is the relation between any given amounts of any types of available training and the maximum level of effectiveness which that training can achieve, given the levels of experience, intelligence, etc. of

the units being trained. An example of an EPF might be E = TIL T2 8 A

where E = effectiveness, T, and T2 are numerical measures of the amounts

of two types of training received, A is a combined measure of experience

and intelligence, and a, a, and 8, are positive parameters.

Assumption 3 states the EPF is known. In order to carry out the

*[

Valuation Methodology, this must be the case. Determining the EPF is .

Step 1 in the CTEA Methodology. 4

14_

4w 1P 1w 'U W or
The EPF is assumed to be continuous. This simply means there

is an effectiveness level associated with all values of the training

variables within the relevant range. Differentiable means the function has derivatives everywhere, there are no kinks or corners in the

function. That the EPF is strictly quasi-concave means its contours -.7

(iso-effectiveness surfaces) are strictly convex to the origin. A

nonnegative range and domain of the EPF simply means the values taken

by inputs and the output of the function cannot be negative.

The following figures give three different perspectives on the

EPF. In general, one can expect an EPF to have a form like that

suggested by the figures, although deviations cannot be ruled out

a priori. The reader might bear in mind that there are an infinite

number of mathematical specifications of the EPF consistent with the

* displayed form, so these figures do not obviate the need for Step 1

of the CTEA Methodology. Figure 1 displays a cross-sectional view of

the EPF. Holding all other variables constant, it shows the relation

between one type of training and unit effectiveness. Three points are

noteworthy. First, zero amount of training does not imply zero effectiveness. This is because the other factors affecting effectiveness, '.

% held constant for the purpose of this Figure, contribute to the unit's

effectiveness. Even if none of this particular type of training is

received, the unit can exhibit a positive level of effectiveness, such

as A. Second, the curve in the figure approaches the effectiveness

value B asymptotically. This means there is a maximum level of effec-

.5V N,
,,."p

Ef fect iveness

B

jA

A"

16,

%,'.%

5'.

.... Fiuei rs -etoa iwofteEfcieeS rdc in..,

.' Funtion Effctienes Verus Tainig.

-Wl *5,"

- Q" ' qP qg' 1 q' q • P qP "
tiveness which can be produced from any amount of this type of

training, given the fixed levels of the other factors. This assumption is in good accord with both the economic theory of production

functions and psychological theories of training. Third, the curve

in Figure 1 is non-decreasing as the level of training increases.

In other words, more training never causes a decrease in effective-

* ness.

Figure 2 shows an alternate cross-sectional view of the EPF.

Holding all factors constant except for two types of training, the

Figure shows all combinations of the two types of training, other

factors constant, which can lead to a given level of effectiveness.

The two curves shown in the figure correspond to two different levels

of effectiveness, the greater effectiveness being associated with the

more north-easterly curve. These curves can be referred to as isoeffectiveness contours, and there are four noteworthy points about

these contours. First, they slope downward from left to right. This

indicates that one type of training can be substituted for another to

maintain a given level of effectiveness. Second, the contours are

V. -'%

non-intersecting. If intersections occurred, this would imply that

both contours were associated with the same effectiveness level, which

would be a contradiction. Third, the contours are convex to the origin,

i.e., their slopes increase to the right. This means that as one type

of training is substituted for another, it takes more and more of the

increasing type of training to compensate for unit decreases in the

17

W W1 %..:~:. ~ '

* . . - ...
Training T 2

N'... ,.,

4"4

-a'.', , 

H

Training T1

Figure 2. Cross-sectional View of the Effectiveness Production I

Function: Training Versus Training.

18

*IV 'U 0 U U U U 74N 

~~*~ L - I I~

*".. ,:4

I- *.:.'; '-' * -..-..... ''''''...''' '' ""-," - " :b - .','--- -".'.::
other type training if the initial level of effectiveness is to j

be maintained. The fourth point is that the iso-effectiveness

contours may or may not intersect the axes. Typically one might

expect the contours associated with the lower effectiveness levels

to intersect the axes while the higher contours may not. This

means that for lower effectiveness levels (say E) some types of

training may be dispensed with entirely (other types substituting)

and that level of effectiveness maintained. For high levels of

effectiveness (say E), at least some amount of certain types of

training may be indispensable if that effectiveness level is to be

maintained.

Figure 3 is a three dimensional view of the EPF. It combines

the views of the previous two figures. Holding all variables constant except two types of training and effectiveness, the EPF can W6,

be represented as surface ADE. A is greater than 0 here as in Figure

1. As T1 increases with T 2 = 0, the value of effectiveness asymptotically approaches B. Likewise, as T 2 increases with T1 = 0, the value .,

of effectiveness asymptotically approaches C. If effectiveness is

fixed at level E, locus GF is the iso-effectiveness contour representing all combinations of Ti and T 2 yielding E. JH is the projection of

GF onto the plane of Tl and T2 represented in Figure 2.

19

% Wd %.,r.

L 4 

N 

wwwww -~~~ ~ %C %%a~*.......

'a V .- -
Effect iveness

E

C...

Trainingg T

AA

Coil

Figure 3. Three Dimensional View of the Effectiveness Production

Functio..

20

W lp

~~~,~ U U U U ~ U% 

% N -~ * ;~k % ,
The Model. We are now prepared to construct the economic model

of training resource allocation, which is the basis for the CTEA

Methodology. We employ the following notation.

E a unit's level of overall effectiveness

T i the quantity of training of type i received by

the unit, i = I ... I

T* the quantity of the new training program received by the unit

C i the per unit cost of T i

c, the per unit cost of T*

B the dollar amount of the unit's training budget -F

A all other factors affecting the unit's mission

proficiency

C(T*) the compensating budget variation associated with

E(T*) the equivalent budget variation associated with T*

V(T*) the net value of T* to the unit.

In what follows we employ an implicit form of the EPF, in order to make

our development of the CTEA methodology as general as possible. Of

course, in an actual application, a specific EPF would be employed.

Denote the EPF as

E = F ( Tl, T2 , ... TI, A ). (i)

Remember that E=F(") incorporates all the assumptions about the EPF

21

V P 6a 4 S P W U 0 S A * SV 0 

% %4,4
mentioned previously. The trainer's objective is to maximize E, but

he is constrained by limited resources. Specifically, his choices

over the T1, T2, ... , Ti (his control variables) must obey (assuming

he spends his entire training budget, B)

B =C l T + c2 T2 + ... + cI T, (2)

and

T i 0, Vi (3)

Equation (2) states the trainer can spend only as much budget as he

has, and (3) is merely a technical statement that the amount of training of any type cannot be a negative number. Equations (1), (2), and :.

(3) constitute a nonlinear programming problem. Since this is the

resource allocation problem providing the foundation for our CTEA 1

Methodology, some investigation into its characteristics is useful.

Specifically, let us characterize the solution to the problem and then

investigate the effects of changes in parameter values (B, Cl, ...) on

the solution.

The method of Lagrange Multipliers1 is a useful solution technique

for maximizing (1) subject to (2) and (3). Form the function

L = F (TI, T 2 , ... , T ) + X (B-clT I- c 2T 2- ... - cyTI ).

First order conditions for a maximum are

.4 5

1 See almost any text on advanced calculus.

'. %22

IF w IR 'w "0 w w w w w w V IA w

% %
31Lf T2 3 F/3T2 Ac2 0

=LDI W~ - Xc 1 0

=L3 B - c T1 - C2 T2 ... -cTI

-'.4

The first I conditions in (4) can be reduced to the I-1 conditions

3T.%

1 - ci for i=lItoI1-1(5

3F c

3T1 I

For the two variable case, the right side of (5), cl/c2 , is Simply

the negative of the slope of (2) in TlT 2 space. The meaning of

the left side of (5) can be seen as follows. For the two variable

case the total differential of (1) is

dE = dT + - dT.

T1 1 aT 2 2

Along an iso-effectiveness contour dE = 0, so that

dT 9F T1 22

Therefore, the left side of (5) is the negative of the slope of the

optimal iso-effectiveness contour. It follows that (5) implies that

the optimal values of T1 and T2 are such that at that point the slope

of the iso-effectiveness contour equals the slope of the budget constraint. Moreover, according to the 1+1st condition in (4), the 9

23

14 4 I

% N N

w 'wWA -~~~~~ a.fQ~.-.~4;4. ~~ ' Yoe.. J.j PP*~,
optimal T, and T2 must be on the budget constraint. Figure 4

summarizes these conclusions by graphically illustrating the

solution to the two variable problems. The solution occurs where

the highest iso-effectiveness contour has a point in common with

the budget line. Tangency of the two lines occurs at that point.

Note that the rationale for the assumption of strict quasiconcavity of the EPF is now evident. Strictness insures that the

iso-effectiveness contour has no linear segments. If it exhibited

linearity, the optimum solution either could be indeterminate (if

the iso-effectiveness line and the budget line happened to have the

same slope) or could occur at a corner, such as at (TI, T2 ) =

(B/cl, 0) or (0, B/c 2 ). The corner solution is the most likely N?

event, yet is not plausible in most cases. Thus, the assumption

of strict quasi-concavity of the EPF helps rule out the implausible

event, and helps the model conform (presumably) more closely to

reality.

As a check on the reasonableness of the model, and as a way of

investigating its characteristics, we can determine the model's implications for training resource allocation when (a) the per unit

cost of one type of training changes; or (b) the training budget

00. . changes. Both these situations can be investigated with the aid of

diagrams akin to Figure 4. Suppose we begin with the solution es24

%" %,

U~ ~~ ~~~ U U U U U U U -v U % U i

9, * - . J*.)* ~ 9

, ' ~ ~
B

Iso-effectiveness contour

_.slope - 1c 2

Budget '."

Constraint b Optimal point: slopes equal and

constraint satisfied

T *F

I--slope = F

0 TB T1

r -,..,

Figure 4. Illustration of Solution to the Training Resource

Allocation Problem (T*T 2 ) is the optial solution.

25

% 

%'

U I l l+l , iI ' .I + I +k+ + 

.+i -. i +-,i +- IiI +m.4"m Im+-, i+ U L QI."m'Ii Im- ' l ' l m _mmII"I+

, -I • Iii( . + t - . U. . ii • $ U S " U +-"- U- + U U + Um +. *' .Ui I ,+ Ul+ . +
IA

tablished in Figure 4 and then assume an increase to cl(the per unit

cost of TI) occurs, but B and c2 remain constant. An increase in cI

means that B/cI decreases, which means that the budget constraint

rotates (about B/c2 ) inward. Figure 5 illustrates this change in the

budget constraint, and the tangency of a lower iso-effectiveness contour to the new constraint line. Figure 6 isolates the effect of the

change in the per unit cost of T, on the amount of T, which is optimally 15

employed. In almost all cases , the optimal level of TI declines with

increase in cI . In this case, the optimal level of T, goes from T, to

T1 . Many points like (TI, cl) can be generated by arbitrarily changing

cl in Figure 5, and tracing the effect in Figure 6. The locus formed

by all such points (in Figure 6) would be know-n as the training demand

curve for Ti . D in Figure 6 is such a curve, here drawn from only

two points. The downward slope at D reflects the law of demand, which

states that there is an inverse relation between the unit price of a

good and the quantity which will be demanded. The law of demand is a

fundamental concept in microeconomic theory.

Now, beginning again from Figure 4, consider the effect of an increase in the training budget on T1 (the effect on T2 could just as

*° easily be considered). An increase in B shifts the budget constraint

outward, but parallel to the initial constraint. The effect of the ,

shift on the optimal level of T, cannot be predicted without detailed

information on the shape of the iso-effectiveness contours. The optimal

.J

' That is for almost all sets of convex iso-effectiveness contours.

boyd 26

V. " V or1 - 2". -"' ='= ma t~ " ;I i ak~ 'tl . k'J, ' '- ,q* n'. "':% ' ." , .""" % , .'%-
T 2I"

Figure 5. A Shift in the Per Unit

Cost of One Type of

Training.

Per Unit

Cost T2

I I, L-' L

SB/c, B/c TI

1. 1 '

Per Unit

Cost T I c Figure 6. The Training Demand Curve.

c

D

'. I..

r N

-. 2 7 , -.
16 ~t"

level of TI could increase, decrease or remain unchanged as B

increases. Figures 7, 8, and 9 illustrate these possibilities.

In each figure B represents the initial budget line, and B' the

new budget line. TI* represents the initial optimal value of

TI, and T, the new optimal value; I* and I are the initial and

final iso-effectiveness contours.

The characteristics of the model revealed in equations (4) and

(5), and in Figures 4, 5, and 6 parallel the characteristics of the %

standard microeconomic theory of the consumer. The latter having

withstood a good deal of initial investigation, it is not imprudent

to accept the reasonableness of the former, and use it as the basis .7

of the valuation methodology.

' " 16 A decrease could occur in one type of training if that training *

i:-'•" were inexpensive but not very effective. An increase in the.-,

r ] training budget could cause the trainer to substitute more costly,,

' . '. but more effective, training for that initial training. Thus, one '-

--- might observe a decrease in a certain type of training associated ,

Swith an increase in anothier type of training",

28'f, ,

.4.

% % 'ft'ft

%f ft%.
Figure 7. An increase in Budget

T 2 causes an Increase in T 1 "r .

T1.

Figure 8. An.

causes a Decrease in T"

I B

T1

T2

2 Figure 9. An Increase in Budget

causes no change in

T.

B

T TT

29 %

IFir 9. A I in B-dget

cassnhnei

% 1
The Valuation Methodology. Having, presented our model of training resource allocation, we are now prepared to develop the training

valuation methodology. The methodology exploits the economic aspects

of the resource allocation model to infer the economic value of a ~

specific type of training. %

Suppose our interest in a new type of training program. T*, whose

per unit cost is c*. At issue is whether, and to what extent, the

benefits derived from T* exceed the costs. Or in other words, is the

net value of the T* training program positive, negative, or zero? In

yet other words, are we economically better or worse off by having the

T* training program?

A-. These questions can be answered by calculating the compensating

budget variation and/or the equivalent budget variation associated

with T*. These concepts were developed in the field of welfare economics to measure the impact on consumers of price changes. Our use of

the concepts here extends their applicability beyond the measurement

of price effects to the measurement of the value of training programs.

The compensating budget variation may be defined as the reduction

in the initial training budget which will maintain the initial level

of effectiveness when the new type of training is introduced where *

the initial effectiveness level was achieved with the initial budget

level but without the new training program.

30

~~~9 It. I.* .. ~ .
This concept of compensating budget variation can be clarified

by resorting to the symbolic notation introducel above. Let B denote

the initial training budget level, and I the initial number of training programs. The trainer's resource allocation problem is, using-

(1), (2), and (3) introduced above. ___

MAXIMIZE E = F (TV T ..., TI, A) (1)

given that B = cI T1 + c 2 T 2 + ... + c I T1 (2)

Let us denote the solution to this problem as (T 1 , T 2 , ... , TI), which

results in an effectiveness level E. ON

...P. Now suppose a new type of training, T,, is introduced, with unit

cost c,. The trainer is free to use as much or as little of T, as he

chooses (subject to c, and B). How much of the budget can now be saved

by incorporating T, into the training schedule? We answer this by find- 9.

ing the minimum budget which will permit E to be achieved. >7'"

Anlytically, the problem is

MINIMIZE B c T + c T + c+c T T(4)

1 1 2 2 I *

given that E =F (Ti, T TI, T*, A) (5)

T T, 0, Vi (6)

Denote the solution to this problem as (Ti'p T 2 ... T', T*), and the

corresponding minimized budget value as B. Then the savings achieved

by the use of T* in the achievement of E level of E are B - B. The

quantity is known as the compensating budget variation associated with

T*, which we denote as C(T*). That is

C (T,)j B - . (7)

J[.- W ] -PJ qJ -. -s ' 1 lJ .# " #" ,PIP " , +" " " P ? ", " ,, 1

"W W W " ' 1", - -%% 1.W N,,.+, " % ' ,,' +,,~~.0 " " "P . ' /'" , +..,." -" # . ' " " 7 ,411" 
We note that C (T,) is necessarily non-negative. For the maximum

value of B in (4) is B achieved by setting T, = 0. If T, is brought

into the solution at a positive level, it cannot be a detriment to

the value of B, otherwise it simply would remain at 0. In other

"a words, if T. is not useful in cutting costs and maintaining effectiveness, it will not be used. It can help, but it can't do any

17 harm. Therefore, if the maximum value of B is B, the minimum

value of C (T,) is 0.

The compensating budget variation is an appropriate measure of

the value of a training program when the goal of that new training S.

is to save money, that is, when the new training is neither expected

nor desired to result in an overall increase in effectiveness. When

an implicit or explicit goal of the new training is to enhance effectiveness, not simply to cut costs, the compensating budget variation

alone is not an appropriate measure of training value. The appropriate

measure in this latter case must include the equivalent budget variation,

which measures the value of the increased effectiveness attributable to e'-

the new training program as the minimum incremental cost of achieving

the higher level of effectiveness without the new training program.

. Once again, resort to symbolic notation may clarify the issue. If

T, is introduced into the training milieu, and the initial budget - -

17 At this point the development cost of T has yet to he considered .

32 ,'.

%' .

N).

w.~~~~~7 ~ ~ ~ U 7-- 
level B prevails, the resource allocation problem is

MXIMIZE E = F (TI, 2' " TI' T,, A) (8)

given that B=c T + c 2 T + + c T + C T (9)-.

b >

T±, T* - 0, Vji (10)

• .The reader will note that problem (8), (9), (10) is different from

problems (1), (2), (3) and (4), (5), (6). Denote the solution to

(8), (9), (10) as T , T, .... 2 TV T*) with resultant effectiveness

E. Then solve

MINIMIZE B = c, T1 + c 2 T2 + c, T, (11)

given that E=F (TI, T 25 ... , TI, A) (12) t

Ti 0 Vi (13)

Denote the solution to this problem as TI, T2 , ... , T, and the

resulting minimized budget value as B. Then, the equivalent budget

variation is

3".- E (T*) = B - B (14)

Note that E (T*) must be non-negative. A comparison of (1), (2), and

(3) with (8), (9), and (10) reveals the only difference is that the

latter problem allows the possibility of using T,. The extreme worst

case would be T, = 0, which would mean that E would be equal to E. In

that case, the solution to (11), (12), (13) would be B = B, and E (T,)

would be 0. If the extreme worst case does not prevail, i.e. T,> 0,

then E (T,) >0.

There is no necessary relation between the magnittidos C(T and :.

E (T,), that is, C (T,) > E (T,). Hov,'e;er, if V (T,) is defined as

the best estimate of the value of T, we should have

33 %'

~~~ 'U w IV 3 w W w w w w wW

%, % %

• . o o • ', "
1F7forV (T,) =X-C (T,) E (T,) c T* (15)wee 

. ~for 0 < A i

That is, V (T,) should be chosen so it falls between C (T,) and E

(T*), inclusively. X is a weight denoting the relative importance of

the cost saving objective in the introduction of T*. (1 - X) is the

weight associated with the effectiveness enhancement objective of T*.

To allay the reader's concern that the introduction of A necessarily

causes V (T*) to span so broad a range as to mitigate its usefulness,

we note here that for a broad and useful class of effectiveness proi '" duction functions, C (T*) = E (T*). Thus, V (T*) can be independent 4

4 of A. For other types of EPF's, C (T) and E (T*) can be reasonably

close.

#" 34

."S.

%S~
.. 4-

Three Geometric Expositions of the Valuation Methodology. An

appreciation of the methodology developed in the previous section

can be gained by considering several geometric interpretations. Each

geometric interpretation makes slightly different assumptions in ab-

* stracting the essence of the model to achieve a two dimensional

* interpretation. In the first interpretation we assume there are two

types of training (TI and T2 ) in addition to the new training T*. We

also assume that while T1 and T 2 have positive unit costs, cI and c2 ,

the unit cost of T,, c,, is zero. In Figure 10, the line designated

L is the initial budget constraint. That is, L is the line determined

by the equation B = c 1 T1 + c2 T 2 . The curved lines are the isoeffectiveness contours of the production function. The initial optimal

solution is TI, T2 yielding E = E, when T, = 0. When T, becomes available a new optimization results in solution values TI, T 2 . yieldiug E

at point D. E can be achieved with T, > 0 at point K with a budget of

B. E can be achieved without T, (T, = 0) only by expanding the budget

to B, causing a solution at M. Then we can determine C (T,) = B - B

and E (T*) B - B.

Alternatively, we can assume there is only one other type of training beside T*, and that T* has unit cost to the group of c,. Figure 11 N.

represents this abstraction. Initially, with budget B and only training

type T 1 available, the group receives T B/c I units of TI, resulting

N in an effectiveness level of E. When T, becomes available, the new

V V V ' 'W W W W W W W W W Ow W

--,,e,,
PTT

2

B

c 2i

B

c

2

I,. B2

I E T,= 0

T2 tD

I

K

E/T*>0

T T 1 B B

4 

,

1 c, 1

Figure 10. First Geometric Interpretation of Methodology.

36

w.~~~~~ ' ww w *w V V

ItT

e S. , --
Tk

B

C

B

ci

B% B

C*C

Fiue1 . Sc n emt i ntrrtto fM t oooy

37J

M w wv w w*

~%

%I

% J A %

I% - L% 
optimum training package is denoted by A, resulting in a higher

effectiveness level of E. E can be achieved with T- with a budget

of only B, as indicated by point D. And E can be achieved without

T, only with the greater budget of B, as indicated along the vertical axis. Once again, C(T ) = B - B, and E(T,) = B - B.

Finally, we can avail ourselves of what might be referred to II as the Effectiveness Cost Function (ECF), a relationship between the

budget expenditure and the level of effectiveness, showing the greatest level of effectiveness which can be achieved with any given

budget level. In Figure 12, the lower curve is the ECF when T, is

-"p.4' unavailable, and the upper curve is the ECF with T,. In this representation, we can assume an arbitrary number of training types besides . $

T,, as well as an arbitrary unit cost of T,. Initially the budget

level is B, T, is unavailable, and the resulting effectiveness level

is E. When T* is introduced, the same budget, B, can now achieve E.

The initial effectiveness level, E, can now be achieved with the lower

budget, B. And, in the absence of T*, E could be achieved only by expanding the budget to B. Again, C(T*) = B- and E(T) = -B.

:0 :

W'.. be

38

1P w *P W W w W w W wW to 
ECFj T,>0

EF IT.=O

" .

E

3

B B

Figure 12: Third Geometric Interpretation of Methodology.

39

w w @ w w IW w 

Iw 41
ALGEBRAIC EXAMIPLE OF THE VALUATION METHODOLOGY

The purpose here is to illustrate the calculation of the compensating budget variation and the equivalent budget variation for a

given effectiveness production function and a given budget constraint.

For the given effectiveness production function consider there

are two types of conventional training - garrison training and field

training - and TEC training, the new training, the value of which is

to be determined. Consider, also, a single variable A represents the ',

effectiveness of a unit based on "all other factors" before the unit '-

commences conventional training and TEC training, and another variable

E represents the effectiveness of a unit based on its ARTEP or other

evaluation after the unit personnel have received conventional and

TEC training. Consider, further, that for a number of like units,

say k, over the course of their training periods, data are collected

from which to determine for each unit

0 the number of hours expended on

each of the three types of training

0 a numerical value (range 0 - 1) for A,

and a numerical value (range 0 - i)

for E.

.6

I .

40

~ W 1W 'W Mw . w w w W dw
t . I-~ V

Finally, consider that several • candidate effectiveness production 'V.

functions are used in regression or other analyses of the k data

points, and that the best "fit" EPF is

E = 1 + [A- - m (1 - e G [- m ( e -tT fF] (16)

where G, T, and F are variables representing hours of garrison, TEC,

and field training; g, t, and f are parameters representing learning

rates associated with the respective types of training; m is the

maximum level of effectiveness a unit might attain from either all

garrison or all TEC training; and A and E are as explained above.

For the given budget constraint consider first that the hourly

cost for each type of training exists so that the expression

B = YG + T T + K + 0 F (17)

represents the cost of various mixes of the three types of training

where y, T, and are the hourly costs of garrison, TEC, and field

training, and K is a fixed cost associated with any amount of TEC

training (provided that TEC is used). Consider second that B denotes

a fixed budget available for training so that

S= yG +TT + K + F (18) V

is the budget constraint - i.e., only those mixes of C, T, and F whose

total cost does not exceed B are permissible.

Now, there are four steps to carry oit. In turn they determine

V

41

PEW,"
0 the maximuIm value of E attainable

with B and with no T. This value

is denoted E.

0 the maximum value of E attainable

with B and with T. This value is

denoted E.

0 the minimum budget with which E

can be achieved with T. This

budget amount is denoted B.

0 the minimum budget with which E

can be achieved with no T. This

budget amount is denoted B.

Note that this first is necessary to perform the third, and the second -_,.

is necessary to perform the fourth. When B and B are determined, C(T) -

and E(T), the compensating budget variation and the equivalent budget

variation, can be calculated, since B is known a priori. Specifically,

C(T) = B - and E(T) = B - B.

The first step is accomplished by solving the problem

MAX E = 1 + [ - -m (1-e [m (e e

given B = yG + TT + K + F

T= 0 (19)

G, F->0.

42

.- 'W 10 'w -0 W Iw W W W w W W
w.' -j W

"2" . , i

The relevant Langrangian expression is .

L = 1+ [A - j [i- m (1 - e-gG)] [e - fF] + ?. [B - G- F] (20)

First order conditions are .

A G e -f F] [e-g 1 m - Xy =0 (21)

S--- -I [- m (1 - e-gG)] [efF] [f] - = 0 (22)

3L i

B - yG - F = 0 (23)

Solving (21), (22), and (23) as a set of simultaneous equations yields

G ln m (qg -Yf)l g- and (24) Lyf (1 -m)l

B Y - in Fm - Yf) g (25)

,Lyf (r) -

Expressions (24) and (25) constitute the solution to (19), and substituting them into the effectiveness production function, (16), yields

Yf -fB

- [+g(l-m)- m(c-g-Yf) -1 1-g E = 1 + LA - 1]Lg-f L'fl-m) [e (26)

Remember that the regression or other analysis used to select (16)

provided numerical estimates for the parameters A, g, t, f, and m;

also remember that B, y, T, 0, and K are known.

S..

Going on to the second step, the problem to solve is

-tT

MAX E = 1 + A- 1] [1- m (1 e-g)] [ m (1 e-tT)] [e-f

given B = yG + TT + K + F (27)

G, T, F ? 0.

43

W Ww w ' W w w w w w

% % -

U" % % -4
Proceeding similarly as in the first step, the solution to (27) is

Fm,~y) 1

G = In ( 1-9) ay 

(28) ..

[m(Ot-Tf) 1 t ]

T = in [Tf (t-) I

B-K Fm(¢g-Yf) g -in Ltf(l-m)_ F = --In fn "m(Pt-.

.

which upon substitution into (16) yields

'yf *Yf B- _ _(1 m)1 m - _ g t(1-m)] [ (# t[f) - K

= 1E + [A - 1] - f Lyf( z -m) L6t - f Tf(l-m) L ] (29)

Step three involves the problem:

MIN B = yG + TT + K + OF )lie F00) "'

given E = 1- - - m (i - e- m (1 e - tT

C, T, F ->0.

Here the relevant Lagrangian expression is

."-

L yC +TT +K + OG +Xi~ A-m (1 -g,,] (31)

- m - e- tT L' F]]

Here, as in the first and second steps, taking the partial derivatives

of L with respect to G, T, F, and X and setting the partials equal to

zero gives the first order conditions

3L Y + [A- i] - m (I- e-tT)] [e - f F] [tme - t ] = 0

(32) %

- + X[A 1lL Ir e ) I m (I- e LtT) [0 e40

__ - - [A 1] [1 - m( - e-gC) m (I - tT e fF ] 0•

/44 I,,,..w

*1 .1,
Solvn- 32)as a set of Sir.L1~tafleous equations yields

'V 7

G =in yf (1-M) J

1 (33) L m(-Tf) t

.-Th~~F T= f(1-mj

-- in f n (t-f)

[7f 1-MI) In [Tf(i-m) 

and substituting (33) into (17) gives

B = B + K + In Tt (M -l in [ -f 1 (34)

As the last step, the fourth requires solving the problem

MIN B LyG + TT + K + F

given 1i+ [A 1-] L m (I e [ m (1 -tT efF

T= 0

G, F 0.

The solution is_

T:= in F 4-i-m g

Lyf(1-rn) J T(6

B-K M(g-Yf) g - M(t f)] F=- -In -f-l-- ) I In L -) t - sYf (-M) (f g)vesL (1- 1TrMY

and therefore

-F(t-Tf - F(t-Tf 17 B =B -+ K + in L in Lf-m)l f(l) . (37)

N Now, using (34) the compensating budget variation, C(T) B, is

F(tivn +m(It-ef)l- f r K; (38)

and Using (37) the equivalent budet variation, E(T) B B is

i nTf f t - K. (39)

B - + B n-- n L

45

W V OW W-74

.R.P~ .1 % d' r " Ln rt - -- -a _ iN - J - K;38
Two points relative to (38) and (39) are noteworthy:

C(T) and E(T) are determined

entirely by the known (estimated)

parameters f, t, m, K, T, and '-

-." o C(T) and E(T), in this example, .*

are equal. This is not always %-:

necessarily so.

46 _, ...-

.5".--,,d

,5.. , "4" ," ," ," , , " ' .. ., '',, " , '.
1.. -~~~~~" - -- ----

PRESENT VALUE ANALYSES

An important part, of a TEC CTEA is the aggregation of TEC costs

and benefits over time, step four of the methodology. That is, an

intertemporal aggregation methodolog: must be a component of a TEC -

CTEA methodology. Intertemporal aggregation permits a series of

.

annual costs and benefits to be distilled to the single number which

is the sought measure of the value of TEC. While there exist a large

number of aggregation schemes, most are variants of the following

four well recognized approaches.

o Payback Period. The investment is

judged worthwhile if it generates

enough cost savings to pay back the

initial cost of the investment within

some given ma::imum period of time,

e.g., five years.

o Benefit Cost Ratio. The investment

is deemed worthwhile if the ratio of

the present value of benefits to the

present value of costs equals or exceeds unity. r".r

0 Internal. Rate of Return. The investment is accepted as worthwhile if its

rate of return eqals or exceeds some ,% ,. ,

!72 predetermiLned hlrdle rate.

47

M W I* W W 0 W 0 W 0 W W 0 IV V --m

* . ... . . . , -

. .- .

}: y , .y N .-. . % , . . , - , - v ,, T. ,, : . - . - . . ... . . , . ° . . . . . , , - , . . . . . . . . . . .-

L - 1 -Z- ",6 1. , ., ' " . - . . -. - . J . . . - . . , . . -. . . . ' . . . . . .. , . --- o . , .,.,,.,.,i- . . . ' . ' . . . . , - '.- . . - , - .- . .
0 Net Present Value. The investre"t

is determined to be worthwhile if

the algebraic sun of the discounted

costs and benefits exceeds zero.

Table 1 presents a more formal comparison of these approaches.

Interestingly, while each of these measures of investment worth

performs the requisite intertemporal aggregation, each results in

a single number of different dimensions, and a different accept/

reject criterion. Today, Net Present Value is generally regarded

as the best approach to intertemporal aggregation. This is because

the Payback Period approach ignores all costs and benefits beyond P

the period required to recover initial costs, and both the Benefit

Cost Ratio and the Internal Rate of Return ignore the total size of

the net benefits in favor of rate measures, i.e., dollars of bene- -'

fits per dollar cost or dollar per year of benefits per dollars per

year of costs. " ."

Our goal in a CTEA is the development of methodology to evaluate

specific training concepts or programs. The central element in the

methodology is the determination of the value of the training program

to a unit. This determination is accomplished using the concepts of

compensating and/or equivalent budget variations. Of course, the

methodology must consider all the ele.ents which affect the "bottom

line" value of the entire program, and tie all these considerations

48

4p'w M- a .

~. A-A ~ .~ ~ A .. ~* ~ t.A % ~'.*.~~ A4~J~ '. ~A. A . h - - - - - -
C)d

.r-C) 4

4.4

03

'-H

4-1 C)- u 4

4.4 CC)

-4 A-I

Al AlA> P-0.

$4 -)

In

C) C)

C t444.

0 :jw >4

M - 0-: 4.4. 4' Cd41 W 4

4-4 (

C) z

E- 41C

-'~4 r-i )

C)4 41 1 .~. .01

C E-.- U ~1

0l 0 + + s-1

C)4 Al--444-

C0 Al 41U 4- rn c. .

4-. U -- 4 C.41 0 0~44

8) -4 C) 440 f 4 .4 ~~).-4 8C-0- 0JJI .

C11 0 4- CJ

4-1 .ci - U1 "i 0

-H :I > r- En

E-4 ;>- 5-1 

cc cc ,-4 V-I 4 4 r-4 Q)

4-) 4.4

4J 4..,4

50., ~L 4. J $.4C

do0. r-~ A4- 4-J 4.l4

cl cC3 ici c k0 )r & a4c 4--I 

49

*~ %c
together to arrive at an ultimate dollar valuation of the program. ,.

The relevant elements include

D the program development costs in year t

t

M the program maintenance costs in year t

t

V(T ) the value of the program to unit 1 in year t

H the time horizon for the program, i.e., t = 0,

1, 2, ... , H

d the discount rate

L the number of groups using the program, i.e.,

1=1..., L

NPV(T,) net present value of the training program

The value of the program can be determined using an extension of

the Net Present Value expression in Table 1. The Net Present

VAlue of the training program is computed as

L1

V (T ) = H 

E Dt + Mt + Z

t=O (1 + d)t

By conventional standards, a program is deemed worthwhile if NPV>O;

and not worthwhile otherwise.

With regard to the variables in ( ) we right expect Dt to be

high in the early years and taper off thereafter. We would expect the %

opposite for Mt and V(T*)Zt. The selection of a discount rate can be

a complex issue. However, there exist military guidelines and these

* should be considered. Presently, a value of 10% is suc:;,ested for d.

50
.,.-

SUMMARY AND FURTHER CONSIDERATIONS

SUMMARY

This is the second report dealing with CTEA methodology directed

at determining the relative value of TEC training to unit combat

effectiveness. The initial document1 8 reported on a review and evaluation of literature relevant to a TEC CTEA and should be viewed as

an introduction and background to this report. As discussed in the

preceding report, because of TEC's unique complementary role, it does

not fit the usual training program mold; therefore, analytical techniques available in current literature are not completely suitable for

a TEC evaluation. This report develops a TEC-specific evaluation

methodology - a new approach to training evaluation. The approach

adopted in the development of this CTEA methodology is an extension

of a model and several concepts from economic theory. The model used

here to guide the choice among various types of training and to assess

the value of TEC is an extension of the microeconomic theory of the

consumer. Basically, we substitute a training effectiveness production

function for the traditional utility function, where the former has

much the same mathematical structure as the latter. The key concepts -

employed here in developing dollar values for the TEC program are the -

compensatory budget variation and the equivalent budget variation. .

These concepts are central to theoretical welfare economics, where

they provide measures of the welfare effects of price changes on consumers. We extend these concepts to provide dollar measures of the

value of training programs. The TEC CTEA methodology can be used to

provide an objective economic evaluation of the contribution TEC training makes to unit performance. The methodology involves four steps:

o Specification and estimation of an Effectiveness

Production Function.

o Estimation of the "per unit" costs of the different

types of training contributing to overall effectiveness,

as well as determination of the magnitude of the training budget.

o Calculation of the current value of the TEC Program.

o Calculation of the present value of the TEC Program.

18 Sassone, Literature Review - Cost and Training Effectiveness.

51

-.- " l .0 . , S S S S U S- U 9 *
Each of these steps involves a number of substeps. The four steps are

discussed in the second section.

FURTHER CONSIDERATIONS

This report presented a new approach to training evaluation; however, Q

at hispoitit is appropriate to pit out some problems which remain

to be solved before the methodology can be called complete, and before

it can be implemented in a step-by-step, if not mechanical, fashion. Unfortunately, the remaining problems are essentially empirical, and can be

resolved only by collecting and experimenting with actual data. These are

three related areas of concern.

o Units of Observation.

The methodology is compatible with units of any size,

from individuals, squads, and platoons to entire

battalions. However, different choices imply different

data requirements, different data availability and

quality, and different aggregation schemes. In the

same vein, the methodology is compatible with units 4

of different types, e.g. rifle squads, tank crews,

artillery sections, etc., where the observational

units might be grouped accordingly. The time period

covered by a single observation is also open to choice.

.4 Observations might refer to a month, 3 months, year,

etc. In all these choices, theory is of little value

as a guide. Quite si.-iply, the question is which

approach works best.

o Inference.

Very likely it will be neither possible nor desirable

to gather data on all instances of TEC usage. It

follows that the required estimate must be constructed

using some type of inferential procedure.

52

0W le M 0 * W 9 W W a 9
o Size of the Data Set.

The number of variables, and the number of observations of each variable depend on, inter alii, the

variances and invariances in the data. Clearly,

only experience with actual data will reveal these

statistics.

The importance of the foregoing issues should not be minimized.

While none of the problems are insurmountable, their substantial

resolution is a prerequisite to a complete CTEA methodology. And as

already stressed, th-eir resolution depends on the collection of and N

experimentation with actual data.

i 'r 1* APP V W V O --- W 0- W W- W IP MP V --

4%%

414-

it.4 - I