Technical Report 646

Evaluation of the HARDMAN

(Hardware vs. Manpower)

Comparability Methodology El

Wayne Zimmerman, Robert Butler,

Valerie Gray, and Leigh Rosenberg

NASA Jet Propulsion Laboratory 2

0 and

Daniel T. Risser

CD Army Research Institute

Systems Manning Technical Area

Systems Research Laboratory Li

L1

* a.. ~r%-.". H '

C.).

LLU

_ U.S. Army L * L Research Institute for the Behavioral and Social Sciences :

mAugust 1984

At'~rpe 'or oubilc release. diStr bulio jnlimZeCd

?-6 1 2 6-

. .... .,*5 .
U. S. ARMY RESEARCH INSTITUTE

FOR THE BEHAVIORAL AND SOCIAL SCIENCES

A Field Operating Agency under the Jurisdiction of the

Deputy Chief of Staff for Personnel

. 4.

L. NEALE COSBY -U..-

EDGAR M. JOHNSON Colonel, IN

Technical Director Coiniander

Research accomplished under contract

for the Department of the Army Accession For

,' Jet Propulsion Laboratory kI'IC TAB

" O~~~~u:t i/ i~ "''

Technical review by ByQUALITY Dtr;tr ' - IJane M. Arabian 'NSP -,D

Mark Palmisano -,A .

,Di°.° -

NOTICES

DISTRIBUTION: Primary distribution of this report has been made by ARI.

Please address correspondence concerning distribution of reports to: U.S.

Army Research Institute for the Behavioral and Social Sciences, ATTN:

PERI-POT, 5001 Eisenhower Avenue, Alexandria, Virginia 22333.

FINAL DISPOSITION: This report may be destroyed when it is no longer

needed. Please do not return It to the U.S. Army Research Institute for

the Behavioral and Social Sciences.

NOTE: The findings In this report are not to be construed as an official

Department of the Army position, unless so designated by other authorized

documents. t..

.47

. * .!.U
·•· 

THIS DOCUMENT IS BEST 

QUALITY AVAILABLE. THE COPY 

FURNISHED TO DTIC CONTAINED 

A SIGNIFICANT NUMBER OF 

PAGES WHICH DO NOT 

REPRODUCE LEGIBLYo 
BLANK PAGES 

IN THIS 

DOCUMENT 

WERE NOT 

FILMED 
*~~ .- - - -v -. _ Trj-k-.-S 

UNCLASSIFIED

SECURITY CLASSIFICATION OF THIS PAGE (when Dot& Entered)

REPORT DOCUMENTATION PAGE READ INSTRUCTIONS BEFORE COMPLETING FORM

1. REPORT NUMBER 12. GOVT ACCESSION NO. - RECIPIENT'S CATALOG NUMBER

-" ARI Technical Report 646 , +'_'

4. TITLE (and Subtitle) _, -1 TYPE OF REPORT & PERIOD COVERED ~..

EVALUATION OF THE HARDMAN (HARDWARE VS. FINAL

MANPOWER) COMPARABILITY METHODOLOGY 6. PERFORMING ORG. REPORT NUMBER

7. OU.4OR(o) B. CONTRACT OR GRANT NUMBER(e)

Wayne Zimmerman, Robert Butler, Valerie Gray, &

Leigh Rosenberg (NASA-Jet Propulsion Laboratory); 13AR119, 13AR183-35

and Daniel Risser (ARI) 9. PERFORMING ORGANIZATION NAME AND ADDRESS 10. PROGRAM ELEMENT. PROJECT, TASK

Jet Propulsion Laboratory AREA & WORK UNIT NUMBERS

California Institute of Technology 2Q263739A793

Pasadena, CA 91109 -.

II. CONTROLLING OFFICE NAME AND ADDRESS 12. REPORT DATE

US Army Research Institute for the Behavioral August 1984

and Social Sciences, 5001 Eisenhower Ave., i. NUMBER OF PAGES

Alexandria, VA 22333-5600 186

"r4. MONITORING AGENCY NAME A ADDRESS(II different from Controlling Office) 15. SECURITY CLASS. (of thi. report)

UNCLASSIFIED

Iee. DECLASSIFICATION DOWNGRADING

-____SCHEDULE

I6. DISTRIBUTION STATEMENT (ot this Report)

Approved for public release; distribution unlimited

* .:17. DISTRIBUTION STATEMENT (of the abstract entered In Block 20, If different from Report)

IIII. SUPPLEMENTARY NOTES

Dr. John Whittenburg was the ARI contracting officer's representative.

Dr. Daniel Risser was the ARI project manager. The work was performed by JPL

and sponsored by ARI through a reimbursable agreement (ARI Nos. 13ARI19 Change 1

and 13AR183-35, respectively dated Feb 15 and Mar 9, 1983) with NASA (continued)

1S. KEY WORDS (Continue on teverso mide If neceserV end Identify by block number)

Manpower requirement estimation -Training costs.

Human resources in LCSMM Training device requirements

. Training requirements estimation Manpower costs

* QQPRI Logistics analysis

Task IiL gnerntn P 20. AmrTIACT ff~ae m ,evwm e* i m mf weow d idweUify by block nu.mber) .

This evaluation is part of an overall effort to improve the Army's Hardware

versus Manpower (HARDMAN) methodology for projecting manpower, personnel, and

training (MPT) requirements to support new acquisitions. Several different

validity tests are employed to evaluate the methodology: (1) a general examination of the basic reasonableness of the technique; (2) a reliability measure :21

of how well other trained individuals, given the same data inputs, can derive

the same answers as an already-completed test case; (3) a (Continued)

* D , ~uh 173 EVIT10,1 OF I NOV 45IS OWICLETIE

UNCLASSIFIED

SECURITY CLASSIFICATION OF THIS PAIE (W en Data Entered)

%. 

.

., DD. o O4.. 

.

o~ o • o o O o % % , o o o .o o - . +o Oo Oo o , . o. - o o . . - o . .• o O o , . • . • o o7
SuY UNCLASSIFIED

$19CURITT CLASSIFICATIOu Of TNIl PAGI9(VWhfta, 0 # #Wd

ARI Technical Report 646

20. (Continued)

qualitative accuracy check based on talking to individuals who have already

applied the same, or a similar, methodology; and (4) a simple measure of how k7"

accurately the methodology must predict the real world. The evaluation reveals that the methodology conforms fairly well with both the Army's MPT

users' needs and other accepted manpower modeling techniques. Additionally,

audits of three completed HARDMAN applications and the reliability study reveal only a small number of potential problem areas, in particular, a need

* to tighten up select judgmental portions of the methodology and to improve

- those aspects of the technique used for making MPT support and budget decisions. Although no firm accuracy judgment can be made at this time, the

.* results suggest that the manpower life-cycle cost component is only marginally sensitive to changes in other related cost variables. Therefore, the

methodology seems to be sound and to have good near-term utility to the

Army. Recommendations for improving the methodology are provided.

18. (Continued)

(Task RD-182, Amendment 256).

UNCLASS IFI ED

ii %9CuUNT Y CLAIIPIC& Io.. COP wmS #&GCf"*' V&- ~,*

U.'_..~t__ z ..~ -* **- .t..:~ * .. ..-* - - - -- -. . ,. -. --
- -

-% -- - -. -- b - W1 . WJ

-- lTechnical Report 646

Evaluation of the HARDMAN

(Hardware vs. Manpower)

Comparability Methodology

Wayne Zimmerman, Robert Butler,

Valerie Gray, and Leigh Rosenberg

NASA Jet Propulsion Laboratory

and

Daniel T. Risser

Army Research Institute

Submitted by

John L. Miles, Jr., Acting Chief

Systems Manning Technical Area

Approved as technically adequate

and submitted for publication by

Jerrold M. Levine, Director

Systems Research Laboratory . -i

U.S. ARMY RESEARCH INSTITUTE FOR THE BEHAVIORAL AND SOCIAL SCIENCES

5001 Eisenhower Avenue, Alexandria, Virginia 22333

Office, Deputy Chief of Staff for Personnel

Department of the Army

August 1984

Army Project Number Human Factors in Training &

2026373OA793 Operational Effectiveness

Aporovd lot public release: disltrbution unlimited.

iiij

% ° .. . .- , ... .j.. . . ." . .-. .-. ..-. ,. . •, .. % . .. ,, - .-. . . , . . _ . . • .."o. . ° . " . .. , .,
ARI Research Reports and Technical Reports are intended for sponsors of

R&D tasks and for other research and military agencies. Any findings ready

for implementation at the time of publication are presented in the last part

of the Brief. Upon completion of a major phase of the task, formal recommendations for official action normally are conveyed to appropriate military

agencies by briefing or Disposition Form. A

pw

iv

.

......... • *.." •*
FOREWORD

I

The Systems Manning Technical Area of the Army Research Institute (ARI) is

concerned with improving the relationship between soldiers and the equipment

they use. This report provides an evaluation of one particular set of procedures, the HARDMAN (Hardware vs. Manpower) Comparability Methodology (HC),

which is used by the Navy to estimate manpower, personnel, and training

requirements for proposed systems during the very early stages of system design --.

and development.

Like the Navy, the Army has been observing problems with the mismatch between advancing technology and decreasing availability of soldier skills. In

1980, ARI initiated an effort, in support of the Army-wide interest in the

system manning problem, to establish whether the Navy's HARDMAN methodology

could be tailored for use on new Army Acquisitions. Recognizing that approximately seventy-five percent of the system acquisition cost is committed by

Phase II of the development cycle, the Army was also interested in determining

whether the technique could provide early, reasonably accurate, manpower, personnel, and training (MPT) estimates. Because it appeared that the HCN was the

most encompassing and mature of the available MPT models, ARI decided to perform test applications of this technique as well as a validation of the methodology. To date, the Army has funded four test applications: Corps Support

Weapon System (CSWS), Division Support Weapon System (DSWS), Remotely Piloted

Vehicle (RPV), and Single Channel Ground Airborne Radio System (SINCGARS). 0._

The evaluation described in this document represents the first segment of

* the Army's validation effort and draws on the results of three of the test - . -

applications. The HARDMAN methodology has good near-term utility as a solution

to the Army's MPT dilemma.

Technical Director

V

4,24.

*. . . . . . . . .. .. . . . . . . . ....- .
ACKNOWLEDGMENTS _• ....

4,...-,

This document is the product of a team effort. Wayne Zimmerman (JPL), the

task manager and prime author, was assisted by several colleagues. Valerie ..-

Gray (JPL) was responsible for the user requirements portion of the study and

assisted in performing the reliability study. Leigh Rosenberg (JPL) organized

and conducted the reliability study. Robert Butler, of the Assessment Group

consulting firm, performed the event-series validity evaluation. Mark Franklin

(JPL) and Steven Parks (JPL) participated and provided assistance, respectively,

in the SINCGARS replication and technical audit efforts, which were part of the

validity studies. Dr. Uldi Shvern (ARI), Mr. Theo Feng (ARI), CPT John Smith

(Soldier Support Center, National Capital Region), Mr. Bernard Schuster (Soldier

Support Center, National Capital Region), and MAJ Brad Loo (Soldier Support Center, National Capital Region) all participated in the SINCGARS replication effort. Dr. Daniel Risser (ARI) provided a broad range of technical and administrative assistance and guidance throughout the project. Shirley Stroup (JPL)

was responsible for typing the document.

The Army Research Institute for the Behavioral and Social Sciences (ARI)

would like to acknowledge the cooperation and assistance provided by the Soldier Support Centers (SSCs) at Fort Benjamin Harrison, IN, and Alexandria, VA,

and the SINCGARS Program Office, which participated in the study. A special

acknowledgment is extended to the Dynamics Research Corporation (DRC) HARDMAN

team for their time, cooperation, and participation.

vi
EVALUATION OF THE HARDMAN (HARDWARE VS. MANPOWER) COMPARABILITY

METHODOLOGY

EXECUTIVE SUMMARY , % 

Requirement:

To improve the Army's capability to project manpower, personnel, and

training (MPT) requirements for proposed weapon systems (NEW) by evaluating

the credibility and utility of the HARDMAN (Hardware vs. Manpower) Comparability Methodology (HCM), a Navy methodology modified for use in an Army

environment.

Procedure:

Several different validity tests were employed to evaluate the methodology and form Lhe basis for recommendations for improvement. These tests

included (1) a general examination of the basic logic and appropriateness of

the technique; (2) a reliability measure of bow well other trained individuals, given the same data inputs, can derive the same answers as an alreadycompleted test case; and (3) a qualitative accuracy check based on talking .'.

to individuals who have already applied the same, or a similar, methodology.

Findings:

HCM conforms well with both the Army's MPT users' needs and other accepted i.anpower modeling techniques. Audits of three completed HARDMAN applications reveal only a small number of potential problem areas. The reliability study results conform well with the problem areas uncovered through

the audits. The major problems discovered basically revolve around a need

to tighten up select judgmental sections of the methodology and to improve

those aspects of the technique used for making MPT cost estimates. Recom- . -

mendations for improving the methodology are provided.

Utilization of Findings:

The evaluation reveals that even with minor problems, the methodology

is sound and has good near-term utility for the Army. Development of an

analyst guide is underway. Of equal importance is the initiation, by the

Soldier Support Center, National Capital Region, of five analysis applications

to be executed in FY 85. These analyses will support on-going weapon development programs and contribute to the further refinement of the methodology.

vii

. . .. 
GLOSSARY

AOSP Army Occupational Survey Program

AR 570-2 Army Regulation for Manpower Authorization Criteria

AR 611-201 Army Regulation for Enlisted Career MOS Management Fields

ARI Army Research Institute for the Behavioral and Social Sciences

ARTEP Army Training Evaluation Program

ASARC Army Systems Acquisition Review Council BOIP Basis-of-Issue Plan

CDB Consolidated Data Base

CFP Concept Formulation Package

COEA Cost and Operational Effectiveness Analysis

CRT Cathode-Ray Tube

DA Department of Army

DCSCD Deputy Chief of Staff Combat Development

DCSLOG Deputy Chief of Staff Logistics

DCSPER Deputy Chief of Staff Personnel

DCST Deputy Chief of Staff Training

DID Data-Item Description

)OD Department of Defense

HARDMAN Hardware vs. Manpower

HCM HARDMAN Comparability Methodology

HIS HARDMAN Information System

HLCCM HARDMAN Life-Cycle Cost Methodology

HQDA Headquarters, Department of the Army

HTRRDM HARDMAN Training Resource Requirements Determination Methodology

ICTP Individual and Collective Training Plans

ILS Integrated Logistic Support

ILSP Integrated Logistic Support Plan

ISS Integrated System Support

JPL Jet Propulsion Laboratory

LCC Life-Cycle Cost

LEA Logistics Evaluation Agency

LOA Letter of Agreement

LO;CEN Logistic Support Center

LSA Logistic Support Analysis

L.SAR Logistic Support Analysis Record

MACR IT Manpower Authorization Criteria

MANPERS Manpower and Personnel Requirements Determination Methodologies

MMII Maintenance Man Hours

MOS Military Occupational Specialty(ies)

MPIT Manpower Personnel and Training

( and 0 Operational and Organization

OICTP Outline Individual and Collective Training Plan

1101 Program of Instruction

1'QA Preliminary Quanti tat ive Assessment

QQPRI Quantitative and Qualitative Personnel Requirement Information

RAM Reliability, Availability, Maintainability

R]I Request for Proposal

SI) Soldier Development 1)i rec torate

SQ'I Skills Qualification Test

SSC Soldier Support Center

ix
SSC-NCR Soldier Support Center-National Capital Region

SSG Special Study Group

TAD Target Audience Description

TG Training Guide

TOE Table of Organization and Equipment

TQQPRI Tentative Quantitative and Qualitative Personnel Requirements

Info rma tion

TRADOC Training and Doctrine Command

TSM TRADOC Systems Manager

Ix

I .

ir ± - ~ ~ - ~ - - . . - - .- - - *-,. - . . - *
EVALUATION OF THE HARDMAN (HARDWARE VS. MANPOWER) COMPARABILITY

* METHODOLOGY

* ~CONTENTS _______________________________

Page

I. INTRODUCTION AND SUMM'ARY................... . . .... . .. .. ......

A. Objectives.......................... .. .. ...... . .. .. .. .....

B. Background.......................... .. .. ...... . .. .. .. .....

C. Assumptions and Constraints..........................2

D. Summary of Evaluation Approach and Results of Analysis . . . . 3

E. Structure of Document...............................9

* ~II. HARDMAN BACKGROUND........................................................10

A. Overview.......................................10

B. HARDMAN History.....................................................10

C. Army Involvement in HARDMAN........................................12

D. Description and Intended Use of the Army HARDMAN System .. 13

Ili. STRUCTURE OF HARDMAN ANALYSIS..........................................16

A. Overview.....................................16

B. Validity Testing...................................16

C. User Requirements Analysis........................17

D. Operational Analysis.............................18

E. Reliability Analysis...........................18

IV. USER REQUIREMENTS ANALYSIS..........................20

A. Overview.....................................20

B. Approach.......................................20

C. Tabulation of Interviews.............................22

D. Translation to User Requirements.....................31

E. Relevance of User Requirements to Existing HARDMAN Design .. 34

V. OPERATIONAL ANALYSIS....................................................40

A. Overview.....................................40

B. Approach.......................................................40

C. Analysis of Existing DO0D MPT Projection Guidelines.............42

D. Critique of Other Example MPI Projection Methods................44

E. Comparison of DOD and Other I4PT Guidelines Against HCM. . . . 51

F. Use of Audits to Examine Face Validitv .................. 52 .-

KG. Validity Implications..............................59

xi
CONTENTS (Continued)

Page

VI. RELIABILITY ANALYSIS.........................65

*A. Overview.............................65

B. Approach.............................66

C. SINCGARS Replication Results...................72

D. Qualitative Accuracy Study....................83

E. Results of Manpower Sensitivity Analysis.............85

F. Reliability Implications.....................92

VII. CONCLUSIONS AND RECOMMENDATIONS...................95

A. Overview.............................95

B. Summary of Results........................95

C. Prioritization of Validity Issues.................95

D. Recommendations.........................100

REFERENCES.................................115 6

APPENDIX A. INTERVIEW QUESTIONS FOR USER REQUIREMENTS ........... A-1

B. USER REQUIREMENTS SUMMARY MATRIX OF QUESTIONNAIRE

RESPONSES...........................B-1

C. AUDIT QUESTIONS.......................C-1

D. DEBRIEFING QUESTIONNAIRE..................D-1

E. DATA INPUTS FOR THE MANPOWER SENSITIVITY MODEL ....... E-l

LIST OF TABLES

Table 1-1. Net results of HARDMAN evaluation................5

4-1. User groups/representatives interviewed............21

4-2. Key MPT drivers and associated user rationale ......... 23

4-3. Comparison of user requirements to the HARDMAN

Comparability Methodology...................35

5-i. Comparative overview of t4PT methodologies............53

5-2. Summary of audit results in terms of face-validity

elements.............................62

6-1. Data package problem statements and descriptions ........ 67r

6-2. Baseline Iife-cyc.e cost sensitivity analysis run.......71

xii
CONTENTS (Continued)

Page

Table 6-3. Functional requirements answer sheet ... ........... .. .. 74

6-4. Work Package 1 - respondent vs. Dynamics Research

Corporation results ....... .................... ... 75

6-5. Work Package 2 - respondent _i, Dynamics Research

Corporation results ....... .................... ... 76 .- *-i

6-6. Work Package 3 - respondent vs. Dynamics Research ,:

Corporation results ....... .................... ... 77

6-7. Corrective maintenance - workload worksheet .......... ..... 78 

6-8. Work Package 4 - respondent vs. Dynamics Research

Corporation results ....... .................... ... 79

6-9. Sample Dynamics Research Corporation answer sheet ..... .. 81

6-10. Work Package 6 - Dynamics Research Corporation vs.

respondent answers ........ ..................... ... 82 .

6-11. Work Package 8 - Dynamics Research Corporation vs.

respondent results ........ ..................... ... 84

6-12. Qualitative accuracy study contacts ... ............ ... 84

7-1. Net results of HARDMAN evaluation .... ............. ... 96

7-2. Common validity issues ....... ................... ... 97

7-3. Recommended plan for improving HARDMAN methodology ..... ... 101

7-4. Credibility rating of data sources and manpower projections

for proposed and reference system comparison .. ....... .104 ..

7-5. HARDMAN methodology tradeoffs ............... 108

LIST OF FIGURES

Figure 2-1. Steps in application of the HARDMAN methodology ...... . 14.....

4-I. Effect of design decisions on life-cycle system cost . 244. 

5-1. MACRIT manpower cruve ................... 43

5-2. Typical Norden/Raleigh manpower curve ... ........... ... 45

6-1. Errors in input variables producing 15% error in manning 88

.i ° 

xiii-i

".°..'.-. -- •:" -•-.-'. ."- -. " .f., '.-',:£.7#.:, "-",° . " ".' " ' °'- . " ." ).".- 7- '-" -" x • • " ." ,'"-" " """ ," ° " ", - " "i " '
CONTENTS (Continued)

Page

*. Figure 6-2. Response of life-cycle cost and manning to equal

changes in other LCC variables .... .............. . 89

6-3. Comparison of spares and manning errors .. .......... ... 90

6-4. Relationship of joint errors to manning .. ............. 91-

7-1. Revised MACRIT manpower projection .... ............ . 107 $ .

7-2. Use of confidence intervals for manpower projections . . . 109

xiv
EVALUATION OF THE HARDMAN (HARDWARE VS. MANPOWER)

COMPARABILITY METHODOLOGY

SECTION I

INTRODUCTION AND SUMMARY

A. OBJECTIVES

The Jet Propulsion Laboratory (JPL), sponsored by the U.S. Army Research Institute for the Behavioral and Social Sciences (ARI), was asked to

provide the Army with an objective, independent assessment of the validity

and reliability of the HARDMAN (Hardware vs. Manpower) Comparability Methodology (HCM) for projecting manpower, personnel, and training (PT) requirements. JPL was also asked to make recommendations for needed improvements

in the methodology that would ultimately help the Army implement an accurate

MPT system that could be applied early in the acquisition of new systems. r

The following paragraphs discuss the background behind the Army manpower and training issue, the major assumptions and constraints that established the framework of the study, the evaluation approach and results, and

the overall structure of the document.

B. BACKGROUND

Over the past several years, the Army has recognized a critical manpower situation revolving around the widening gap between increasing weapon

complexity and declining manpower availability. The problem of declining

manpower availability has been aggravated by the apparent decay of entrants'

skill levels. In 1980, as part of an Armywide interest in improving the

manpower problem, ARI initiated an effort to determine whether the Navy's

HARDMAN methodology could be applied to Army systems. The Army was interested in determining whether an existing manpower, personnel, and training

projection technique could be modified to provide early, accurate inputs to

the acquisition process. The Army chose to examine the HCM (one of several

methodologies developed under the auspices of the Navy HARDMAN Office) because 

eral Navy 

it was 

acquisition 

an integrated 

programs. 

MPT technique 

The other 

and 

Navy 

had 

methodologies 

already been 

appeared 

used in sevto be

K

too narrow in scope or required input data not available under the Army's

existing data collection systems. Although these other systems have not

been ruled out for Army use, it seemed that in the near term, the HCM would .-*-.

provi le the greatest payoff. The HCM involves the following six major steps:

(I) Establish a consolidated data base composed of functional descriptions of the proposed and similar predecessor systems and

associated inputs, such as hardware reliability, personnel information, costs, and training data. This information is used to

1.,-
support succeeding steps in the analysis. This step is essential

because the Army does not usually organize and store data in

effective structure to conduct human resource analysis.

(2) Determine the manpower requirements and skills necessary to

operate and maintain the system.

(3) Determine training requirements likely to be imposed by the

proposed system. 0

(4) Determine personnel requirements (i.e., personnel flows that will be necessary to support the manpower requirements). ''"

(5) Conduct an impact analysis to establish likely manpower and

training shortages and identify parts of the system that represent

high-cost drivers.

(6) Perform a tradeoff analysis to alter features of the system (e.g.,

reliability or required skill levels) to reduce or eliminate

unreasonable demands.

Each of these steps involves several judgments pertaining to the

selection of comparable predecessor systems, the identification and assembly

of associated data on the predecessor systems, expected performance of the

proposed system as compared to predecessor systems, and required skills.

Additionally, gaps in the supporting data often exist at the conceptual design

stage. At these junctions, expert judgment is exercised to merge the various

pieces of information into a comprehensive, consistent picture to be used to

generate the personnel projections. The Army's concern is that these judgment

areas could act as major error sources in the manpower projection.

Consequently, JPL was asked to perform an objective evaluation of the

methodology to identify both strengths and weaknesses of the technique and

make recommendations, as appropriate.

C. ASSUMPTIONS AND CONSTRAINTS

One of the methodology validity tests involved a comparison of the user

needs against the proposed outcomes of the projection scheme. To establish

the user requirements, a detailed interview process was designed to use the

bank of experience already existing in the Army. The major assumption

involved with the user requirements effort centered on the user contacts.

Because the user contacts in the various Army commands were established via

the sponsor and recommendations from the users themselves, it was assumed that

the suggested sources were sufficiently experienced and knowledgeable in the

MPT arena so that their answers were representative of the respective commands

in general. Even though the sample size was small, the consistency of the

responses suggested that the answers were a good representation of the Army as

a whole.

2

. .. .. . .... . .
Several limitations were imposed on the study. The size of the sample

user population was limited primarily by the available time to complete the

evaluation. Some of the users contacted were more informed or aware than

others about MPT processes or projection models such as the HCM. Therefore,

it was understood that some of their responses could reflect an already

established preference, depending on their knowledge of the various elements

present in the CM or other MPT models. To circumvent this bias, the

questionnaire and interview technique employed in this evaluation focused on

getting the user to provide requirements for a "generic" MPT projection system

for the Army.

It should be noted that the user requirements determined by this study

reflect the needs of a present or near-future MPT system. Any changes to this

environment could result in additional or different requirements (e.g.,

implementation of Integrated System Support (ISS)). Therefore, it is

important that the Army consider periodic reevaluation of its MPT requirements

to ensure that any in-place MPT system remains responsive to the needs of the

various MPT players.

Although it was originally planned to do a thorough validity and

reliability evaluation of the methodology, internal time and budget limitations resulted in the sponsor opting to defer (1) the accuracy check on the

MPT methodology, (2) the life-cycle cost (LCC) and methodology sensitivity

analysis, and (3) the quantitative accuracy check on the methodology input

data (e.g., reliability and maintainability data). Finally, other duty

commitments required that the internal validity experiment be simplified so

that the Army participants could finish the replication within a maximum

three-week time frame.

D. SUMMARY OF EVALUATION APPROACH AND RESULTS OF ANALYSIS

The preceding discussion outlined the various steps and judgments

necessary to exercise the HCM. Because of the breadth and complexity of MPT

issues covered by the technique, a multifaceted evaluation was structured to

examine potential validity aspects. The validity aspects, evaluation

approach, and study results are summarized in the following paragraphs.

I. Evaluation Approach

As stated earlier in this section, time and budget constraints

limited the original scope of the study. Validity measures related to testing

model accuracy were deferred to the next study phase. However, within the

imposed constraints several other validity tests were conducted. Face

validity, a measure of whether the model appears logical and reasonable, was

the first aspect investigated. Although not a rigorous credibility test, a

face-validity study does afford some insights into the general utility of the

model, potential error sources, data availability problems, logic foundation,

and possible ambiguities in logic structure or results. The approach for

performing the face-validity examination was fourfold. First, a user

requirements study was performed to establish the Army's needs within the MPT

3 .,er-'e

........ -.. -_ .. '---- -....................... - _?_ _'_- --- .... . .. .
JPf

community. A total of nine distinct user groups (thirteen individuals) were

interviewed in person to establish user HPT needs in areas, ranging from the

basic design of the HPT projection system to the kinds of information desired

from the system outputs. A basic utility test was then performed by comparing

each user requirement against the same respective element in the HCM. Second,

existing DOD and Army directives were examined to see whether there were any

established, accepted 1PT techniques presently in use that could be used to

confirm the general logic structure and data foundation. Third, a larger

literature search was conducted, encompassing other government agencies and

the private sector. Again, the object was to develop a firm basis for .*,

evaluating the logic and data support structures of the HCM, as well as

examining potential sources for error. Last, detailed audits were conducted

on three applications of the methodology. Independent analysts attempted to

replicate major empirical and judgmental steps in each application with the

intent of examining overall logic, potential sources of errors, data

manipulation or availability problems, and ambiguities in how the projections

were derived and interpreted.

Another test, internal reliability, was employed to get an understanding

of the ability of others to successfully apply the technique and obtain

consistent answers. An internal validity test is essentially a measure of the

variance in model outputs when a group of independent analysts replicate an

already completed application given the same inputs. Ultimately, the

objective is to establish the clarity and succinctness of the model, as well

as the ability to have any reasonably trained individual perform the process

reliably. An actual Army system was selected for the internal validity test.

A group of seven individuals (four civilians and three Army officers) were

initially trained in the execution of the HCM and then provided with data

packages that allowed them to conduct a "limited" manpower and personnel .-.-

assessment on the proposed system (i.e., because of other duty comitments and

resultant time restrictions, only four major components of the system were

studied).

It was decided that an attempt should be made to examine model accuracy

from at least a qualitative standpoint. To this end, several individuals in

both the Air Force and Navy, who had experience with applications of the same

(or similar) MPT model, were contacted for information pertaining to any tests

or actual comparative field data that might indicate the relative accuracy of

the methodology.

Finally, the last validity aspect examined was an event-series test.

Event-series validity is basically a measure of how accurately a simulation

must reproduce a real-world event. A top-level, life-cycle, cost-sensitivity

analysis, keying on the manpower variable, was employed as the tool to achieve

this measure. The required accuracy of the manpower variable was determined

by performing iterations of the life-cycle cost projection (using an example

acquisition) until all other dependent variables, manning, and total

life-cycle cost fell within prescribed error bounds. This measure was

particularly important for obtaining a further understanding of the model

usefulness as a decision making tool.

4

* .***

• . * *. .- ** . ~:~ .*..<~ .- .-. ::.-
* . - ....- - . T 

. -- .- =--.-,. - ...- - -.- . - - - . - ---- . . . . .. . . . .

Throughout all of these validity exercises several strengths and parallel -

problem areas surfaced. The complete results of the evaluation are summarized

in the following discussion. "

2. Summary of Results

Table 1-1 provides an overall summary of the study results. Each

of the above evaluation areas is discussed briefly in the following paragraphs.

Table 1-1. Net Results of HARDMAN Evaluation

Evaluation Areas Net Results

User Requirements Technique complied with majority of mandatory

user requirements

Operational Analysis

Comparison with Methodology conformed with other known, accepted

other MPT methods MPT modeling schemes, and data foundations

Audits Methodology demonstrated sound logic and

reasonable results for 85% of test issues

examined (remaining issues considered reparable

in near term)

Reliability Analysis

Internal reliability Test group correctly replicated half of selected

test points (two of the four remaining test

points found to be repeatable after clarification)

Qualitative Two of nine individuals having experience with

Accuracy Check HARDMAN applications indicated a rough accuracy

of 80-90% with actual manning requirements; P

remaining individuals had insufficient experience

to comment

Event Series Validity Manpower variable found to be least sensitive of

all life-cycle cost variables, demonstrating good

utility of existing methodology even without F Z

suggested near-term improvements

5
a. User Requirements Analysis. The user requirements analysis

generated a total of twenty-two mandatory MPT system requirements. Thirteen

of the mandatory requirements applied to system outputs, and the remaining

nine requirements were related to system design. An additional four .require-

£ ments were given lower priority by the users and were therefore listed as

"desirable." The major thrust of the user needs revolved around the key design

drivers of affordability, supportability, and sustainability. By comparison,

it was found that the HCM complied with seventeen of the twenty-two mandatory ".

requirements. The areas of noncompliance were as follows:

(1) Not indicating the accuracy of the MPT projection system to aid

budget control and program-level decisions.

(2) Not including demographic data such as age, sex, size, etc., as

part of the man-machine matching and support process.

(3) Not considering the demands of other acquisitions on the available Z

manpower pool (i.e., the HCM still focuses primarily on the

individual equipment system).

(4) Not performing a complete life-cycle cost tradeoff analysis (i.e.,

the HCM focuses only on HPT costs).

(5) Not indicating weaknesses in the MPT analysis along with

respective resolution actions.

Overall, it seems that the HCM is reasonably responsive to the Army's

needs.

b. Operational Analysis. The literature search and audit

aspects of the face validity study were jointly conducted under the

operational analysis portion of the evaluation. This portion of the overall

study was broadly termed "operational analysis" because the ensuing

investigation required a rigorous exercising of the HCM. The review and

comparison of related DOD and Army directives provided three major findings:

(l) DOD system acquisition guidelines stated a requirement to consider MPT in

the design process but did not appear to afford a means of projecting MPT

requirements; (2) the Army Manpower Authorization, Criteria (MACRIT) guide did

provide a simple manpower projection algorithm; and (3) the HCM appeared to be

designed around the MACRIT manpower algorithm, with its outputs generally

organized to conform with broader DOD acquisition directives. These insights,

although serving as good initial data points, only addressed a fraction of the

total methodology. The follow-up government and private-sector literature

searches were far more intensive. Nineteen other MPT models and papers,

encompassing all aspects of the HCM, were studied. Generally, it was

discovered that the other modeling approaches were reasonably equivalent in

their assumptions, algorithms, and data inputs. In one area, the personnel

pipeline projection, the HCM approach was found to be stronger because it

S***,*., 6 .. * --...
- . - .

employed a modeling technique that reduced the need for difficult-to-obtain

data such as present and future manpower availabilities. On the negative side,

it seemed that there were four distinct areas where potential inaccuracies

could be interjected: (1) The use of the MACRIT manpower algorithm could

result in early or late life-cycle manpower projection errors unless system .

deployment and phase-out rates are considered; (2) non-consideration of major

socio-economic factors such as the draft, economy, change in federal govern-

- ment administrations, or massive military reorganizations could influence the

* attrition and retention characteristics of personnel career paths; (3) the

non-use of some type of long-term manpower availability projection to indicate

shortages at the time of system deployment could result in sizeable personnel F.-

errors; and (4) the lack of a complete life-cycle cost model could prevent the

full range of potential cost and system design tradeoffs from being provided

to the decision makers.

The last aspect of the face-validity analysis, the audits, revealed that

the model successfully passed eighty-five percent of the test issues examined. -

The remaining potential problem areas discovered were (1) the lack of a " 2

structured process of selecting, adjusting, and manipulating the source data

could sizeably affect the accuracy and credibility of the projections; (2) the

use of a single reference system (composed of similar predecessor components)

to model several widely varying proposed designs could also affect the

accuracy and credibility of the results; (3) the absence of an adjustment to

the complete predecessor historical data to consider the design improvements

of the proposed system means that it would not be feasible to make a valid

comparison between real world performance and the often overly-optimistic

projection of the contractor; (4) the lack of a structured process for

approximating the value of peculiar design changes or induced human error as

related to component failure rates presents a means of interjecting computational mistakes as well as reducing both the audit and usability capacities of

the method; (5) the absence of indicators for data quality and projection

accuracy could affect the credibility and usefulness of the results; and (6)

the complexity of the technique requires the use of a well-seasoned, multidisciplinary team which, in turn, could impact the ability of others to apply

the method.

c. Reliability Analysis. The last parts of evaluation, the

internal and event-series validity tests and qualitative accuracy check, were

performed under the reliabiity analysis. This was done because all of these

validity aspects related to both the internal and external reliability of the

process. The results of the internal validity check were closely aligned with

the audit findings. As a whole, the test group discovered that, given the

same inputs, it could not replicate the original SINCGARS results. Although

all of the study group seemed to understand roughly half of the early steps in

the process reasonably well, they had problems with (1) developing a reference

system that jointly met the functional requirements and was representative of

the two widely varying contractor designs; (2) not understanding the thought

process for determining impacts of design differences, and how and when to

perturb historical predecessor data to model components of the proposed

system; and (3) not comprehending the exact process of matching or selecting

skills. After items (1) and (3) were clarified, the study group had a fairly

*... .. 

................. ..--...... ..... .... -,. . . ._.. .. ......... . .. .'
. . . ,.. ~ - -. -. _ .. . . . .-. -. _° . _ . ." - < : -=T . - rv: -7 .rr. . .- . o-- -. . . _

good grasp of the majority of the early steps in the process. The test

subjects uniformly agreed that none of them singularly had enough training or

educational background to exercise the complete methodology with any .. g

confidence.

The major finding in the event-series validity study was that the MPT

projections could easily vary by plus or minus fifteen percent without causing

major problems at the upper DOD budget and cost control management levels.

Component usage rate was found to have the greatest influence on causing

unacceptable errors in manpower projections. Although it was jointly discovered that there is still not sufficient historical experience to make a

firm qualitative accuracy judgment on the process, the low sensitivity finding

on the manpower variable was a promising indicator of the utility of the model

even with its minor drawbacks.

d. Conclusions and Recommendations. In conclusion, it was felt

that, overall, the HCM appeared to be reasonably sound. The flaws discovered

did not appear to be irreparable. The major near-term recommendations

suggested to tighten up the problem areas were as follows: (1) Establish

subjective criteria for rating the quality of various data sources; (2)

provide an indication of the credibility of the projections; (3) establish a

top-level method for comparing the technology of the reference and proposed

systems on an equal basis; (4) establish solid guidelines and information

sources for building reference systems and consider the use of two reference

systems to model two widely varying contractor-proposed designs; (5) firm up

guidelines for the functional requirements level of indenture; (6) provide a

better structure for MOS and skill selection; (7) periodically update manpower "

projections (every two to four years) over a system's life cycle to offset

possible end-point errors; (8) provide a more structured tradeoff process; and

(9) standardize the assessment of study weaknesses and resolution actions.

The intermediate term (two years from the present) priorities were

(1) project manpower availability out to the actual system deployment date;

(2) incorporate a more complete life cycle cost tradeoff analysis; (3) firm up

ranges for perturbation values, induced failures, and indirect productivity

factors.

Far-term recommendations (for the three- to five-year time frame

following the present) included (1) consideration of other emerging system

manpower demands; (2) development of a quantitative means to differentiate

between critical and non-critical tasks; (3) expansion of the demographic data

base; and (4) consideration of major socioeconomic fluctuations. Potential

solutions for all of the preceding recommendations were also provided.

In addition to these recommendations, it was also suggested that, based

on the audit and reliability findings, future HARDMAN training programs solicit

a multidisciplinary team of individuals with backgrounds in the military,

engineering, information systems and data management, and personnel. Further,

combinations of two of more of these disciplines in each individual as part of

the overall multidisciplinary team seem to form the best foundation for accurately exercising the methodology.

8... -

. - . .i

- -. -. - .. _ ".- "-. ' -- .-. '- - .-.. ' ' '., . .. *. -. , *.' . -" ' '.. ;'. '. . . . " - . ". " " .. .- . ' - .-'. . - "". --'. .. '- ' "
E. STRUCTURE OF DOCUMENT

This document is structured so that a brief history of both the HARDMAN

effort and Army involvement in HARDMAN is provided first in Section I1 as a

basis for the study. Section III discusses the overall design of the analysis

and broadly explains the purpose and content of the various validity studies.

Sections IV through VI provide the detailed results of the methodology evaluation in the major areas of user requirements, face validity (operational

analysis), and reliability (internal and event-series validity). Finally, '.

Section VII discusses the implications of the results and provides the overall

assessment of the technique with recommendations. i. .-t

r

9I

- -A°.'
SECTION II

HARDMAN BACKGROUND

A. OVERVIEW

The previous section provided a brief summary of the evaluation approach

and results. As stated earlier in Section I, the Army has been concerned

about the widening gap between growing technology and decreasing availability

of needed skills. Manpower guidelines already exist through DOD documents

such as the 5000 series and Military Standard (MIL-STD) 1388. These documents,

which establish acquisition and support procedures for new systems, address

the requirement to have a proper man-machine balance but do not specify

techniques to accomplish the balance. Given the ever-widening technology and

skills gap, the Army has been pressed to develop clear, accurate manpower

projection techniques to fill the void in existing DOD acquisition guidelines.

The following paragraph4 provide a brief summary of the history and progression

of events leading up to the Army's decision to develop an appropriate MPT projection system...

-., .t .

B. HARDMAN HISTORY

The weapon system acquisition process has evolved over a long period of

time. The overall system acquisition strategy suggests that the number of

competing alternatives should be narrowed by eliminating those concepts no

longer considered viable as a function of four major measures (1,2): (1) the

ability of a design to meet the proposed mission; (2) the compatibility of the

design with the anticipated threat; (3) a comparison of each concept with

operational requirements; and (4) a comparison of each design's threshold

values (i.e., cost, performance, readiness, and supportability impacts).

These measures are employed throughout the system development process. The

development process consists of four phases containing three major milestones or acquisition decision points (2):

(1) Phase I (Milestone I) - Concept development.

(2) Phase II (Milestone II) - Concept demonstration and

validation.

(3) Phase III (Milestone III)- System development.

(4) Phase IV - System production

and deployment.

At the end of each phase or milestone, a go/no-go decision is made by

the Army System Acquisition Review Council (ASARC) and Defense System Acquisition Review Council (DSARC) as to whether to proceed to the next milestone

(1,2). A key consideration in all of the preceding measures and subsequent

decision points is the integration of manpower, personnel, and training (MPT)

1i.
into new designs. Recent studies (3, 4) by some of the services (i.e., Air

Force and Navy) indicate that operations and support costs can consume from 40

to 50% of the total life-cycle cost of a weapon system. Additionally, the

manpower component of the support portion of life-cycle cost can consume as

much as 60%. These studies underline the growing importance of MPT. %.k

Although the Air Force was the first service to develop a major thrust

to resolve the MPT dilemma, the actual formalization of the HARDMAN concept,

as presently defined, started with the creation of Navy's HARDMAN Development ____

Office in 1977 (5). The Navy's HARDMAN effort was initiated partially because

of studies similar to References 3 and 4 and to the fact that present DOD

directives do not adequately define methods for projecting M'PT requirements

for new acquisitions. The first Navy HARDMAN study, initiated in 1975, was

funded by the Chief of Naval Operations (CNO) Studies and Analysis Program.

This study established that (1) systems were being designed without adequate

attention given to MPT costs and (2) logistics support planning (and in

particular MPT) for new acquisitions was inadequate. The CNO study outlined .

the following solutions:

(1) Develop analytical tools and methods for MPT to influence the

weapon system acquisition process.

(2) Develop an MPT information system.

(3) Revise existing directives governing the acquisition process to be

more sensitive to MPT.

As a result of the CNO study and some aggressive management, the Navy

established the HARDMAN office and set about developing a modeling system

composed of four linked components consistent with the aforementioned CNO

guidelines and tailored to the various design phases.

The linked modeling system, which is the present HARDMAN design,

basically consisted of the following:

(1) A manpower projection model.

(2) A training resource assessment.

(3) A life-cycle cost analysis.

(4) A HARDMAN information system. K

The manpower projection link of the system (HCM) was designed by

Dynamics Research Corporation (DRC) and was developed around the concept of

using information from cimilar predecessor technology as a basis for manpower

projections (6). Drawing on an extensive historical data base as well as the

new design inputs, the manpower for the proposed system is estimated by

altering the historical data as a function of the peculiar design improvements

and fitting these results into a simple manpower equation consisting

principally of total workload manhours and total systems to be procured. The

HCM proceeds to establish required skills, training, MP costs, and potential

personnel shortages through the remaining steps in the methodology. MPT

estimates are then refined as the input data matures through subsequent design

stages.

11-i
The PACER Corporation developed the training resource link (HTRRDM),

designed to provide a checklist and straightforward algorithms for defining

elements such as training sites, instructors, instructional devices, course

material, and training costs (7).

The life-cycle cost link of the HARDMAN system (HLCCM) was developed by

the Assessment Group firm and designed in two levels of detail to be commensurate with the different stages of design development. Drawing on standard

life-cycle cost algorithms, a skill-cost model, and designer inputs, the technique provides both the total-system, life-cycle cost and the ability to

quickly perform life-cycle cost tradeoffs between various logistic elements (8).

The last system link, the HARDMAN Information System (HIS), was designed

by Ackman Associates. Although not completed, the HIS is structured to be the

repository for all predecessor system MPT and life-cycle cost data as well as data for new potential acquisitions.

The Navy effort has considerable value for several reasons. Most

important is the clear demonstration of the necessity to bond MPT securely

with the acquisition community. Another clear value of the Navy's effort

revolves around the many manpower models developed by DOD and the various

services over the past several years. These models generally have only

addressed select aspects of the MPT problem such as manpower flows in and out

of various skill areas or total manpower requirements. The Navy's thrust has

been to develop a total MPT system, which extends beyond these smaller

studies. Additionally, the Navy's program has produced a large bank of

experience and MPT tools.

C. ARMY INVOLVEMENT IN HARDMAN'

Like the Navy, the Army has been observing similar problems with the

mismatch between advancing technology and decreasing availability of skills.

In 1980, the Army Research Institute (ARI) initiated an effort, in support of

the Army-wide interest in the system manning problem, to establish whether the

Navy HARDMAN effort could be tailored for use on new Army acquisitions.

Recognizing that approximately seventy five percent of the system acquisition

cost is committed by Phase II of the development cycle, the Army was also

interested in determining whether the technique could provide early,

reasonably accurate MPT estimates. As indicated in the preceding section,

there was considerable overlap between the comparability, training, and

costing links. Because it appeared that the HCM was the most encompassing and

mature of the MPT models, ARI decided to perform some test applications of

this technique as well as a validation of the methodology. To date, the Army has funded four test applications: Corps Support Weapon System (CSWS),

Division Support Weapon System (DSWS), RemoLely Piloted Vehicle (RPV), and

Single Channel Ground Airborne Radio System (SINCGARS). The study performed

in this document represents the first segment of the Army's validation effort

and draws on the results of three of the test applications. The HCM presently 1.-

represents a possible near-term solution to the Army's MPT dilemma.

", ...-'--- ' :.... -: '-5-:,'-'~ ,, . -" ,".".".' -,. . . .- -,-...,.... .. ., ... . . . ... ... .... . . .. . ... . . . . . . . .
D. DESCRIPTION AND INTENDED USE OF THE ARMY HARDMAN SYSTEM

The Army HARDMAN system is presently emphasizing only the HCM. The

technique consists of six major steps as shown in Figure 2-1 (6). Step 1,

Establish Consolidated Data Base, includes the development of functional

descriptions of the alternate systems and the collection of the input data

(hardware, personnel, training, and cost) needed to support the succeeding

analyses. Step 2, Determine Manpower Requirements, refers to the process of

identifying the number and skills of soldiers necessary to operate and

maintain the system. Step 3, Determine Training Requirements, is done in

parallel with Step 4 and requires the specification of the numbers and types

of instructors, courses, media, and training devices, etc., which are

necessary to accommodate the new equipment. Step 4, Determine Personnel

Requirements, addresses the activity of identifying the peculiar occupational

speciality requirements. Step 5, Conduct Impact Analysis, focuses on the

activity of comparing assessed demands (from Steps 2-4) and available supplies

and notes unreasonable resource demands in terms of both manpower and training

costs and personnel. Step 6, Tradeoff Analysis, refers to the activity of

altering features of the system and repeating Steps 1 through 5 to reduce or

eliminate unreasonable demands.

Within each step there are several judgments required that are basically

satisfied by using experience from the new system designers or a group of

outside experts (6). For example, in Step 1, the evaluator(s) is required to

interpret documents and use personal judgment in establishing the functional

requirements for the new system and in selecting similar predecessor

components that most closely meet these requirements. In Step 2, generic

kinds of tasks must be defined and assigned to the various echelons of

maintenance (i.e., organizational level in the field and intermediate level

removed from the field). Step 2 also requires the evaluator(s) to make

educated guesses as to the split between inherent versus induced component

failures, design differences, selection and assignment of skills for the

various tasks, and probable system and component usage rates. In Step 3, the

analyzer must decide which kinds of predecessor training resources are

applicable to the proposed system and, where no training scenario exists,

generate new training resource requirements. Steps 4 and 5 involve judgments

about the movement and availability of people both within various skills and

the Army as a whole and also require the development of options to offset high

demands. Step 6 does not require any major judgments other than a selection

of different operating, reliability, or manning scenarios that represent a

modified system designed to offset high manpower demands or costs.

In reviewing the Army MPT problem outlined earlier in this section, it

appears that a technique such as the HCM could certainly contribute to

resolving most of the man-machine mismatches with new acquisitions.

Ultimately, it is the Army's primary intent to use the results of such an

approach to influence the ASARC, DSARC, and system program management

communities to be more supportive of the MPT portion of system-integrated

logistics support. At a secondary level, it is also the Army's desire to

examine techniques such as the HCM with the intent of determining the

feasibility of enhancing and managing its own MPT system. For both the ASARC

13 - .. . . . .. . . .
00 6i0

to-i

000

'-4

0

0

z Li z U)

U30. WC ce.J 4c

L3 0040 - 0000 I-a o 0i 

U3 L2 U343 w 061-4

4I

Ia.

C4 z PC r

Jz3.

-4 0000

-- U4

14
, "

"-* decision and internal MPT management objectives it is important that the Army .

*: establish a credible MPT projection scheme. It then follows that any

anticipated MPT projection techniques be validated before being approved for

Army-wide use. The following section describes the structure for the .HCM

validity evaluation and sets the foundation for the final results provided

later in the report.

| -

-

,,,)..

- ~:-i.- 
SECTION III

STRUCTURE OF HARDMAN ANALYSIS

A. OVERVIEW

The previous section described how important it is for the Army to

develop a credible, comprehensive MPT system. There are several different

kinds of validity tests to consider when establishing the true credibility of

.* a modeling technique. For example, some kinds of mathematical models may obey &

other already proven algorithmic or empirical relationships. Validating

techniques using this method is rather simple. However, the problem becomes

more difficult when a proposed simulation is projecting the future based on

experiential judgment and scant, or non-existent, historical evidence. As

shown in Section II, the proposed Army MPT methodology not only fits this

description but is further complicated by the need to draw on a considerable

amount of data and data sources. In situations such as this there are several

different kinds of validity tests that can be employed to test "first-time"

models. In this section the various kinds of validity tests are first

described, followed by a detailed discussion of the structure of each portion

of the evaluation in the context of the validity tests.

B. VALIDITY TESTING

The two important outcomes of testing the validity of a simulation or

projection are the degree of credibility of the model and results, and the

respective usefulness of the results. The credibility of a projection

revolves around several operations research-type measures. Two .met

very important validity measures are as follows:

(1) Variable parameter validity - a measure of how well the simulation

parameters compare with their respective counterparts in the real,

or observable, world; and whether the parameters, when varied over

an expected range, effect the results in the direction of known

historical data.

(2) Hypothesis validity - a measure of how well various relationships

within the model and its outcomes correspond with the real world.

Both of these validity measures fit under generic tests of "model

accuracy." As stated earlier in Section I-B, the detailed accuracy

examination has been postponed to a later phase.

Within the constraints of this study, the following validity tests were

exercised:

(1) Face validity - a general indication of whether the model is

logical, credible, and the results reasonably based on the inputs.

p I""''I

%' .- " £ -" -. " .-..... %-%" .. .." ..- .. -.. ".. -.-

., .'. . i ....... ..-o." ...-. . -.". .'. .... .-.. ..... . .. .,.. ", . .. . .. . .'.. .. 

. .-... "...'.-... ..... ... * . . .. -.. ,-.... ... . . .. i'i
This indication can be obtained by talking to individuals i .

experienced with the same, or similar, models; comparing the

structure of the model to other established simulations; or

tracing and evaluating test case results for reasonableness.

(2) ntenalrelabiity- a measure of the variance in model outputs

when replicated by a group of independent assessors given the same inputs.

(3) Event series validity - It is understood that inherent within the

design of any model is the inability to duplicate exactly the real

world. Event series validity is a measure of how accurately the

simulation must reproduce real-world events. This test is

extremely important because it relates to both the credibility and

utility of the results as viewed by the people using the outputs

for their decision process (i.e., the ASARC and Program Managers).

In the following paragraphs, the various analytic procedures employed in

evaluating the Army MPT methodology are discussed in terms of the preceding - -

validity tests.

C. USER REQUIREMENTS ANALYSIS

One of the more obvious tests of face validity is a straightforward,

initial comparison of the user needs against the outputs of the model. By

querying knowledgeable individuals in the Army MPT arena for problems, and

their respective needs to resolve those problems, it was possible to evaluate

whether the proposed methodology could illuminate the key personnel issues.

The effort designed to establish the relevance of the HCM MPT system to the

Army's needs was fourfold:

(1) Establish a list of experienced, interested individuals from the

Army MPT community and attempt to draw on individuals from as many

participating commands as possible to establish a broad user base.

(2) Design a comprehensive set of questions to probe both major MPT

problem areas and needed solutions and/or tools to resolve

personnel issues.

(3) Obtain answers to these questions via personal interviews with

each individual.

(4) Compare the results against the applicable portions of the

methodology to establish whether RARDMAN is reasonably responsive

to the stated issues. 4

Each step in the above user analysis, along with both the user requirements and relevance results, is discussed in greater detail in Section IV.

4 =

%°°
D. OPERATIONAL ANALYSIS

As a continuation of the face-validity evaluation, it is also useful to

examine applications of the methodology as a means of establishing credibility.

In this study the examination encompasses both actual test cases and comparison i_

with other accepted or proven MPT projection techniques. The structure and

results of both of the aforementioned examinations are provided in greater

detail later in Section V. However, as a basis for the outcomes provided in

Section V, it is important to note that the results afforded insight into

several credibility issues. First, the results provided information on how '- rational the model was. For example, the test-case audits were established to

examine the consistency and reasonableness of the results relative to the

inputs. Similarly, by comparing the technique against other accepted models,

it was possible to see whether the basic assumptions and logic structure

appeared sound. Second, the audit results provided useful information on the

complexity, and subsequent utility, of the model. Third, the audits provided

an understanding of whether the model logic and interpretation of the

projections were perhaps ambiguous or masked, due to model over-complexity.

Fourth, the audits indicated the presence and relative occurrence of error

sources in the methodology. Last, the use of both test cases and comparisons

with other MPT models highlighted potential problems in input data quality and

availability.

E. RELIABILITY ANALYSIS

An examination of the "reliability" or accuracy of the methodology and

its projections was the original intent of this portion of the evaluation.

Although it was possible to retain part of the original reliability analysis .-. scope, time and funding constraints required the actual accuracy effort to be

C." postponed until the next evaluation phase. The portions retained for this

part of the evaluation address the issues of internal reliability, a pre- IN

cursory accuracy examination using the experience of individuals who have

already applied the HCM, or a similar methodology, and event-series validity.

For the internal reliability test, it was decided to use one of the Army

test cases (i.e., SINCGARS) and design a controlled experiment to establish

whether a group of individuals, initially trained to use the methodology,

could both replicate the original answers and demonstrate a low variance in C answers among themselves. A group of seven individuals (four civilians and

three career Army personnel) of varying backgrounds were selected for the

experiment. Given the small sample size (primarily due to restrictions in

available people, time, and budget) it was decided to dispense with detailed

statistical comparisons and simply establish general conclusions about the

repeatability and consistency of the results. The replication was also used

to develop a list of critical factors that have major impacts on the MPT

projections.

To round out the accuracy examination within the constraints of the

study, it was also decided to perform a qualitative accuracy check, using past

Air Force and Navy experience with the same type of methodology.

18 . "

. . . . . . . . . . . .. ,

. .* . . . * . *. * ~ .. ___...__...__-- .. -'-_--'-,_-_,'_ __ -.. -,__ _- __-.-.-.-_..' .__ .- .-... .. '. -. '-'.'.'? " " - .-. i'- . .. :. -. . - .
-. - .. ... - .• *- . . ., J.- Zw& " 

V I- . - r r - _..w_.. x . .,iw-* m.r..r-r r.. C. o -. • .. ,.

As stated earlier, the event-series validity test is extremely important

for determining the required accuracy of a model and, subsequently, the

utility of a new simulation. To obtain an estimate for the required accuracy

of the HCM, a simple top-down cost sensitivity analysis was employed to

establish the allowable variance in the manpower projection, as constrained by

other life-cycle cost elements and an allowable error band around the total

life-cycle cost projection. This estimate of the allowable variance in the -. *

manpower projection was performed for an example acquisition, using actual

Army weapon system data (i.e., the SINCGARS system). Once the level of

accuracy was established, it was possible to formulate corrective actions (for

the various problem areas revealed through the face-validity and internal

reliability investigations) commensurate with a greater or lesser need for

accuracy. It was for this reason that the sensitivity test was reserved for

the end of the evaluation. The complete details of the internal validity and

accuracy studies are provided in Section VI.

. . . .

m . . . . . . . :.. 
-- ILA

SECTION IV

USER REQUIREMENTS ANALYSIS I.. I..

I

A. OVERVIEW

As a part of the approach developed to determine the face validity of

the HCM MPT projection model, an MPT user requirements study was conducted.

This analysis included a definition and documentation of Army user requirements for a manpower, personnel, and training (MPT) prediction model. Sub- f. *

sequently, the user requirements were compared with the capabilities of the

HCM to determine the degree to which the technique conformed with the needs of

intermediate as well as end users of MPT information. The users included

those who currently develop the MPT information as well as those who primarily

use the information (Table 4-1). The details of the requirements analysis are

provided in the following paragraphs. :-

B. APPROACH

This study focused on exploring Army user requirements for a "generic" 2

MPT methodology that would serve as comparison against an existing MPT projection technique for the Army's Integrated Logistic Support (ILS) process.

The user requirements provided a useful measure for determining the extent to

which the HARDMAN model fulfilled the Army's stated requirements. As stated

earlier in Section III-D and III-E, detailed audits and a test case replica- tion were also conducted to assess face validity and internal reliability.

The additional familiarization with the HCM technique achieved through these

parallel analyses augmented the comparison of the user requirements with the

methodology.

The basic approach, outlined in the preceding discussion, is explained

in greater detail in the following paragraphs:

I. Selection of Individuals

Determination The Methodologies Interim Report (MANPERS) for Manpower and Personnel Requirements states that "the three principal

organizations responsible for the research, development, and deployment policy

of a system under development are DARCOM - the materiel developer; TRADOC -

the training and doctrine (i.e., combat) developer; and HQDA - the force

modernization planner. With the exception of the "industry" user, all of the

users interviewed in this study were from these three organizations in the

Army. These commands were also confirmed by the sponsoring agencies (Army

Research Institute (ARI) and the Soldier Support Center (SSC)), as key actors

in the Army's MPT process.

A predominance of users were from the TRADOC community. This is a

reflection of the fact that "front end analysis," such as MPT projections,

done prier to Phase I in the systems acquisition process, is largely a TRADOC-"

responsibility. TRADOC has the lead in ILS management prior to the appointment of a DARCOM Project Manager at Milestone I.

20
.- ." - - -

Table 4-1. User Groups/Representatives Interviewed

Number .of

Users Interviews .

ILS Managers (DARCOM) 2

Soldier Support Center/SSC-NCR (TRADOC) 3

Logistics Center/LOGCEN (TRADOC) I

Deputy Chief of Staff, Combat Development/DCSCD (TRADOC) 1

Deputy Chief of Staff, Training/DCST (TRADOC) 2

Special Study Group-TRADOC Systems MGR/SSG-TSM (TRADOC) I

Logistics Evaluation Agency-DCSLOG/LEA 1

DCSPER 1

Industry (based on results of survey) I

Total User Groups: 9 Total Respondentsa: 13

aUsers listed were selected on the basis of their role/function in the MPT

process. All users contacted participated.

Inputs from the SSC-NCR-II (TRADOC) and industry users were not gathered

from actual interviews. The information needed from these users was not

obtainable within the allocated time frame for the study and was, therefore,

extracted from documents that adequately portrayed their respective user needs

for an MPT projection model. One document was a memorandum about requirements -

for an MPT model (9) and the other was an industry survey conducted by the

Soldier Support Center to determine industry requirements for MPT information

in developing proposals (10).

2. Questionnaire Design

The questionnaire was designed as a discussion guide to identify

user requirements. Some questions addressed user requirements directly by

identifying the specific MPT information input they require to fulfill their

responsibilities for the Army MPT process. Other requirements directly

addressed were imbedded in questions concerning the user's interpretations of

the timing and interfaces required by the process. Through interface and

timing questions, an understanding was developed about when critical MPT

information is needed and about the relationships between principal organizational players, so that it could be determined if the design or structuring of

an KPT projection system would be sensitive to these relationships. Another

direct way of focusing on user requirements was to ask respondents for -

21
suggestions about the design of an MPT methodology. Suggestions included

comments on outputs, schedule, and the process of actually projecting personnel needs. Questions indirectly addressing user needs provided confirmation, context, or further elaboration of the direct user data. These questions

referred to the following contextual information:

(1) The user's understanding of the present technique used for

MPT projections.

(2) Users seen as responsible for MPT projections and the nature

of their "use" of the information (intermediate or end use).

(3) MPT regulations/documents that impact the user.

(4) Users' perceptions of the strengths or weaknesses of the

current MPT process.

(5) The primary charter and responsibilities of the user.

(6) The key drivers underlying the need for MPT projections,

according to each user interviewed.

The complete questionnaire is shown in Appendix A.

3. Interview Procedure

User requirements were determined primarily through interviews

with the representative users who were identified and recommended by the

sponsor. Initial contacts made by ARI provided JPL with direct access to

appropriate individuals with whom formal follow-up requests for assistance

were made and visits scheduled.

The study received a substantial level of user cooperation in conducting

the interviews; an example was that users frequently called upon people within

their organizations having specialized knowledge of that organization's

involvement in the MPT process. Each scheduled interview was treated as a

single-user interview even if more than one person was present for the

discussion. A single-interview form consolidated the responses of all present

from that organization.

C. TABULATION OF INTERVIEWS

1. General Responses

Most of the users interviewed thought that MPT projections should

be available at the front end of the acquisition cycle. This was frequently

linked to supportability and affordability drivers for needing MPT information,

particularly because these affect the user organization's roles and responsibilities for MPT projections. One user cited sustainability as a key driver

of MPT information. This was considered an important insight because it

highlights life-cycle aspects of system operation. The predominant responses

pertaining to these three major drivers are summarized in Table 4-2.

,, .~- -A.-

. . - . . . - - . .
Table 4-2. Key MPT Drivers and Associated User Rationale

Key MPT Drivers Predominant Rationale

Affordability MPT system must provide all MPT cost elements associated

with acquisitions

MPT system must reflect cost tradeoffs with force

structure/mission requirements

Supportability MPT system must support readiness goals

MPT system must fully support new acquisitions

(operations, maintenance, supply, training, etc.)

MPT system must demonstrate sensitivity to horizontal

MPT impacts (Integrated System Support concept)

Sustainability MPT system must support manning/training system(s)

plans over long periods of time

MPT system must consider extended impacts of mission/

scenario changes with time

Most users expressed the idea that system design should be influenced by

various manpower constraints emerging from early MPT analysis before the

design is fixed. A related point is that most users thought that the primary

tool for getting MPT projections into the system through the Quantitative and

Qualitative Personnel Requirements Information (QQPRI), which is developed

concurrently with the Basis of Issue Plan (BOIP), was inadequate and/or

untimely. By the time QQPRI data is available (after Milestone I), most of a

system's design is fixed (Figure 4-1). Another problem mentioned was that

DARCOM and TRADOC's shared responsibility for developing the QQPRI and BOIP

requires additional clarification of lines of responsibility. This would be

particularly important if the initial development of MPT projections shifts to

an earlier point in the acquisition cycle (i.e., TRADOC would become

responsible for the initial, therefore tentative, QQPRI).

Occasionally, a user expressed a viewpoint that MPT projections made

later in the acquisition cycle were adequate, providing that sufficient lead

time was given to acquire and train the personnel. This statement assumed

that funding was already available for the personnel.

. . .o.
100

V1,11YEND OF FULL-SCALE

DEVELOPMENT (MILESTONE 1I1

_857 BY END OF DEMONSTRATION AND

VALIDATIIA (MILESTONE 11)

-BY END OF CONCEPT 1EXPLORATION (MILESTONE 1)

504

0

year SYTE AQISTO LIECCE( LSON )

SOURCE:~~~~~ Poednsf mNa ia SeuiyIdsr lAscatn

Symposium~ onNv Iytm custoOtbr2-8 97

Figure~~~~~ 41 Efeto Dsi Deiin o Lf-yl SstmC t
*".'

2. User Group Responses

User responses are reviewed in this section. Responses from

multiple users in DARCOM and SSC-NCR are incorporated under single headings

* for these respective organizations. Individual responses, by organization,

are detailed in Appendix B. The discussion on user responses includes a '

summary of the user organization background to establish the appropriate

context for the responses that follow.

a. DARCOM/Integrated Logistic Support (ILS) Managers Responses.

ILS managers in DARCOM are responsible for ensuring the planning and monitoring

of logistic support elements for systems under development from Milestone I

throughout the development process. The prime analytical tool used in ILS is

Logistic Support Analysis LSA).

The LSA process consists of analysis efforts to completely define the --.

support system and its interfaces with a materiel system. The analysis

incorporates manpower/logistics analysis and tradeoff analyses associated with

maintenance concepts (repair levels, etc.). LSA Record information (LSAR) ...

includes all tasks required to operate and maintain a materiel system. LSA

data then becomes feeder data for the Quantitative and Qualitative Personnel

Requirements Information (QQPRI) system. As the chief means of developing

both the rumbers and kinds of personnel required for operating and maintaining

systems, QQPRI development is a shared responsibility of DARCOM and TRADOC.

As a result, LSA data needs to be as accurate as possible for both the DARCOM

and the TRADOC user communities.

LSA normally includes operator/maintainer task and supply functions for

repair and test equipment as well as the system under development. In determin-

. ing system supportability, ILS managers required information about the exact

tasks to operate and maintain systems. They also needed projections of reliable manloading requirements to help establish expected Military Occupational

Specialty (MOS) and skill demands, plus any MOS changes anticipated.

ILS managers stated a need for methods to analyze tradeoffs between

force structure, personnel constraints, hardware performance, and cost. K Essential to this analysis was information (output) about support and test

equipment ("playing field") associated with a system. The total QQPRI

associated with an expected density of these systems, when deployed, was cited

as an important element of this analysis.

Early maintenance concept information was also requested, including

maintenance manhours (MMH), reliability, availability and maintainability

(RAM) data, and maintenance-level information.

DARCOM suggestions for the design of an MPT model included providing/

displaying MPT data for alternative (proposed), as well as predecessor systems

to provide support documentation for decision-making. An MPT methodology

would need to provide credible quantitative data to the ILS process, the

source(s) of which could be documented or traced. Formats for the data should

be designed to present data efficiently to users who have limited time for

reviewing information (e.g., graphs, charts, etc). Timing for initiating the

process should be prior to Milestone I to achieve maximum benefit from

,. .. *.. ... *. -. .'.',............-. .. .. *. * ... * *> i.... .... ... . . .. _ . . ..- ,.. . , . - . ___"________ '________ "-________________.'.,_.....____._____._ ".."..- .'.- .''. . •- .' ..'. - ' 'r
hardware/manpower tradeoffs; however, there should also be follow-up on

existing systems in the field to verify LSAR data and system supportability ".

in format ion. i. F b. SSC-NCR (TRADOC) Responses. SSC-NCR is responsible for

assessing the feasibility of allocating personnel and materiel to specified

organizations within the Army (i.e., the Tables of Organization and Equipment

(TOE). SSC also ensures that the user communities identify both demand and

supply MPT factors at the front end of the system acquisition process to

address supportability and affordability issues.

SSC-NCR stated that detailed data for the following general categories

of information is required:

(1) Manpower.

(2) Comprehensive task breakdown.

(3) Personnel.

(4) Training.

(5) Logistic support.

Wi (6) Integrated System Support (ISS).Graphic display capability was highlightd as useful to SSC for inclusion

in reports comparing demand and supply. In addition, detailed MPT analyses

comparing proposed to predecessor systems is required.

SSC also needed MPT projections prior to Milestone I to support the Army

Systems Acquisition Review Council (ASARC) decision process. An additional need cited was timely and useful cost information for the Army's funding

process. In providing such information, the projection process should also

expose any assumptions employed so that an understanding of the limitations of

the analysis is clear.

SSC-NCR further suggested using a solid, historical data base containing ..

Army personnel statistical data, comprehensive task-related files, and RAM

data associated with systems. It was also indicated that the MPT system

shovId be compatible with existing Army reporting structures and command roles

to ensure that the model could be integrated into the Army's normal systems

aquisition process.

c. Training Developers (TRADOC) Responses. A key concern of

training developers within TRADOC Headquarters is the integration of the

Army's training function with the other critical Army functions - manning,

force structure, facilities, equipping, sustaining, mobilizing, and

deploying. Matching the Army training system to manning and the systems

acquisition process is crucial to the support of force modernization.

Supportability of force modernization requires specific information about the

availability of manpower qualified for training, as well as sustainability

(personnel flow and replacement) of the force required.

21)

.. .. .-
Another driver of training developers' needs for MPT information was

system affordability, i.e., an understanding of a system's life-cycle cost.

The general categories of information needed by training developers to

determine supportability and affordability were (1) manpower and personnel

information (demand/supply); (2) training information (exact operator/

maintainer task and target audiences); (3) equipment information (capability

and man/machine tradeoffs); and (4) accurate maintenance concept information.

d. Combat Developers/Soldier Development Directorate (SDD) and

TRADOC) Responses. Combat developers in TRADOC/Soldier Development

Directorate are concerned with personnel doctrine that addresses air/land

battle mission needs. They also promote combat effectiveness through MOS and

force structure change and ensure that soldiers' physical and mental capabilities are taken into account by the various elements of force modernization. In their interface between SSC and TRADOC Headquarters, combat

developers at SDD are involved in the process of optimization of resources

between proponent schools.

The combat developer's key drivers for needing MPT information were

supportability, in a readiness sense; performance, as it relates to combat

effectiveness (e.g., through man/machine design issues); and cost (e.g., as might be determined by impact analysis).

Combat developers' design suggestions focused on the need for an

interactive management information system having a time-line signaling

mechanism that could alert participants to needed (or missing) actions in the

current MPT process. Suggested outputs from the MPT model were a projection

of the numbers and kinds of operators and maintainers associated with a

system. A timely breakdown of task and training data was also cited to

influence system design (prior to Milestone I).

It was also suggested that some information concerning the impacts of

new systems on the existing personnel structure and resulting tradeoffs would

be useful in optimizing the selection of new systems. This requirement also

implies a need to apply MPT analysis to systems on a horizontal plane (i.e.,

to consider the drain on the projected manpower pool caused by other developing systems).

1.

e. Tradoc System Managers (TSM) Responses. During the

gestation period of a system prior to Milestone I, TRADOC assumes the lead in

conducting mission area studies. A TRADOC Systems Manager (TSM) is concerned

with the identification, analysis, and selection of system concepts that meet

needs of combat elements of the Justification of System Major New Starts

(JSMNS) document. Other responsibilities of the TSM include preparing the

Operational and Organization (O&O) plan, as well as monitoring the preparation

of a number of different documents required for logistic support. Among these .**

are (1) the Integrated Logistic Support Plan (ILSP); (2) the Concept Formula- ..

tion Package (CFP); (3) the Outline of Individual and Collective Training Plan

(OICTP); (4) the Letter of Agreement (LOA); (5) the Cost and Operational Effective Analysis (COEA); and (6) the Preliminary Qualitative Assessment (PQA). The

-2-7-

............................................-- ' '. '-,'''' .'. ".'. ,. *..''' .- '-- .--. '._ ' "''--.
Preliminary Quantitative and Qualitative Personnel Requirements Information LA:

*" (PQQPRI) could, by interpretation of the current regulation, be initiated by

the TSM although it is not normally done at this time. -_

The TSMs had a clear requirement for a model that would contain a comprehensive task file data base with task data taken from a number of taskcentered Army manuals and documents. This would assist the TSMs in ensuring

the timely completion of these documents by facilitating the identification of e.

specific tasks, MOS, and skills associated with hardware. Finally, the TSMs

suggested the development of both a manpower and personnel (demand/supply)

[* data base that would permit tradeoff analysis to determine the best fit of

* people with equipment.

f. LOGCEN/LOGISTICS CENTER (TRADOC) Responses. Force

developers, training, and materiel systems managers are all MPT actors within

LOGCEN. The critical role of LOGCEN is as an executive agent for the ILS

process. Issues such as man/machine interface, system and non-system training

devices, logistic system proponency, and Manpower Authorization Criteria

(MACRIT) are included in LOGCEN's charter. Some of the LOGCEN responsibilities are (1) impact analysis of system/design characteristics; (2) support

equipment training plan and technical publications development; and (3) MACRIT

system improvement. As a TRADOC integrating center, LOGCEN is similar to a

proponent school in its proponency role for selected logistic systems. LOGCEN 7-7

also develops some TOEs for maintenance units.

LOGCEN indicated they had specific information requirements related to

system considerations for affordability and supportability, including

(I) accurate RAM data (based on historical data and accounts for realistic

utilization rates, actual failure rates, and combat damage); (2) workload data

(as a function of direct productive and indirect productive annual maintenance

manhours (AMMH), plus available productive manhours); (3) density of equipment

(TOE-related); (4) unit-focused manpower requirements; and (5) LSAR-format

task data associated with MOS and equipment needed for training analysis.

LOGCEN's MPT timing requirements were twofold: (1) LOGCEN needed early

(front-end) training device information to prepare for both early system

testing and the final system design configuration, as well as (2) solid

manpower requirements for supporting accurate MOS and organizational design

decisions.

Design discussions about MPT projection models focused on LOGCEN's

requirement for accurate and reliable (RAM) data. The Annual Maintenance

Manhour (AMMH) data that LOGCEN presently receives should be made separate and

distinct from reliability data. MPT information was needed in an LSAR data

format and would be more useful if broken down by component, task, and skill

level. It was felt that this could then be used with an MPT simulation to

determine impacts of utilization rate, combat damage, and/or hypothetical _._

changes in RAM characteristics.

A concern was expressed about the accuracy of data to be used for MPT

analysis. The availability of "hard" LSAR data is critical to the development

of useful MPT information. Given an absence of system fielding experience and

28
correspondingly mature LSAR data (such as would be the case when substantial

technological changes in equipment are introduced), it was recognized that it

would be difficult to obtain reliable data. However, it was also recognized

that, if there is minimal technological change, the MPT information should be

readily available.

g. Devices and Systems Training Directorate (TRADOC) Responses.

This organization is involved in policy development, training-device proponency, and training-plan approval.

In general, this directorate indicated needs for both manpower and

- training information for the purpose of developing the individual and

collective training plans (ICTP) of systems under development. A new Army

"* thrust called Integrated System Support CISS) now requires the generation of

MOS information that is system- as well as unit-specific. It was also

suggested that an improvement to the current MPT process would be to provide

sufficient front-end task manloading and training information to allow the

design to be influenced early in the acquisition process. One final expressed

requirement was to have an MPT projection model that more effectively flags

* MOS "overload" (i.e., a model that detects and documents additional needed

manpower, both numbers and skills).

h. Logistics Evaluation Agency - LEA (DCSLOG) Responses. As a

field-operating agency of the DA staff, LEA is the independent logistician in

the materiel acquisition process responsible for monitoring ILS requirements

for systems under development. Supportability of the more than 800 systems to

be evaluated was cited as a key driver for needing MPT information, along with

other factors such as degree of interoperability and performance levels.

LEA users indicated a need to be able to use MPT projections (before

Milestone I) as constraints during system development. These MPT projections

should include RAM goals, new skills and training device concepts. Following

Milestone I, it was suggested that readiness considerations include a 3- to

5-year lead time to acquire and prepare logistic support personnel prior to

system fielding. LEA information requirements also included a complete and

accurate listing of tasks and skills required to maintain the system

(including all subsystems). Finally, there was an expressed requirement to

look across systems (horizontal analysis) to assess overall impact on logistic

support and training.

i. Force Modernization Cell - Manpower Programs and Budget

(DCSPER) Responses. This organization provides staff support to DCSPER for

the Army Systems Acquisition Review Council (ASARC). They are chiefly

concerned with resource management for force modernization and bridging demand

and supply issues in both manning and force-structuring. DCSPER is concerned

with matching the number and kinds of "spaces" needed to operate and maintain

systems with the required qualified personnel (i.e., "faces"). In this

respect, they are also concerned with the personnel training. Because of

congressionally-mandated "end strengths" for the Army, information concerning L..

distribution of personnel is important. Recognizing that affordability and

29~
* Z W W-7

user supportability concerning the supply of personnel is a realistic

constraint, DCSPER therefore suggested that a horizontal look across systems

would permit analysis of the impacts of shifts in the numbers and types of

manpower required. Tradeoff analysis at a high level was also cited as useful

for conducting Army-wide optimization studies (e.g., hardware vs. manpower vs. I

system performance).

Finally, it was indicated that the design and output of an MPT model

should be compatible with existing reporting processes (QQPRI, BOIP, ASARC

Milestones, etc.). In line with this requirement, it was also suggested that

there is a need for a more effective (interactive) information system, with

appropriate access to relevant data bases of HPT information.

j. Industry Responses. Three industrial associations

(Aerospace Industries Association, Electronic Industries Association, and

National Security Industrial Association) were surveyed about their needs for

logistic support (MPT) information in responding to Requests for Proposals

(RFPs). It was discovered that industry is very interested in improving its

understanding of logistic support and human factors concepts associated with

systems under development. Some suggested means to accomplish this were (1)

earlier user contact, (2) user involvement in RFP preparation, (3) better

implementation of human factors analysis, and (4) improved information

dissemination to industry. Two basic concerns of industry have been to

address human factors before the system design is fixed and to specify

contract arrangements for implementing MPT analysis. -

Industry also indicated a need for reliable information concerning the

interded/planned concept of system operation and supporting organizational

design. The participants also suggested DOD strengthen its estimates of the

kinds of personnel expected to operate and maintain the system. Scenarios

depicting both threat, environment, and realistic usage rates were cited as

important in preparing RFPs. The expected performance of a system, plus

maintenance concept, were two other critical data points contributing to

industry's ability to conduct or contribute to early MPT analysis.

The industry-users interviewed also indicated a need for MPT information

to be centrally located for accessibility. Charts and graphs were suggested

as being especially helpful in transmitting to industry the necessary

information. An historical data base was also suggested to permit comparisons

with other developed systems or analysis of system designs that were rejected.

2. Tabulation of Individual Interviews into Matrix Format

After all user interviews were completed, the raw discussion notes

were assembled, reviewed, and transcribed onto interview forms for clarity.

Key words and ideas were highlighted. Two requirements analysts then

separately analyzed the information and jointly tabulated the responses to

facilitate analysis of the results. Once the answers were transformed into

rough requirement statements, they were entered into a matrix format to permit

a clear indication of which users were associated with each user need. The

detailed matrix of users and user answers is shown in Appendix B. -

....................... . . ......... . ' ' .-...
D. TRANSLATION TO USER REQUIREMENTS

1. Procedure

The user needs were divided into two categories - one category

containing "output" (data) needs, and the other containing "design" needs for

an MPT projection system.

User requirement statements were generated from the Appendix B matrix

information and the interview forms. When appropriate, similar ideas or

* requirements expressed by different users were translated into a comprehensive

user requirement statement.

The separate requirement statements developed by both analysts were then

compared to ensure agreement and completeness. To establish priorities among

the requirements, criteria were developed to evaluate the requirements in the

matrices. A final user-requirements list was developed after applying these

criteria.

2. Criteria for Assigning Priorities

The following criteria were considered essential to determine

" which requirements were necessary or desirable for an MPT projection model:

(1) The requirement's role in addressing key drivers associated

with MPT projections for Army systems (i.e., affordability,

supportability, and sustainability).

(2) The importance of the requirement in providing useful and

timely MPT information to the Army system acquisition

process.

(3) The breadth of users stating the requirement.

3. Requirements List

As stated earlier, the user needs were categorized by "output" and

"design" needs and then prioritized according to the degree of fit with the

three criteria stated previously. Once this procedure was completed, the user

needs were grouped into mandatory and desirable requirements. Below is the

final list of user requirements (including output and design requirements) for

a projection system that conforms to the needs of the key Army players in the

MPT arena. For the purpose of this evaluation, all the following mandatory .. F requirements were given equal weight because, ultimately, they were primarily

used to test the "reasonableness" of the methodology:

(1) Mandatory Output Requirements:

r (a) Operator/Maintainer task data should be provided as . '

required for a system and its support equipment

(including task description and frequencies), based on

a replaced or comparable system.

"ii'

- . . * ... v."* *-*.
(b) Maintenance manhour projections must be based on a

specified maintenance concept (levels of maintenance,

specific RAM data).

(c) Operation and maintenance manpower, personnel, and

training (MPT) projections must reflect realistic

system usage rates as determined by acquisitionspecific mission/scenario drivers.

(d) Reasonably accurate (i.e., acceptable to the

budget-control community) total manpower demand

(number, MOS, skill level, additional skill

indicators) is required before Milestone I. "'

(e) The MPT model must include and document projected

availability of personnel for the system (demographic

data such as number, age, sex, size, skills, k,,

education, MOS, term of service).

(f) The MPT model should document MOS alignment data

showing current status; authorizations vs.

requirements to indicate MOS overloads.

(g) The model must incorporate, document, and use target

audience description (TAD). This includes reading

ability, grade levels, experience, skills, and

previous training of available personnel to determine

accurately testing/training requirements (the TAD

would be input to the Letter of Agreement).

.h) The "T" of MPT must include the formulation, description and use of comprehensive training concept

elements: (1) classes or training required (including

any increases) for unit or institutional training;

(2) time to train; (3) training frequency; and

(4) training support (devices, facilities,

instructors).

(i) There must be documentation of MPT impacts and costs on the Army as a whole.

(j) Trade-offs must be conducted between hardware and

people to determine the optimal balance for best

system performance, costs (LCC), and support

requirements (Decision/Tradeoff matrix needed).

(k) Output should include reports with well-labeled,

easy-to-read graphic formats such as graphs, tables,

matrices, charts, as well as an executive summary of

the report that could stand alone.

(1) Outputs should be in both hardcopy as well as on CRT

display for permanent archiving.
(m) Reporting procedure must include any pertinent caveats

about the data source availability/reliability or

assumptions about the system. This information should

be provided at major decision points or steps .in the ___,

analysis. Appropriate actions to resolve identified 6

weaknesses in the analysis should also be noted.

(2) Desirable Output Requirements:

(a) A complete historical MPT data base is highly desirable. MPT projections should be updated throughout

the life of the system to increase accuracy of LSA/

other data.

(b) MPT impacts should account for geographical constraints

on manpower pools "available" to run the system.

(c) MPT system output should be compatible with accepted

LSA reporting format.

(3) Mandatory Design Requirements:

(a) Projected MPT impacts/costs must be available at the

front end of the acquisition cycle (before Milestone

I) for early design influence and early MPT (cost)

warnings.

(b) Front-end determination of a system's candidacy for

MPT analysis should include criteria such as the

potential for high payoff (e.g., selection should be

based on type and amount of change in the technology

that drive significant MPT costs/changes).

(c) To facilitate training analysis and to support

proponency for MOS changes or additions, the system

should have the capability to compare the proposed

operator and maintainer tasks associated with a new

system with a comprehensive, automated data base of

tasks detailed in the Soldier's Manual, ARTEP, POI,

SQT, T.G. and AR 611-201 (Enlisted Career MOS

Management Fields).

(d) Output should include comparable MPT projection data

for both alternative/proposed and existing systems as

support for decision milestones.

(e) The model should incorporate horizontal analysis

across systems to determine total MPT and logistic

impacts and to facilitate Army-wide equipment and

personnel optimization, as well as support

end-strength decisions.

(f) The MPT model should provide compatability with

existing DOD documentation and reporting structures.

It should, however, evolve into an interactive

- - V .-
information system with remote terminals, including a

time line oriented to both the budget and systems

acquisition processes. Signalling mechanisms are

needed for late/missing/incomplete data.

(g) The MPT system must identify and provide resolutions

to both weaknesses in data inputs and inaccuracies

resulting from the use of the data.

(h) There must be a documented and accessible audit trail.

i) A universal information system should be designed to

allow multi-user access and update.

(4) Desirable Design Requirements:

A more detailed O&O plan is needed to act as a "hingepin" to 6 4

initiate MPT analysis at the front end of the acquisition

cycle.

E. RELEVANCE OF USER REQUIREMENTS TO EXISTING HARDMAN DESIGN

Having stated the user requrements in the preceding section, it is now

possible to determine the extent to which the HCM has application to the

requirements for the Army's existing (or near-term) MPT process. Through a comparison of the HCM to user requirements, it was possible to establish one

measure of the face validity of HARDMAN (i.e., how well the HCM fits the

Army's MPT needs). This was achieved by checking the relative applicability

of each step in the HCM to specific portions of the requirements. As stated

earlier, both the ARI replication exercise and the HARDMAN applications audit

were conducted in parallel with the requirements analysis. These parallel

studies afforded an in-depth understanding of each step in the methodology.

The additional familiarization with the complete array of data inputs,

algorithms, methodology logic process, and outputs, provided the essential

background to perform a comparison. Table 4-3 presents the results of the

compatibility analysis by indicating which portions of the HCM have high,

medium, or low applicability to the user requirements. The degree of

applicability provided an indication of the utility of various portions of the

technique to the Army's MPT users.

Out of the thirteen total mandatory output requirements, the HCM

demonstrated good compliance in essentially six of the user need areas. Of

the seven requirements where elements of the methodology appeared to exhibit

low applicability, only five of the user areas were considered of immediate

importance to developing a sound MPT projection scheme. These key user areas

were (1) establishing the accuracy of the MPT methodology; (2) including other

demographic data such as age, sex, size, etc., in personnel allocations;

(3) considering the demands of possible future acquisitions on MOS, (4) performing a complete life-cycle cost analysis as part of the tradeoff studies;

and (5) indicating weaknesses in the analysis and respective resolution

actions. Items (2) and (3) appear to be somewhat of an Army-wide constraint

resulting from the general lack of readily available information. Therefore,

only requirements (1), (4), and (5) seem to be achievable, close-range

deficiencies that should be incorporated as improvements to the HCM.

.4,
* * - -. ~- oW . r r. *. -. - .- *.-. -. -r ;6. -~r . .

Table 4-3. Comparison of User Requirements to the

HARDMAN Comparability Methodology

Applicability of HCM Technique

User Requirements to User Requirements

High Medium Low -

* Manadatory Output Requirements

Operator/maintainer task data as required X X

for system and support equipment (Primary (Support

(description/frequencies) based on System) Equipment)

replaced or comparable system

Maintenance manhour projections must be X

based on specified maintenance concept

(levels of maintenance, accuracte RAM

j data)

Operation/maintenance requirements must X

reflect realistic system usage rate as

determined by mission/scenario drivers

Reasonably accurate total manpower demand X ?

estimate (number, MOS, skill level, addi- (Manpower (Accuracy

tional skill indicators) required before demand) unknown)

Milestone 1

Model must include and document projected X X

availability of personnel for the system (No., (Age/sex/

(Demographic data such as - number, age, skills, size

sex, size, skills, education, MOS, term MOS, education)

of service) Term of

service)

Model should document MOS alignment data X X

showing current status; authorizations (Current (Future,

vs. requirements to indicate MOS overload system multidemands) system

demands)

Model must incorporate and document target X

audience description - including reading, (Model

grade levels, experience, skills, previous assumes

training of available personnel to accu- prederately determine testing/training require- cessor '"*

ments (would be input to Letter of training

Agreement) is

adequate

" " ......... ...-... ......... *.....~ . **_,_.-..... -... .. S.,.....,.
*~-. . .7 - -- 7 

Table 4-3. (Cont'd.)

Applicability of HCM Technique

User Requirements to User Requirements

High Medium Low

Manadatory Output Requirements

The "T" of MPT must include the formula- X ?

tion, tranig description tie otri; and utilization rinn of (Insti- (Colleccomprehensive training concept elements: tutional tive unit

classes required (including any increases) train- training)

for individual; unit or institutional ing training: time to train; training....

frequency; and training support (devices,

facilities, instructors)

Documentation of MPT impacts and associ- X

ated costs for MPT projection

Tradeoffs between hardware and people X X

needed to determine optimal balance (Limited LCC and

between performance: costs (LCC) and tradeoff other O&S

support requirements (decision/tradeoff capability) elements)

matrix needed)

Output should include well-labeled, easy- X

to-read formats such as graphs, tables, (Executive

matrices, charts, as well as an Executive Summary)

Summary of the report that can stand

alone

Outputs should be in hardcopy as well as X X

CRT display for permanent archiving (Hardcopy) (CRT)

Reporting procedure must include briefing X X

re: any pertinent caveats about the data (Not consis- (Resolusource availability/reliability or tently done) tion

assumptions about the system, at major actions

decision points or steps in the analysis. on major

Actions should be noted to resolve weakweaknesses identified nesses)

.......... "-

.._.... .-....... ....... fl .*.~=. .*..:-..:...:.....
Table 4-3. (Cont'd.)

Applicability of HCM Technique

User Requirements to User Requirements -.

High Medium Low

Desirable Output Requirements

Historical MPT database highly desirable. X X

MPT projections should be updated through- (Histor- (Update

out life of system to increase accuracy of ical over

LSA/other data database) system

life)

MPT impacts should account for geo X

graphic constraints on M.P. pools "available" to run the system

MPT system output should be compatible X

with accepted LSA reporting format

Mandatory Design Requirements

Projected MPT impacts/costs must be avail- X

able at the front end of acquisition cycle

(before Milestone I) for (a) design

influence and (b) early MPT (cost)

warnings

Front-end determination of system's candi- X

dacy for MPT analysis should include

criteria such as potential for high

payoff, e.g., selection should be made on

type and amount of change in the technology that mightttdrive significant MPT

costs/changes

To facilitate training analysis and X X

proponency for MOS changes or additions, (New (Autothe model should have the capability to skill mation)

compare operator and maintainer tasks identifiassociated with a selected system with a cation)

comprehensive automated database of tasks

detailed in the Soldier's Manual, ARTEP,

POI, SQT, T.E. and AR 611-201 (enlisted

carreer-MOS Management Fileds)

37

. . . . . . . .. .. . . . . . . . . ' ~ . *%'. . - .
K Table 4-3. (Cont'd.)

Applicability of HCM Technique

User Requirements to User Requirements

High Medium Low

Mandatory Design Requirements

Output should include comparable MPT X

projection data for alternative (proposed)

and existing systems as support for

decision milestones --

Should incorporate horizontal analysis X

across systems to determine total MPT and

logistic support to facilitate ARMY-wide

optimization and support end strength

decisions

MPT model should provide compatibility X x

with existing DOD d.-umentation and (Exist- (Remote

reporting structures; however, it should ing docu- terminals

evolve into an interactive information menta- with

system with remote terminals, including a tion) time-line

time line oriented to both the budget and overlay)

systems acquisition processes. Signaling

mechanisms are needed for late/missing/

incomplete data

MPT system must identify/resolve weak- X

nesses in data inputs and resulting

analysis

* Documented and accessible audit trail is X

essential

Universal information system (CDB) should X X

be included and designed to allow multi- (Consol- (User

user access and update idated access)

database)

Desirable Design Requirements

More detailed O&O Plan required as X

"hingepin" to initiate MPT analysis at (Army

front end responsibility)

;'" _- -- : -'- _ -- -- -- - -" _ ,_ -_ :_ - _.,... -" .. -' " _ • ..' .* . . - . , .- .-* .*. . .- . . -_ .. _
~.- -. -, -~- - ~'~ - -- .. .-.- - - +.,,

Out of the nine mandatory design requirements, the technique seemed to

be in reasonable agreement with seven of the user needs. The two areas where

there was low agreement were considered critical for the near term. These two

areas were, (1) consideration of the horizontal effects of other systems on

the manpower drain and (2) construction of a means to flag and resolve

weaknesses in the MPT analysis. Both Items (1) and (2) were already discussed

as being an equally important consideration on the "output" side of the system.

Overall, this discussion suggests that the HCM is reasonably applicable

to the Army's major user needs in its present design. More specifically, the

present HCM basically meets seventeen of the twenty-two crucial user needs.

Therefore, by being largely commensurate with the present user requirements,

the technique seems to have considerable near-term utility to the Army.

-71

39 -

. . "..

. . .. .. . . . .- . .
SECTION V

OPERATIONAL ANALYSIS

A. OVERVIEW 4.

Section IV provided one measure of face validity by indicating the

relative compliance of the HCM with independently-derived user requirements.

This part of the analysis suggested that the methodology had considerable

utility to the Army. Other important face-validity considerations were.

introduced in Section III. These considerations refer more to the general

integrity of the technique and include such elements as potential error

sources, logic, reasonableness or credibility of the projected results,

inherent ambiguities in the logic or outcome, and how supportable the model is

by its respective input data sources. To close out these last elements of the

face-validity analysis, a multi-leveled approach was structured, basically

consisting of (1) a comparison of the HCH against any MPT modeling schemes

delineated in existing DOD and MIL-STD instructions; (2) a comparison of the

HC2 against a host of other known MPT projection or modeling schemes; and

(3) an independent, detailed, step-by-step audit of actual applications of the

technique. This part of the overall study was termed "operational analysis"

because every aspect of the methodology was actually exercised quite

rigorously. The following paragraphs first provide the detailed rationale and

approach used to investigate the various remaining face validity issues,

followed by the actual analysis and conclusions concerning the "general"

integrity of the HARDMAN system.

B. APPROACH

To ensure that the HCM was generally sound, beyond the user requirements

compliance, several additional face-validity elements were considered. The

additional face-validity elements and respective tests were as follows:

(1) Logic - A simple examination of whether the proposed technique

maintains a well-structured, credible train of logic throughout

the model.

(2) Usability - A simple study of both the utility of the model and

the ease with which the model can be applied by others.

(3) Error Sources - An investigation of whether the methodology

incorporates data inputs or internal procedures that can cause

inaccuracies in the projections.

(4) Data Availability - A basic check on whether the modeling technique draws on real-world, available data sources to act as a

foundation for the projections.

4() . -"

.....................................
(5) Ambiguity - An examination of the ability of the model to be both

consistent and succinct in its respective use of data and overall

design (e.g., having each successive step build on previous

steps). The ambiguity test also includes a look at how cl.early

the results relate to the model's original objective(s).

(6) Credibility - A simple test of whether the results seem

reasonable, based on the inputs and whether the technique

incorporates a means of weighing the quality of the data inputs

and results.

A three-phased analysis was structured within the context of the ... '

preceding face-validity tests. The first and second phases of the analysis

primarily addressed the logic, usability, error source, and data-availability

elements. The third phase of the face-validity analysis encompassed all six

of the above elements.

As an initial step in evaluating any new modeling scheme, it is useful

to compare the new methodology against similar techniques that appear

reasonable and/or are already tested and in use. This comparison serves

several purposes in the context of the above first four validity elements.

First, it provides a basis to examine whether the overall logic and structure

of the model are reasonable. Second, the comparison establishes a yardstick

against which to measure improvements in the versatility and subsequent

utility of the model. Third, it assists in flushing out inherent sources of

errors. Last, the comparison sheds light on the availability and quality of

data.

To these ends, the first phase of the validity analysis examined

existing DOD and MIL-STD guidelines to see if any credible MPT projection

schemes have already been established. Similarly, the second phase consisted

of conducting a much larger literature search inclusive of past and present

MPT models from both the government and private sectors.

The third phase of the face validity study was a detailed audit of three

applications of the projection technique. This part of the operational

analysis proved to be a pivotal component because it encompassed all the

validity insights provided by the above literature searches, as well as a

grasp of (1) the ease with which the model could be applied by others; (2)

whether the technique contained discontinuities between its inputs, internal

mechanics and results (therefore introducing ambiguities); and (3) whether

there was a clear indication of the quality and limitation of the outcomes.

The three applications chosen for the audit were the Single Channel Ground

Airborne Radio System (SI14CGARS), Remotely Piloted Vehicle (RPV), and the

Division Support Weapons System (DSWS). The results of the three remaining

phases of the face-validity analysis are detailed in the following paragraphs.

.* *..~ S .•.. -

o. -. -.-..-. . . ,....... ..- . . .,.... -.... . ,..... . -... ,--•, ,,......-... . . ','..... .. .- . .- • '"i ; -2,- '-:, ','* '-" -,.-. _.'.. _i..,* .. " " " "" - * -.. -- . "-" % -- - " " - - - " . . . . ." "
C. ANALYSIS OF EXISTING DOD MPT PROJECTION GUIDELINES r

As indicated in Section II, there are several DOD and MIL-STD documents b

that refer to MPT. For example, DOD 5000.1 and 5000.2 describe the structure

and supporting procedures for planning and completing major system acquisitions (1,2). As a precursor to 5000.2, the 5000.1 directive calls out the

necessity to define, estimate, and budget accordingly, MPT at the acquisition

management level. The 5000.2 directive, acting in a supplementary role, more

specifically addresses how the new acquisition will be evaluated against the

various support parameters such as MPT. For example, using a top-down e. ,-

* approach, it first defines the acquisition review council (DSARC or ASARC)

advisory roles and then establishes both the content and approximate time

frames for the major acquisition and system development review phases. From

the standpoint of threat mitigation, affordability, and supportability,

detailed guidelines are provided for developing the system acquisition and

support strategy. The 5000.2 directive states the necessity to consider MPT

needs under the "System Readiness, Support, and Personnel (ILS)" portion of

the document. Although this subsection requires that MPT estimates consider

both peacetime and wartime manpower demands, skills, costs, and training, it

does not indicate a methodology to assist in making these estimates.

Similarly, the 5000.1 directive provides even less distinct guidelines for

determining, and budgeting for, MPT costs. Because MPT falls under the

heading of Integrated Logistics Support (ILS), the next step was to examine

directives more specifically related to system support. Perhaps one of the

best of the family of directives associated with system support is MIL-STD

1388A (11). This MIL-STD states up front the requirement to develop human

engineering and safety programs as part of the overall system support analysis

plan. The document then proceeds to delineate key MPT variables and supporting data that must be considered to determine whether the proposed system

meets manpower readiness, supportability, and affordability goals. These

S-variables include (1) the number of systems to be supported; (2) missions per

Sunit of time; (3) mission duration; (4) operating parameters (days, miles,

hours, flights, etc.); (5) maintenance and transportation periods; (6) descriptions of system components and operating environment (in terms of their

impact on MPT); (7) task descriptions; (8) costs; and (9) special support and

test equipment MPT impacts. Additionally, MIL-STD 1388A requires that all the

various support scenarios for each proposed design alternative be compared

against each other to optimize the design selection. r

As in the case of the previously described DOD directives, MIL-STD 1388

also fell short of providing an analytic structure for projecting new

acquisition MPT demands. Although a dead-end from the analytic standpoint,

MIL-STD 1388A did provide more depth on some positive characteristics of the

HCI. These characteristics revolved around the reasonably close match between

the DOD MPT guidelines and structure of the data inputs and outputs of the

* HaM. Because MIL-STD 1388 provided more specific guidelines on MPT support

considerations than DOD 5000.1 and 5000.2, it was decided to probe one level

-. deeper into specific Army directives having to do with MPT. At this final

level it was anticipated that all key MPT variables and data elements would be

defined along with the supporting analytic structure. Probably the most

concise Army directive in the MPT area is Army Regulation (AR) 570-2 (12).

242

." ... ..• ... • -- - ... .- - - : .- - "-.. .-. -' .. .'-'- .. : . ' . . . -°-' . -/ o - . .. . * --. .. * .-. . -
W LIE

This service regulation, entitled the "TOE Manpower Authorization Criteria

(MACRIT)," provides criteria and standards that help establish an "equitable

relationship between services performed and types of personnel utilized." To

achieve this match the MACRIT requires that system manpower studies be

conducted and the results approved by the Deputy Chief of Staff for Personnel I-

(DCSPER) for the Army. The studies must provide the number of direct workers

necessary to effectively perform a specified work activity. Direct (actual

task performance), nonproductive (sleep), and indirect (travel and task

preparation) times must be considered in the manpower calculation. The MACRIT

defines some approximate time factors for these three elements and ultimately

provides a simple equation to calculate manpower. This equation is as follows

(12):

WORKLOAD FORCE STRUCTURE

(in direct and X (in number of MANPOWER

indirect manhours per system) systems to be procured) = (number of

AVAILABLE PRODUCTIVE HOURS people)

Figure 5-1 shows that the above expression can be depicted graphically

as a rectangular function.

One interesting aspect of the MACRIT manpower equation is that the

number of support people is calculated for the complete procurement totally

independent of the acquisition time cycle. Therefore, if the manpower requirement per year was graphed as a function of the system life cycle, one would

observe the same type of step function as shown in Figure 5-1, indicating no

change in the manpower requirements over time. This function is discussed in

greater detail later in this section. The most important aspect of the above

discussion is that AR570-2 presently represents the key manpower projection

and planning document in the Army. It is now also obvious, based on the HCM

summary provided in Section II-B, that the methodology is strongly aligned

with the manpower guidelines as specified in the MACRIT. as well as the

LU

Ck. - o

NUMBER OF SYSTEMS

-.-. - " " -' .'. "........ .• .......... - .. - -" -. - " . , • - -." - ' - -. , . ". -'. '. -'
..

tt r WON

broader policy guidelines spelled out in major support and acquisition

planning documents such as MIL-STD 1388A, DOD 5000.1 and DOD 5000.2.

It is clear from this discussion that related DOD directives do not seem

to provide much analytic guidance in the area of new acquisition MPT projec- .

tions. Although one appraisal element was obtained via the MACRIT, this data

point only represented one aspect of evaluating a much more complicated methodology encompassing task and skill analysis, personnel identification, training

identification, costing, and tradeoff analysis. To obtain more data points on

the manpower algorithm and evaluate the other methodology elements, it was

obvious that a broader literature search was required. The following section

provides the results of this broader effort.

D. CRITIQUE OF OTHER EXAMPLE MPT PROJECTION METHODS

The summary of directives provided in the previous discussion was not

meant to be an exhaustive review of all related MPT DOD documentation. For

the purpose of this study it sufficed to point out that the immediate and most

widely used DOD acquisition and support guidelines are not fruitful sources of

detailed MPT modeling techniques. A somewhat similar approach was taken with

the literture search. Again, the intent was not to cover all modeling

developments in MPT. For this study it was adequate to examine a sufficient

number of related papers and models to obtain a solid understanding of other

accepted techniques and logic employed in government and the private sector,

compare shortcomings of both the DRC and other models, and ensure that a large

enough array of models was examined so that all aspects of the HARDMAN

methodology could be evaluated. This approach turned out to be very efficient "

because the literature search revealed a considerable amount of redundancy in ..

the MPT material. In line with this finding, the following summary and

analysis encompass a select group of papers that appeared to have the best

data and analytic foundations as related to their respective aspect of the MPT

problems. As stated earlier, the HCM covers a broad number of MPT topics.

Therefore, to prevent any confusion in understanding how the various papers

apply to the different topics, the following discussion is organized by each

step in the methodology. The methodology steps described earlier in Section

II-D are restated here in a little more depth to assist the reader in seeing

the distinctions, or parallels, between the different projection techniques.

1. Summary of Papers Related to Building the HCQ Historical Data Base

(Step 1)

Step 1 of the HCQ requires the development of a reference system

that is representative of the proposed system (meets the functional requirements) and is generally composed of technically and operationally similar

predecessor components. The development of the reference system is a pivotal

point in the MPT analysis because a conceptual error in system configuration

could result in gross MPT errors later in the analysis. The premise for

conceptually building the proposed system out of predecessor components is

based on the idea that technology growth is usually evolutionary rather than

414
.. *

" revolutionary in nature. Two references were discovered, Fong's unmanned

spacecraft cost model and Gordon's review of mining equipment R&D, that

illustrate the "evolutionary" concept of technology development as being a

reasonable and supportable assumption (13,14). Both the Fong and Gordon

papers were chosen to show that the accepted concept of evolutionary

technology development in fact spans several industries (i.e., the

cerospace/communications industry to the mining industry).

2. Summary of Information Related to the Manpower Algorithm (Step 2)

As stated earlier in this section, one type of manpower projection

algorithm is the simple Army MACRIT form (12). This form is the one employed

in Step 2 of the HCM where the mission, operational, and maintenance data

(extracted from the proposed and adjusted predecessor system data bases), are

combined to establish the new system manpower requirements. Quade and

Boucher, in their book entitled Systems Analysis and Policy Planning, employ

the same basic rectangular function for projecting weapon system manpower

support (15). They suggest that this algorithm is desirable largely because

it provides a simple approximation of manpower (15). Another interesting

insight, somewhat in conflict with the Quade and Boucher approach, is provided

by Putnam in his paper dealing with projecting manpower to operate a new

computer software system (16). Putnam provides essential reference backup

supporting the theory that medium-to-large sized systems follow a life-cycle

pattern of (1) rise in manpower, (2) peaking, and (3) gradual tailing off as

the system gets older and is attrite. This cycle is termed the Norden/

Raleigh cycle and closely approximates a critically damped oscillating system

or a Weibull curve as shown in Figure 5-2.

-i* .% *

0

OC

Z2

,p

Bt t P-"-

t= LIFE CYCLE IN MONTHS OR YEARS

Figure 5-2. Typical Norden/Raleigh Manpower Curve

r f

" .~-.r

~~~~~~~~~~~................................. -''- -.- ......... s .. ,............. , .. ,... .. .. . * *. . .. . . .'-.' '.. . . . . . ... ... .. ..
rT

The equation, depending on the boundary conditions and how gradually the curve 'C ~

peaks and tapers off, is of the form: H .- bt -bt 2

MP C1 + C2 t e , or, MP = C1 + C2 t e 2

where C1 , C 2 , and b are constants determined by the boundary conditions MP - 0 @ t - 0, dMP/dt 0 @ t - t p (t peak), MP = MPmax @ t - tp

(t peak). The total manpower required for the life of the system is

determined by integrating the above funtion(s) over the total life cycle. The

important thrust of this paper is not determining the exact MP equation, but

the fact that the author provides substantial backup indicating that small,

short-lived projects can be modeled fairly accurately, using a rectangular MP

function (16). However, large, long-lived programs (such as a large-weapon

system) may require a manpower curve reflecting the more accurate buildup and

tapering off of real-life systems (16). This could have major impacts on early- or later-stage, life-cycle MP requirements because the rectangular

function would show a larger requirement than actually needed. Consequently, a new system being jointly phased-in at the same time or at a later stage might not have access to a needed manpower pool if the rectangular curve were might

used as an acquisition planning guide.

3. Su-mary of Papers Related to the HO( Training Resource Assessment

(Step 3)

Step 3 of the HC4 addresses the determination of training

resources. Step 2 of the procedure provides both the number and types

(skills) of people required for the proposed systems. Drawing on this

information, simple programm-atic matchups based on existing schools and

training algorithms are employed to calculate the kinds and number of courses,

instructors, training devices, and total training costs. The approach uses

present curricula, devices, and associated costs where predecessor system

components, similar to the proposed system, already exist. However, where new

skills and training may be required the methodology provides logic algorithms

(in the form of decision trees) for (1) determining whether additional

training is required and (2) whether (and what kinds of) generic training

devices are required. Ballpark cost estimates are then developed by using

present costs of the generic devices as they exist in the other services or

industry.

The PACER document, introduced earlier in Section II, is almost

uniformly in consonance with the HCM (7). One effort derived from the HCM and

developed by Winston, "A Preliminary Report on the Early Comparability

Analysis (ECA) Methodology," examines the difficult problem of identifying,

focusing in on, and training for, new hard-to-find skills (17).

46

. p.!i

- ~j.. ,* 
Winston's effort primarily addresses the issues of force modernization

versus the recruitment/retention of manpower. The object of the comparability

analysis is to identify potentially high MPT drivers early in the acquisition -

process. Data on specific problem tasks or "high drivers" would be supported

'" by historical, current, or projected data on MPT. Examples of data sources

i include (1) Army Occupational Survey Program (AOSP), (2) proponent schools

(TRADOC), (3) subject matter experts, (4) Logistic Support Analysis Records -

(LSAR), and (5) Army Publications. The basic methodology involves the

following steps:

(1) List the required tasks and MOS for the proposed system by using a

similar predecessor.

(2) Establish task criteria such as % of people performing a task

(PPT), task difficulty (TD), frequency of task performance (FTP),

risk involved if task not periormed properly (RI), task learning

difficulty (TLD), training time (TT), and task decay rate (TDR).

(3) Assign subjective values of 1-4 (where 1 is "low" and 4 is "high")

to each criterion, for every task identified with the proposed

system.

(4) Calculate a subjective task score for each task (i.e., PPT x TD x

FTP x RI x TLD x TT x TDR = Task Score).

(5) Identify high drivers.

(6) Conduct task analysis of high drivers.

(7) Establish learning/training requirements for high drivers.

(8) Determine personnel required based on test scores and available

skills.

The logic algorithms employed in the HCQ decision tree analysis of new skills

use the same subjective task criteria as listed in Step 2 of the preceding

comparability methodology.

4. Summary of Papers Related to the HC4 Personnel Requirements

Analysis (Step 4)

Step 4 of the HCH establishes the necessary number of people in

the training pipeline to support the manpower and skill projections made in

Step 2. The pipeline figure considers that personnel move in and out of

various skill categories for reasons such as retirement, promotion, or

changing to different skill areas. The fractional change in people moving in

and out of various skill areas is based on historical data.

47 . . - * . .. .. . .
I .. . . . . "° " " "- " -- - - -. " ' T" %'-. - - --. -.--. -. - - -. - -. . ~

Several papers associated with modeling this flow process were found.

Driven by a desired manpower ceiling for various skills, each model basically

establishes the monthly or yearly proportional ebb and flow of people within

each skill category (based on historical data) and compares the final remaining stock of people in each skill area against the desired ceiling to establish manpower shortages. Papers by Enke (18), Holz (19), Safeer (20), and

Doong (21), apply specifically to the military (i.e., Army and Navy); while

Spapers by Drummond (22), and Leeson (23), examine the problems of statewide

agricultural 

terms in the equations 

labor shortages 

describing 

and police 

the personnel 

shortages, 

flow 

respectively. 

process vary 

Although 

from paper

the a

to paper, the general algorithms are the same. The algorithms have the

following basic form:

n1 n " "J

ill 

NP A) i=l i - fOM- fIM + fR i

,".. °-

M P ( M (" i ,.A 

KE = > (HCi Z 1A

i=l i=l. .

where

(MPA = Manpower available in the ith skill category after reducing

the existing pool

P. = Existing or prjected pool of people in the ith skill category

fM = Fraction of people out-migrating from skill category i

f IM Fraction of people in-migrating

f Fraction of people retiring

R

(NP )i Desired manpower ceiling for the ith skill category

Ci

MP Total entrant manpower required to sustain the desired

E

manpower ceiling

Enke suggests using probabilities in place of the fractional adjustments of

the manpower pool to determine the possibility of joint events occurring

(e.g., the chances of an individual leaving the Army within the interval t + n

given that he has attained a certain level of training in a given skill area)

(18). Enke further indicates that variables such as the state of the economy,

r ."-
whether there is or is not a draft, or massive reorganizations can effect the

personnel flux greatly (18). Of all the papers, the most satisfactory approach

to dealing with these variables is provided by Enke. Enke points out that

historical manpower flows are only useful if (1) stable socio-economic conditions have existed and will continue to exist in the near future, (2). there is

enough cause-effect data to catch most reasons for reenlistment/loss, and

* (3) the cause-effect data are supportable by accurate statistics. However, to

gain experience on how historical manpower flows are perturbed by, for example,

a varying economy, he suggests using the Decisions Elements Analysis (DEA)

technique (18). DEA is a survey approach whereby statistically significant

. sample populations of personnel enlisting, leaving, and staying are queried to

establish the main drivers affecting their respective decisions. Modeled

after similar successful consumer product-oriented sales DEAs, this approach

provides a more accurate picture of how the population will largely behave i ~during periods of rapid, sizeable socio-econmic change. The different splits ,-.

in the sample populations are then used to proportionall' adjust the pools

within various skill areas so that the true manpower availability more closely

resembles the real world. Enke anticipates that, with time, sufficient

* l experience with widely varying socio-econcmic conditions can be gathered so

*. that personnel flow models can employ different sets of probabilities as they

apply to the given (or projected) national or international situation.

By comparison, the HCQ differs in the use of a starting manpower pool

and desired ceiling. Both of these variables are required data inputs to all

of the previous models examined. The HCa calculates the desired manpower

ceiling from Step 2 of the methodology and does not depend on a historical

starting pool. Instead, the HCM uses an optimization routine initiated by

placing one hypothetical individual (unity) in the first open-skill area

associated with a given task. The value of unity is multiplied by the

historical fractional manpower flows and what initially remains is a fraction

of an individual. The model then proceeds to input more individuals in the

same manner, stepwise, until all the skill areas are filled in accordance with

the manpower requirement. This stepwise calculation results in the total number of people in the pipeline that must be recruited and trained to

compensate for the various personnel losses. The HCM does not appear to

address the use of compensating factors to adjust its historical personnel

loss rates according to socio-economic fluctuations.

5. Summary of Papers Related to the H(10 Determination of Manpower

Shortages (Step 5)

Step 5 of the 1104 draws on the results of the personnel flow

analysis and ultimately makes a comparison between the pipeline requirements

and the present (or two-year projection) of available manpower. The resultant

difference indicates manpower shortages in the various skill areas. The key

difficulty with this step is finding a reasonable estimate of available

manpower. If the manpower shortage is required for a near-term assessment,

then the existing manpower pool is a reliable number. However, if the

potential shortages must be calculated for a system deployed five to ten years

.', 9

-. i~
from now, then the current (or near-future) availability number may be

inadequate. Several papers address the long-term manpower projection

problem. All of the studies (Lindsay's future Army enlistment regression

(24), Barnes' soldier 90 regression (25), and Grissmer's year 2000 manpower

supply scenario (26)) were service-specific. Both the Lindsay and Barnes

papers were standard linear regression models that project outyear manpower e-*, ,.

requirements. Grissmer employed a non-linear regression model that further

adjusted its manpower trends up or down through the use of elasticities (i.e.,

weighting factors such as declining economy, unemployment, pay changes, etc.),

which are projected to take effect at various time intervals and therefore

distort the curve up or down as they occur. The major drawback of this

techniques is that adjustments are not made to account for the lower

confidence associated with outyear projections.

6. Summary of Papers Related to the HCM Tradeoff Analysis (Step 6)

The last step in the HCI is reflective in that various "what if"

scenarios are developed in response to excessive costs or manpower and

training shortages revealed by the results of Step 5. Many of the "what if"

scenarios revolve around cost because it is a common denominator for many

design decisions. In performing the various tradeoff analyses as a function

of cost, the HCH usually does not extend beyond those elements directly

associated with MPT (e.g., tradeoffs related to billet definition and costs,

task definition, system reliability, task training, or training devices).

Other life-cycle cost elements such as spares, support and test equipment, and

technical documentation are not addressed.

One paper summarized in Section II was the HARDMAN Life-Cycle Cost Model

(HLCCM) developed for the Navy (8). This model was specifically designed for

performing a wide range of system tradeoffs. The complete program consists of

several interactive life-cycle cost models that allow the designer to change

input values for one or more of the life-cycle cost variables and observe the

impact on the relative cost of the other support elements. The cost variables

include (1) production, (2) wages, (3) spares, (4) support and test equipment,

(5) maintenance, (6) technical documentation, and (7) maintenance and operator

training. For example, if a new, expensive acquisition demands a large number

of highly skilled maintenance people, it may be worthwhile to examine the

impact of reducing the tasks and skill level by investing more money in

support and test equipment to prevent the reliability from decreasing and

subsequent spares from increasing. Similar to other life-cycle cost models,

the HLCCM technique seems to explore more adequately the range of possible

cost tradeoffs.

7. Summary of Papers Encompassing the Complete HOC

One immediate implication of the preceding discussion is that

there are apparently very few modeling processes that cover the complete MPT

arena. Indeed, the HCM seems to be the one process that achieves this level

of coverage. However, another MPT procedure examined, which appeared to be 0

r

50"

-:~~~~~~.-.-..v..-............................-...,,.......-,"' "'" " " " '" -.. "" .. '- - .. . -,
P:, designed to the same level of detail, was the Air Force "Acquisition of

Supportable Systems Evaluation Acquisition Technology (ASSET)" methodology

(27). One of the reasons the Air Force procedure demonstrates reasonable

equivalence to the overall 11CM is because ASSET was originally designed by the

same contractor. In its present form, ASSET represents the combined past and

present design efforts of the HQI design team, Westinghouse Corporation, and

the Air Force (27). ."-

In contrast to the HQ~s six major steps, ASSET has eight:

(1) Define the specific application and identify major data sources

and planning documents.

(2) Construct the consolidated data base (similar to the H110 CDB).

(3) Define all operator and maintenance tasks, tools, and test or

support equipment. r

(4) Develop maintenance action networks.

(5) Establish other logistic support resources based on the task

analysis and maintenance network.

(6) Compare the new system alternative designs against a baseline

system to identify different maintenance and resource demand

scenarios.

- (7) Perform a life-cycle cost assessment using all of the above

logistics support inputs.

(8) Develop a design option decision tree that indicates various

tradeoff situations critical to logistics and human resources.

Although the Air Force technique seems to use more detailed modeling

routines, it admits to having some data availability, data manipulation, and

subroutine integration problems (e.g., problems with obtaining data for

maintenance network probabilities or the mismatch between the ASSET personnel

availability model and the equivalent information generated by the Air Force

Manpower and Personnel Center) (27).

In summary, it seems that the H10 does not strongly diverge from either

standard DOD MPT doctrines or other accepted MPT modeling practices. Although

distinctions were made throughout the preceding discussion, the following

section summarizes the major positive and negative differences more concisely.

E. COMPARISON OF DOD AND OTHER MPT GUIDELINES AGAINST HCM

The major differences between the 1104 and the other various MPT modeling

techniques were pointed out in the preceding paragraphs. In some cases the

distinctions of other modeling practices represent improvements because they

' -: . :-.. . . . . . . . . '.-...'-. . . . . .... '. ... .' .. ". ..--. '"-,,", ... './'-.- ..-- . "--.-. .. - , -. '. ..
address real-uorld considerations that could affect the projection accuracy.

In other situations, the HCM strengths provide a stronger theoretical foundation. Table 5-1 summarizes the areas in which the strengths and weaknesses

were apparent.

Table 5-1 clearly indicates that overall the HC4 is fairly similar to

other HPT models in its logic, use of associated HPT parameters, and input

data. The major differences considered as weaknesses originated from (1) using

a rectangular manpower function, (2) not making allowances in the manpower

flows for the possible impacts of major socio-economic fluctuations, (3) not

considering long-range manpower trends on personnel availability, and (4) not

considering all operations and support variables in the tradeoff and lifecycle cost assessments. Item (1) above is important because of the potential

life-cycle end point problems cited earlier during the discussion of Putnam's

approach. Enke's point reiterated in item (2) cannot be overlooked because some socio-economic impacts, such as a rapid defense drop-off with change in

administration, are not minor perturbations. Perturbations such as this can

affect the supply of manpower for several years. Item (3) is an important

consideration because the normal seven- to ten-year acquisition period far

exceeds the two-year manpower planning projection used in the Army and H04.

Admittedly, the Grissmer, Lindsay, and Barnes papers referred to in Table 5-1

do not include a confidence factor in their manpower projections. However,

the use of confidence intervals is standard practice and could easily be

incorporated. Recommendations such as this are discussed in greater detail

later in this report. The last area, item (4), was considered a weakness

because all life-cycle elements must be considered to completely assess costs

and design tradeoffs.

The HCl clearly exceeded some of the other studies in the areas related

to building the data base and describing tasks and the personnel pipeline

calculations. The data base and task area descriptions were comparatively

better than the Fong or Gordon papers because these studies were not

projection models. The advantage of the HIQ over che series of papers related

to the pipeline projection was the optimization routine. By not requiring - -

either manpower ceiling or starting pool data inputs, the model reduced the

chance of contaminating the outputs with incorrect or obsolete data.

Having identified some of the strengths and weaknesses of the HCa via

the literature search, the next step was to investigate some actual applications of the methodology, drawing on the familiarization provided by the

preceding comparative examination. Inferences about the overall face validity

of the HCH were withheld until the audits were completed, and any obvious

parallels between the literature search findings and applications analysis

were identified. The final results are provided in the following paragraphs.

F. USE OF AUDITS TO EXAMINE FACE VALIDITY

The audit phase of the face-validity analysis was designed to follow the

literature searches so that the inquiry might be better organized (having

52
I.,.

C oIi- o°

0 C 0 0

0 V. M

4a Q)o 0)

to)-

4P4

... 44..0.,

" 5 t . C U0. 

0 " u 0 0 

,.~~ ,. oo ~ ..

"" I-I 0 .,,- 4), t 4) O;.

IV Enc. 0 . . .

CO *-00 0

0 0- 5-,

* ""' Cu C I

02 0 .4.' 0 ) "4 

|-.~ to (na .-

0. -4 ix

0 ,,4 024 I. C) 0 - 14)- 0E-' L .

0 i 00 -b '. W) ., 

000 go i,.

44 402 F- 4v .C o

0 . r_ w 0 r.4 W ,cc Q

0-. . CL > F 4. -. C -. cc 0 ) 5 0

00 0 ") ' JE- 0E- 024)" "

> 2 4 0~ a C

.4 I.. 24)0 J Cu~~,4 J ) 0. ', 4-4 .00 C . 1.C ) 0 a " "0'

'..4,4), -4'' ,, Z.. E 0 ,1.u+ u .-,. +

- 20 4.'45 2 .40 o" -0U. 

,. 0 C" .0 o4) 

Ia'.o0~ 0n U~C) -4)0 40 2

4) .,4 00 4)054) 0

0. .C".A"-,

j 02I00' 0 C.'0

4)) 4) 0... .0 w ~ E~. 00..

4-n toC04 ~ -1k'I J4) 0to w bA) U) 

U~L)022 :5 (n WJ(0 .1. c/ .u W

*j * ' .. *.. IA u C. 0 m~ V., - -

. .. . . . . .. . . . . ..4 $4 ... -, W - -4. 4 C* . . w c 4 ..) 

.0~~ p :. -4 w a) U) C ot .0 

co.. . . . . .. .. .. . .. . . . . . . . . . . .

(A" -,- C. "- . " "- ".
- -. - - . - - - o 

[." obtained some foreknowledge of the HCH process and its shortcomings). As

stated earlier in this section, the three applications selected for the audits

were SINOGARS, RPV, and DSWS. These three systems were chosen primarily

because they represented the most recent applications of the H1d. These cases

were also selected because, being at different stages in the acquisition

process, they also typified different levels of study detail that is useful

for examining potential data availability or credibility issues. Finally, the

RFV analysis was executed by a different contractor team than the one :..-.

responsible for SINCGARS and DSWS.

As an introduction, SINCCARS is a sophisticated inter-communication

radio system consisting of a manpack subsystem (i.e., radio backpack) and six :.,

other radio configurations that can be mounted on different types of vehicles,

i.e., trucks, tanks, etc. (28). The RPV, a small, drone-type aircraft that is

remotely piloted from a mobile command center, provides surveillance information on enemy target locations (29). Last, the DSWS system is a self-propelled

Howitzer linked with three other respective ammunition resupply, operations

center, and maintenance vehicles (30).

Each design is fairly complex because several major components make up

the complete system. Given the time and budget constraints for the total

evaluation, it was decided that only select components from each system could

be examined in any depth. The three following selection criteria were

developed in an effort to choose key components that could surface validity

problems:

(1) The component should be a major manpower driver in terms of

quantity and skills.

(2) The component should require maximum exercise of the mechanics and

judgmental portions of the methodology to explore validity issues

such as error sources, data availability, logic, and credibility.

(3) Where possible, the component should demonstrate the least

quantity and quality of source data, therefore requiring an

exercise of the most judgmental portions of the methodology.

These criteria would provide insight into the most likely areas

where error sources, credibility, logic, or ambiguity problems

could arise.

Although it was difficult to find components that simultaneously met all

criteria, generally the four to six components selected for each system

satisfied at least two of the guidelines. Once the studies were reviewed and

the components analyzed, a detailed list was assembled that addressed all the

areas where various face-validity issues could arise. The next step was to

interface with the HC4 designers to trace and understand exactly what was done

to confirm the problem areas. This was accomplished by converting the problem

list into a questionnaire, visiting the contractor's facility, and replicating

in detail the analysis performed (31). The complete questionnaire is shown in

Appendix C.

54.. .'.., . .... .
Two caveats pertaining to the audits should be stated at the onset.

First, only the RPV and DSWS audits were actually conducted at the contractor's facility. The SINCGARS audit was performed off-site under the same

guidelines as a test of the internal reliability of the methodology (32). The

test of internal reliability requires that an independent group of trained

individuals, given the same data inputs, attempt to replicate the results of a

previous application of the methodology. Although the details of the internal

validity test are provided later in Section VI, it suffices to state here that

the face-validity issues surfaced during the on-site RPV and DSWS reviews also

appeared in the separate SINCGARS replication.

The second caveat applies to the presentation and interpretation of the

audit results. Appendix C contains approximately eighty questions related to

both the three specific systems and the methodology as a whole. It should be

noted that of the eighty issues examined, which covered the camplete

methodology, roughly 85% (or 68 of the potential issues) were adequately 7--

addressed and resolved during the audits. A summary of the remaining problem

areas follows.

1. Results of the SINCGARS Audit

The major findings of the SINCGARS audit revolved around the

clarity of the methodology, usability of the technique, and data management.

The clarity issues primarily applied to the early Step 1 and 2 analysis done

in support of the ultimate manpower and personnel projections. For example,

functional requirements had to be developed as a subset of selecting

predecessor components to both construct a reference system and load the

historical data bank. The participants in the replication found that the

detail to which the requirements could be defined had a direct relationship to

the number of technically different camponents selected to build a reference

system and, therefore, could affect the number of people needed to support the

system. There were no clear guidelines provided for determining the depth to

which the requirements should be developed.

Another area of contention developed over the construction of the

reference system. The reference system is constructed using predecessor

components that meet the functional requirements and, in turn, are similar to

the proposed system(s) one desires to model. The study group discovered

through the SINCGARS example the possibility that, if two or more contractorproposed designs differed substantially, it could be very difficult to construct one reference system similar to both designs. This dilemma could

result in a weaker manpower projection for one case than the other. Although

the obvious solution was to develop two reference systems, the larger problem

was the matter of defining how much two proposed designs could diverge before

one reference system was insufficient to resemble the performance of both.

One area closely associated with the manpower projection is the process

of evaluating design differences between the predecessor and proposed components and defining the fractional improvements in maintainability and

reliability performance. The first major difference occurred over the selection of the so-called "perturbation values," which are the ratios used to

adjust reference data to better represent the performance of the proposed

• . .i
component. The study participants found that, because there were not estab- ,

lished guidelines for selecting perturbation values, their perceptions of

certain design improvements varied widely. This variance could have a major

affect on the manpower outcomes.

The study group suffered similar confusion over guidelines associated

with the selection of indirect productivity and induced failure values.

Although distinct values were provided for SINCGARS, the group recognized that

the PACRIT, which is the source of these values, provides a range of selection. Again, not being given guidelines in the methodology on how to select

either a high or low factor over a fairly wide range of potential values, the

study participants felt that the final manpower projection could be affected -ArA

considerably.

The last, most important, difference was associated with a misunder-

* standing of how the perturbation values were actually used in the manpower

" calculation. It was discovered that the HC! presently uses perturbation

values only to fill data gaps in the proposed designs. Occasionally, HM•

analysts find that contractors do not provide performance data on all the

components in the proposed design (such as SINCGARS). While constructing the

reference system out of similar predecessor components, the analysts will

select the predecessor component equivalent to the missing component in the

proposed design and "perturb" the historical performance data in order to

complete the data set for the new design. However, in the final manpower and

personnel projections the total reference system predecessor data are not

upgraded (using perturbation values) to allow the complete new design to be

modeled based on proven historical information. This procedure represented a

major logic problem to the control group because one of the primary reasons

for modeling new system manpower requirements is to prevent contractors from

being overly optimistic about the impacts of new technology improvements. By

not upgrading the old technology to a roughly equivalent proposed configuration

and establishing a revised historical data base against which to equally compare proposed designs, the methodology cannot test the integrity of contractors. Without making an equivalent technology-based comparison, the same

trivial outcome will always result; the new technology will appear to perform

better than old technology (i.e., the new technology will require either fewer

or lower-quality skilled people).

One area of contention that arose over the personnel assessment portion

of Step 2 in the methodology was the matching process between tasks and

skills. The participants found that the cookbook approach outlined in the

methodology was not completely accurate. This definitive approach is normally

followed up with the analysts actually contacting various Army training

schools to confirm, or update, the task and skill matchups. Because this

caveat was not passed on to the participants, there was general disagreement

between the skills selected by the original methodology designers and the

skills selected by the control group.

The last two areas pertaining to the usability and data issues were more

related to the methodology as a whole. The analysts discovered that, although

they jointly had backgrounds in mathematics, engineering, and personnel, they

still lacked the range of training and experience necessary to completely

comprehend the pitfalls and nuances of the methodology. The overall conclusion was that the methodology could not be properly exercised unless a

50)
.: well-seasoned, multidisciplinary team having joint military, engineering, data

management, and personnel backgrounds was employed. The last problem area,

data management, was not necessarily viewed by the study participants as a

shortcoming of the methodology. The observation was made that the complex

nature of the MPT issues alone caused the methodology to contain many places

where errors could be introduced due to (1) the prodigious quantity of data r

required and (2) the large amount of data manipulation required. ..

2. Results of the RPV Audit

The RPV audit surfaced several potential validity problems

.- involving elements of the early data-sorting process, manpower and training

projections, as well as the later tradeoff analyses. The most important

finding related to the initial data base step was the lack of a clear decision

process for selecting and using different sources of performance data (e.g.,

reliability and maintenance data). For example, it was stated earlier in this

*- section that the methodology normally requires historical data to be used for

- modeling new system performance. However, in the RPV case, most of the

' components had already been prototyped so that actual reliability and

- maintenance test data existed. Nevertheless, the decision was made to use 7.

only some of the contractor test daca. The remaining maintenance data were

discarded and relaced by comparable information listed in the design specifications that were part of the original proposal. The reason cited for this

action was that experience dictated the contractor data was not reasonable.

* Another example of non-distinct direction for data selection pertained to

modeling component performance on the small RPV using comparable components on

full-sized aircraft. The reason the contractor opted to use this information

. was because the Army, having no history of using drones, did not have comparable predecessor data. Closely associated with this decision was the fact

that the original designers did not differentiate between the different

* performance and environmental envelopes of the aircraft. These differences

could affect the stress levels and resultant wearout rates experienced by the

* RPV components. The confusion over the selection of data sources, although

somewhat linked to concerns about accuracy, more specifically revolved around

(1) identifying the various data sources and ranking them as a function of

quality and (2) determining when to use one source in preference to another.

In the manpower and personnel step following Step 1, the analysts found

the same problems with selecting perturbation and induced failure values that

were discovered with SINCGARS. Similarly, the step of matching skills with

tasks lacked a clear audit trail as to how the final skills were selected.

This particular audit problem was derived from the fact that the Army did not

list skills for operating and maintaining drone-type aircraft. Therefore, the -

HOI analysts had to search outside the normal cookbook procedure to define the

new skill areas and obtain associated wage and training information. Regarding

the final manpower projection made in Step 2 of the methodology, the step

function deployment assumption discussed earlier in the literature search was

confirmed. As indicated before, this assumption is considered a problem

because of the potential personnel allocation conflicts imposed by other

acquisitions under concurrent development.

57

,~~~-

* *** *
Step 3 of the methodology, the training resource assessment, surfaced

some interesting problems with respect to RPV. The analysts discovered that,

although the methodology provides for a very detailed instructor, training

resource and cost asessment, it has only been marginally exercised because of

lack of data. Critical training considerations such as the identification of

safety related tasks or task difficulty were not highlighted. Additionally,

the audit team found that the methodology did not differentiate between the

instructor contact hours required for critical versus non-critical tasks. The

analysts felt that this was particularly important because RPV appeared to be a new technology area not previously employed by the Army.

An in-depth investigation of the personnel shortage projection in Step 5

of the RPV study was very revealing. The investigators found that, although

the modeling algorithm for calculating shortages was sound, the weakness

pertaining to the use of the short-term manpower availability projection cited

in the previous literature sumary was verified.

In the final step of the RPV study, tradeoff analysis and conclusions,

the analysts experienced considerable confusion over the criteria for

(1) performing certain tradeoffs and (2) identifying the one MPT projection

that represented the best estimate of the real-world performance of the

proposed system. For example, in the training assessment several cases were

found where the design dictated the use of low pay grades in certain task

areas. However, when the training courses were structured based on pre-

* decessor systems, it was found that rather high skills were actually required

for the same tasks. Although the study indicated that this mismatch was a

major conflict, the training and cost implications were never pursued in the

tradeoff analysis. This ambiguity about the relative importance of various

tradeoffs did not give the analysts confidence that (1) all possible tradeoffs

had been considered; (2) important tradeoffs, such as the above example, had

not been overlooked; and (3) the tradeoffs finally pursued represented the

pivotal issues. The largest source of ambiguity in the RPV study revolved

around the presentation of the final NPT and cost projections. The prosent

summary format for the projections displays both the reference and proposed

results together. However, where wide variations between the projections

occurred (such as SINCGARS) or when several proposed systems were presented - -

(such as RPV), it was not clear which set of results represented the most

accurate approximation and MPT planning guide. The audit team found that the

source of the confusion originated from not having an indication of the

relative credibility of the data inputs and outcomes.

One final interesting aspect noted by the analysts applied specifically

to RPV but not to SINCGARS. The observation was made that because the RPV

study was completed roughly one year before this evaluation, the members of

the original HCM study team did experience memory decay with respect to their

decisions and judgments about the use of certain data sources and the ultimate

selection of one of the proposed system designs as the best planning figure.

This last finding was important because it demonstrated that, even with a

seasoned corporate bank of knowledge, the judgment areas in the methodology

must be better structured to allow a firmer audit trail.

..* . . . . . . ... 
3. Results of the DSWS Audit

The findings of the DSWS audit were quite similar to both SINCGARS

and RPV. The main reasons for the similarities were (1) the DSWS reference

system was compared against several, widely varying proposed designs; (2) a

significant amount of data sorting and selection was required; and (3) the

original DSWS analysis was also completed well before the methodology validity

evaluation. 

item 

for the 

(1) 

functional 

above 

The 

were 

chief 

requirements 

those 

problems 

associated 

and 

encountered 

building 

with defining 

by 

one 

the 

reference 

the 

investigators 

proper 

system 

level 

related 

to 

of 

model

detail

to U

four widely varying contractor concepts. Like the SINCGARS audit, it was felt

that the lack of sufficient detail on functional requirements and singular

reference system constraints could cause major variance and errors in the

- operational and maintenance manpower support projections. Item (2) above was

extremely important because of the confusion generated in trying to sort and

categorize generic component repair times by specific task, establish the

*. credibility of the various data sources, and understand which data sources had

priority over others. The DSWS data question was analogous to the earlier

defined RPV data problem. Item (3) addresses the audit trail decay dilemma

similarly revealed with the RPV study.

As was the case with both the SINCGARS an RPV investigations, the

- analysts did not clearly see the selection criteria for the DSWS perturbation

values, indirect productivity factors, and final choice of skills. Finally,

the investigators were confused by the way the data were presented for the

" final design option selected as the MPT planning guide. Similar to RPV and

"" SINCGARS, the study team felt that the best option should have been

respectively chosen as a function of (1) the relative quality of the various

* data inputs and (2) a comparison against a reference system that had been

perturbed so that the historical and proposed designs were compared on an

equal technological basis.

G. VALIDITY IMPLICATIONS

The preceding discussion highlighted many strengths and weaknesses of

the HQ(. The overriding strengths were:

(1) The methodology conformed well with DOD and Army doctrines.

(2) The technique basically resembled other known, accepted MPT

modeling techniques in its logic, use of MPT variables, and input

data.

(3) The methodology was reasonably well supported by its data sources.

(4) The HCI surpassed most other models in its completeness and

integration of various MPT modeling techniques.

(5) The methodology provided a better modeling foundation in some

areas than other simulations (e.g., manpower pipeline projections).

(6) The HCM stood up reasonably well under an extensive audit (i.e.,

85 percent of the test issues were adequately answered).

•- ° -. -° . . . . .. - - . • 9 . . .
However, the audits confirmed some problem areas outlined in the

literature search. During the process of replicating the three case studies

the remaining 15 percent of the test issues revealed that (1) the HI-.

*= conformed with the MACRIT in its use of a rectangular manpower curve with no

consideration of the Norden-Raleigh reduced end point demands; (2) the HCH did

not address Enke's potential impacts of major socio-economic fluctuations on

personnel flows in and out of various skills, and the Army as a whole; (3) the

technique did not project manpower availability and shortages at the actual

piojected deployment time of the new system; (4) the HCM incorporated, but did

not exercise, a technique similar to Winston's subjective critical task

assessment; and (5) che HCO MPT system examined only a small number of lifecycle cost tradeoffs (e.g., wages, reliability changes, task redefinition,

training programs and devices). Additionally, in the data base construction

phase, early in Step 1 of the methodology, there was a wide variance in (1)

the types and sources of data selected, (2) the reliability and maturity of

the data, and (3) the methods used in screening and manipulating the data to

conform to the methodology or to fill data gaps. This wide variance caused a

major problem in establishing the credibility of both the analysis and final

projections. It was also discovered in Step I that, by not reaching the

proper level of indenture for the functional requirements, key tasks and

subsequent manpower support could be left out. Another major finding

associated with the reference system portion of Step I was particularly

apparent in the SINCGARS and DSWS audits. It was discovered that where

several widely varying new designs were being considered, the use of a singlereference system could be an inadequate performance comparator for some of the

proposed concepts. Also linked to the reference system procedure was the

problem of not adjusting the predecessor historical data as a function of the

proposed design improvements. This was a pivotal discovery because the

contractor-projected manpower and costs could not be assessed on an equal

basis with what would be considered a reasonably equivalent extension of

existing technology.

The manpower and personnel projection phase of the audits uniformly

revealed that a better logic structure was required for selecting perturbation

values, indirect productivity factors, and induced failure rates. It was

found that arbitrary selection of these different values would affect the

ultimate manpower projection, particularly where small variations in manpower

support (such as at the field level) could cause critical shortages. In the

personnel selection step it was found that HC4's actual skill matching process

extended beyond the scope of the stated methodology. The resultant skill

mismatch and cost impacts were felt to be potentially sizeable.

In the tradeoff and conclusion portion of the methodology, Step 6, the

audits revealed a definite need for (I) a clear structure for selecting

tradeoffs and (2) a subjective weighting mechanism of the various data and

judgmental elements of the methodology to establish the credibility of both

the final design selected and the accuracy of the analysis as a whole.

Overall, the findings of the audits suggested that the structure of the

judgmental portions of the methodology still needed to be improved to offset

the observed audit trail decay. Furthermore, the sophistication of both the

HPT problem and the HCH design did not make the process amenable to use by lay

people. Although the preceding discussion indicates the various sources of

6 ".
potential errors, logic problems, ambiguities, or credibility issues, Table 5-2

more concisely illustrates the link between these findings and complete array

of respective face-validity elements defined earlier in this section.

The face-validity analysis provided an indication of the strengths and

* weaknesses of the RCM, as well as a first-cut, general examination of the

". integrity of the technique. The next step was to investigate and tailor

validity tests to examine more specifically the accuracy and reliability of

both the overall methodology and its projections. The validity tests selected

for the reliability portion of the study, selected to conform with the project

time and budget constraints discussed in Section III-E, are discussed in

" detail in Section VI along with the results of the complete reliability

analysis.

. . . . . . . . . . . . . . . . . . . . . . . . . "1.:. ."-. 
40. ...

-. 4 -4 4

FA .40

4)I

00 0

0

j 0 p) 0

-4 r4

to0 4) . ,.

4) 4 04.3

- 40 

06

to 0

00

4-41CC-.

4) 4---.400o U4C)cu a

to0 >4 U4uq L -4

41 60A

4)t0: -. 4 .0 u )0t

IV 40 A4)U ,4u>

00 9)1- to -. ) Ai>4o to t 1

0 AiP cth4

4) 41 44)

00u 41 4 to 0 0

u ,4 u . v > 0 0 0h 1-. 41 uO 4- 0 )1

4) 4.0 V04 cc4 - C ) 0 4)4a- 4 0-4000I- a A "a i 4) 9 l Q. )0-Li .0 a 40

-~~~~~~~~~~ ) 4 0 "10 0 0 -Q9.4 4
-4 .4

--4 -4

J -.4 -4

0

o 9

0 i

-4

CIA

FA V

.0) .0

J--4 0)

0 .0.0

to u 00r

Cu..

*0000

0 0 0 u 

44 0) 0 L0t

0- 04 14 M- 

*-m4. A o 4 C 'Li P-4c-404 C 

0. >, u kCO)ccc o4 . r LV

4.40 a 00.0 s ) hb . u. 0 a r. w u00 0 .C'.V-V . 0 AiC '4 0 cc U) 0- -4 '4 0 0.)0 0AWC j 0 0 w>,0 0 0

1:: - 0W 0)) 4 ~ *- CX (1) A cSEr 0 'o 4

0.4 $w4W L Q %.40 0I~ -O .0 C

M00 1.00 0 0) r ) 0 4 -4.00

w go -0 C 1.4 00 0 U4 0 4 "q A -4Or

~j 4W 00 0 -4C u r 41 = 0 0 0~

.4 O- r) v-41. 4 0 0u . E0 $4A

0 0 0- '4- - cc4-I 0) 0. 00 00

0 w 0 0) 4-4 w- 0)0 -4. 4) >.4 

z 0 0 -4 U) 0.1 44 Z -. U).- M.

C000)~~~~~~~I ) A'-i 05 0-- 0.) ) 0 0
0 A. 0) 

2 0).

x x.

0 I

-4

4) 0

4)0 

5.4 *0

.00

m 00

0 r

.4-0 -44

*.0 OD wu0b

4) w o o ) ,44

0 u 0

) c 4 AK:O

JDo 0 0b 0 4J 0 '-4 o

$ 40 0 ) SD .4 c Aj 0 6 - c .

-4 40,4 x.Q 0

0 04 t.4 IV4 I. boC CL4 (A 00) 0' cI $ .,cC 04 

u0 to 9. ti 41'- w0-414 c oF

r 0$ c *' 0 -4- 0 , C- .

go4. u4. PA4. V.~*. tj 0 W u l .4J

0) %I 00 O 0O C - 1.I b..640
SECTION VI

RELIABILITYL ANIALYSIS

- A. OVERVIEW

As explained in Section III, the reliability, or accuracy, of the HCM .

was explored via three avenues. The reliability of the HARDMAN methodology

was assessed by examining internal reliability, to a limited extent an

examination of the experience of individuals who have already applied the HCM

(or other similar methodologies) to real-life applications, and event series

validity.

For the internal reliability test, it was decided to use one of the Army

test cases (i.e., SINCGARS). A controlled experiment was designed to

establish whether a group of individuals, trained to use the methodology,

could both replicate the original answers and also demonstrate a low degree of

variance in their answers. A group of seven individuals (four civilians and

three career Army soldiers) of varying backgrounds, were selected for the

experiment. Given the small sample size (primarily due to restrictions in

available people, time, and budget) it was decided to dispense with detailed

statistical comparisons and simply establish general conclusions about the

repeatability and consistency of the results. The replication was also used

to develop a list of critical factors that have major impacts on MPT

projections. For purposes of consolidating the internal reliability findings

and to find ways to improve future versions of the HARDMAN methodology, each

participant went through a formal debriefing. Furthermore, a discrepancies

resolution conference, attended by the study group, ARI, and HCM designers,

was held following the completion of the replication.

Another important step in the reliability examination was a qualitative

accuracy check of past Air Force and Navy experience with the same (or

reasonably close) methodology. This was done by personally contacting

pertinent project personnel.

The last step in the reliability examination was a manpower sensitivity

analysis. Sensitivity analysis is a useful tool for testing how well a technique like the HCM must conform with the real world. This type of test is

often referred to as an "event series validity test" because it ultimately

provides an acceptable accuracy or reliability envelope for the outcomes of a

modeling process. If the envelope is rather large, then a quick-and-dirty

approximation is acceptable. As the envelope shrinks, greater emphasis must

be placed on improving the quality and accuracy of each data input and

algorithm. In this evaluation, the sensitivity (or event series) analysis was

done last to assist in formulating and prioritizing the closing recommendations for improving potential problem areas in the methodology. To obtain an

indication of the required accuracy of the HCM, a simple top-down cost

sensitivity analysis was designed to establish the allowable variance in the

manpower projections which would still provide accurate MPT estimates. The

estimates of the allowable variance in the manpower projections were performed

for one sample acquisition using an actual Army system. Each element of the

reliability analysis is described in greater detail in the next subsection.

65
B. APPROACH

1. Internal Reliability

The replication effort for evaluating internal reliability

* involved the application of the first two steps of the HARDMAN methodology to

. the Army's Single Channel Ground Airborne Radio System (SINCGARS). These two :. steps were:

(1) Establishing a consolidated data base.

(2) Determining manpower requirements. -'-i

The remaining four steps were not included for several reasons. First,

preliminary review of the HCH process suggested that there were a substantial

number of judgments, having considerable influence on the manpower estimates,

in the first two steps. Second, at the time the replication was performed,

HCM designers had only completed their analysis on the first two sections of

the methodology. Third, other duty commitments mandated that the experiment

be simplified so that the Army participants could finish the replication

within a maximum time frame of three weeks. The SINCGARS components included

in the replication were as follows:

(1) Receiver/Transmitter Unit (R/T).

(2) Electronic Counter-Countermeasures Unit (ECCM).

(3) ECCM fill device.

(4) Securable Remote Control Unit (SRCU).

These components were confirmed as offering the greatest number of judgment

exercises.

A group of seven individuals with varying technical backgrounds (i.e.,

math, engineering, psychology) and experience (JPL, Soldier Support Center,

""" Army Research Institute) were selected for the testing. This group attended

*-' the HCM training course, which was held at the HCM designer's facility in

Boston, Massachusetts, during February 1983. The goal of the course was to

familiarize the participants with the key analytical and judgment areas in the

" first two steps of HARDMiAN. During the training sessions, the methodology was

applied to an example subsystem of the Army's Multiple Launch Rocket System.

For the SINCGARS replication effort the two steps were divided into

eight different data packages. Each package required the determination of key

judgment and calculational outputs and was preceded by (1) a problem statement

related to the judgments that were required, (2) a list of the enclosed data

items to facilitate making the judgments and necessary calculations, and (3) a

brief statement about how to use the data. The problem statements and descriptions of the data packages are given in Table 6-1.

In general, the participants were to proceed independently and

sequentially through the eight data packages until the four components of

SINCGARS identified for the replication effort were evaluated. Three weeks

66
Yr. W- W- 77 7 .-

Table 6-1. Data Package Problem Statements and Descriptions

'a

Problem Statement Date Package Description

Develop system functional Descriptive data for proposed system

requirements and one worksheet for each subcomponent

Construct reference system Candidate list of predecssor systems

and description of contractor proposed

systems

Define tasks List showing range of generic classes of

tasks and contractor task definitions

for proposed systems

Reliability and maintenance Task time data, component failure data,

determinations for inherent induced failure data, equipment breakversus induced component down structures, predecessor task echelon

failures and assignment of assignments - one worksheet for each

task echelons component

Determine design differences Data extracted from 1, 2, and 4, above,

(design difference analysis, summaries of contractor correspondence

impact of design differences, with component contractors and one

* derivation and application of worksheet for each component

perturbation values (PV) to

"" appropriate tasks)

*Select and assign MOS List of candidate MOS from which to

choose appropriate specialties for each

task

. System/component Select data extracted from the operational

usage rates and organizational plan

Determine workload and Data extracted from 1-7, above, available

manpower requirements manhours - one worksheet for summary of

results

were allowed to complete the SINCGARS replication. The study group was

permitted to ask the HCM designers questions of clarification only by way of a

mediator. It was originally planned to provide feedback to the students at

selected audit points in the analysis for the first two SINCGARS components

(to allow mid-course corrections) and to provide no feedback for the last two

components to get a feel for what the end-point variance might actually be,

given no student coaching. In actuality, because of the huge quantity of data .-

involved and the relatively short time frame, feedback was given to the

students after each data package completion for all four components.

-. 67
,.'..-'. -~~~~~~. -P - - - - -N 7

: .. >*:./ 

The internal reliability was primarily assessed by reviewing the study

group's answers to the data packages and by comparing them to the baseline HCM

designer answers. Besides a reliability measure, convergence of answers would

suggest that the judgment activities could be done by any individual with ,

appropriate training whereas divergence would indicate inadequacies with the

methodology, student background or experience, or HCM training (or possible

combinations of these problems). Another major step in the analysis was a

structured telephone debriefing of each participant to identify subjectively

major problem areas in the replication or methodology.

The analysis of each data package within the reliability effort involved

either a subjective review of the participants' responses or a numerical /1

evaluation of the key answers as compared to those of the 1CM design team. To

protect the confidential nature of the replication, the participants were

referred to by number only (from 1 to 6). After the replication was

completed, each participant was debriefed to define individual problem areas

with the packages and to get feedback for possible future improvements in the

methodology. The debriefing specifically addressed relevance to present

assignments, difficult portions of the replication, discrepancies between the

HCM designer and study group answers, and the adequacy of the HARDMAN training

course (see Appendix D for a sample debriefing questionnaire).

The last major component of the internal reliability analysis was a

discrepancies resolution conference, which was held during August 1982 at the

Soldier Support Center at Ft. Benjamin Harrison, Indiana. This conference was

attended by the study group, representatives of JPL, the Army Research

Institute, and the HCM designers. The purpose of the meeting was to provide a

structured means of resolving 'ifferences and misunderstandings resulting from

the replication. This conference was a two-day activity in which each of the

eight data packages was examined and discussed in detail.

2. Qualitative Accuracy Check

The qualitative accuracy check was conducted by using participants

in existing military projects that have used some form of similar manpower

estimating methodology. The projects covered included the Advanced Lightweight --

Torpedo (ALWT), the DDGX destroyer, the VTXTX flight trainer, and the SUBACTS

submarine combat system. The pertinent project personnel were asked their

opinions on the validity of the HARDMAN methodology as compared to other

methodologies, as well as real-life data.

3. Manpower Sensitivity Analysis (Event-Series Validity)

7ie objective of the sensitivity effort was to roughly approximate

the permiss1''e variation in the manning variable. To accomplish this objective, four tasks were undertaken. The first was to develop a data set of

actual costs for the SINCGARS system. The second was to develop approximate

manning data for SINCGARS. The third task was to determine broad, acceptable

limits of error on total life-cycle cost estimates according to authorities

engaged in the budget and acquisition review process. The final, and largest,

task was to determine the extent of error allowable in manpower and training

cost estimates, given that other cost elements cannot exceed the limits of

their allowable ranges.

68
Within Task 1, data were collected reflecting typical costs of various

elements of life-cycle cost. The major components of total cost were ...

identified in an attempt to determine approximate ranges. It was decided that "'"

if possible, the more difficult-to-identify elements of life-cycle cost would

be ignored if their relative sizes were small. In Task 2, actual training,

manpower, and training estimates associated with SINCGARS were determined. In

Task 3, a variety of participants in the acquisition process were interviewed

to determine what guidelines they use for the range of acceptability of

errors. The groups contacted included OSD or CAIG, who are involved in the

review of DOD acquisitions. The results of these interviews were resolved

into perceived admissible limits of error on total life-cycle cost and each of

its components. In Task 4 the allowable variation in manpower cost in the

presence of other acceptable errors in the estimate was determined. This

study used an already developed life-cycle cost and level-of-repair model to

study the interrelationships between projection errors for several quantities

(8). The central idea was to determine how large or small the permissable

error in manning estimates could be as a result of these relationships.

The first step in Task 4 was to identify all the variables that

influenced manning and manning estimates. Next, other readiness-critical

quantities estimated in the model that depended on the same inputs were

identified. There are five major sources of variance in manning requirements. These are the equipment operating program, the deployment plan,

reliability, maintainability, and manpower characteristics. Together they

determine actual manning, and estimates of the variables that describe these

areas subsequently impact the accuracy of manning estimates.

The operating program describes the number of hours during which the

force is expected to operate the equipment. This might be twenty hours a

week, one or one hundred sixty-eight. During the life of the system, these

numbers will aggregate to a total usage that will have a fundamental effect on

total life-cycle cost. But for any given period within the life cycle, the

rate or pace of operations will help determine the demand rate for repairs.

The deployment plan simply tells how many units will be purchased and

deployed in the field. By extrapolation, the equipment operating hours are

determined, both over the entire life cycle and per unit of time. The

deployment plan also usually explains how usage rates are distributed among

operating units. This is important because of the effect of geographic

separation on various costs (i.e., pipeline delay times, number of maintenance

facilities, etc.). For example, a division may generate the requirement for

only one half a manyear of labor per year to repair a type of equipment.

While most labor costs would therefore be accounted for at half cost, training

would not, because a whole man must be trained for each division.

The reliability of an equipment expresses the rate at which failures

occur as a result of usage. In company with the operating program and

deployment plan which determine the usage rate, the reliability measure

(either mean time between failures (MTBF) or failure rate) gives the demand

rate for repair services. These three areas jointly determine how many failures must be repaired per unit of time.

69 S.

.
The maintainability of an equipment is expressed by the average time

* .required to repair it. In most systems this can be divided into two repair

times. The first is system restoration time estimated as the mean time to

remove and replace a faulty component (MTRR). The second is the mean-time to

*: repair the faulty component after the system has been restored (MTTR). The

system support policy is usually developed by analyzing the relative costs of

undertaking these two repair actions in different ways and at different

places. An item is discarded if the cost of repairing it, including all the

training, equipment and manpower consumed, exceeds the cost of buying a

replacement. When the converse is true, then the difference between repair on '" the spot versus repair at a depot must be computed. At the organizational

level, the system is generally repaired through fault isolation and replacement of the failed module. The failed module may then either be repaired or

discarded.

The last major determinants of manning are characteristics of the L

manpower itself. These will include quality (skills) and a version of

quantity. The quality of manpower will have an effect on repair times.

Generally speaking, these skills are set at the lowest level capable of beintg

trained through the service schools and cannot be dealt with in the estimation

process. Quantity is related to the amount of work available from a single

individual in a period of time. Typical times are 40 to 60 hours a week. Depending on the circumstances, these limits may also be exceeded.

It should be noted that other LCC variables such as special support

equipment, tooling, etc., can effect manpower and, therefore, represent .-.

additional manning drivers. However, for the purposes of this top-level

sensitivity study, variables such as these were not major manpower

determinants and were therefore not considered.

SINCGARS was selected as the system to illustrate the error bound I

interrelationships because it had been studied previously in the HARDMAN

project and because it represented a small (therefore, easy to model) but

important electronics system. The cost model used to perform the analysis is

an adaptation of the LCC model summarized earlier in Section V developed by .'

- the Navy HARDMAN Office (8). The bulk of the modifications made to the LCC

model were aimed at converting the computer software to a fast turnaround

system suitable for extensive sensitivity analysis. In addition, certain

values were added to the printed output, along with the ability to impose

predetermined support policies at the line replaceable unit (organizational)

level.

The model is a level of repair and life-cycle cost model that was

developed especially to be sensitive to the relationships that drive manpower

costs. It is roughly based on the MIL STD 1390 group of level of repair

models although the errors implicit in those models have been corrected, and

several innovations have been added to the computational procedures (8).

An example baseline cost estimate is shown in Table 6-2. The SINCGARS

life-cycle cost estimate is $784 million, excluding batteries. The batteries

were left out of the cost estimation process because they consumed a great

deal of computational time and added little to the analysis. Manning (in

manyears) is summarized in the lower right corner of the table. Real manning

70

"~~~~~~~~~~~~~~~~.'"..'.':... .';'':.L . . . ..... ; ."'- .... ,'.. .. -.....,-..,. .. . -... ... ,. .,. . ... . . .. ;,:-"
Table 6-2. Baseline Life-cycle Cost Sensitivity Analysis Run

Cost Elements, M$ System Rates

. Life-Cycle Cost 783846.0 Number of LRA Types 17

System-Unit Cost 11.1 Number of LRAs 17

INITIAL COSTS: Confidence Against 0.999

Production 312296.0 Stockout

Maintenance Training 1104.2

Operator Training 70280.0 System MTBF 472.3

Misc. Manpower 0.0 System MTTR 3.0

Spares 2512.2

S+TE 67048.6

Technical 55.2

Documentation

TOTAL 453296.0 SUPPORT POLICY SUMMARIES:

LRAs coded COD 0

OPERATION AND SUPPORT COSTS: LRAs coded MOD 9

Maintenance Wage 27457.5 LRAs coded local repair 4

Operator + Office 31019.4 LRAs coded discard 4

Wage

Maintenance Training 3390.6

Operator Training 215920.0

Misc. Manpower 0.0 TRAINING BILLETS BY TYPE:

Spares 9577.7 Equipment Operators 28112

Repair 1962.7 Org. and Int. Maint. 420 .

Transportation 17.6 Depot Technicians 6

S+TE 41198.5

Technical 5.5 REAL MANNING REQUIREMENTS:

Documentation Org. and Int. Maint. 205.12

TOTAL 330550.0 Depot Maintenance 4.27

OPERATORS 7028.00

requirements indicate the total number of manyears of labor, for all skills,

actually consumed by SINCGARS each year. These manyears are not necessarily

consumed in full manyear increments because labor requirements are distributed

over a number of geographical units (28 divisions, in this example). The

effect of geographical distribution is to expand the real manning requirement 4

to a larger number of billets required for training purposes. Because it is

not possible to train portions of people or to give people part of the

training required, training billets are constrained to be full integers; and

one must be developed for every individual who is required to work on the

system, no matter how little time is spent with it.

71 - - ,

,,... .... ..... .... .... .. • . ... . .. . . . ., i

.: ., -- ,~. . . . . .: _.-. . '.' .. . ,, ... ..... . . -. .- , . -. .' -. -,, ..... ..- ,- .' . - .. * .. ., .. . -.- -, . , .
The most extensive manpower requirement is for operators. This shows up '..

* in the consumption of real manning requirements as well as in the establishment of training billets. A glance at the list of costs also shows that, with o,"

the exception of production cost, operator training and billet costs constitute

the largest elements of life-cycle cost for the STNCGARS example. However,

while operator costs appear to be important in the estimation of total cost,

" they are unimportant in this study because they are unresponsive to changes in

most of the input variables that influence maintenance manning. It is

understood that the relationship between operators and maintainers may vary .-

from system to system.

The SINCGARS manpack version was modeled as a collection of 17 (18,

including the battery) line-replaceable units. In fact, the SINCGARS program

office basically views the equipment as consisting of four modules (R/T,

handset, whip antenna, battery) among which the R/T breaks down into LRUs.

Because the other three modules have only one piece and are discard items

anyway, the equipment's structure was simplified to accommodate the computa-

*. tional problem, treating modules as LRUs along with the LRUs that make up the

* R/T.

The model is driven by inputs that describe the support and deployment

environment, the system design, and the particulars of component design. The

data used in this exercise were selected as representative acquisition values

and are reported in Appendix E, along with the component cost sheets.

The model operates by estimating life-cycle cost for each component in

four different support postures. Normally, the least costly of these is

determined, and the output data associated with it are forwarded to the system

cost estimate. In the current example, support policies had already been

determined, and these were used to override the cost-minimizing routine of the

program.

At the same time that costs are aggregated by the system model, certain

variables of concern are aggregated as well. These include reliability,

maintainability, and system-configuration summary data, as well as an

accounting of the support postures selected for the different LRUs. While it

is understood that skill tradeoffs cannot be examined using aggregate costs,

the use of total costs for a top-level sensitivity study such as this was

acceptable. Finally, the cost tableau is broken down into investment of

initial cost and operating and support costs. Initial costs include both

production and logistic investments. Drawing on both the Table 6-2 baseline

costs and perceived acceptable LCC error bounds, the above cost model was

iterated until the relationship between manning and the previously identified

five major LCC variables could be clearly plotted.

C. SINCGARS REPLICATION RESULTS

1. Work Package I - Develop System Functional Requirements

The functional requirements data package consisted of three

subtasks: '

(1) Identification of functions and performance standards of

SINCGARS Manpack system.

72

.- . . . . . i... 

** * ** * * . . ..... . . . . . . . . . . . . . . . . . . . . . . .

• "
(2) Organization of the functions and standards into a

functional hierarchy.

(3) Allocation of the functions to equipment and people.

The HCM designers' answer sheet was a matrix that matched subsystem

elements and performance standards to functions (see Table 6-3 for a sample

answer sheet).

There were no numerical answers to evaluate in this package. Part of

the study group adjusted the answer format. Additionally, differing semantics

among the participants complicated the analysis. That is, a "participant"

could mean the HCM designers but use different wording. Neither of these last

two factors necessarily made an individual's answers incorrect. In order to

evaluate the answers, it was decided to determine whether or not the participants included the major functional requirements in their answers. As

shown in Table 6-4, none of the respondents had the function "Operate

SINCGARS." However, the remainder of the answers indicated that the study

group understood that this was included. All respondents had "Communicate"

and four (2,3,4,6) had "Accommodate Mobility," but only two had "Command

Control." It should be noted that an "X" indicates matching answers between

the HCM contractor and respondents.

In terms of format, one participant had three extra element columns

(adaptor, shorting block, COMSEC), while another individual never used the

backpack column. Two individuals itemized tasks rather than functions. Three

individuals missed at least some performance standards. On the other side of

the spectrum, five of the study group included extra items such as "pass

through retransmission" and "monitor net," which the original HCM designers

did not have. In this sense, the study group's answers were more complete.

In the debriefing it was found that two of the group were confused as to

the HCM usage of "end item." The study group as a whole was unsure about much

detail to give in its answers. Regardless of some of the above confusion, the a functional areas were basically understood and covered by the

participants even though they were limited by an open-ended answer format.

2. Work Package 2 - Reference System Selection

The objective in this package was to select the reference system

based on the following given information:

(1) Functional requirements.

(2) Reference system candidate equipment.

(3) Proposed system design data.

The analysis of this package was based on the ability of the replicants

to specify the four major system components: R/T, ECCM, ECCM Fill, and SRCU.

Given the correct selection of the radio/transmitter (AN/ARC-114), the

remainder of the components generally followed. As it turned out, all

73

.. , . _ . . . . . . .... .. '. . . .,...,. '.. .. ,. , ,,,.:..... ...... .- , "'... .. . . . . .... . . . .".. ., " . . .. "._. .'. '
mFmTmJm .m -- -- -- ---iin - - mjin~ ~m~T 1 -

~ 1 II ii

I.

3 Ii

(Jb 'a - - - - I - a ~ b

* a * - ft - N . St

* - S. -. -. F* E* ft ft ft

N N N N N

* SN 4) uu N

* 'N-a.

* 4) - - - --

.~ I - 4 -----------------------

- N N

0) ~ U- N --

-~ -'- ~~sJ * N

~ I .

- - N N - - -

~. - 5 N N

St N

- ~ m mu *E urn mm mm.. mm inm U -~

0

C

~0

4)

'.7 .5.

CO

H

3. .3

I

I.

- N

-~ - S

U

- m - m - - * * - U iniinI U m mum m m

3.

74
Table 6-4. Work Package 1 - Respondent vs. Dynamics Research

Corporation Results

Total

SMa j or Respondents Number

Functional Requirements DRC 1 2 3 4 5 Correct

Operate SINCCARS X Not specified but

inferred

Communicate X X X X X X X 6/6

Command Control X X X 2/6 -.

Accommodate Mobility X X X X X 4/6

participants had doubts concerning the H1CM designers' answer. Several felt

that their selections were better suited as the baseline system. In the

discrepancy review the HCM designers explained that the 114 had the most

available data. This was obvious from a data source matrix that was not given

to the study group. This mismatch also suggested that the group did not -

understand the relationship between comparable component selection and data

availability for that component.

This package was among the easiest to grade of all the packages (see

Table 6-5 for the answer comparison). Three individuals (3,5,6) had the --

correct R/T. Two of these three had the remaining major components correct as

well. As in Package 1, the level of indenture was again a concern. All

participants were missing at least some detail. Minor subsystems such as fill

cable and battery were missed by everyone except Participant 1. Two individuals had some detail (i.e., programmer gun, antenna) that was not included by

the HCM contractor. Aside from the doubts on the HCM designer selection, most

participants felt that the material was too technical, while two participants

felt that there was not enough time allotted for this task.

In summary, without knowing the complete HCM contractor rules used for

the reference system selection, and without all necessary input information,

it was difficult to obtain the same answer.

3. Work Package 3 - Task Identification

The objective of this part was to take the previous reference .

system design as specified by the HCM designers, plus the data included in

work package 3, and develop generic and reference task lists and assign the

tasks to equipment.

75

- .....' ' . ' .. '. * * . ""-° - , *L' S. ,S. . " - . " C. "* <- .° C L S ' - - .*' * *',"" ".*. - - - ". - "" - - --"" ". ". * -" -" " "
Table 6-5. Work Package 2 - Respondent vs. Dynamics Research

Corporation Results

, Major Respondents Number

System Components DRC 1 2 3 4 5 6 Correct

AN/ARC-114 R/T X X X X 3/6

SN 416/APX 76 ECCM x X x 2/6

KYK-13/TSEC Fill X X X X 3/6

C-2328/GRA-39 SRCU X X X X X X 5/6

The answers to this package were similar to those in Package 1 in that

there was a fair amount of subjective matching of tasks to equipment. This

. again complicated the grading. Therefore, it was decided to check for major

categories of equipment only. A major problem arose for the study group here

because in Package 3 there were three new major equipment categories (antenna

coupler, digital data device, COMSEC), which were not included on the original

work Package 2 answer sheet. All three of these categories were missed by

almost everyone (see Table 6-6). Only one participant broke his answers into

* generic and reference categories. Even the 11CM designers' answers were not

broken down this way. The HCM contractor did not use the workload worksheets

for his answers even though they were provided to the participants. Only one

participant used the appropriate worksheet from the methodology. This had

been done in the training course but was not a requirement of the replication.

The level of indenture was again a problem for the study group. Everyone was ...

missing a' least some detail.

The HCM designers pointed out during the discrepancies review that this

type of effort would typically be a multi-person effort and that missed areas could be picked up within later data packages. Almost all of the participants

felt that either not enough detail was provided or the solution did not

logically follow from the methodology instructions. Upon clarification of the

problems within this element of the replication, it appeared that the participants had a reasonable understanding of how to apply tasks to equipment.

4. Work Package 4 - Reliability and Maintainability (R&M)

Determination

In this package the analysts were directed to use given input data

and then to perform a reliability and maintainability analysis of the SINCGARS

system manpack configuration. The final result of this effort was the determination of the system equipment maintenance workloads. The participants were

76 .. .. ..
Table 6-6. Work Package 3- Respondent vs. Dynamics Research

Corporation Results

Major Respondents Number

Equipment Categories DRC 1 2 3 4 5 6 Correct

M anpack X X X X 3/6

Receiver Transmitter X X X X X X X 6/6

Antenna Coupler X X 1/6

ECCM Unit X X X X X X X 6/6

Digital Data Device X 0/6

SRCU X X X X X X X 6/6

COMSEC X 0/6

to take the tasks and equipment on the workload worksheets, perform the calculations as specified in the instructions, and ultimately derive maintenance

ratios (see Table 6-7, column 14, for a sample workload worksheet). This was

to be done for three types of systems: a reference system and two proposed

systems, one from Cincinnati Electronics (CE) and one from ITT. This exercise

resulted in an examination of approximately 300 maintenance ratios for each V

participant. Unfortunately, wrong frequency information was given in the

instructions for using operating hours. The wrong usage metric (annual

operating hours) was given for the CE ECCM module. The contractor answer

sheets had several problems. General and direct support were interchanged for

the reference system ECCM module. CE SRCU answers were given even though the

participants were only supposed to use reference system data. A few minor

components (e.g., battery) were missing. Most importantly, several answer

sheets were wrong and had to be replaced by corrected ones.

In view of the above problems, it was decided to grade the study group

based on what they thought was the proper procedure. The use of reference

system data for components of the CE and ITT were not included. Because one

participant (1) did contact the designers and obtain the correct CE ECCM usage

metric, the use of the incorrectly given value was counted as wrong. Table 6-8

gives the percentage of correct answers with clarifying comments. The use of

the correct usage metric for the CE proposed system is obvious for Respondent

1. His percentage correct was 94 while everyone else varied between 59 and

71. Several respondents (3,4,5) were missing some system components. Participant 6 disregared some columns on the workload worksheet but still managed -

to obtain high scores. Similarly, Participant 3 seemed to differ from the HCM

designers in his approach but still arrived at correct answers.

77

.. . .. .. . . . .. "- . . :.,-- - . ; ;.i= , ,i v - >. .-..- ?..- - . , - ~ -. > .. >-. .
-- f- - 7.I

4 CU4

;7 c

-~ 

-0 C0 0C, 00 0 0 0 01 0 L

S 00 V

I II I I I

IT 0 0 

K 00 0 0 :

C

* a.) 0 0 0

C 0

) 0 

CU41 CD

AJ~- W4. Uc ~ U ID 1. U0

-: ) a) a o

1) !~ ~ ; oF- Z -0 r

0 w

78
Table 6-8. Work Package 4 - Respondent vs. Dynamics Research

Corporation Results

(% Correct-Maintenance Ratios)

Respondent Reference CE ITT Comments

1 70 94 87 E

2 95 61 94 A

3 95 59 82 A,B,D

4 40 60 95 A,B,E

5 50 71 89 A,B

6 100 71 89 C

X 75 69 89

A - Wrong usage metric for CE

B - Missing items - Reference, ITT

C - Missing calculation steps

D - Different calculation procedure than DRC

E - Wrong total actions - ITT, Ref S.

The study group had several comments on the incorrect and confusing

replication instructions. Most commented on the extensive calculations, time

limitations, and tedium caused by the task. One participant (3) resorted to

using a programmable calculator.

During the discrepancy re.iew, the HCM designers noted that this

procedure has now been simplified by using updated LSA data sheets and by

automating the worksheet.

Despite all the problems, the number of answers that matched the contractor's calculations were relatively high (generally between 60 and 95%).

This indicated that at least for SINCGARS, the study group probably understood

how to apply this phase of RARDMAN.

5. Work Package 5 - Design Difference Index (DDI)

The participants were directed to develop the SINCGARS proposed

design in the manpack mode for the two contractors (CE and ITT) by evaluating

and compiling a list of reference system versus proposed system design differences. The study group was then supposed to determine the impacts on

system workload of each design difference. The resultant perturbation values

.. . ... ...... ... . '," - - - * "' '- . ' ,'. ? ,- ' , . - -. . _ , . ' .
a V.

(PV) were to be used for adjusting the reference system design so that equitable comparisons could be made between the reference and proposed systems.

This task generated the most discussion during the discrepancy review.

The discussion centered on the nature of the comparisons. For instance, should

-, a reference system be used "as is" or should there be an adjusted reference

system if reference and proposed systems have significantly different technologies and time frames. This question was not addressed during the training

course or the replication. Because this problem was discussed earlier in

Section V, it will not be pursued here.

As in packages 1 and 3, the work that Package 5 answers were quite

subjective (see Table 6-9 for a sample contractor answer sheet). This package

had the highest variance among participant answers. The contractor answers

had only two PVs. Participant 3 had 30 values. Individual 6 had 16. Participants 4 and 5 had none. Participant 5 left out the ECCM, ECCM Fill, and

SRCU subsystems. Individuals 4, 5, and 6 made no reference to the LSA-02

reference data.

It was found in the debriefing that the study group had trouble

determining the design differences and did not understand the use of the PV.

In summary, it is apparent from the study group feedback and work package

answers, that this step could not be closely replicated and further

Sclarification is needed.

6. Work Package 6 - Select and Assign MOS

The objective in this part of the replication was to record the

a. MOS and skill levels that are currently performing each operation and

maintenance task for the reference system equipment in column 6 of the

workload worksheet. MOS and skill levels were also to be recorded for the

contractor proposed systems. Lastly, for those tasks omitted by the

contractor, the reference system data were to be used.

Because the study group had previously calculated the column 14

maintenance ratios on the workload worksheets, it is likely that the

replication effort would have proceeded more easily if this task had been done

earlier. This was reiterated during the discrepancy review. Additionally, it

was pointed out that the MOS selection process needs additional refinement by

the HCM designers. Presently there are no systematic decision procedures for

deciding which tasks are to be matched with the appropriate MOS and skill

levels.

Once again, there were problems with the given answers. The grader was

not given all the answer sheets. LSA answers were supplied for components

where reference data should have been used. Lastly, CE R/T answers were given

when the study group had not been given the inputs. As opposed to Package 4,

the reference components jointly included in the proposed systems were

included in the grading. This inclusion is not very significant (6 out of 151

answers for the CE (4%) and 13 of 141 for the ITT (9%)), but it was done here

because the participants were directed to do so in the instructions.

•- . - . - - . 80).. * * . . . *
41

.CU

ILI

10

0

.0

I,-

I.' it4

S81
All participants primarily had problems with, or could not understand,

* the contractor MOS selection process. In fact, one individual used predecessor system MOS instead of those of the reference system. Even though it

, was only necessary to copy MOS from the correct data sheets, the average scores

were not particularly high (55-80%) (Table 6-10). Two possible contributing

factors are (1) the recurring problems with the methodology instructions and

(2) the boredom caused by the tedious nature of filling in approximately 300 MOS

and skill levels.

7. Data Package 7 - System/Component Usage Rates

In this package the usage rates for the components of SINCGARS

were to be determined. This data is required for column 15, usage rate, of

the workload worksheets (see Table 6-7). Because only one value for each

component (R/T, ECCM, ECCM Fill, SRCU) was required, this was the most

straightforward assignment of the entire replication effort. The entire study

group had the correct answers. Note that in doing a firsthand application in

the real-weapon development environment, usage data might be difficult to

acquire.

Table 6-10. Work Package 6 - Dynamics Research Corporation

vs. Respondent Answers (% Correct MOS Answers)

Respondent Reference CE ITT Comments

1 50 95 79 A

2 77 87 73 C

3 73 79 67 C

4 50 97 63 C,D

5 45 44 91 A,C,D

6 36 85 69 B,D

X 55 81 74

A - Missing Reference System R/T MOS. Z-,

B - Assumed DRC used predecessor rather than reference.

C - Missing CE or ITT R/T MOS.

D - Missing Reference Components for ITT.

82."

.- '."'i -'i-' i--i-...-........"......- " ...-..- i...'........-.'.".........".-.-..-..-. . 4-. .-. .
8. Work Package 8 - Determine Workload and Manpower Requirements

The goal of this final package was to determine the workload

* results and manpower requirements for the reference and proposed systems. The

workload results were obtained by multiplying the maintenance ratios from Work

Package 4 by the scenario usage rate from Package 7. That result was to be multiplied by the indirect productivity factor (IPF) to obtain the workload ,

results. The next step was to add the workloads, by component, to get the

total. The manpower requirements were to be obtained by multiplying the

workload totals by one of two manpower availability factors that were supplied

by the contractor. Because this last step was trivial, only workload totals

were analyzed in detail.

The HCM design team answers had several shortcomings. Calculation

mistakes were found that had to be corrected, and LSA data were incorrectly

used in place of reference system data. With respect to the study group, the

use of wrong MOS was evident for five respondents. Reference data were left

out by four of the participants. Four individuals also made calculation

mistakes. The participants were confused by the contractor justification for

selection of IPF and manpower availability factors. Two also commented on

time limitations. Given the above problems, in addition to the problems with

the previous packages, it was surprising that the study group got any of

Package 8 correct. Actually, the average percent correct was between 25 and

45 (Table 6-11). One individual (2) averaged 69% correct for each of the

three systems.

D. QUALITATIVE ACCURACY STUDY

The names of pertinent military personnel were supplied by ARI and the

Navy RARDMAN office. These contacts covered a wide range of Navy and Air

Force procurement projects that required manpower and training forecasts

(Table 6-12). The feedback on HARDMAN with respect to these real-life

applications varied from highly positive to negative. It was noted that these

impressions could be at least somewhat affected by the personal biases of

people contacted. Nevertheless, as a rough estimate of accuracy, it was felt

that these contacts were a good initial step.

The Advanced Lightweight Torpedo (ALWT) project logistician was highly

favorable (33). He was impressed with HARDMAN in that its initial application -

could start with very little input data. Its resulting projections appeared

to be within about 10% of actual manpower data. His immediate experience

applied to the reliability and maintainability areas. He also pointed out

that these estimates went through the Navy's Logistics Review Council (LRC)

unscathed. This typically happens only thirty percent of the time. Most of

those projects that do not pass have unsatisfactory manpower and training

estimates. HARDMAN also saved ALWT large sums of money because of its ability

to highlight possible problems. These comments were reiterated by the ALWT

ILS manager although he added that the HCM was difficult to understand (34).

He also pointed out that HARDMAN provides a consolidated data base that

facilitates sensitivity analyses. The ALWT and SIRCS MPT logistic element

manager felt that once the total integrated methodology (HARDMAN plus training

requirements determination plus life-cycle costing) is developed, it will be

...--.-7.
Table 6-11. Work Package 8 - Dynamics Research Corporation

vs. Respondent Results

% Workload Results Correct

Respondent Reference CE ITT Comnents

1 0 70 25 B,C,D

2 86 80 42 A

3 29 60 50 AD

4 0 10 8 A,B,D

5 29 30 17 A,B

6 14 20 8 A,B,C

X 26 45 25

A - MOS Discrepancies

B - Component answers wrong

C - Left out reference maintenance

D - Did not use all reference data for CE or ITT

Table 6-12. Qualitative Accuracy Study Contacts

Project Contact

SUBACTS - Submarine Combat System Training Manager

ALWT - Advanced Lightweight Torpedo ALWT Logistician

ALWT - Advanced Lightweight Torpedo ALWT ILS Manager

DDGX - Destroyer Personnel and Training

Analyst

ALWT - Advanced Lightweight Torpedo MPT Logistics Element

Manager

SIRCS - Shipboard Intermediate Range Combat

System MPT Logistics Element

Manager

VTXTX - Flight Trainer Project Manager

Air Force - ASSET and other similar ASSET Operations Research

techniques Analyst

84
very useful for the early identification of manpower requirements (35). In

comparison with other similar methodologies he felt that HARDMAN was less

cumbersome and more accurate. He also pointed out that one drawback of this

type of methodology is that it does not interface with hardware design

requirements.

The training manager for the SUBACTS program felt that, although the

HARDMAN manpower forecasts have been fairly accurate, there were problems in

understanding the approach (36). The SUBACTS program brought in an independent

contractor to use HARDMAN, which helped make the approach more understandable.

Concurrently, the new projections proved to be as accurate as the original

forecasts.

One of the DDGX destroyer training analysts has had MPT experience on

pilot programs within NAVC, NAVAIR, and NAVLX. He stated that there was no

standard, overall methodology for manpower estimates at present although it is

planned (by January 1984) to evaluate a Navy program (SYSCOM) with the help of

the HARDMAN office. He felt that, although total feedback on HARDMAN has not

been received as yet, it is probably quite useful and will have many

applications within the Navy (37).

The project manager for the VTXTX flight trainer was too far removed

from forecasting to say how closely HARDMAN resembled real life. With respect

to other similar methodologies, he stated that it has typically been difficult

to match their outputs to actual performance. This has usually been due to

the differing ways the methodologies define system components as compared to

the actual system design (38).

An operations research analyst at the Air Force's Human Resources

Laboratory has been involved with several Air Force projects. She has worked

with HARDMAN-type methodologies (e.g., coordinated Human Resources Technology,

ASSET) but has found to date they have not been widely applied (39).

E. RESULTS OF MANPOWER SENSITIVITY ANALYSIS

The problem explored in this portion of the reliability analysis was how

accurate manning estimates must be in a new system acquisition. Because the

penalty and cost of estimation can be quite high, it is important to be clear

about the benefits of accuracy and the costs of inaccuracy, particularly when

developing a projection technique such as the HCM. Further, because manning

is not estimated in a vacuum, it is important to establish whether concentration on accuracy in other related variables will have any impact on the

manning estimation.

In this portion of the analysis, permissable error bounds were

determined and applied to the SINCGARS program to study the interplay of

manning estimate accuracy and the accuracy of other variables. Error bounds

were determined through interviews with decision makers involved in program

review offices such as the Office of the Secretary of Defense (OSD) or the

Cost Analysis Investigation Group (CAIG). For this small study, most of the

individuals contacted were associated with OSD. Data descriptive of the

SINCGARS system were obtained from the SINCGARS program office. Error

85

I• .

* . .. . . .. . . . - . .**' -*-**-* 

--- ' ' *'-- - ' -S ''''' ._. . '' , '- . ' "- ',-. . .. " ,-.', -, .--. , ", " ''' '-
N-7

relationships were investigated through the use of an interactive life-cycle

*. cost model developed for the U. S. Navy and adapted to this particular

exercise (8).

The overall conclusion of this brief study is that accuracy of manning L -.

estimates may not be a significant issue in the development of tools and..-

methods for acquisition support and cost analysis. For example, if manning

estimates were scrutinized by the budget controllers and were found to be as

poor as the 15% boundaries allow, the error in spare stockage costs (a major

LCC 

likely 

variable) 

flag other 

would 

important 

be almost 

quantities 

50%. Therefore, 

long before 

budget 

the 

controllers 

manning estimates 

would most 

would

i i

exceed their permissable limits. This conclusion is explored in greater ,.,

detail in the following paragraphs.

1. Permissable Error Criteria

The costs of error vary by circumstance and direction. Many

believe, for example, that the "cost growth" phenomenon marking system

acquisitions of the past decades is partly explained by poor cost estimating

in the early stages of acquisition. Nonetheless, this facet of many programs

has been a contributing factor to cancellation.

A theoretician would relate the permissable range of error to the

trade-off between the cost of error and the cost of reducing it. In the case

of manning estimation error, the major variables have to do with establishing

appropriate personnel pipeline flows at the right times to make required

resources available on time. Conversely, excess flows are to be avoided.

When these flows are of the wrong size, costs are incurred in adjusting them.

Costs of concern to the Department of Defense come in two forms: dollars and

readiness. These are looked upon differently by decision makers.

OSD (or CAIG) personnel responsible for questioning the estimates of

cost and support requirements for new systems look for (1) gross error in cost

estimates and (2) errors in readiness-critical quantities. If total dollars

is the only issue, an estimated error of about $50 million (in an estimated

quantity) seems to be required before decision makers are willing to slow down

the progress of a program to check for more accurate estimates. By the same

token, very small amounts of dollars may be involved in such decisions if the

variable being estimated is critical to readiness. Here the rule is that the

cost estimate may not be off by more than about 15%, regardless of how small

the total dollar amount is.

The mechanism by which these limits are employed is the application of

experience and judgment to assess cost projections from program offices. The

decision makers queried have all studied the relevant issues for many years on

a variety of systems. It was assumed that their experience provided them with

a reasonable, independent assessment of the value of specific quantities of

interest. If their assessments differ from those presented by program offices

by more than the amounts indicated above, this gives rise to a question and

request for further analysis.

To formalize this mechanism, it can be compared to a statistical test of

hypothesis. The decision maker's estimate is the population mean and the

86

. ---.- -
%

program office submission is the sample mean. The decision maker acts as if a 7_ - 15% difference is sufficient to reject the hypothesis that the sample is drawn

from the same population.

2. Results of the Analysis

The procedure followed was to drive manning to +15% of its value .... ,

in the baseline estimate by varying several different input values. The

results for MTTR, MTRR, MTBF and usage rate are shown in Figure 6-1. The

figure shows the positive error boundary for manning on the vertical axis as a ,

function of various LCC input values on the horizontal axis. The closer the

plot of each variable is to the horizontal axis, the less responsive manning

is to variations in that variable. Reading the figure, the farther to the

right a line intersects the +15% line, the greater the error in that variable

necessary to produce the fixed error in manning.

The most powerful variable is usage rate because this simply acts as a

scalar against all demand rates. Failure rate (the inverse of mean time

between failure) is the next most powerful determinant, driving manning to its

lower boundary at half its baseline value. The two maintainability variables

are the least powerful, mean time to remove and replace having a greater

effect than mean time to repair. This is explained by the fact that in the

basic data set, the latter is only one hour while the former is three hours.

This difference is accentuated by the fact that four of the LRU are coded .* ".

discard at failure, which means that they will be removed and replaced but not

repaired.

Figure 6-2 is a plot of life-cycle cost variation against manning levels

produced by changes in the four major variables. It is clear that usage rates

have a strong effect throughout the cost estimate while reliability and

maintainability statistics have a much smaller influence. Notice also that

mean time to repair, even when driven to zero, cannot produce the -15% manning

error.

Another comparison of value is the different effect of MTBF errors on

manning and spares. The error bounds on spares are reached at 86.7% and

118.5% of baseline MTBF. The corresponding manning error, however, is only

4.7% and -4.6% at these limits. To reach the +15% manning errors, the

multiples required are 66.8% and 200% of baseline MTBF. These relationships

are illustrated in Figure 6-3. Thus, it appears that for this particular

case, reasonable scrutiny of the spares cost estimate (and hence reliability)

would be likely to confine the value of the manning error well within

prescribed limits. Similar comparisons cannot be made for the maintainability

variables because they do not affect readiness-critical outputs other than

manning and training.

The remaining possible source of excess error in manning estimates is

from joint errors in input variables. This problem can be illustrated by

joint errors in maintainability and reliability data. MTRR is used for the

former and MTBF for the latter.

Figure 6-4 illustrates the accelerated effect of joint errors in MTBF

and MTRR in the presence of errors in the manning variable. The two graphs

are the locus of manning errors produced by different levels of error in MTRR,

............................ 87

. .
CL.-

LLJ 0

onc

* U

CI u.

z-

*LUJ

U-.

5> LJJ 0)

ZD - c

Z15~

IULu

04-

L'-.

'0

too

% 'NOIIVIA30O NINNYW 1N1K)d-
~~~~.~~3 .,-- - WX V-6\~ - -'T -

900

USAGE

RATE

850

MEAN TIM

S800 BETWEEN FAILURE 0

U

o ~MEAN TIME MATM TO RPAIR REMOVE & REPAIR _ 750

700

178 209 241

MANNING, no. of manyears

Figure 6-2. Response of Life-cycle cost and Manning to Equal

Changes in Other LCC Variables

89Y
+15-

SPARE S

ERROR, %

15

4• - - -4.6 

0

OD 0.6 1.0 1.5 2.0 -

z 15MULTIPLES OF

I MEAN TIME -4,7 -- BETWEEN FAILURE

Figure 6-3. Comparison of Spares and Manning Errors

90

.. .. .. . .. .. . . . .

**** - - **~ j . . . . . . - -. '

. . . . . . .---- . . . . . . . .~--- . . . . . . .--. . . . . . °.
A *. .j . *. - - - -. - -T

40

MTBF: 86.7% OF BASELINE

(SPARES UPPER BOUNDARY)

30 MTBF: 118.5% OF BASELINE

(SPARES LOWER

BOUNDARY)

20

0-

0---------- -MANNING: UPPER LIMIT

S10

z4.

- - LMULTIPLE OF MEAN TIME

---- --- ---- --- --- MANNING: LOWER LIMIT

Figure 6-4. Relationship of Joint Errors to Manning

9 1

-~ .>. - ... 7.,. ...... -<2,
7 -7

given a certain error has already occurred in MTBF. The graphs have been

drawn for those values of MTBF that produced the upper and lower boundary

errors in spares estimates.

Notice that even under the joint error condition, a relatively large

error in MTRR is required to produce a 15% error in manning. For an MTBF

(118.5% of the baseline) that produces a 15% underestimate in spares, MTRR

must also be understated by almost fifty percent (0.542) of its true value to

produce a manning estimate 15% lower than its true value. Under the same MTBF error an overestimate of 87.6% in MTRR is required to produce a 15% overstatement of manning requirement. For an understated mean time between failure

(86.7% of the baseline), the MTRR must be misestimated at about two thirds its

value (0.37) and almost a third again its real value (133.9%) to produce manning estimates off by minus and plus 15%, respectively. This one example suggests that manning may not be a particularly sensitive variable in comparison

to other dependent LCC variables.

F. RELIABILITY IMPLICATIONS

1. Internal Reliability

The four major findings of the SINCGARS replication were as

follows:

(1) There were problems in understanding how to construct the reference system.

(2) There was confusion over the selection and use of design difference indexes, perturbation values, and indirect productivity

factors.

(3) There was a lack of understanding of MOS/skill-level determination.

(4) it was apparent that this type of analysis should be performed by

a multidisciplinary team, not one analyst.

Among the causes for these findings, beyond the general problems of lack

of methodology clarity and inaccuracies caused by not having the technique

automated, were problems with the instructions, insufficient or missing supplemental input data, and excess emphasis on calculations instead of reasoned

judgments. It was also apparent that the replication deviated too much from

the training course content. Additional constraints included incorrect answer

sheets upon which later analyses were based, syntax differences in the subjective work packages, and the tedium brought on by the enormous quantity of

answers required by Work Packages 4 and 5. Interestingly, the average number

of answers that matched the HCM design team was high (78% in Package 4, 70% in

Package 6) in some of the packages. However, in the all-important Data Package 8, only 32% of the answers matched the corrected contractor response.

This is a function of both the HCM clarity problem and the snowballing effect

of previous administrative irregularities already present in the inputs of

Package 8. Even with all the problems previously discussed, the study group

appeared to understand how to apply HARDMAN to SINCGARS within the regulated

....
-°-

atmosphere of the replication. This finding was derived from the reasonably

good performance on roughly half the data packages and the apparent improvement in methodology comprehension, once the discrepancies review conference

was held. It is probably only now that the designers understand all of the

problems that existed within the replication. If the original replication

were used as a pilot study, a comprehensive and consistent reliability check

could be performed.

2. Qualitative Accuracy Study

trainin havIt seems that some of military contacts who deal with manpower and

training have somewhat consistent opinions with the results of the face

reliability and replication analyses. That is, the methodology is complicated

and hard to understand, but when understood and applied correctly, has the

potential to provide reasonable manpower and training forecasts for new

procurements. This difficulty in understanding HARDMAN negatively influenced

some of the personnel queried, or else others surmounted this obstacle and

were able to make good use of the technique. It can be conjectured here that

the comprehension difficulty experienced by the personnel contacted is similar

to the problems encountered during the replication experiment. Once the

methodological loose ends are corrected, it appears that HARDMAN could have

widespread acceptance.

3. Manpower Sensitivity Study

The sensitivity analysis suggests that manning estimates may be

fairly robust; they may not be as responsive to errors in input variables as

the other important LCC elements. Because several of these quantities, such

as MTTR and MTBF, are scrutinized for accuracy during the review process, and

because they are more responsive, it appears likely that errors in inputs will

be detected through them before they drive manning estimates out of the

permissable boundaries.

With regard to the data that drive these estimates, the results suggest

that it would be unwise to devote extensive resources to their refinement for

the purpose of improving manning estimates. The more serious sources of

significant error in manning estimates that can be isolated from this study

are the inappropriate selection of an operating scenario or deployment plan.

The TRADOC and DARCOM communities control cost and manning estimates to a far

greater extent than normally realized. By setting the number to be purchased,

the number of operating units, and the hours of operation for the equipment,

the program offices have the largest single effect on estimated quantities of

anyone in the design process. Because early in the acquisition process these

values are frequently not known with any precision, they are often considered

unimportant when, in fact, they are pivotal LCC elements. For example, a

system of which 100 will be purchased should be designed and built differently

from one ahere 10,000 will be built. As a second example, usage rate differences of plus and + 13% will produce error in manning estimates +15%. h Tese can be produced-through an error in the estimated hours of operation per year or in the number of units to be purchased. If both are off

S I
by 6.3% or more, then the joint effect on usage rate will be 15% or more and

the manning error will exceed the permissable limits. These remarks should be

taken as a caution to program offices to devote the time and energy necessary to establish reasonable, stable values for these variables. Understknding |

that the sensitivity analysis provided here is somewhat limited by (1) the

sample size and variety of acquisitions studied and (2) assumptions concerning

the use of Navy billet data as representive of the Army billet structure, the

findings are significant enough to warrant repeating the analysis once the C.C

Army has its own billet costing system completed.

94

............................................................... I.-.. V.Z . ~ .- --.. . 2 
SECTION VII

CONCLUSIONS AND RECOMMENDATIONS

A. OVERVIEW

The preceding secticons provided the findings in the four major validity

areas of face, internal reliability, event-series, and qualitative accuracy.

At the end of each section, the validity implications were assessed, along

with supporting rationale. In this last section, the findings are first

pooled and then summarized to illustrate the strengths and common major

weaknesses that became apparent after examining and comparing all the validity

implications. Next, the resolution of these common problem areas is addressed

within the context of (1) the ranking of the weaknesses as a function of the

user requirements and their impacts on the accuracy of the methodology; (2)

the practical aspects of improving weak areas in terms of difficulty and

poten'tial for other organizations besides the original designer to adopt the

technique; and (3) the reasonably low requirement to have extremely accurate

*" MPT projections. Ultimately, a condensed, prioritized list of weaknesses and

recommended corrective actions is provided. Additionally, follow-on study

areas deemed important to closing out issues that could not be addressed

within the time and budget constraints of this evaluation are also recommended.

B. SUMMARY OF RESULTS r

In total, the HCM proved to be reasonably sound. Table 7-I summarizes

the strengths. The net strengths were based primarily on the high degree of

conformance with the user requirements, the reasonable similarity to other MPT

modeling processes, and the closure of eighty-five percent of the audit

issues. It was also promising to discover that the projections did not have

to be extremely accurate, considering the remaining twelve to fourteen problem

areas that were found to potentially affect the accuracy and credibility of

*the projections. The notential concerns or problem areas uncovered, which

generally spanned all the user requirements, operational, and reliability

analyses, are shown in Table 7-2.

C. PRIORITIZATION OF VALIDITY ISSUES

Before any recommended revisions to the methodology can be made, it is

essential to examine each of the issues from Table 7-2 and weigh their

importance to meeting the Army's MPT needs and their implications relative to

effecting the accuracy and usefulness of the projections. It is important to

recognize in the near term that the methodology is reasonably sound. It is

equally important to know the major limitations and the supporting fact that

allowances can temporarily be made for some of the limitations in light of the

°"" inherent acceptable error bounds on the manpower projection. For the near and ""9

" intermediate terms it would be useful to improve those areas most important

and amenable to immediate correction, followed by a plan for improving the

remaining far-term areas, perhaps requiring more supporting data.

2. 95 '.A
77. --------

.* .

Table 7-1. Net Results of HARDMAN Evaluation

Evaluation Area Net Results

User Requirements Technique complied with majority of

mandatory user needs

Operational Analysis

Comparison with other Methodology conformed with other known,

MPT methods accepted MPT modeling schemes and data

foundations

Audits Methodology demonstrated sound logic and

reasonable results for 85% of test issues

examined (remaining issues considered

reparable in near term)

Reliability Analysis .

Internal reliability Test group correctly replicated half of

selected test points (two of four remaining

test points found to be repeatable after

clarification)

Qualitative Two of nine individuals having experience

Accuracy check with HARDMAN applications indicated a rough

accuracy of 80-90% with actual manning

requirements; remaining individuals had

insufficient experience to comment "- :

Event Series Validity Manpower variable found to be least

sensitive of all life cycle cost variables,

demonstrating good utility of existing

methodology even without near-term

improvements

96

. . . . . . . . . . . . . . . . . . . . . . . . .

.. . . .. . .. . . . . . . . . . .. . . . .. . . . . .
-- TR7 7 -T - r 777 r:

Table 7-2. Common Validity Issues

' Methodology Step Validity Issues

Step 1 Data base construction No firm criteria for selecting, altering,

and functional requirements manipulating, or rating data sources

Unclear indication of functionalrequirement level of indenture

Step 2 Build reference system No firm guidelines for constructing

and establish manpower/ reference system (particularly when modeling

skill requirements two widely varying proposed designs)

No consideration given to observed reduced

manpower at end points of life cycle

No firm range and criteria for selecting

design difference indices (perturbation

values), induced failure factors, or

indirect productivity factors

No complete means provided for equally

comparing reference and proposed

technologies

Incomplete guidelines provided for

selecting MOS/skills

Step 3 Determine training No quantitative means of differentiating

requirements between critical/non-critical tasks and

resultant impact on training

Steps 4 and 5 Determine Projection does not consider manpower

personnel requirements/ available at actual time of system

shortages deployment

Step 6 Perform tradeoff Unstructured tradeoff process

analyses and present results

No indication provided for either relative

credibility of projections or best

estimate for manpower planning

Incomplete life-cycle cost analysis

No standardized assessment of study

weaknesses and proposed resolution actions

Overall methodology Low usability by anyone other than a

well-seasoned, multidisciplinary team

9
1. "Near-term Issues

The issues having both sizeable user value and accuracy -

implications, and jointly found amenable to solution within the immediate

- one-year time frame were as follows:

(1) No standardized criteria for selecting, altering, manipulating, or

rating data sources.

(2) Lack of proper level of indenture for functional requirements.

(3) Insufficient guidelines for constructing reference system.

(4) No consideration for reduced manpower demands at system life-cycle

end points.

(5) Unequal comparison of perturbed state-of-the-art technology

against new proposed designs.

(6) Lack of guidelines for selecting MOS and skills. ..-.- .

(7) Unstructured tradeoff process.

(8) No credibility indication of projections.

(9) Non-standardized assessment of study weaknesses and resolution

actions.

Logic and credibility issues such as items (1), (3), and (5) above, are

probably most important because they affect both the accuracy and usefulness

of the methodology. Although removal of these problem areas would not

guarantee improvement in ibsolute accuracy, it would certainly provide the

users with a better subjective, relative indication of accuracy to aid

decisions on budgeting of additional hardware tests or favoring one projection

over another for planning purposes until additional information is received.

All of these items were considered candidates for immediate resolution [*. "

because elements of the solution for each of the issues often already existed

in both the HCM design team and the Navy concerning the relative quality of

various data sources (31, 40). For example, actual field or controlled

collection data probably ranks highest on the list of preferred information

for modeling new system performance because of its maturity, followed by

operational or field-test data (reasonably mature), development test data, and

free-flow or semi-controlled data collection information (limited sample

size); and finally, contractor estimates (no test or sample data).

As with the inherent experience available on data sources, it was

discovered that solution structures similarly existed for items (3) and (5)

above. The use of two reference systems to model two widely varying

contractor designs was introduced earlier in Section V-F (the SINCGARS

audit). Because the structure already existed for perturbing historical

predecessor component data to fill data gaps in the proposed system, it t.

98

- . , -- * ". ' ."- "" "-. " . - .

* * % * % • *-** . .. * . * . * * * *. .7= ,_.. L *,. . . . ,, -* ' , _ - - - .
appeared that a simple extension of the same procedure would allow a

"top-level" comparison of the total predecessor and proposed systems on an

equal basis.

The next near-term priority closely associated with credibility is the

resolution of potential error sources. Items (2), (4), and (6) serve as

potential error sources and therefore have accuracy and credibility

implications. As indicated in the preceding paragraphs, it was learned that -

much of the solution framework already existed within the joint experience of

*; the HCM design team who normally perform the HARDMAN analysis (34). The

"" reduced end-point manpower demand issue, item (4), was considered a fairly

simple error source to correct, considering that the deployment period and

life cycle could be broken down into yearly time periods to be sensitive to

the gradual buildup or tapering off of systems. Item (6) above, was another

near-term issue that was found easily rectified based on knowledge obtained at

• both the audits and discrepancy review.

The final near-term priority area is the resolution of some of the

ambiguity issues. Items (7) and (9) are somewhat linked together and are

rated as the last priority because their solutions serve more to improve the

clarity of the process and projections than to contribute directly to accuracy

and credibiity. Again, because the solution components for listing all

potential tradeoffs, weaknesses, and proposed actions already existed to

varying degrees, it was felt that these adjustments to the methodology could

be a simple enhancement to the decision-making quality of the technique.

2. Intermediate-term Issues

Some of the validity issues discovered during the evaluation were

found to have accuracy implications, but, upon closer examination during the

audits and SINCGARS discrepancy review, were also found to be not amenable to

immediate solution. Although some data and models were available and appeared

to represent means of closing the issues, it was obvious that a certain amount

of additional research and methodology restructuring were required and would

probably consume cn to two years of effort. The issues from Table 7-1 that

are relevant to this intermediate category are as follows:

(1) No firm range and criteria for selecting perturbation values,

induced failure factors, or indirect productivity factors.

(2) No consideration for manpower availability at the projected system

deployment date.

(3) Incomplete life-cyle cost analysis.

In line with the reasoning provided earlier, items (2) and (3) above

would have to be given highest priority because they both impact the accuracy

and credibility of the projections. For both of these issues modeling .

processes, introduced earlier in Section V, have already been developed and

with slight redesign could be incorporated in the existing methodology. Item

(1) is fairly important because it could represent a major error source. Of

99

* -.. *....... . ...... >. ............. ,.. . .:
the three different types of factors, the perturbation value is probably the

most crucial element. Some information presented during the SINCGARS

discrepancy review is available to help define the problem of determining what

can reasonably be expected regarding improvements in new technology

performance. The induced failure and indirect productivity factors were .--w

considered slightly larger problems because it is difficult to determine when

mistakes in selection of an arbitrary value for either one will generate a

serious error in the outcome.

3. Far-term Issues

Some of the validity issues uncovered were found to require a

substantial amount of data and time (probably about three to five years) to

resolve them. In most cases the issues, although important, only surfaced in

either the user requirements or the operational analysis and were therefore of

a singular nature. The issues fitting in this category were as follows:

(1) No quantitative differentiation between critical and non-critical

tasks.

(2) Incomplete consideration of demographic data such as age, sex, or

education.

(3) No consideration of the manpower impacts of other emerging systems.

(4) No consideration of major socioeconomic fluctuations on manpower

availability.

Probably the largest potential error source is item (3) because the

overall acquisition and planning process for new systems spans several years.

Consequently, new concepts being approved have a very good chance of

overlapping each other in their respective design and support planning

stages. The remaining items are fairly equivalent in their level of

importance.

Great care was taken in the preceding paragraphs to refrain from

including suggestions for improvements to concentrate on presenting a clear

rationale for the near-, intermediate-, and far-term ordering of the various

problem areas. Having prioritized all the various validity issues as a

function of their potential for weakening the methodology and ease of

solution, the final step is to explore sensible solutions.

D. RECOMMENDATIONS

One way to develop a plan for implementing recommendations is to first

convert each issue into a positive proposed action and then summarize the

total scope of intended actions. Table 7-3 first provides a summary plan for

improving the existing methodology, followed by a detailed discussion of each

suggested corrective action.

100)
Table 7-3. Recoimmended Plan for Improving HARDMAN Methodology

Potential for Improving Methodology

Prioritized List Near-Term Intermediate Far-Term

of Recommendations High Med. High Med. High Med.

Establish criteria for rating, X

selecting, altering, or

manipulating data

Provide indication of credibility X

of projections

Establish top-level method for x

comparing technology of total

reference/baseline and proposed

systems r

Provide better guidelines for X

constructing reference systems

Firm up guidelines for functional- X

requirements level of indenture

Provide a better structure for MOS X

and skill selection

Reduce MACRIT manpower projection X

down to one- or two-year projections to be sensitive to lower

end-point demands

Provide tradeoff a process more structured X

Establish standardized assess- X

ment of study weaknesses and

resolution actions

Project manpower availability X

out to actual system deployment

date

Provide more complete lift-cycle X

cost/tradeoff analysis

K.

IC)"1.

° p
Table 7-3. (Cont'd)

Potential for Improving Methodology

Prioritized List Near-Term Intermediate Far-Term

of Recommendations High Med. High Med. High Med.

I.

Firm up ranges for perturbation X

values, induced failure, and

indirect productivity factors

Incorporate potential manpower x

impacts of other emerging

systems

Provide quantitative x

differentiation betwen critical

and non-critical tasks

Incorporate more demographic X

data into personnel/availability

projections

Consider major socioeconomic X

fluctuations in manpower

availability projection

The following paragraphs develop each of the recommendations in

Table 7-3 by suggesting possible corrective actions. First, consideration is

given to the methodology design, followed respectively by suggested training

for potential HARDMAN analysts as well as follow-on research.

1. Methodology Design

Because the key credibility issues were primarily centered in the

near- and intermedite-term time frames, the following discussion focuses on

those recommendations that fit in that one- to two-year time span. Far-term

corrective actions are discussed in the "follow-on" portion of this subsection.

a. Suggested Corrective Actions for Data Management and

Credibility Indication. It is generally the intent of any modeling technique

to act as a guide or tool to assist decision makers. As suggested earlier in

Section IV, the Army users indicated a need to be able to apply HARDMAN as an

assist to the ASARC or program management budget decisions. Paramount to

making program direction, budget priority, and type manpower decisions is

102
understanding both the kinds and quality of data employed in the analysis. Consequently, a simple data rating matrix is suggested as a means of both

summarizing the net credibility of the projections in the executive summary of

HARDMAN applications as well as aiding the construction of the consolidated

data base in Step 1 of the methodology. Table 7-4 is provided as an example.

Table 7-4 basically applies to the proposed design. The listings of

both the weighting criteria and data source quality were based on the joint

experience of the Navy and contractor HARDMAN analysts. The table has

considerable flexibility in that the analyst can fill it out solely for the

proposed system if the predecessor reference or baseline system data are good

or evaluate both the reference and proposed systems separately if each contains suspect information. Similarly, the data evaluation can be done on a

component-by-component basis. In all these schemes the analyst merely has to

indicate the availability, maturity, and type of data employed that automatically places the outcomes in a low, marginal, or good credibility 'ii

quadrant. Once the data source(s) has been located, the remaining table

elements can be answered. Probably the final credibility assessment will span

two quadrants (i.e., low to marginal or perhaps marginal to good). Ultimately,

a summary table such as Table 7-4 could provide an overall assessment of the

data and projections to the decision makers so that MPT planning and budgeting

can always be supported by the more reliable projection throughout the system

development phases.

Processes for changing or manipulating data were also recommended as

part of the suggested improvements. Although not as complicated as the

data-rating scheme, a simple delineation of helpful hints for when to alter

data inputs (along with supporting examples) is necessary to help clarify the

data management process in Step 1 and improve the audit trail. Based on the

findings of the previously described audits, the basic guidelines for handling

data appear to be as follows:

(1) When building the consolidated data base, first check the proposed

system operator and maintenance data for completeness; if data are

missing for any components, select appropriate component data from

a siil1ar existing system, perturb the data per instructions

provided in Step 2 of the methodology, and complete the proposed

system data set (see the SINCGARS HARDMAN results for an example)

(28).

(2) All proposed system data inputs should be filtered by a group of

experts to establish confidence that the data are reasonable. For

example, basic maintenance repair times should be checked to

determine if sufficient time is allowed for obtaining test

hardware and tools by comparing them against similar tasks on

predecessor components. If erroneous failure rate, indirect

productivity, or induced failure data are encountered, replace the

data with comparable historical values (best choice) or select

appropriate values from the original minimum performance criteria

stipulated in the request for proposal, RFP (next best choice)

(see the RPV HARDMAN results for an example) (29).

103

.. .... . ..
IIW

0 in

0

o to)

w rC

0C C

I 'a" 4.*

0.0 0. 0

c. 0 e . r

w .0 4

~~104
(3) Where necessary, it sometimes saves time to use more recent

historical data in the data base because it is easier to collect.

For example, if the more recent data are component test data, then

a sample sensitivity study should be conducted to determine

whether the less mature (but easier to obtain) test data produces

significantly different projections than the older, more mature

(but harder to gather) historical data. If not, then replace the

historical component data with the comparable test data where

appropriate (see the RPV HARDMAN results for an example) (29).

(4) Occasionally, it may be difficult to find a predecessor duplicate

of a new proposed component. In this situation a larger or

smaller scale component with similar design characteristics may be

used in the consolidated data base as a last resort. However, it

is extremely important to rate the credibility of these data in

terms of both operational similarities and environment (see RPV

HARDMAN results for an example) (29).

b. Suggested Corrective Actions for Perturbing Reference System

to Model New Proposed Design. A framework for perturbing predecessor data to

fill gaps in the proposed system data base was sunmmarized earlier in the

audits. To provide the decision makers with a better means of evaluating the

credibility of contractors' projections, it is necessary to compare both the

total reference and proposed designs on an equivalent technology basis. In

support of the "evolutionary" technology growth concept, several studies have

been done which suggest that industry is often too optimistic about new

technology performance (42). Regardless of the fact that much of the overly

inflated performance is sometimes derived from inadequate testing, it seems

that a 20% improvement in reliability or performance can reasonably be

expected (41,42). With respect to Step 2 of the methodology, a simple

top-level adjustment of the predecessor reliability and maintenance data might

be structured according to the following guidelines:

(1) If the design difference indicates a positive improvement and the

operating environment for the new component is not as harsh as the

predecessor component, then employ a 20% increase in reliability

with a "less than" or "maximum" label on the manpower projection

(e.g., < 5,000 operators required).

(2) If the design difference indicates a positive improvement and the

operating environments are essentially the same, increase the

component reliability by 20%.

(3) If the design difference is insignificant, or the positive design

improvement is offset by a harsher operating environment, then

leave the reliability and maintenance data the same.

'.. -
(4) If the operating environment is considerably harsher than the

predecessor system and the design and functional differences are

insignificant, then do not perturb the predecessor data, and

indicate with good confidence that the resulting manpower.

projections represent a minimum (e.g., a minimum of 5000 operators

are required; or, the operator requirement is > 5000 people).

With these indications on the manpower projection for the updated

predecessor or baseline design, one can start to identify whether the

contractor projection for new technology performance is reasonable because the

projections will either converge or diverge. Along with the earlier credibility ratings on the various kinds of data, the decision makers have a better

means of making budget decisions in support of the revised predecessor data or

the contractor projection.

c. Suggested Corrective Actions for Constructing the Reference

System. During the audits it was discovered that the HCM designers used a

more sophisticated way of selecting predecessor components to build the

reference system than explained in the methodology. The following additional

steps are suggested to help clarify and better structure the Step 1 reference

system construction process:

(1) Include a complete listing of documents that help select

predecessor components similar to the proposed system (e.g., U.S.

Army Field Manual 24-24, Washington, D.C., 20 May 1977; Department

of Army (DOA), Aviation Electronics Configuration Directory,

December 1970).

(2) Include a listing of typical Army and industry technical contacts

(by organization and division) in various component classifications

(e.g., radios, surveillance sensing, etc.) that can confirm

similarities or differences between the reference and proposed

systems.

(3) Employ two reference systems if two proposed systems, although

functionally similar, employ completely different technologies

(e.g., a Howitzer versus a rocket launcher).

d. Suggested Corrective Actions for Functional-Requirements Level of

Indenture. Similar to the preceding reference system suggestions, the audits

revealed that the HCM designers employed a decision process which, although ."

simple, provided more functional and task requirements guidance than the

published methodology. It is suggested that the following steps be included

in Step 1 of the methodology to better define the level of indenture of the

functional requirements:

(1) Go to a level in which the associated component tasks stop

branching off (i.e., new tasks requiring different people cannot

be identified, such as a mechanic who can repair both the truck

engine and the chassis).

I 6 .%°
p.

(2) Go to a level in which no further technology differences can be

delineated between components.

(3) Go to the same level as available sample predecessor component

maintenance data in the consolidated data base.

e. Suggested Corrective Action for MOS and Skill Selections.

The audits discovered that the cookbook approach outlined in the methodology

was not complete. The additional step not included in the methodology, which

ensures the proper matchup between tasks and skills, is the use of experts

from various proponent schools. Therefore, it is suggested that the

methodology include a list of Army proponent school contacts (for all types of

component groups and technologies) that can confirm or alter the task and

skill selections as necessary.

f. Suggested Corrective Action for Potential End-Point Manpower

Errors. The solution to the end-point manpower shortage problem was outlined

earlier in this section. The basic solution revolved around shortening the

intervals between manpower projections, particularly when considering manpower

demands of emerging systems during the early acquisition or later life cycle

phases. Figure 7-1 graphically displays the solution by showing MACRIT projections at one-year increments during periods of rapid changes in deployment

and attrition, and the typical long term MACRIT rectangular projection for the

mid-cycle, steady-state period.

g. Suggested Corrective Action for Structured Tradeoff

Process. To instill confidence in the reader that the methodology encompasses

all possible tradeoffs, a simple table could be constructed as shown in

Table 7-5. Table 7-5 could be placed in the executive summary as a means of

displaying (1) the range of potential tradeoffs that the methodology could

provide and (2) the actual tradeoffs conducted for a particular application

POTENTIAL ERROR - EMERGING SYSTEMS

INDUCED BY NOT

SHORTENING PRO- STEADYJECTION INTERVAL STATE

MAC RI T

Z SOLUTION LATE

S- TAPERING

3::: OFF

0 / 0MACRIT SHORTENED -

WPROJECTION _

EARLY 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

BUILDUP LIFE CYCLE, yr

Figure 7-1. Revised MACRIT Manpower Projection

2- -
- "V. T

Table 7-5. HARDMAN Methodology Tradeoffs S...

4 .0 . .

I-', 4, ,, 

LIFE-CYCLE COST ' "4, . TRADEOFF ," -.-.-. PARAMETERS kT

DECREASE BUDGET CEILING

DECREASE RELIABILITY .......

'"" ":: ............ ', "- "

DECREASE SPARES ............ o; ..

DECREASE MANPOWER

DECREASE SKILLS

........ ii... ..... DECREASE TRAINING iiiii!"'::

DECREASE TECH PUBS

DECREASE SSE

DECREASE TOOLING

as constrained by time or the sponsoring agency. The actual tradeoffs

explored would be indicated by placing X's in the appropriate slots on the

matrix. It should be noted that Table 7-5 is also useful if it is desired to

display a clean summary of the various parameter sensitivities. Typically,

the sensitivities would be indicated by a relative percentage change (e.g., a

1% decrease in reliability results in a 10% increase in manpower). These more

detailed cost sensitivity relationships could be summarized by placing the

percentage "deltas" in the appropriate matrix slot. For example, continuing

with the reliability illustration, one could read horizontally across the

matrix and immediately see the net impact on the other LCC variables. This

more detailed cost-sensitivity matrix could be used to cite pivotal cost and

tradeoff benefits and jointly close out Step 6, tradeoff considerations, of

the methodology.

h. Suggested Corrective Action for Summarizing Study

Weaknesses. Although listing study weaknesses is a simple procedure, it is

extremely important information for decision makers. In conjunction with a

I:::8
" . . - --

systematic listing of weaknesses, it is also important to suggest possible

[ .resolutions. For example, it is perfectly feasible that both the reference

and contractor manpower projections could have "low-to-marginal" credibility

ratings assigned because of poor or insufficient data. Consequently, the data

problems would be listed as weaknesses and a suggested resolution might be to --.

fund test programs for the major components that are driving the projection

credibility down. A simple, dual-heading table consisting of "Major Study

Problem Areas" and "Suggested Resolution Actions" could be placed in the

executive summary to assist the decision makers. This adjustment would be

fairly minor because the 1CM designers already address problem areas in some of their applications (see the HARDMAN SINCGARS application) (28).

i. Suggested Corrective Action for Projecting Manpower Out to

System Deployment. Examples of manpower regression models were provided

earlier in Section V. The key problem identified with these models was the

lack of any indication of uncertainty in the manpower availability

projections. Nevertheless, it is extremely important to compare the projected

manpower requirement against manpower available starting at the point of

system deployment. A simple initial approach to this dilemma might be to

employ a linear regression technique, such as Barnes' Soldier 90, and allow an

end point spread representative of only 75 or 80% confidence (i.e., select a

confidence interval low enough to prevent too large a spread at the projected

system deployment time) (43). The final projection may appear as shown in

Figure 7-2, where the shaded area might represent the zone of confidence in

which 75 or 80% of the observed data points center around the least squares

projection (43).

If the projected manpower requirement resulted in the situation depicted

in Figure 7-2, then obviously there is potential for a critical manpower

shortage. However, if the projected requirement fell within the shaded area,

then for purposes of early planning, it would appear that there

CoACTUAL MANPOWER --- PROJECTED O AVAILABILITY MANPOWER

Z:R REQUIREMENT

J /PLANNED "--

STARTING SYSTEM MANPOWER DEPLOYMENT

1 2 3 4 5 6 7 8 9 10

PHASE 0 TIME, yr

Figure 7-2. Use of Confidence Intervals for Manpower Projections

.-.'-4-"

109

%%
was no immediate problem. The advantage of this approach is that the

projection can be made with greater confidence as the deployment date is

approached and more data points on manpower availability are obtained.

Probably the actual confidence in the projection would not have to be any

better than one minus the allowable error on the manpower projection (i.e.,

roughly 0.8, as determined in Section VI).

j. Suggested Corrective Action for Improving LCC Tradeoff

Analysis. As shown earlier in Section V, some life-cycle cost elements are

"- already included in the HCM. One of many typical life-cycle cost models was

also briefly summarized (i.e., HLCCM). It is merely suggested here that other

life-cycle cost models be reviewed and the remaining missing cost parameters

be incorporated in the methodology to form a complete cost and impacts

assessment.

k. Suggested Corrective Actions to Firm Up Perturbation,

Induced Failure, and Indirect Productivity Value Ranges. The 20% upper bound

on the expected performance and reliability improvement of new technology was

discussed earlier in this section. This expected improvement, however, is

only one part of the complete picture. The previous discussion also pointed

out the possibility that a different operating scenario or harsher environment

could drive the reliability of a new design below that of its predecessor. At

a top level it was merely suggested that the final manpower projection be

stated as "greater than X number of people." The next step would be to try to

establish an approximate estimate of the potential drop in reliability. A

threefold approach is suggested to obtain this estimate:

(1) Studies such as the earlier referenced Dobbins paper suggest

roughly an order of magnitude difference between contractor

estimates of reliability and actual field operation (42). It may

be enlightening first to determine the components involved in the

above Dobbins reliability tests, choose equivalent predecessor

components, compare the operating environments in terms of

relative harshness, and calculate the incremental difference

between the respective reliabilities to get an idea of whether new

technology (operating in a different environment and adjusted to -.

its realistic performance level according to Dobbins) experiences

a reduction in reliability below its predecessor.

(2) As a next step it may be useful to conduct a survey of program

managers in all four services to gather examples (and maintenance

data) of systems and components where the reliabilities fell far

short of their original goals. Again, establish the equivalent

predecessors that were replaced by those new designs, compare the

harshness of the operating environments, and calculate the

incremental difference in reliability between the two. Several

different kinds of components should be examined to determine

which technologies are more sensitive than others (i.e.,

computers, propulsion components, firing devices, etc.). p., ,,

* .. . . . . . . .

-.. *:% .....'V" -
(3) Finally, establish a group of experts from various technical

fields in DOD to adjust the estimates considering that research

and development programs provide engineering fixes that allow some

of the reliability to be regained before a fairly steady-state

reliability level is reached.

The induced failure and indirect productivity variables can effect the

outcome of the manpower projection even though their ranges are not as broad

as the perturbation value. As a first step it is suggested that a sensitivity

analysis be conducted, using completed HARDMAN applications, by simply

"- calculating the spread in manpower for the complete ranges of both variables.

. If the spread does not aggravate the sensitivity of the manpower variable,

* then a single, perhaps mid-point, value should be selected to remove any

confusion in picking a value from the possible ranges. If the spread exceeds

the allowable error, then more detailed breakdowns using industry and services

industrial engineering studies (for indirect productivity factors) or the Navy

200,000 Hour Test Study (for human induced failure rates) will have to be used

(41).

2. Training of HARDMAN Analysts

One of the major results of the audits and reliability analyses

was that the methodology appeared usable by only a select group of

individuals. Comments made during the debriefing following the internal , -

validity test universally suggested that no one individual could exercise the

methodology reliably; a team effort was absolutely required. It was

discovered that the net expertise required was a combination of the following:

(1) Sufficient military experience to provide both an understanding of

where to obtain training and technical information, as well as the

structure of the military logistics support, personnel and %

training environments.

(2) Technical capability in at least one of a wide range of technical

fields encompassing mathematics (particularly modeling),

behavioral sciences, aerospace engineering, mechanical

engineering, electrical engineering, and chemical engineering.

(3) Experience in computer information and data management systems.

(4) Experience with DOD personnel assignment and tracking processes.

Again, based on comments made by replication analysts during the

*" debriefing and discrepancies review (41), it seems that the most suitable

combination of the above abilities was either items (1), (2), and (3), or

*items (1), (2), and (4). Note that the joint military and technical

* experience recur in both combinations. The personnel experience could be an

extension of the military experience; and, similarly, the computer information

system experience could greatly enhance the technical experience. All of

these attributes exist to varying degrees in the individuals employed by the

HCM contractor who execute the HARDMAN methodology (31). While some of the

first-year methodology design recommendations are being explored, the Army

* . * * ,
'* - - - - - . '-'- .2' . C .~~~ *~ . . ;Y

''

might consider using these qualifications to select and staff its HARDMAN

"" analyst team(s). In preparation for possibly conducting in-house applications

of the methodology or managing and auditing contractor applications, it is

extremely important that the Army firm up the various judgment areas per the

preceding recommendations. This effort will also provide for a clearer

training foundation. Drawing on both the results of the preceding evaluation

and debriefing comments, the following training suggestions are provided:

(1) The original objective of the HARDMAN training session (in

preparation for te internal validity test) was to concentrate on

developing skills for the judgmental portions of the methodology.

This objective should remain the major training thrust because the

algorithms and number manipulation are fairly straightforward.

Comments made on the debriefing questionnaire suggest that the

first training session concentrated too heavily on the numbers and

data manipulation. r

(2) The handouts (particularly the LSA data and examples) should be

designed and their presentation associated more clearly with each

step and judgment in the process. The emphasis should be placed

on defining the judgment process and, where calculations are

required, concentrate on only a few select cases that demonstrate

the possible range of judgments that may be encountered. Note

that the suggested corrective actions made earlier in this section

could help refine the judgment process and that some of the

peculiar data management, reference system construction, and

credibility problems discussed earlier as part of the SINCGARS,

RPV, and DSWS audits could be employed as example problems.

(3) Break down the process by areas of expertise and provide different

focuses for the backgrounds of various individuals (i.e., teach

the team approach and define participant responsibilities).

(4) Train for the complete six-step methodology (e.g., as part of the

training program it may be useful to have the Army analysts work

as apprentices at the HCM contractor's facility as a means of

obtaining experience).

These recommendations should not be construed as only applying to

HARDMAN analysts. Even if the Army only chooses to monitor contractors

performing applications in response to a Data Item Description (DID) in a

contract, it is still imperative that contract monitors have the recommended

training to ensure quality control of the outcomes and answer ASARC or Program

Office audit requests.

3. Suggestions for Follow-on Research

There were four far-term areas identified earlier in this

section. These areas revolved around consideration of emerging systems

(horizontal analysis), quantitative differentiation between critical and

non-critical tasks, expansion of the demographic data base, and consideration

of major socioeconomic fluctuations. Except for horizontal analysis, the

.". 7
remaining areas are probably not critical to early Phase 0 and Phase I

top-level budget planning. As a first step, it is suggested that the Army V. .

initiate appropriate directives to require early planning of "most likely" -

candidates to be approved as emerging systems, inclusive of early operational .

and projected logistics support data. It is recognized that this is a large

. and perhaps sketchy task. However, if the Army wishes to meet its user needs

and consider major drains of other systems on the personnel pool, it must

*, start the mechanism to gather the needed information as soon as possible.

The three remaining areas are considered far-term because they represent

large data-gathering activities. For example, the quantitative critical and

non-critical task differentiation effort might be accomplished by (1) first

establishing criteria for critical and non-critical tasks (e.g., life or

mission endangering, task difficulty); (2) selecting an array of systems

representative of obvious task extremes for operator or maintainer performance

(e.g., aircraft pilot versus an RPV operator); (3) calculating the ratios of

instructor hours for the extremes; and (4) developing a simple rule of thumb

to increase instructor contact hours by "X" percent in Step 3 of the

methodology for those tasks judged critical. The time-consuming aspect of an

effort such as this is the task sorting and rating study associated with item -

(1) followed by the gathering of system examples and instructor task hours

associated with items (2) and (3). It is for these reasons that the "obvious

task extreme" qualifier was suggested as a starting point.

The completion of the demographic data base is a massive effort because

-" it not only involves the gathering of data on age, sex, education, etc., but

also requires expansion of the decision network in Step 2 of the 1CM (i.e.,

skill and MOS assignments). It may be worthwhile for the near and

intermediate terms not only to require manufacturers contractually to adher '.7

strictly to accepted human engineering guidelines but also to establish a

design auditing mechanism to ensure strict compliance with sound human

- engineering standards.

The last area, socioeconomic impacts on manpower, should be pursued in

.. the manner expressed by Enke earlier in the document. In the interim, a

subjective weighting mechanism might be employed to indicate the general

. direction of manpower trends (up or down), based on expert experience with the

various variables (such as the economy). Referring back to the earlier

recommendation on the use of confidence intervals around the long-term

manpower availability projection, the final manpower projection might simply

indicate that the most likely availability population would be either above or

below the least squares linear projection, depending on the historical effect

of the socioeconomic variables.

The final far-term research area refers to one of the original

constraints imposed on this evaluation. Due to time and budget constraints,

- only a qualitative examination of methodology accuracy was feasible. It is

strongly suggested that a feedback loop be constructed so that actual field

data on completed applications are gathered as the new systems are deployed.

If the actual system MPT performance falls outside the allowable error

discussed in Section VI, then appropriate steps should be taken to backtrack u

through the studies and strengthen those weak areas in the methodology.

:" 11 3. . . ::*. 

. . . . . . . .. . .. . . . . .. . . . . .

• .'.-,"...-.---' ,: : ,C-, ." .'_.. .-- '-. .'.,' - .'.'" . .''''.-' .-.. ",- ..-..- -'. "-'..'-.,-. ..-. .,.-"... ... -. ,.-.. .-.-. -. ,'.. .-. -,. '
1. WA

In conclusion, several different validity examinations were structured

to evaluate the HCM. Table 7-1 indicates that the technique was generally

sound and basically integrated all aspects of the MPT problem well. Although ' .4

some weaknesses were discovered, solutions (or solution structures) were

apparent in most cases and appeared attainable within a reasonable time

period. If the Army chooses to pursue the proposed recommendations, several

benefits might be realized:

(1) The Army will move closer to developing its own in-house HARDMAN

capability (if it elects to do so).

(2) The Army will improve the audit trail on MPT projections.

(3) The Army will have a sound tool for producing MPT requirements,

weighing the credibility of projections, and making better

personnel recruitment and budget decisions in the manpower support

arena.

114

A

............

p:-:
REFERENCES

1. "Major System Acquisitions," Department of Defense Directive 5000.1, .

March 29, 1982.

2. "Major System Acquisition Procedures," Department of Defense Directive

5000.2, proposed revision, October 30, 1981. '. .*

3. Ritchie, M., "Manpower Implications in the Design of Air Force Equipment," National Technical Information Service (NTIS), February 1974.

4. Eckstrand, G., "Technology Projection: Manpower and Logistic Factors in

Weapon System Development," NTIS, August 1980.

5. Thomas, G., and Butler, R. "An Informal History and Critique of the

Navy's HARDMAN Program," working paper, December 1982.

6. HARDMAN Methodology Handbook, Dynamics Research Corporation, Vol. 1-3,

November 1980.

7. Training Requirements Determination Handbook, PACER Corp., Vol. 1-5,

June 1982.

" 8. Butler, R., "HARDMAN Cost Model System," Assessment Group Co., Santa

Monica, California, Volumes 1-3, August 1981.

9. Rose, Col. Louis, "HARDMAN Data and ASARC I Boilerplate," Memorandum,

Soldier Support Center, Soldier Development Directorate, March 23, 1983.

10. Wildrick, Maj. Edward III, "Industry Survey of MPT Needs," Briefing in

Minutes from Tenth Meeting of Department of Defense, Human Factors

Engineering Technical Advisory Group, El Paso, Texas, March 8-10, 1983.

11. "Weapon System and Equipment Support Analysis," MIL-STD 1388A, November

1981.

12. "TOE Manpower Authorization Criteria (MACRIT)," AR570-2, December 6, 1974.

17

13. Fong, F. K., et al., "Unmanned Spacecraft Cost Model," Air Force Space

Division, June 1981.

14. Gordon, P., et al., "A Review of Underground Mining Equipment Research

and Development," JPL Document 5030-482, November 1, 1980.

15. Quade, E., and Boucher, W., Systems Analysis and Policy Planning,

Elsevier Publishing Co., New York, 1968.

16. Putnam, G. H., "Example of an Early Sizing, Cost, and Schedule Estimate

for an Application Software System," Proceedings of Third Summer

Software Engineering Workshop, NTIS, September 18, 1978.

" ..-% 7 -
17. Winston, M., et al., "A Preliminary Report on the Early Comparability

Analysis (ECA) Methodology," Soldier Support Center, Regional HQ,

Alexandria, Virginia, undated.

18. Enke, S., "Army Manpower Prediction System (AMPS), Final Phase I

Report," NTIS, February 15, 1973.

19. Holz, B., "The ELIM-COMPLIP System of Manpower Planning Models,

Vol. 3, User's Manual Part 2," NTIS, December 1973.

20. Safeer, H., et al., "A Computer Model for Projecting Civilian and

Military Manpower," NTIS, February 1966.

21. Doong, L., "A Manpower Model to Determine Future Inputs to NEC335X and

NEC336X Billets," NTIS, December 1981.

22. Drummond, H. E., and White, F., "Sources of Entrant Requirements in

Manpower Models," Growth and Change, Vol. 7, No. 3, NTIS, July 1976.

23. Leeson, G., "A Projection Model for Hierarchical Manpower Systems,"

Journal of Operational Research Society, Vol. 31, No. 3, NTIS, 1980.

24. Lindsay, W., et al, "Simple Regression Models for Estimating Future

Enlistment and Reenlistment in Army Manpower Planning," NTIS,

September 1970.

25. Barnes, H., et al., "Soldier 90 Demographic Data and Trends," NTIS, October 1982. e"

26. Grissmer, D., et al., "DOD and Navy Manpower Supply Scenarios Through

2000," MATHTECH Inc., NTIS, June 10, 1977.

27. Liberati, G., "Technology for Acquiring Supportable Systems:

Managerial Overview of Test and Evaluation," Westinghouse

Integrated Logistics Suppport Div., May 1983.

28. "Application of the HARDMAN Methodology to the Single Channel

Ground-Airborne Radio System (SINCGARS)," Dynamics Research Corp.,

Technical Report E-8735U, September 1983 (only results of Steps 1 and 2

audited).

29. "Application of the HARDMAN Methodology to the Army Remotely Piloted

Vehicle (RPV)," Dynamics Research Corp., Technical Report R-408U.

Vol. 1 and 2, January 11, 1983.

30. "Application of the HARDMAN Methodology to the Division Support Weapon

System (DSWS)," Dynamics Research Corp., Technical Report E-8247U,

Vol. 2, December 1982.

31. Zimmerman, W., and Franklin, M., Technical Audit of RPV and DSWS HARDMAN

Applications, Dynamics Research Corp., July 18-22, 1983.

32. Joint JPL, Army Research Institute (ARI), and Soldier Support Center

(SSC) SINCGARS Replication Effort (seven individuals), May 9-27, 1983.

-:.

..................................... .........-*...
33. Knight, A., Advanced Lightweight Torpedo (ALWT) Logistician, personal

communication, November 17, 1983.

34. Wright, J., Lt. Cdr., ALWT Integrated Logistics Manager, personal

communication, October 11, 1983. -w -.

35. Vroom, 0., ALWT/SIRCS MPT Logistics Element Manager, personal .

communication, October 6, 1983.

36. Horn, G., Sgt., Submarine Combat System (SUBACTS) Training Manager, ,

private communication, November 10, 1983.

37. Fagae, R., Destroyer (DDGF) Personnel and Training Analyst, personal

communication, October 11, 1983.

38. Ritter, W., Capt. Flight Trainer (VTXTX) Project Manager, personal

communication, November 17, 1983.

39. Preidis, R., Airforce Operations Research Analyst, ASSET, personal

communication, November 30, 1983.

40. Lugen, D., Lt. Cdr., personal communication, Assistant Manager, Navy

HARDMAN Office, Washington, D.C., April 4-7, 1983.

41. Joint meeting of JPL, Army Research Institute (ARI), Soldier Support

Center (SSC), and Dynamics Research Corporation (DRC), SINCGARS

Discrepancies Review Conference, August 22-24, 1983.

42. Dobbins, D. A., "The Dangers of Relying on Industry as a Partner in

Material Development," Army Research, Development, and Acquisition

Magazine, Vol. 23, No. 2, March-April 1982.

43. Miller, I., and Freund, J., Probability and Statistics for Engineers,

Prentice Hall Inc., New Jersey, (pp. 231-234) 1965.

117

%
T-7-7E 7-7-

APPENDIX A

INTERVIEW QUESTIONS FOR USER REQUIREMENTS

A-i
USER REQUIREMENTS

QUESTIONNAIRE

1. To what organizational division of the Army do you belong?

VERBATIM

2. What is the primary charter of your organization?

VERBATIM

3. What are your specific duties within the organization?

RESPONSIBILITY: (Areas of Concern, Deliverables, etc.)

4. What are your key needs in making or using manpower personnel and training

projections (i.e., what do you need to have to do this?)

INPUTS: Type of data, Level of Detail

INTERFACE CONSIDERATIONS

Other Organizational Actors (input/output)

Timing

5. What are the key drivers in needing such MPT projections, e.g., cost,

planning, safety, performance, readiness/force modernization, etc?

SPECIFY AND EXPLAIN, if possible

6. Which ARMY MPT regulations/documents pertain to your organization's

involvement in the M.P.T. process?

7. What technique is presently employed for getting MPT inputs into system

acquisition process? -

8. Who has responsbility for this techniquwr

9. To your knowledge, does it provide good results?

SPECIFY

10. In your organization, who is (are) the most likely user(s) of a

methodology to make MPT projections?

A-3...%

.. .. . .. . . .. ... .-. . .. . . . .
11. What other users of the the methodology could you identify within the Army

organizational structure?

12. Who in your organization is (are) the most likely user(s) of the results

of such a methodology?

I _I

13. What other users of the results of such a methodology could you identify

within the Army?

14. What suggestions do you have for the design of a methodology to accomplish

HPT projections? (OUTPUTS, SCHEDULE, PROCESS, ETC.)

15. What would be the most useful format(s) for the results?

16. At what point in the acquisition process (i.e. Phase 0, I, etc.) do you

require HPT results for (a) design changes, or (b) cost control?

A-4

.............................
APPENDIX B

USER REQUIREMENTS SUMMARY

MATRIX OF QUESTIONNAIRE RESPONSES

B-1

.- - , .. . .
. . .. . . . . . . . . . . . .....-.

,, 

.' • ,-

iII Ii - - ": 

a am

g! ! I ; = =-

~ - I___,. '.-"-

~ I - , =

I. I HE =

B--2
* d

-

>1.-

i ".;-° V

* 0

I-I I.-. . iiJ... ~zI i- -. i '

Cl ,, il "

i ! E~ i '-"

ii - U • •

El u :°

.' .-" " ."' "- ." ." -. _ . - " .% ' ' '_ ..t ' '' ' ' . ..' '' ' ' ' ' .-q / '.. -. .- . " _Z _J _ ' t ' : ' . _ ._ .-:.I. ( _ ' ., . ,
HARDMNAN USER REQUIREMENTS SUMMARY MATRIX (OUTPUT)

sums b " mm ucAW

uM 1 ~wI'Uuwm Its U11 w

orsAAToolilMAiUaiE TASK OAm ASSOCITS WITH A SYSTEM AND ITS SH#9ORTI TEST EDUTUEST atsciWTmS S x x AND FIGUIEHS1

2. MAINTENANCE NMINIAiPOWSS KOM SAMES es I A x x ACCURATE AN DATA

3 IMOS. $KILL LEVEL NGUWUS. A~IUUA SKILL INDICATORS Int. AND TOTLW4hIMS It SPACES x x x x

4. MAWYISAOc COCEPT. INCLDIN NAWULUUi ISLS. A x x I x MTHA. MYHEF

-CLASSES HEOIIIHS SIHITIUNDftSSTITTIHAA x -TRAINING TIME. FEUESY A I -QUANTITATIVE AND DUALITEIVE TINING INCESES S S

TSMSIHSWPPSHT AES INSTRUICTIOS AS. ITES) x X.

I. PEISONNEL AIEASIICSUAITS x X I PERSONNEL PSDJSCTIOHS. TOMS x

7- UIPT COSTS ASSOCIATED WITH SYSTEM OPERATIOH ASO It I x x x

SUPPORT

H. TRADEDFF ISHETMEN HABOIDD AND PEOPLE x I K I .,. 1

I. ACCURATE ISA DATA, HSil 04 COMPARSON TO REPLAED x I x S

SYSTEM

IH HNOISMIA SGM NASE WPMT CAFAIKITTI

12. USIMAL (TMAL 8YSTM AST 90UI WY S x K

1- 1W? "1ATA SUES WUS T OF BSIN HUIT OF UWITIihWP Off. AS A BATTALION. A

I1*565 SHUNS So ESCRITIO A 4MASS ANS SEAM LEVELS

SUL$ (11111411 TEADY TiS SEl- tAI

As.BmS POmS Sxtcml r

II ISICATS TPETS1DA 111Ai S AD I A~lC~C 155THSY ASED NOCIHOS T M A I1211

17- V~ RHEA INOMATION S MS.135.HUUK

II S fLA M FOMFIN SYTSTOVRY

2P- RAMOSPY AM CRT WEPLX CAPOAIITY WINAAIH ANHEVWI

23. EXECUIVESiUMAY HEM, TO ACCOMPANY eIIPUsS

24- OUTPUT SIULS ISER SUMD 0 LETTER OF AiUUHHT 60111

URELECTS ITEGRATED SYSTEM SIWoI 0SSI FOCUIS

B- 4
HARDMAN USER REQUIREMENTS SUMMARY MATRIX (OUTPUT) (Continued)

man MAN fK WO nNTANI KMM VNP T " 00 AIO

x x

xX X

x x x x x

xX X

x x x x

x

x K K x

x

x x x

x x x

x

x x x

x A

K K B-5

NK
APPENDIX C

AUDIT QUESTIONS

C-1
METHODOLOGY AUDIT 22]

Generic

Steps 1 and 2

. What judgment determines the level of indenture necessary in determining

the functional requirements?

2. How does one determine the maturity of the data presented?

How does on ensure selecting the most mature data source(s)?

3. The judgments used to determine manpower drivers in the design

difference index are not clear to me:

How are the key drivers flagged?

How is the perturbation value assigned?

When and where is the perturbation value applied?

4. What judgment is required and how is it applied to select the lowest

skill level MOS for any maintenance task when the AR611-201 seems to

define the performance of each MOS quite clearly?

5. What judgments temper the use of the indirect productivity factor? Is

this a blanket application or a selected application? Define why and

when.

6. What judgments allow the incorporation of two tasks, normally performed

by two separate MOS's, into one MOS?

7. What judgments are used in creating a new MOS?

8. When and where are the perturbation values applied to the reference

system to achieve a baseline system with which to compare the contractorproposed data?

9. Is the function to address the manpower loading a rectangular or weible

curve?

10. How is the accuracy of the reference system ensured when contractorproposed data is used in building a reference system component for which

no predecessor data is available?

11. How do you construct a reference system to be similar to the proposed

system when the proposed systems vary widely?

12. Does the HARDMAN Methodology take into account factors other than those

addressed by the contractor? Are the DRC reference system manpower

requirements more credible than the contractor-proposed requirements?

C-3

,. . -.......... .-. .... '-:- .'-::.- . ". °'''..'. " .:.-. . " .- . , -.-. : ". - . . . " ," * -- ,°" : -, - -. , ;, '.°,:,'< , ":,%--' '
METHODOLOGY AUDIT

Generic

* Step 3

1. Is the training necessary for the instructors taken into account?

Training for the instructor's instructors?

2. Are the costs of training the instructors taken into account? The costs

of training the instructor's instructors?"-

3. What judgments are used in identifying and constructing training courses

for any new ratings developed?

4. When the technical factors for determining the baseline skill levels are

analyzed, is memory decay taken into account?

5. Explain the two levels of indenture for determining training

requirements; the Methodology seems to address only the first.

6. What determines the frequency or need for refresher (retraining) courses?

7. How does the TRRA estimate the cost of devices required for training?

8. Are the costs for personnel movement between training schools and

training periods included in training costs? Are these large costs?

9. How often is it required to send personnel back through the training

schools? Is this a large cost? Z-

*10. How are the perturbation values determined in steps 1 and 2 applied to

the reference system training courses?

C-4
METHODOLOGY AUDIT

Generic

Step 4

1. How are the personnel required for new MOS's accounted for?

2. Where is the flow of second enlistment personnel taken into account?

3. How is a personnel flow conducted for a new MOS when no historical data

is available?

4. Where do the probabilities to calculate the flow of personnel come

from? How are they applied to the MOS populations to establish the

expected values of availability?

5. Are major socioeconomic fluctuations considered?

6. Is a steady-state population within the Army assumed?

7. How are the perturbation values determined in steps I and 2 applied to

the reference system personnel requirements?

Conclusions

1. Why is a Life Cycle Costing (LCC) model not included in the Methodology?

2. When, if at all, is a wartime scenario addressed?

3. How much credibility is lent to the results of the analysis?

F2

C-5

=-- ',-."-: -'-'-''-,''.-.,._-"-'2.............-.......-.'.-....'.'....-.-..".-......'-.-..........--..'..'..-.....i.-'-. ° .' " " --",-°
- -.. ,.,... 

METHODOLOGY AUDIT

g DSWS/RPV-Specific

* Steps l and 2

DSWS

1. What justification does PACCAR have for utilizing an "M-2-like" chassis

in a potential overload situation?

2. What accounts for the vast differences in manpower between the Norden

M109E4 system and the reference and other proposed systems?

3. Why weren't personnel and training analyses performed on the PACCAR

system?

4. The Norden M109E4 system has no Ammunition Resupply Vehicle (ARV). The

PACCAR system uses only one ARV per two Howitzers. How do these two

systems get considerations as proposed systems if the functional

requirements were developed properly? Do you consider the fact that the

PACCAR system uses one ARV per two Howtizers a potential error source?

5. Why was a shipboard gun autoloader chosen for the reference system?

6. For the following subsystems, what judgments were used in determining

the design differences? What judgments were used in assigning a

perturbation value? How were these values applied to the reliability

and maintainability data? How did they affect the manpower projections

for the reference system?

a. Autoloader p

b. Fire Control Processor

c. Internal Navigation System

d. Auxiliary Power Unit

e. Suspension: 1. Howitzer

2. ARV (where applicable)

f. Howitzer Engine

g. Suspension/Chassis Combination

7. In building the reference system, what criterion were used to determine

the maturity of the data?

8. How does the use of a trailer with the FMC system affect the reliability

and maintainability of that system? How does the use of a trailer

affect the munitions-loading capabilities of that system?

9. What accounts for the vast differences in manpower projections between

the reference and proposed systems in the following MOS categories:

31V, 31E, 35E, 63D.

10. What accounts for the extremely low number of the following MOS's in the

Norden M109E4 system: 31V, 31E.

C-6

__N* 21: -
METHODOLOGY AUDIT

DSWS/RPV-Specific (cont'd.)

Steps 1 and 2

DSWS (cont'd.) -

11. Why was an airborne unit selected aa the reference navigation system?

12. What justification exists for Norden and FMC systems not using an

Auxiliary Power Unit?

13. How was the H-i tank Auxiliary Power Unit data altered to take into

account its lack of predicted performance?

RPV

1. Track the source of the Mean-Time-To-Repair (MTTR) values of the

reference system for the following subsystems:

a. Data link terminal (airborne unit)

b. Airframe assembly

c. Launcher

2. Identify the reference equipment and selection rationale for the

following subsystems:

a. Central Processor Unit (CPU)

b. RPV control and display panel

c. Airframe assembly

d. AV engine

e. Launcher

3. Track the Contractor-Furnished Equipment data from the baseline system

equipment that was used in the reference system for the following

subsystems:

a. Launcher

b. CPU

c. Airborne unit data terminal

4. Identify the subsystems of the baseline system that did not meet the

functional requirements.

5. Identify the baseline equipment that used altered contractor data.

Explain the rationale for altering this data.

C-7

:- " -L-- - - - *-'. . .' . . . . . . ... ... .... . . . .L" .._- . . .. .... -, -,-_ '. .. , - . -
METHODOLOGY AUDIT K:.

DSWS/RPV-Specific (cont'd.)

Steps 1 and 2 V

RPV (cont'd.) - -

6. Identify and provide the rationale for the perturbation values (PV)

assigned to the following subsystems:

a. Airborne unit data terminal

b. CPU

C. AV airframe

d. AV engine

If no PV was used in the above subsystems, identify where the PV was

used.

7. Identify baseline equipment where no reliability and maintainability

(R&M) data was available.

8. Track MOS/Skill level assignments for the following subsystems:

a. Airborne unit data terminal

b. CPU

c. Airframe assembly

d. AV engine

9. Page 48 of Volume I, RPV states that the assumption is made that the MAC

charts exclude the make-ready/put-away times, a value which would

normally be adjusted by the Indirect Productivity Factor. In the

SINCGARS replication, the instructions stated that the make-ready/

put-away times were included in the MAC chart data. Explain this

discrepancy.

10. Explain how full-sized aircraft data was altered in order to meet

miniaturized aircraft data projections.

II. Page 75 of Volume 1, RPV makes reference to certain modeling equations.

Explain these.

12. What is the basis for a two-to-one improvement of the M939 truck over

the M809?

13. What criterion are used in determining the number of systems to be

fielded? Are these criterion adjusted for system availability?

C-8

- . * * -.. .**p.* ~ *~_-...--

p. . ~ - * ~-.--.-:-* - -.

-" -:.: -",' :, .- '-: .-.- .-.... ,' - -,- :" : '. ."' ", .-.... "." .'" c ; .-...-.. - € ,,'., '-,"- ... '- . .... . '-,"-,.. .,'... .,.. . .-.-..-... . . . . . . . ..- " , -- "-
METHODOLOGY AUDIT

DSWS/RPV-Specific (cont'd.)

Step 3

DSWS

1. How did the perturbation values assigned to components of the following

subsystems of the reference system affect the training resources

requirements analysis?

a. Autoloader

b. Fire Control Processor

c. Internal Navigation System

d. Auxiliary Power Unit

e. Suspension: 1. Howitzer

2. ARV (where applicable)

f. Howitzer Engine

g. Suspension/Chassis Combination

2. What training impacts are effected by the use of a trailer in the FMC

system?

RPV: For Questions I and 2

References: RPV Appendix C - pp. 3, 4, 27, 30, 57

Subsystems: Air Vehicle, airborne data terminal, remote ground

terminal

1. What was the rationale for assigning the PV's? What are the values that

were arrived at?

2. What is the rationale for choosing the respective representative

equipments for training estimation? What would the procedure be if a

detailed Training Resources Requirements Analysis (TRRA) was conducted?

3. Identify the nine new courses in the reference system; the seven

modified courses in the baseline system; the two deleted courses in the

baseline system.

4. Discuss the implications of the fact that, in many cases, Direct Support

(DS) level maintenance was not specified in the Logistics Support

Analysis (LSA) data.

5. Discuss the low performance that may result from high skill requirements

and low paygrade assignments as indicated on page 119, RPV; explain why

this was not identified in the recommendations; how was this addressed

in the reference system?

C-9

""." .", ,- ", ." "- '",°"-i °o "°'-°'" . "'*'-'°,°',°'.- '.% . • - - ,.'.-° •, , , , . . .. , . . ,. ." / 2 4
METHODOLOGY AUDIT

DSWS/RPV-Specific (cont'd.)

Step 3 (cont'd.)

RPV: (cont'd.)

6. Explain, by example, the four steps in determining the number of

instructors for one course.

7. Show an example of a Program Of Instruction (POI) used for obtaining

training man-days.

8. Since the HARDMAN Methodology does not address Life Cycle Costing (LCC),

and only addresses a potion of the training costs, why address it at

all? Why wasn't it addressed in RPV?

9. In general, how does this step interrelate with the MOS/ASI selection in

step 1?

Step 4 _

DSWS

1. How did the perturbation values assigned to components of the following 1 subsystems of the reference system affect the personnel requirements?

a. Autoloader

b. Fire Control Processor

c. Internal Navigation System

d. Auxiliary Power Unit

e. Suspension: 1. Howitzer

2. ARV (where applicable)

f. Howitzer Engine

g. Suspension/Chassis Combination

2. What personnel requirements are affected by the use of a trailer in the

FMC system?

RPV

p1. Does the primary output of this step, personnel to be trained per year,

represent requirements at IOC, 5 yrs., 10 yrs., . . . ?

2. Discuss the sources of personnel flow rates. Why does DRC rely on Army

data and not develop its own models, or use the algorithms outlined in

the HARDMAN Methodology handbook? What is the expected error in the "-

primary output due to inadequate data inputs?

3. Explain, by example, personnel flow in the 13T OS structure.

4. Are there other Army data sources available, such as TRADOC?

C- 10

-: .-:: --.-:-2 _ : :::.. :; ---.::..,.. _ :. . :.-:,:-.; .- . . . ::::::. . .. . ,. -... .... . . .... .:::-.;:,: * .. .-:- :- -: -: :; :-:.:::.: ::: . -. : i
Step 5

RPV

1. Page 154, RPV states that meaningful results can be obtained in the case

of RPV since it is a completely new system. Does this imply that

meaningful results could not be obtained if this were not the case?

2. Does DRC provide inputs or actually run the Army's Personnel Policy

Project Model (p3M)?

3. The p3M Model projects personnel suply based on Army internal

variables; what of external variables such as the economy or politics?

4. Are the P3M personnel inputs different from those used by DRC in step 4 of the Methodology? =

5. Given the short horizon, one to three years, of the P3M Model, is

there a better way and why?

6. How do authorizations relate to the TO&E?

-St

RPV

1. How long does it take to conduct a tradeoff study?

2. Discuss, by example, the tradeoff study of the M939 truck versus the

ML809 truck.

Conclusion

1. What conclusion can you draw from the chart on page 5, RPV; what

conclusions can we draw on how credible the answers are?

C-

. . . . • "'-

.* * . * . *"...",.. ** : *" * ,
APPENDIX D

DEBRIEFING QUESTIONNAIRE

D- 1
TRAINEE DATA COLLECTION SHEET

HARDMAN PROJECT

In order to analyze and identify possible explanations for divergence in

trainee's solutions to problems worked during the training course, we need to

determine the following background data on each person.

* 1. TRAINEE I.D. # (NAME)

2. LEVEL OF EDUCATION

NO. YEARS COMPLETED___ _________

ADVANCED DEGREES?_____________

CERTIFICATES? _______________

COLLEGE DEGREE _______________

HIGH SCHOOL_________________

3. TYPE OF EDUCATION

ADVANCED DEGREE FIELD(S)_____ ___________

COLLEGE MAJOR________________ ______

H.S. EMPHASIS_

- COLLEGE PREP DATALLECTION _EET _- - - TECHNICAL DMA PRO

- VOCATIONAL __________________

- OTHER ___________________

* 4. PRESENT EMPLOYMENT

TITLE:

EMPLOYER:

DUTIES:[

LENGTH OF TIME (IF ANY) AWAY FROM PAST EDUCATION FIELD OF EMPHASIS

(MAJOR, FIELD, ETC.)

5. PAST EMPLOYMENT

IN REVERSE CHRONOLOGICAL ORDER. PLEASE LIST RECENT JOB EXPERIE~NCE (UP

TO 5 YEARS)

D--.. 3

- - . .J G*P SCOO -/,---- .:,-
. DEBRIEFING QUESTIONS , .1*

1. Did you complete all 8 sections of the replication puckage?

If NO0, state which portion ins incompletea...

, .s

2. Were the recommended time frames reolstic?

If NO, state specific packages which tobk more/less time than reconmended. "

,...

3. Was the subject material or methodology of this replication process directly" "

or indirectly related to your present job?

If YES, state how (briefly)

D- 4

.. . . . . . ....

~ * *.o-. ... -
,, - - - .- r.r-w ~r.=w N'fl = W ps - - -. ., - - w ,r =j ,T r ., lrb , r"'J~==m' '-:' " LL: '=4=

4. Were there portions of the repllcation t)hat yoos felt were partLirclarly

difficult? (Beginning with package 1, please give any significanit notation"

you made about the package while working on the replication. Please omit

problems which were readijy resolved by a telephone call of DRC. 'Also include comments about possible reasons and solutions for the problem.)

Prompts: Complexity; Course training; Time factors, e.g., lag between course

& replication or time to finish package; Materiels, Preparation of Individual..)

D-5

'. °--%

,o.°

S'-.

* .o....
-D-6
- ''r- - .. W r C '' dr C. - - r , - .;--.-"-. €-' -%- - -- - .-. . -- -. * .r. - . . r~r - r .r -r .a .. a.

6. Were the instructions in the data packages generally adequate for inLerpretitig

what was actually required, i.e., calculating the required answers?

If NO, please explain possible reasons and state which data packige.

7. Did you note any significant descrepancies between your solutions and the

"school" (DRC) solutions during the replication?

If YES, please state reasons, if known. (Prompts: Replicator background;

Training course; Data package instructions, materials; Other (errors,etc.)

,.1

D-7

- ', .
8. What would have been required of you to arive at the school solution (DRC

answers)? Apply comments to specific data packages.

9. Did you read the pre-replication patkage materiels for the SINCCARS?

If YES, were they useful? "

10. Did the course prepare you adequately for the replicatlon?

EXPLAIN...

-. 11. What criteria could you suggest for selecting analyst trainees to apply

HARDMAN? (Answer may vary for different steps in the process)

12. Do you have any suggestions for training course improvements?

D-8p
APPENDIX E

SDATA INPUTS FOR THE MANPOWER SENSITIVITY MODEL

E- I
This appendix contains the data sets and baseline runs of the SEDCOM

cost model. The first table is the baseline run used in the analysis. The

equipment used there is SINCGARS without the battery. This was done because

the battery, with extremely low mean time between failures, takes a long time

to run on the computer and adds nothing to the analysis. For the sake of

completeness a second baseline cost estimate is presented, based on the

inclusion of the battery. It is labeled SNCGARS2.

Most of the input data used to describe both SINCGARS and its operating

and support environments were provided by the SINCGARS Program Office and are

presented in the next several tables. Because the computer program used for

this example was derived from the Navy SECDOM model, the billet structure and -..

some maintenance level terminology (for example, ship level is organizational

level) were merely adopted to prevent reprogramming. The use of some Navy

terminology and data did not affect the accuracy of the sensitivity study

because this effort was conducted strictly as a top-level (order-of-magnitude)

analysis.

E-3

~ - ."-"S-S - -

.. . .. i- ... . . . . . . . ~S . . . . . . . . . . . . .
ti -. ,

SINCCARS

i'.. '.,

, COST EEENT ($-000) i tSYSTEM RATES

* Life Cycle Cost 885323.0 Number of LRA Types 17

System Unit Cost 11.1 Number of iRA's 17

* INITIAL COSTS:

Production 359265.0 Confidence Against Stockout .999

Maint. Training 1251.2

Op. Training 80850.0 System TBF 472.3 hrs.

Misc. Manpower 0.0 System MTTR 3.0 hrs.

Spares 2844.2

S+TE 67048.6

" Tech. Doe. 55.2

**TOTAL 511314.0 SUPPORT POLICY SUMMARIES:

-LRA's coded COD 0

OPERATION AND SUPPORT COSTS: LRA's coded MOD 9

Maint. Wage 31587.0 LRA's coded local repair 4

Op. + Off. Wage 35684.7 LRA's coded discard 4

Maint. Training 3842.2

Op. Training 248394.0 . Misc. Manpower 0.0 TRAINING BILLETS BY TYPE:

Spares 11018.2 Equipment Operators 32340

Repair 2257.9 Org and Int Maint 476

Transportation 20.2 Depot Technicians 6

S+TE 41198.5

Tech. Dec. 5.5 REAL MANNING REQUIREMENTS:

**TOTAL 374009.0 Org. and Int. aint. 235.97

Depot Maintenance 4.92

Operators 8085.00

E- 4

A1 ....... .
SNCCARS2

* COST ELEMENTS ($'000) * * SYSTEM RATES *

Life Cycle Cost 1126040.0 Number of LEA Types 18 j

System Unit Cost 11,2 Number of LRA's is

INITIAL COSTS:

* Production 314545.0 Confidence Against Stockout .000

Maint. Training 1618.7

Op. Training 70280.0 System MTSF 22.8 hrs.

Misc. Manpower 0.0 System Nfl 0.4 hrs.

Spares 54779.7

S+TE 67048.6

Tech. Doc. 55.2

**TOTAL 508327.0 SUPPORT POLICY SUMMARIES:

LRA's coded COD 0

OPERATION AND SUPPORT COSTS: LRA°s coded MOD 9

Maint. Vage 39707.1 IRA's coded local repair 4

Op. + Off. Wage 31019.4 LIAsa coded discard 5

Maint. Training 4971.3

Op. Training 215920.0

Misc. Manpower 0.0 TRAINING BILLETS BY TYPE:

Spares 282912.0 Equipment Operators 28112

Repair 1962.7 Org and Int Maint 616

Transportation 17.6 Depot Technicians 6 S+TE 41198.5

Tech. Doc. 5.5 REAL MANNING REQUIREMENTS:

#,TOTAL 617714.0 Org. and Int. Maint. 296.63

Depot Maintenance 4.27

Operators 7028.00

E-5

.............. ... .

o , ..
- -- VT -1

File name: SIrhCGARS

NAVY ENIVIRONIMEITAL AND COST FACTORS *

U Uaily cost of C-school training (S/day/student) ........................ 250

1 Cost of A-school, maintenance technician (S/student) .................. 0

2 Cost of A-school, operator (S/student) ................................... 0

3 Annual billet cost, maintenance technician ........................... 30583

4 Annual billet cost, operator ......................................... 28235

5 Annual billet cost, maintenance technician at depot .................. 38018"

6 Annual billet cost, officer .............................................. 0

7 Available weekly work hours, maintenance technician ..................... 48

8 Available weekly work hours, operator ................................... 60

9 Available weekly work hours, depot technician ........................... 40

10 Annual turnover rate, shipboard personnel (decimal) ...................... 5

11 Annual turnover rate, depot personnel (decimal) ........................ .33

12 Annual discount rate (decimal) .......................................... .1

13 Condemnation rate (decimal) ............................................ .01

14 S+TE support cost : S+TE purchase cost (ratio) .............................. .1

15 Cost of technical documentation development (S/page) ................... 200

16 Annual tech. documentation maintenance rate ($/page/yr) ................. 20

17 Cost of insured freight ($/Ib/mile) .................................. .0002

•** ***1 * SYSTEM OPERATING ENVIROI'91ENT * L,.-

O Number of ships on which system is deployed .............................. 23

1 Number of systems per ship ............................................ 1004

2Length of system life cycle (yrs) ....................................... 10

3 Length of deployment period (days) ................................. 5

4 Number of deployments per year .......................................... 52

5 Average system operating hours (hr/wk) ................................. 15

6 Peak system operating hours (hr/wk) ..................................... 30

7 Reduction rate (learning curve slope)..............................1

8 Number of stockage depots ................................................ 3

9 Number of repair depots .......................................... 2

10 Cost of LRA repair at contractor depot ................................ 1000

11 Ship response time (days) ................................................ 2

12 Military depot response time (days) ..................................... 16

13 Contractor operated depot response time (days) ...................... 20

14 Distance between repair and supply depots (miles)...................750

15 Other personnel costs (S/person) ......................................... 0

**************4*********** SYSTEM DESIGN PARAMETERS * I Required systea confidence level against stockout ...................... .95

2 System assembly cost at given lot size ................................ 5204

3 Lot size for assembly .......................................... 100

4 Average LRA weight (lb) ............................. ........... 1.5

5 Fault isolation hardware cost (S/system) ....................... 2.35714E+06

6 Support and test hardware cost (/systen) ................................ 0

7 Support and test software development cost .................................. 0

8 Average LRA specific software development cost ....................... 80000

9 Technical documentation for systen description (pages) ................ 3

10 Technical documentation for fault isolation (pages) ................... 0

11 System scheduled maintenance requirement (manhr/wk/ship)............... .25

12 Required number of operators per system ..................................-

13 Required no. officers per platform ....................................... 0 

14 Required days C-school for operator training ........................ 10

15 Required days C-school maintenance trng (sys orlien + LRA R&R) ........... In

E-6

. .. ~ ~ *. -. - . . ,. - * -...- .- :%..:- .-.:'- 2 ,.- ;:":? *.*- .' -; :,* * ::: z ::
IRA Configuration for SICGARS

CHASSIS: 1

0 Estimated unit cost .............................................. 1400

1 Corresponding lot size............................................. 100

2 IRA mean time between failures (hours) .................................14006

3 Mean time to fault isolate, R & R this IRA (hours)..................... 3

4 Mean time to repair IRA (hours)......................................1I

5 LRA duty cycle (decimal) ............................................ 1

6 LRA specific support and test hardware cost .......................... 67

7 Training hours required for repair of IRA............................. 1

8 LRA repair rvaterial cost per failure................................. 20

9 Pages of documentation for repair description ........................ 21

1U Quantity of similar assemblies....................................... 2

CONTROL: 1

0 Estimated unit cost ............................................... 375

1 Corresponding lot size ............................................ 100

2ILRA mean time between failures (hours)............................ 27855

3 M~ean time to fault isolate, R & R this IRA (hours)..................... 3

4 Mean time to repair IRA (hours) ...................................... 1

5 IRA duty cycle (decimal) ............................................ 1

6 IRA specific support and test hardware cost........................... 67

7 Training hours required for repair of IRA ......................... 1

8 LRA repair material cost per failure.,.......... o.................... 30

9 Pages of documentation for repair description .......... 0.......... 21

1U Quantity of similar assem~blies.......... 0*0- .....-... 00.... .... 1

PWR ASSY: 1

0 Estimated unit cost ............................................... 180

1 Corresponding lot size.......................... .................. 100

2 IRA mean time between failures (hours) .......... ................. 25974

3 Mean time to fault isolate, R & R this IRA (hours) ..... o............. .. 3

4 Mean time to repair LRA (hours).o............ ................... 0..... 1

5 IRA duty cycle (decimal) ............................................ 1I

6 IRA specific support and test hardware cost .......................... 67

7 Training hours required for repair of IRA............................. 1

8 LRA repair material cost per failure ................................ 30

9 Pages of documentation for repair description ........................ 21

10 Quantity of similar assemblies ........................... ... 0...... 1

TU14R/MXR: 1

0 Estimated unit cost........... ......... ....................... 370

1 Corresponding lot size .......................................... 100

2 IRA mean time between failures (hours)............................ 12937

3 Mean time to fault isolate, R & R tils IRA (hours) ....... o ........ 3

4 Mean time to repair IRA (hours)......................... ........... 1

5 IRA duty cycle (decimal) ............................................ I

6 IRA specific support and test hardware costo......... o................67

7 Training hours required for repair of IRA ........................... 40011

8 LRA repair material cost per failure........................... 0..... 30

9 Pages of documentation for repair description ........................ 21

10 Quantity of simnilar assemblies ........................ o........ ..... 1

F-7
IZ-, - N7 1*

LRA Configuration for SliNCGARS

IF UEMUU: 1

0OEstimnated unit cost......................................... ...... 325

1 Corresponding lot size ............................................ 1loo

2 LRA mean time between failures (hours) ............................ 18868

3 mean time to fault isolate, R 3 R this IRA (hours)..................... 3

4 Mean time to repair LRA (hours)......................................1I

5 LRA du ty cyclIe (dec iimaI) .................. *.........................1

6 LRA specific support and test hardware cost.......................... 67

7 Training hours required for repair of IRA............................. I

8 LRA repair material cost per failure ................................ 30

9 Pages of documentation for repair description ........................ 21

10 Quantity of similar assemblies....................................... 1

EXCITEk: I

o Estimated unit cost ............................................... 950

1Corresponding lot size ............................................ 100

2 LRA mean time between failures (hours)............................ 11737

3 Mean time to fault isolate, R & R this LRA (hours)..................... 3

4 Mean time to repair IRA (hours)...................................... 1

5 LHA duty cycle (decimal) ............................................ 1

6 LRA specific support and test hardware cost.......................... 67

7 Training hours required for repair of LRA................................. 1

8BLRA repair material cost per failure ................................ 30

9 Pages of documentation for repair description ........................ 21

10 Quantity of similar assemblies....................................... 2

SYNTrISIZER: 1

0 Estimated unit cost ............................................... 525

1 Corresponding lot size ............................................ 100

2 LRA mean time between failures (hours)............................ 15674

3 mean time to fault isolate, R & R this IRA (hours)..................... 3

4t'ean timeto repair IRA (hours) .................................... 1.5

5 LRA duty cycle (decimal) ............................................ 1

6ILRA specific support and test hardware cost.......................... 67

7 Training hours required for repai of IRA............................. 1

8 IRA repair material cost per failure ................................ 30

9 Pages of documentation for repair description ........................ 21

10 Quantity of similar assemblies....................................... 2

TWO-WIRE: 1

0 Estimated unit cost ............................................... 160

1 Corresponding lot size ............................................ 1ion

2 IRA mean time betw.een failures (hours)............................ 22624

3 flean time to fault isolate, R & R this IRA (hours) .................... 3

4 mean time to repair IRA (hours)...................................... 1

5 IRA duty cycle (decimal) ............................................ 1

6 IRA specific support and test hardware cost.......................... 67

7 Training hours required for repair of IRA............................. n

8 IRA repair material cost per fdilure ............................... 200

9 Pages of documentation for repair description ........................ 21

10 Quantity of similar assemblies....................................... 3

- E-8
r. • .. V J W5'1.'W WI P .IIg .''-'~

LRA Configuration for SINCGARS

-

SUITCII: 1

0 Estimated unit cost .................................................... 400

I Corresponding lot size ................................................. 100

2 IRA mean time between failures (hours) ............................. 12837

3 Mean time to fault Isolate, R I R this LRA (hours) ..................... 3

4 Ilean time to repair LRA (hours) ........................................ 1

5 LRA duty cycle (decimal) ................................................. 1

6 IRA specific support and test hardware cost ...... .............. 67

7 Training hours required for repair of LRA ................................ 1

8 LA repair material cost per failure .................................... 30

9 Pages of documentation for repair description ........... ...... 21

10 Quantity of similar assemblies ........................................... 2

REMOTE 1/0: 1

0 Estimated unit cost ................................................ 225

1 Corresponding lot size.......... ................ 100

2 IRA mean time between failures (hours) ............................... 44444

3 Mean time to fault isolate, R & R this LRA (hours) ..................... 3

4 Mean time to repair LRA (hours) .......................................... I

5 IRA duty cycle (decimal) ............................................ 1

6 IRA specific support and test hardware cost ............ ..... 67

7 Training hours required for repair of LRA............... .....

8 LRA repair material cost per failure... ......................... 30

9 Pages of documentation for repair description....... .. .. 21

10 Quantity of similar assemblies..................................... 1

F%'.

ANT CUhTHL: 1 %

0 Estimated unit cost............... .................... 175

1 Corresponding lot size . ............................. ... 100

2 IRA mean time between failures (hours) ..... ...... 28818

3 Mean time to fault isolate, R & R this LRA (hours)........ ........ 3

4 Mean time to repair RA (hours). .......... .............. 1

5 LRA duty cycle (decimal).............. ...... ...... ... 1

6 IRA specific support and test hardware cost.......................... 67

7 Training hours required for repair of LRA ... ........... .. 1 I -"'' 

8 LRA repair material cost per failure............. .......... 30

9 Pages of documentation for repair description.......................... 21

10 Quantity of similar assemblies.............. ...... ... ...... ...... 1 "

AUD) PWR SUP: 1

0 Estimated unit cost................................. 210

I Corresponding lot size..... .. ......... ....... --.......... 100

2 IRA mean time between failures (hours)................. 32573

3 Mean time to fault Isolate, R & R this LRA (hours) .................... 3

4 Mean time to repair LRA (hours)....... ............................ 1

5 IRA duty cycle (decimal) ......................................... "

6 LRA specific support and test hardware cost.......................... 67

7 Training fours required for repair of LRA ............................. 1 "

8 IRA repair material cost per failure.................................... 30

9 Pages of documentation for repair description .......................... 21

10 Quantity of similar assenblies....................................... I

E-9
LA Configuration for SINCGARS

AUD VATA I/0: 1

0 Estimated unit cost .................................................... 260

1 Corresponding lot size .................... . ...................... 100

2 LRA mean time between failures (hours) .......................... 14881 . 3 Mean time to fault isolate, R & R this LkA (hours) ..................... 3

4 Mean time to repair LRA (hours) .......................................... 1

5 LRA duty cycle (decimal) ................................................. 1

6 LRA specific support and test hardware cost ............. .... 67

7 Training hours required for repair of RA............................... 1

8 LRA repair materlal cost per failure ................................... 30

9 Pages of documentation for repair description ........................... 21

10 Quantity of similar assemblies .............. ...................... 1

BATTERY CASE: 1

0 Estimated unit cost ................................... ............ 50

1 Corresponding lot size .................................. .............. 100

2 LRA mean time between failures (hours) ............................. 86957

3 Mean time to fault isolate, R & R this IRA (hours) ..................... 3

4 Mean time to repair LRA (hours).................... ............... 10

5 LRA duty cycle (decimal) ................................................. 1

6 LRA specific support and test hardware cost ........................ 67

7 Training hours required for repair of LRA .............................. 0

8 LRA repair material cost per failure ...... ................... 60

9 Pages of documentation for repair description ........................ 21

10 Quantity of similar assemblies ......................................... 3

AUU DAT CONTRL: 1

0 Estimated unit cost..... ... ........... "18

1 Corresponding lot size. ..... ... . .................. 100

2 IRA mean time between failures (hours) ............................... 21978

3 Mean time to fault isolate, R & R this LRA (hours) ............... 3

4 Mean time to repair LRA (hours) ............................. 1

5 LRA duty cycle (decimal) ......................................... 1 6 LRA specific support and test hardware cost..... .................. 67

7 Training hours required for repair of LRA ........................... I

8 LRA repair material cost per failure......-...................... 30

9 Pages of documentation for repair description ....................... 21

10 Quantity of similar assemblies ........................................... 1

ANTENNA: I

0 Estimated unit cost ........................ ...... 40

1 Corresponding lot size ................................................. 100

Z LRA mean time between failures (hours) ............................. 13158

3 I-ean time to fault Isolate, R & R this LRA (hours) ....................... 3

4 Mean time to repair LRA (hours) ........... ......................... 2

5 LAA duty cycle (decimal) ............ ............................. 1

6 LRA specific support and test hardware cost .............................. 0

7 Training hours required for repair of LRA .............................. U

8 LRA repair material cost per failure ............................... 0

9 Pages of documentation for repair description ............................ )

10 Quantity of similar assemblies ........................................... 3

E-1O

.................... ... ... ... ... ... .... . . ..---
LRA Configuration for SItICGARS

* IAULSET: 1

0 Estiw.dted unit cost ..................................................... 80

1 Corresponding lot size .................................................. 100

2 LRAmean time between failures (hours) ................................. 780

* 3 Mean time to fault isolate, R 0 R this LRA (hours).............. 3

4 Mean time to repair LRA (hours)........................................ 2

5 LHA duty cycle (decimal) ................................................. 1

6 LRA specific support and test hardware cost............. ........ ** 0

7 Training hours required for repair of LRA ................................ 0

8 LRA repair material cost per failure ..................................... 0 9 Pages of documientation for repair description ............................ 0

10 Quantity of siriilar assenblies.........................*******~****..... 3

E-11
"% CHASSIS

INPUT DATA '

Quantity I UC(Iot) 1400 lot 100 PITUF 14006 k.__

HTkR 3.0 flTT 1.0 Duty cycle 1.0 S.TE 67

Trng hrs 1 Repair parts $ 2U Tech doc pp Z )SA 2 ,.. ,*.

SUPPORT PULICY SUI-IARIES "*h*"*'*

Contractor Military Local

Depot Depot Repair Discard

LIFE CYCLE COST 46846.4 40340.8 40227.0 49635.4

InITIAL CUSTS:

Production 39356.8 39356.8 39356.8 39356.8

Training 0.0 0.1 0.9 0.0

Spares 284.5 280.3 172.5 1830.2

S+TE 0.0 80.1 81.9 0.0

Tech. Doc. 0.0 4.2 4.2 0.0

**TOTAL 39641.3 39721.5 39616.2 41187.0

OPERATIO14 AiU SUPPORT COSTS:

Wage 251.9 344.9 335.8 251.9

Training 0.0 0.1 2.7 0.0

Spares 82.0 82.0 82.0 8196.5

Repair 6871.3 137.4 137.4 0.0

Transportation 0.0 3.1 0.0 0.0

S+TE 0.0 49.2 50.3 0.0 .

Tech. Doc. 0.0 2.6 2.6 0.0

1*TUTAL 7205.1 619.3 610. 8 8448.4

Support Posture Assigned: Local Repair

:U

.. -

E-12
r:T:

r

Cu14THOL

INPUT UATA

Quantity I UC(lot) 375 lot 100 MTBF 27855

HTTR 3.0 MTTe 1.0 Duty cycle 1.0 S.TE 67

Trng hrs I Repair parts $ 30 Tech doc pp 21 QSA 1

- SUPPORT POLICY SUrMARIES " *

Contractor M.1111 tary Local -

Depot Depot Repair Discard

LIFE CYCLE COST 14184.4 11016.6 11001.7 12029.9

INIITIAL COSTS:

Production 10542.0 10542.0 10542.0 10542.0

Training 0.0 0.1 0.9 0.0

Spares 49.7 48.6 33.6 257.4

S+TE 0.0 80.1 81.9 0.0

Tech. Doc. 0.0 4.2 4.2 0.0

**TOTAL 10591.7 10675.0 10662.6 10799.4

OPER ATIOI Aria SUPPORT COSTS:

Wage 126.7 173.4 168.9 126.7

Training 0.0 0.1 2.7 0.0

Spares 11.0 11.0 11.0 1103.9

Repair 3455.0 103.6 103.6 0.0

Transportation 0.0 1.6 0.0 0.0

S*TE 0.0 49.2 50.3 0.0

Tech. Doc. 0.0 2.6 2.6 O.C

**TOTAL 3592.7 341.6 339.1 1230.6

Support Posture Assigned: rilitary Depot

) -A 

. . . . . . . . .f . . . . *- 5**** * a. - 3
---

PkIR ASSY

INPUT .**oeel* UATA

(Quantity 1 UC(lot) 180 lot 100 HTF 25974

I1ITRH 3.0 1ITTR 1.0 Duty cycle 1.0 S+TE 67

Trng hrs 1 Repair parts $ 30 Tech doc pp 21 QSA I

*"'************* SUPPORT POLICY SUf4ARIES "

Contractor Military Local

Depot Depot Repair Discard

LIFE CYCLE COST 8930.8 5524.9 5516.8 5895.6 .-.

INITIAL COSTS:

Production 5060.2 5060.2 5060.2 5060.2

Training 0.0 0.1 0.9 0.0

Spares 23.9 23.9 16.2 131.4

S+TE 0.0 80.1 81.9 0.0

Tech. Doc. 0.0 4.2 4.2 0.0

**TOTAL 5084.1 5168.5 5163.3 5191.6

OPERATION A1D SUPPORT COSTS:

Wage 135.8 186.0 181.1 135.8

Training 0.0 0.1 2.7 0.0

Spares 5.7 5.7 5.7 568.3

Repair 3705.2 111.2 111.2 0.0

Transportation 0.0 1.7 0.0 0.0

S+TE 0.0 49.2 50.3 0.0

Tech. Doc. 0.0 2.6 2.6 0.0

**TOTAL 3846.7 356.4 353.5 704.1

Support Posture Assigned: Military Depot

i:-1 4:.

. ...

• . - . . ..-. -. * . %j •~ 
TUHiR/WxM

.. u v. do .,

QJuantity 1 UC(lot) 370 lot 100 MTBF 12937

f1TNM 3.0 NTTR 1.0 Uuty cycle 1.0 S TE 67

Trnq hrs 1 Repair parts S 30 Tech doc pp 21 QSA 1 . ..

**..o*.** *.**'***'** SUPPORT POLICY SUILIARIES *'****"***'

Contractor mi I i tary Local

Depot Uepot Repair Discard

LIFE CYCLE COST 18213.3 11236.7 11200.1 13538.3 v

INITIAL COSTS:

Production 10401.4 10401.4 10401.4 1U401.4

Training 0.0 0.1 0.9 0.1)

Spares 76.6 75.5 45.9 519.u

S+TE 0.0 80.1 81.9 O. r Tech. Doc. 0.0 4.2 4.2 O.C

**TOTAL 10478.1 10561.4 10534.3 10920.4

OPERATIO4 A14U SUPPORT COSTS:

Wage 272.7 373.4 363.6 272.7 -

Training 0.0 0.1 2.7 0

Spares 23.5 23.5 23.5 2345.

Repair 7439.0 223.2 223.2 0..

Transportation 0.0 3.3 0.0 0.0

S TE 0.0 49.2 50.3 0.(1

Tech. Doc. 0.0 2.6 2.6 0..

**TOTAL 7735.2 675.3 665.8 2617.9

Support Posture Assigned: Military Depot

. .. .... .... .... ... .... .... .... ...

:72

K- L "
IF UEhOU

11?UT LhAdA

'e.**...'

Quantity 1 UC(lot) 325 lot 100 MTBF 18868

14T14R 3.0 MTTR 1.0 Duty cycle 1.0 S.TE 67

Trnq hrs 1 Repair parts S 30 Tech dC Pp 21 QSA 1

*******.*******SUPPORT POLICY SUf~iARIES *~*~o..e

Contractor Military Local

Depot Depot Repair Discard

LIFE CYCLE COST 14493.2 9752.3 9725.4 11057.0

INITIAL COSTSProduction 9136.4 9136.4 9136.4 9136.4

Training 0.0 0.1 0.9 0.0

Spares 55.0 54.0 30.0 321.1

S.TE 0.0 80.1 81.9 0.0

Tech. LUoc. 0.0 4.2 4.2 0.0

**TOTAL 9191.4 9274.8 9253.4 9457.5

OPEkATIO4 A140 SUPPORT COSTS:

Wage 187.0 256.0 249.3 187.0

Training 0.0 0.1 2.7 0.0

Spares 14.1 14.1 14.1 1412.4

Repair 5100.6 153.0 153.0 0.0

Transportation 0.0 2.3 0.0 0.0

S.TE 0.0 49.2 50.3 0.0

Tech. Doc. 0.0 2.6 2.6 0.0

**TOTAL 5301.7 477.4 472.0 1599.4

Support Posture Assigned: Military Depot

V:- 16E
EXCI TER r.

irWUT UATA Iu ( ;antity 1 UC(Iot) 950 lot 100 NTdIF 11737

-14TiRR 3.0 tiTTR 1.0 Duty cycle 1.0 S.TE 67

Trn9 hrs I Repair parts S 30 Tech doc pp 21 QSA 2

SUPPORT POLICY SUmmARIES *******.*..

Contractor i i tary Local

Uepot Depot Repair Uiscard -'

LIFE CYCLE COST 35476.6 27774.0 27681.1 35102.8 .

INITIAL COSTS: , ,

Production 26706.4 26706.4 26706.4 26706.4

Training 0.0 0.1 0.9 0.0

Spares 203.6 203.6 119.1 1458.7

S+TE 0.0 80.1 81.9 ".0

Tech. Doc. 0.0 4.2 4.2 0.0

**TUTAL 26910.0 26994.4 26912.4 28165.1

OPERATIO4 AND SUPPORT COSTS:

Wage 300.6 411.6 400.8 300.6

Training 0.0 0.1 2.7 0.0

Spares 66.4 66.4 66.4 6637.1

Repair 8199.6 246.0 246.0 0.0

Transportation 0.0 3.7 0.0 0.0

S+TE 0.0 49.2 50.3 0.0

Tech. Doc. 0.0 2.6 2.6 0.0

"TOTAL 8566.6 779.6 768.7 6937.7 . . "

Support Posture Assigned: Local Repair

E-17
SYIJTHSIZEM C

INPUT UATA

Qu antity 1 UC(lot) 525 lot 100 I4TLF 15674

11TkR 3.0 ITTRt 1.5 Duty cycle 1.0 SiTE 67

*Trng hrs I Repair parts $ 30 Tech doc pp 21 QSA 2

*,*.**..*..**..SUPPOR~T POLICY SU1#4ARIES *****.**

Contractor Military Local j Depot Depot Repair Discard

LIFE CYCLE COST 21245.9 15552.3 15500.0 18342.8

INITIAL COSTS:

Production 14758.8 14758.8 14758.8 14758.8

Training 0.0 0.1 0.9 0.0

Spares 94.5 92.9 49.3 612.3

S+TE 0.0 80.1 81.9 0.0

Tech. Doc. 0.0 4.2 4.2 0.0

**TOTAL 14853.3 14936.1 14895.1 15371.1

OPERATIOU A14D SUPPORT COSTS:

Wage 225.1 349.8 337.6 225.!

Training 0.0 0.1 2.7 0.0

Spares 27.5 27.5 27.5 2746.F.

Repair 6140.0 184.2 184.2 0.0

Transportation 0.0 2.8 0.0 0.C

S+TE 0.0 49.2 50.3 0.0

Tecn. Doc. 0.0 2.6 2.6 0.0

'1Y~.6392.6 616.1 604.9 2971.7

Support Posture Assigned: Local Repair

F, L 8

.1
TWO-WINE

I;IPUT DATA

Quantity 1 UC(lot) 160 lot 100 14TBF 22624

i~iTxN 3.0 mTTI4 1.0 Duty cycle 1.0 S+TE 67

Trng hrs 0 Repair parts S 200 Tech dcc pp 21 QSA 3

***************SUPPORT POLICY SUMtAltIES

Contractor Military Local

Depot Depot Repair Discard

LIFE CYCLE COST 8936.4 572R.9 5715.9 5366.3

[INITIAL COSTS:

Production 4497.9 4497.9 4497.9 4497.9

Training 0.0 0.0 0.0 0.0

Spares 22.9 22.9 14.5 132.5

S.TE 0.0 80.1 81.9 0.0

Tech. Doc. 0.0 4.2 4.2 0.0

**TOTAL 4520.8 4605.1 4598.5 4630.4

OPERATION A14D SUPPORT COSTS:

Wage 155.9 213.5 207.9 155.9

Training 0.0 0.0 0.0 0.0

Spares 5.8 5.8 5.8 579.9

Repair 4253.8 850.8 850.8 0.0

Transportation 0.0 1.9 0.0 0.0)

S.TE 0.0 49.2 50.3 0.0

Tech. Doc. 0.0 2.6 2.6 0.0

**TOTAL 4415.6 1123.8 1117.4 735.9

Support Posture Assigned: Discard

F-1 9
. b

SuI TCHe

.. ,..

IfPUT uATA

Ouantity 1 lC(Iot) 400 lot 100 MTBF 12937

hTNM 3.0 HTTR 1.0 Outy cycle 1.0 S+TE 67

Trng hrs 1 Repair parts $ 30 Tech doc pp 21 QSA 2

SUPPORT POLICY SUMIIARIES ***********"***

Contractor Military Local

Depot Depot Repair Discard

LIFE CYCLE COST 19125.0 12093.0 12053.9 14639.6

INITIAL COSTS:

Production 11244.8 11244.8 11244.8 11244.8

Training 0.0 0.1 0.9 0.0

Spares 82.9 81.7 49.7 564.8

S TE 0.0 80.1 81.9 0.0

Tech. Doc. 0.0 4.2 4.2 0.0

**TOTAL 11327.7 11410.9 11381.4 11809.6

OPERATION ND SUPPORT COSTS:

Wage 274.8 376.3 366.4 274.8

Training 0.0 0.1 2.7 0.0

Spares 25.6 25.6 25.6 2555.1

Repair 7497.0 224.9 224.9 0.0

Transportation 0.0 3.4 0.0 0.0

S+TE 0.0 49.2 50.3 0.0

Tech. Loc. 0.0 2.6 2.6 0.0

**TOTAL 7797.4 682.1 672.5 2329.9

Support Posture Assigned: Local Repair -.

E.. - 20

E-20 '" "

kk . . . . .. I. . . . . . . . . . " . ..... . " . . . . . . . .. . . . .- 
-k4- i --.- V .d7i

*. -. .

REmOTE flU '*

WiPUT DATA

Quantity 1 UC(lot) 225 lot 100 ITBF 44444

IHTiKR 3.0 HTTR 1.0 Duty cycle 1.0 S+TE 67

Trng hrs 1 Repair parts S 30 Tech doC pp 21 QSA 1

*************** SUPPORT POLICY SUMIARIES **************

Contractor Military Local -'.

Depot Depot Repair Discard

LIFE CYCLE COST 8596.5 6662.0 6656.1 6919.9

INITIAL COSTS:

Production 6325.2 6325.2 6325.2 6325.2 & A -

Training 0.0 0.1 0.9 0.0

Spares 22.4 21.7 13.4 100.2

S+TE 0.0 80.1 81.9 0.0

Tech. Loc. 0.0 4.2 4.2 0.0

**TOTAL 6347.6 6431.3 6425.5 6425.4

OPERATION AND SUPPORT COSTS:

Wage 79.4 108.7 105.8 79.4

Training 0.0 0.1 2.7 0.0

Spares 4.2 4.2 4.2 415.1

Repair 2165.4 65.0 65.0 0.0

Transportation 0.0 1.0 0.0 0.0

S+TE 0.0 49.2 50.3 0.0

Tech. Doc. 0.0 2.6 2.6 0.0

**TOTAL 2248.9 230.7 230.5 494.5

Support Posture Assigned: Military Uepot

?i,

-.- '

-.... ..... ........ .... ..... ....
* r r.--rw *- - . -- . ~ w7 - 7 F

AnlT COliTRL

INPUT JATA

Quantity I UC(Iot) 175 lot 100t 100 HiHF Hr8F 2831R

14TRR 3.0 MTTR 1.0 Uuty cycle 1.0 S.TE 67

Trnq hrs 1 Repair parts $ 30 Tech doc pp 21 QSA 1

' *********** SUPPORT POLICY SUMMARIES ********* *

Contractor Military Local

Depot Depot Repair Discard

LIFE CYCLE COST 8409.7 5352.9 5346.2 5656.8

INITIAL CUSTS:

Production 4919.6 4919.6 4919.6 4919.6

Training 0.0 0.1 0.9 0.0

Spares 23.2 22.7 15.7 116.8

S+TE 0.0 80.1 81.9 0.0

Tech. Doc. 0.0 4.2 4.2 0.0

**TUTAL 4942.8 5026.6 5022.2 5036.4

OPERATION ANO SUPPORT COSTS:

Wage 122.4 167.6 163.2 122.4

Training 0.0 0.1 2.7 0.0

Spares 5.0 5.0 5.0 498.0

Repair 3339.5 100.2 100.? 0.0

Transportation 0.0 1.5 0.0 0.0 r-.,

S+TE 0.0 49.2 50.3 0.0

Tech. Doc. 0.0 2.6 2.6 0.0

**TOTAL 3466.9 326.2 324.0 62U.4

Support Posture Assigned: Military Depot

1E--2 2

..................................................................
T 77° ... 

AUD PUR SUP

I, UT DATA

,Nantity 1 UC(lot) 210 lot 100 MTBF 32573

14TkR 3.0 ITTR 1.0 Duty cycle 1.0 S+TE 67 Trn 9 hrs I Repair parts $ 30 Tech doc pp 21 QSA 1

.- ..... ,

SUPPORT POLICY SUP.NARIES **********-

Contractor MIilitary Local

Depot Depot Repair Discard

LIFE CYCLE COST 8998.7 6309.8 6297.2 6666.9

IITIAL COSTS:

Production 5903.5 5903.5 5903.5 5903.5

Training 0.0 0.1 0.9 0.0

Spares 27.0 26.4 12.8 126.4

SiTE 0.0 80.1 81.9 0.0

Tech. Doc. 0.0 4.2 4.2 0.0

*TOTAL 5930.6 6014.3 6003.2 6029.9 ' " "

OPERATION AND SUPPORT COSTS:

Wage 108.3 148.3 144.4 108.3

Training 0.0 0.1 2.7 0.0

Spares 5.3 5.3 5.3 528.7

Repair 2954.6 88.6 88.6 0.0

Transportation 0.0 1.3 0.0 0.0

S+TE 0.0 49.2 50.3 0.0

Tech. Doc. 0.0 2.6 2.6 0.0

**TOTAL 3068.1 295.5 293.9 637.0

Support Posture Assigned: Military Depot

-. - . ~ ~ . . . . . . ..-.. ".--

- - °.
AUD DATA 1/0

IN4PUT UATA

Quantity 1 ~~t 26U lot 100 PHTIF 141481 7

3UMit1.0 Vuty cycle 1.0 S+TE 67

rng hrs 1 Repair parts $ 30 Tech doc pp 21 USA 1

~~ SUPPORT POLICY SUtViARIES

Contractor Military Local

Depot Depot Repair Discard

LIFE CYCLE COST 14080.4 8033.2 8000.7 9301.7

INITIAL COSTS:

Production 7309.1 7309.1 7309.1 7309.1

Training 0.0 0.1 0.9 0.0

Spares 52.7 51.9 24.6 322.8

SaTE 0.0 80.1 81.9 0.0

Tech. Doc. 0.0 4.2 4.2 0.0

*TTL7361.8 7445.4 7420.6 ~ 7631.9

UPERATIONd A14U SUPPORT COSTS:

Wage 237.1 324.6 316.1 237.1

Training 0.0 0.1 2.7 0.0

Spares 14.3 14.3 14.3 1432.7

Repair 6467.2 194.0 194.0 0.0

Transportation 0.0 2.9 0.0 0.0

SaTE 0.0 49.2 50.3 0.0

Tech. Doc. 0.0 2.6 2.6 0.0

**TOTAL 6718.6 587.8 580.0 1669.8

Support Posture Assigned: Military Depot

r

E-24
..

BATTERY CASE

INPUT DATA

."uantity 1 UC(lot) 50 lot 100 mTBF 86957 ?- -,

HTKR 3.0 NTTR 10.0 Duty cycle 1.0 S.TE 67

Trng hrs 0 Repair parts S 60 Tech doc pp 21 QSA 3

* SUPPORT POLICY SUMMARIES %

Contractor Military Local

Depot Depot Repair Discard - 4

LIFE CYCLE COST 2556.6 1802.7 1790.1 1505.4

INITIAL COSTS: . "

Production 1405.6 1405.6 1405.6 1405.6

Training 0.0 0.0 0.0 0.0

Spares 3.2 3.2 2.9 12.1

S+TE 0.0 80.1 81.9 0.0 4

Tech. Doc. 0.0 4.2 4.2 0.0

**TOTAL 1408.8 1493.1 1494.6 1417.7

OPERATION AND SUPPORT COSTS:

Wage 40.6 190.4 175.R 40.6

Training 0.0 0.0 0.0 0.0

Spares 0.5 0.5 0.5 47.1

Repair 1106.7 66.4 66.4 0.0

Transportation 0.0 0.5 0.0 0.0

S+TE 0.0 49.2 50.3 0.0 

Tech. Doc. 0.0 2.6 2.6 0.0

**TOTAL 1147.8 309.6 295.6 87.7

Support Posture Assigned: Discard

F..- 

1E-29 ..,' '
AUI) VAT CUJ4THL '.

InFUT vATA

Quantity I UC(lot) 180 lot 100 MTHF 21978

WAR~4 3.0 NTTR 1.0 LOuty cycle 1.0 S+TE 67

Trng hrs I Repair parts S 30 Tecle doc pp 21 QSA I

SUPPORT POLICY SUIVARIES

Contractor Military Local

Depot Depot Repair D~iscard

*LIFE CYCLE COST 9635.4 5585.5 5571.2 6048.4

* IN~JITIAL COSTS: .-

Production 5060.2 5060.2 5060.2 5060.2

Training 0.0 0.1 0.9 0.0

Spares 29.2 29.2 16.4 156.2

S.TE 0.0 80.1 81.9 0.0

Tech. Doc. 0.0 4.2 4.2 0.0

" TOTAL. 5089.3 5173.7 5163.5 5216.3

OPERATION ANID SUPPORT COSTS:

Wage 160.5 219.8 214.0 160.5

Training 0.0 0.1 2.7 0. C.

Spares 6.7 6.7 6.7 671.6

Repair 4378.9 131.4 131.4 0.0

Transportation 0.0 2.0 0.0 0.0

S+TE 0.0 49.2 50.3 0.0

Tech. Doc. 0.0 2.6 2.6 0.0

**TOTAL. 4546.1 411.8 407.7 832.1

Support Posture Assigned: Military Depot

F-2 6

.~~~~ ~ ~ . . .

. . . . . . . . . . . . . .! --. .. . I. , )-- 7
ArTEN fA *

INPUT DATA

Quantity I UC(Iot) 40 lot 100 14TUF 13158

HTRR 3.0 HTTR 2.0 Duty cycle 1.0 S.TE 3-

Trng hrs 0 Repair parts 0 Tech doc pp 0 USA 3

SUPPORT POLICY SUWARIES "

Contractor Military Local

Depot Depot Repair Discard

LIFE CYCLE COST 8717.5 1733.7 1707.9 1697.2

I.ITIAL COSTS:

Proauction 1124.5 1124.5 1124.5 1124.5 -

Training 0.0 0.0 0.0 0.0

Spares 8.3 8.2 5.0 55.3

S+TE 0.0 80.0 80.0 0.0

Tech. Doc. 0.0 0.0 0.0 0.0

**TOTAL 1132.8 1212.6 1209.4 1179.8

OPERATION AND SUPPORT COSTS:

Wage 268.1 466.1 446.9 268.1

Training 0.0 0.0 0.0 0.0

Spares 2.5 2.5 2.5 249.3

Repair 7314.1 0.0 0.0 0.0.

Transportation 0.0 3.3 0.0 0.0

S TE 0.0 49.2 49.2 0.0

Tech. Doc. 0.0 0.0 0.0 0.0

"TOTAL 7584.7 521.1 498.5 517.4

Support Posture Assigned: Discard

E-2 7

7, ,-.-,

------------------------------------------------------- --'--.

.............................. .. °-'.

-. . . . . . . . . . . . . . . . . . . ... .
"AUDSET

IN.PUT JATA

Quanttty 1 UC(Iot) 80 lot 100 W.THF 730

iTrTK 3.0 HTTR 2.0 Uuty cycle 1.0 SiTE 0

Trng hrs 0 Repair parts $ 0 Tech doc pp 0 USA 3

**tttt**tt**tt**ltt SUPPORT POLICY SUPMtARIES *

Contractor tt I tary Local

Depot Depot Repair Discard

LIFE CYCLE COST 130370.0 10511.9 10063.6 16903.1

IITIAL COSTS:

Production 2249.0 2249.0 2249.0 2249.0

Training 0.0 0.0 0.0 0.0

Spares 130.6 130.6 63.1 1721.0

S+TE 0.0 80.0 80.0 0.0

Tech. Doc. 0.0 0.0 0.0 0.0

**TOTAL 2379.6 2459.6 2392.1 3969.9

UPERATIOUl A1U SUPPORT COSTS:

Uage 4522.9 7863.5 7538.2 4522.9

Training 0.0 0.0 0.0 0.0

Spares 84.1 84.1 84.1 8410.3

Repair 123383.0 0.0 0.0 0.0

Transportation 0.0 55.5 0.0 0.0

S+TE 0.0 49.2 49.2 0.0

Tech. Doc. 0.0 0.0 .0 0.0

*"TOTAL 127990.0 8052.3 7671.5 12933.2

Support Posture Assigned: Discard

b E-28

-. '-7

r *~**.~ . ,
~77i7r7. -- ~-----~-• I,.IBATTERY

IdPUT UATA

Quantity 1 UC(lot) 80 lot 100 1 TBF 24 ..

flTRK 0.3 MTTR 2.0 Duty cycle 1.0 S+rc 0

Trng hrs 0 Repair parts $ 0 Tech doc pp 0 QSA 3

11Z

•'*" " " SUPPORT POLICY SUMMARIES *."***"",,-""

Contractor Military Local

Depot Depot Repair Discard

LIFE CYCLE COST %4030330.11 130887.0 116801.0 342672.0

IHITIAL COSTS:

Production 2249.0 2249.0 2249.0 2249.0

Training 0.0 0.0 0.0 U.0

Spares 3153.5 3153.5 1442.7 54139.4

S*TE U.0 80.0 80.0 0.0

Tech. Doc. 0.0 0.0 0.0 0.0

"TOTAL 5402.5 5482.5 3771.7 57088.4

OPERATIOU AND SUPPORT COSTS:

Wage 12249.6 120818.0 110247.0 12249.6

Training 0.0 0.0 0.0 0.0

Spares 2733.3 2733.3 2733.3 273334.0

Repair %4009950.0 0.0 0.0 0.0

Transportation 0.0 1804.5 0.0 0.0

S.TE 0.0 49.2 49.2 0.0

Tech. Doc. 0.0 0.0 0.0 0.0 "

**TOTAL 4024930.0 125405.0 113029.0 285584.0

Support Posture Assigned: Oiscard

122785

E--29

............................................... ..-