DAU

PROGRAM

MANAGERS

TOOL KIT

Fourteenth Edition (Ver 1.0)

March 2005

For sale by the

U.S. Superintendent of Documents, Government Printing Office

Internet: bookstore.gpo.gov Phone: (202) 512-1800 Fax: (202) 512-2250

 Mail Stop: SSOP, Washington, DC 20402-0001
DAU PROGRAM MANAGERS TOOL KIT

ii
DAU PROGRAM MANAGERS TOOL KIT

iii

PREFACE

This Fourteenth Edition (ver 1.0) of the DAU Program Managers Tool

Kit contains a graphic summary of acquisition policies and managerial

skills frequently required by DoD program managers. It is a current

version of a “Tool Box” that was first developed by Mr. Charles F. Schied

of Program Management Course (PMC) 92-1. For convenience, the

Tool Kit is sized for insertion into a 3-hole, 5-1/2” x 8-1/2” “Day Runner.”

The information in the Tool Kit is extracted from Defense Acquisition

University (DAU) course material and is based on DoDD 5000.1 (12

May 2003), DoDI 5000.2 (12 May 2003), the Defense Acquisition Guidebook (DAG), CJCSI 6212.01C (20 November 2003), and CJCSI

3170.01D (12 March 2004). Material from the DAU Acker Library &

Learning Resource Center was also used.

Users of the Tool Kit are reminded that this summary is a guide only

and should not be used as a substitute for official policy guidance. Periodic review of official policy guidance is recommended.
DAU PROGRAM MANAGERS TOOL KIT

iv

ACKNOWLEDGMENTS

As sponsor of the Tool Kit, I wish to recognize the following members of

the Defense Acquisition University (DAU) faculty and staff for their input to this Fourteenth Edition: Mr. Bill Bahnmaier, formerly of DAU’s

Program Management and Leadership Department on the Capital/Northeast Regional Campus, for coordinating the input and editing the material from various DAU departments; and, from the Visual Arts and Press

Department, Mrs. Collie Johnson for editing, Mrs. Kathryn Sondheimer

for layout/graphics and editing, and Mrs. Frances Battle for liaison with

the Government Printing Office and printer. Other significant contributors were Mr. John Kelley, Program Management Department; Ms. Leslie

Deneault, Contract Management Department; Ms. Roberta Tomasini,

Ms. Siobhan Tack, Mr. Robert Carlson, Mr. Jack Cash, and Ms. Miriam

Cohe of the Business, Cost Estimating and Financial Management

Department; Dr. John Claxton, Dr. John Snoderly, Mr. George Prosnik,

Mr. Bill Motley, and Mr. Randy Fowler of the Curricula Development &

Support Center; Maj Mark McNabb, USAF, and Dr. Bob Lightsey of the

Technology & Engineering Department; and Mr. Larry Heller of the Logistics Management Department. I also want to thank Mr. Tim Shannon, Dean of the Capital/Northeast Regional Campus for both his encouragement and support for this effort.

C.B. Cochrane

Director, Center for

Program Management
DAU PROGRAM MANAGERS TOOL KIT

v

TABLE OF CONTENTS

I. ACQUISITION MANAGEMENT .................................................. 1

Policy ...................................................................................2–14

Program Manager’s Bill of Rights and Responsibilities ................ 2

Defense Acquisition Decision Reviews & Phases ........................ 3

Acquisition Categories (ACAT)..................................................... 4

Acquisition Strategy Considerations ............................................ 5

Acquisition, Technology, and Logistics (AT&L) Knowledge

Sharing System (AKSS) ........................................................... 6

Acquisition Community Connection (ACC) .................................. 6

Defense Acquisition Guidebook (DAG) ........................................ 7

Defense Acquisition Board Timeline Milestones B, C & FRPDR ...... 8

Milestone Decision Information—A Possible Construct ............... 8

Information for Milestone/Decision Reviews ................................ 9

C4ISR Interoperability.................................................................10

S&T Linkage to Defense Acquisition Process .............................11

Adv Concept Technology Demonstration Initiation Process ........12

Acquisition Program vs. ATD & ACTD.........................................13

Program Structure/Schedule (Example) .....................................14

International ......................................................................15–16

DoD International Armaments Cooperation Policy ......................15

The Scope of Defense Cooperation ............................................15

Defense Sales vs. Cooperative Acquisition .................................16

Internat’l Activities Associated with Def Acquisition Phases .......16

Funds Management ..........................................................17–28

Resource Allocation Process—Overlap ......................................17

PPBE —Planning Phase .............................................................18

PPBE On-Year Concurrent Program/Budget Review (Even Year) ...19

PPBE Off-Year Program/Budget Review (Odd Year) ...................19

Resource Allocation Process ......................................................20

Congressional Enactment Timetable ..........................................20

Procurement Appropriations .................................................21–22

RDT&E Appropriations ...............................................................23

Sample Navy Appropriations and Budget Activities ....................24

Appropriation Life........................................................................25

Below Threshold Reprogramming Actions ..................................25

Life Cycle Cost Composition .......................................................26

Product Improvements ................................................................26

Cost Estimating ..........................................................................27

Cost Estimating Relationships (CER)—Parametric ....................27

Performance Measurement.........................................................28
DAU PROGRAM MANAGERS TOOL KIT

vi

I. Acquisition Management (continued)

Contracting .......................................................................29–73

Contracting—Components of Contract Price ..............................29

Types of Contracts ......................................................................29

Contract Type Features .........................................................30–31

Pre-solicitation Process ..............................................................32

Post-solicitation Process .............................................................32

Other Ways to Buy ......................................................................33

Defense Industry Business Management .......................34–40

Contractor Profitability Ratios .....................................................34

Aerospace Defense Industry Contractor Profitability Ratios .......35

Lockheed Martin Corp—Dupont Formula Analysis (An Example) ....36

Cash Cycle .................................................................................37

Contractor Financing and Payments ...........................................37

Direct and Indirect Costs ............................................................38

Assigning Indirect Costs .............................................................38

Life Cycle of Indirect Cost Rates ................................................39

Contractor’s Cost Proposal .........................................................39

Contractor Business Planning Process Outputs .........................40

Logistics ............................................................................41–46

Life Cycle Logistics .....................................................................41

Principal Life Cycle Logistics Goals/Objectives ..........................41

System Operational Effectiveness (SOE) ...................................41

Best Practice: Support Elements ................................................42

Support Element Definitions .......................................................42

Life Cycle Logistics Linkage to Defense Acquisition Process .....43

Performance Based Logistics .....................................................44

Performance Based Logistics Implementation Model .................44

Post-production Support Decision Process ................................45

Operational Availability ...............................................................45

Logistics Management Community of Practice (LOG COP) .......46

Defense Acquisition Guidebook—Life Cycle Logistics ................46

Organization ......................................................................47–49

Program Office Organization Structures ...............................47–49

Manufacturing ...................................................................50–53

The Role of Manufacturing in the Acquisition Process ................50

Common Production Risks that Greatly Impact Cost,

Schedule & Performance ........................................................50

Producibility ................................................................................51

Producibility—Defense Acquisition Guidebook,

4.4.6.1 Producibility .................................................................51

Quality Management Systems ....................................................52
DAU PROGRAM MANAGERS TOOL KIT

vii

I. Acquisition Management (continued)

Key Characteristics and Variation Reduction ........................52–53

Production Readiness Review (PRR) .........................................54

Additional Manufacturing Information Sources............................54

Test & Evaluation (T&E) ...................................................55–58

Test & Evaluation (T&E)—Types and Tasks ................................55

Modeling & Simulation (M&S) Planning ......................................56

Hierarchy of Models and Simulations..........................................57

The Evolution of Modeling & Simulation......................................58

Planning and Control .......................................................59–65

Planning and Control Tools ...................................................59–60

PERT* Network Charts ...............................................................61

Network Schedule Development .................................................62

Lead-Time Chart.........................................................................63

Line of Balance Technique ..........................................................64

Acquisition Program Baseline .....................................................65

Risk Management .............................................................66–68

Risk Management Process Model ..............................................66

Risk & Tradeoff Analysis .............................................................67

Program Manager’s Checklist for Review of

Tradeoff Planning and Studies ................................................68

Technical Performance Measurement .........................................68

Systems Engineering .......................................................69–90

What is Systems Engineering? ...................................................69

A Simple Model of a Systems Engineering Process ...................69

Systems Engineering Tasks ........................................................70

Systems Engineering Policy in DoD ...........................................70

Systems Engineering Process “The Basic Engine” .....................71

Systems Engineering Process “Basic Engine Operations” .........72

Generic SE Process—Detailed Approach ..................................73

Concept Refinement Phase SE Activities ...................................74

Technology Development Phase SE Activities ............................74

System Development and Demonstration Phase SE Activities ...75

Production and Deployment Phase SE Activities ........................75

Operations and Support Phase SE Activities..............................76

Important Systems Engineering Design Considerations .............77

Requirements (User Needs) Analysis Questions ........................78

Attributes of a Well-defined Requirement....................................78

System Analysis and Control ......................................................79

Specifications Development & Technical Reviews ......................80

Technical Review Definitions ......................................................81

Technical Review Best Practices ................................................82
DAU PROGRAM MANAGERS TOOL KIT

viii

I. Acquisition Management (continued)

Specifications and Standards .....................................................83

Performance vs. Detail Specifications .........................................83

Program-unique Specifications ...................................................84

Configuration Management.........................................................85

Interface Management ................................................................86

Interface Control Concept ...........................................................86

How to Create a Work Breakdown Structure ..............................87

Basic Purposes of WBS..............................................................87

Software Management ................................................................88

II. Leadership & Managerial Skills ..............................................89

Working Groups ....................................................................90–91

Management Tradeoffs for Working Groups ................................91

Team Performance Model ...........................................................91

Delegation ............................................................................92–93

Effective Meetings ......................................................................94

Personal Communications ....................................................95–96

Time Management................................................................96–97

Decision Briefing ....................................................................... 98

Management Tools & Techniques .......................................99–100

III. Problem Solving Tools ..........................................................101

Brainstorming ................................................................... 101–102

Cause-and-Effect Diagram .......................................................103

Force Field Analysis .................................................................103

Histogram .................................................................................104

Scatter Diagram ........................................................................104

Surveys ....................................................................................105

Affinity Diagram ................................................................ 106–107

Pairwise Ranking .............................................................. 108–109

Pareto Chart .............................................................................109

Benchmarking...........................................................................110

Flowcharting .............................................................................111

Standard Flowchart Symbols ....................................................112

Deployment Flowcharts .................................................... 112–113

Nominal Group Technique (NGT)...................................... 114–115

Creative Problem Solving .........................................................116

Knot Chart ................................................................................117

Qualitative Problem Solving ......................................................118

Gantt Charts .............................................................................119

Notes ........................................................................................120
DAU PROGRAM MANAGERS TOOL KIT

1

I

ACQUISITION MANAGEMENT

• Things that make you go “Hmmm?...”

“The only thing most auditors fix is the blame.”

“Experience is something you got just after you needed it.”

“People are smarter than they look; listen to them.”

“The last 10 percent of the performance sought generates one-third

of the cost and two-thirds of the problems.”

“Never open a can of worms unless you want to go fishing.”

“Those who believe it cannot be done, will you please get out of the

way of those who are busy doing it?”

• Things we should always remember.

“Be honest in everything you say, write and do.”

“Be good to your people, and they will be good to you.”

“Forgiveness is easier to obtain than permission.”

“Keep everyone informed; when in doubt, coordinate.”

“Be the first to deliver bad news.”

“Bad news does not get any better with time.”

“If you are sitting at your desk, you are not managing your

program.”
DAU PROGRAM MANAGERS TOOL KIT

2

THE PROGRAM MANAGER’S BILL OF RIGHTS

AND RESPONSIBILITIES

RIGHTS:

Program Managers have the RIGHT to:

• A single, clear line of authority from the Defense Acquisition

Executive.

• Authority commensurate with their responsibilities.

• Timely decisions by senior leadership.

• Be candid and forthcoming without fear of personal consequences.

• Speak for their program and have their judgments respected.

• The best available training and experience for the job.

• Adequate financial and personnel resources.

RESPONSIBILITIES:

Program Managers have the RESPONSIBILITY to:

• Accept program direction from acquisition executives and

implement it expeditiously and conscientiously.

• Manage their programs to the best of their abilities within approved

resources.

• Be customer-focused and provide the user with the best, most

cost-effective systems or capabilities.

• Innovate, strive for optimal solutions, seek better ways to manage,

and provide lessons-learned to those who follow.

• Be candid about program status, including risks and problems as

well as potential solutions and likely outcomes.

• Prepare thorough estimates of financial and personnel resources

that will be required to manage the program.

• Identify weaknesses in the acquisition process and propose

solutions.
DAU PROGRAM MANAGERS TOOL KIT 3 Pre-Systems Acquisition

User Needs &

Technology Opportunities

PRODUCTION & DEPLOYMENT OPERATIONS

& SUPPORT

LRIP/IOT&E

TECHNOLOGY

 DEVELOPMENT

SYSTEM DEVELOPMENT

 & DEMONSTRATION

FRP

DECISION

REVIEW

DESIGN

READINESS

REVIEW

C IOC B

Systems Acquisition

DEFENSE ACQUISITION DECISION REVIEWS & PHASES

FOC

Increment 3

Increment 2

Sustainment

• Process entry at Milestones A, B or C• Entrance criteria met before entering phase• Evolutionary Acquisition or Single Step to Full Capability

A

CONCEPT

REFINEMENT

CONCEPTDECISION
DAU PROGRAM MANAGERS TOOL KIT

4

• DAB Review

• Designated by DAE

• Decision by DAE

• Component Review

• Designated by DAE

• Decision by Svc Sec/CAE

• ITAB Review*

• Designated by ASD(NII)**

• Decision by ASD(NII)

• Component Review

• Designated by ASD(NII)

• Decision made by

 Svc Sec/CAE

• Does Not Meet ACAT I Criteria

• Designated by Svc Sec/CAE

• Decision by Svc Sec/CAE

• Does Not Meet ACAT I, IA

or II Criteria

• Designated IAW Component

policy

• Decision at lowest appropriate

Level

• Not otherwise designated

ACAT I, IA, II or III

• Designated IAW Component

Policy

• Navy/USMC ACAT IVT/IVM

• Decision at lowest appropriate level

ACAT ID:

ACAT IC:

ACAT IAM:

ACAT IAC:

ACAT II:

ACAT III:

ACAT IV:

Major

Defense

Acq Pgms

Major AIS

Acq Pgms

Major

Systems

All Others

(except

 Navy and

USMC)

Navy

USMC

ACQUISITION CATEGORIES (ACAT)

SECNAVINST 5000.2_

No Fiscal

Criteria

$140M RDT&E or

$660M Procurement

(FY00 Constant $)

$378M Life Cycle Cost or

$126M Total Prog. Cost or

$32M Prog. Cost in any

single year

(FY00 Constant $)

$365M RDT&E or $2.190B

Procurement (FY00

Constant $)

* Information Technology Acquisition Board

** ASD, Networks & Information Integration (NII); formerly ASD (C3I)
DAU PROGRAM MANAGERS TOOL KIT

5

ACQUISITION STRATEGY CONSIDERATIONS

(Defense Acquisition Guidebook, Chapter 2)

• Program Structure

• Acquisition Approach

• Capability Needs

• Test & Evaluation

• Risk Management

• Resource Management

— Funding Under an Evolutionary

Acquisition Strategy

— Advance Procurement

• Systems Engineering Plan

• Interoperability

— Information Interoperability

— Other than Information Interoperability

• Information Technology

• Research & Technology Protection

— Protection of Critical Information

— Anti-Tamper Measures

• Information Assurance

• Product Support Strategy

• Human Systems Integration

• Environmental Safety, and Occupational

Health

• Modular Open Systems Approach

• Business Considerations

—Competition

◆Fostering a Competitive Environment

–Competition Advocates

–Ensuring Future Competition for

Defense Products

◆Building Competition into Individual

Acquisition Strategies

–Applying Competition to Acquisition

Phases

–Applying Competition to Evolutionary

Acquisition

–Competition and Source of Support

–Industry Involvement

◆Potential Obstacles to Competition

–Exclusive Teaming Arrangement

–Sub-Tier Competition

◆Potential Sources

–Market Research

–Commercial and Nondevelopmental

Items Dual-Use Technologies

–Use of Commercial Plants

–Industrial Capability

◆Small Business Innovative Research

(SBIR) Technologies

— International Cooperation

◆International Cooperative Strategy

◆International Interoperability

◆International Cooperation Compliance

◆Testing Required for Foreign Military Sales

—Contract Approach

◆Performance-Based Business Strategy

◆Modular Contracting

◆Contract Bundling

◆Major Contract(s) Planned

◆Multi-Year Contracting

◆Contract Type

◆Contract Incentives

◆Integrated Contract Performance

Management

◆Special Contract Terms and Conditions

◆Warranties

◆Component Breakout

—Leasing

—Equipment Valuation

◆Program Description

◆Accounting Review

◆Contract Implications

• Best Practices

• Relief, Exemption, or Waiver

• Additional Acquisition Strategy Topics

NOTE: In addition to the Acquisition Strategy, there are five plans required: Acquisition Plan

(FAR/DFARS), Program Protection Plan and TEMP (DoDI 5000.2), Information Support Plan

(ISP) (CJCSI 6212.01C), and Systems Engineering Plan (USD AT&L Memo 20 Feb 2004).
DAU PROGRAM MANAGERS TOOL KIT

6

ACQUISITION, TECHNOLOGY, AND

LOGISTICS (AT&L) KNOWLEDGE SHARING

SYSTEM (AKSS)

• AKSS Web Site URL: http://akss.dau.mil

• Central gateway for all of the AT&L resources and information (Replaces “Deskbook”)

• Communicates acquisition improvement initiatives to the AT&L workforce

• Provides links, shared information and references from various DoD organizations

and disciplines as part of integrated information system

• Quick links to knowledge sources include:

Defense Acquisition Resource Center (DARC); Defense Acquisition Guidebook

(DAG); Defense Contract Management Agency (DCMA) Instruction/Guidebook;

Security Assistance Management Manual for Foreign Military Sales (FMS);

Defense Contract Audit Agency (DCAA) Manual; DoD Instructions, Directives, and

Memoranda; Federal Acquisition Regulation (FAR), Defense Federal Acquisition

Regulation Supplement (DFARS); Community Areas; Guidebooks and Handbooks;

Education and Training; AT&L Web Sites; Software Tools; Glossaries and

Acronyms; News and Publications; Acquisition Events; and Ask-a-Professor

ACQUISITION COMMUNITY CONNECTION (ACC)

• ACC Web Site URL: http://acc.dau.mil

• Collaborative arm of the AT&L Knowledge System, consisting of publicly accessible

knowledge communities of practice

• Complements the existing information resources located on the AKSS

• Goal is connecting people with know-how across all DoD organizations and industry.

• Quick links to communities of practice and special interest areas include:

Communities of Practice: Contract Management; Data Management; DoD

Wireless; Facilities Engineering; Information Technology; Logistics Management;

Program Management; Risk Management; Systems Engineering

Special Interest Areas: Acquisition Center for Exellence (ACE) for Services;

Acquisition Research; Alternative Dispute Resolution; Business, Cost Estimating &

Financial Management; Contingency Contracting; Test & Evaluation/Science &

Technology [log in required]; Environment, Safety & Occupational Health; Earned

Value Management; Performance-Based Service Acquisition; Production Quality &

Manufacturing; Reliability & Maintainability; Software Acquisition Management;

Government Property; Spectrum Compliance; Total Ownership Cost
DAU PROGRAM MANAGERS TOOL KIT

7

 DEFENSE ACQUISITION

GUIDEBOOK (DAG)

• DAG Web Site URL: http://akss.dau.mil/dag

• Interactive, Web-based capability for DoD acquisition workforce and industry

partners providing instant reference/guide to best practices and supporting policies,

statutes, and lessons learned

• Quick links on Guidebook Web Site include:

Document View: Highlighted links to navigate from policy requirements in DoD

Directives to suggested solutions and processes in the Guidebook

Life Cycle Framework View: Explore the Guidebook through the Life Cycle

Framework Chart

Functional/Topical View: Review information related to each major functional

discipline, and how each discipline contributes to the acquisition process

Features:

• Easy navigation of the DoDD 5000.1, DoDI 5000.2 and the DAG

• Highlighted links for easy navigation from policy requirements to suggested

solutions and discretionary processes in Guidebook, other DoD sites, or

external resources

• Tables detailing statutory and regulatory requirements, guidebook links, and

associated Internet links.

• Multiple navigation and search tools to suit user needs

• Glossary of Acronyms

• Quick access to AT&L Knowledge Sharing System (AKSS) and to Acquisition

Community Connection (ACC)
DAU PROGRAM MANAGERS TOOL KIT

8

DEFENSE ACQUISITION BOARD TIMELINE

MILESTONES B, C & FRPDR

MILESTONE DECISION INFORMATION —

A POSSIBLE CONSTRUCT

OIPT

Review

OIPT

Review

Overarching Integrated Product Team (OIPT) Meetings

JROC

Review

JROC

Review

3-4 WEEKS

2 WEEKS 2 WEEKS

Milestone

45 DAYS

180 DAYS

30 DAYS

DAB DAB

TEMP to 

USD(AT&L)/

DOT&E

JCB

Review

JCB

Review

21 DAYS

CAIG Review

Draft POE

Draft 

CARD 

to CAIG

Final 

CARD 

to CAIG

CAIG briefs 

Preliminary 

LCCE to PM

AoA 

Results to 

PA&E

60 DAYS

CAIG final 

LCCE to PM

10 DAYS

Final POE 

& Component 

Cost Position to 

CAIG

3 DAYS

CAIG 

Report to 

OIPT

. ADM - Acquisition Decision Memorandum . CAIG - Cost Analysis Improvements Group . CARD - Cost Analysis Requirements Description . DAB - Defense Acquisition Board . FRPDR - Full Rate Production Decision Review . JCB - Joint Capabilities Board . JROC - Joint Requirements Oversight Council . LCCE - Life cycle cost estimate(s)

WHY ? WHAT ? HOW ?

RISKS ? COST ? MANAGEMENT ?

AGREEMENT ?

12 3

4 5 6

7 ☛ Have I presented all necessary information ?

☛ Does the information flow logically ?

☛ Is the information clear and accurate ?

☛ Is it concise, executive-level information?

• Threat

• Capability

• Requirement • Analysis of Alternatives • Acquisition Strategy

• Risk Mgmt Plan

• T&E Plan • T&E Results

• CAIV Objectives

• LCCE

• ICE

• PMO Structure

• IPT Structure • WIPT - OIPT Structure

• APB

• ADM

• Exit Criteria
DAU PROGRAM MANAGERS TOOL KIT

9

INFORMATION FOR MILESTONE/DECISION REVIEWS

(See DoDI 5000.2, CJCSI 3170.01D and CJCSI 6212.01C)

Milestone/Review

Information CD A B DRR C FRP

Acquisition Decision Memorandum5 X X XX XX

Acquisition Program Baseline5 X XX

Acquisition Strategy5 (see Page 5) X X X

Affordability Assessment X X

Analysis of Alternatives3&5 (AOA) X X X X

AOA Plan X

Benefit Analysis & Determination1&8 (bundled acquisitions) X

Beyond LRIP Report2 X

Capabilities Development Document (CDD)5 X

Capabilities Production Document (CPD) X

Certification of Compliance with Clinger-Cohen7 X X XX

Certification of Compliance with BEA7 (FM MAIS only) X X X X

Clinger-Cohen Act Compliance5&7 (MS-A, MAIS only) X X X X

Competition Analysis1&8 (depot-level maintenance rule) X

Compliance with Strategic Plan X

Component Cost Analysis5&9 (MAIS; optional MDAP) X X

Consideration of Technology Issues X X X

Cooperative Opportunities1 X X

Core Logistics/Source of Repair Analysis1&8 X

Cost Analysis Requirements Description5&9 (MDAP & MAIS) X X X

Economic Analysis (MAIS)7 (may be combined w/AoA at MS-A) X X X

Exit Criteria5 X XX XX

Industrial Capabilities1 (n/a MAIS) X X

Independent Cost & Manpower Estimate5 (MDAPs; n/a MAIS) X X X

Independent Tech. Assessment (ACAT ID only) (DDR&E Option) X X

Information Support Plan1&5 X X

Initial Capabilities Document (ICD)4&5 XXX X

J-6 Interoperability & Supportability Certification X X

J-6 Interoperability & Supportability Validation X

Live Fire T&E Waiver2 (covered systems) (n/a MAIS) X

Live Fire T&E Report2 (covered systems) (n/a MAIS) X

LRIP Quantities (n/a AIS) X

Market Research X X

Operational Test Agency Report of OT&E Results X X X

Post Deployment Performance Review X

Program Protection Plan1 X X

Pgm Environ, Safety & Occup Health5 (w/NEPA schedule) X X X

Registration of Msn Critical & Msn Essential Info Sys5&7 X XX

Spectrum Certification Compliance8 X X

System Threat Assessment5&6 X X

Systems Engineering Plan X X X X

Technology Development Strategy (MDAP & MAIS) X X X

Technology Readiness Assessment5 X X

Test & Evaluation Master Plan (T&E Strategy only due at MS A) X X X X

1. Summarized in Acquisition Strategy

2. OSD T&E oversight programs only

3. MDAP: A, B, C; MAIS: A, B, FRPDR

4. Milestone C if program initiation

5. Program initiation for ships

6. Validated by DIA for ACAT ID;

AIS use capstone InfoOps

sys threat assessment decision

7. Milestone C if equivalent to FRP

8. Milestone C if no milestone B

9. MAIS whenever an economic

analysis is required
DAU PROGRAM MANAGERS TOOL KIT

10

COCOM/

Service/

Agency

C4ISR INTEROPERABILITY

DoD Policy: (CJCSI 6212.01C)

• All C4I systems developed for use by U.S. forces are considered to be for Joint use.• All C4I requirements shall be reviewed/updated at every milestone.• Joint Staff will certify all requirements documents for conformance with Joint C4I Policy & Doctrine,

architectural integrity, and Interoperability standards.

Recommends:• Certify• Certify with Comments• Not Certify

Document Certification

Joint

Staff

Joint

Staff DISA

System Interoperability Validation

• Supports clarification of Interoperability test issues• Witnesses/conducts Interoperability tests

Interoperability

& Supportability

Validation

TEMP/

Test Results DISA/JITC

ICD

CDD

CPD

Certification

* These documents replaced the MNS and ORD.

ICD: Initial Capabilities DocumentCDD: Capability Development DocumentCPD: Capability Production DocumentDISA: Defense Information Systems AgencyJITC: Joint Interoperability Test CommandCOCOM: Combatant Commander

*

Interoperability

& Supportability

Certification

COCOM/

Service/

Agency
DAU PROGRAM MANAGERS TOOL KIT

11MDA

DECISION

Oversight

Panel

 Adv Tech Dev (6.3)

ACTDATDLab/field demoWarfighting Experiments

Warfighting Needs

&

R&D Objectives

 Tech Base (6.1/6.2) Basic Research Applied Research

Oversight

Panel

STOP

Systems

STOP

S&T

Options

(1)Variation on concept for new system/upgrade

of system out of production.

(2)Insert into ongoing technology or systems

development, or complete ACTD development.

(3)Upgrade system in production/fielded

systems, or produce mature ACTD.

(4)Use of new technology for

demilitarization/disposal.

S&T LINKAGE TO DEFENSE ACQUISITION PROCESS

(1) (2) (2) (3) (4) (3)

LRIP/IOT&E

FRP

DECISION

REVIEW

DESIGN

READINESS

REVIEW

FRP

DECISION

REVIEW

PRODUCTION & DEPLOYMENT OPERATIONS

& SUPPORT

TECHNOLOGY

 DEVELOPMENT

SYSTEM DEVELOPMENT

 & DEMONSTRATION

C IOC B FOC A

CONCEPT

REFINEMENT
DAU PROGRAM MANAGERS TOOL KIT

12

ADVANCED CONCEPT TECHNOLOGY DEMONSTRATION INITIATION PROCESS

Breakfast Club

briefing/

assessment

DUSD(ASC)

Review

JROC

briefing/

assessment

Overall

prioritization/approval

(USD(AT&L)-VCJCS)

FCB

ACTD

Concept Candidates

Services

Agency

Industry

Allies

Needs

JROC

Services

Unified Cmds.

Agencies

Developer Sponsor

NoteJROC – Joint Requirements Oversight Council; FCB – Functional Capabilities Board
DAU PROGRAM MANAGERS TOOL KIT

13

ACAT: Acquisition CategoryDT/OT: Developmental/Operational TestingDUSD(AS&C): Deputy Under Sec Def (Advanced Systems & Concepts)FAR: Federal Acquisition Regulation

FYDP: Future Years Defense ProgramICD: Initial Capabilities Document

MotivationRequirementOversightFundingACATConfiguration& TestingRulesRole of User

 Acquisition Program• Develop, produce and field system• Cost, schedule, performance

ICD/CDD/CPD

Milestone Decision

Authority

fully FYDP funded

all ACATs

system/subsystem

prototypes DT/OT

DoD 5000 Series/FAR

max involvement

 Advanced Technology Demonstration (ATD)• Demonstrate feasibility and maturity• Reduce technical risks and uncertainties at relatively low cost

not required

RDT&E

not ACAT effort

technology

demonstrations

informal/FAR/OTA

some involvement

CDD: Capability Development DocumentCPD: Capability Production DocumentOTA: Other Transactions AuthorityRDTE: Research, Development, Test & Evaluation (appropriation)

ACQUISITION PROGRAM VS. ATD & ACTD

DUSD(ASC) Oversight Panel

(Breakfast Club)

RDT&E (2 yrs in field)

not ACAT effort

tech demonstrations in field

environment with users

implementation directive/FAR/OTA

max involvement

 Advanced Concept Tech Demonstration (ACTD)• Gain understanding of and evaluate utility prior to acquisition decision• Develop concepts of operation and doctrine

labs/R&D centers

user sponsor & JROC prioritization
DAU PROGRAM MANAGERS TOOL KIT

14

PROGRAM STRUCTURE/SCHEDULE (EXAMPLE)

Contract

Award

(increment 1)

RDT&EProcurementO&MMILCON

FY01 FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 FY12 FY13 FY14

CR Tech Dev

Sys Integ Sys Demo

System Devel & Demo

CR

A

B

Prototype

Competition

DRR

FRP C IOC

B FRP IOC

FRP C

SI SD PROD

C

Milestones

Reviews

& Phases

Technical

Reviews

(increment 1)

Testing

(increment 1)

Deliveries

(increment 1)

OA

FOT&E

EDM

EOA

LRIP Production

LRIP Full-Rate Prod & Depl Sustainment/Disposal

Incr 2

Incr 3

(eng dev models)

DT&E

LFT&E

IOT&E

SRR

DDR

PCA PDR

TD

ASR

Prod & Deployment Operations & Support

CD

LRIP

SFR

Full-Rate Prod & Depl

B IOC

Full-Rate Prod

DDR

CDR SRR
DAU PROGRAM MANAGERS TOOL KIT

15

DoD INTERNATIONAL ARMAMENTS

COOPERATION POLICY

“PMs shall pursue international armaments cooperation to the maximum extent feasible, consistent with sound business practice and with

the overall political, economic, technological, and national security goals

of the United States. International agreements for international armaments

cooperation programs shall complete the interagency consultation and

Congressional notification requirements contained in 10 U.S.C. 2350a,

section 2751 of the Arms Export Control Act, and 10 U.S.C. 2531.”

— DoDI 5000.2 (Para E1.1)

THE SCOPE OF DEFENSE COOPERATION

Production &

RDT&E Procurement Follow-on Support

Information Foreign Military Sales Cooperative Logistics

Exchanges Supply Support

Engineer & Direct Commercial Sales Mutual Support

Scientist Exchanges Exchanges

Cooperative R&D Cooperative Production Logistics Support

(Joint Funds)

Comparative or Coproduction/Licensing Host Nation Support

Joint Testing (Foreign Funds) Defense Industrial

Base

Standardization Reciprocal Procurement

 The Program Manager’s Focus
DAU PROGRAM MANAGERS TOOL KIT

16

INTERNATIONAL ACTIVITIES ASSOCIATED WITH

DEFENSE ACQUISITION PHASES

Cooperative Production

Coproduction

 Licensed Production

 Production Sharing

Foreign Military Sales

Technology Opportunities

& User Capability Needs

NATO Forums

DEAs/IEPs

Staff Talks

S&E Exchanges

Cooperative Development

International Testing

Concept Refinement

and

Technology Development

System Demonstration

of SDD Phase

Production

and

Deployment,

Sustainment

DEFENSE SALES VS. COOPERATIVE ACQUISITION

They are Different

Note: DEA = Data Exchange Agreement

 IEP = Information Exchange Project

• Defense Sales

• Any Nation

• U.S. Contracts (FMS)

• U.S. Manages (FMS)

• Production & Support

• DoS or DoC

+ DoD – USD (Policy)

• Foreign Initiated

• Foreign Funds (or U.S.

Credit/Grants)

• Cooperative Acquisition

• Allied or Friendly

• U.S., Ally or NATO

• Jointly Managed

• All Acquisition

• DoD – USD (AT&L) +

Dept of State and

Dept of Commerce

• U.S. and/or Foreign

• Foreign + U.S. Funds
DAU PROGRAM MANAGERS TOOL KIT

17

RESOURCE ALLOCATION PROCESS—OVERLAP

D J J FM AM J A SON J F MA M J ASOND J J FMAMJ A SON D

FY05FY06FY07FY08FY09

CY05 CY06 CY07

Programming &

Budgeting Enactment Execution Planning

Programming &

Budgeting Enactment Execution Planning

Enactment Execution

Execution

J

Programming &

Budgeting Planning

FY06 

FY05 & prior 

FY06 & prior 

PCPs FY07-11BCPs FY07

FY07 FY07 & prior 

SPG/JPG 

FY08-13 

PCPs FY09-13BCPs FY09

POM FY08-13 

Budget FY08/09

FY08 FY08 &

prior 

SPG/JPG*

FY09-13

SPG/JPG* 

FY09-13

*SPG/JPG SECDEF Option in off-years

PCP: Program Change ProposalBCP: Budget Change ProposalPOM: Program Objectives

Memorandum

SPG: Strategic Planning GuidanceJPG: Joint Programming Guidance
DAU PROGRAM MANAGERS TOOL KIT

18

○○○○○○○○○○○○○○○○○○○○○○○○○○○○ JPD: Joint Planning Document NMS: National Military Strategy CPR: Chairman’s Program Recommendation SPG: Strategic Planning Guidance JPG: Joint Programming Guidance PB: President’s Budget NSS: National Security Strategy QDR: Quadrennial Defense Review

PLANNING, PROGRAMMING, BUDGETING AND EXECUTION (PPBE)—PLANNING

PHASE

FEB MAR SEP DEC FEB MAR APR/MAY

SECDEF

JPD SPG JPG

CPR

PresidentNational Security CouncilCIA/DIA/CJCS/OSD

CJCSJoint Staff/OSDCmbt CmdsDepts/Agencies

NSS

1st Year 2nd Year

Joint Staff/OSDCmbt CmdsDepts/Agencies

NMS

(every 4 years)

QDR

NOTE; In 1st year of a new administration the NSS is due 150 days after President takes office. QDR

starts after NSS is issued; QDR report is due concurrently with President’s Budget to Congress.
DAU PROGRAM MANAGERS TOOL KIT

19

PPBE ON-YEAR CONCURRENT

PROGRAM/BUDGET REVIEW (Even Year)

AUG SEP OCT NOV DEC JAN/FEB

Services

PEO/PM, SVC HQs

COCOMs

OSD/

OMB

Joint

Staff

POM

BES

FYDP

Issue

Resolution

CPA

Questions/

Hearings

PDM

DoD Components – PEO –

PM Answer/Reclama

FYDP

SECDEF

SLRG

PBD’s MBI PB

AUG SEP OCT NOV DEC JAN/FEB

Joint

Staff

PCP’s

BCP’s

Accepted PCP

Resolution

CPA

PBD’s MBI PB

PDM

DoD Components – PEO –

PM Answer/Reclama

SECDEF

SLRG

Adv Questions/

Hearings

PPBE OFF-YEAR PROGRAM/BUDGET REVIEW

(Odd Year)

COCOM: Combatant Commander

POM: Program Objectives Memorandum

BES: Budget Estimate Submission

CPA: Chairman’s Program Assessment

PBD: Program Budget Decision

Common Acronyms

3 Star Group

OSD/

OMB

OSD/

OMB

OSD/

OMB

3 Star Group

Services

PEO/PM,

SVC HQs

COCOMs

Notes: 3-Star Group replaces Program Review Group (PRG); SLRG replaces Defense Resources Board (DRB)

MBI: Major Budget Issues

PB: President’s Budget

PDM: Program Decision Memorandum

PCP: Program Change Proposal

BCP: Budget Change Proposal

PPBE: Planning, Programing, Budgeting

& Execution

SLRG: Senior Leadership Review

Group

OMB: Office of Management & Budget

FYDP FYDP
DAU PROGRAM MANAGERS TOOL KIT

20

RESOURCE ALLOCATION PROCESS

DoD

President

& OMB

Congress Authorization

Committees

Appropriation

Committees

Testimony Appeals

Authorization/

Appropriation

Acts Passed

Phase I:

PPBE

President’s

Budget

Apportionment

Allocation/

Execution

Phase IV:

Budget

Committees

Phase III:

CONGRESSIONAL ENACTMENT TIMETABLE

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT

15 15

Budget

Resolution

House

Senate

House

Senate

House

Senate

Authorization

Appropriation

Press

Budget

 Committee

 Inputs

 Floor

HRINGS MARKS FLOOR

HRINGS MARKS FLOOR

 Floor

 BILL/RPT

 HASC HRINGS MARKUP FLOOR FLOOR

CONF ACT

/RPT ^

 SASC HRINGS MARKUP FLOOR FLOOR

 BILL/RPT

 BILL/RPT

 HAC HRINGS MARKUP FLOOR FLOOR

CONF ACT

/RPT ^

SAC HRINGS MARKUP FLOOR FLOOR

BILL/RPT

CONF RESOLUTION

 ^

“TARGET”

FISCAL

YEAR

HAC – House Appropriations Committee

HASC – House Armed Services Committee

HRINGS – Hearings

SAC – Senate Appropriations Committee

SASC – Senate Armed Services Committee

Phase II: Enactment
DAU PROGRAM MANAGERS TOOL KIT

21

- 2031

- 2032

- 2033

- 2034

- 2035

- 1506

- 1507

- 1508

- 1611

- 1810

PROCUREMENT APPROPRIATIONS

(ACCOUNT NUMBERS AND BUDGET ACTIVITIES)

Appropriation Budget Activity

1 Aircraft

2 Modification of Aircraft

3 Spares & Repair Parts

4 Support Equipment & Facilities

1 Other Missiles

2 Modification of Missiles

3 Spares & Repair Parts

4 Support Equipment & Facilities

1 Track Combat Vehicle

2 Weapons & Other Combat Vehicles

3 Spares & Repair Parts

1 Ammo

2 Ammo Production Base Support

1 Tactical & Support Vehicle

2 Comm & Electronics Equipment

3 Other Support Equipment

4 Spares & Repair Parts

1 Combat Aircraft

2 Airlift Aircraft

3 Trainer Aircraft

4 Other Aircraft

5 Modification of Aircraft

6 Aircraft Spares & Repair Parts

7 Aircraft Support Equipment & Facilities

1 Ballistic Missiles

2 Other Missiles

3 Torpedoes & Related Equipment

4 Other Weapons

5 Not Used

6 Spares & Repair Parts

1 Ammo, Navy

2 Ammo, Marine Corps

1 Fleet Ballistic Missile Ships

2 Other Warships

3 Amphibious Ships

4 Mine Warfare & Patrol Ships

5 Auxiliary, Craft & Prior Year Costs

1 Ship Support Equipment

2 Comm & Electronics Equipment

3 Aviation Support Equipment

4 Ordnance Support Equipment

Army (21 -)

Aircraft

Missile

Weapons

Ammo

Other

Navy (17 -)

Aircraft

Weapons

Ammo

Shipbuilding

& Conversion

Other
DAU PROGRAM MANAGERS TOOL KIT

22

- 1810

- 1109

- 3010

- 3020

- 3011

- 3080

- 0300

- 0350

- 0360

- 0390

Appropriation Budget Activity

PROCUREMENT APPROPRIATIONS

(ACCOUNT NUMBERS AND BUDGET ACTIVITIES)

(Continued)

Other (cont.)

Marine Corps (17 -)

Procurement

Air Force (57 -)

Aircraft

Missile

Ammo

Other

Defense (97 -)

Defense-wide

National Guard

 & Reserve Equip.

Defense Production

 Activity Purchase

Chemical Agents

 & Munitions

 Destruction

6 Supply Support Equipment

7 Personal & Command Support Equipment

8 Spares & Repair Parts

1 Not Used

2 Weapons & Combat Vehicles

3 Guided Missiles & Equipment

4 Common & Electronics Equipment

5 Support Vehicles

6 Engineering & Other Equipment

7 Spares & Repair Parts

1 Combat Aircraft

2 Airlift Aircraft

3 Trainer Aircraft

4 Other Aircraft

5 Modification of In-Service Aircraft

6 Aircraft Spares & Repair Parts

7 Aircraft Support Equipment & Facilities

1 Ballistic Missiles

2 Other Missiles

3 Modification of In-Service Missile

4 Spares & Repair Parts

5 Other Support

1 Ammo

2 Weapons

1 Not Used

2 Vehicular Equipment

3 Electronics & Telecommunications Equipment

4 Other Base Maintenance & Support Equipment

5 Spares & Repair Parts

1 Major Equipment

2 Special Operations Command

3 Chemical/Biological Defense

1 Reserve Equipment

2 National Guard Equipment

1 Defense Production Activity Purchases

1 Chemical Agents & Munitions Destruction-RDT&E

2 Chemical Agents & Munitions Destruction-Procurement

3 Chemical Agents & Munitions Destruction-O&M
DAU PROGRAM MANAGERS TOOL KIT

23

RDT&E APPROPRIATIONS

Relationship Between MFP 6 (Research and Development)

Categories and RDT&E Appropriations Budget Activities

RDT&E APPROPRIATIONS

(ACCOUNT NUMBERS)

MFP 6 RDT&E

R&D Budget RDT&E

Category Activity Budget Activity Title

6.1 BA 1 Basic Research

6.2 BA 2 Applied Research

6.3 BA 3 Advanced Technology Development

6.4 BA 4 Advanced Component Development &

Prototypes

6.5 BA 5 System Development & Demonstration

6.6 BA 6 RDT&E Management Support

 --- BA 7 Operational System Development

*NOTE: Although similar, titles of the Major Force Program (MFP) six categories

(which are not shown above) are not exactly the same as titles of the RDT&E

Appropriation Budget Activities. In addition, the “Operational System Development”

Budget Activity for RDT&E BA 7 is not considered MFP 6. While correctly funded with

RDT&E dollars, these efforts do not fall under a MFP 6 Category; rather, for MFP

purposes, the efforts are considered part of the Major Force Program within which

the fielded operational system falls.

Account

Appropriation Number

RDT&E, Army 21 - 2040

RDT&E, Navy 17 - 1319

RDT&E, Air Force 57 - 3600

RDT&E, Defense-wide 97 - 0400

Development T&E, Defense 97 - 0450

Operational T&E, Defense 97 - 0460
DAU PROGRAM MANAGERS TOOL KIT

24

SAMPLE

NAVY APPROPRIATIONS AND BUDGET ACTIVITIES

*Below Threshold Reprogramming (BTR) amount limits are cumulative over entire period of time the

 specific fiscal year appropriation is available for obligation purposes (i.e., 1, 2, 3 or 5 years).

PROCUREMENT

SCN-1 Ship Conv - FBM Ships $20M** Lesser of 5 Full

SCN-2 Ship Conv - Other Warships $20M or

SCN-3 Ship Conv - Amphib Ships 20% of

SCN-4 Ship Conv - Mine & Pat Ships amount

SCN-5 Ship Conv - Aux, Craft & PY Costs appropriated

WPN-1 Weapons Proc. - Bal. Msl 3

WPN-2 Weapons Proc. - Other Msl

WPN-3 Weapons Proc. - Torp & Eq.

WPN-4 Weapons Proc. - Other Wpn

WPN-5 Weapons Proc. - Other Ord.

WPN-6 Weapons Proc. - Spares & Repair Parts

OPN-1 Other Proc. - Ship SE

OPN-2 Other Proc. - Comm/Elec Eq

OPN-3 Other Proc. - Aviation SE

OPN-4 Other Proc. - Ordnance SE

OPN-5 Other Proc. - Civil Engr SE

OPN-6 Other Proc. - Supply SE

OPN-7 Other Proc. - Pers & Com SE

OPN-8 Other Proc. - Spares & Repair Parts

APN-1 Aircraft Proc. - Combat

APN-2 Aircraft Proc. - Airlift

APN-3 Aircraft Proc. - Trainer

APN-4 Aircraft Proc. - Other

APN-5 Aircraft Proc. - Mods

APN-6 Aircraft Proc. - Spares

APR-7 Aircraft Proc. - SE & Fac.

Below Threshold Years Available

Reprogramming Rules for Obligation Funding

Other Appropriations / Titles Max In Max Out Purposes Policy

O&M, N Operations & Maintenance $15M No Congressional 1 Annual

Restriction

MILPER, N Military Personnel $10M No Congressional 1 Annual

Restriction

MILCON, N Military Construction Lesser of No Congressional 5 Full

+$2.0M or 25% Restriction

Appropriated

6.1 BA 1 Basic Research $10M** Lesser of 2 Incremental

6.2 BA 2 Applied Research $10M or

6.3 BA 3 Advanced Tech. Devel. 20% of

6.4 BA 4 Adv Comp Dev & Prototypes amount

6.5 BA 5 Sys Dev & Demo appropriated

6.6 BA 6 RDT&E Mgmt Supp

 (T&E Ranges)

 (Civilian Salaries)

BA 7 Operational Systems Devel.

 (Post-Production)

Procurement Below Threshold Years Available

Budget Buget Activity Reprogramming Rules for Obligation Funding

Activity Description Max In Max Out Purposes Policy

 [At Line Item Level]

 MFP 6 Below Threshold Years Available

 R&D RDT&E Budget Activity Reprogramming Rules For Obligation Funding

Category Number and Title Max In * Max Out * Purposes Policy

 [At Program Element Level]

 **RDT&E changed to $10M and procurement changed to $20M for FY03 – per 15 May 2003 OSD Comptroller memo.

Changes apply to FY04 and FY05.
DAU PROGRAM MANAGERS TOOL KIT

25

LINE ITEM

BELOW THRESHOLD REPROGRAMMING ACTIONS

 APPN MAX INTO MAX OUT LEVEL OF CONTROL OBL AVAIL

BUDGET ACTIVITY

SOME BA 1 SUB- ACTIVITY

LIMITATIONS ON DECREASES

(OPERATING FORCES)

LESSER OF

+ $ 2M

 + 25%

3 YEARS

(SCN: 5 YEARS)

PROJECT

RDT&E

PROC

O&M

MILPERS

MILCON

NONE, UNLESS

OTHERWISE

SPECIFIED

NO SPECIFIC

CONGRESSIONAL

RESTRICTION

NO SPECIFIC

CONGRESSIONAL

RESTRICTION

LESSER OF

- $ 10M

 - 20 %

+ $ 10M* PROGRAM ELEMENT 2 YEARS

+ $ 20M*

+ $ 15M

+ $ 10M BUDGET ACTIVITY 1 YEAR

1 YEAR

5 YEARS

LESSER OF

- $ 20 M

 - 20 %

Amounts are Cumulative Over Entire Period of Obligation Availability

APPROPRIATION LIFE

Current Period: Available for new obligations, obligation adjustments, expenditures, and outlays

O&M

RDT&E

Procurement

Ships

MILCON

MILPERS

 YEARS

1 2 3 4 5 6 7 8 9 10 11

12345

12345

1234567890123456789012

1234567890123456789012

1234567890123456789012

1234567890123456789012

1234567890123456789012

1234567890123456789012

1234567890123456789012

1234567890123456789012

1234567890123456789012

1234567890123456789012

1234567890123456789012

1234567890123456789012

1234567890123456789012

1234567890123456789012

1234567890123456789012

Expired Period: Available for obligation adjustments, expenditures, and outlays

Cancelled: Unavailable for obligations, obligation adjustments, expenditures, and outlays

1234567890123456789012345678901212 1234567890123456789012345678901212

1234567890123456789012345678901212

1234567890123456789012345678901212

1234567890123456789012345678901212 1234567890123456789012345678901212

1234567890123456789012345678901212

1234567890123456789012345678901212

1234567890123456789012345678901212 1234567890123456789012345678901212

1234567890123456789012345678901212

1234567890123456789012345678901212 1234567890123456789012345678901212

1234567890123456789012345678901212

1234567890123456789012345678901212 1234567890123456789012345678901212

1 2

1234567890123456789012345678901212

123456789012345678901234567890121 123456789012345678901234567890121

123456789012345678901234567890121

12345678 12345678

12345678

* RDT&E changed to $10M and Procurement changed to $20M for FY03, FY04, FY05 per OSD Comptroller.
DAU PROGRAM MANAGERS TOOL KIT

26

LIFE CYCLE COST COMPOSITION

PRODUCT IMPROVEMENTS

Funding Decision Tree

Does Proposed

Modification (Mod)

Increase System

Performance?

Is System

Currently In

Production?

NO

Procurement $ O&M $

NO

YES YES

IF . . . . .

THEN . . . . .

AND . . . .

Fund Purchase of the Mod Kits

and Installation of those Mod Kits

on the Fielded System with . . . . .

Fund Development

and Testing with. . . . .

(To Include the Mod

Kits used for Testing)

Is DT or IOT&E

Required?

Procurement $

YES

RDT&E $

NO

RDT & E

Development Costs of

PME & Support Items

Systems Engineering

Program Management

Test & Evaluation

PROCUREMENT

Prime

Equipment

Flyaway Cost

Development Cost

Procurement Cost

PROCUREMENT

Initial Spares

MILCON

Facilities

Operations &

 Support

• O&M , MILPERS

Disposal

• O&M (or others

PROCUREMENT as appropriate

Support Items

Weapon System Cost

Program Acquisition Cost

Life Cycle Cost
DAU PROGRAM MANAGERS TOOL KIT

27

Estimate Methods Comments

Analogy Comparison to one similar existing system;

based on judgments. Little or no data available;

relatively quick, easy, flexible. Used in early

phases (e.g., Concept Refinement & Tech Dev).

Parametric Comparison to many similar existing systems;

based on statistical analysis. Determine

primary cost drivers and Establish Cost

Estimating Relationships (CERs). Used in early

to mid-phases (e.g., Concept Refinement &

Tech Dev, and System Dev & Dem).

Engineering or Summation of “all” individual items in the system.

“Bottoms-Up” Uses Work Breakdown Structure (WBS)

for estimating purposes. Used in mid-phases

(e.g., System Dev & Dem).

Extrapolation Comparison to historical cost of same system.

Based on extrapolation from actuals. Uses

Learning Curve Theory. Used in late phases

(e.g., production and replenishment spares).

COST ESTIMATING

 1. Make sure cost data are relevant and homogeneous. Caution: Watch out for

 historical data in times of change. Prior actuals may include uncompensated

 overtime or were priced as a “buy-in.”

 2. Focus on cost drivers.

 3. Test sensitivities and data relationships.

Guidelines

COST ESTIMATING RELATIONSHIPS (CER)

— PARAMETRIC —

Parameter

(size, wt., etc.)

Regression Line

(Curve of Cost Means)

+ Sigma

– Sigma

= Similar Systems

Est. Cost ($)

+ Sigma

Average

(Expected)

Cost

– Sigma

Parameter

(e.g., size)
DAU PROGRAM MANAGERS TOOL KIT

28

Defining, Planning and Budgeting

Work Packages Planning Packages

(6-month coverage) (remainder of effort)

PERFORMANCE MEASUREMENT

COST & SCHEDULE PERFORMANCE MEASUREMENT

$45

1. Define the work (WBS)

2. Schedule the work

3. Allocate budgets } Cost Account

WBS

Task A

Task B

Task C

Tasks D-X

$10

$ 3

Element/Cost Account - 300

$ 4

4. Prepare and monitor performance profiles

TERMINOLOGY VARIANCES

BCWS - Budgeted Cost of Work Scheduled Cost Variance CV = BCWP - ACWP

BCWP - Budgeted Cost of Work Performed Schedule Variance SV = BCWP - BCWS

ACWP - Actual Cost of Work Performed

MR - Management Reserve Cost Variance % CV% =

EAC - Estimate at Completion (Govt)

NCC - Negotiated Contract Cost

BAC - Budget at Completion Schedule Variance % SV% =

CBB - Contract Budget Base (NCC+AUW)

TAB - Total Allocated Budget

PMB - Performance Measurement Baseline Variance at Completion VAC = BAC - EAC

AUW - Auth Unpriced Work

BCWP - ACWP

 BCWP

BCWP - BCWS

 BCWS

}

}

CBB

SCH VAR

ACWP

BCWP

MR

- COST VAR

NOW

BCWS

 BAC

EAC

TAB

PMB }

TIME

COST

BCWP

ACWP

BCWP

BCWS

 BCWP (cum)

BAC

 ACWP (cum)

BAC

PERFORMANCE INDICES

 Cost Performance Index CPI =

Schedule Performance Index SPI =

Percent Complete =

 Percent Spent =

=

ESTIMATE AT COMPLETION

 EAC BAC__

(Lowest Est.) CPI(cum)

 EAC = ACWP(cum) +

(Highest Est.)

BAC - BCWP(cum)

EAC - ACWP(cum)v

 BAC - BCWP(cum)

{CPI(cum) *SPI(cum)}

TCPI(eac) =

TO COMP PERFORMANCE INDICES
DAU PROGRAM MANAGERS TOOL KIT

29

TYPES OF CONTRACTS

Cost Family: Appropriate when product not well defined; high risk; contractor provides best efforts; Government pays all allowable costs. Fee varies by type.

Cost Plus Fixed Fee (CPFF) - Fee same regardless of actual cost outcome.

Cost Plus Incentive Fee (CPIF) - Actual fee earned computed by applying share

ratio to over/under run, subject to min/max fee limits.

Fixed Price Family: Product well defined, low risk; contractor must deliver product.

Firm Fixed Price (FFP) - Price fixed regardless of actual cost incurred.

Fixed Price Incentive Firm Target [FPI(F)] - Final price computed by applying

share ratio to over/underrun, subject to ceiling price limitation.

Award Fee (AF) - Either stand-alone Cost Plus Award Fee (CPAF) or combined

with cost or fixed price types. AF unilaterally determined by government based on

subjective evaluation of contractor’s performance.

Fee Limits: CPFF - Fee limited to 15% for R&D; 10% for Production and services.

No statutory or FAR/DFARS regulatory limitation on other contract types.

 Contract Price = Cost + Profit/Fee

 Direct Cost Indirect Cost

Direct Labor Direct Material Overhead G&A FCCM

 Other

Direct Cost

 (ODC)

Engrg Mfg Raw Purchased

Labor Labor Material Parts Engrg Mfg Mat’l

 Std Comm Spt Spt Handling

 Items

Subcontracts

 Interdivisional

 Transfers

TYPICAL CONTRACT TYPE BY PHASE

 CR TD SDD/SI SDD/SD PROD

 CPFF, FFP CPFF,FFP CPFF, CPIF CPIF, CPAF FPI(F), FFP

CONTRACTING—COMPONENTS OF

CONTRACT PRICE
DAU PROGRAM MANAGERS TOOL KIT

30

CONTRACT TYPE FEATURES

Promise

Contractor Risk

Cash Flow

Progress Payments %

Administration

Fee Limit %

FIXED

PRICE

Delivery

High

Delivery

75/90/95

Low

None

COST

REIMBURSEMENT

Best Efforts

Low

As Incurred

N/A

High

15/10/6 on CPFF

FEE

FIXED

FEE

(SHARE 100/0)

COST ESTIMATED COST

PRICE = COST + FIXED FEE

Risk Highest to the Government

Obtains Fee Regardless of Cost

CPFF

FEE

COST TARGET COST

(Target) PRICE = (Target) COST + (Target) FEE

All Reasonable Cost Paid

Shared Risk Between Min/Max Fee

MAX

TARGET

MIN

CPIF

Share

 Ratio
DAU PROGRAM MANAGERS TOOL KIT

31

COST

CONTRACT TYPE FEATURES

(Continued)

COST TARGET COST

 (Target) PRICE = (Target) COST + (Target) Profit

0/100 SHARE

FPI(F)

FFP

PRICE = COST + PROFIT

PTA

Share

 Ratio

CEILING

PRICE

 PROFIT

PROFIT

TARGET

PROFIT

Point of Total

Assumption = CEILING PRICE - TARGET PRICE + Target Cost

(PTA) GOVERNMENT SHARE
DAU PROGRAM MANAGERS TOOL KIT

32

PRE-SOLICITATION PROCESS

POST-SOLICITATION PROCESS

FBO/CBD – FedBizOps/Commerce Business Daily

RFP – Request for Proposal

SSA – Source Selection Authority

SSAC – Source Selection Advisory Council

Market

Research

Acquisition

Strategy Requirement

Acquisition

Plan

Post Draft RFP

on Electronic

Bulletin Board

FBO/CBD Notice

Advisory Multi-Step

Finalize

RFP

CBD Notice of

RFP

Release

RFP Release

to Industry

Source

Selection Plan/

Strategy

RFP Release

Briefing to

SSA

SSA

Decision

Receipt of Oral

& Written

Proposals

Initial Eval Clarifications

Limited Communications

Competitive

Range

Determination

Face-to-Face

Discussions/

Negotiations

Receive & Analyze

Field Surveys

(if requested)

Prepare for Discussions

with

Remaining Offerors

Debrief

Unsuccessful

Offerors

Contract Award

(Distribution)

 Brief

SSAC

Request Final

Proposal

Revision

Receive &

Analyze

Final Revision

Brief

SSA
DAU PROGRAM MANAGERS TOOL KIT

33

• GSA Multiple Award Schedules (MAS)

– General Services Administration contracts for both products and

services–available to all agencies.

• Government–wide Agency Contracts (GWACs)

– Similar to MAS but more restricted in products and services

available.

• Indefinite Delivery/Indefinite Quantity Contracts

– Task orders (services) & Delivery Orders (products) issued under

omnibus umbrella contract.

• Other Transactions (OT)

– Defined: Vehicles used for basic, applied and advanced research

projects and prototype development. OTs are not Contracts,

Grants, or Cooperative Agreements.

– Objective: Attract commercial companies and consortia that

historically have not done business with the Department of

Defense because of statutory and/or regulatory requirements.

OTs are not subject to the Federal Acquisition Regulation.

Designed to increase DoD access to dual use Technologies.

– Research Projects:

◆ Where practical, government cost share should not exceed

cost share of other parties.

◆ Use OT when standard Contract, Grant or Cooperative

 Agreement is not appropriate.

– Prototype Projects:

◆ Must be directly relevant to weapons or weapon systems

proposed to be acquired or developed by DoD.

– Constraints:

◆ At least one nontraditional contractor participating.

◆ If no nontraditional contractor participates, 1/3 of cost paid by

parties other than federal government, or, senior procurement

executive justifies transaction

– OT Guide for Prototype Projects, January 2001.

OTHER WAYS TO BUY
DAU PROGRAM MANAGERS TOOL KIT

34

CONTRACTOR PROFITABILITY RATIOS

The basic concept of profitability ratios is to measure net income against revenue or against the

investment required to produce it. There are three principal profitability ratios with which you should

be familiar. They are:

1. Return on Sales which shows what percentage of dollars are left after the company has paid

for all costs, interest, and taxes. It is expressed as:

Return on Sales = Net Income

Sales

2. Return on Total Assets, which looks at the efficiency with which management has used its

resources, the company’s assets, to generate income. It is computed as:

ROA = Net Income

Total Assets

As noted, Return on Assets (ROAs) addresses how well management utilizes the assets of the

firm in generating income. The ROA formula reflects the combined result of Return on Sales and

the total asset turnover ratio (total sales/total assets), broken down as follows:

ROA = Net Income X Total Sales

Total Sales Total Assets

3. Return on Stockholders’ Equity measures the rate of return on the owners’ investment—their

equity in the company. This is also known as Return on Equity (ROE).

ROE = Net Income

 Stockholders’ Equity

ROE can also be broken into two components: these being return on assets and financial leverage

(a ratio reflecting the relationship of creditor to owner financing—expressed as total assets/

stockholders equity). This is shown by:

ROE = Net Income X Total Assets

Total Assets Stockholders’ Equity

These profitability ratios give three different viewpoints concerning the “bottom line” on the income

statement—how much net profit is being made on each sale, how much is being made for the

assets that are employed, and how much is being made for the company owners. Contractor

profitability ratios for the Aerospace/Defense Industry for the period of 1980 to date are shown on

page 35.

From an owner’s perspective, another profitability ratio you may be aware of is Earnings Per

Share (EPS):

Earnings Per Share = Net Income 

Number of Shares of Common Stock Outstanding
DAU PROGRAM MANAGERS TOOL KIT

35

AEROSPACE DEFENSE INDUSTRY

CONTRACTOR PROFITABILITY RATIOS

Aerospace/Defense Industry

Contractor Profitability Ratios

RETURN ASSET RETURN FINANCIAL RETURN

ON SALES TURNOVER ON ASSETS LEVERAGE ON EQUITY

(NI/S) (S/TA) (NI/TA) (TA/SE) (NI/SE)

1980 4.3 1.21 5.2 3.08 16.0

1981 4.4 1.18 5.2 3.06 16.0

1982 3.3 1.12 3.7 3.24 12.0

1983 3.5 1.17 4.1 2.98 12.1

1984 4.1 1.15 4.7 3.00 14.1

1985 3.1 1.13 3.6 3.17 11.1

1986 2.8 1.07 3.1 3.13 9.4

1987 4.1 1.07 4.4 3.32 14.6

1988 4.3 1.02 4.4 3.39 14.9

1989 3.3 1.00 3.3 3.24 10.7

1990 3.4 1.00 3.4 3.38 11.5

1991 1.8 1.06 1.9 3.21 6.1

1992 -1.4 0.86 -1.2 4.33 -5.2

1993 3.6 0.97 3.5 3.80 13.2

1994 4.7 0.92 4.3 3.44 14.8

1995 3.8 0.92 3.5 3.17 11.1

1996 5.6 0.91 5.1 3.35 17.1

1997 5.2 0.92 4.8 3.60 17.3

1998 5.0 0.96 4.8 3.73 18.0

1999 6.5 0.95 6.2 3.52 21.8

2000 4.7 0.91 4.3 3.30 14.2

2001 3.9 0.92 3.6 3.22 11.6

2002 4.1 0.90 3.7 3.16 11.7

2003 3.1 0.84 2.6 3.81 9.9

 AVERAGE 3.8 1.0 3.8 3.4 12.7

Source: Aerospace Industries Association
DAU PROGRAM MANAGERS TOOL KIT

36

XX

LOCKHEED MARTIN CORP — DUPONT FORMULA ANALYSIS (AN EXAMPLE)

==

4.00%3.00%2.00%1.00%0.00%

-1.00%-2.00%-3.00%-4.00%-5.00%

Return on Sales (%)

(Net Income/Sales) 

1999 2002 2003 2000 2001

5.00%4.00%3.00%2.00%1.00%0.00%

-1.00%-2.00%-3.00%-4.00%-5.00%

Return on Assets (%)

(Net Income/Total Assets)

1999 2002 2003 2000 2001

5.0004.5004.0003.5003.0002.5002.0001.5001.0000.5000.000

Financial Leverage

(Total Assets/Shareholder Equity) 

1999 2000 2001 2002 2003

5.00%4.00%3.00%2.00%1.00%0.00%

-1.00%-2.00%-3.00%-4.00%-5.00%

Return on Assets (%)

(Net Income/Total Assets)

1999 2002 2003 2000 2001

Fiscal

Year

Total Asset

Turnover

Return 

on Assets % 

Financial

Leverage

Return

on Equity %

Return on

Sales % =

X X 4.757 =

X = X 4.249 =

X = X 4.301 =

X X 4.600 =

1999

2000

2001

2002

2003

1.53%

-2.11%

-4.36%

1.88%

3.31% X

0.826

0.807

0.866

0.985

1.216

1.26%

-1.71%

-3.77%

1.85%

4.02%

1.26%

-1.71%

-3.77%

1.85%

4.02% X 3.874 =

6.01%

7.25%

-16.23%

8.53%

15.59%

Return 

on Assets % X =

=

=

=

X

20.00%15.00%11.00%10.00%

5.00%0.00%

-5.00%

-10.00%-15.00%-20.00%

Return on Equity (%)

(Net Income/Shareholder Equity) 

1999 2002 2003 2001 2000

1.41.21.00.80.60.40.2

0

Total Asset Turnover

(Sales/Total Assets) 

1999 2000 2001 2002 2003
DAU PROGRAM MANAGERS TOOL KIT

37

CASH CYCLE

— Delivery

CONTRACTOR FINANCING AND PAYMENTS

*Internal Contractor Financing

 — Retained Earnings

— Govt. specified

— Offer or proposed

— Interim

— Advance

—Private

• Trade Credit

• Bank Credit

— Revolving Credit

— Term Loan

— Government

— Periodic

— Partial

FINANCING (External*) PAYMENTS

Commercial Noncommercial Commercial Noncommercial

• For Non-Commercial

— Progress Payments

• Performance Based

• Cost Incurred Based

• % Complete

— Unusual Progress Payments

— Assignment of Claims

— Guaranteed Loans

— Advance Payments

Cash

received

Accounts

receivable

Sale

(DD 250) Finished goods

inventory

Raw material

inventory

Contract

award

Cash

disbursed

Cash

disbursed

Wages

payable

Work in process

inventory

Accounts

payable
DAU PROGRAM MANAGERS TOOL KIT

38

Indirect costs

• Costs that can’t be traced

to a single contract

because they are

associated with

multiple contracts

• Example: electricity for

the company’s facilities

Direct costs

• Costs that can be

traced to a single

contract

• Examples: material

and labor to assemble

an aircraft

DIRECT AND INDIRECT COSTS

Indirect costs are

assigned to

contracts using

indirect rate(s).

(Traced directly)

(Traced directly)

Direct

material

Direct

labor

Contract

ASSIGNING INDIRECT COSTS

Calculation of Indirect Rates

Note: See DAU Indirect-Cost Management Guide at

http://www.dau.mil/pubs/gdbks/icmguide.asp

INDIRECT RATE = Indirect Cost Pool

Allocation Base
DAU PROGRAM MANAGERS TOOL KIT

39

LIFE CYCLE OF INDIRECT COST RATES

BIDDING ON

CONTRACTS

ADJUSTING

PAYMENT &

CLOSING

CONTRACTS

PAYING

CONTRACTS

FORWARD

PRICING

RATES

BILLING

RATES

ACTUAL

RATES

Direct material $ 40,000

Material handling 10% 4,000

Direct engineering labor 6,000

Engineering overhead 100% 6,000

Direct manufacturing labor 12,000

Manufacturing overhead 150% 18,000

Other direct costs 6,000

Subtotal 92,000

General and administrative 25% 23,000

Total cost 115,000

Profit 15% 17,250

Cost of money for facilities capital employe 1,500

Price $133,750

CONTRACTOR’S COST PROPOSAL

EXAMPLE
DAU PROGRAM MANAGERS TOOL KIT

40

CONTRACTOR BUSINESS PLANNING PROCESS

OUTPUTS

MANPOWER PLAN

Rates

for

Estimating

SALES

ANNUAL

OPERATING PLAN

CAPITAL

INVESTMENT PLAN

PRODUCTION &

ENGINEERING PLAN

MASTER DELIVERY

SCHEDULE

IR&D/B&P PLAN

TOP MANAGEMENT

GOALS, OBJECTIVES,

STRATEGIES

OVERHEAD

FORECAST

BUSINESS BASE

SALES FORECAST

$ $

TREND
DAU PROGRAM MANAGERS TOOL KIT

41

System

Performance

Technical

Effectiveness

System

Effectiveness

System

Availability

Process

Efficiency

Capabilities

Functions

Priorities

Reliability

Maintainability

Supportability

Producibility

Operations

Maintenance

Logistics

Life Cycle Cost / Total Ownership Cost (TOC)/

Cost as an Independent Variable (CAIV)

Affordable

Operational

Effectiveness

◆ System Training

◆ System Documentation

◆ Supply Support (including Spares)

◆ Sustainment Planning

◆ Test and Support Equipment

◆ Facilities

◆ Packaging, Handling, and Transportation

◆ Manpower

SYSTEM/PRODUCT SUPPORT PACKAGE:

SYSTEM OPERATIONAL EFFECTIVENESS (SOE)

PRINCIPAL LIFE CYCLE LOGISTICS

GOALS/OBJECTIVES

• Influence product design for affordable System Operational

Effectiveness

• Design and develop the support system utilizing Performance

Based Logistics

• Acquire and concurrently deploy the supportable system, including

support infrastructure

• Maintain/improve readiness, improve affordability, and minimize

logistics footprint

LIFE CYCLE LOGISTICS

• The planning, development, implementation, and management of a

comprehensive, affordable, and effective systems support strategy,

within Total Life Cycle Systems Management. Life cycle logistics

encompasses the entire system’s life cycle including acquisition

(design, develop, test, produce and deploy), sustainment

(operations and support), and disposal.
DAU PROGRAM MANAGERS TOOL KIT

42

Maintenance Planning – establishes maintenance concepts and requirements.

Manpower & Personnel – identification of personnel skills, grades and

quantity required to support operation and maintenance of system.

Supply Support – determine requirements to acquire and manage spares

and repair parts.

Support Equipment – identify all equipment required to support operation

and maintenance of the system.

Technical Data – scientific and technical information used to support

systems acquisition.

Training & Training Support – determine requirements to acquire training

devices and conduct training of operators and maintenance personnel.

Computer Resources Support – identification of facilities, hardware,

software and support tools to operate and support embedded computer systems.

Facilities – identify real property required to support system.

Packaging, Handling, Storage and Transportation – identify designs and

methods to ensure the system is preserved, packed, stored, handled

and transported properly.

Design Interface – relationships of logistics-related design parameters to

readiness and support resources requirements; influence design for

supportability.

SUPPORT ELEMENT DEFINITIONS

BEST PRACTICE:

Support Elements

Maintenance

Planning

Manpower &

Personnel

Supply

Support

Support

Equipment Training &

Training Support

Technical

Data

Design

Interface

Computer

Resources

Support

SYSTEM

Facilities Packaging, Handling,

Storage and

Transportation

Note: Under Performance-Based Logistics (PBL), Support Elements are still valid even though

Government is buying results and solutions, not specific resources or processes.
DAU PROGRAM MANAGERS TOOL KIT

43

Acquisition

Strategy

Support

Strategy

LIFE CYCLE LOGISTICS LINKAGE TO DEFENSE ACQUISITION PROCESS

PRODUCTION & DEPLOYMENT OPERATIONS &

SUPPORT FULL-RATE

PRODUCTION &

DEPLOYMENT

LOW-RATE

INITIAL

PRODUCTION

CONCEPT

REFINEMENT

TECHNOLOGY

 DEVELOPMENT

SYSTEM DEV & DEMONSTRATION

SYSTEM

 DEMONSTRATION

SYSTEM

 INTEGRATION

FRP DECISION REVIEW DESIGN READINESS REVIEW

User Needs & Technology Opportunities

C IOC FOC

Pre-Systems Acquisition Sustainment

Sustainment Disposal

Produce Product

Produce

Support System

Demonstrate

Supportability

& Life Cycle

Affordability

GOAL 1

GOAL 4

GOAL 2

Design Product

for Supportability

Design Develop

Support System

Acquire & Deploy

Supportable System

GOAL 3

Maintain/Improve

Readiness &

Improve Affordability

Readiness &

Total Ownership

Cost Objectives

Performance

Based Logistics

(PBL)

Evaluate

Alternative

Logistics

Concepts

Support

Concepts

IOC

FACTORY

CLOSED

Systems Acquisition

B

A

INCREMENT 3 CONCEPT DECISION

SE Process SE Process

Evaluate

Product Support

Capabilities

Develop Initial

Product

Support Strategy

Product

Support Plan

Demo Product

Support

Capability

PBL & Product

Support

Implementation

PBL & Product

Support

Management

Operations/

Sustainment

INCREMENT 2

Inputs to:• CDD• TEMP• APB• Contract

Demonstrate

Supportability

& Life Cycle

Affordability

Single Step or

Evolution to Full

Capability

Demil

&

Disposal

Modifications &

Technology Insertion

Post-Deployment

Evaluations

Post-Deployment

Support
DAU PROGRAM MANAGERS TOOL KIT

44

PERFORMANCE-BASED LOGISTICS (PBL)

• The purchase of support as an integrated, affordable, performance

package designed to optimize system readiness and meet

performance goals for a weapon system through long-term support

arrangements with clear lines of authority and responsibility.

• PBL is DoD’s preferred approach for product support

implementation.

PERFORMANCE-BASED LOGISTICS

IMPLEMENTATION MODEL

Integrate

Requirements

 & Support

Form the

PBL Team

Baseline

The System

Identify

Performance

Outcomes

Select

Product

Support

Integrator

Supply Chain

Management

Strategy

Establish

Performance

Based

Agreements

Business

Case

 Analysis

Accomplish

Contracts

Financial

Enablers

Implement

And

Oversight

Workload

Allocation

Strategy
DAU PROGRAM MANAGERS TOOL KIT

45

Sufficient Spares

And Repair Parts for

System Life?

Yes

Yes

No

No

Monitor

Usage Data

Multiple

Sources?

Evaluate

Alternatives

Monitor

Vendors

Sole Source

Reprocurement

Competitive

Reprocurement

Substitute

Parts

Component

Redesign

Life of Type

Buy

POST-PRODUCTION SUPPORT DECISION PROCESS

Logistics

 Down Time

(LDT)

Corrective

Maintenance Time

(CMT)

Administrative

Delay Time

(ADT)

Preventive

Maintenance Time

(PMT)

• Locating tools

• Setting up test

 equipment

• Finding personnel

 (trained)

• Reviewing manuals

• Complying with

 supply procedures

• Parts Availability

“In the Bin”

• Needed items

awaiting

transportation

• Preparation time

• Fault location time

• Getting parts

• Correcting fault

• Test and checkout

• Servicing

• Inspection

OPERATIONAL AVAILABILITY

Standby

Time

Operating

Time

Uptime

Downtime

AO =

Uptime +
DAU PROGRAM MANAGERS TOOL KIT

46

LOGISTICS MANAGEMENT

COMMUNITY OF PRACTICE (LOG COP)

Transforming the Way We Work

Where YOU can ....

Find Helpful Tools & Templates

• Access Latest PBL Resources

• Supportability Best Practices

• Contracting Lessons Learned

Get Ahead In YOUR Career

• Logistics Training & Education

• Latest OSD Policy and Direction

• Logistics Conferences/Events

• Link to Top DoD Web sites

Connect With Professionals

• Share Experiences & Ideas

• Start and Join Discussions

• Locate DoD & Industry Experts http://log.dau.mil

DEFENSE ACQUISITION GUIDEBOOK

— LIFE CYCLE LOGISTICS —

http://akss.dau.mil/dag
DAU PROGRAM MANAGERS TOOL KIT

47

FUNCTIONAL

DIVISIONS

FUNCTIONAL

DIVISIONS

FUNCTIONAL

DIVISIONS

STAFF STAFF STAFF

PM

“Pure” Product Structure

PM

SYSTEM A

PM

SYSTEM B

PM

SYSTEM C

PROGRAM OFFICE ORGANIZATION STRUCTURES

(Examples)

ENGINEERING BUSINESS PRODUCTION LOGISTICS

PM

STAFF

Functional Structure
DAU PROGRAM MANAGERS TOOL KIT

48

ENGRNG

PROGRAM OFFICE ORGANIZATION STRUCTURE

(Continued)

Matrix Structure

PEO

PM PM PM

STAFF STAFF STAFF

ENGRNG PRODUCTION

BUSINESS LOGISTICS

FUNCTIONAL

DIVISIONS

FUNCTIONAL

DIVISIONS

SYSCOM

HEAD

FUNCTIONAL

DIRECTORS

LOGISTICS

PRODUCTION

BUSINESS
DAU PROGRAM MANAGERS TOOL KIT

49

IPT = Integrated Product Team

PIT = Program Integration Team

Note 1: IPTs mirror Work Breakdown Structure

PROGRAM OFFICE ORGANIZATION STRUCTURE

(Continued)

PIT

AIR VEHICLE

IPT

ENGINE

IPT

SUPPORT SYS

IPT

TRAINING SYS

IPT

Note 1

INTEGRATED PRODUCT TEAMS

SYSTEM PROGRAM DIRECTOR

DIR OF

LOG

DIR OF

TEST

DIR OF

CONTRACTS

DIR OF

PROJECTS

DIR OF

ENG

DIR OF

FIN MGT
DAU PROGRAM MANAGERS TOOL KIT

50

THE ROLE OF MANUFACTURING IN THE

ACQUISITION PROCESS

SUSTAINMENT FULL

RATE CR TD LRIP DEMONSTRATION

 INTEGRATION

A B C

• Execute the Manufacturing Plan

– Reflect Design Intent

– Repeatable Processes

• Continuous Process Improvement

PRODUCTION

• Influence the Design

Process

• Prepare for Production

RDT&E

○○○○○○ ○○○○○○○○○○○

• Producible Design

• Factory Floor

Characterized

MILESTONE C EXIT

REQUIREMENTS

CURRENT DoD 5000 PROCESS

• Unstable requirements/engineering changes

• Unstable production rates and quantities

• Insufficient process proofing

• Insufficient materials characterization

• Changes in proven materials, processes, subcontractors,

 vendors, components

• Producibility

• Configuration management

• Subcontractor management

• Special tooling

• Special test equipment

COMMON PRODUCTION RISKS THAT GREATLY

IMPACT COST, SCHEDULE & PERFORMANCE

• Uniform, Defect-Free

Product

• Consistent Performance

• Lower Cost

NET RESULT:
DAU PROGRAM MANAGERS TOOL KIT

51

PRODUCIBILITY

DEFINITION:

The measure of relative ease of manufacturing a product. The product

should be easily and economically fabricated, assembled, inspected,

and tested with high quality on the first attempt that meets performance

thresholds.

PRODUCIBILITY ISSUES:

• Design engineering, NOT manufacturing, is the technical group responsibility for producibility.

Program offices and design engineers often dislike producibility because it usually requires

performance functionality sacrifices (especially if cost is a set value, i.e., CAIV).

• Many design engineers do not have proper training or experience in designing for producibility.

Manufacturing facilities must be explicitly recognized as a major design constraint. This

includes process capabilities and rate capabilities at each facility.

* The PM is responsible for Producibility*

PRODUCIBILITY

Defense Acquisition Guidebook,

4.4.6.1 Producibility

• Producibility: degree to which system design facilitates timely, affordable, optimum-quality

manufacture, assembly, and delivery of system

• Producible system design should be a development priority.

• Design engineering efforts concurrently develop:

– Producible and testable design

– Capable manufacturing processes

– Necessary process controls to:

◆ Meet requirements

◆ Minimize manufacturing costs

• PM should use existing manufacturing processes whenever possible

• When design requires new manufacturing capabilities, PM needs to consider process

flexibility (e.g., rate and configuration insensitivity)

• Full-rate production necessitates:

– Stable systems design

– Proven manufacturing processes

– Available production facilities and equipment
DAU PROGRAM MANAGERS TOOL KIT

52

QUALITY MANAGEMENT SYSTEMS

Defense Acquisition Guidebook, 4.4.7 Quality

• The program manager should allow contractors to define and use

their preferred quality management system that meets required

program support capabilities.

• The program manager will not require ISO registration of a supplier’s

quality program. ISO compliance is just one means that a program

manager uses to distinguish between multiple bidders.

• Contractor’s quality management system should be capable of the

following key activities:

– Monitor, measure, analyze, control, and improve processes;

– Reduce product variation;

– Measure/verify product conformity;

– Establish mechanisms for field product performance feedback; and

– Implement an effective root-cause analysis and corrective action

system.

Notes: ISO 9000 Series International Quality Standard is considered a Basic Quality system,

but the focus is still on, “Document what you do. Do what you document.”

Advanced Quality Systems (AQS) such as the new SAE AS9100B Aerospace industries

Quality Standard focus on achieving Customer Satisfaction via use of Key Characteristics

Identification & Control, Variation Reduction of Key Characteristics, Flow-Down of similar

process control requirements to Suppliers, and many other advanced process-oriented

control and improvement techniques

KEY CHARACTERISTICS AND

VARIATION REDUCTION

Minimize and control variation on both key product characteristics and corresponding key manufacturing process characteristics:

• Key Characteristics: The features of a material, process or part

whose variation has a significant influence on product fit,

performance, service life or manufacturability—Per SAE AS9100B

• Major Sources of Variation: Insufficient design margins, process

(manpower, machinery, methods, etc.), measurement systems,

supplier’s products

WHY: Direct correlation between deviation from nominal value (i.e., variation) on key characteristics, and product quality and functionality

TOOLS: Quality Function Deployment (QFD), Design of Experiments

(DOE), Statistical Process Control (See control chart on next page.)
DAU PROGRAM MANAGERS TOOL KIT

53

KEY CHARACTERISTICS AND VARIATION

REDUCTION

(continued)

The (X bar) and R Control Charts are used to monitor manufacturing processes. Upper or

Lower Control Limits (UCL or LCL) are NOT design specification parameters. Instead, they are

predicted boundaries for stable processes, calculated using (X double bar) (average of

sampled process Means), (R Bar) (the average of the sample Ranges, which are the spreads

between extreme values per sample), plus the selected data sample size and process-keyed

statistical formulas. Values outside the UCL and/or LCL indicate possible process instability, likely

due to uncommon “special” causes of variation. Caution: A process in control is desirable

because it is predictable, yet it could fail to meet design requirements due to inherent “common”

variation and/or because the process average isn’t centered on the design nominal value.

Reference: The Memory JoggerTM II; ©1994 by GOAL/QPC

.90

.85

.80

.75

.70

.65

.60

.55

_

X

UCL

LCL

1 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 20 21 22 23 24 25 2 3

.40

.30

.20

.10

R =

.178

UCL

19

1 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 20 21 22 23 24 25 2 3 19

_

X (Control Chart)

_

R (Control Chart)*

*Note: No lower control limit for R Chart for sample size below 7.

R

x
DAU PROGRAM MANAGERS TOOL KIT

54

PRODUCTION READINESS REVIEW (PRR)

“WHY WE DO THEM”

• Risk Management Tool: Identify program risks, issues & opportunities Early & Often (small,

incremental, proactive—vice big, single, reactive)

• Assess capability of contractor (and subcontractor) to deliver on time, within cost, a product

that meets performance & quality requirements

• Assess actual contractor performance (metrics)

• Assess effectiveness of contractor’s corrective/preventative actions

• Measure improvement of contractor’s performance

“HOW TO DO THEM”

• Write a charter that the program office and contractor both understand.

• Coordinate with the Defense Contract Management Agency (DCMA)—use their capability.

• Establish areas of assessment with metrics

– Producibility

– Engineering Change Orders (ECO) / design stability

– Manufacturing process control (key characteristics)

– Cost, time of scrap, rework, and repair

– Tooling status

– Subcontractor management (same metrics as listed above)

• Ask questions, touch things, talk to shop floor workers

– See what is actually happening on the factory floor rather than the conference

room, i.e., go see and talk to the people doing the work.

“WHEN TO DO THEM”

• Early and often (see Defense Acquisition Guidebook, 4.3.3.9.3, Production Readiness Reviews)

• Concurrently with other technical reviews, such as the System Functional Review (SFR),

Preliminary Design Review (PDR), and the Critical Design Review (CDR)

• In Systems Integration & Systems Demonstration

• “Final” PRR occurs at end of Systems Demonstration (before Milestone C).

• PRRs should be held in LRIP and beyond IF major changes (to design, manufacturing

processes, rates/quantities, etc.) occur during LRIP.

ADDITIONAL MANUFACTURING

INFORMATION SOURCES

• DAU’s Production, Quality and Manufacturing Information Web site:

– Go to www.dau.mil, click Knowledge Sharing, click Acquisition Community Connection,

then see Special Interest Areas, and click Production, Quality & Manufacturing

– Contains references to subjects including DoD Manufacturing Requirements, and Best

Business Practices, such as Lean Enterprise, e-Commerce, Six Sigma, Basic and

Advanced Quality Systems, Supply Chain Management, etc.

• Best Manufacturing Practices Center of Excellence—www.bmpcoe.org

• Lean Aerospace Initiative (LAI)—http://web.mit.edu/lean
DAU PROGRAM MANAGERS TOOL KIT

55

Use Combined DT/OT – single integrated Contractor/Gov’t DT and OT Team;

shared test events & test data; independent data analysis & reporting.

ACAT I & II Programs – require an independent, dedicated IOT&E to proceed

Beyond Low Rate Initial Production.

T&E Required before going Beyond Low Rate Initial Production

Production Qualification T&E – Verify Design Article meets Spec/PM responsible

Performed by Contractor &/or Government/DPRO assistance valuable. Readiness

for IOT&E.

• Live Fire T&E (LFT&E) – Vulnerability and Lethality/Dev’l Agency fund and execute.

DOTE oversight, approval and Congressional reporting for ACAT I, II and selected

programs.

• Initial Operational T&E (IOT&E) – Operational Effectiveness and Suitability/

Independent Svc OTA plan and manage. DOTE oversight, approval, and

Congressional reporting for ACAT I and selected systems.

Models and Simulations used throughout the Acq Process

 REQUIREMENTS DEVELOPMENT

 ENGINEERING DESIGN

 FABRICATION & TEST (BENCH, LAB)

 INTEGRATION & TEST

 (H/W IN THE LOOP)

 OT&E:

TEST & EVALUATION (T&E)—TYPES AND TASKS

Developmental T&E (DT&E)/Operational T&E (OT&E) Comparisons:

OT&E

• Operational effective/suitable

• Operational Test Agency (OTA) resp.

• “Typical” User Personnel

• Many test articles/each test

• “Combat” environment/Threats

• “Production Rep” Test Articles

• Contractor may not be allowed

DT&E

• Tech. perf. measurment

• Dev. agency rsp. (PM)

• Technical Personnel

• Ltd. test articles/each test

• Controlled environment

• All types of Test Articles

• Contractor involved

T&E Tasks & Events

• Test Rqmts

• Test Interfaces

• Eval Strategy

• Sys. Engineering

• Design for Test

• S/W Human T&E

• TEMP

• Subsystem T&E

• Software Only T&E

System DT&E

• CSCI T&E

• R. A. M.

• Supportability

• Interoperability

• Production Qual

• Live Fire T&E

• Cert of Readiness

 for IOT&E

AGONIZE OVER THRESHOLDS!

To Support T&E during:

 RQMTs Definition

 T&E Planning

 Engr Design

 Fabrication

 Integration

 Sys DT&E

 OT&E

 Training

 Ops

System OT&E

• Effectiveness

• Suitability

• Acceptance Test

• Manf. Test

• Data Collection

• Reporting

DEVELOPMENT T&E:

QUALIFICATION T&E,

ACCEPTANCE T&E, LFT&E

PRODUCTION (PAT&E, FOT&E)

 DEPLOY & OPERATIONAL SUPPORT

EOA, OA, IOT&E(OPEVAL)
DAU PROGRAM MANAGERS TOOL KIT

56

MODELING & SIMULATION (M&S) PLANNING

• ACCESS YOUR SERVICE CENTERS OF M&S EXPERTISE

• Establish a simulation coordinating group; the EARLIER the better

• Design long-term M&S applications and the Integrated Digital Environment

through the acquisition strategy, TEMP, STEP, SSPs

• CONTINUOUS PLANNING PROGRAM PLANNING

 TEMP

Integrated Digital

Environment (IDE) Planning

• Establish a business process

improvement team

• Identify High Payback Process

Areas

• Identify potential legacy systems

and data repositories

• Identify user base including

remote sites

• Capacity of PC workstations

• Bandwidth of communication lines

• Where servers are/will be located

• Identify legacy system host

platforms

Simulation-Based Acquisition

(SBA) Planning

• Establish a Simulation & Verification,

Validation, and Authentication (VV&A)

planning process

• Identify High Payback Process

Areas

• Identify potential legacy systems,

Service/Joint-standard simulations,

Architectures and data repositories

• Identify where user and simulators

are/will be located

• Capabilities & Architectures of

Existing Simulations

• Network Bandwidth Requirements

• IDE utilization opportunities

• Interoperability/Interface/Immersion

Requirements

• Required Capability Gap

• Design SBA Architectures

• Establish long-term plan, budget,

DOCUMENT & implement

• Manage it, update, evolve, reiterate

HOW DO WE PLAN?—A NOTIONAL APPROACH

Simulation Support Plan

(A,MC), Single Acquisition

Mgt Plan (AF), Acquisition

Strategy, TEMP

DSMC

Dec 98

M&S in
DAU PROGRAM MANAGERS TOOL KIT

57

HIERARCHY OF MODELS AND SIMULATIONS

 FUNCTIONS

RESOLUTION SUPPORTED MODELS & SIMULATIONS FORCE OR SYSTEM LEVEL

Increasing

Resolution

Actual

Performance

Increasing

Aggregation

Comparative

Results

The Hierarchical Aggregation of Models reduces Fidelity

significantly & must be evaluated. - (PM, DIS & DMSO)

Test &

Evaluation

Operational Req’ts. DevelopmentEffectiveness AnalysisTactics DevelopmentMission Planning & Rehearsal

Design

Manufacturing

Cost

Support

Development

Tech Req’ts Dev.

Joint/Combined

Forces

Air Wings Battle Groups

Air Wing

EW A/C

AircraftSystem

Fire Control

Radar

Corps

Air Vehicle

(System/Subsystem/Component)

(One-on-One)

(Many-on-Many)

Theater/

Campaign

Mission/Battle

Engagement

Engineering

(Groups of SystemsForce Package)
DAU PROGRAM MANAGERS TOOL KIT

58

THE EVOLUTION OF MODELING & SIMULATION

VIRTUAL PROTOTYPE

PHYSICAL MOCK-UP

FINAL PRODUCT

Distributed

Interactive

Simulation

(DIS)

Full

System

Prototypes

CAD

CAM

CAE

CADAM

Simulation-Based

Acquisition

The Advancing Computer & Communications Technology Revolutions

SimulationBased Design

(Virtual

Prototyping

with mfg design,

risk/cost analyses,

mat’l designation)

CONCEPTDEVELOPMENT

DEPLOYMENT& SUPPORT

MANUFACTURING

ENGINEERING DESIGN

& ANALYSIS

INTEGRATED

SYSTEM DATA

Oper ational 

Training

Oper ational Simulation

Par ametr ic Design Detailed Design

Manufacturin

g

Acquisition Planning

Advanced Surface Combatant DARPA

TEST &

EVALUATION
DAU PROGRAM MANAGERS TOOL KIT

59

TYPICAL TIMES FOR PROGRAM ACTIVITIES

Event

Time

(months)

6 – 9

 9 – 12

6 – 8

24 – 30

30 – 36

2 – 3

 9 – 12

2 – 3

 6 – 12

 3

 18 – 30

(ADDITIONAL TYPES OF PLANNING CHARTS ON NEXT 5 PAGES)

TYPES OF PLANNING CHARTS

MILESTONE CHART (Gantt)

PLANNING AND CONTROL TOOLS

Procurement Request Development Time

Contract Lead-time

DAB Lead-time

SI Design, Fab and Test

SD Design, Fab and Qual

Test Readiness Review Lead-time

DT&E

OT Readiness Review Lead-time

OT&E

OT Report Preparation

Production Lead-time

Time Period

 J F M A M J J A S O N D J F M A M

Design

Fab

Integrate

Gnd. Test

Qual

Flt. Test

Produce

• Advantages: Simple

• Disadvantages: Difficult to show dependencies between

 activities unless computer-constructed chart.

Activity:
DAU PROGRAM MANAGERS TOOL KIT

60

NETWORK CHART

= CRITICAL PATH

TIME

 5

 4

 7

 4

 5

21

17

 6

 7

COST

2,200

15,000

2,500

4,600

0

28,000

23,000

13,500

9,000

PLANNING AND CONTROL

(Continued)

3 5

12 6 7

TASK

Brief

Transport

Ship GFE

Ship system

Inspect

Train maint.

Train oper.

Integ. sys.

Dry Run

TASK #

1-2

2-3

2-6

2-4

4-6

3-6

3-5

6-7

7-8

4

8

ACCELERATE

COST TIME

 -

 500

 600

 750

 -

 800

 800

 -

 400

5

3

4

2

5

14

12

6

5

• Advantages: Shows dependencies; computes critical path

• Disadvantages: Complex; computerized support required to maintain

 Does not provide any chronology

17

21

5

4

6 7

4

7

5

• Advantages: Shows chronology and dependencies (“Swan-like”)

• Disadvantages: Complex; computerized support required to maintain

SWAN CHART

0 5 9 12 14 26 30 36 43 DAYS

• USE TASKS & TIMES FROM

 CHART ABOVE

 6-7 7-8

2-3

3-6

1-2 2-6

2-4

3-5

4-6
DAU PROGRAM MANAGERS TOOL KIT

61

34

6 7 8 9

10

11

SHIP 7GFE

 8INTEGRATE

DRY 7RUN

21

TEST

GOVT 20ANALYZE

GOVT 5REPORT

KTR 45ANALYZE

KTR 7REPORT

TRAIN 17OPS

TRANS 4TROOPS

TRAIN 21MAINT

SHIP 4SYS

5

INSPECT

Normal PERT display for manual method:

TRAIN

OPS

TRANS

TROOPS

7

SHIP GFE

TRAIN

MAINT

SHIP

SYS

4

4

5

21

INSPECT

8

INTEGRATE

7

DRY

RUN

21

TEST

GOVT

ANALYZE

GOVT

REPORT

KTR

ANALYZE

KTR

REPORT

30 5

45 7

PERT* NETWORK CHARTS

Most widely used PERT display using scheduling software:

2

*PERT = Program Evaluation & Review Techniques

5

12 1

5

BRIEF

5

BRIEF
DAU PROGRAM MANAGERS TOOL KIT

62

Project Work

Breakdown

Structure (WBS)

Task/Responsibility

Matrix

“ When ”

Networks

“ How Much ”

“ How ”

NETWORK SCHEDULE DEVELOPMENT

T&E

a

b

M

a

b

S

a

b

c

C

10

1112

20

21

22

30

31

3233

40

414243

50

51

52

30

20

10

Schedule

abcd

abcd

abcd

MNS

AOA,

ORD

Technical

Criteria

“ What ”

“ Who ”

Work

Packages

Manpower

& Material

Estimates

Project

Plans

Org.

WBS

10

20

30

40

11

12

21

22

31

32

41

T&E M S B

a b a b a ba

p s1

s1 p

p

s1

p s1

s2

p

C
DAU PROGRAM MANAGERS TOOL KIT

63

26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0

WORKING DAYS PRIOR TO COMPLETION (LEAD TIME)

(SUBCONTRACT

PART)

(PURCHASE PART)

(PURCHASE PART) (PURCHASE PART)

(SUBASSEMBLY “B”)

(SUBASSEMBLY

 “A”)

FINAL ASSEMBLY

(FABRICATED PART

IN-HOUSE)

TEST SHIP

6

8 7 4

1

2

3 5

12 11 10

9

LEAD TIME CHART
DAU PROGRAM MANAGERS TOOL KIT

64

80

70

60

50

40

30

20

10

0

80

70

60

50

40

30

20

10

0

O N DJ F MAMJ J 1 2 34 5 6 7 8 9 10 11 12

MONTHS CONTROL POINTS

80

70

60

50

40

30

20

10

0

Planned

Actual

LINE OF BALANCE TECHNIQUE

26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0

PRODUCTION LEAD TIME CHART 22 WORK DAYS/MONTH

6

1

4 7 8

2

3 5 9

10 11 12

Snapshot in time: 1 May
DAU PROGRAM MANAGERS TOOL KIT

65

ACQUISITION PROGRAM BASELINE

(Performance) - An Example

Baseline

 Values

Objective

Threshold

X

.85

.80

.75

.70

.65

(Prob)

Exit

Criteria

(.77)

System Integration System Demonstration Low-Rate Initial Prod Full-Rate Prod

Current

Estimate

Growth (Maturity) Curve

 MS B DRR MS C FRP DR

S

U

R

V

I

V

A

B

I

L

I

T

Y

Exit

Criteria

(.73)

System Development &

Demonstration Phase

Production &

Deployment Phase

* In this example the current estimate falls below the threshold—

this represents a baseline breach of performance.

*
DAU PROGRAM MANAGERS TOOL KIT

66

RISK MANAGEMENT PROCESS MODEL

List of

Top Risks

Risk

Assessment

Risk 

Monitoring/Reporting

Risk Event

Identification

Process Plans & Controls

Organization & Training

Revised BestHandlingOptions

Best Options

Revised Top

Risk List

Risk Mgt Plan(Implemented)

Process

Preparations

Risk

Monitorin

g

& Reporting

Risk Handling

Option Analysis

 ScheduleBudget & MIS

Risk

Integration

Risk 

Management

Planning/Prep

Risk Handling

Risk

Analysis

Risk Handling

Option

Identification

Risk and

Risk Handling

Option

Integration

List of Risk

Events

Risk

Descriptions

Status

Reports

List of HandlingOptionAlternatives 
DAU PROGRAM MANAGERS TOOL KIT

67

RISK MANAGEMENT

1. Develop program plans to the work package level.

2. Assess risk at the lowest work package/WBS level.

3. Manage the highest risk work packages; most others will work out.

TRADEOFF ANALYSIS*

1. Identify alternative solutions

2. Select evaluation criteria/factors & MOEs,

 i.e., cost, schedule, performance criteria

3. Weight evaluation criteria

4. Develop utility functions for each factor

5. Conduct evaluation (weighted utility summary table where

 weight is multiplied by utility function value)

6. Perform sensitivity check

7. Select highest scored alternative

RISK & TRADEOFFS ANALYSIS

Risk Planning

Risk Mgmt Plan

 (The Process) Risk ID

Technical

Cost

Schedule

Lessons learned

WBS

Risk Analysis

Networks

Simulation

Watch lists

Templates

Risk Handling

Avoidance

Control

Assumption

Transfer

Research

Risk Management

Risk Assessment

*With Cost As an Independent Variable (CAIV), aggressive cost objectives

are established as a result of trading performance and schedule for cost.

SOW DELIVERABLES WBS

OBS

WORK PKG

PLANNING PKG

GANTT/

NETWORK

CHART

COST

EST.

COST

VS

TIME

PLAN RISK MGT

PROCESS

ASSESS RISK

HANDLE RISK

Specs Contract
DAU PROGRAM MANAGERS TOOL KIT

68

TECHNICAL PERFORMANCE MEASUREMENT

THE CONCEPT

TECHNICAL

PARAMETER

VALUE

e.g.,

 MTBF

Weight

Fuel

Consumption

Tolerance Band

Threshold

Planned

 Value

Current

Estimate

TIME

Variation

Planned

 Profile

Achievement

 to date

Milestones

}

1. ARE ALL VIABLE ALTERNATIVES BEING

EXPLORED?

– Is each alternative clearly defined?

– Have the alternatives been

prescreened? How?

– Are affordability limits established?

– Can all of the screened-out

alternatives be defended?

2. ARE SELECTION CRITERIA IDENTIFIED?

– Are all significant criteria identified?

– Do the criteria discriminate among

alternatives?

– Are the criteria measurable?

– Have the criteria been preapproved?

3. IS THE CRITERIA WEIGHTING SYSTEM

ACCEPTABLE?

– Are rationales for criteria weights

explained?

– Are criteria weights consistent with

guidance?

– Are criteria weights consistently

distributed in the tree?

4. ARE UTILITY (SCORING) CRITERIA

DETERMINED?

– Is defensible rationale established

for each criterion?

– Are criteria developed from

operational measures of effectiveness where possible?

– Do all plans use the same numerical

scale?

– Is the location of the “zero point”

explained?

5. ARE EVALUATION METHODS

DOCUMENTED?

– Are test data reliability estimates

(confidence levels) incorporated?

– Are models validated? When? Who?

6. HAS SENSITIVITY BEEN ESTIMATED?

– Are error ranges carried through with

worst-on-worst case analysis?

– Have the effects of changes in the

utility curve shapes been examined?

– Have rationales for the limits been

developed?

PROGRAM MANAGER’S CHECKLIST FOR REVIEW

OF TRADEOFF PLANNING AND STUDIES
DAU PROGRAM MANAGERS TOOL KIT

69

WHAT IS SYSTEMS ENGINEERING?

• International Council on Systems Engineering (INCOSE) definition:

Systems Engineering is an interdisciplinary approach and means

to enable the realization of successful systems. It focuses on

defining customer needs and required functionality early in the

development cycle, documenting requirements, then proceeding

with design synthesis and system validation while considering

the complete problem...

Focus is on technical systems development and integrative

engineering to meet requirements.

-- www.incose.org

Integration

software/information

hardware/machines

humans/organizations

A SIMPLE MODEL OF A

SYSTEMS ENGINEERING PROCESS
DAU PROGRAM MANAGERS TOOL KIT

70

SYSTEMS ENGINEERING TASKS

• Ensure essential technical things get done

• Verify technical solutions to satisfy customer capability

requirements

• Develop a total system design solution

– Design in downstream life-cycle needs (open system approach)

– Balance cost, schedule, performance and risk

• Generate and track technical information needed for decision

making and configuration management

SYSTEMS ENGINEERING POLICY IN DoD

(Signed by the Honorable Mike Wynne, USD (AT&L) (Acting),

20 Feb 2004)

• All programs, regardless of ACAT shall:

— Apply an SE approach

— Develop a Systems Engineering Plan (SEP)

◆ Describe technical approach, including processes, resources,

and metrics

◆ Detail timing and conduct of SE technical reviews

• Director, Defense Systems (DS), USD (AT&L) tasked to provide

guidance for DoDI 5000.2

— Recommend changes in Defense SE

— Establish a senior-level SE forum

— Assess SEP and program readiness to proceed before each

DAB and other USD (AT&L)-led acquisition reviews
DAU PROGRAM MANAGERS TOOL KIT

71

SYSTEMS ENGINEERING PROCESS

“THE BASIC ENGINE”

PROCESS INPUT

Design

Verification Loop

Loop

* Sometimes referred to as "Logical" Analysis PROCESS OUTPUT

Requirements

(User Needs)

Analysis

Systems Analysis

and Control

(Balance)

Requirements

Loop

Functional Analysis/

Allocation*

Synthesis
DAU PROGRAM MANAGERS TOOL KIT

72

SYSTEMS ENGINEERING PROCESS

“BASIC ENGINE OPERATIONS”

X

F1

R1

F2 F3 FN

X

X

X X

X X

R2

R3

RN

• Customer Needs

• Tech Base

• Prior Systems Engineering Output

• Program Decision Requirements

• Budget

Requirements

(User Needs) 

Analysis

• Trade Studies

• Risk Mgt

• Config Mgt

• TPMs

• Tech Reviews

Do what? Functions

How well? Performance

Environment? Interfaces

ICD Req’ts Spec Req’ts

Implied Req’ts

Questions for Requirers

Systems Analysis & 

Control

Integration, Verification/Validation, T&E

Design Loop Requirements Loop

Synthesis

Functional 

Analysis

Implementation

Decision

Database

TIMELINE

HI / LO

Baselines

Functional

(System) Allocated

(Perf)

Product

(Detail) Specifications

System

Item Perf Item Detail

Process

Material

TO

CRUISE

CLIMB

LAND DESCEND

CLIMB

CRUISE

LOITER

ATTACK

(67 min., 50 km range)

FLY

TAKEOFF CRUISE LAND

(2 min.) (60 min.,

50 km range)

(5 min.)

AIRCRAFT

AIR ENGINE COMMUNICATIONS

FRAME

• Analyze Functions

• Decompose Function

• Allocate Requirements

F1

F2

F3

FN

X X

X X

X

X X

HW1 HW2 SW1 SWN
DAU PROGRAM MANAGERS TOOL KIT

73

GENERIC SE PROCESS—DETAILED APPROACH

Systems Engineering Logical (Functional) Analysis

Stakeholder

Requirements Definition

Design

Documentation

Requirements

Development

Hardware Fabrication Operator/Maintainer

Training

Software

Creation/Coding

Note: Parts of “V” process

in Dark Gray boxes match

the “Basic Engine” Model

Operational Expectations

User Expectations

Functional Expectations

Configuration Item

Expectations

Component

Expectations

Output/Transition Input

Requirements (User

Needs) Analysis

Component Engineering

(Implementation) of

System Elements

Development &

Definition

Realization &

Assessment

Test & Evaluation

Validation

Verification

Integration

Conformation

Audit

Design

Implementation

Design Solution
DAU PROGRAM MANAGERS TOOL KIT

74

CONCEPT REFINEMENT PHASE SYSTEMS ENGINEERING

ACTIVITIES

TECHNOLOGY DEVELOPMENT PHASE SYSTEMS

ENGINEERING ACTIVITIES

Demo & Validate Sys

Concepts & Technology Maturity

Versus Defined User Needs Trades

INPUTS OUTPUTS

• ICD & Draft CDD

• Preferred Sys Concept

• Exit Criteria

• T&E Strategy

• Support & Maintenance

Concepts & Technologies

• AoA • SEP • TDS

Develop System Perf (&

Constraints) Spec & Enabling/

Critical Tech Verification Plan

Demo/Model Integrated System

Versus Performance Spec

Develop Functional Definitions for

Enabling/Critical Technologies &

Associated Verification Plan

Interpret User Needs.

Analyze Operational Capabilities

& Environmental Constraints

Demo System Functionality Versus Plan

Demo Enabling/Critical Technology

Components Versus Plan

Develop System Concepts, i.e.,

Enabling/Critical Technologies, Update

Constraints & Cost/Risk Drivers

Trades

Decompose Functional Definitions into Critical

Component Definition & Tech Verification Plan

• Sys Performance Spec

• LFT&E Waiver Request

• TEMP • SEP • PESHE • PPP • TRA

• Validated System Support & Maintenance

 Objectives & Requirements

• Inputs to: –IBR –ISP –STA –CDD –Acq Strat

 –Affordability Assessment

 –Cost/Manpower Est. 

SRR

ICD

AoA Plan

Exit Criteria

Alternative Maintenance & 

Logistics Concepts

Prelim Sys Spec

T&E Strategy

SEP

Support & Maintenance Concepts & Tech

Inputs to: -Draft CDD -TDS -AoA -Cost/Manpower Est.

Decompose Concept Performance

 into Functional Definition 

& Verification Objectives

Analyze/Assess

System Concept Versus

Functional Capabilities

Analyze/Assess Concepts 

Versus Defined User Needs & 

Environmental Constraints

Assess/Analyze Concept &

Verify System 

Concept’s Performance

Develop Component Concepts, i.e., 

Enabling/Critical Technologies, 

Constraints & Cost/Risk Drivers

Analyze/Assess

Enabling/Critical Components

Versus Capabilities

Decompose Concept Functional

 Definition into Component Concepts 

& Assessment Objectives

Develop Concept 

Performance (& Constraints)

Definition & Verification Objectives

Interpret User Needs,

Analyze Operational Capabilities &

Environmental Constraints

INPUTS OUTPUTS

Trades

ITR ASR

Trades
DAU PROGRAM MANAGERS TOOL KIT

75

SYSTEM DEVELOPMENT AND DEMONSTRATION

PHASE SYSTEMS ENGINEERING ACTIVITIES

PRODUCTION AND DEPLOYMENT PHASE

SYSTEMS ENGINEERING ACTIVITIES

Trades

INPUTS OUTPUTS

Trades

• Sys Performance Spec

• Exit Criteria

• Validated Sys Support &

Maintenance Objectives &

Requirements

• APB • CDD • SEP • ISP • TEMP

Interpret User Needs, Refine

System Performance Specs &

Environmental Constraints

Develop System

Functional Specs &

System Verification Plan

Evolve Functional Performance

Specs into CI Functional (Design to)

Specs and CI Verification Plan

Evolve CI Functional Specs

into Product (Build to)

Documentation and Inspection Plan

• Initial Prod Baseline • Test Reports

• Elements of Product Support

• Risk Assessment • TEMP

• SEP • TRA • PESHE

• Inputs to: –CPD –STA –ISP

 –Cost/Manpower Est.

Fabricate, Assemble, Code to

“Build-to” Documentation

System DT&E, LFT&E & OAs,

Verify System Functionality &

Constraints Compliance to Specs

Individual CI

Verification DT&E

Integrated DT&E, LFT&E &

EOAs Verify Performance

Compliance to Specs

Combined DT&E/OT&E/LFT&E

Demonstrate System to Specified User

Needs & Environmental Constraints

FCA SVR PRR

CDR

TRR

SFR

SRR

PDR

FTA

RCM

LORA

MTA

FMECA

OUTPUTS

J-6 Interoperability

& Supportability Validation

Full-Up System Level LFT&E

Independent IOT&E

LFPT

Report to

Congress

INPUTS

JITC Interoperability

Certification Testing

OTRR

BLRIP

Report to

Congress

Analyze Deficiencies To

Determine Corrective Actions

• Test Results

• Exit Criteria

• APB • CPD • SEP •TEMP

• Product Support Package

Verify & Validate

Production Configuration

• Production Baseline

• Test Reports

• TEMP • PESHE • SEP

• Input to: –Cost/Manpower Est.

PCA

Modify Configuration (Hardware/Software/

Specs) To Correct Deficiencies
DAU PROGRAM MANAGERS TOOL KIT

76

OPERATIONS AND SUPPORT PHASE

SYSTEMS ENGINEERING ACTIVITIES

INPUTS OUTPUTS

Data for In-Service Review:

• Input to CDD for Next Increment

• Modifications/Upgrades to Fielded

Systems 

• SEP

• Process Change—

Hardware/Support

• Materiel Change

Trades

• Service Use Data

• User Feedback

• Failure Reports

• Discrepancy Reports

• SEP

Monitor and Collect

All Service

Use Data

Analyze Data to

Determine

Root Cause

Determine

System Risk/

Hazard Severity

Develop

Corrective

Action

Implement and

Field

Assess Risk of

Improved System

Integrate & Test

Corrective Action

Trades

In-Service

Review

•
DAU PROGRAM MANAGERS TOOL KIT

77

SystemPerformance

TechnicalEffectiveness

SystemEffectiveness

SystemAvailability

ProcessEfficiency

Capabilities

Functions

Priorities

Reliability

Maintainability

Supportability

Producibility

Operations

Maintenance

Logistics

Life Cycle Cost / Total Ownership Cost

PROCESS EFFICIENCY

Operations: Interoperability, Information Assurance, 

Survivability & Susceptibility, HIS, System Security, Anti-tamper 

Maintenance: Corrosion Prevention and Control, Accessibility, 

Interoperability, Unique Identification of Items 

Logistics: Supportability Producibility: Value Engineering, Quality, Manufacturing Capability 

SYSTEM AVAILABILITY

Reliability / Maintainability / Supportability:

RAM, COTSProducibility: Value Engineering, Quality,

Manufacturing Capability

Affordable

Operational

Effectiveness

SYSTEM PERFORMANCE

Architectural

Impacts on System

Open Systems Design Interoperability 

Design Considerations

Survivability &  Susceptibility Software Anti-tamper HSI Accessibility Insensitive Munitions System Security Information Assurance Corrosion Prevention COTS Disposal and  Demilitarization Environment, Safety,  Occupational Health 

Design Tools 

Modeling & Simulation 

IMPORTANT SYSTEMS ENGINEERING DESIGN CONSIDERATIONS

“The Fishbone”
DAU PROGRAM MANAGERS TOOL KIT

78

REQUIREMENTS (USER NEEDS)

ANALYSIS QUESTIONS

• What are the reasons behind the system development?

• What are the customer expectations? How will they measure

the performance of the system?

• Who are the users and how do they intend to use the product?

• What do the users expect of the product?

• What are their levels of expertise?

• With which environmental characteristics must the system

comply?

• What are existing and planned interfaces?

• What functions will the system perform, expressed in customer language?

• What are the constraints – hardware, software, economic,

procedural – with which the system must comply?

• What will be the final form of the product – model, prototype,

mass production?

• Specific, Clear and Unambiguous: Contains no vague

terms

• Understandable: Stated with sufficient detail in everyday

 language

• Concise: Contains no unnecessary words

• Consistent: Top-to-bottom consistency with identical usage

 of terms and conformance to standards

• Stable: Baselined and under configuration control

• Traceable: Derived from the mission profile or the contractor’s

 design policies

• Verifiable: Determine whether the product is satisfying the

 requirement

• Feasible: Can achieve, produce, and maintain the requirement

ATTRIBUTES OF A WELL-DEFINED REQUIREMENT
DAU PROGRAM MANAGERS TOOL KIT

79

HOW DO I MAKE DECISIONS ?WILL IT DO JOB/WORTH THE $$ ?ARE WE DOING THE RIGHT THING ?HOW DO I KNOW IT WORKS?WILL IT MEET THE PERFORMANCE CRITERIA?WILL IT ALL WORK TOGETHER ?DO WE KNOW WHAT WE HAVE ?ARE WE READY TO GO ON ?HOW DO I RUN THIS PROGRAM ?

SYSTEM ANALYSIS AND CONTROL

(MGMT TOOLS)

H E L P !

TRADE STUDIESEFFECTIVENESS ANALYSISRISK MGMTTECHNICAL PERF MEASURESMODELING & SIMULATIONTECH PERF MEASCONFIG MGMTTECHNICAL REVIEWSINTEGRATED PLANNING
DAU PROGRAM MANAGERS TOOL KIT

80

SPECIFICATION DEVELOPMENT & TECHNICAL REVIEWS

Pre-Systems Acquisition Systems Acquisition Sustainment

System Development & 

Demonstration

Operations & 

Support

Concept 

Refinement

Technology 

Development

System 

Integration LRIP FRP & Deployment

IOC FOC

Sustainment

Disposal

Specifications System Specification

Item Performance Specifications (Design to)

ASR SRR SFR

S/W Reviews

PDR CDR TRR

Item Detailed Specification (Build to)

Configuration Baselines

Functional Baseline

Allocated Baseline

Product Baseline

PRR SVR

• Increased emphasis on evolutionary acquisition and spiral development• Technology development strategy: Approved by Milestone Decision Authority (MDA) at Milestone A• Systems Engineering Plan: Integrated with acquisition strategy and submitted for MDA approval at each milestone

review, regardless of Acquisition Category (ACAT)

Technical Reviews and Audits

A

B

C

SE Process

Configuration Control

Product Upgrades

Planning for Disposal

(Repeated in Each Phase)

DRR

PCA

FRP

Initial Capabilities Document 

(ICD)

Capability Development 

Document (CDD)

Capability Production 

Document (CPD)

Production & Deployment

System 

Demonstration

Pre-Systems Acquisition Systems Acquisition Sustainment

System Development & 

Demonstration

Operations & 

Support

Concept 

Refinement

Technology 

Development

System 

Integration LRIP FRP & Deployment

IOC FOC

Sustainment

Disposal

Specifications System Specification

Item Performance Specifications (Design to)

ASR SRR SFR

S/W Reviews

PDR CDR TRR

Item Detailed Specification (Build to)

Configuration Baselines

Functional Baseline

Allocated Baseline

Product Baseline

PRR SVR

• Increased emphasis on evolutionary acquisition and spiral development• Technology development strategy: Approved by Milestone Decision Authority (MDA) at Milestone A• Systems Engineering Plan: Integrated with acquisition strategy and submitted for MDA approval at each milestone

review, regardless of Acquisition Category (ACAT)

Technical Reviews and Audits

A

B

C

SE Process

Configuration Control

Product Upgrades

Planning for Disposal

(Repeated in Each Phase)

DRR

PCA

FRP

Initial Capabilities Document 

(ICD)

Capability Development 

Document (CDD)

Capability Production 

Document (CPD)

Production & Deployment

System 

Demonstration
DAU PROGRAM MANAGERS TOOL KIT

81

TECHNICAL REVIEW DEFINITIONS

ASR (Alternative Systems Review) – Assess that preferred system

solution meets needs.

SRR (System Requirements Review) – Ensure system performance

capabilities are consistent with technology solution and traceable to

ICD and draft CDD.

SFR (System Functional Review) – Ensure functional performance

capabilities are consistent with cost, schedule, and risk constraints

and system ready to proceed into preliminary design.

PDR (Preliminary Design Review) – Ensure system preliminary design and functional/allocated baseline captured in item performance

specifications for each configuration item in system.

CDR (Critical Design Review) – Determine that system meets stated

performance capabilities, product baseline captured in item detail

specification, and system ready for fabrication, demonstration, and

test.

PRR (Production Readiness Review) – Verify design is ready for production and adequate production planning accomplished; system

capabilities traced to final production system.

TRR (Test Readiness Review(s)) – Assess test readiness and approve

test plans.

SVR (System Verification Review) (synonymous with Functional

Configuration Audit) – Verify configuration items perform to specification and system ready to proceed into Low-Rate Initial Production and Full-Rate Production within cost, schedule and risk constraints.

PCA (Physical Configuration Audit) – Verify item produced (product

baseline) matches design documentation (e.g., drawings, etc.) and

item detail specification in the contract.
DAU PROGRAM MANAGERS TOOL KIT

82

TECHNICAL REVIEW BEST PRACTICES

Easy in principle, Easy in principle,

difficult in practice

Technical reviews:

• Are a fundamental part of the SE process and serve as a technical

assessment product for the program manager

– Should be event-based

– Objective entry criteria need to be defined up front

– Are only as good as who conducts them

– Engagement of Technical Authority

– Chair independent of program team

– Independent subject matter experts, determined by Chair

– Involve ALL STAKEHOLDERS

• Should review entire program from a technical perspective

– Cost, schedule, and performance

– By ALL STAKEHOLDERS

– Involve all technical products (specs, baselines, risks, cost

estimates)

• Should result in program decisions and changes

– Rather than a “check in the box”

• Taken as a whole series, form a major part (backbone) of the SEP
DAU PROGRAM MANAGERS TOOL KIT

83

SPECIFICATIONS AND STANDARDS

A New Way of Doing Business (Acquisition Reform)

(Sec Def Memo of 29 June 1994)

1. Use Performance-Based Specifications

2. Cancel/Convert Manufacturing and Management Standards to

Performance or Nongovernment Standards (NGSs)

3. Encourage Contractors to Submit Alternative Solutions to Military

Standards/Specifications

4. Prohibit Use of Military Specifications/Standards Except when

Authorized by Service Acquisition Executive or Designee

Design/Fab Require desired Specify exact parts and

outcomes or functions, components

Specific design to

contractor

Processes Few, if any Specify exact processes

Physical Give specifics only for Specify more physical

Characteristics interfaces, environment, characteristics, than

or human factors needed for interfaces,

environment, etc.

Interface Reqmts Detailed interface data do Detailed interface data

NOT solely make a perf.

spec a detail spec.

Materials Leave specifics to Require specific materials

contractor

Test & Eval State performance Prescribed testing

need; contractor picks process

test procedure

 PERFORMANCE DETAIL / DESIGN

PERFORMANCE VS. DETAIL SPECIFICATIONS
DAU PROGRAM MANAGERS TOOL KIT

84

• Program Unique Specifications advantages:

– Helps avoid duplication and inconsistencies.

– Enables good estimates of necessary work and resources.

– Provides consistent communication among players as people

rotate.

– Can be used to prepare test plans.

– Can be used a long time after the system has been put into

operation.

– Serves as an interface between customers, developers and

designers.

• Can act as negotiation and reference document for engineering

changes.

MIL-STD-961

Standard Performance Specification

Standard Design Specification

Program-Unique Specifications

No waiver required to use.

STD PRACTICE

PROGRAM-UNIQUE SPECIFICATIONS

• Defines mission/technical performance

 requirements. Allocates Requirements to

 functional areas. Defines interfaces.

• Defines performance characteristics of CI’s

 (form, fit, function). Details Design Require-

 ments only to meet interfaces. “DESIGN TO.”

• Includes “how to” and specific design

 requirements. Usually includes specific

 processes and procedures. “BUILD TO.”

• Defines process performed during fabrication.

• Defines production of raw materials or

 semi-fabricated material used in fabrication.

System

(HW or SW)

Item

Performance

(HW or SW)

Item

Detail

Process

Material

Functional

(“System”)

Allocated

(“Design to”)

Product

(“Build to”)

Product

Product

Specification Content Baseline

Specs
DAU PROGRAM MANAGERS TOOL KIT

85

CONFIGURATION MANAGEMENT PLANNING

• The decisions on

– Which baselines the government should eventually control

– The data needed

– When that control should be established

 ... are strategic management decisions that involve

– Acquisition strategies—sources, competition, etc.

– Logistics support plans—repair levels, data needs, open

systems, etc.

– Technology insertion—stable vs. rapidly moving technologies, etc.

• Government should control the Functional Baseline (document system

level requirements)

• DoD PMOs increasingly choose to leave Allocated Baselines under

contractor control until late in development. (Documents the

Configuration Item (CI) level design requirements.)

– Promotes contractor design flexibility

– Relieves PMO from administrative burdens of managing design

Engineering Change Proposals

– Requires effective implementation of Integrated Product and

Process Development

• When and if to control baselines is dependent on support philosophy

and acquisition management strategy.

“A management process for establishing and maintaining consistency

of a product’s performance, functional and physical attributes with its

requirements, design and operational information throughout its life.”

• Identify and document the functional and physical characteristics of

configuration items.

• Control changes to configuration items and their related documentation.

• Record (or Status Accounting in DoD terms) and report information

needed to manage configuration items effectively, including the status of

proposed changes and implementation status of approved changes.

• Audit configuration items to verify conformance to specifications,

drawings, interface control documents, and other contract requirements.

CONFIGURATION MANAGEMENT

Non-Government Std: EIA Standard-649

Also see MIL HNBK 61

—Adopted by DoD, 22 Nov 96
DAU PROGRAM MANAGERS TOOL KIT

86

• The Government PMO:

– Identifies External Interfaces

– Establishes interface standards

(baselines)

– Maintains interface stability

• The Contractor:

– Manages Internal Interfaces

– Establishes Interface

◆ Requirements to Include

◆ Internal & External Interfaces

– Controls Interfaces to Ensure

◆ Accountability & Timely

◆ Dissemination of Changes

The Government increasingly chooses to manage interfaces

leaving design details to contractors.

• Identifies, documents, & controls all functional and physical

characteristics

• Interfaces:

– What?

◆ Common boundary

◆ Types: Mechanical, Electrical, Operational, Software

◆ Functional & Physical characteristics

– Where?

◆ Within one contractor’s design

◆ Among contractor’s items and GFE

◆ Among multiple contractors’ items

◆ Among systems

– Controlled by Interface Control Working Group

– Documented in Interface Control Documents

?

INTERFACE MANAGEMENT

Will it all work together?

INTERFACE CONTROL CONCEPT
DAU PROGRAM MANAGERS TOOL KIT

87

BASIC PURPOSES OF WBS

HOW TO CREATE A

WORK BREAKDOWN STRUCTURE

Sys Eng

System

Training Test Mfg

computer sensor

Crew

Simulator

HW

System XXX

SW

Application

SW

displays

Grouping for

Spec.

Development

Interface

Earned Value Management

Evaluation

Risk

Assessment

Tech

Review

Structure

$$$

Management

Product

Tree

ECP

Impact

IPT

Setup

CATT Support

Appendix

B

C

MIL HDBK System

Define

the Product

Tailor

(Supporting Processes)

Create and Refine

as Design Matures

SE Process
DAU PROGRAM MANAGERS TOOL KIT

88

SOFTWARE MANAGEMENT

• Nine Principal Best Practices to Improve Software Development,

Reduce Costs, and Increase User Satisfaction*

– Formal Risk Management

– Agreement Interfaces

– Peer Reviews/Inspections/Walk-throughs

– Metric-Based Scheduling and Management

– Binary Quality Gate, at Inch-Pebble Level

– Program-wide Visibility of Project Progress vs. Plan

– Defect Tracking Against Quality Targets

– Configuration Management

– People-Aware Management Accountability

• Nine Project “Breathalyzer ” Questions to provide “Quick Look” at

Software Project Health**

– Do you have a current, credible activity network supported by a

work breakdown structure (WBS)?

– Do you have a current, credible schedule and budget?

– Do you know what software you are responsible for delivering?

– Can you list the current top 10 project risks?

– Do you know your schedule compression percentage?

– What is the estimated size of your software deliverable? How

was it derived?

– Do you know the percentage of external interfaces that are not

under your control?

– Does your staff have sufficient expertise in the project domains?

– Have you identified adequate staff to allocate to the scheduled

tasks at the right time?

 *“Little Yellow Book of Software Management Questions” (Software Program Managers Network)

**“Project Breathalizer Questionnaire Software Health”; Software Program Managers Council
DAU PROGRAM MANAGERS TOOL KIT

89

II

LEADERSHIP & MANAGERIAL SKILLS

• More things that make you go “Hmmm?... ”

“An authority is a person who just happens to know the source.”

“A conservative is a person who believes nothing should be done

the first time.”

“Diplomacy is the art of hearing all parties arguing in a dispute and

nodding to all of them without ever agreeing with any of them.”

“The meeting raised our confidence that the contractor can actually

accomplish the task and that it will occur in our lifetime.”

“This is the earliest I've been late.”

“The world would be a much better place if people weren't allowed

to have children until they've proven they can successfully

manage a DoD program.”

“Everyone is bound to bear patiently the results of his/her own

example.”

“The superior person is firm in the right way, and not merely firm.”
DAU PROGRAM MANAGERS TOOL KIT

90

TYPICAL WORKING GROUPS

• Logistics Support Management Team (LSMT)

• Test & Evaluation Working Group (TEWG)

• Computer Resources Working Group (CRWG)

• Requirements Interface Working Group

• Interface Control Working Group (ICWG)

• Technology Assessment Working Group

• "Tiger" Team

• Process Action Team

• Integrated Product & Process Teams

RECOGNIZE WHICH PHASE OF

TEAM DEVELOPMENT YOU ARE IN

AND TAKE POSITIVE ACTION TO

WORK THROUGH

WORKING GROUPS

TEAM DEVELOPMENT WHEEL

Performing

Creative

Trusting

Effective

Confident

Forming

Milling

Confusion

Polite

Purposeless

Norming

Cohesion

Purpose

Feedback

Relevancy

Storming

Conflict

Frustration

Resistance

Cliques

Note: There can be an additional phase – “Adjourning” – when the team

disbands, says good bye, and reflects on lessons learned. This is a

“celebration” phase.
DAU PROGRAM MANAGERS TOOL KIT

91

 Disadvantages

• Takes more time

• Hard to terminate

• Paralysis by analysis

WORKING GROUPS

(Continued)

Group Consensus - a general agreement by all team members that they

can live with, and be committed to a particular course of action. Until you

have this agreement, you don't have consensus. Guidelines for achieving:

1. Avoid arguing for your own opinion.

2. Go for "win-win" solutions.

3. Do not change your mind to avoid conflict.

4. Avoid majority vote, coin-flipping, horse-trading.

5. Expect differences of opinion.

MANAGEMENT TRADEOFFS

FOR WORKING GROUPS

 Advantages

• More ideas & solutions

• Consensus positions

• Strong commitments

TEAM PERFORMANCE MODEL

• Decision Making

• Resolving Issues

• Communicating

• Planning

• Executing

• Controlling

• Customer Focus

• Leadership

• Values

• Vision

• Purpose

• Goals & Objectives

• Critical Success Factors

• Diversity

• Conflict

• Comfort Zones

• Communications

• Focus

• Organizational Climate

• Trends

• Awareness

• Roles & Responsibilities

• Operating Agreements

• Team Accountability

• Empowerment

• Trust

• Five Cs

• Team Identity

• Self-assessment

Communication

Commitment

Cooperation

Contribution

Caring

Team Processes

Team Principles

Team Dynamics

Team Foundation

Thinking

Learning

Charter
DAU PROGRAM MANAGERS TOOL KIT

92

DELEGATION

REASONS FOR DELEGATING

 1. Improve manager's time management

a. Increase manager's span of control

b. Increase time allocated to long-range planning

c. Increased management efficiency

2. Assure tasks performed by most qualified

3. Build organizational depth

4. Improve employee motivation

5. Increased teamwork (IPTs/TQM)

6. Maximize resources

7. Appropriate organizational responsibility

12 STEPS FOR DELEGATING

1. Set clear objectives and task statements

2. Select "Delegate"; check qualifications

3. Provide training, if necessary

4. Solicit input from Delegate

5. Assign task and deadline

6. Provide any relevant guidance

a. Critical information required to do tasks right

b. Potential approaches—only as suggestions!

c. Describe results desired

7. Makes a delegation "contract" (see next page)

8. Establish controls

9. Maintain controls

10. Provide feedback

11. Identify lessons learned

12. Evaluate performance
DAU PROGRAM MANAGERS TOOL KIT

93

DELEGATION

(Continued)

DELEGATION RECORD

Description of Action: Date:

Person Assigned:

Authority Level (specify):

1 – Take action; do not report

back

2 – Take action; report back

(see Frequency)

3 – Prepare plan; proceed

upon approval

4 – Do only as directed below

Frequency of Contact (specify):

1 – daily

2 – weekly

3 – monthly

4 – other

Suspense Action:

Performance Assessment:

Suspense Date:

Delegation Guidance/Agreements:

(SAMPLE)
DAU PROGRAM MANAGERS TOOL KIT

94

EFFECTIVE MEETINGS

PRE-MEETING

A. Establish type of meeting

1. Information (quick, crisp)

2. Planning/Strategizing (slow, deliberate)

3. Problem solving (divergent/convergent)

4. Decision (deliberate)

5. Staff/Conference (repetitive, short)

6. Feedback/Evaluation (slow, contemplative)

7. Training (smooth, flowing)

8. Social (rambling)

B. Select participants

1. Based on purpose; relevant; decision auth.

2. Size: 4-7 ideal; 10-12 tolerable; >13 unsat.

C. Circulate agenda (3-5 days in advance)

1. Type, purpose, date, place, start/finish times

2. Topics, time allocated (minutes), speakers

3. Assign recorder

CONDUCTING MEETING

A. Opening

1. Start on time

2. Repeat type and purpose of meeting

B. During

1. Facilitate the meeting

2. Encourage openness and communication

3. Develop cohesion

4. Use active listening

5. Stick to agenda

C. Closing

1. Set time and date of next meeting

2. Summarize agreements, actions, decisions

3. Close on time or before

AFTER MEETING

A. Review minutes with recorder

B. Publish minutes
DAU PROGRAM MANAGERS TOOL KIT

95

DIRECTIVE

• Give advice

• Evaluate

• Motivate

• Explain

• Reassure

NONDIRECTIVE

• Don't display authority

• Listen carefully

• Don't advise

• Facts only; no opinions

• Employee find solution

Advantages

• Effective with inexperienced

 personnel

• Quick

• Take charge attitude

Advantages

• Develops commitment

• Good training

• Employee responsible

• Supports delegation

Disadvantages

• Perceived insulting

• Does not support delegation

• Manager keeps responsibility

Disadvantages

• Takes time

• Skill/patience required

• Ineffective with

 inexperienced personnel

COUNSELING PROCESS

1. Set up interview - private, confidential,

unhurried

2. Encourage discussion - open questions,

active listening

3. Help employee think it through - deal with

facts, no opinions or own views

4. Let them find the solution - their solution to

their problem

COUNSELING

PERSONAL COMMUNICATIONS
DAU PROGRAM MANAGERS TOOL KIT

96

PERSONAL COMMUNICATIONS

(Continued)

WIN-WIN NEGOTIATIONS

FOCUS: Defeat the problem; not the person

APPROACH:

Resolve conflict

Reach agreement

Normalize relationships

Combine efforts

GOAL:

Acceptable gains by both parties

INTERPERSONAL NEGOTIATIONS

1. Separate people and emotions from the problem

2. Focus on interests, not positions

3. Generate options for mutual gain

4. Insist on objective criteria

– Screen for importance

– Limit to 2 minutes

– Do all at one time

– Itemize topics before calling

– Don't socialize

– Screen for importance

– Do not invite into office

– Remain standing

– Re-delegate

– Stay focused on subject

 area and on schedule

1. Incoming telephone calls

2. Outgoing telephone calls

3. Unscheduled visitors

4. Improper delegation

5. Poorly conducted meetings

TIME MANAGEMENT

TIME ROBBERS AND AVOIDANCE TECHNIQUES
DAU PROGRAM MANAGERS TOOL KIT

97

TIME MANAGEMENT

(Continued)

1. List all tasks.

2. Categorize tasks using matrix.

3. Review quadrant 3 items; reassign as 1, 2, or 4 as appropriate.

4. Do quadrant 1 tasks first; consider delegating!

5. Strive to maximize time for quadrant 2 tasks (be proactive!).

6. When all 1 and 2 tasks are complete, do quadrant 4 tasks.

KEEP A "TO DO" LIST

1. List all goals and tasks.

2. Categorize as:

A – High value

B – Medium value

C – Low value

3. Prioritize within each category (e.g., A-1, A-2, etc.).

4. Accomplish all A tasks, then all B. Do C if time permits.

5. Review list and priorities daily.

HIGH IMPORTANCE LOW

HIGH

URGENCY

LOW

DO IT

NOW

VERIFY

&

REASSIGN

MID- TO

LONG-RANGE

PLANNING

BUSY

WORK

1

2

3

4
DAU PROGRAM MANAGERS TOOL KIT

98

DECISION BRIEFING

Elements of a Decision Briefing

• Purpose – Issues

• Outline – Agenda

• Background

• Assumptions

• Alternatives Identified

• Evaluation Criteria

• Analysis of Alternatives

• Recommendation

• Implementation Plan

Things to Expect (from Briefee)

• Challenges to assumptions, definitions, methodology

• Does it comply with or change policy?

• Is the situation sensitive to change?

• Issues with analysis, tradeoffs, recommendations, implementation

• Open/closed questions
DAU PROGRAM MANAGERS TOOL KIT

99

MANAGEMENT TOOLS & TECHNIQUES

1. Activity-based Management (ABM). Uses detailed economic

analyses of important business activities to improve strategic and

operational decisions. ABM increases the accuracy of cost information by more precisely linking overhead and other indirect costs to

products or customer segments. Traditional accounting systems

distribute indirect costs using bases such as direct labor hours,

machine hours, or material dollars. ABM tracks overhead and other

indirect costs by activity, which can then be traced to products or

customers.

2. Balanced Scorecard. Defines what management means by

“performance” and measures whether management is achieving

desired results. The Balanced Scorecard translates mission and

vision statements into a comprehensive set of objectives and

performance measures that can be quantified and appraised. These

measures typically include: financial, customer value, internal

business process, learning and growth, and employee performance.

3. Cycle Time Reduction. Decreases the time it takes a company or

program to perform key activities throughout its value chain. Cycle

Time Reduction uses analytic techniques to minimize waiting time,

eliminate activities that do not add value, increase parallel processes, and speed up decision processes within an organization.

Time-based strategies often emphasize flexible manufacturing, rapid

response and innovation in order to attract the most profitable

customers.

4. Groupware. Refers to a broad range of technologies that allow

people in organizations to work together through computer networks.

These products range from sophisticated electronic mail packages

to applications that link offices and employees. Organizations use

such technology-aided communications to better inform strategic

and financial decisions and to more effectively and economically

bring together working groups. (DAU has a Groupware capability in

its Management Decision Center, which is used for management

decision making by offices and agencies throughout DoD.)
DAU PROGRAM MANAGERS TOOL KIT

100

 MANAGEMENT TOOLS & TECHNIQUES

(Continued)

5. Outsourcing. Occurs when a company or Government agency uses

third-parties to perform non-core business activities. Contracting

third-parties enables a company or agency to focus its efforts on its

core competencies. Many companies find that outsourcing reduces

cost and improves performance of the activity. Third-parties that

specialize in an activity are likely to be lower cost and more effective,

given their scale. Through outsourcing, a company or agency can

access the state of the art in all of its business activities without

having to master each one internally.

6. Business Process Reengineering. Involves the fundamental

redesign of core business processes to achieve significant improvements in productivity, cycle times, and quality. In Business Process

Reengineering, companies start with a blank sheet of paper and

rethink existing processes to deliver more value to the customer.

They typically adopt a new value system that places increased

emphasis on customer needs. Companies and/or Government

agencies reduce organizational layers and eliminate unproductive

activities in two key areas. First, they redesign functional organizations into cross-functional teams. Second, they use technology to

improve data dissemination and decision making.

7. Strategic Planning. Is a comprehensive process for determining

what a commercial business or Government agency should become

and how it can best achieve that goal. It appraises the full potential

of a business and explicitly links the business objectives to the

actions and resources required to achieve them. Strategic Planning

offers a systematic process to ask and answer the most critical

questions confronting a management team – especially large,

irrevocable resource commitment questions.
DAU PROGRAM MANAGERS TOOL KIT

101

III

PROBLEM SOLVING TOOLS

BRAINSTORMING

PURPOSE: To stimulate the free flow of ideas.

METHOD: Group members take turns generating ideas. One idea

stimulates another and then another. Freewheeling of ideas is encouraged. Brainstorming stops when all group members run out of

ideas. The next page lists questions that may suggest new ideas for

you.

GROUND RULES:

Put prejudgment aside. Remember, all ideas can be thought of as

starters.

No criticism allowed. This is not the time to prejudge an idea. Don't

criticize other ideas no matter how ridiculous they may seem. The

ideas can be discussed in detail later; at this time, the objective is

to generate more ideas.

Welcome free-wheeling or blue-skying. Let those wild ideas come

out—otherwise you may conceal your creative process. The impractical ideas may trigger other ideas that are possible to use.

Strive for quantity, not quality. The more ideas brought out, the

better the chance of a great solution.

Combine and rearrange ideas. Single ideas aren't the only way to

make a suggestion. You can make additions or combinations of

previously suggested ideas to create still better ideas.

Record all ideas exactly as expressed. This keeps the mind free of

remembering what was said and allows you to build on previous

ideas.
DAU PROGRAM MANAGERS TOOL KIT

102

BRAINSTORMING

(Continued)

Why does it work?

Some of the reasons why brainstorming enhances a group's creativity

are that it:

• Increases involvement and participation.

• Produces the most ideas in the shortest time.

• Reduces the need to give the "right" answer.

• Frees up the group; allows the members to have fun and is

 interesting.

• Reduces the possibility of negative thinking.

QUESTIONS TO STIMULATE YOUR BRAIN CELLS:

1. Can we use this idea elsewhere? As is? With changes?

2. If we change it, is there anything else like it? Any related issues?

3. Modify? Change? Rearrange? Meaning, color, motion, sound, odor,

taste, form, shape, layout, etc.?

4. Magnify? Add what? More, stronger, larger, newer?

5. Minimize? Subtract what? Eliminate, smaller, lighter, slower, split?

6. Substitute? Who, what, when, where?

7. Reverse? Opposite, backwards, upside down, inside out?
DAU PROGRAM MANAGERS TOOL KIT

103

FORCE FIELD ANALYSIS

A Force Field Analysis illustrates the relationship and significance of

factors that may influence the problem or goal. This analysis helps us

better understand driving and restraining forces.

Used for making decisions, FORCE FIELD ANALYSIS can help:

• Identify realistic improvement opportunities;

• Develop systematic action plan for problem resolution; and

• Create criteria for evaluating effectiveness of improvement actions.

CAUSE-AND-EFFECT DIAGRAM

The cause and effect diagram is a graphical illustration of the relationship

between a problem or goal (the effect) and its potential contributors (the

causes). Sometimes called the “fishbone” or Ishikawa diagram.

Used for analyzing problems, a CAUSE-AND-EFFECT DIAGRAM can

help:

• Determine root causes of a given effect, and

• Identify areas where there is a lack of data.

GOAL: Stop Smoking

+ Driving Forces

Poor Health

Burned Clothing

Poor Example

Cost

Impact on Others

Restraining Forces –

Habit

Addiction

Taste

Advertisements

Stubborn

ENVIRONMENT PEOPLE MACHINE

POOR

PHOTOCOPY

QUALITY

METHODS MATERIALS

Original

Settings

Pencil

Hardness

Drying

Time

Bad Paper

Handling

Poor

Maintenance

Bad Settings

Contamination

Paper

Alignment

Hands

Dirty

Bad

Originals Humidity

Operating

Hours

Sharpness

Lamp Dirty

Speed

Roll Condition

Not

Cleaning-up

Not Level

Poor

Quality

Poor

Storage

Copy Paper

Writing

Pressure
DAU PROGRAM MANAGERS TOOL KIT

104

SCATTER DIAGRAM

A scatter diagram depicts the correlation between two variables (X and Y).

Used for data analysis, a SCATTER DIAGRAM can help:

• Confirm a hypothesis that two variables are related.

• Provide both visual & statistical means to test the strength of a

potential relationship.

HISTOGRAM

The histogram chart displays the distribution of a measurable characteristic (for example: weight, length, speed, etc.). A histogram shows what

the variability of the data is in a graphical or pictorial manner.

Used for data analysis, a HISTOGRAM can help:

• Present a picture of how the process is operating;

• Compare actual process measurements with an expected

distribution;

• Observe patterns in the data; and

• Investigate process stability.

Y

X

FREQUENCY

MEASURABLE CHARACTERISTICS
DAU PROGRAM MANAGERS TOOL KIT

105

SURVEYS

Surveys are used to collect data from a variable number of items or

people for a comparative study. They are used when a new project is

planned, to prove the need and the demand of the customer.

Surveys can be used anywhere in the organization to find out specific

information that is necessary to make improvements in a process.

Surveys:

• Are an inexpensive way to test a system or product.

• Can be used with a large number of people or a small group.

• Can give you an overall view, determined by the questions you ask.

• Show if an organization is meeting its quality goals.

• Help identify satisfied and dissatisfied customers or employees.

Survey Process

1. Determine the group to be studied.

2. Determine what questions will be asked.

Note: Train your data collectors thoroughly. Everyone must know how to ask

the questions, who to approach, and how to approach them.

3. Compile your results in chart form using a Pareto chart, histogram,

and other tools that will give you clarification.

4. Use the compounded data to form a base for improvement.

5. Continue to take data to monitor improvements, and make sure the

improvements you have made are working.

Caution!

• Data must be collected honestly and consistently.

• An untrained collector can skew the data to reflect personal biases.

• A poor, inconsistent survey will give you invalid data.

• Make sure there is enough time allowed for the collecting process.

Brainstorming

SURVEY

1.

2.

3.

4.
DAU PROGRAM MANAGERS TOOL KIT

106

AFFINITY DIAGRAM

An affinity diagram is a technique for organizing verbal information into

a visual pattern. An affinity diagram starts with specific ideas and helps

you work toward broad categories. This is the opposite of a cause and

effect diagram, which starts with the broad causes and works toward

specifics. You can use either technique to explore all aspects of an

issue.

Affinity diagrams can help you:

• Organize and give structure to a list of factors that contribute to a

problem.

• Identify key areas where improvement is most needed.

How to do it:

1. Identify the problem. Write the problem or issue on a chalkboard

or flip chart.

2. Generate ideas. Use an idea-generation technique to identify all

facets of the problem. Use index cards or Post-it® notes to record

the ideas.

3. Cluster your ideas (on cards or paper) into related groups. Use

questions like “Which other ideas are similar?” and “Is this idea

somehow connected to any others?” to help group the ideas together.

4. Create affinity cards. For each group, create an affinity card, a

card that has a short statement describing the entire group of ideas.

5. Cluster related affinity cards. Put all of the individual ideas in a

group under their affinity card. Now try to group the affinity cards

under even broader groups. You can continue to group the cards

until your definition of “group” becomes too broad to have any meaning.

6. Create an affinity diagram. Lay out all of the ideas and affinity

cards on a single piece of paper or a blackboard. Draw outlines of

the groups with the affinity cards at the top of each group. The

resulting hierarchical structure will give you valuable insight into the

problem.
DAU PROGRAM MANAGERS TOOL KIT

107

Environment

Equipment Ergonomics

Training

Original Document

No Definition

of Quality

Noise

Lighting

Desk Height

Chair Height

Comfort

Handwriting

Grammar

Punctuation

Spelling

Draft Copy

Final Copy

Distribution

Font

Computers

Printers

Typewriters

Typing Skill

Editing Skill

Computer Skill

Proofreading Skill

No Measurement

No Feedback

Technical Jargon

Slang

Affinity Diagram

Interruptions

Unreasonable

Deadlines

Time of Day

Typographical Errors

Author Skill Requirements

AFFINITY DIAGRAM EXAMPLE

A publication team wanted to reduce the number of typographical errors

in their program’s documentation. As part of a first step, they conducted

a brainstorming session that produced the following list of factors that

influenced errors.

Computers

Proofreading Skill

Unreasonable

Deadlines

Lighting

Chair Height

Desk Height

Technical Jargon

Handwriting

Slang

Spelling

Draft Copy

Distribution

Final Copy

Computer Skill

No Measurements

No Feedback

Printers

Noise

Typewriters

Comfort

Time of Day

Interruptions

Grammar

Punctuation

Font

Editing Skill

Typing Skill

The following affinity diagram helped them to focus on areas for further analysis.
DAU PROGRAM MANAGERS TOOL KIT

108

PAIRWISE RANKING

Pairwise ranking is a structured method for ranking a small list of items in

priority order.

Pairwise Ranking can help you:

• Prioritize a small list.

• Make decisions in a consensus-oriented manner.

How to do it:

1. Construct a pairwise matrix

2. Rank each pair. For each pair, have the group (using a consensusoriented discussion) determine which of the two ideas is preferred.

Then, for each pair, write the number of the preferable idea in the

appropriate box. Repeat this process until the matrix is filled.

3. Count the number of times each alternative appears in the matrix.

4. Rank all items. Rank the alternatives by the total number of times they

appear in the matrix. To break a tie (where two ideas appear the same

number of times), look at the box in which those two ideas are compared.

The idea appearing in that box receives the higher ranking.

Each box in the matrix represents the intersection (or pairing) of two items. If your list has five

items, the pairwise matrix would look like this,

the top box representing idea 1 paired with idea 2:

1

 2

3

 4

2

3

4

5

1

2 2

1 3

 4

2

3

4

5

1

2 2

3

 4

2

3

4

5

1

2 2

1 2 3

1 2 3 4

5 5 5 5

2

3

4

5

1 and 2 compared

2 is better

1 and 3 compared

1 is better

4 and 5 compared

5 is better

...and so on

until...

Alternative 5

ranks 1st overall

Alternative 5

appears 4 times in

the matrix

Alternative 1 2 3 4 5

Count 2 3 1 0 4

Rank

Alternative 123 45

Count 231 04

Rank 3rd 2nd 4th 5th 1st
DAU PROGRAM MANAGERS TOOL KIT

109

PAIRWISE RANKING

Example

A program team was asked to recommend a site for testing a unique portion

of a system. A feasibility study produced a list of six possible locations. The

team then used pairwise ranking to determine that Nellis AFB was best

suited for this particular test.

1. Fort Huachuca 4. Nellis AFB

2. Edwards AFB 5. Eglin AFB

3. Kirtland AFB 6. Hanscom AFB

1

2 2

1 3 3

4 4 4 4

5 5 5 4 5

1 6 6 4 5

2

3

4

5

6

Site 1 2 3 4 5 6

Count 2 1 1 5 4 2

Rank 3rd 6th 5th 1st 2nd 4th

PARETO CHART

A bar chart that arranges contributing factors/causes to a problem in order with

respect to their degree of contribution to the problem.

Used for analyzing problems, a PARETO CHART can help:

• Select improvement opportunities;

• Identify root causes with greatest impact from a cause and effect diagram;

• Check results of improvement efforts by comparing Pareto charts before

and after action is taken.

40

30

20

10

Customer Complaints

 (Food Service)

Poor

Service Cost Quantity Taste Other

CAUSES

%

CONTRIBUTION
DAU PROGRAM MANAGERS TOOL KIT

110

BENCHMARKING

Benchmarking is the process of measuring products, services, and practices against the toughest competitors or those known as leaders in

their field. Benchmarking can help you:

• Understand how you compare with similar organizations.

• Identify areas for process improvement.

HOW TO DO IT:

Identify the process to be benchmarked. Select a process (as

opposed to a product) that is important to both your organization and your

customers. Be sure the process in your organization is similar to, and

measured in the same manner as, the one to which it’s being compared.

Study other organizations. Develop a list of organizations with

comparable products and services. Determine what specific processes

the organization performs. Based on this information, rank the organizations from best to worst.

Compare and evaluate. Compare your process to the best and worst

cases, and list the important differences. These differences can suggest

potential improvements to your process.

Note: Benchmarking is not replicating a process from an organization that excels. (Unless

your goal is to be 2nd best.) It is studying the process, clearly understanding the theory

behind the process and then restudying your own process to determine improvements.

BENCHMARKING EXAMPLE:

Using inputs their customers provided, the executive leaders at

AF Product Division B decided

that their source selection process needed improvement. As

part of the initial analysis, they

wanted to see how their process

compared with others. They determined that the average number of days required for source

selection was an important process measure.

As a result of this analysis,

representatives visited AF Product

Division A and Navy Division B

and studied their source selection

procedures.

AF Product Division A 75 days

Navy Product Division B 79 days

AF Product Division B 95 days

Army Product Division A 110 days 

140 days

Government Agency C 200 days

BEST

WORST

(Our Organization)

(Average number of days required)

Source Selection 

Navy Product Division A
DAU PROGRAM MANAGERS TOOL KIT

111

FLOWCHARTING

A flowchart is a graphic representation of the steps of a process. Flowcharts help us understand the process by mapping out the steps in as

much or as little detail as needed.

Enlisted Accessions and Training

Used for analyzing a process, a FLOWCHART can help:

• Understand the existing process;

• Locate improvement areas in a process;

• Document changes to a process;

• Show relationships between different steps in a process; and

• Identify critical stages of a process.

There are standard flowchart symbols. When you are developing a

flowchart, especially in a group environment, the goal is to chart the

process. Don't waste time debating which shape a symbol should be.

A flowchart that doesn't use the standard symbols can be just as useful

as a chart that does use them.

FIRST DUTY

ASSIGNMENT

HAS

PRIOR

SERVICE

RECRUITING

DISCHARGED

YES ASSIGNED

TO A 

SPECIALTY

REQUIRES

TRAINING

TECH

TRAINING FAILS

YES

YES

YES

NO

NO

YES

NO

NO

NO

RETAIN

ENTERS

BASIC

TRAINING

PASSES
DAU PROGRAM MANAGERS TOOL KIT

112

A

B

This Symbol... Represents...

Start/Stop

Decision Point

Activity

Document

Connector (to

another page or

part of the

diagram)

Some Examples:

Receive Trouble

Report

Machine operable

Approve/Disapprove

Accept/Reject

Yes/No

Pass/Fail

Drop off travel

voucher

Open access panel

Fill out trouble

report

STANDARD FLOWCHART SYMBOLS

DEPLOYMENT FLOWCHARTS

A deployment flowchart shows the process flow and the work groups

involved in each step. It provides a graphic representation of a given

process or system work groups, or individuals responsible for each

activity.

A deployment flowchart is used anytime individuals or groups need to

analyze a process in order to improve a system.
DAU PROGRAM MANAGERS TOOL KIT

113

Deployment flowcharts are used for various functions such as training

agenda, daily schedule, meeting analysis, emergency procedures, purchasing process, communication procedures, maintenance process, etc.

Deployment Flowcharts:

• Identify involvement in a process, as it relates to the whole process.

• Define work processes, and identify existing loops through people

or departments.

• Visualize a process or system.

Deployment Process

1. Select a process or system to analyze.

2. Identify the cast of characters (people involved in the process).

3. Document the existing process using the flowchart symbols.

4. Discuss changes to be made in the process with all those involved

with the process.

5. Update the deployment flowchart with the proposed changes and

implement the new process

6. Study the effectiveness of the change and return to step #1 above.

Time correlation if applicable

Process Identification

Cast of Characters

Task Box

Circle denotes added support

Meeting

Decision Diamond

Decision Line

Shadow box means more information

on another flow chart

Report

End Source: Dr. Myron Tribus

Deployment Flow Charting

Quality & Productivity, Inc.,

Los Angeles, CA 90024

NO

YES

DEPLOYMENT FLOWCHARTS (continued)
DAU PROGRAM MANAGERS TOOL KIT

114

NOMINAL GROUP TECHNIQUE (NGT)

Ranking of consensus

Why?

Allows a team to come to consensus on relative importance of issues,

problems, or solutions by combining individual importance rankings into

a team’s final ranking.

What?

• Builds commitment to the team’s choice through equal participation in the process

• Allows every team member to rank issues without being pressured

by others

• Puts quiet team members on an equal footing with more dominant

members

• Makes a team’s consensus (or lack of it) visible; the major causes

of disagreement can be discussed.

How to do it:

1.Generate the list of issues, problems, or solutions to be prioritized

 In a new team with members who are not accustomed to team

 participation, it may feel safer to do written, silent brainstorming, especially

 when dealing with sensitive topics.

2.Write statements on a flip chart or white board

3.Eliminate duplicate and/or clarify meanings of any of the statements

 As a leader, always ask for the team’s permission and guidance

 when changing statements.

4.Record the final list of statements on a flip chart or white board

Example: Why does faculty have inconsistent output?

A Lack of training

B No documented process

C Unclear quality standards

D Lack of cooperation with other departments in developing

standards

E High turnover

 Use letters rather than numbers to identify each statement so that team

 members do not get confused by the ranking process that follows.
DAU PROGRAM MANAGERS TOOL KIT

115

5. Each team member records the corresponding letters on a piece of

paper and rank orders the statements

 Example: Larry’s sheet of paper looks like this:

A 4

B 5

C 3

D 1

E 2

 • This example uses “5” as the most important ranking and “1” as

the least important. Since individual rankings will be later

combined, this “reverse order” minimizes the effect of team

members leaving some statements blank. Therefore, a blank

(value = 0) would not, in effect, increase its importance.

6. Combine the rankings of all team members

 John Paul George Ringo Mary Total

 A 4 5 2 2 1 = 14

 B 5 4 5 3 5 = 22

 C 3 1 3 4 4 = 15

 D 1 2 1 5 2 = 11

 E 2 3 4 1 3 = 13

“No documented process,” B, would be the highest priority. The team would

work on this first and then move through the rest of the list as needed.

Variation:

Weighted Multivoting

Each team member rates, not ranks, the relative importance of choices by

distributing a value, e.g., 100 points, across the options. Each team member

can distribute this value among as many or as few choices as desired.

Example:

 John Paul George Ringo Mary Total

 A 20 10 = 30

 B 40 80 50 100 45 = 315

 C 30 5 10 25 = 70

 D 5 10 20 = 35

 E 10 10 20 10 = 50

With large numbers of choices, or when the voting for the top choices is

very close, this process can be repeated for an agreed-upon number of

items. Stop when the choice is clear.

NOMINAL GROUP TECHNIQUE (NGT) (continued)
DAU PROGRAM MANAGERS TOOL KIT

116

CREATIVE PROBLEM SOLVING

PROCESS STEPS

1. List perceived problems

2. Gather relevant data

3. Define actual problem

4. Determine alternative solutions

5. Analyze and evaluate alternatives

6. Select solution

7. Validate solution

DIVERGENT THINKING*

1. Accept all ideas and alternatives

2. Defer judgment or evaluation

3. Discuss, combine, hitchhike, improve ideas

4. When exhausted, move to converge

CONVERGENT THINKING*

1. Establish categories of alternatives

2. Develop evaluation criteria

3. Avoid premature closure

4. Keep eye on objective

5. List strengths and weaknesses

6. Select best alternative or idea

*Used sequentially during all problem-solving steps
DAU PROGRAM MANAGERS TOOL KIT

117

KNOT CHART

As you work your way through the problem, everything should move

into the left column – Know.

The Knot Chart is useful for:

• Initially sorting the wheat from the chaff

• Organizing/coordinating the next steps of the problem-solving

process

Think

Know Need to Know Opinion We Know
DAU PROGRAM MANAGERS TOOL KIT

118

QUALITATIVE PROBLEM SOLVING

(Kepner - Tregoe)1

Deviation Statement: (Describe the actual performance vs. should performance)

1. Define deviation.

2. Describe what deviation IS and IS NOT.

3. List distinctions between what deviation IS and IS NOT.

4. Do distinctions indicate or suggest a change?

5. Determine possible causes based on distinctions and changes.

1Copyright Kepner Tregoe, Inc. (1981). All rights reserved. Reprinted with permission.

(Kepner-Tregoe, Inc., Research Road, P.O. Box 704, Princeton, N.J. 08542)

Specifying

Question

What?

(Identify)

Where?

(Location)

When?

(Timing)

Extent?

(Magnitude)

Does the distinction

suggest a change?

What is distinctive about

"Is" vs. "Is Not"? Is Is Not

Possible Causes:

Most Likely Cause:
DAU PROGRAM MANAGERS TOOL KIT

119

GANTT CHART

A Gantt Chart is used for planning schedules and managing projects. It

is a method for basic planning and work instruction.

How to do it:

1. The Gantt Process begins by listing the activities of a project in

order of execution.

2. Place the number of each activity across the top of your chart. Time

duration such as days, weeks, years, etc., can replace activity numbers if appropriate.

3. Draw vertical lines across the chart for each item.

4. Starting with number 1 begin comparing the activities. Can number

1 be done at the same time as number 5 or 6?

5. Draw horizontal lines to indicate which activities can be done

simul taneously.

6. You now have an overview of your project giving you a starting

point, and time saving measures to help you complete the project

on time.

 ACTIVITIES 1 2 3 4 5 6

1. Requirements are written

2. Finances are arranged

3. Bidding takes place

4. Contractor is selected

5. Prototype is built

6. Testing begins

 ACTIVITIES

1. Requirements are written

2. Finances are arranged

3. Bidding takes place

4. Contractor is selected

5. Prototype is built

6. Testing begins
DAU PROGRAM MANAGERS TOOL KIT

120

Notes