# Presentation: 168551

## FORTY YEARS OF NASA COST ESTIMATING:ARE THE GOOD GUYS WINNING OR LOSING?NASA COST ESTIMATORS’ SYMPOSIUMHumboldt C. Mandell, Jr.Research FellowThe University of Texas at Austin Center for Space Research

***March 10, 2004***

## ARE THE GOOD GUYS WINNING?

*This presentation summarizes 40 years of cost estimating history at JSC*
*I’m sure that all of the problems I talk about have been fixed in the year that I have been retired*
*There have been good times and bad*
*This is a series of mini-case studies that will illustrate the best and worst*
*I will show how cost estimation (red) and **management cultural reform (blue)** have been intertwined*

## Case 1, Apollo 

***NASA, and NACA before it, had NO parametric cost analysis or estimating capability***
***Although some costing was done, legend says the $20 billion dollar (1961$) Lunar Program estimate was done in a limo by Administrator Webb on the way to talk to the President***
***The estimate was “supply side,” i.e., what Webb perceived the market would bear***
***Fortunately, the estimate became self-fulfilling, BUT:***
***May have created too much infrastructure, too much bureaucracy and***
***Left a legacy of high cost programs, high agency overhead***
***And, with smaller budgets, fewer programs could be afforded***

***NASA STATE OF THE ART: None.  Into the vacuum came Hermann Koelle at MSFC, to create cost estimation capability for the first time in the agency***

## The First NASA Cost Teams

***In 1961, Hermann Koelle at MSFC compiled “Handbook of Astronautical Engineering,” which contained the first rudimentary costing equations found in NASA literature.***
***Koelle Hired Terry Sharpe (1961), Cary Rutland, others, as first NASA parametric cost estimators ***
***In cooperation with JSC, began planning programs to follow Apollo***
***JSC followed in 1962 (Mandell, Ivy Hooks Schneider, Gil Chisholm, Richard Lupo ~62-65) Under Dave Hammack, Bill Stoney (Advanced Spacecraft Technology Division)***
***NASA commissioned RAND to develop early methods ***
***Methods drew heavily on DOD work, primarily at the RAND Corporation (Novick, Campbell, Large, Fisher, Margolis)***

## The First Cost Teams, Cont.

***JSC Contracted with GD Ft. Worth (Tom Brents) for Spacecraft methods (1966)***
***But data were so scarce that early NASA methods were statistically questionable***
***Schneider and Lupo published the first NASA cost model (Manned Spacecraft Costing Technique, MASCOT) in 1967***
***Lupo, a Georgia Tech mathematician, created the first “spreading” algorithms based on beta curves (TIPP)***
***First use of computer graphics (Riedlinger, Smith, 1968, at JSC) for cost “Spreading”***
***The first Scheduling Model was produced by LTV for JSC (1971)***

***The First Cost Teams, Cont.***

## Case 2, Space Shuttle 

*The next element of the “Von Braun” Plan on the way to MARS*
*Better estimating techniques available*
*Shuttle orbiter originally estimated from ONE data point (Boeing B-52)*
*Frank Freiman first came on the scene with PRICE, but did not want to market it! (NASA reluctant to use it, but gave Freiman the FIRST contract to develop PRICE for NASA)*
*Agency Planners had acute awareness of high costs*
*Shrinking budgets*
*Limited NASA ability to do what they wanted to do (go to Mars)*
*Several studies of how to reduce costs*
*Had not yet discovered the tie between management culture and cost*

## Case 2, Space Shuttle 

*Essentially MET the DDT&E estimates!!*
*Unrealistic “traffic model” of 50 flights per year, and lack of cultural change produced erroneous costs per flight ($10.45 million 1971$)*
*Management Studies were largely ignored, overpowered by the “culture”*
*Costs actually drove the configuration*
*Use of SRB’s vs. Two Stage Fully Reusable (Compromised Ops costs for Development Cost Peak Funding Reduction*
*Use of Carrier Aircraft instead of airbreather engines.*

***NASA STATE OF THE ART: “RAND” style parametrics; Few Data Points; First user of RCA PRICE (Freiman); inter-Center costing teams; It is all about ANNUAL costs***

## Major Findings (from Fox)

**1     Conduct training for program/project ****	****	****management personnel**
** ****2.****	****Conduct annual meeting of project    ****	****	****managers**
** ****3.****	****Develop realistic cost estimates**
**	****4.****	****Clarify HQ role in project management**
**	****5.****	****Improve adequate front-end planning      ****	****	****definition**
**	****6.****	****Need for long range vision and agency  ****	****	****roles**
**7.****   Conflict between institutional and ****	****	****program needs**
**	****8.****	****Attention to operations and logistics**
**	****9.****	****Need for adequate requirement definitions**
**10.****	****Contractor and NASA buy-ins**
**	****11.****	****Clarity and communication of mission ****	****	****goals and  objectives**
**	****12.****	****Need for communication at all NASA levels ****	****	****and contractor teamwork**
**	****13.****	****Improve management of contingency funding**
**	**
**14.****	****Eroding in-house technical expertise**
**	****15.****	****Need for risk assessments**
**	**
**16.****	****Increasing technical complexity of projects**
**	**
**17.****	****Develop formal top-down planning process**
**	****18.****	****Formalize S/E process**
**	****19.****	****Maintain more consistent documentation**
**	****20.****	****Better manage Congressional issues**
**	**
**21.****	****Over-commitment**
**	****22.****	****Need to establish improved international ****	****	****involvement**
**	****23.****	****Improve program control function — develop ****	****	****agency models, control to baseline, etc.**
**	**
**24.****	****Acquisition reform**

- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 

- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 

- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 

- 
- 
- 
- 
- 
- 
- 
- 
- 
- 

- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 

- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 

- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 

- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 

- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 

- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 

- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 

- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 

- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 

- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 

- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 

- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 

- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 

- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 

- 6
- 1
- 6
- 9
- 8
- 6
- 3
- 3
- 7
- 7
- 10
- 9
- 7
- 8
- 7
- 5
- 7
- 4
- 3
- 1
- 4
- 4
- 6
- 3

**Mandell/Feasible Changes in Mgmt Style/gjf — 5/20/93**

**Issues**

- Information provided by the program/project initiative

***Common Issues in the Management of NASA Programs and Projects***

## Corrective Action History(Richard Fox’s Chart)

**PM Workshop (1975)**
**SEP (1976)**
**SEP (1985)**
**SEP (1986)**
**PM Colloquium (1980)**
**NASA Cost Studies (1986)**
**(Ride) Leadership (1987)**
**NASA HQ Mgmt. Phillips (1988)**
**90-Day Study (1989)**
**Exec. PM Colloquium (1991)**
**APM Follow-on Colloquium (1991)**

- *  Published 4 years later in NMI 7120.3

***Implementation Success***

**Presentations made:**
**No follow-up or**
**implementation**

**Recommendations**
**selectively or partially**
**implemented**

**Formal follow-up**
**and partial**
**implementation**

**Formal follow-up**
**and full**
**implementation**

**Hearth Study (1981)***
**PM Colloquium (1982)**
**Phillips SSF Report (1986)**
**NASA Prog. Ctrl. (1989)**

**Phillips Mgmt. Report (1986)**
**Future of U.S. Space (Augustine) (1990)**

## THE “COMMANDMENTS”  A Summary of Lessons Learned From Previous NASA Management Studies

***The ingredients of successful low-cost, high technology programs are well known and universally recommended by the many successful program managers interviewed:***
***Government (NASA) should only define and verify requirements***
***Keep requirements fixed:  once requirements are stated, only relax them; never add new ones***
***Place product responsibility in a competitive private sector***
***Specify end results (performance) of products, not how to achieve the results***
***Minimize government involvement (small program offices)***
***Insure that all technologies are proven prior to the end of competition***
***Utilize the private sector reporting system:  reduce or eliminate specific government reports***
***Don’t start a program until cost estimates and budget availability match***
***Minimize or eliminate government imposed changes***
***Reduce development time:  any program development can be accomplished in 3 to 5 years once technology uncertainties are resolved***
***Force people off of programs when development is complete***
***Incentivize contractors to keep costs low (as opposed to CPAF, CPFF of NASA).  Reward only product performance.***
***Use geographic (or virtual?) proximity of contractor organizations when possible ***
***To reduce the number of interfaces and keep responsibilities clean, use the major prime contractor as the program integrator***

***NASA has known these principles for many years. ***
***Implementation has been resisted by the culture***

## Case 3, Space Station

*A story of intentional deception?*
*Costing Task Force at NASA HQ*
*First given $1 billion cap*
*Then given $8 billion cap*
*Inter Center task force led by MSFC (O’Keefe Sullivan)*
*There never was an $8 billion space station, despite many compromises, configuration changes*
*Cost (peak annual funding) actually drove the configuration*
*Renewed awareness of the importance of lowering costs: More management /cost studies*

## Case 3, Space Station, Cont.

*Again, “lessons learned” not implemented*
*HEAVY Government Involvement in development vs. lesson to use performance specs*
*Mixing of Contractor and Civil Service vs. Lesson to Separate them*
*Work packages ADDED management complexity*
*International work packages*
*KSC Processing vs. “Ship and Shoot”, etc.*
*The “days of reckoning” *
*One vote victory (mainly because of cost growths)*
*Loss of “Freedom” Advent of “Alpha”*
*The saga continues today*
*OMB Administrator and Comptroller to “fix the problem”*
*This performance damaged further NASA’s management credibility and is a significant reason that Mars missions are not in the foreseeable future*

***NASA STATE OF THE ART: Extensive use of inter-center costing teams; mature parametrics; still data limited; rediscovery by Cyr of cultural effects on costs; first algorithm to quantify effects of adding work packages (Whitlock)***

## Effect of Development Organization Typeon Program Development and Production Cost*(Another Cyr Chart)

**Relative**
**Development**
**Cost**

**Developing Organization Type**

- *  From AMCM Database

- Aircraft

- Missiles

- Spacecraft

***Small changes in development management style can have huge influences on the costs of space programs***

- Reference

- 3

- 1.74

- 1.92

- 2.04

- 2.34

**+**

**+**

- 2

**+**

- 1

- 0

**+**

## Cost Versus Management Style(Kelley Cyr’s Chart)

**For a given product, cost is primarily a function**
**of the manner of doing business of the developer**

## Cost vs. Mass and Specification(Kelley Cyr’s Chart)

## Case 4, SEI 

*SEI costing is a story of over- compensation by NASA to criticism of cost overruns*
*Early caution to avoid high peak funding (a lesson learned) not only ignored, but ridiculed by HQ management “This is the President’s Initiative . .”*
*Cohen led study team*
*Mandell led the cost team: inter-center working group*
*Mature Cost analysis methods used very early in the process, BUT, NOT fully integrated*
*HQ insistence on rising “MERTS” base, independent of SEI*
*Highly ambitious lunar and Mars programs to run in parallel*
*High reserve levels*
*Peak funding called for tripling the NASA budget*

## Case 4, SEI, Cont.

*Cost leaks to the press were not friendly, misperceptions created that persist to today*
*BAD NEWS: It was an election year, and the Congress and the Presidency were in the hands of different parties*
*GOOD NEWS: The established management structure eliminated one full level of management (center PM reported directly to AA)*
*Full buy-in to “lessons learned” by AA Griffin*
*Recognition that continued success required independence from “old Culture”*
*“**New Culture” Management Plan was written and implemented*
*Set a record that may still stand for a procurement over $20 million (~65 days?)*
*Delegation of procurement authority to the program by the Administrator!*
*Forced us to realize the importance of early CUSTOMER INVOLVEMENT!*
*Management benchmarking studies gave us more knowledge*

***NASA COSTING SOA: First extensive use of CAD in conjunction with parametrics; Costing was very mature, and incorporated early in the design process, BUT, STILL NOT FULLY INTEGRATED (IDC’s came a little later)***

## Case 5: Faster, Cheaper, Better

*When SEI failed, Dan Goldin was brought into NASA from TRW (~1992)*
*Goldin’s outsider view of NASA was essentially the same at that of the people who had studied the NASA culture (very similar to “Commandments”)*
*Established several teams to look at NASA*
*“**Blue Team” on Management was headed by Jack Lee, MSFC Director, Fox was on it*
*Early Faster, Cheaper, Better initiative was designed to force management innovations*
*The end of “Big Science” unmanned missions*
*Many lower cost missions in their place*
*Cancellation review after 15% cost overrun*
*Speech to JPL:  “Lower costs or be shut down”*

## Case 5: Faster, Cheaper, Better, Cont.

*By this time, cost estimation was very advanced, incorporated by JPL into their IDC, and by JSC into their Design Laboratory*
*DOD and DARPA’s developments in the ’80’s and ’90’s*
*Lockheed Martin Denver’s Integrated Design Center*
*DARPA technology adapted by TRW*
*TRW assisted JPL Integrated Design Center*
*JPL Activity Based Modeling is very mature*
*Failures of the two 1999 Mars spacecraft BLAMED on FCB*
*JPL: “FCB is dead”*
*The cultural pendulum, which had produced a low-cost management system for SEI had swung back*

***NASA COSTING SOA: More extensive use of CAD in conjunction with parametrics; Advent to Design Centers with integrated Costing.  Still residual data availability issues.***

## Case 6: Costing a Mars Mission

*The President’s 2004 Space Exploration Policy:*
*Took too much advice from lunar advocates *
*Basing a Mars Mission at the Moon is a silly idea (John Glenn)*
*Lacks specifics regarding Mars Exploration*
*Cost analysts of my generation share a lot of the blame*
*Costs of SEI*
*How we dealt with the press on SEI*
*Haven’t shown what is possible within realistic funding constraints*
*So, the wrong conclusions have been drawn by policy makers*
*Reversion to the “Old Culture” NASA?*
*“**No heroics” *

***NASA COSTING SOA: YOU TELL ME!***

## CUSTOMER ACCEPTANCE

***60***

***40***

***20***

***0***

***ANNUAL BUDGET, $B***

***COMBINED MOON AND MARS MISSIONS, APOLLO- LIKE, GO EVERY OPPORTUNITY, TRIPLE NASA BUDGET***

***CURRENT***
*** ******BUDGET***

***YEAR FROM GO-AHEAD***

***5***

***10***

***0***

***CUSTOMER ACCEPTANCE***

## Effect of Development Organization Typeon Program Development and Production Cost*

**Relative**
**Development**
** ****& Production Cost**

**Developing Organization Type**

***  From NASA AMCM Database**

**Aircraft**
**Culture**

**Missiles**
**Culture**

**Spacecraft**
**Culture**

**Small changes in development management style can have huge influences on the costs of space**** ****programs**

**Reference**

**3**

**1.74**

**1.92**

**2.04**

**2.34**

**+**

**+**

**2**

**+**

**1**

**0**

**+**

**Our**** Models Tend to Put us HERE**

**Our requirements can be met HERE**

**~34%**

## HELP FROM MISSION DESIGN

***1.0***

***.50***

***.25***

***.75***

***1989 ESTIMATES***
***(REJECTED BY CUSTOMER)***

***WHAT IF WE USE MARS ATMOSPHERE FOR PROPELLANTS?***

***RELATIVE MASS***

***WHAT IF WE USE MARS WATER FOR PROPELLANTS & DRINKING?***

***APOLLO ANALOGY***

***WE RECOGNIZE THAT MARS HAS AN ATMOSPHERE***

***WE FIND LOTS OF WATER ON MARS***

***HELP FROM MISSION DESIGN***

***TIME***

***1989***

***NOW***

***WHAT IF WE USE EXISTING LAUNCH VEHICLES***

***THE MARKET- PLACE PROVIDES LOWER COST TO ORBIT***

## CUSTOMER ACCEPTANCE

***WHAT MIGHT SELL TO OUR CUSTOMERS NOW***

***60***

***40***

***20***

***0***

***ANNUAL BUDGET, $B***

***COMBINED MOON AND MARS MISSIONS, APOLLO- LIKE, GO EVERY OPPORTUNITY, TRIPLE NASA BUDGET***

***BUILD A MARS BASE, USING EXISTING LAUNCHERS, OVER 3-4 MARS OPPORTUNITIES, EXISTING BUDGET LEVELS***

***CURRENT***
*** ******BUDGET***

***YEAR FROM GO-AHEAD***

***5***

***10***

***0***

***CUSTOMER ACCEPTANCE***

## The Bush2 Plan 

*A great boost for the NASA Explorers*
*But really only announced the end of the Shuttle Program*
*Everything else is beyond this President’s ability to influence it*
*Some of the content, the timing, is contradictory to 40 years of exploration studies*
*Policy based on some questionable advice by opponents of Exploration*

## So What? 

*Bad policy means:*
*A great agency is not living up to its potential*
*A generation of spacefaring youth may be denied their “destiny”*
*In the past, before hardware was built, good estimates were made, often after the policy was established (e.g., ISS)*
*And, hopefully, this will happen again,*
*To influence the policy to be more ambitious *

## P.S.:  Good EstimatingCan Change Our Future

*There is only one major adventure left to mankind in our lifetimes, i.e., the human exploration of Mars*
*Human Mars missions, costed as though developed in the current NASA culture, are not affordable in our lifetimes, based on current cost models and budgetary trends*
*With a cultural change, NASA can send people to Mars well within our lifetimes*
*Cost estimates, based on existing culture, have created a pessimistic planning climate*
*Well-conceived estimates can be the key to the future*
*But major cultural change is also required*
*Cost estimators can play a major leadership role*

## So, ARE the good guys winning?

*Positive trends*
*Methodology,*
*Integration with the Design Process*
*Turning Engineers into estimators*
*Influencing all hardware configurations for the past 30 years*
*Recognition that NASA has customers with expectations about cost, demands for better accuracy up front.*
*Negative trends*
*Little evidence that the President’s Exploration policy had much cost analysis behind it*
*The culture still rules, and the pendulum is back firmly in the old “Apollo” culture.*
*The positives probably outweigh the negatives*

## Perception of Change

*“**Nothing is more difficult to undertake, more perilous to conduct or more uncertain in its outcome, than to take the lead in introducing a new order of things.  For the innovator has for enemies all those who have done well under the old and lukewarm defenders amongst those who may do well under the new.”*
*Machiavelli*