# Presentation: 439889

## Perspectives on NASA Mission Cost and Schedule Performance TrendsPresentation at GSFC Symposium3 June 2008

**David Bearden****Acknowledgments: Claude Freaner, Bob Bitten, Debra Emmons, Tom Coonce**

## Typical Questions

- What is the magnitude of cost and schedule growth?
- How reliable are estimates in the conceptual design stage?
- Why does cost growth occur?
- What is the relationship between cost, schedule and “complexity”?
- Are there any improvements that can be made in estimating the costs of future design concepts?

## Forty NASA Robotic Science Missions Experienced 27% Cost and 22% Schedule Growth

**Forty NASA Robotic Science Missions Experienced 27% Cost ****and 22% Schedule Growth**

## While Significant Variability is Evident, for Every 10% of Schedule Growth, there is a Corresponding 12% Increase in Cost

## Comparison of Schedule Growth Data with Agency Guidelines: NASA Telescope Missions

**Comparison of Schedule Growth Data with Agency Guidelines: NASA Telescope Missions**

**General Rule of Thumb**
**1 Month per Year**

**NASA/JPL Guidance**
**1.8 Month per Year**

**Four of Six Telescope Missions Exceeded Common Schedule Reserve Guidelines **

## Comparison of Schedule Growth and Success for Planetary Missions vs. Earth-orbiting Missions

- Development times for Planetary missions less than Earth-orbiting missions due to constrained launch windows
- Planetary missions experienced less schedule slip on average than earth-orbiting missions
- However, planetary missions failed or impaired twice as often

_**Average Development Time**_

## Typical Questions

- What is the magnitude of cost and schedule growth?
- How reliable are estimates in the conceptual design stage?
- Why does cost growth occur?
- What is the relationship between cost, schedule and “complexity”?
- Are there any improvements that can be made in estimating the costs of future design concepts?

## How Reliable are the Projects’ Estimates at the Conceptual Design Stage and How Does Confidence Progress?

**Ten Missions Demonstrate How Accuracy of Project Estimates Increases Over Time however Cost Growth, Over and Above Reserves, Still Occurs Deep into the Project Life Cycle**

## In What Phase Does Cost Growth Occur?

**Greatest Growth Occurs During Integration and Test (Phase D) When Trying to Get Hardware & Software to Function as Designed**

## Typical Questions

- What is the magnitude of cost and schedule growth?
- How reliable are estimates in the conceptual design stage?
- Why does cost growth occur?
- What is the relationship between cost, schedule and “complexity”?
- Are there any improvements that can be made in estimating the costs of future design concepts?

## Some of the Reasons

- Inadequate definition of technical and management aspects of a program prior to seeking approval
- 				(NASA’s Project Management Study, 1980)
- Program and funding instability; difficulties in managing programs in an environment where funding must be approved annually and priorities change
- 	(Advisory Committee on the Future of the U.S. Space Program, 1990)
- Lack of emphasis on technological readiness and requirements on the front end of a program
- (NASA’s Roles and Missions Report, 1991)
- Program redesign, Technical Complexity, Budget Constraints, Incomplete Estimates
- (GAO Report on NASA Program Costs, 1992)

## The Reasons for Growth - Study of 40 NASA Missions:Internal versus External Factors Driven-Growth

- Internal Growth
-    (within Project’s control)
- Technical
- Spacecraft development difficulties
- Instrument development difficulties
- Test failures
- Optimistic heritage assumptions
- Programmatic
- Contractor management issues
- Inability to properly staff an activity
- External Growth
-    (outside Project’s control)
- Launch vehicle delay
- Project redesign
- Requirements growth
- Budget constraint
- Labor strike
- Natural disaster

**Distribution of Growth**** **

**Distribution of **
**Internal Growth**** **

**Distribution of **
**External Growth **

## Mass Growth Exceeds Typical Guidance 

- Average mass growth for ten missions studied is 43% which exceeds the typical industry guidelines of 30% mass reserves (over CBE) at the start of Phase B

**Typical Reserves**

## Assessing Relationships for Causality:Inherent Optimism in Initial Design & Estimates

**Progression of Average Cost Growth for Discovery Selections May be Indicative of Competitive Pressures Leading to More Aggressive Designs**** **

## Typical Questions

- What is the magnitude of cost and schedule growth?
- How reliable are estimates in the conceptual design stage?
- Why does cost growth occur?
- What is the relationship between cost, schedule and “complexity”?
- Are there any improvements that can be made in estimating the costs of future design concepts?

## Hypothesis

***Complexity Index*** could be derived using a broad set of parameters to arrive at a top-level representation of the system
- Correlation to spacecraft cost and/or development time based on actual program experience might be apparent
- Data assembled for most spacecraft launched during past two decades (1989 to present) including technical specifications, costs, development time, mass properties and operational status
- Complexity Index calculated based on performance, mass, power and technology choices for purposes of comparison
- Relationship between complexity and “failures” investigated compared with adequacy of cost and schedule resources
- Method to assess complexity at the system-level should allow more informed overall decisions to be made for new systems being conceived

- Illustrations reprinted courtesy of NASA

## Complexity Index Example

**Complexity Index Example**

## When is a Mission Too Fast?

**When is a Mission Too Fast?**

## When is a Mission Too Cheap?

**When is a Mission Too Cheap?**

## 3-D Trade Space – Intuitive Result: Missions that have the greatest complexity, are highest cost and longest development 

## Complexity Bands vs. Cost and Schedule Help Proposers Define Scope of Mission to Fit Fixed Cost & Schedule

**Complexity Bands vs. Cost and Schedule Help Proposers Define Scope of Mission to Fit Fixed Cost & Schedule**

**    ****2012**

## NASA’s Report Card Following Mars ’98 Failures

- Complexity of Failed Missions High in Both Catagories!
- Planetary Missions are “Fastest”
- But fail more often than earth-orbiters
- NASA Earth-Orbiting Missions are “Cheapest”
- But longer to develop than planetary
- Overall Success Record is About 3 out of 4 !

- Reprinted with permission of Aviation Weekly and Space Technology

- AW&ST 12 June 2000

## For a project that has fixed requirements and schedule, the inevitable outcome is that cost will grow if developmental problems occur

**For a project that has fixed requirements and schedule, the inevitable outcome is that cost will grow if developmental problems occur**

- Case Study: Mars Exploration Rover (MER)
- 90-day surface lifetime; ~9-mos cruise
- Launch Mass: 1050 kg (Delta II)
- Mobile platform: 1000-m range
- Assessment found that:
- 33-month development appeared inadequate
- “Open Checkbook” and heritage offset shortfall
- Mitigations:
- Focused on rapidly deploying staff to front load schedule (dual/triple shifts)
- Developed extra hardware test-beds
- Cost grew from $299M to $420M

- AW&ST, 26 May 2003

- Reprinted with permission of Aviation Weekly and Space Technology

**Notes:**
Findings and recommendations of the complexity-based risk assessment are summarized above.

## Typical Questions

- What is the magnitude of cost and schedule growth?
- How reliable are estimates in the conceptual design stage?
- Why does cost growth occur?
- What is the relationship between cost, schedule and “complexity”?
- Are there any improvements that can be made in estimating the costs of future design concepts?

## Example: Substantial Differences Exist between STEREO Science Definition Team (SDT) and Final Implemented Configuration

_**SDT Configuration**_

_**Final Configuration**_

- Illustrations reprinted courtesy of NASA

## Effect of Increased Complexity on Flight System Cost: STEREO Complexity Increased from 40% to 60%

## Effect of Increased Complexity on Development Time: STEREO Complexity Increased from 40% to 60%

**Effect of Increased Complexity on Development Time:**** STEREO Complexity Increased from 40% to 60%**

## Typical Cost-risk Analyses Won’t Capture Large Changes During Concept Evolution

- $299M Estimate with 20% Reserve

- Final Cost
- $551M

## Inadequate Budget Planning for One Project Results in a Domino Effect for Other Projects in the Program Portfolio

_**STP 2000 Planned Funding**_

_**STP Actual Funding History**_

_**Total Program Funding 1999-2006**_
-  Planned = $689M
-  Actual = $715M

**Although the total program funding remained essentially the same over this time period, implementation of successive missions (e.g. MMS) was substantially affected**

**MMS**

**STEREO**

**MMS**

**STEREO**

## Summary

- Methods exist to estimate cost and schedule at the conceptual phase albeit with some level of uncertainty
- While estimates become more accurate as project matures, the greatest growth manifests itself late in project during Integration & Test
- Data highlighted that the primary reason for cost and schedule growth is internal project technical and development issues often associated with instruments
- Initial project estimates may be unreliable due to design and technology immaturity and inherent optimism
- Success dependence on system complexity and adequacy of resources observed with identification of a “no-fly zone”
- Better technical and programmatic appraisal early in lifecycle is needed along with independent assessment of design and programmatic assumptions

## References and Further Reading

- Bearden, David A., “A Complexity-based Risk Assessment of Low-Cost Planetary Missions: When is a Mission Too Fast and Too Cheap?”, Fourth IAA International Conference on Low-Cost Planetary Missions, JHU/APL, Laurel, MD, 2-5 May, 2000.
- Dornheim, Michael, "Aerospace Corp. Study Shows Limits of Faster-Better-Cheaper", *Aviation Week and Space Technology*, 12 June 2000.
- Bearden, David A., "Small Satellite Costs", *Crosslink Magazine*, The Aerospace Corporation, Winter 2000-2001.
- Dornheim, Michael, “Can $$$ Buy Time?", *Aviation Week and Space Technology*, 26 May 2003.
- Bitten R.E., Bearden D.A., Lao N.Y. and Park, T.H., “The Effect of Schedule Constraints on the Success of Planetary Missions”, Fifth IAA International Conference on Low-Cost Planetary Missions, 24 September 2003.
- Bearden, D.A., “Perspectives on NASA Robotic Mission Success with a Cost and Schedule-constrained Environment”, Aerospace Risk Symposium, Manhattan Beach, CA, August 2005
- Bitten R.E., Bearden D.A., Emmons D.L., “A Quantitative Assessment of Complexity, Cost, And Schedule:  Achieving A Balanced Approach For Program Success”, 6th IAA International Low Cost Planetary Conference, Japan, 11-13 October 2005.
- Bitten R.E., “Determining When A Mission Is "Outside The Box": Guidelines For A Cost- Constrained Environment”, 6th IAA International Low Cost Planetary Conference, October 11-13, 2005.
- Bitten R., Emmons D., Freaner C., “Using Historical NASA Cost and Schedule Growth to Set Future Program and Project Reserve Guidelines”, IEEE Aerospace Conference, Big Sky, Montana, March 3-10, 2007.
- Emmons D., “A Quantitative Approach to Independent Schedule Estimates of Planetary & Earth-orbiting Missions”, 2008 ISPA-SCEA Joint International Conference, Netherlands, 12-14 May 2008.
- Freaner C., Bitten R., Bearden D., and Emmons D., “An Assessment of the Inherent Optimism in Early Conceptual Designs and its Effect on Cost and Schedule Growth”, 2008 SSCAG/SCAF/EACE Joint International Conference, Noordwijk, The Netherlands, 15-16 May 2008.