# Presentation: 186737

## NAFCOMProbabilistic Cost Analysis Modulepresented bySharon WinnNASA Cost Symposium March 10, 2004

- NAFCOMProbabilistic Cost Analysis Module*presented by*Sharon Winn*NASA Cost Symposium **March 10, 2004*

## NAFCOM is a parametric estimating tool for space hardware. 

- NAFCOM is a **parametric **estimating tool for space hardware.
- It is intended to be used in the **very early phases** of a development project.
- NAFCOM can be used at the **subsystem or component** levels and estimates **development and production** costs.
- NAFCOM is applicable to various types of missions (**manned spacecraft, unmanned spacecraft, and launch vehicles**).
- There are two versions of the model: a **government version** that is restricted and a **contractor releasable version**.

**NAFCOM Description**

## NAFCOM Evolution

- Since 1990, eight versions of NAFCOM have been developed and distributed across NASA and other government agencies.

- NASCOM database in hardcopy
- Estimators hand-entered data into spreadsheets
- Database contained 65 data points

- Allowed online searching & copying of data
- Cost estimates developed in spreadsheets with CERs created by individuals
- Database contained 70 data points

- Fully functional cost model with user defined WBS and data access
- CERs built within NASCOM using “1st Pound” method
- Database contained 91 data points

- Combined NASA and Air Force data
- Enhanced search and filtering of data
- Standardized WBS elements created
- Database contained 102 data points

- First non-weight based CERs for five subsystems (Complexity Generators)
- Gov and contractor versions distributed
- Database contained 114 data points

- Total re-write of all NAFCOM program code
- Complexity Generators for all subsystems
- Major user interface improvements
- Database contains 122 data points

**1990**

**1992**

**1994**

**1996**

**1998**

**2000**

**2002**

- NASCOM Books

- NASCOM Automated DB

- NASCOM Ver. 3.0

- NASCOM Ver. 4.0

- NAFCOM96

- NAFCOM99

- NAFCOM Ver. 2002

- NAFCOM Ver. 2004

- Complexity Generators reworked
- Fully integrated version of SOCM
- Probabilistic Cost Risk Analysis module

**2004**

## NAFCOM provided a point estimate as opposed to a probabilistic range estimate.
The addition of a probabilistic cost risk analysis module:
provides the ability to identify, analyze, and quantify risk/uncertainty 
Expands NAFCOM from a static analysis tool to one that provides probabilistic analysis of uncertain values

- The addition of a probabilistic cost risk analysis module:
- provides the ability to identify, analyze, and quantify risk/uncertainty
- Expands NAFCOM from a static analysis tool to one that provides probabilistic analysis of uncertain values

**NAFCOM Probabilistic Cost Analysis*****Rationale***

## NAFCOM Probabilistic Cost AnalysisDevelopment

- SAIC was tasked with developing a Probabilistic Cost Analysis capability for NAFCOM.
- Experts in the risk field, including representatives from MCR, Aerospace, NASA, IPAO, and Mitre participated in the methodology development.
- A meeting was held at NASA HQ to discuss issues including sampling approaches, incorporation of correlation and model implementation.
- Dr. Steve Book has been working directly with SAIC and NASA to ensure that the best possible approaches are considered for integration into NAFCOM and to consult on the methodology implementation.

## NAFCOM Probabilistic Cost AnalysisModel Selection

- Choice between Analytic approach (FRISK) or Monte Carlo-based sampling approach
- Analytic Approach
- Pros: Simple for summing WBS elements, instantaneous results, simple development
- Cons: Difficult to apply risk to algebras beyond summation (i.e., inflation, phasing)
- Monte Carlo Approach
- Pros: Allows probability analysis on different algebras
- Cons:  Difficult to develop or interface w/COTS, time intensive to run (I.e., 5,000 to 10,000 samples)
**Decision:  **_**Use Analytic Approach**_

## NAFCOM Probabilistic Cost AnalysisImplementation 

***B***

***0 ***

***L***

***H***

**Best-Estimate Cost**

**Distribution of ****Estimating Error**

**Cost Implications of Complexity Drivers (Technical, Management and New Design)**

***SEE***

***B***

_**At Estimating Level**_

## NAFCOM Probabilistic Cost AnalysisImplementation 

- +

- +

- .

**WBS Element 1**

**WBS Element 2**

**Total Cost**

**  ****Approximate by lognormal distribution**
**  ****Compute analytically**
**  ****Mean**
**  ****Mode (“Most likely”)**
**  ****Confidence percentiles**

_**At Summation Levels**_

- .

- .

## NAFCOM Probabilistic Cost AnalysisModel Interface

- Users can elect to run NAFCOM in Risk On or Risk Off mode via toolbar

**NAFCOM Probabilistic Cost Analysis*****Model Interface***

## NAFCOM Probabilistic Cost AnalysisModel Interface

- These distributions are combined with CER errors to produce distributions for nonrecurring and recurring cost for each subsystem

**NAFCOM Probabilistic Cost Analysis*****Model Interface***

## NAFCOM Probabilistic Cost AnalysisModel Interface

- Both intra- and inter-subsystem correlation is taken into account
- Users can elect to modify the default correlation matrix
- The default correlation matrix was developed based on Dr. Steve Book’s recommendation of 0.2 correlation across all elements (“knee of the curve”)

**NAFCOM Probabilistic Cost Analysis*****Model Interface***

## NAFCOM Probabilistic Cost AnalysisModel Interface

- Result data includes summary statistics, probability densities, and cumulative distributions for each major estimating element (i.e. stage, bus)

**NAFCOM Probabilistic Cost Analysis*****Model Interface***

## NAFCOM Probabilistic Cost AnalysisSummary

- Uses well-documented analytical method to calculate risk
- User can input low, most likely, and high values for all NAFCOM complexity generator and conventional CER inputs
- Incorporates both technical and estimating uncertainties
- Incorporates correlation between subsystem costs
- Results displayed to the user are summary statistics, probability densities, and cumulative distributions for DDT&E, Flight Unit, Production, and Total Costs for each major estimating element (i.e. stage, bus, etc.)
- User can select either the Normal distribution or the Lognormal distribution to approximate the final results

## NAFCOM Probabilistic Cost AnalysisCurrent Status

- Model is in final test and debug phase
- A beta version of the model was delivered to MSFC last week
- Roll out of model to cost estimating community is planned for early April
- NAFCOM can be downloaded from the NAFCOM home page at _[http://](http://nafcom.saic.com/)__[nafcom](http://nafcom.saic.com/)__[.](http://nafcom.saic.com/)__[saic](http://nafcom.saic.com/)__[.com](http://nafcom.saic.com/)_
- _[winns@saic.com](mailto:winns@saic.com)_
- 256-971-7275        256-705-8539

## NAFCOM Probabilistic Cost AnalysisPlanned Enhancements

- Initial delivery of NAFCOM probabilistic cost analysis module does not address engine CERs, the allocation of risk dollars, or an assessment of the accuracy of the sampling method used.
- The the cost risk analysis capability will be expanded to include all engine CERs including liquid, combined cycle, and air breathing estimating algorithms allowing the user to perform risk analysis on all current NAFCOM CER types.
- The ability to determine which elements have the most cost risk associated with them and to allocate risk dollars back to those WBS elements will be added.
- User will be allowed to select a point estimate (most likely,mean, median, mode) and a level of confidence above the point estimate to determine “risk dollars.
- There is no unique “optimal” way of allocating the risk dollars, but NAFCOM will implement a technique acceptable to the NASA estimating community that takes into account inter-element correlations and other features of the estimate.
- The accuracy of the analytical versus Monte Carlo approaches to sampling will be studied to determine the most cost effective capability.
- Determine if there is any value added in using the Monte Carlo sampling approach over the statistical method.
- Document findings to support/refute the need to use one approach over the other.