# Presentation: 261440

## Using Parametric Models to Demonstrate Cost Estimate Realism (or not)

**Tim Hohmann**
**Director of Professional Services**
**March 10, 2004**

## Basic Estimating Truthsor, “Abandon Hope All Ye Who...”

**Truth #1:  Your estimate will be wrong.  **Your inputs are uncertain and your model isn’t perfect.  And besides...
**Truth #2:  There is no “right answer” **at the time you perform your estimate.  The best you can hope for is a well-documented estimate of the item you’re estimating, where the eventual actual value falls within your predicted range.  Even then, you’ll *still* be wrong, because...
**Truth #3:  They never wind up building what you estimated.**
**Truth #4:   You can make the estimate anything you want.**  Any model and/or any methodology can be “gamed” to get an answer which is “politically correct” or makes the project *appear* affordable.
**Truth #5:  Wishing will not make it so.  **Reducing your estimate will *not* reduce the cost or of an unaffordable project or get a project completed in half the time.  If budgets and schedules are based on unrealistically low estimates, the project will probably overrun more than it would ordinarily have, if it ever reaches completion at all.
**Truth #6:  This will be blamed on poor estimation. **

## On the positive side...

**Parametric models work right...  **While “prior performance is no guarantee of future results”, we know that SEER, COCOMO, PRICE, NAFCOM, etc., as well as many “homegrown” cost models, all have had wide usage for many years, and have produced results which *support good management decision making*.
**...****when (if) you work them right.**
- Reasonable, properly defined inputs
- Proper assessment of technical uncertainty (parameter ranges)
- Calibration against historical data
- Documentation
- Documentation
- Documentation

## Estimate Realism:How do you know?

**How can you tell if your prime contractor/program office estimate is realistic, and will provide good guidance for decision makers?**

## Estimate Realism:SEER-SEM Example

**Step 1:  Evaluate Goals (project office estimate) against current parametric estimate**

## Estimate Realism:SEER-SEM Example

**Step 2:  Estimator selects “degrees of freedom” in calibrating the parametric model.**
- Technology (people, processes)?
- Product Characteristics?
- Probability?
**	****Make parameter adjustments (redefine the development scenario) to make estimate match goal.**

## Estimate Realism:SEER-SEM Example

**Step 3:  Evaluate whether what you had to do to predict achievement of the goal was reasonable.**

## Estimate Realism:SEER-SEM Example

**Step 3:  Evaluate whether what you had to do to estimate achievement of the goal was reasonable.**
- In this instance, we matched the contractor/program office estimate by:
- Applying “Lake Woebegone” analysts
- Reducing SLOC count by 10% (!)
- Reducing labor rate (!!)

## Estimate Realism:SEER-SEM Example

**Step 4:**
- If the changes to the parametric model are deemed realistic:
- Document your assumptions
- Feed the new scenario into your program plan
- Declare victory and quit
- If not...
- Reevaluate the parametric model—will a different, more reasonable set of parameter adjustments make the estimates match?
- Reevaluate the contractor/program office estimate—why does it disagree with the independent parametric estimate?
- Continue until you’ve resolved discrepancies (at least enough to declare victory and quit)

## Ignore Estimate Realism at your Peril(with apologies to Tim Anderson)

**Joe Hamaker:  Sir, our best estimate for this new program is $150M.  Uncertainty ranges from $100M at the 10% confidence level to $220M at the 90% level.  We recommend funding it at the 70% confidence estimate, which is $185M.**
**Administrator:  Joe, I’m sorry, but the number needs to be $75M.  We won’t be able to get the program funded at a higher number, and we need a cost estimate which supports the budget request.  I need you to bring me a cost estimate at $75M.**
**Joe:  Aye, aye, sir.  We’ll see what we can do.**

## Later...

**Joe:  Well, sir, here’s our estimate.  As you’ll note, it contains certain assumptions, which include flying pigs, hockey in hell, and full cooperation between NASA and the United States Air Force.  That gets us to a 5% probability estimate of $75M.  Of course, our 50% probability estimate is still...**
**Administrator:  Thanks, Joe.  $75M on the nose.  This is just what we needed.**

## Later still...

**Administrator:  Joe, this program is a disaster.  It’s got a 100% cost overrun already, and it’s still climbing!  We based our budget on your cost estimate, which turned out to be lousy!**
**Joe:  But sir, we told you.  The pigs didn’t fly, the rink in Hades melted, and as for the Air Force...**
**Administrator:  Joe, I don’t want to hear any excuses.  Cost estimating is broken, and I’m holding you responsible.**
**Joe:  D***ed Carpio!  I’m sure this is his fault...**

## *