# Presentation: 940829

## Value Measuring Methodology

- May 2003

*Council for Excellence in Government:  Benefits Assessment Workshop*

## Why isn’t “traditional” business case analysis providingthe information OMB is looking for? 

**Why isn’t “traditional” business case analysis providing****the information OMB is looking for? **

**Primarily focused on financial benefits (e.g., ROI) ****that impact the *****government only*****...**

**Analysis is viewed as a means to get funding, ****not *****a tool for on-going management & evaluation******...***

## How can traditional analysis be supplemented to better address the challenges of the e-Government environment?

**How can traditional analysis be supplemented to better address the challenges of the e-Government environment?**

- In July 2001, the Social Security Administration (SSA), in cooperation with the General Services Administration (GSA), took on the task of developing an effective methodology to assess the value of electronic services that would be:
**1. Compliant with current federal regulations & OMB guidance**
**2. Applicable across the federal government**
**3. “Do-Able” **
- A team of Booz Allen analysts and thought-leaders affiliated with Harvard University’s Kennedy School of Government were contracted to support this effort.

## The approach used to develop VMM was built upon the foundation of a public / private partnership

| Traditional / Emerging Approaches | Federal Agency Staff |
| --- | --- |
| Legislation and OMB Circulars | State Government Staff |
| Government Developed Training Documents | Private Sector |
| Academic Institutions | OMB |
| Think Tanks | GAO |

**development**** **

**presentation**

**research & analysis**

**discussion**

**The approach used to develop VMM was built upon the foundation of a public / private partnership**

## The output of this effort was the Value Measuring Methodology (VMM)

- First articulated in ***Building a Methodology for Measuring the Value of e-Services*** (1/02)
- Refined & tested through application to two cross-agency e-Government initiatives (e-Travel & e-Authentication)
- Release of the ***VMM How-To-Guide ***and ***VMM Highlights ***document by the Best Practices Committee of the CIO Council (10/02)
- VMM Roll-Out, held by the Council for Excellence in Government in cooperation with the CIO Council’s Best Practices Committee, OMB, and GSA (4/03)

## VMM Overview

## It is important to understand what VMM IS and ISN’T... 

**VMM IS****... **
**A scalable and flexible approach for quantifying and analyzing value, risk, and cost and evaluating the relationships among them **
- Helps to create a roadmap for on-going management and evaluation
- Supports the development of critical management plans

**It is important to understand what VMM ****IS**** and ****ISN’T****... **

**VMM IS NOT****...**
- One Size Fits All
- A Way to Avoid Analysis
- Only Useful for e-Government Initiatives

***COMPLIANT WITH***
***GPRA******CCA******OMB A-11***
***CONSISTENT WITH THE PHILOSOPHY OF THE PMA***

## The Essential Factors...

## A Decision Framework

| V A L U E | Value Factors |  | priority |
| --- | --- | --- | --- |
|  | value measure(s) | metric, target, scale | priority |
| R I S K |  |  |  |
| COST | CustomizedCost ElementStructure | 1.0 ... |  |
|  |  | 2.0...
 |  |
|  |  | 3.0 ... |  |

**Risk Inventory**
**Risk  Tolerance Boundary**

**Define User ****Needs & Priorities**
**Quantifiable Measures of Performance (Metrics, Targets)**
**Foundation for Analysis & On-going Performance Measurement**
**Early Consideration ****of Risk**

**Notes:**
This Chart is a visual representation of the VMM Decision Framework. The Decision framework revolves around 3 key elements of Value, Risk, and Cost. In the early steps of the VMM process, a value structure is developed then a risk structure as well as a cost structure. Once these pieces are developed, the user has a “Decision framework” to use in evaluating IT investment alternatives. This chart represents that concept. 

## OR

**Communicating Value to Customers and Stakeholders**

- Using digital LMR will prevent the catastrophic communications malfunctions and inefficiencies that cost lives in the aftermath of 9/11 in NYC. Digital LMR will accomplish this by...

- What will make an Appropriations Committee staff member or OMB care about an investment in digital Land Mobile Radio (LMR) equipment for public safety agencies across government?

## VMM Effective in Building “WINNING”OMB Exhibit 300s

**PMA Imperatives**
**Captures All Value Factors/Benefits**
**Analytic Rigor**
**Clarity**
**Completeness**
**Focus On Results**

**VMM Effective in Building “WINNING”OMB Exhibit 300s**

## Value

## The Value Factors...

**DIRECT USER (CUSTOMER) VALUE** Benefits directly realized by users or multiple user groups.  Users or customers will vary based on the type of initiative being assessed.  Users may include, but are not limited to, government employees, other government organizations, and citizens
**SOCIAL (NON-DIRECT USER/PUBLIC) VALUE **Benefits not related to direct users (e.g., society as a whole)
**GOVERNMENT OPERATIONAL / FOUNDATIONAL VALUE**Order of magnitude improvements realized in current government operations and processes and in laying the groundwork for future initiatives
**GOVERNMENT FINANCIAL VALUE**Financial benefit (e.g., cost savings, cost avoidance) realized by the government, including financial benefits received by the managing or sponsor agency as well as other federal agencies
**STRATEGIC / POLITICAL VALUE**Benefits that move an organization closer to achieving its strategic goals, the priorities established by the Executive Office of the President and congressional mandates

**Notes:**
0:15 to 0:40 – Value (Jerry Mechling Michele Kratky, Nancy McBain)
Slides 12-18

Michele Kratky (10 minutes)

Slide 13 - 14: Here's how we think about value with VMM. Five value factors, Measures

## L A Y E R  1  

**Definition**

**L A Y E R  1  **

**Project Value**

**Definition**

**Definition**

**Project Value**

**Definition**

**(Measures)**

**Definition**

**Direct User**
**(Customer)**

**Social**
**(Non-Direct User)**

**Government Financial **

**Government ****Operational/ ****Foundational **

**Strategic/ Political**

**L A Y E R  2  **

**Identifying and Defining Value**

## Structured Approach to Identifying and Defining Value Measures

|       Concise, Illustrative Name | Robust Reliable Service |  |  |
| --- | --- | --- | --- |
|        Brief Description | Service with:
Minimal or no disruptions
Consistent service regardless of normal fluctuations in demand
High fault tolerance with built-in redundancy
Adequate speed to meet business needs |  |  |
|  |  |  |  |
| Performance Metrics |  | Target | Scale (0-100) |
| Frequency of service disruptions |  | None | 0 disruptions = 100
1 disruption  = 95
4 disruptions = 60 |
| Length of service disruptions |  | ≤ 10 minutes | ≤ 10 min. = 90
11–14 min. = 60 |
| Is an executable Continuity of Operations plan in place (with a backup NOC) sufficient to pass annual certification? |  | Yes | No = 0 / Yes = 100 |
| Latency |  | ≤ 75 milliseconds  |  |

**1**

**2**

**4**

**3**

**The way measures are articulated can directly impact the way they are perceived and understood. The definition must consist of four parts:**

**Structured Approach to Identifying and Defining Value Measures**

**Notes:**
This chart reflects a defined Value Measure. There are 4 parts to a defined Value Measure. 1: A concise, illustrative name – in this example,”Robust Reliable Service” is used. 2: A Brief description – in this example bullet points read, “Service with minimal or no disruptions, consistent service regardless of normal fluctuations in demand, high fault tolerance with built in redundancy, and adequate speed to meet business needs. 3: Performance Metrics  - in this example, ‘Frequency of service disruptions”, “Length of service  disruptions”, “In an executable Continuity of Operations plan in place sufficient to pass annual certification?” and “Latency”. 4: Target and Scale. In this example targets are given to the previously mentioned Performance Metrics, ‘frequency of service disruptions = none”, “length of service  disruptions = less than or equal to 10 minutes”, “In an executable Continuity of Operations plan in place sufficient to pass annual certification? = yes” and “Latency = less than or equal to 75 milliseconds”. Those targets are then normalized across a 100 point scale, “frequency of service disruptions, 0 disruptions = 100; 1 disruption = 95; 4 disruptions = 60”, “length of service  disruptions, less than or equal to 10 minutes = 90; 11 to 14 minutes = 60”. Latency does not have a scale in this example.

## Building A Direct User Measure  What Do Users Want?

| E-Travel Initiative |
| --- |
| Travelers / Managers
Anytime Access to Data
Real-Time Data Availability 
Simplified/Automated Trip Planning 
Speed of Reimbursements 
Budget &  Accounting Staff 
Standardized and Electronic Auditing Function / Simplified Application and Monitoring of Travel Entitlements 
Access to Reporting Information 
Businesses (Travel Management Companies)
Timely & Accurate Receipt of Bill Payments |

## Analytical Hierarchy Process (AHP) tools are designed to help groups enhance the quality of their decisions.  These tools: 
Bring structure to the decision-making process;
Elicit ideas, feelings and the judgments of stakeholders;
Represent those judgments as meaningful numbers;
Synthesize the results; and
Analyze the sensitivity of those judgments to changes. 

- Bring structure to the decision-making process;
- Elicit ideas, feelings and the judgments of stakeholders;
- Represent those judgments as meaningful numbers;
- Synthesize the results; and
- Analyze the sensitivity of those judgments to changes.

**Through the use of pair-wise comparisons, the relative importance of each of the criteria is calculated**
**Attention is focused on areas of disagreement**

**Prioritizing Value Factors  - The Analytic Hierarchy Process**

## Summary of  VMM Weighting & Scoring for Title XVI “Check Your Benefits”

|  | Agency Direction | Program Mgt. Direction | Sub-Criteria |  |  |
| --- | --- | --- | --- | --- | --- |
| Value Factors and Value Measures(% share of total weight) | Maximum Value of the  Value Factors | Maximum Measure Value | Scoring (“Normalized”) | Sub-Criteria Points (%) | Value Score |
| Direct User | 25% |  | Scoring Specialists: OPB, OAS, OQA, OCOMM |  | 21.0% |
| Expanded Access (50%) |  | 12.5% | 10 of 10 | 12.5% |  |
| User Time Saved (30%) |  | 7.5% | 6 of 10 | 4.5% |  |
| Increased Satisfaction (20%) |  | 5.0% |   8 of 10 | 4.0% |  |
| Social | 15% |  | Scoring Specialists: OCOMM, OQA, OSM |  | 13.0% |
| Increase Public Confidence (33%) |  | 5.0% | 10 of 10 | 5.0% |  |
| Access for “Hard to Reach” (33%) |  | 5.0% |   8 of 10 | 4.0% |  |
| Equity and Fairness (33%) |  | 5.0% |   8 of 10 | 4.0% |  |
| Government Financial | 10% |  | Scoring Specialists: OB, DCS, OQA |  | 9.0% |
| Effectiveness and Efficiency  (50%) |  | 5.0% |   8 of 10 | 4.0% |  |
| Return on Investment (50%) |  | 5.0% | 10 of 10 | 5.0% |  |
| Operational/Foundational | 30% |  | Scoring Specialists: DCS, OES, OPB, OAS |  | 25.0% |
| Supports Future eService Transactions (50%) |  | 15.0% | 10 of 10 | 15.0% |  |
| Supports Transformation (33%) |  | 10.0% |   6 of 10 | 6.0% |  |
| Supports Organizational Learning (17%) |  | 5.0% |   8 of 10 | 4.0% |  |
| Strategic / Political | 20% |  | Scoring Specialists: OSM, OES |  | 18.0% |
| Satisfies External Mandates / Requirements (50%) |  | 10.0% | 10 of 10 | 10.0% |  |
| Supports ASP (50%) |  | 10.0% |   8 of 10 | 8.0% |  |
| TOTAL | 100% | 100% |  | 86% | 86% |

- Applying VMM to Title VXI “Check Your Benefits,” we determined the following scores for each of the Value Factors and their respective value measures

**Notes:**
This chart represents the calculation of a Value Scores for a Social Security Administration initiative, “Check your Benefits” which used VMM to analyze IT investment options. In the chart the value measures have been prioritized with in the 5 Value Factors which have also been prioritized. The individual value measures have been scored based on what is thought to be a  likely outcome of the individual measures. These scores are then applied to the weight of the measure and associated value factor to arrive at a value score. 

## Risk

## IDENTIFYING RISKS:
Consider “standard” IT project risks 
Identify project specific risks via input from technical & policy staff, representatives of partner agencies collected from:
Working Sessions
Survey Efforts

**Risks that are not mitigated can cause a project to fail either in the pursuit of funding or, more dramatically, while the project is being implemented.**

**IDENTIFYING RISKS:**
- Consider “standard” IT project risks
- Identify project specific risks via input from technical & policy staff, representatives of partner agencies collected from:
- Working Sessions
- Survey Efforts

**EXAMPLE OMB RISK CATEGORIES:**
- Project Resources / Financial
- Technical / Technology
- Business / Operational
- Organizational & Change Management
- Data / Information
- Security
- Strategic
- Privacy

**Identifying and Defining Risk**

## Organizational Tolerance For Cost Risk (increased cost)
Organizational Tolerance for Value Risk (slippage in performance)

**Organizational Tolerance For Cost Risk (increased cost)**
**Organizational Tolerance for Value Risk (slippage in performance)**

**What is the decision process****behind the following:**
- Buying a $1 lottery ticket for the chance to win $1 million. Odds are 1 in 1,000.
- Buying a $100 lottery ticket for the chance to win $1 million. Odds are 1 in 1,000.
- Buying a $100 lottery ticket for the chance to win $10 million. Odds are 1 in 1,000.

**Defining Risk Tolerance**

## As the estimated most likely value score increases, risk tolerance is likely to increase.

**As the estimated most likely value score increases, risk tolerance is likely to increase.**

**As the estimate most likely ****cost increases, risk tolerance****is likely to decrease.**

**Value and Cost Risk Tolerance Boundaries communicate the upper limit of the range of risk an organization will accept in both areas.  **

**Notes:**
Two charts represent the concept of a risk tolerance boundary. A risk tolerance boundary is a graphical representation of the tradeoffs an individual, organization, or agency is willing to make among cost and risk or value and risk. The horizontal axis represents Cost or Value Risk and the vertical axis represents Value Scores or Cost. Once defined and plotted, a Risk Tolerance Boundary allows individuals, agencies or organizations to define acceptable and unacceptable areas. IT alternatives or investments can then be plotted to determine in which area they fall or, how far outside the acceptable area they fall.  

## Cost

## Identifying & Defining Costs

**Consider Value and Risk**

**Identifying & Defining Costs**

| Direct User Value  | Training
Marketing
Access (e.g., kiosks)
Incentives |
| --- | --- |
| Social Value  | Communications 
Public awareness Advertising
Public Relations |
| Government Operational / Foundational Value  | Maintain legacy systems and processes during transitions
On-going maintenance of paper process  |

**Ensure a complete, comprehensive cost estimate ...**
** ****Alleviate the risk of missing costs or double-counting...   **
**by developing a **
***Cost Element Structure***
**Investments made on incomplete or inaccurate estimates are likely to run out of funding and, therefore, require justification for additional funding or a reduction of initiative scope  **

## Estimating and Comparing Value, Cost, & Risk

## Identifying and Defining Viable Alternatives

**Alternatives Must Address People, Process & Technology!**

- trainingoutreachmanagementstaffingcommunicationsrecruitmentsocializationuser support508 requirementslanguage requirementsEA / FEA

- PEOPLE

- PEOPLE

- PROCESS

- PROCESS

- BPRAcquisitionOutsourcing/in-sourcingconcept of operationsrisksecurityprogram managementfundingcollaborationcommunicationsevaluationlegislative req.policy req.EA / FEA

- TECHNOLOGY

- hardware
- software
- interface
- data req.
- EA / FEA

- TECHNOLOGY

**Identifying and Defining Viable Alternatives**

**Notes:**
This graphic represents three key elements of any system: people, process, and technology. It argues that that any IT Investment alternative must address all 3 elements.

## The Base Case

- Projects the results of maintaining current systems and processes while attempting to keep pace with changes over time.

**T          I          M          E**

- rising demand

- workforce
- attrition

- customer
- satisfaction

**status quo**

**base case**

**The Base Case**

## Collecting Data

- Data sources and detail depend upon the initiative’s stage of development
- Use the best information available rather than looking for information that doesn’t exist
- Update this information as “better” information becomes available
**ALWAYS DOCUMENT DATA SOURCES &  ASSUMPTIONS**

| Stage of Development | Data Sources |
| --- | --- |
| Strategic Planning | Strategic & Performance plans
Subject Matter Expert Input
New and existing user surveys
Private/public sector best practices, lessons learned and benchmarks
Enterprise Architecture
Modeling & simulation
Vendor / market survey |
| Business Modeling & Pilots | Subject Matter Expert input
Data from analogous government initiatives
New & existing user surveys for each business line
Private/public sector best practices, lessons learned & benchmarks
Refinement of modeling & simulation |
| Implementation & Evaluation | Actual data from phased implementation
Actual spending/cost data
User group / stakeholder focus groups / surveys
Other performance measurement |

**Collecting Data**

**Notes:**
This table in this slide has 2 columns: “Stage of Development” and “Data Sources”. Row one reads: “Stage of Development: Strategic Planning”, “Data Source: Strategic & performance plans; Subject matter expert input; New and existing user surveys; Private/public sector best practices, lessons learned and benchmarks; Enterprise Architecture; Modeling and simulation; Vendor/market survey”. Row two reads: “Stage of Development: Business Modeling and Pilots”, “Data Source: Subject matter expert input; Data from analogous government initiatives; New and existing user surveys for each business line; Private/public sector best practices, lessons learned and benchmarks; Refinement of modeling and simulation” Row three reads:”Stage of Development: Implementation and Evaluation”, “Data Sources: Actual data from phased implementation; Actual spending/cost data; User group/stakeholder focus groups/surveys; Other performance measurement”.

## EXAMPLE

**USE RANGES TO INCREASE CONFIDENCE IN COST ESTIMATES!**

- 200

- 150

- 100

- # of Employees to be
- Trained/year
- Annual Cost per Employee Trained

**High**

**Med**

**Low**

**Projected Range of Training Costs Inputs**

- $1500

- $1200

- $1000

**Inputs**

**Using Ranges**

**Notes:**
This chart provides an example of Low Medium and High values to be used in estimates. The two headings read: “Inputs” and “ Low, Medium, High”. Row one reads: “Input: # of employees to be trained/year”, “Low: 100, Medium: 150, High: 200”. Row two reads:  “Input: Annual Cost per Employee trained”, “Low: $1000, Medium: $1200, High: $1500”.

## Uncertainty Analysis 
Based on considerationsof requirement, cost estimating, and technical uncertainty
Increases confidence in the estimate. Doesn’t increase the precision of the estimate
Tool: Monte Carlo Simulation
Output: “Most Likely” or “Expected” Cost & Value

**Conduct Uncertainty and Sensitivity Analyses on Both Cost & Value Estimates**

- Uncertainty Analysis
- Based on considerationsof requirement, cost estimating, and technical uncertainty
- Increases confidence in the estimate. Doesn’t increase the precision of the estimate
- Tool: Monte Carlo Simulation
- Output: “Most Likely” or “Expected” Cost & Value

- Sensitivity Analysis
- Based on the output of the Monte Carlo Simulation
- Sensitive variables have a significant impact on the ***overall estimate***
- Output: Identification of which variables have a significant impact on the overall estimate. Can be used to determine which variables merit additional research

**Uncertainty and Sensitivity Analysis**

**Notes:**
This slides contains 2 boxes that describe Uncertainty Analysis and Sensitivity Analysis. “Uncertainty Analysis: Based on considerations of requirement, cost estimating, and technical uncertainty; Increases confidence in the estimate. Doesn't increase the precision of the estimate; Tool: Monte Carlo Simulation; Output: “Most Likely” or “Expected” cost and value”. “Sensitivity Analysis: Based on the output of the Monte Carlo Simulation; Sensitive variables have a significant impact on the overall estimate; Output: Identification of which variables have a significant impact on the overall estimate. Can be used to determine which variables merit additional research.”

## Analyzing Cost Risk and Value Risk

| ALTERNATIVE 1 - COST RISK ANALYSIS |  |  |  |
| --- | --- | --- | --- |
| Risk | Probability | Cost Impacted | Impact |
| Cost Overruns | Med | 1.0 System Planning & Development | Low |
|  |  | 2.0 System Acquisition & Imp. | High |
|  |  | 3.0 System Maintenance & Operations | Med |
| Cost of Lost Info / Data | High | 1.0 System Planning & Development | Med |
|  |  | 2.0 System Acquisition & Imp. | Med |
|  |  | 3.0 System Maintenance & Operations | Low |

| ALTERNATIVE 1 - VALUE RISK ANALYSIS |  |  |  |
| --- | --- | --- | --- |
| Risk | Probability | Value Impacted | Impact |
| Cost Overruns | Med | Total Cost Savings to Investment | Low |
|  |  | Total Cost Avoidance to Investment | Low |
| Cost of Lost Info/ Data | High | Total Cost Savings to Investment | Low |
|  |  | Total Cost Avoidance to Investment | Low |
| HW/ SWFailure & Replacement | Med | Accessibility of e-Gov services to Users | High |
|  |  | User Trust In Internet Transactions | High |
|  |  | Application Owner Confidence in Identity of Users | High |
|  |  | Reduction of Identity Fraud | High |
|  |  | Regulatory Compliance | High |
|  |  | Total Cost to Savings Investment | High |
|  |  | Total Cost Avoidance to Investment | High |

- The probability of a specific risk occurring remains constant through out the analysis of a specific alternative, regardless of where it impacts the value or cost of a particular alternative

- The impact of a single risk factor may differ in magnitude at each point where it interacts with cost and value

**Analyzing Cost Risk and Value Risk**

**Notes:**
This slide shows an illustrative example of applying risk to value and cost. It represents one possible way to link cells in spreadsheet to calculate the affects of identified risks on value and cost. The point is that for each identified risk, the probability of occurrence and impact on cost or value must be determined. This will allow for the calculation of risk adjusted cost and risk adjusted value in the context of VMM

## Pulling Together the Information

- What is the estimated cost of each alternative?
- What is the financial return on investment associated with the alternatives?
- What is the value score associated with the alternatives?
- What are the cost and value risks associated with this alternative? What effect do they have? (value and cost risk scores)
- How do the value, risk and cost of the alternatives compare?
- Does the cost risk and value risk associated with the alternatives fall within the range represented by the relevant risk tolerance boundaries?

**Pulling Together the Information**

## Comparing Value to Cost

**Investment Cost To Value**
**(Expected & Risk-Adjusted)**

- 0

- 10

- 20

- 30

- 40

- 50

- 60

- 70

- 80

- 90

- 100

- $-

- $5

- $10

- $15

- $20

- $25

- $30

- $35

- $40

**C O S T   ($M)**

**V A L U E**

- Expected Alt 1

- Risk Adjusted Alt 1

- Expected Alt 2

- Risk Adjusted Alt 2

- Expected Alt 3

- Risk Adjusted Alt 3

**Alt 1**

**Alt 2**

**Alt 3**

- Based on This Information, Which Alternative Would You Choose?

**Comparing Value to Cost**

**Notes:**
This chart is an example of the value and risk associated with 3 alternatives. 3 examples are plotted on a graph in which the vertical axis is Value and the horizontal axis is Cost. The examples are separate plots of the value and cost of each connected by an arrow. The example allows for the comparison of the three alternatives.

## Comparing Value to Value Risk,and Cost to Cost Risk

**The risk associated with all of the value scores fall within the acceptable area. ****Alt. 2 bears the lowest value risk.**** **

**The only alternative that falls squarely within the Cost Risk Boundary is Alt. 2.  **

**Comparing Value to Value Risk,****and Cost to Cost Risk**

**Notes:**
This slide contains the two risk tolerance charts that were previously explained in slide 22. In this slide examples have been plotted with their associated cost and value as described in slide 33, to show how the chart would be used. Some fall into the “Unacceptable Area” some fall in the “Acceptable Area”. The chart allows for decision makers to understand howthe investments compare to the defined risk tolerance boundary.

## The VMM Guide

## The VMM How-To Guide provides best practice analysis techniques, real examples and required resources

**Notes:**
This slides describes the nomenclature used in the VMM How to Guide. Throughout the How to Guide there are sections that present different perspectives on the material: “Key Concepts”, “Best Practices”, “VMM in Action”, “Summary”, and “Required Resources”. The slide shows the color scheme or format used to differentiate each section.

## VMM  Step 1: Develop a Decision Framework

| V A L U E | Value Factors |  | priority |
| --- | --- | --- | --- |
|  | value measure(s) | metric, target, scale | priority |
| R I S K |  |  |  |
| COST | CustomizedCost ElementStructure | 1.0 ... |  |
|  |  | 2.0...
 |  |
|  |  | 3.0 ... |  |

**Risk Inventory**
**Risk  Tolerance Boundary**

**Task 4:**Begin Documentation

**Task 2:**Identify & Define the Risk Structure

**Task 3: **Identify & Define the Cost Structure

**Task 1: **Identify & Define the Value Structure

**Define User ****Needs & Priorities**
**Quantifiable Measures of Performance (Metrics, Targets)**
**Foundation for Analysis & On-going Performance Measurement**
**Early Consideration ****of Risk**

**Notes:**
This Chart is a visual representation of the VMM Decision Framework with step1of the 4 step VMM process called out. The Decision framework revolves around 3 key elements of Value, Risk, and Cost. In the early steps of the VMM process, a value structure is developed then a risk structure as well as a cost structure. Once these pieces are developed, the user has a “Decision framework” to use in evaluating IT investment alternatives. Step 1 is “Develop a Decision Framework” and contains 4 tasks, which are listed on the slide:  Task 1: Identify and Define the Value Structure; Task 2: Identify and Define the Risk Structure; Task 3: Identify and Define the Cost Structure; Task 4: Begin Documentation.

## VMM  Step 2: Alternatives Analysis 		    (estimate value, cost, & risk)

**R I S K**

- priority

- priority

**3.0 ...**

**2.0...**

**V A L U E**

**COST**

**1.0 ...**

- metric, target, scale

- value measure(s)

**Value Factors**

**Customized****Cost Element****Structure**

**VMM  Step 2: Alternatives Analysis ****	****	****    *****(estimate value, cost, & risk)***

**Viable Alternatives**
**Base Case - ****What will happen if nothing changes?**
**Match levels****of information ****to the **
**phases of development**

**S E N S I T I V I T Y  **

**U N C E R T A I N T Y   **

**U N C E R T A I N T Y   **

**Risk Analysis**

**S E N S I T I V I T Y  **

** ****Step 1**

**Task 1:** 	Identify & Define Alternatives

**Task 4:** 	On-going Documentation

**Task 2:**Estimate Value & Cost

**Task 3:** Conduct Risk Analysis

**Risk Inventory**
**Risk Tolerance Boundary**

- Low    High Expected

- Low    High Expected

**Notes:**
This Chart is the same visual representation of the VMM Decision Framework as in slide 37 with step 2 of the 4 step VMM process called out.  Step 2  is “Alternatives Analysis” and contains 4 tasks which are: Task 1: Identify and Define Alternatives; Task 2 Estimate Value and Cost; Task 3 Conduct Risk Analysis; Task 4 On-going Documentation.

## VMM  Step 3: Pull Together the Information	

**VMM  Step 3: Pull Together the Information****	**

**Government Cost Savings/Avoidance**

**Expected****Value Score**

**Expected Cost**

***Risk Adjusted***** Expected****Value and Cost**

**Risk Scores**

**Expected ROI**

***Risk******Adjusted***** Expected ****ROI**

**S E N S I T I V I T Y  **

**U N C E R T A I N T Y   **

**U N C E R T A I N T Y   **

**S E N S I T I V I T Y  **

** ****Step 1**

- Low    High Expected

**R I S K**

- Low    High Expected

- priority

- priority

**3.0 ...**

**2.0...**

**V A L U E**

**COST**

**1.0 ...**

- metric, target, scale

- value measure(s)

**Value Factors**

**Customized****Cost Element****Structure**

**Risk Analysis**

** ****Step 2**

**Task 5: **	Compare Value, Risk, & Cost

**Task 1:**Aggregate the Cost Estimate

**Task 2: **	Calculate the Return-on-Investment

**Task 3: **	Calculate the Value Score

**Task 4: **	Calculate the Risk Scores

**Risk Inventory**
**Risk Tolerance Boundary**

**Notes:**
This Chart is the same visual representation of the VMM Decision Framework as in slide 37 with step 3 of the 4 step VMM process called out.  Step 3  is “Pull Together the Information” and contains 5 tasks which are: Task 1: Aggregate the Cost Estimate; Task 2 Calculate Return on Investment; Task 3: Calculate the Value Score; Task 4: Calculate Risk Scores; Task 5: Compare Value, Risk, and Cost.

## VMM  Step 4: Communicate and Document

**	****Reporting****	****	****Consensus Building****	**
**	****	****Investment Planning   ****	****Management Planning**
**	****	****	**

**Expected****Value Score**

**Expected Cost**

***Risk Adjusted***** Expected****Value and Cost**

**Risk Scores**

**Expected ROI**

***Risk******Adjusted***** Expected ****ROI**

**Government Cost Savings/Avoidance**

**S E N S I T I V I T Y  **

**U N C E R T A I N T Y   **

**U N C E R T A I N T Y   **

**Risk Inventory**

**S E N S I T I V I T Y  **

** ****Step 1**

- Low    High Expected

**R I S K**

- Low    High Expected

- priority

- priority

**3.0 ...**

**2.0...**

**V A L U E**

**COST**

**1.0 ...**

- metric, target, scale

- value measure(s)

**Value Factors**

**Customized****Cost Element****Structure**

**Risk Analysis**

** ****Step 2**

** ****Step 3**

**Task 2:**
- Prepare Budget Justification Documents

**Task 3:**
- Satisfy Ad Hoc Reporting Requirements

**Task 4:**
- Use Lessons Learned to Improve Processes

**Task 1: **
- Communicate Value to Customers and Stakeholders

**Risk Inventory**
**Risk Tolerance Boundary**

**Notes:**
This Chart is the same visual representation of the VMM Decision Framework as in slide 37 with step 4 of the 4 step VMM process called out.  Step 3  is “Pull Together the Information” and contains 5 tasks which are: Task 1: Aggregate the Cost Estimate; Task 2 Calculate Return on Investment; Task 3: Calculate the Value Score; Task 4: Calculate Risk Scores; Task 5: Compare Value, Risk, and Cost.

## Q & A

## VMM establishes an even scale for quantifying and analyzing value, risk, and cost

- Measures tangible and intangible benefits
- Accounts for risk in cost and value calculations
- Increases reliability of ROI through simulation
- Tested and proven in multiple E-Gov projects
- Flexible and adaptable
- Results and outcome driven
- Allows examination of the relationships among Value, Cost and Risk
- Feasible for portfolio management

**VMM establishes an even scale for quantifying and analyzing value, risk, and cost**

| V A L U E | Value Factors |  | priority |
| --- | --- | --- | --- |
|  | value measure(s) | metric, target, scale | priority |
| R I S K |  |  |  |
| COST | CustomizedCost ElementStructure | 1.0 ... |  |
|  |  | 2.0...
 |  |
|  |  | 3.0 ... |  |

**Risk Inventory**
**Risk  Tolerance Boundary**

## Building a Methodology for Measuring the Value of e-Services

- http://www.estrategy.gov/documents/measuring_finalreport.pdf
***VMM How-To-Guide ***and ***VMM Highlights***
- http://www.cio.gov/   –  best practices page:
- http://www.cio.gov/documents/ValueMeasuring_Methodology_HowToGuide_Oct_2002.pdf
- http://www.cio.gov/documents/ValueMeasuring_Highlights_Oct_2002.pdf