# Presentation: 322609

## NOAA Observing System Portfolio Management

- A Decision Analysis Approach

## Background

- NOSA tasking:
- Develop observing system baseline architecture
- Develop observing system target architecture
- Base prospective observing systems on validated requirements
- Investigate system consolidation and ownership transition
- Conduct a systemic review of all observing systems

## Purpose

- Introduce a study based on the observing system portfolio management approach to help:
- Develop observing system target architecture
- Investigate system consolidation and ownership transition
- Conduct a systemic review of all observing systems

## The Portfolio Question

- The question *“How valuable are NOAA’s observing systems?”* decomposes into two:
*How effective are NOAA’s observing system projects?*
*What are the costs of NOAA’s existing and candidate observing system projects?*

## The Effectiveness Question

- Effectiveness should be measured
- Measurement should reference NOAA mission goals
- Question becomes:* ****To what extent do NOAA’s existing and candidate observing system projects contribute to NOAA mission goals?***

## Measuring Contribution to NOAA Mission Goals

- Methodology based on Analytical Hierarchical Process (AHP)
- Structured, traceable decision-making process
- Used by USDA, DOE, HUD, VA, FAA, SSA
- Cited by CIO Council as an important decision support methodology

## Methodology Outline

- Link observing system projects to NOAA mission goals
- Determine relative weights of NOAA mission goals
- Determine observing system projects’ relative contributions to NOAA mission goals
- Synthesize (2) and (3) to determine relative values of observing systems
- Introduce cost; derive cost-effectiveness comparison
- Generate improvement possibilities

## 1. Linking Observing Systems to Goals

- Example linking:

## 2. Determining Weights for Goals

- Present a series of pairwise comparisons to senior leadership:

**Equal**

**Slight**

**Slight**

**Moderate**

**Moderate**

**Large**

**Large**

**Extreme**

**Extreme**

**Understand Climate**

**Weather Water Info**

- Which goal is more important to realizing NOAA’s mission to “understand predict changes in the Earth’s environment and conserve and manage coastal and marine resources to meet our Nation’s economic, social, and environmental needs?”

- Derive weights from comparisons:

## 3. Determine Observing Systems’ Contribution to Mission Goals

- Present a series of pairwise comparisons to goal teams:

- Derive weights from comparisons:

**Equal**

**Slight**

**Slight**

**Moderate**

**Moderate**

**Large**

**Large**

**Extreme**

**Extreme**

**Fish    LIDAR**

**Marine Optical Buoy**

- Which observing system contributes more to understanding the climate?

## 4. Synthesize

- Apply goal weights to observing system weights
- Obtain relative values of observing systems to NOAA

## 5. Introducing the Cost Question

- Cost (pv)

- Effectiveness

- Low Cost / High Effectiveness

- Low Cost / Low Effectiveness

- High Cost / Low Effectiveness

- High Cost / High Effectiveness

**+**  -  existing project

**x**  -  candidate project

**+ **

**+ **

**+ **

**+ **

**+ **

**+ **

**+ **

**+ **

**+ **

**x**

**x**

**x**

**x**

**x**

**+ **

**+ **

- 100%

- 100%

- 0%

- 0%

## 6. Improvements - Phase A

- Resource allocation
- Select cost-effective observing systems
- Knapsack budgeting: given finite funds what is the best mix of existing and candidate observing system projects?

## Phase A: Best Mix of Observing Systems

- Cost (pv)

- Effectiveness

- Low Cost / High Effectiveness

- Low Cost / Low Effectiveness

- High Cost / Low Effectiveness

- High Cost / High Effectiveness

**+**  -  existing project

- x  -  candidate project

**+ **

**+ **

**+ **

**+ **

**+ **

**+ **

**+ **

**+ **

**+ **

- x

- x

- x

- x

- x

**+ **

**+ **

- 100%

- 100%

- 0%

- 0%

##  6. Improvements - Phase B

- Internal improvements
- Increase effectiveness, and or
- Cut costs

## Phase B: Internal Improvements

- Cost (pv)

- Effectiveness

- Low Cost / High Effectiveness

- Low Cost / Low Effectiveness

- High Cost / Low Effectiveness

- High Cost / High Effectiveness

**+**  -  existing project

- x  -  candidate project

**+ **

**+ **

**+ **

**+ **

**+ **

**+ **

**+ **

**+ **

**+ **

- x

- x

- x

- x

- x

**+ **

**+ **

- 100%

- 100%

- 0%

- 0%

**Lower Costs**

**Increase Effectiveness**

**+ **

## Phase B: New Mix of Observing Systems

- Cost (pv)

- Effectiveness

- Low Cost / High Effectiveness

- Low Cost / Low Effectiveness

- High Cost / Low Effectiveness

- High Cost / High Effectiveness

**+**  -  existing project

- x  -  candidate project

**+ **

**+ **

**+ **

**+ **

**+ **

**+ **

**+ **

**+ **

- x

- x

- x

- x

- x

**+ **

**+ **

- 100%

- 100%

- 0%

- 0%

**+ **

**+**** **

## Study Execution and Timeline

## Back-up Slides

## AHP and Expert Choice

- T. L. Saaty, *The Analytic Hierarchy Process: Planning, Priority Setting, Resource Allocation*, McGraw-Hill, New York, 1980
- Software: Expert Choice 2000 for Groups
- Price: ?