GCDAMP Multi-Attribute Evaluation 

Multi-Attribute Evaluation of 

Experimental Options

GCDAMP Technical Working Group

Feb 2 2004
GCDAMP Multi-Attribute Evaluation 

Meeting Objectives

Purpose

– Identify and evaluate experimental optio

Background

– Build on December MATA workshop resu
GCDAMP Multi-Attribute Evaluation 

Agenda

9:00 Introduction and summary of MATA workshop9:30 Approach to evaluating experimental options

10:00 Define options and evaluation criteria12:00 Lunch

1:00 Evaluate experimental options4:00 Summarize areas of agreement and next steps4:30 Close
GCDAMP Multi-Attribute Evaluation 

MATA Workshop: 

Purpose

Develop a framework for decision making

– Address multiple objectives– Address trade-offs– Address uncertainties

Gain insight into priority experimental opti
GCDAMP Multi-Attribute Evaluation 

MATA Workshop 

Context: Sequenced Decision Makin

Develop Evaluation Framework

Decide: Adopt or experiment? Which experimen

Conduct experiments

Decide to adopt or continue 

Stop .
GCDAMP Multi-Attribute Evaluation 

MATA Workshop 

Tasks

Define endpoints and attributesIdentify optionsDevelop consequence tableEvaluate/rank optionsDiscuss implications for experimental priori
GCDAMP Multi-Attribute Evaluation 

MATA Workshop 

Results

Results of Workshop

– Good progress on consequence table– Good understanding of the decision cont– Beginnings of deliberative discussions– General agreement (?) that the MATA 

framework is a useful approach 
GCDAMP Multi-Attribute Evaluation 

MATA Workshop 

Results 

Time limitations meant:

– Limited review of consequence table– No opportunity to refine the options– Little opportunity to reflect and deliberate (either te

or value based issues) 

– Possibility of misinterpretation

• Options, the attributes, the scores, the ranking/weight

instructions

USE CAUTION IN INTERPRETING RESULTS

The draft report provides ideas about how you coul

process to gain insights rather than definitive r
GCDAMP Multi-Attribute Evaluation 

MATA Workshop 

Some Key Findings

Eliminate

– MLFF– SASF

Include

– MECH– BHBF

Candidates for testing

– Power, FSF, TCD
GCDAMP Multi-Attribute Evaluation 

Approach to Experimental 

Design and Evaluation
GCDAMP Multi-Attribute Evaluation 

Sequenced Decision Making

Develop Evaluation Framework

Decide: Adopt or experiment? Which experimen

Conduct experiments

Decide to adopt or continue 

Stop .
GCDAMP Multi-Attribute Evaluation 

Experimental Design 

Experimental Continuum

1. Adopt

– No uncertainty, or clear winner rega

uncertainty

2. Adopt, Monitor, Review

– Clear winner with uncertainty

3. Sequential Experiment (Titration)

– Several candidates, logical order, ad

success

4. Concurrent Experiment (Factorial)

– Several candidates, commit to full 

experimental period

Fixed

Adaptive
GCDAMP Multi-Attribute Evaluation 

Experimental Design 

Experimental Options

Water 

Year

Adopt and 

Monitor

Sequential 

(Titration)

Concurrent 

(Factorial) 

Fine

200520062007200820092010201120122013201420152016201720182019
GCDAMP Multi-Attribute Evaluation 

Experimental Design

Questions for TWG to Address

Adopt or experiment?Long term plan; staged commitment?Which experimental design: titration or facWhich treatments? Sequence? Timing?Monitoring: short or long response variablesWhat milestones and process for review?

Purpose: To resolve technical uncertainties; 

resolve value differences
GCDAMP Multi-Attribute Evaluation 

Experimental Design

Candidate Treatments

Candidates for Testing

– POWER– MECH– BHBF– FSF– TCD– Other?
GCDAMP Multi-Attribute Evaluation 

Experimental Design

Tasks

Define candidate treatmentsDevelop alternative experimental programsDefine criteria for evaluating programsEvaluateAreas of agreementNext steps