DRAFT

PROSPECTUS FOR EVALUATING TRADEOFF AND DECISION SUPPORT METHODS FOR THE GCDAMP

GCDAMP SCIENCE ADVISORS  

DECEMBER, 2010

SCIENCE ADVISOR CHARGE

The Technical Work Group (TWG) and the Science Advisors (SAs) have for several years

discussed the need to consistently incorporate more objective based approaches in evaluating tradeoffs

of various science and management programs.  At the TWG 2010 summer meeting it was proposed to

the Adaptive Management Work Group (AMWG) that the Science Advisors be charged to evaluate

decision support methods that could be utilized by the GCDAMP, especially the TWG.   The AMWG at

their fall 2010 meeting charged the SAs to conduct the assessment.   

BACKGROUND

Over the past decade the GCDAMP has used decision support methods to evaluate alternative

science and management direction.  These have ranged from simplistic consensus and voting

approaches to use of analytic preference tracking procedures to better inform the consensus and voting

process. TWG and AMWG have also utilized models such as ECOSIM (MATA, Ecosystem, SPG

Workshops) to characterize biophysical tradeoffs occurring in differing science and management

approaches.  Less use has been made of decision support methods which incorporate treatment of

tradeoffs, costs, benefits, risks, etc.

   Generally, decision making methods incorporate a mental cognitive process resulting in the

selection of a course of action among several alternatives.  It should be made clear that in these

processes, use of more complex analytical methods do not necessarily insure better decisions.  In fact,

for large groups such as TWG and AMWG significant continued investment is needed in effective

collaborative processes to permit application of more complex decision processes.  This can prevent

concerns by individuals that important information is being glossed over in the complex models.  

Further, and probably just as important, stakeholders sometimes prefer much less complicated

information sets and models for use in decision processes.   More simplistic models that are populated

with accurate and effective information can often provide effective decision outcomes on many issues.

Decision processes are a necessary part of life and are common to all aspects of living.  Examples

include deciding what car to buy, what college to attend, and what appropriate course of actions are

necessary to get to the moon or bring GM back from bankruptcy.  Some individuals have unique

capabilities to request and process quickly the most important information sets necessary to arrive at

multiple decision points required in a complex program.  Others may struggle over the most simple

1
decision, often unable to correctly formulate the question or issue needing resolve, a basic first step in

the process.    

    The disciplines of “decision making” and “decision methods” have been developed extensively

in science and management to assist the process, using both qualitative and quantitative  approaches

that range from simplistic to complex.  Most often the decision process must rely on a blend of

qualitative and quantitative approaches simply because complete knowledge never exists on all aspects

of an issue.  This is especially true of more complex problems with high uncertainty such as encountered

in GCDAMP.  

Qualitative decision approaches, especially more simplistic methods, have significant appeal

because they are more easily structured, understood and accepted by groups.   However, often even

they require quantitative scales and matrices with assigned numeric values to permit assessment such

as the following example.

SCALE PROBABILITY RELATIVE

GOODNESS

AGREEMENT FREQUENCY   IMPORTANCE QUALITY

1 Very high   Much better Strongly

agree

Always Very

important Excellent

2 Medium Same as       Not decided Occasionally Neutral Average

3 Low Worse than Strongly

disagree

Never Not

important

Poor

Quantitative approaches to assist decision making has had continued  support  from the science

and management communities since the 1960s.  Development of academic disciplines such as systems

theory and operations research advanced concepts such as Program which have evolved today into very

articulate and often complex Decision Support Systems (DSS).

CHARACTERIZING APPROPRIATE AND EFFECTIVE DECISION METHODS FOR THE GCDAMP   

In the GCDAMP both TWG and AMWG have sufficient science and technical support to evaluate

the application of the full range of decision processes.  As noted above, over time both very simplistic

and very complex information sets and decision methods have been utilized in tradeoff and decision

assessments by the TWG, albeit not consistently.    Effective decision support methods could therefore

be selected for evaluation from the full range of simplistic and complex approaches in existence.   

The TWG conducts most of the in depth assessments of science and management alternatives

for the AMP.  The TWG then would be the primary focus of the SAs in this assessment. However, the

AMWG may also have a need for application of more qualitative decision methods.  

In accomplishing its tasks the TWG is challenged constantly by time constraints resulting from its

work load, but possibly because it also does not use decision methods in a consistent manner.  Because

of this challenge, and also the wealth of qualitative decision methods, the SAs recommend that its

evaluations in 2011 be focused on more qualitative methods and models.  This does not mean that

2
quantitative tradeoff techniques, such as models like ECOSIM will not be incorporated, they will.

However, initially it may be more productive to evaluate 2‐4 approaches that represent a more

qualitative approach overall.

Even though more qualitative methods are proposed in this first year evaluation, these methods

will be expected to respond to selection criteria that define more robust decision methods.  For

example, developed screening criteria will require that methods chosen for evaluation incorporate

capabilities for assessment of cost and benefits, uncertainty, risk, comparative tradeoffs, etc.  

If the process proves effective and the TWG makes more consistent use of decision

methodology it is proposed that the SAs evaluate 1‐3 more quantitative decision support systems. This

two step approach might best support the TWG and AMWG in their joint deliberations on science and

management needs over the next five years, and the current expected timeline of AMP

accomplishments.    For example, the assessments may demonstrate that more qualitative approaches

would best benefit AMWG decision processes whereas more quantitative methods may be appropriate

to the TWG.

ASSESSMENT APPROACH

Decision support methods exist and are used at some level in most management and science

disciplines.  The area of Business Management has provided much of the leadership over time on this

issue. Yet, the area of Natural Resource Conservation and Management, especially since the 1960s, has

developed excellent methods that demonstrate capabilities to incorporate environmental cost and

benefits, risk assessment, uncertainty analysis, etc.  

The assessment approach used will be Literature Review and Case Study Evaluation.  The

following tasks will be performed between January, 2011 and August, 2011.  

• A review of literature that provides overviews of development of decision analysis

methodology; qualitative and quantitative methods; appropriate applications of differing

qualitative methods, etc.

• Develop criteria for coarse screening and selection of 4‐8 qualitative decision support methods

in current use (past five years) in natural resource management and business.  Conduct

screening and selection.   

• Refine criteria for fine screening and selection of 2‐4 methods for in depth evaluation for

application as a GCDAMP decision support tool.   

• Evaluate candidate methods potential application to GCDAMP needs using hypothetical case

evaluations.  

• Develop report for TWG and AMWG.

Presentations to TWG and AMWG and discussions of methodologies with members will depend on

AMP schedules and available resources.   Recommendations to AMWG may require a short workshop

with TWG.    

3
4

SCHEDULE AND RESOURCE AVAILABILITY

All project tasks and activities for completion of the final report are proposed to be conducted

during the Science Advisors contract year, i.e., January to January.  Expansion of the project assignment

or multiple presentations and or a workshop could require the SA Coordinator to adjust completion

schedules or budgets to accommodate the  $20,000 assigned to the project.  If this schedule needs to be

revised to accommodate the TWG, the SAs will attempt to respond with an alternate schedule.

   

   

    

   
AMP SCIENCE ADVISORS

MARCH , 2011 TWG MEETING
 QUALATATIVE APPROACHES

 QUANTATATIVE METHODS
 SIMPLISTIC METHODS AND MODELS

 EASILY UNDERSTOOD AND APPLIED

 LOW USER COST AND TIME INVESTMENT

 LIMITED USE OF COMPLEX ASSESSMENTS

 CONSTRAINED TO MORE COARSE 

ANALYSIS

 USEFUL FOR REAL TIME APPLICATIIONS
 COMPLEX METHODS AND MODELS

 MORE DIFFICULT TO UNDERSTAND 

SYSTEM DETAIL

 REQUIRES ANALYSTS TO OPERATE

 HIGH DEVELOPMENT COST

 SUMMARY OUTPUTS USEFUL IN MORE 

SIMPLISTIC MODELS

 USEFUL FOR COMPLEX ASSESSMENTS 

AND MICRO-ANALYSIS

 MORE LIMITED REAL-TIME APPLICATION
 INTEGRATED ASSESSMENTS OF GOAL 

OUTCOMES

 COST ASSESSMENTS

 BENEFIT ASSESSMENTS

 ASSESSMENT OF RISK

 EVALUATION OF LEARNING AND 

UNCERTAINTY

 TRADEOFF ANALYSIS

 EASE OF USE AND UNDERSTANDING
 LITERATURE AND USER REVIEW

 CRITERIA FOR COARSE SCREENING, 

SELECT 4-8 METHODS IN CURRENT USE 

 REFINE EVALUATION CRITERIA AND 

SELECT 2-4 METHODS FOR ANALYSIS

 EVALUATE APPLICATION TO AMP 

 FINAL REPORT TO TWG : SUMMER 2011 
 DEVELOP A GENERAL CORE MONITORING 

PLAN

 CONDUCT INFORMATION NEEDS 

WORKSHOPS WITH TWG

 CONDUCT PEPS FOR EACH RESOURCE 

GOAL

 DEVELOP INDIVIDUAL RESOURCE GOAL 

CMPs
 MANAGER AND STAKEHOLDER 

GUIDELINES FOR SPECIFYING DFCs; 

GOALS; ARTICULATING PRIORITY 

INFORMATION NEEDS; LEVELS OF 

NEEDED RESOULUTION IN DATA AND 

ACCURACY FROM ANALYSES; RESOURCE 

IMPACT PROJECTIONS; TRADEOFFS; 

LEVELS OF PROJECTIED RISK; ETC., NEED 

TO BE INCLUDED IN THIS 

DOCUMENTATION .
 AMWG PRIORTY 

 MOs AND CMINs

 COMPLIANCE 

 LEGACY

 DATA QUALITY AND AVAILABILITY

 COST/BENEFIT AND RISK ASSESSMENT 

 STATUS OF KNOWLEDGE 

 METHODOLOGY