REVIEW OF TWG DEVELOPED APPENDIX AND RELATED CHANGES TO THE GCMRC DRAFT CORE

MONITORING PLAN

L. D. Garrett, Executive Coordinator Science Advisors

November 5, 2010

SCIENCE ADVISOR BACKGROUND RELATING TO THE CMP

The need to develop a science based cost effective CMP that is embraced by stakeholders and

scientists has been a goal of the GCDAMP since inception in 1996.  Completion of effective plans for this

critical program is long overdue and is a primary goal of the GCMRC and TWG for FY‐2011.

The Science Advisors (SAs) have been requested to review four different drafts of CMP plans

since 2003, including drafts in 6/2004, 11/2004, 4/2005 and 9/2009.  The SAs were also engaged in

assisting the GCDAMP in the 2005/2006 Science Planning Group that developed several criteria to assist

in refining the CMP.   

Following are general statements that capture some of the observations by the SAs on previous

drafts of the CMP.  General and specific positions of SAs in past reviews are also the basis for other

review comments of the SA Executive Coordinator in this document.   

SA general comments on the 6/2004 CMP draft was that it lacked appropriate science

justification for several areas of monitoring and selected individual resource impacts from management

activities.  The SAs requested expanded documentation to respond to critical science questions, define

current levels of knowledge, establish science based relationships, assumed and validated resource

interactions, and most critical science questions.   

In March 2005 the GCMRC and the TWG Core Monitoring Team requested coarse review input

on a draft Provisional Core monitoring Plan and a new CMP process outlined in chapter 1 of the plan.  

The new process would be used to develop a new version of the CMP by 2008.  The five day review

clarified several areas of needed improvements including the following.

• A unifying ecosystem approach for the monitoring program still needs to be developed.  Further,

when developed it must be woven into the complete fabric of the plan and not just referenced

in the introduction.

• The CMP must address priority information needs of managers and stakeholders, the most

recent developed in 2004.

• Clear linkages of what is known, needed to be known and how the CMP will improve on both is

not developed well for each goal area.

• Proposed analysis approaches are not sufficiently developed.

• The proposed new process for development of a new CMP was applauded by the SAs for

addressing several problems in the existing draft.

The General Core Monitoring Plan provided to the SAs for review in 9/2009 was based in part on

the new CMP process provided in the 2005 draft Provisional CMP.  The approach of first developing and
approving a general plan strategy that would be used to guide development and approval of individual

resource CMPs was supported by the SAs as the approach that should be pursued.  Several

recommendations were provided including the following.  

• As possible, dfcs or best available surrogates should be used to guide general and individual

resource CMP development.

• Stakeholder groups and managers proposed management directions and activities, etc need to

be more prominent in the CMP program.  Manager and stakeholder guidelines for specifying

dfcs;  goals; articulating priority information needs; levels of needed resolution in data and  

accuracy from analyses; needed resource impact projections, tradeoffs, levels of projected risk,

etc need to be included in this documentation.   

• Well defined science and management questions are needed to clearly link management needs

and science direction.

• Articulation of how science and management program integration is accomplished in both data

development and analysis needs to be improved.

It is important to note that the 2005/2006 activities of the Science Planning Group of managers,

stakeholders and scientists provided similar recommendations in several developmental areas of the

CMP.  The SAs have also raised several of these points in reviews of annual and biannual workplans,  

strategic plans, etc.

EXECUTIVE COORDINATOR COMMENTS ON PROPOSED TWG CHANGES TO DRAFT CMP

The comments in this document by the SA Executive Coordinator are provided on request from

the TWG Chair on sections of chapter 1 and the new proposed Appendix B of the 2010 Draft CMP.  

These comments of the Executive Coordinator do not supplant or displace any existing positions taken

by the SAs in previous reviews.  To this point, many previous comments of SA reviews are reiterated in

this document.

The EC and the SAs support the approach outlined in the draft CMP.  The EC also understands

concerns that have been expressed by the TWG Chair and the general TWG  in their reviews and past

meeting discussions of the draft CMP.  These concerns have resulted in rewrites of sections of the CMP

by the TWG Chair and TWG members.  The most recent TWG recommendations for revisions of the CMP

are the subject addressed by the EC in this document.

Following are the ECs general comments on proposed TWG changes to and additions to the

CMP.  More specific comments are provided in following sections.   

• In recent SA reviews of other planning documents of the GCMRC the SAs have proposed

development of planning documents that better express needed integration of management

and science activities of the GCDAMP.  This perspective applies to the current  document .  For

example, the SAs have proposed that the Annual and now Biannual Monitoring and Research

Program plans and budgets should have greater presence of planning and implementation of

management actions, risk assessments, program decision alternatives, etc., and their

appropriate integration with science programs.  And, in the recent reviews of the Non‐Native

Fish Control Plan drafts, the SAs proposed that improved leadership was needed by managers to

assure required balance of management and science activities.   
• Generally, the draft CMP can be improved by the recommended TWG proposals for change in

chapter 1 and in the proposed new appendix.  Manager and stakeholder guidance in a general

monitoring strategy document are important in many areas including addressing what goals

should be emphasized, what information needs are most critical, levels of resolution needed in

information, accuracy levels acceptable to the, manager, degree of risk acceptable to managers,

tradeoffs of knowledge, risk , benefits, costs, etc. Needed recommendations are now being  

provided by the TWG in many of these areas. However, these recommendations must be refined

and given greater specificity to become effective guidelines and criteria that will guide and

structure the actual CMP plans.  

• With the integration of management guidelines and criteria the General CMP appears more as a

strategic document.  Possibly GCMRC and TWG might want to call it a Core Monitoring Strategy.

• This strategy should reflect the importance of science/management interaction and integration

in all approaches for developing the individual resource CMPs.  

• The plan proposes that once individual resource plans are designed and implemented, they

should be free of political and budget pressures for change.  TWG and GCMRC should provide

more specific guidelines that address conditions under which change could be addressed and

protocols to follow in such cases.

A RECOMMENDATION FOR REVISION OF APPENDIX B

The SA Executive Coordinator recommends a rewrite of Appendix B to create specific focus on

protocols, guidelines, criteria and processes for review, evaluation, development and implementation of

the General CMP Strategy and Individual Resource CMPs.  This does not mean that the information in

the current Appendix is not critical to the CMP process.  In fact all of the information in the current

Appendix B is critical to resolving the most appropriate direction for the TWG and GCMRC to take.  And,

all sections of the original text are proposed for inclusion in the revised text.  The change will insure that

the revised text contains only the critical elements of the original text to gain needed focus on the

established task.  The changes are as follows.

• It is proposed that the extensive documentation of the accomplishments of previous work on

the CMP by the CMT, AMWG, GCMRC, TWG, SPG, SAs and others be deleted.   What this critical

documentation provides is the clarification and justification for the TWG AD Hoc to select the

protocols, processes, guidelines and criteria it has chosen to guide development of the final Core

Monitoring Program.  This contributed research by the TWG Chair should be retained, perhaps

rewritten into a white paper by the Chair on AM processes.  However, for this document pages

2‐7 detract from the critical mission to succinctly define criteria and processes to accomplish the

task at hand.  

• Proposed orderly and more specific definition of protocols, processes, guidelines and criteria for

evaluation, development and implementation of the Core Monitoring Program (CMP).  Appendix

B is now a critical component of the CMP.  It is a rudder from the manager/stakeholder group

that provides critically needed social guidance to the planning process that includes critical

normative assessments.  Additional specificity is needed to strengthen definitions.

The Science Advisors Executive Coordinator Recommends the rewrite of Appendix B use the

following general outline.
1.0 INTRODUCTION

Include all text on page one.  Editing could shorten text to three paragraphs.

2.0 BACKGROUND

Reduce pages 2‐6 to 1‐1 1⁄2 pages.  Mention briefly the processes but focus on the critical

outcomes of these processes that the TWG has decided to utilize as protocols, processes,

guidelines and criteria to guide the CMP development.  This includes: AMWG priority goals as

revised by SPG;  Ranked CMINS of CMT as revised by SPG;  CMT evaluation criteria as revised by

TWG Ad Hoc; CMT budget range for CMP as revised by TWG Ad Hoc; TWG Ad Hoc DSM and

tradeoff processes;  TWG and GCMRC collaborative development and evaluation procedures by

TWG Ad Hoc and GCMRC.

3.0 CASTING ALTERNATIVE PROGRAM OPTIONS

The TWG needs to provide some definition and guidelines for specifying the three different

program levels they are recommending.  At the minimum they should set the maximum budget

level for each of the three alternatives and specify general guidelines for development, i.e. they

will be developed collaboratively with GCMRC; they must address at least priority goals and

CMINs; meet specified minimal standards for accuracy; integrate with ongoing management and

research, etc.  The text does need rational for approaches taken but does not need extensive

justification for selected protocols and guidelines.  However the protocols must exist to guide

the process.

4.0 PROPOSED STRUCTURED EVALUATION AND DEVELOPMENT PROCESS FOR CMP

The first part of this section is absolutely critical and should state in chronological sequence the

Collaborative steps that will be taken by TWG and GCMRC to accomplish the General CMP

strategy in 2011 and implement at least one individual resource review.  A quarterly schedule

seems most realistic, i.e., Nov‐ Feb, Mar‐June, etc. Simple but comprehensive bullet statements

of projected accomplishments are all that are necessary in each quarter, i.e., “TWG AD Hoc and

GCMRC complete specification of three alternative CMP program levels”,  “TWG Ad Hoc and

TWG approve revised General CMP Strategy”, “TWG Ad Hoc initiates evaluation of  Lake Powell

Resource CMP”, etc.  Both GCMRC and TWG must be committed to accomplishment of the

listed actions, not just their discussion.  What is critical in this section is the recognition by all

participants of the GCDAMP that the past 14 years is not an acceptable timeline for any

government agency and stakeholder group to develop a Core Monitoring Plan. One quickly

realizes the necessary interdependence of TWG and GCMRC to create timely outcomes for the

program to progress.   

The second part of this section is the nuts and bolts of additional needed specification,

clarification, definition,  etc of adopted protocols, processes, guidelines, criteria,  etc. to

develop, evaluate and implement the General CMP Strategy and Individual Resource CMPs.  The

focus of this part should be in providing additional specification of the 10 evaluation criteria and

the six steps of the SDM process TWG has chosen as its evaluation tool.   

Input on the 10 TWG selected criteria and the 6 SDM steps in this second part are provided in

the following to assist TWG in their efforts to provide more clarification to each.  Most of the 10 criteria

have been part of the CMT process in developing the CMP for half a decade.   
• AMWG Priority Goals:  For both GCMRC and TWG, the concern for this criteria is not just that

one can recognize the AMWG priority is being addressed, but how, i.e.,  is it a set of multiple

studies,  and over what time period will the results of this project provide outcomes to resolve

the goal.  Statements use general assessments of “ direct or indirect effects” which are relevant

but fail to provide guidance of needed sequential accomplishment over time.   

• MOs and CMINs:  The TWG has established the criteria for priority goals and information needs

for the CMP in Appendix A.  However many of the CMINs can only be resolved by multiple years

of data collection and assessments, including modeling.  The GCMRC criteria provide a sequence

that should force the scientist to evaluate the proposed monitoring project and the sequence of

follow‐up monitoring and assessments to resolve the task at hand.  However, from a science and

management perspective and to benefit C/B, tradeoff and decision analysis, the scientist needs

to provide at least a general overview of the monitoring and assessment process through time,

with both number and type of assessments and resource needs.  Otherwise, the TWG is simply

evaluating a list of monitoring and assessments for 2011/2012, with limited knowledge of the

science assessments needed to gain final resolve.  For example suppose ground based

vegetation monitoring surveys to determine when critical tamarisk habitat for flycatcher is at a

critical point in the canyon for new introductions (90% confidence level) would take six years for

monitoring and two years for developing a prediction model at a total cost of 6x.  An alternative

approach using remotely sensed data with interpretive analysis capability would produce similar

data of less resolution and gain only 80% confidence,  but could accomplish it in 4 years at a cost

of 4x.  Knowing time intervals and successive assessments in a monitoring regime across time is

critical to determine the best path today.

• Compliance:  Both the GCMRC and TWG specification of this criteria would indicate if the

compliance requirement exists for the data, and that the project activity must be performed.  

The GCMRC requirement evaluates underperformance and specifies needed correction.  Do all

the compliance requirements in the GCDAMP have to result from dam operation impacts?  And

will criteria be established to define those compliance issues addressed by the AMP?

• Legacy:  The SAs have not been asked to review the Legacy data report to GCMRC and TWG.

Although legacy data in general are excellent resources for many applications, and many long

term monitoring programs contribute to legacy data sets, a question exists regarding why it is a

criteria for this monitoring program.  That is, the GCDAMP monitoring program and associated

data is an applied science program designed to resolve specific issues and answer specific

questions, requiring specific data needs, i.e. CMINs. If data satisfies this requirement and other  

criteria, it should be included.  Does inclusion of this criteria mean that data also has to fit legacy

data guidelines, or does it mean that data that fits legacy requirements are more important to

the program? Criteria 2 and 6 seemingly satisfy specific requirements for data.  Having a

separate criteria for legacy data does not appear necessary.  If there are established legacy data

that are necessary to the program it could be referenced in criteria 6.  

• Ecosystem Importance:  Another important factor that relates to ecosystem importance is the

ability of the parameter or variable studied to drive major parts of an ecosystem.  In many ways

“system indicators” and “system drivers” account for much of the same variance in an aquatic

system. Yet, many indicators are responders, where as drivers act as impactors.  Drivers in this

system such as diatoms act both as response indicators of system health and as drivers in higher

trophic systems. A variable such as temperature is a primary driver of aquatic biology and is

critical to aquatic system linkages and overall system performance.  An assumption exists that

most of the critical drivers and indicators in this system are known.  This may well be the case
and yet we also know that how they interact changes and creates changing direct and indirect

response in resources of concern.  As such an additional point could be added under TWG; “Can

thresholds of known interacting variables be evaluated in the CMP”.  For example specification

of how low flows, low DO, rising temperature create interactive impacts on the response

variable diatom biomass.

• Data Quality/Availability:  When one reads this criteria and criteria on methodology a key

element of the monitoring program seems missing, assessments.  It is evident in the draft of the

text, but the criteria for evaluating both the GCMRC and TWG collaborative direction does not

highlight the monitoring assessment needs. It certainly refers to it in requirements for statistical

accuracy.  Without a criteria for data assessment and finding of analysis we are left to wonder

how we will move from extensive data sets to answering the managers’ questions and resolving

resource issues. The SAs asked for improvements in the science direction regarding this issue in

previous reviews of the CMP and improvements have been made in the text.  If managers are

truly concerned about confidence they can place in accuracy of outcomes, whether from GAP

analysis, multivariate analysis or simulation models, emphasis needs to be placed on evaluation

of assessment methodologies used in various monitoring programs.  PEP evaluations are the

best activity to which to clarify this requirement, but it should also appear as a criteria of the

manager evaluation.   

• Cost/Benefit Assessment (C/B:.  The GCMRC requirement must also include the expected time

period for gaining significant intermediate and final resolves specified in goals and CMINs. That

is, this monitoring program is not just about accounting for state of resources,  it is an applied

science program to resolve resource issues in a timely manner.  The TWG in its treatment of

costs, benefits and risks of pursuing one monitoring approach over another must give critical

treatment to time requirements for differing monitoring and assessment approaches.  Time

always increases risks of changes in other interacting resources and variables.

• Status of Knowledge:  A TWG approach regarding use of this criteria should relate to when and

how a particular monitoring and/or assessment approach should be modified through time, i.e.,  

“evaluate needs for modifications in the monitoring/assessment approach.” Although

monitoring programs should not be subjected to the annual pressures of changing budgets and

political desires, there is a need to have scheduled periodic reviews of monitoring and

assessment approaches. One of the primary outcomes of a 5 year interval status of knowledge

assessment should be to help inform needed improvements in a monitoring program.

• Trade‐off Analysis.  As noted earlier, TWG needs to specify a maximum budget level for each

program alternative rather than providing ranges.  Otherwise be willing to accept a 10% budget

reduction for a medium level program.  In the TWG tasks it states, “Consider all variables above

in the SDM framework”.   The evaluation approach is structured to use the outcomes,

constraints, etc of the evaluation criteria as inputs to the SDM process, which is the most

objective way to perform the analysis.  As such, outcomes from your trade‐off analysis and C/B

analysis are inputs to your SDM assessment. It is a very acceptable and robust approach.  Use of

trade‐off methodologies to evaluative differing program alternatives especially more analytic

procedures requires that quantified criteria on each alternative be accurate.  If some criteria are

more important than others a weighting system must be developed and approved.  If these

tasks cannot be accomplished to everyone’s satisfaction, more subjective models can be used.  

Even if this approach is chosen, the process is analytic by nature and must be well specified,

clearly understood and implemented well.  That is, it will require much more time initially to do

this well than the more simplistic process of verbally describing program differences and voting.  
The SAs continue to recommend this approach but have always encouraged the TWG to anoint a

general executive committee to executive this type of process. It permits the full TWG the

needed time to focus on the assessment outcomes and the SDM process.  

5.0 Using Structured Decision Processes To Evaluate Alternative Monitoring Programs

The 6 Structured Decision Making steps provided are the general basis for most decision

processes used today.  As noted earlier, specialists in natural resource management recognize

them as the basis for NEPA processes, especially EAs and EISs.   Also as noted earlier, effort is

needed by the TWG now to add some specificity to each of the six steps, so the acceptance of

the proposed draft of the CMP by GCMRC and TWG will be accomplished with significant clarity

about how the process will work.  The TWG has requested the SAs to evaluate existing SDM

methods that could be used in the proposed decision processes.  The following comments

address additional definition briefly.   

Decision Context:  Articulate the outcome the SDM process must perform, i.e. select monitoring

programs that will receive funding. What is the decision body and how will it accept external

input.  How will it make the final decision?   

Objectives and Evaluation Criteria:  As noted these criteria are defined and being refined.

Develop Alternatives:  The process has been defined and is being refined.  TWG needs to be

specific on budget levels, where saved funds are going to be allocated, etc.  For example, if

funds go to compliance, both GCMRC and TWG might want to consider a different monitoring

method that could also compensate for inability to have needed strategic research to support

monitoring.

Estimate Consequences: How this is performed depends upon the SDM method selected.  The

SAs will provide proposals on alternative methods.

Evaluate Tradeoffs and Select:  One process is mentioned.  The overall evaluation process has

to be structured and applied in a consistent manner.  As noted above the TWG should consider

establishing a subgroup that is willing to spend some time to establish this process, conduct

evaluations, summarize outcomes and conduct a briefing process with TWG.  If the Chair feels

the full TWG must be involved in both development and assessments perhaps four workshops

would be more appropriate.   

Implement and Monitor:  The three critical steps to this process are specified.