PICES Press Vol. 22, No. 2

North Pacific Marine Science Organization

OSM Workshop on “Bridging the divide between models and decision-making:
The role of uncertainty in the uptake of forecasts by decision makers”
by Edward J. Gregr

Convenors and invited speakers (left to right): Kai M.A. Chan (Canada), Lee Failing (invited speaker, Compass Resource Management Ltd., Canada),
Georgina A. Gibson (invited speaker, International Arctic Research Center, University of Alaska Fairbanks), Edward J. Gregr (Canada), Vladimir Kulik(Russia), Hal Batchelder (PICES Secretariat), Motomitsu Takahashi (Japan), Shin-ichi Ito (Japan), Missing: Naesun Park (Korea), Ian Perry (Canada),
Jameal Samhouri (USA).

The FUTURE science program recognizes the need to more
directly address uncertainty in products such as ocean climate
forecasts, and to improve how the knowledge produced byPICES is disseminated. In a series of presentations anddiscussions, our workshop (W2), held April 14, 2014,
examined both the nature of uncertainty in model systems,
and how uncertainties can be included in the decision making
process. The workshop was well attended, with broad
representation from PICES member countries. We identified
a number of opportunities for the PICES community toimprove how uncertainty is characterized, and to highlightseveral advantages that would emerge from tailoring model
outputs, including uncertainties, for diverse audiences.

design and contain uncertainties about model boundaries andresolution, among other things. Once model data and scope
are defined, decisions and assumptions about process must
be made, for example, which ecosystem components interact
and the nature of these interactions, some of which are alsouncertain.
For the purposes of communication and decision-making,Gregr and Chan added two additional classes ofassumptions, Communication and Relevance (Table 1).
Assumptions around communication obscure uncertainties
related to things such as language and disciplinary
epistemology. Perhaps most importantly, the relevance of
ecosystem model results to decision-making is often
assumed to be quite high by model developers. However,this is far from certain, and evidence suggests that it isoften quite low (Failing, this workshop). This class ofassumptions thus relates to uncertainties about indicator
selection and the context relevance. In many cases,
comprehensive treatments of model uncertainties are notnecessarily desirable (or tractable). However, Gregr andChan argue that a more explicit recognition and discussionof model assumptions is necessary for improving ourunderstanding and communication of model results, and theassociated uncertainties.

Understanding uncertainty
The first step in addressing uncertainty is to understand its
source. Gregr and Chan (in review) consider three classes(Data, Scope, and Process) of uncertainty based on theassumptions necessary at various steps in the model design
process. Assumptions about data relate to uncertainties about
things such as sampling bias, representativeness, and theoverall relevance of the data to the study under consideration.
Decisions about model scope (e.g., specification of spatial,temporal, and compositional extents) are central to model
Summer 201424

North Pacific Marine Science Organization

PICES Press Vol. 22, No. 2

Table 1 Assumption classes and the associated types of uncertainty.

Assumption classassume unbiased data, but analyses often contain implicit,
potentially false assumptions about the relevance orsuitability of such data, which may contain geographic orseasonal sampling biases. Gryba also considered
conceptual assumptions implicit in such models, where, forexample, it is often assumed that mammal sightings arecorrelated with high prey concentrations. She showed how
this conceptual assumption is testable using independent
data on prey distributions, thus providing insights into theuncertainty associated with this key habitat modelingassumption.

Uncertainty

Data

Observational

Scope

Design uncertainty

Structure
Process

Parameter estimation
Natural variation
Inherent randomness

Communication

Ambiguity

The challenge of coupling models was discussed by ShinIchi Ito (Japan), who presented the results of a fisheries
production model for Pacific saury forced using sea surface
temperature predictions from 12 different global climate
models developed by the Intergovernmental Panel on
Climate Change (IPCC). While a number of correlations
were found, uncertainty in fish growth projections weredominated by uncertainties in the physical forcing. Thisemphasizes the need for appropriate scaling methods whenmoving from global to regional study areas. Ito suggested
that to effectively couple models across scales, more
attention needs to be paid to key processes at the interfaces.
For example, zooplankton dynamics play a key role insaury abundance. Thus, it is critical to appropriately
capture the relationships between physics and zooplankton,
and between zooplankton and higher trophic levels (HTLs).
Given that HTLs typically respond to multiple drivers
operating at different scales (e.g., Palacios et al. 2013), a
better understanding is needed about how HTLs respond to
short-term forecasts.

Under-specification
Vagueness
Relevance

Context dependence
Relativism

Representing uncertainty
Several presentations illustrated methods for examining
model uncertainties. Invited speaker, Georgina Gibson(USA), discussed the role of assumptions in thedevelopment of lower trophic level (LTL) ecosystemmodels. Describing how the complexity of model structure
and parameterization can increase quickly, she emphasized
the associated need for assumptions to manage this. She
demonstrated how to use sensitivity analysis to identify
critical parameters, but noted that the large computational
demands limit the extent to which it can be applied. Gibson
and Spitz (2011) used a one-dimensional lower trophic
level model to examine a suite of 135 biological and 8
environmental factors, and ranked these factors according
to their influence on model outputs. Although the approach
identified parameters deserving closer scrutiny, similar
analysis has not been applied to 2- or 3-dimensionalmodels because of the computational limitations, leaving
important parameters untested.

The need to understand such processes and theirinteractions was nicely illustrated by Bill Peterson (USA),
who showed how the correlation between the Pacific
Decadal Oscillation (PDO) and Chinook salmon ocean
survival, which had shown a robust negative correlation for
15 years, suddenly failed dramatically in 2011. The causal
relationship appears mediated by copepods, which provide
an index of the lipid richness at the base of the food chain.
This ‘lipid rich copepod index’ is, in turn, correlated with
Chinook survival. However, the decoupling of therelationship highlights new uncertainties about the scale
and process of the presumed mechanism. Once again, thisemphasizes the need to understand the process, thougheven so, surprises should be expected. For HTL models inparticular, the need to transition from correlative tomechanistic model frameworks is increasingly relevant(Palacios et al. 2013).

Exploring the parameter uncertainties in such simulation
models is typically handled using established Monte Carlo
methods. However, knowing the range over which tosample parameters is critical to such efforts. Unfortunately,such ranges (which are necessary to parameterizetheoretical, mechanistic models) are not always known, andthus represent important design assumptions. Similarly,
initial or starting conditions for models may be unknown,
which can have a significant effect on the trajectory ofmodel predictions (Gibson and Spitz 2011).

Decision making and communication

Rowenna Gryba (Canada) examined assumptions about therelevance and utility of data, and how this influences theevaluation of habitat suitability in models of North Pacific
Right whales. Standard cross-validation approaches toevaluating models of habitat suitability are sensitive topotential biases in the data. Analytical methods typically

Lee Failing (Canada), our second invited speaker, provided
an important perspective on the role of research anduncertainty in decision making. Failing noted that while
many frameworks exist to support integrated management,
the process of actually making decisions and managing the25Summer 2014

PICES Press Vol. 22, No. 2

North Pacific Marine Science Organization

risks arising from uncertainty are rarely emphasized.
Rather, the decision-making components are often
presented as post-hoc interactions with the principal
science represented in prominent detail (e.g., Figure 1).
Treating decision-making as an afterthought introduces
many implicit and likely false assumptions about the roleof science in the decision-making process. Suchperspectives are grounded in the information deficit modelof science communication, an approach that is increasingly
understood to be false (see http://en.wikipedia.org/wiki/Inf
ormation_deficit_model). Only a small portion of science
as currently practiced is typically salient to decision makers.
To improve the relevance of science to policy and decision
making, Failing emphasized the transformative power thatcomes from “making the decision” the goal of the scientificeffort and analysis. This leads to immediate identificationof what is important, and informs where science could best
contribute to the process. The salience of such contributions
would be greatest if they helped inform the trade-offs faced
by decision-makers and their stakeholders.

also means being explicit about unquantified assumptions
to help understand the associated risk. And perhaps mostimportantly, it means recognizing that there is no single
audience or stakeholder, but rather a diversity of interests
for whom different metrics and presentation methods may
be required. Targeting – identifying what matters, and how
it is best measured, for each decision scenario – will be key
to effectively communicating FUTURE products beyond
the PICES scientific community.
Challenges and opportunities
In addition to the presentations, we devoted considerable
time to discussion, including a joint session with theparticipants of workshop W3 on “Climate change andecosystem-based management of living marine resources:
appraising and advancing key modeling tools”. The joint
session acknowledged that the fundamental challenge forthe modeling community is to identify what resonates withdecision makers. Given the diversity of management andpolicy decisions that are regularly made, this emphasizes
the need to develop communication strategies that canadapt effectively to diverse audiences. Decision makers
would like to reduce risk and reduce surprises. This would
presumably simplify the trade-offs inherent in policy andmanagement decisions.
The role of reliable ecosystem forecasts in reducing risk
and producing fewer surprises is recognized, although therisk of such forecasts being wrong and surprising decision
makers will need to be carefully managed. Integrating datafrom regional Ocean Observing Systems, focusing on
short-term forecasts, and predicting the responses of HTLs
are essential components of such ecosystem forecastsystems. The increasing risks faced by decision makers due
to climate uncertainty provide an opportunity to advocate
for ocean climate forecast services at regional scales,emphasizing that their utility for managing risk is as highas traditional short-term weather forecasts.
Uncertainties related to closure terms (i.e., the parameters
required to represent aspects not included in the model),model structure, and the downscaling of global models willcontinue to present challenges to the development of such
short-term forecasts. Ensemble modeling is increasinglyproviding an opportunity to address the cumulative
uncertainty in highly complex models, allowing theassessment of robustness (Knutti and Sedláček 2013). To
demonstrate their relevance, a key performance challenge
for such short-term forecasts is to achieve not only
statistical accuracy, but to reasonably predict the phase (i.e.,timing) of climatic events. This will be best approachedthrough regional models, which have already met withsome success, such as the prediction of hypoxia events(Siedlecki et al. 2014). Accurate predictions of phasechanges is critical (although emphatically not sufficient)for forecasting the HTL indicators important for many
stakeholder groups.

Fig. 1 Example risk assessment framework emphasizing (red oval,added) the implied post-hoc role for the decision making process.

This leads to the question of science communication, andKai Chan (Canada) argued that this is as much a questionof targeting as it is of understanding the science and theinherent uncertainties. Through various examples, Chanemphasized the need to focus on the complete decisionscenario to identify what really matters and, equallyimportant, what is at risk. From the perspective of theFUTURE program, this means identifying relevant metrics
and understanding the distribution of inputs and outputs. It
Summer 201426

North Pacific Marine Science Organization

PICES Press Vol. 22, No. 2

Other opportunities are emerging due to the consequencesof a changing ocean. As ecosystem boundaries shift,baselines on which stock assessment data are based will
begin to expose the assumption of spatial stationarity. This
provides an opportunity for fisheries scientists toreconsider how the science underpinning management
decisions is conducted, and perhaps refocus it more directlyon the decision and the risks to stocks in a moreunpredictable ocean. This is particularly salient in light ofrecent research suggesting ocean conditions play a muchstronger role in recruitment than previously believed(Szuwalski et al. 2014), re-enforcing the need for reliable
ocean forecast systems.

Knutti, R. and J. Sedláčcek. 2013. Robustness and uncertainties in
the new CMIP5 climate model projections. Nature Climate
Change 3, 369-–373.
Palacios, DM, MF Baumgartner, KL Laidre, and EJ Gregr. 2013.
Beyond correlation: integrating environmentally andbehaviourally mediated processes in models of marine mammaldistributions. Endangered Species Research 22: 191–203. doi:
10.3354/esr00558
Seidlecki, SA, A Hermann, N Bond, S Alin, and R Feely. 2014.
Hypoxia and ocean acidification of the coastal waters of thePacific Northwest: evaluation of seasonal predictions ofhypoxia and pH. Presentation at the 2014 Ocean Sciences
Meeting, Honolulu, HI.
Szuwalski, CS, KA Vert-Pre, AE Punt, TA Branch, and R Hilborn.
2014. Examining common assumptions about recruitment: ameta-analysis of recruitment dynamics for worldwide marine
fisheries. Fish and Fisheries 1467-2979. doi.org/10.1111/faf.12083.

The take-home message for FUTURE from the workshop
is that broader uptake of our knowledge products will
require clearly articulating the decision context to which
they contribute. The extent to which we can explicitly
inform the risks in the choices facing managers and policy
makers will influence the uptake of our science into
decision making. Casting our uncertainties as risks, andtargeting these results at the appropriate audiences, willfurther increase our contribution to evidence-based
decision making. Finally, by considering how we cancontribute to decisions that will be made in the future, theocean science community has an opportunity to move froma reactive, crisis-management role to proactive leadershipwhere best available science provides timely, salient, andsound advice to support ocean management decisions.

Acknowledgments

The author thanks the other convenors for their contributions
to the success of the workshop.

References
Gibson, GA and YH Spitz. 2011. Impacts of biological parameterization, initial conditions, and environmental forcing onparameter sensitivity and uncertainty in a marine ecosystemmodel for the Bering Sea. Journal of Marine Systems 88: 214–231.
Gregr, EJ and KMA Chan KMA. 2014. Leaps of Faith: How
implicit assumptions compromise the utility of ecosystemmodels for decision-making. BioScience. In review.

Edward Gregr (ejgregr@gmail.com) is a marine ecologist withover 15 years’ experience in marine classification and marine
habitat suitability. He is currently completing his PhD at theInstitute for Resources, Environment, and Sustainability at theUniversity of British Columbia, Canada, focusing on trophic
cascades in nearshore ecosystems.

27

Summer 2014