DEPARTMENT OF SCIENCE, TECHNOLOGY, ENGINEERINGAND PUBLIC POLICY (UCL STEaPP)

The ethos of science advice
Arthur Petersen
Professor of Science, Technology and Public Policy(collaborators: Jason Blackstock, Kira Matus, Michael Veale, Pita Spruijt)
STIG PoP Seminar, University of Tokyo, 6 November 2015

Outline
•••••
•
•
•

Literature review advisory roles
Oustanding questions
Climate-change uncertainty
Post-normal science
IPCC
Capacity-building needs
PBL and uncertainty
Ethos of science advice

Literature review Spruijt et al. (2014)“Most theories are well elaborated, but empirical proof for thedescribed changes, roles and processes is limited” (p. 16)

Spruijt et al. (2014)

LONDON’S GLOBAL UNIVERSITY

Spruijt et al. (2014)

SCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)

Jasanoff (2013)“In a world that seems too often to be hurtling toward
planetary self-destruction, we need the capacity – and will –to question our purposes deeply: to ask over and over how
knowledge underpins institutions and policies that are
sometimes serviceable but other times perverse; and toexplore how even esoteric social institutions such as
scientific advice-giving can stay in touch with ongoingreflection on where we have come from and where we aregoing” (pp. 66–67)
Jasanoff, S. (2013), ‘The science of science advice’, in CSaP, Future
Directions for Science Advice in Whitehall, pp. 62-67

LONDON’S GLOBAL UNIVERSITY

Latour: “We have never been modern”

SCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITY

Outstanding questions
Two practical questions from scientific advisory
practitioners about science advice:
1. What evidence is there to help me do my jobbetter?
2. How should I set up effective science advisory
mechanisms in my context?

SCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITY

Research programme: starting points• International Network for Government Science Advice• OECD Global Science Forum Report on Scientific Advice
for Policymaking

SCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITY

Research programme: knowledge gaps
Parliamentary advice‘Top’ advisors vs. more specific/Engineering advice

lower levels of engagement

Roles of ‘boundary organisations’International/national/local advice

Roles of NGOs and think tanks

Internal structures

Technical advice

Capacities

Influence of topical domains

Mobilising and incentivising science

Influence of levels of development
Advocacy roles
Operational vs. agenda-setting roles

and engineering communities
Accountability, quality,communication, participation

SCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITY

Research programme: aspects• Type of issue• Advisory structures• Management and orchestration• Characteristics of the recipients of advice• Skills and characteristics of advisors• Activities of advisors and recipients• Type of knowledge• Methods of communicating and disseminating• Culture and context
SCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITY

Functional framework for analysing scienceadvice to decision-makers
Activity perspective

Actor perspective

Institutional••

Main actors and theirperspective

activities andcharacteristics,•decision-making

resources,

activities

capabilities, interests,•Connecting activities

values and goals

on actors, activities•

Perspectives onPerspectives onand structures

Knowledge-making•

institutions

activities by actors

actors by other actors

and media

and media•

Informal and formal••

Effects of institutions

Conflicts andcomplementarities

Political constellations
SCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITY

Activities: potential graphical depiction

Courtesy Michael Veale (UCL)SCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITYSCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITY

Global and regional models
Global modelRegional model

SCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITYIBM Supercomputer
European Centre for Medium-Range Weather Forecasts
SCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)

IPCC 2001: taking into account all
uncertainties (including model uncertainty):
largest part of warming is ‘likely’ due toanthropogenic greenhouse gases

Warning:take into account uncertaintyin climate simulation

IPCC 2007: taking into account all
uncertainties (including model uncertainty):
largest part of warming is ‘very likely’ due
to anthropogenic greenhouse gases

LONDON’S GLOBAL UNIVERSITY

de Kwaadsteniet versus van Egmond• de Kwaadsteniet:“Computer simulations are seductive due to theirperceived speed, clarity and consistency. However,simulation models are not rigorously compared withdata.”• van Egmond:“Policy makers are confronted with incomplete
knowledge; it is the task of scientific advisers toreport on the current state of knowledge, including
uncertainties. Simulation models are indispensable.”
SCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITY

Funtowicz and Ravetz,‘Science for the postnormal age’, Futures,
1993

SCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITY

The challenge of post-normal science• Expert advisers should be reflexive• Methods for dealing with uncertainty should merely
be considered as tools, not as the solutions• Fear for paralysis in policy making should not beallowed to block communication about uncertainty• Communication with a wider audience about
uncertainties is crucial

SCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITY

Shifting notions of reliability• Statistical reliability (expressed in terms of probability)– How do you statistically assess evidence?

• Methodological reliability (expressed qualitatively interms of weak/strong points)– How do you determine the methodological quality of thedifferent elements in scientific and engineering practice?

• Public reliability (expressed in terms of public trust)– What determines public trust in scientists and engineers?

SCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITY

Example from theIntergovernmental Panel on
Climate Change WG I (2007)“Most of the observed increase in globally
averaged temperatures since the mid-20thcentury is very likely due to the observed
increase in anthropogenic greenhouse gas
concentrations12.” (SPM)
12 Consideration of remaining uncertainty is based on

current methodologies.
SCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITY

Example from the IPCC WG I 2007
(continued)“Very likely” means a chance >90%. But whatkind of probability are we dealing with here?
likelihood using expert judgement, of an outcome“assessed likelihood,of anoutcome or a result”
ora result”

Draft SPM
Final

SCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITY

Example from the IPCC WG I 2013

“Probabilistic estimates of quantified measures of uncertainty ina finding are based on statistical analysis of observations ormodel results, oror expert
both, andexpert judgment.”
judgment.”

Final
Draft SPMSCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITYSCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITYSCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITYSCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITYSCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITYSCIENCE, TECHNOLOGY, ENGINEERINGAND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITYSCIENCE, TECHNOLOGY, ENGINEERINGAND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITYSCIENCE, TECHNOLOGY, ENGINEERINGAND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITYSCIENCE, TECHNOLOGY, ENGINEERINGAND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITYSCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITYSCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITYSCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITY

Latour: “We have never been modern”

SCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITYThe IPCC: science or politics?
• Assessments are social constructs that contain bothscientific and political elements• Successful? Depends on ability to connect to climate
science and policy• Generally voiced criticism: IPCC not open enough toskeptics

SCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITYThe IPCC: science or politics? (II)• Practice: procedures ensure inclusivity; skeptics dohave influence; reflexivity on dissensus is moderate(neither low nor high)• Not: “scientific consensus”. But: “policy-relevant
assessment acknowledging uncertainty”• Still, the communication of uncertainty can befurther improved• The IPCC acts as a Latourian “Parliament of Things”– but the actors won’t admit this!
SCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITY

PBL’s Guidance for
Uncertainty Assessment
and Communication• Offers assistance toanalysts• Not a protocol• Based on post-normal
science

SCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITY

Six uncertainty elements in assessments

42

SCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)

24 Jan 2014 | Arthur Petersen

LONDON’S GLOBAL UNIVERSITY

Typology of uncertainty• Location• Level of uncertainty
statistical uncertainty, scenario uncertainty, recognisedignorance• Nature of uncertainty
knowledge-related uncertainty, variability-related
uncertainty• Qualification of knowledge base (backing)• Value-ladenness of choices
SCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITY

Locations of uncertainty• Context• Expert judgement• Model• Data• OutputsSCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)LONDON’S GLOBAL UNIVERSITY

Capacity-building needs science advice• Dealing with complexity, uncertainty and systems
thinking• Communicating in different languages(understanding of scientific and policy-makingprocesses)• Management of expectations (limits of science)• Negotiating and influencing <> maintaining integrity• ‘Civics’ for scientists• Public education on science–policy interface• Professional career paths
SCIENCE, TECHNOLOGY, ENGINEERING
AND PUBLIC POLICY (UCL STEaPP)The ethos of science advice (1/2)
Explicit reflection on uncertainty and values“Take “normal science” seriously, but also organise reflection on
its uncertainties and value-ladenness.
Addressing methodological and public reliability
Alongside the statistical reliability of results (expressed in termsof probability), devote due attention to their methodologicalreliability (expressed in terms of strengths and weaknesses) andtheir public reliability (expressed as the degree of publicconfidence in the scientists who produce them).

The ethos of science advice (2/2)
Extended peer review
Involve a larger group of specialists and non-specialists who
hold different values in monitoring the quality of scientificassessments.
Acknowledging social complexity
Be wary of accepting the conclusions of actors and practitioners
at face value: try to delve deeper through the layers ofcomplexity by means of narrative methods.