PBL-omslag zonder flappen
340 x 240 mm

Assessing an IPCC assessment
An analysis of statements on projected
regional impacts in the 2007 report

Assessing an IPCC assessment
The Netherlands Environmental Assessment Agency (PBL) has investigated the scientific foundations for the IPCC summary conclusions
of the Fourth Assessment Report of 2007 on projected regional
climate-change impacts, at the request of the Dutch Minister for the
Environment. Overall the summary conclusions are considered well
Hier
komt
bijwere
voorkeur
flaptekst
tesignificant
staan, in plaats
founded
and
none
foundeen
to contain
any
errors.
van deze algemene tekst
The Working
Group II contribution
to the Fourth
Assessment
Het Planbureau
voor de Leefomgeving
is hét
nationale Report
showsinstituut
ample observational
evidence
of regional climate-change
voor strategische
beleidsanalyses
op het gebied
impacts,
have
beenenprojected
vanwhich
milieu,
natuur
ruimte. to pose substantial risks to most
parts of the world, under increasing temperatures. However, in some
Het PBL draagt bij aan de kwaliteit van het strategiinstances the foundations for the summary statements should have
sche overheidsbeleid door een brug te vormen tussen
been made more transparent. While acknowledging the essential role
wetenschap en beleid en door gevraagd en ongevraagd,
of expert judgment in scientific assessments, the PBL recommends to
onafhankelijk en wetenschappelijk gefundeerd, verkenimprove the transparency of these judgments in future IPCC reports.
ningen, analyses en evaluaties te verrichten waarbij een
integrale benadering voorop staat.
In addition, the investigated summary conclusions tend to single out
the most
important
negative
impacts of climate change. Although
Planbureau
voor
de Leefomgeving
this approach
was agreed to by the IPCC governments for the Fourth
Locatie Bilthoven
Assessment
Report,
Postbus
303 the PBL recommends that the full spectrum of
regional
impacts
is summarised for the Fifth Assessment Report,
3720
AH Bilthoven
including
the274
uncertainties.
The PBL believes that the IPCC should
T: 030
274 5
invest F:
more
quality
030 in
274
4479control in order to prevent mistakes and shortcomings,
to the extent possible.
E: info@pbl.nl
www.pbl.nl

Netherlands
Planbureau
Environmental
voor de Leefomgeving,
Assessment Agency,
<maand>
July200X
2010

snijlijn

rug

snijlijn

Assessing an IPCC assessment
An analysis of statements on
projected regional impacts
in the 2007 report

Assessing an IPCC assessment. An analysis of statements on projected regional impacts in
the 2007 report
©Netherlands Environmental Assessment Agency (PBL)
The Hague/Bilthoven, 2010
PBL publication number: 500216002
Cover photo: Hollandse Hoogte
Corresponding author: leo.meyer@pbl.nl
This publication can be downloaded from: www.pbl.nl/en. A hard copy may be ordered from:
reports@pbl.nl, citing the PBL publication number or ISBN.
Parts of this publication may be reproduced, providing the source is stated, in the form:
Netherlands Environmental Assessment Agency, the publication title.
The Netherlands Environmental Assessment Agency (PBL) is the national institute for
strategic policy analysis in the field of environment, nature and spatial planning. We contribute
to improving the quality of political and administrative decision-making by conducting outlook
studies, analyses and evaluations in which an integrated approach is considered paramount.
Policy relevance is the prime concern in all our studies. We conduct solicited and unsolicited
research that is both independent and always scientifically sound.
Netherlands Environmental Assessment Agency
Office The Hague			
Office Bilthoven
PO Box 30314			
PO Box 303
2500 GH The Hague			
3720 AH Bilthoven
The Netherlands			
The Netherlands
Telephone: +31 (0)70 328 8700		
Telephone: +31 (0)30 274 2745
Fax: +31 (0)70 328 8799		
Fax: +31 (0)30 274 4479
E-mail: info@pbl.nl
Website: www.pbl.nl/en

Foreword
In January 2010, the media reported two errors in specific parts of the Fourth
Assessment Report of 2007 by the Intergovernmental Panel on Climate Change
(IPCC). The media reports gave rise to questions about the credibility of the overall
IPCC assessment: could politics and society still rely on the IPCC for an assessment
of the scientific knowledge on climate change?
On 28 January 2010, the Dutch Parliament asked Jacqueline Cramer, the then Dutch
Minister for the Environment, for an investigation into the implications of these
errors in the IPCC report. Minister Cramer subsequently requested the Netherlands
Environmental Assessment Agency (PBL)1) to make an assessment of the reliability
of the regional chapters (9 to 16) of the IPCC Working Group II contribution to the
Fourth Assessment Report (the part on Impacts, Adaptation and Vulnerability), and
to assess the effects of any errors on the summary conclusions drawn by the IPCC.
From the outset it was clear that the PBL would not be able to repeat the original
IPCC Working Group II assessment, given the sheer volume of the scientific work
reported in the IPCC exercise.2) What we were able to do, however, was investigate
the extent to which the IPCC in their summaries had presented existing scientific
knowledge to the world of policymakers, in a way that was supported by the
underlying texts and scientific references. What is more, we also investigated what
could be learned from our findings so that informed choices can be made in future
assessments. This seemed the more reasonable approach, as behind the request
from the Dutch Parliament was a more general concern: could policymakers and
the public at large still trust the IPCC's key messages?
A scientific assessment – an analysis and weighing of the actual scientific
knowledge – of a complex problem such as climate change is a tremendously
difficult task. The IPCC is a bridge between science and policy: a so-called ‘science–
policy interface’. The IPCC is not conducting science, nor is it making policy, but
should provide the guarantee that policymakers can come to their decisions on
the basis of the best available knowledge regarding climate change. We have
1) The Netherlands Environmental Assessment Agency (PBL) is an independent governmental body that by statute provides
the Dutch Government and Parliament – and the European Commission, European Parliament and UN organisations – with scientific advice on problems regarding the environment, sustainability and spatial planning. PBL authors have contributed to several
assessments of the IPCC. During the Third and Fourth Assessments, the PBL hosted the Co-Chair and Technical Support Unit of
Working Group III (Mitigation on Climate Change).
2) Elsewhere we have reported on new insights into climate science since the Fourth Assessment Report (cf. News in Climate
Science and Exploring Boundaries, 2009).

Foreword

5

investigated how this task was taken on by the IPCC Working Group II in their
assessment of the regional impacts of climate change. Based on this investigation
we have formulated some recommendations to further improve the quality of
these assessments.
The investigation was conducted realising that climate science and policymaking
are currently taking place in a new era, characterised by a high degree of
politicisation, a much more dynamic interaction between science and public
discourse, and vocal citizens who either want to know whether policy measures
under discussion are all really necessary, or who are of the opinion that suggested
measures do not go far enough. This new context comes with new requirements
for the IPCC assessments. Our suggestions are thus not always meant as criticism
of the ‘architecture’ of the climate report of Working Group II as this was being
drawn up in the years leading up to its publication in 2007. Yet we believe that, by
being critical and self-critical of work contained in the Fourth Assessment Report,
we can contribute to further improving the reliability of future IPCC reports. It is an
exercise in future-oriented learning.
Accomplishing our task would have been impossible without the cooperation of
the IPCC authors of Working Group II, who were willing to devote a considerable
amount of their time to answering our questions, and did so within the strict
time tables we had set them. It is clear that the responsibility for the research,
as well as for the presentation of the findings, the conclusions drawn, and
the recommendations made, lies solely with the Netherlands Environmental
Assessment Agency (PBL).
Our investigation is focused in character and size. However, a number of
conclusions and recommendations concern the set up of the procedures and
processes of the IPCC. Hopefully, those suggestions can still play a role in the
review by the InterAcademy Council, that will become available by late August 2010,
focusing on procedural and process issues of the IPCC.3)
Professor Maarten Hajer
Director, Netherlands Environmental Assessment Agency (PBL)

3) InterAcademy Council 2010, Review of the IPCC; an evaluation of the procedures and process of the
Intergovernmental Panel on Climate Change, http://reviewipcc.interacademycouncil.net/index.html

6

Assessing an IPCC assessment

Contents
 Executive summary

9

Part I – Main findings
 1
Introduction 19
1.1 Scientific assessment in aid of policy 19
1.2 Some background on IPCC Working Group II 22
1.3 Scope, objectives and limitations of this report 24
1.4 Structure of the report 26
 2
Methodology 27
2.1 Investigative approach 27
2.2 Typology of errors and qualifications

29

Results and discussion 33
 3
3.1 Status of IPCC summary statements on regional climate-change impacts 33
3.2 Errors 35
3.3 Risk-oriented approach 37
3.4 Comments 40
3.5 Conclusion 43
 4
Recommendations 45
4.1 Minimising the risk of errors 45
4.2 Investing in the improvement and transparency of foundations of summary conclusions
4.3 Strengthening the quality control by the chapter teams 47
4.4 Strengthening the review process 47
4.5 Timing of the assessments 48
4.6 Balancing the assessment of climate-change impacts 48
4.7 Investing in climate-change impact science 50

46

Part II – Detailed analysis of regional chapters and summaries
 5
Africa 53
5.1 Analysis of statements in Table SPM.2 of the Synthesis Report 53
5.2 Additional findings 55
5.3 Findings from the PBL registration website 56

Foreword

7

 6 Asia 59
6.1 Analysis of statements in Table SPM.2 of the Synthesis Report 59
6.2 Additional findings 60
 7 Australia and New Zealand 63
7.1 Analysis of statements in Table SPM.2 of the Synthesis Report 63
7.2 Additional findings 64
 8 Europe 67
8.1 Analysis of statements in Table SPM.2 of the Synthesis Report 67
8.2 Additional findings 68
8.3 Findings from the PBL registration website 70
 9 Latin America 73
9.1 Analysis of statements in Table SPM.2 of the Synthesis report 73
9.2 Additional findings 74
 10 North America 77
10.1 Analysis of statements in Table SPM.2 of the Synthesis Report 77
10.2 Additional findings 78
 11 Polar regions (Arctic and Antarctic) 79
11.1 Analysis of statements in Table SPM.2 of the Synthesis Report 79
 12 Small islands 81
12.1 Analysis of statements in Table SPM.2 of the Synthesis Report 81
12.2 Additional findings 81
 Annex A

Table SPM.2 of the Synthesis Report: examples of some projected
regional impacts 83

 Annex B

The error on the melting of the Himalayan glaciers 86

 Annex C

The error on the Dutch land area below sea level

 Annex D

Sea level rise: consequences for the Netherlands 91

 Annex E Abbreviations
 Colophon

8

96

99

Assessing an IPCC assessment

89

Executive summary
The Netherlands Environmental Assessment Agency (PBL) has investigated the scientific
foundations for the IPCC summary conclusions of the Fourth Assessment Report of 2007
on projected regional climate-change impacts, at the request of the Dutch Minister for
the Environment. Overall the summary conclusions are considered well founded and
none were found to contain any significant errors. The Working Group II contribution to
the Fourth Assessment Report shows ample observational evidence of regional climatechange impacts, which have been projected to pose substantial risks to most parts of the
world, under increasing temperatures. However, in some instances the foundations for the
summary statements should have been made more transparent. While acknowledging
the essential role of expert judgment in scientific assessments, the PBL recommends to
improve the transparency of these judgments in future IPCC reports. In addition, the
investigated summary conclusions tend to single out the most important negative impacts
of climate change. Although this approach was agreed to by the IPCC governments for
the Fourth Assessment Report, the PBL recommends that the full spectrum of regional
impacts is summarised for the Fifth Assessment Report, including the uncertainties. The
PBL believes that the IPCC should invest more in quality control in order to prevent
mistakes and shortcomings, to the extent possible.
 o significant errors found in summary conclusions
N
The foundations for thirty-two IPCC Fourth Assessment summary conclusions
on the regional impacts of climate change have been investigated. These
conclusions show examples of projections of climate-change impacts on food,
water, ecosystems, coastal regions and health, for all the earth’s continents. These
conclusions have not been undermined by errors, although one of the conclusions
contains a minor inaccuracy: in hindsight, not 75 to 250 million people, but 90 to
220 million people are projected to be exposed to increased water stress due to
climate change in Africa, by 2020. Given the large uncertainties surrounding such
projections, this difference is not significant.
Provenance of summary statements needs to become more transparent in future
reports
Seven of the investigated 32 conclusions on the regional impacts of climate change
contain information that we were unable to sufficiently trace to the underlying
chapters in the IPCC Working Group II Report or to the references therein. For
two of these conclusions, our critical comments pertain to insufficiently founded
generalisations from existing scientific research, in both cases from local to regional
scales, and for one of them also from one type of livestock to livestock in general.
The PBL recommends to invest in the improvement and transparency of the
foundations of summary conclusions in future IPCC reports.

Executive summary

9

The regional chapters: one significant error and some comments
In the underlying regional chapters, in addition to the two already known errors
about the melting of the Himalayan glaciers and about the Dutch land area below
sea level, another significant error was found: a 50 to 60% decrease in productivity
in anchovy fisheries on the African west coast was projected on the basis of an
erroneous interpretation of the literature references. It appeared to be about a 50
to 60% decrease in extreme wind and seawater turbulence, with some effects on
the anchovy population that were not quantified. We found certain inaccuracies,
ranging from (very) small errors in numbers to imprecise literature references.
In addition, the PBL has some critical comments to make. One of these relates
to the fact that the report does not specify how many of the additional heatrelated deaths projected for Australian cities are actually attributable to climate
change – a sizeable fraction is due to demographic changes alone. However, these
shortcomings do not affect the investigated 32 summary conclusions or other parts
of the IPCC summaries.
Examples of negative impacts dominate at summary level
The IPCC Working Group II Report focuses on climate impacts, adaptation and
vulnerability. It was found that, in the IPCC’s highest level summary, the conclusions
that were derived from the regional chapters of the Working Group II contribution
to the Fourth Assessment Report single out examples of projections of negative
climate-change impacts. The IPCC authors considered these to be the most relevant
to policymakers. The PBL has labelled this as a ‘risk-oriented’ approach, which had
been implicitly endorsed by the governments that constitute the IPCC (including
that of the Netherlands). The PBL subscribes to the importance of an approach
that highlights what may go wrong under unmitigated climate change, but the
Working Group II Report lacked a clear explanation of the choice of approach and
its consequences. Alternatively, it could be argued that policymakers should be
presented with a complete picture in the Summaries for Policymakers, not just with
negative examples (without suggesting that potential positive effects cancel out
potential negative effects). We recommend that, for the Fifth Assessment Report,
the choice of the approach taken is made explicit. Moreover, we suggest that, at
Summary for Policymakers level, two separate sections are included dealing with
projected regional impacts on water, food, ecosystems, coastal regions and health:
 One section that describes the projected full range of climate-change impacts,
including uncertainties, positive impacts, and the relative contributions from
other important areas, such as industrialisation, population growth, and land
use;
 One section that describes the most important negative impacts, including
a worst-case risk approach, based on a clearly explicated risk-assessment
methodology.
No consequences for overarching conclusions
Our findings do not contradict the main conclusions of the IPCC on impacts,
adaptation and vulnerability related to climate change. There is ample
observational evidence of natural systems being influenced by climate change
on regional levels. The negative impacts under unmitigated climate change in the
future pose substantial risks to most parts of the world, with risks increasing at
higher global average temperatures.

10

Assessing an IPCC assessment

Other recommendations
Additional recommendations for future IPCC publications include:
 Create a public website for the submission of possible errors found in the
published reports;
 Provide stronger underpinning of generalisations of case studies to entire
regions or sectors, also making use of regional modelling studies;
 Ensure that statements that attribute impacts to climate change are well
founded in scientific research, including systematic observations, modelling
and statistics. The climate change component of impacts should be carefully
characterised.
 Be careful with phrasing of statements that could be perceived by readers as
heightening the projected impacts of climate change;
 IPCC governments should provide financial support for hiring chapter assistants
to assist with quality control;
 Assure that the reviews of all draft texts are fully covered by several expert
reviewers;
 Strengthen the expert and government reviews of the foundation for and
provenance of statements in the summaries;
 IPCC governments should increase their investments in climate-change
observations and modelling in developing countries.
The Dutch Parliament resolution and the subsequent assignment given to the PBL
In January 2010, worldwide media attention was given to two errors that were
discovered in a part of the Fourth Assessment Report of the IPCC of 2007: an
erroneously high rate of melting of the Himalayan glaciers, and an erroneously high
percentage of land area in the Netherlands lying below sea level. The commotion
in the Dutch media and the subsequent discussion at the political level in the
Netherlands led to a resolution by the Dutch Parliament on 28 January 2010, which
declared in the preamble that the reliability of the IPCC should be undisputed but
was now at issue. In the resolution, the parliament required that the government
would instruct the Netherlands Environmental Assessment Agency (PBL) to provide
a new update on climate science, including the implications of the said errors.
Based on this resolution and the debate in parliament, the Minister for the
Environment decided to limit her question to PBL to an investigation in the
implications of possible errors in the regional chapters of the IPCC report of 2007
on climate-change impacts, adaptation and vulnerability, paying specific attention
to the Himalayan glaciers.
Specification of the assignment
The PBL, subsequently, broadened the assignment somewhat by including an
investigation of the conclusions on regional impacts, as described in the highest
summary level of the combined IPCC 2007 reports (the ‘Synthesis Report’). In
addition, the investigation report would also include annexes on the Himalayan
glaciers, on the percentage of land area in the Netherlands lying below sea level,
and on the protection of the Netherlands against sea level rise. The last annex
was added, since the errors about the Himalayan glacier melting and about the
percentage of Dutch land area below sea level had triggered a political discussion
on whether Dutch policies dealing with sea level rise would need to be revised.

Executive summary

11

For the ‘update on climate science’, as requested by the Dutch Parliament, we refer
to a PBL study ‘News in Climate Science and Exploring the Boundaries’ (December,
2009). That report, drawn up together with the Royal Netherlands Meteorological
Institute (KNMI) and Wageningen University and Research Centre (WUR), revisited
the IPCC conclusion that global warming since the middle of the 20th century is very
likely to have been due to human influence on the global climate. This statement
was found to be robust, even when taking into account peer-reviewed scientific
literature expressing doubts on this relation. We took this and other findings of
the Working Group I as a point of departure for our investigation into the IPCC
statements on the projected regional impacts of climate change.1)
Given the constraints regarding time and capacity, it was not possible for the PBL to
check a hundred per cent of all texts and references in the eight regional chapters
of the Working Group II Report for errors, considering that it had taken hundreds of
authors and reviewers to produce the report over the course of five years. Instead,
we limited ourselves to the IPCC summary statements, and framed the central
questions of this report as follows:
Are the summary conclusions on regional impacts well founded on the underlying
chapters and literature references? Are there errors in statements that have travelled
from the scientific literature references and/or the main texts through to the summary
conclusions? If errors are found, do they affect the validity of these conclusions? What
recommendations can we derive from our investigation in order to further improve the
quality of the assessment process for the Fifth Assessment Report (due in 2014)?
The role of IPCC assessments and the role of the PBL
Policymakers need to be well informed about the state of the art in climate science.
Obtaining a reliable overview of the scientific literature, with its many different and
sometimes contradicting messages, is something policymakers can obviously not
achieve on their own. What is more, a knowledge base that is shared and accepted
as credible and legitimate by all parties involved, can be of great value to the policy
process. Therefore, in 1989, the UN member governments that constitute the
IPCC arranged a process in which worldwide groups of carefully selected expert
authors, periodically, would assess the ever growing amounts of scientific literature
on climate. Working within the confines of the IPCC, authors arrive at collective
judgments that are supposed to present the state of the art in climate science,
in a comprehensive, unbiased, non-policy-prescriptive way. The strict procedures
established by the IPCC member governments are to guarantee precisely this. The
credibility of an IPCC report depends both on the knowledge and integrity of the
author teams and on the thoroughness and expertise of expert and government
reviewers.
The PBL generally has an intermediate role in interpreting IPCC reports, so that
these can be applied in policy-making by the Dutch Government and Parliament.
In this case, the PBL has had to assess the reliability of some key messages derived
from a particular IPCC report; the Working Group II Report. To this end, we needed
1) There is a forthcoming publication by the Royal Netherlands Academy of Sciences (KNAW) that
will examine this fundamental part of climate science in more detail.

12

Assessing an IPCC assessment

to rely on the expert judgments of IPCC authors, to some extent, as it would have
been impossible to track down and assess all theory, measurements, observations,
model calculations, and all arguments that the IPCC authors may have used during
the five-year assessment period in order to arrive at certain conclusions. However,
we could investigate whether the key messages in the summary were being backed
up by material discussed in the main chapters of the IPCC assessments, whether
relevant information in the main chapters, in our opinion, had been founded
on sound scientific sources, and whether there would be any obvious errors or
mistakes in the (presentation of) the material investigated.
IPCC assessments are always snapshots in time – representative of the knowledge
of science as it is at that moment. As science proceeds, insights may change.
However, in our investigation, we have not included new scientific information that
has become available since the Fourth Assessment Report was produced – our task
was not to redo the IPCC assessment, but to investigate its underpinning at the
time of finalisation, in early 2007.
Focus of the investigation
The 2007 Fourth Assessment Report of the IPCC contains almost 3,000 pages. The
focus of our investigation was on only a small part of this report. The IPCC report
contains four volumes: the Working Group I Report on the physical science basis of
climate change; the Working Group II Report on climate impacts, adaptation and
vulnerability; the Working Group III Report on mitigation of climate change; and
a Synthesis Report, integrating the main outcomes of these three Working Group
reports.
The Working Group II Report contains 20 chapters, eight of which deal with the
regions Africa, Asia, Australia and New Zealand, Europe, Latin America, North
America, polar regions (Arctic and Antarctic), and small islands (that is, small-island
states).
In the Synthesis Report a table is presented, in its Summary for Policymakers,
describing ‘examples of some projected regional impacts’, derived from the eight
regional chapters in the Working Group II Report, each containing four statements,
which together form 32 statements (see Annex A for a copy of this table). We
have considered this table in the Synthesis Report as the highest summary level
presenting the most important regional impacts, with the highest visibility to the
policymakers.2)
The primary focus of this investigation, therefore, was on the foundation of these
32 statements, which are a subset of largely identical statements in the Summary
for Policymakers of the Working Group II Report.

2) The Synthesis Report was released a few weeks before the important world climate Summit of
the UNFCCC (the UN climate convention) in Bali in December 2007 (also indicated as CoP13). At this
conference, the UN parties to the convention took the first steps towards a long-term climate policy
strategy following the first commitment period of the Kyoto protocol.

Executive summary

13

Analysis of the main texts and the summaries
Each IPCC report consists of several layers. Each chapter in a Working Group report
contains a main text and its literature references, with an ‘Executive Summary’ up
front. Each Working Group report has both an extensive ‘Technical Summary’ which
synthesises and summarises the information of all chapters, and a brief ‘Summary
for Policymakers’.
We analysed the Executive Summaries of each of the eight regional chapters, and
checked their provenance from the relevant parts of the underlying texts in the
main chapters and their key references. We did the same with regional information
in the Technical Summary and the Summary for Policymakers of the Working Group
II Report. In addition, we checked how these conclusions were entered into the
main text of the Synthesis Report, and how they ultimately landed in its Summary
for Policymakers (see Figure 2.1).
Registration website
The PBL launched a public website that would be available for the course of one
month, in order to give all experts in the Netherlands the opportunity to contribute
to our investigation. We asked for submissions of possible errors found in regional
chapters of the Working Group II Report. By the end of that month, the PBL had
registered 40 submissions; however, most of them were about issues related to the
Working Group I Report. Two submissions qualified to be addressed in our report.
All submissions and PBL’s responses have been published on the PBL website.
Process
We invited the IPCC authors of the regional chapters of the Working Group II
Report to comment on our draft findings, and in our communication we went
through several iterations. In some cases, we also contacted the authors of cited
references. The draft of our report was also reviewed by selected internal and
external, national and international experts. Moreover, this project has been
conducted under the independent supervision of the Royal Netherlands Academy
of Arts and Sciences (KNAW).
Methodology
This investigation has resulted in a report on:
1. findings referring to the foundation for the 32 summary conclusions on regional
impacts in the Synthesis Report (main objective);
2. additional findings at lower levels, such as those referring to the foundations
for statements on regional impacts in the Technical Summary and Executive
Summaries of the regional chapters, and also some findings on parts of the main
text that we came across while working on other findings. As our focus was on
tracing possible errors that would affect the key messages to policymakers, we
were unable to systematically check all statements in the main texts.
For our investigation, we used several criteria for assessing the quality of IPCC
statements. First of all, we employed a distinction between obvious factual ‘errors’
– which we believe require an erratum on the IPCC Fourth Assessment web pages
– and ‘comments’, the latter being critical remarks, made from our specific position
of having to assess these statements on behalf of Dutch policymakers. These

14

Assessing an IPCC assessment

criteria were developed ‘on the job’, while categorising our findings. In total, there
are nine criteria, and associated with these are two types of ‘errors’ (E1-E2) and
seven types of ‘comments’ (C1-C7):
 E1: Inaccuracy of a statement;
 E2: Inaccuracy of literature referencing;
 C1: Insufficiently substantiated attribution, when a certain impact had been
attributed to climate change without convincing substantiation;
 C2: Insufficiently founded generalisation, when case studies had been
extrapolated to whole regions/sectors without convincing foundation;
 C3: Insufficiently transparent expert judgment, when we could not trace the
reasoning behind a statement from underlying texts or literature references
given;
 C4: Inconsistency of messages, when found between the different layers in the
IPCC report;
 C5: Untraceable reference, when a literature reference could not be found;
 C6: Unnecessary reliance on grey referencing, when reference had been made
only to grey literature (literature other than from peer-reviewed journals), while
adequate peer-reviewed scientific journal references would have been available;
 C7: Statement unavailable for review, when new information had been
introduced after the last review, but was not clearly derived from a content issue
raised in this last review.
When a certain statement in the IPCC Fourth Assessment Report would meet all the
criteria, the statement was considered to be well founded and reliable.
Melting of Himalayan glaciers
In response to the request from the Dutch Minister for the Environment to provide
an update on the melting of Himalayan glaciers, we performed an assessment of
the new scientific information that had become available since the publication of
the IPCC’s Fourth Assessment Report (see Annex B). The average, region-wide,
glacier area retreat rate is probably between 0.1 and 0.5% per year. Although the
glacier area will shrink substantially this century, especially in the most vulnerable
eastern zone of the Himalayas, glaciers (such as Khumbu and Imja), will not
disappear entirely, or even mostly, by 2035, as stated in the Working Group II
Report.
The annual contribution from Himalayan glaciers to sea level rise is 0.06±0.04 mm
or ~2% of the current annual sea level rise of 3.1±0.7 mm. The rate of Himalayan
glacier melting appears to have increased over time, but so has sea level rise, and
therefore the 2% contribution may still apply.
Dutch land area below sea level
The error about the percentage of Dutch land area below sea level stems from a
text submitted by the PBL (see Annex C). The Working Group II Report states that
55% of Dutch land area is below sea level. This should have read that 55% of the
Netherlands is prone to flooding: 26% of the country is at risk because it lies below
sea level and another 29% is susceptible to river flooding.

Executive summary

15

Sea level rise and its consequences for the Netherlands
Errors in the IPCC report regarding the Himalayan glaciers and the Dutch land
area below sea level have triggered a discussion on whether Dutch policies that
deal with sea level rise should be revised (see Annex D). The answer is clearly ‘no’:
the contribution from melting Himalayan glaciers to sea level rise is very limited
(around 2%) and the policy concerning safety against flooding in the Netherlands,
as formulated in the National Water Policy Plan 2009-2015 (Ministry of Transport,
Public Works and Water Management, 2009) and elaborated in the Delta
Programme, is well founded on the available knowledge on climate change, sea
level rise, river discharges and uncertainties. The current discussions on the IPCC
have no effect on the assumptions under the National Water Policy Plan and the
Delta Programme.

16

Assessing an IPCC assessment

Part I – Main findings

Part I – Main findings

17

18

Assessing an IPCC assessment

Introduction
1.1 Scientific assessment in aid of policy

1

The characteristics of climate change qualify it as the most complex type of
policy problem. It resists direct perception, hence, making policymakers and the
public depend on experts to describe its potentially problematic features. It is a
phenomenon, the problematic features of which may manifest themselves within
a matter of decades, while preventative measures assume more direct action. It
is also a global phenomenon which can only be handled effectively through joint
participation of all countries (climate change is a classic ‘free rider’ problem).
Addressing climate change is not a matter of controlling one clearly identified
source, as was the case with the depletion of the ozone layer, but rather of
controlling a multiplicity of sources. This characteristic of ‘wickedness’ (cf. Rittel
and Webber, 1973) makes that climate science plays a very central role in the
policy debate. To make things worse, a full understanding of the subject depends
on the input from a large variety of scientific fields, with differing degrees of
standing knowledge relevant to measuring and understanding processes of climate
change, its effects and possible remedial strategies. Since both the stakes and the
uncertainties are high, the problem of climate change needs to be addressed by
using ‘post-normal science’; a problem-solving strategy that involves paying specific
attention to qualitative judgments and value commitments permeating science
(Funtowicz and Ravetz, 1993).
 he role of assessments
T
Historically, the IPCC was established to present international policymakers with
undisputed, policy-relevant knowledge, while acknowledging uncertainties (see
e.g. Petersen, 2006). Since its inception in 1989, the IPCC has been effective at
this. However, the science on climate-change impacts is still beset with large
uncertainties, some of which are irreducible. Next to the nature of climate science,
it is relevant to remember that perceptions of climate-change risk vary widely, both
across the globe and within societies (cf. e.g. Hulme, 2009).
Hence, the issue of climate change is complex, both in causes and effects and in
terms of policy implications. The uncertainties, different interests and different risk
perceptions pertaining to all aspects of causes, consequences and response options
are inextricably linked to the issue of climate change.
In such situations, scientific assessments are the institutional vehicle for presenting
policymakers with information that would help them define and decide on policy
strategies. Scientific assessments present the state of knowledge relevant to a

Introduction

19

particular policy problem. It is a subtle tool. For complex problems such as climate
change, recent history has shown that a traditional, linear science-for-policy model
– one that is based on a clearly formulated policy question followed by a sequence
of scientific research from hypotheses to experiments and, finally, agreed on
answers – will not work. The nature of the issue is too complex and uncertainties
will always remain. However, the fact that uncertainties about climate change
are large does not mean that scientific information should not or could not play a
role in decision-making. On the contrary, well-formulated scientific information is
considered a crucial component in addressing complex policy problems.
The tool of scientific assessment, such as that implemented through the
Intergovernmental Panel on Climate Change (IPCC), has been developed as a way
of organising the science–policymaker dialogue (see, e.g., Social Learning Group,
2001). Scientific assessment is the analysis and review of information derived from
existing research, in the context of specific decision-making processes, in order to
better understand the wide implications of a policy problem and to identify and
evaluate possible response actions. Such assessment usually does not entail new
research. The design of the process of scientific assessment needs to ensure that
uncertainties are properly addressed and that different societal perspectives are
included in the process, so that politicians may obtain a better sense of the possible
futures facing the world, and of the choices that could be made. One of the
functions of independent assessments is to assess and communicate uncertainty,
and to make sure that the problem is analysed from different value perspectives.
The proper course of action then is for policymakers to decide what – if anything –
should be done.
The IPCC assessment process
This report investigates the quality of IPCC’s key messages on regional impacts
of climate change, which were the outcome of a long production process. We
have not reiterated the elaborate IPCC production process with its procedures for
review. This has been described extensively in the Principles Governing IPCC Work
(http://www.ipcc.ch/pdf/ipcc-principles/ipcc-principles.pdf; for a summary, see
http://www.ipcc.ch/pdf/ipcc-principles/IPCC%20Procedures.pdf). These procedures
are meant to guarantee a responsible assessment process.
It is important to note, however, that an IPCC assessment is not the product
of an extensive bureaucratic operation. Instead, its organisational structure is
aptly characterised as being a network organisation that spans the whole globe.
Scientific experts from varying continents collaborate and assemble overviews
of the knowledge in their respective fields. The quality and integrity of the IPCC
process depends, to a large extent, on the quality and integrity of its procedures,
and, hence, on the commitment of all contributing authors to adhere to those
rules and procedures. In response to experiences and criticism, these procedures
have been regularly upgraded over the course of more than 20 years to guarantee
increasing assurance of high levels of quality and integrity.
We have chosen not to elaborate on the institutional set up of the IPCC, but have
considered it a given. For the present report, we had set ourselves the task of
assessing the transparency with which scientific facts on climate-change impacts

20

Assessing an IPCC assessment

have ‘travelled’ from their source (e.g., scientific peer-reviewed literature) to their
destinations: the Summaries for Policymakers. This final destination of scientific
facts within the IPCC process has driven the investigation presented in the present
report. The IPCC process should deliver useful and reliable knowledge (cf. Morgan,
2010) to policymakers; one route to check this – followed for this report – is by
tracing summary conclusion back to their origins.
The role of expert judgment
This report deals with the questions that have been raised on the reliability of the
assessment process run by the IPCC. In particular, questions have been raised
concerning the Working Group II contribution to the Fourth Assessment Report
with its focus on Impacts, Adaptation and Vulnerability and including the regional
chapters that were home to the two errors that caused the upset in the media
and in society, in the winter of 2009–2010. This report has been based on the
assumption that the quality of the IPCC scientific assessment can be measured by
tracing statements on the effects of climate change, for example, as presented
at the highest level of summaries for policymakers (in the Synthesis Report and
the Working Group II Report), back to their original sources. There is obviously
a limit to the traceability of such statements. Given the mismatch between the
state of standing knowledge on complex environmental phenomena, on the one
hand, and the questions raised by policymakers, on the other, those conducting
an assessment often have to arrive at so-called expert judgments: a weighing of
understandings and analyses to come up with the best possible assessment of
observations, or of what one might expect to happen. Such expert judgments are a
crucial component of any assessment. Given the fact that these judgments cannot
be traced back to any one source in the (peer-reviewed) literature, the quality of
these assessments grows as the expert judgments are made more transparent, that
is, by explaining how particular conclusions have been arrived at. This procedure
allows others to follow a certain line of reasoning, similar to how this generally
is done in articles published in scientific journals. Another way of enhancing the
quality of expert judgments is to present them in the form of a deliberation, stating
the way in which key conclusions have been arrived at. This then allows other
scientists to contradict inferences, or come up with alternative explanations.
Expert judgments are inextricably linked to assessments and are used in all phases
of the IPCC assessment process. The lead authors of the regional chapters of the
Working Group II Report were supposed to have read all the literature relevant
to that particular region, and to be familiar with the environmental policy issues
of that region. Based on their individual expertise, the lead authors collectively
arrived at judgments on the impacts of climate change. The review process of the
IPCC relies on two rounds of expert review and one round of government review,
as well as on Review Editors, who umpire the review process. The objective of this
process is to ensure a balance in the lead authors’ judgments as these are published
in the report. The chapters themselves are very detailed. This explains the need for
summary statements that convey the most important conclusions to policymakers.
The Coordinating Lead Authors drafted the Summary for Policymakers of the
Working Group II Report, and a smaller group of authors (consisting of two regional
Coordinating Lead Authors, the two Co-Chairs and the head of the Technical
Support Unit of Working Group II) was involved in the drafting of the Summary for

Introduction

21

Policymakers of the Synthesis Report. Moreover, in both of these Summaries for
Policymakers, the role of expert judgments had been crucial for determining the
main policy-relevant findings from the underlying chapters.
Since it is impossible for most readers of IPCC reports to understand and check
every reference, all data, models, calculations and measurements, they must be
able to rely on the quality control and quality assurances given by the assessment
process. To some extent, those readers have no choice other than to assume an
expert judgment to reflect the best of current knowledge. However, they should
be able to trace the plausibility of a judgment in a Summary for Policymakers back
to the main text and references. When dealing with issues as complex as climate
change, expert judgment is inevitable. But precisely because this is a matter of
sound judgment, there is a virtue in maximising the possibility for others to verify
and validate the analyses made by the IPCC authors.

1.2 Some background on IPCC Working Group II
Since the Third Assessment Report (finalised in 2001), the work of the IPCC has
been organised into the following three distinct Working Groups:
 Working Group I addresses the physical science of climate change;
 Working Group II addresses the impacts of and vulnerability to climate change
and the means for adaptation;
 Working Group III addresses the mitigation of greenhouse gas emissions.
Working Group II addresses impacts, vulnerability and adaptation to climate change
with respect to human and natural systems. This particular focus calls for the
mobilisation of authors mostly from physical, geographical and ecological sciences,
but also from the social sciences. The particular thematic focus of Working Group
II immediately complicates its assessment task. The interactions within biological
systems and between biological systems and physical chemical and social systems,
are very complex. What is more, the patterns of interaction vary, depending on
local situations and on human choices and actions.
Literature on climate impacts is not only very diverse, and the intensity of academic
scholarship is divided very unevenly over the globe. This increases the challenge
of making good assessments of what the impacts, effects and vulnerabilities of
climate change are in those regions that have the least data to draw on. This point
is well illustrated by a map from the IPCC report itself (see Figure 1.1).
Furthermore, impact analyses are not only spread unevenly around the world,
it is also a field of study where often insights are found outside of the strict
academic tradition. For instance, many impact analyses are presented in reports
by governments and inter-governmental or private, non-academic institutions –
given the fact that not all topics in these reports are equally suitable for, or indeed,
targeted to scientific publication – which complicates the task of assessors. The
IPCC guidelines specifically allow for this literature also to be assessed (in order to
provide relevant input about local and sectoral situations and to minimise bias in
information between developing and industrialised countries, for instance, because

22

Assessing an IPCC assessment

Observed changes in biological and physical systems

Figure 1.1

Temperature change oC
1970 – 2004
-1.0 -0.2

0.2

1.0

Observed data series

2.0

3.5

Physical systems (snow, ice and frozen ground;
hydrology; coastal processes)
Biological systems (terrestrial, marine, and freshwater)
* Circles in Europe represent 1 to 7,500 data series.

Europe *
1 – 30
31 – 100
101 – 800
801 – 1,200
1,201 – 7,500

Locations of significant observations of changes in physical an biological systems together
with changes in surface air temperature, over the 1970–2004 period. Source: Climate
Change 2007: Impacts, Adaptation and Vulnerability. Working Group II Contribution
to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change,
Figure 1.8. Cambridge University Press.

impacts in developing countries are reported even less in scientific peer-reviewed
journals).
The scientific uncertainties that the IPCC, including Working Group II, are
confronted with, are large and deep. Since its inception, the IPCC has given
increasing attention to the management and reporting of uncertainties (Swart
et al., 2009). The socio-ecological systems studied by Working Group II are not
always suitable for numerical modelling; hence, there is a lack of information on
quantitative uncertainty. Impact analyses or the study of vulnerabilities often
depend on empirical observation using different scientific techniques, such as field
research or case study analysis. Because of the uncertainties involved, it is difficult
to establish which impacts occur, and why and to what extent observed effects
could be attributed to climate change. Often authors need to make assessments
of the risks that are associated with various levels of climate change, rather than
indicating best-estimates and clearly defined uncertainty levels. In order to estimate

Introduction

23

the possible risks of climate change, also events with very low probability but with
large consequences have to be dealt with.
In the context of this report, it is important to understand the mandate of
Working Group II. Formally, Working Group II ‘assesses the scientific, technical,
environmental, economic and social aspects of vulnerability (sensitivity and
adaptability) to climate change and the negative and positive consequences for
ecological systems, socio-economic sectors and human health, with an emphasis
on regional, sectoral and cross-sectoral issues.’ (see http://www.ipcc.ch/working_
groups/working_groups.htm)

1.3 Scope, objectives and limitations of this report
In January 2010, worldwide media attention was drawn to two errors that were
discovered in a part of the Fourth Assessment Report of the IPCC of 2007: an
erroneously high rate of melting of the Himalayan glaciers, and an erroneously high
percentage of land area in the Netherlands lying below sea level. The latter error
was not the fault of the IPCC but of the PBL (see Annex C). The commotion in the
Dutch media and subsequent discussions at the political level in the Netherlands led
to a resolution by the Dutch Parliament on 28 January 2010, which declared in the
preamble that the reliability of the IPCC, which should be undisputed, was now at
issue.
In the resolution, the parliament required that the government would charge the
Netherlands Environmental Assessment Agency (PBL) with providing a new update
of climate science, including the implications of the said errors. Based on this
resolution and the debate in parliament, the Minister for the Environment decided
for the PBL to limit its investigation to the implications of possible errors in the
regional chapters of the IPCC report of 2007 on climate-change impacts, adaptation
and vulnerability.
The assessment presented in this report was based on an investigation of the
original sources of the IPCC’s Fourth Assessment summary conclusions (in
particular, the Synthesis Report, Table SPM.2, see Annex A) that pertain to a part
of one of the Fourth Assessment Reports (that is, Chapters 9 to 16 of the Working
Group II Report), and was aimed to derive general observations and identify
possible improvements.
Two additional tasks were taken on. First of all, the Minister for the Environment
asked the Netherlands Environmental Assessment Agency (PBL) to make an
assessment of the scientific literature on Himalayan ice, published since 2007.
This was triggered by the need to communicate what scientists do know about
this topic; after all, the public turmoil concerned the IPCC error that stated that
all Himalayan ice would be melted by 2035. The results from this assessment are
presented in Annex B.
Second, the parliamentary debate also showed concerns over the possible
repercussions of the state of knowledge on the issue of sea level rise for climate-

24

Assessing an IPCC assessment

change adaptation measures in the Netherlands. Therefore, the state-of-the-art
scientific foundation on this issue is summarised in Annex D.
The PBL has taken the utmost care to prevent possible prejudice in their present
assessment of the conclusions derived from the regional chapters of the Working
Group II Report. A crucial mechanism to avoid any prejudice was the installation
of a supervisory committee under the responsibility of the Royal Netherlands
Academy of Arts and Sciences (KNAW). The KNAW was asked to supervise the
quality of this PBL study. The KNAW appointed a scientific committee, whose role
it was to judge whether the PBL performed its research without prejudice and
according to established procedures of quality control.1)
Realistically speaking, a thousand-page assessment by hundreds of authors
involving thousands of reviewers conducted within a limited timeframe could
hardly be expected to be free of errors. Therefore, it is to be expected that some
inaccuracies, insufficiently justified statements or other irregularities, escape even
the most thorough drafting and review procedures. Our evaluation methodology
was to take the statements in the various IPCC summaries as starting points and
consider their source of origin (where did they come from?). A full evaluation of
the reliability of statements would entail redoing the original IPCC assessments.
Our limited evaluation could never have repeated or improved on the IPCC drafting
and review process in a comprehensive way, nor was it our aim to redo the IPCC
assessment. Nevertheless, we think that such a limited evaluation by scientifically
educated policy analysts, who are not experts in climate-impacts science, is a useful
means for the IPCC to receive feedback from an extended peer community (cf.
Funtowicz and Ravetz, 1993). Through this evaluation, we reflect on the qualitative
judgments and value commitments that necessarily permeate the IPCC assessment
process.
Without recruiting experts with the range of expertise and following the careful,
multi-step review that has characterised the IPCC reports, any review of the
contents of IPCC chapters only has limited potential. The current PBL assessment,
therefore, extends only to checking whether observations and interpretations had
been captured conscientiously, how statements had been substantiated within
IPCC reports, and to evaluate the transparency with which important expert
judgments had been made. The analysis focused on investigating the ‘internal’
consistency of the report, and did not perform new, fresh reviews of the evidence
available at the time the Fourth Assessment Report was produced. In sum, the
analysis focused on assessing statements by looking for errors (inaccuracies in
statements and referencing) and by applying a short list of quality criteria (see
Chapter 2), which was developed specially for the purpose of this analysis.
In sum, the central questions addressed in this report are the following:
Are the summary conclusions on regional impacts well founded on the underlying
chapters and literature references? Are there errors in statements that have travelled
1) The terms of reference of the supervision by the Royal Netherlands Academy of Arts and Sciences
can be found at http://www.knaw.nl/pdf/Instellingsbesluit_Toezicht_Cie_PBL.pdf (in Dutch).

Introduction

25

from the scientific literature references and/or the main texts through to the summary
conclusions? If errors are found, do they affect the validity of these conclusions? What
recommendations can we derive from our investigation in order to further improve the
quality of the assessment process for the Fifth Assessment Report (due in 2014)?

1.4 Structure of the report
Part I of the report contains the main findings. Chapter 2 describes the
methodology of our assessment. Chapter 3 presents the results from our analysis of
IPCC Fourth Assessment’s statements on regional climate-change impacts. Chapter
4 contains our recommendations for the IPCC assessment process, following from
our analysis in Chapter 3. Part II provides our detailed analysis of the regional
chapters and the summaries.
References
Funtowicz, S.O. and Ravetz, J.R., 1993. ‘Science for the post-normal age’, Futures 25: 739–755.
Hulme, M., 2009. Why We Disagree about Climate Change: Understanding Controversy, Inaction and
Opportunity. Cambridge: Cambridge University Press.
Morgan, M.S., 2010. ‘Traveling facts’, in P. Howlett and M.S. Morgan, eds., How Well Do Facts Travel?,
Cambridge: Cambridge University Press. (http://www2.lse.ac.uk/economicHistory/pdf/Morgan/
Travelling%20facts%20_2_.pdf)
Petersen, A.C., 2006. Simulating Nature: A Philosophical Study of Computer-Simulation Uncertainties
and Their Role in Climate Science and Policy Advice. Het Spinhuis Publishers, Apeldoorn and
Antwerp. Download from http://hdl.handle.net/1871/11385.
Rittel, H.W.J. and M. Webber, 1973. ‘Dilemmas in a general theory of planning’, Policy Sciences, 4 (2):
155-169.
Social Learning Group, 2001. Learning to Manage Global Environmental Risks, Volume 1: A
Comparative History of Social Responses to Climate Change, Ozone Depletion, and Acid Rain,
Cambridge, MA: MIT Press.
Swart, R., L. Bernstein, M. Ha-Duong, A. Petersen, 2009. ‘Agreeing to disagree: uncertainty
management in assessing climate change, impacts and responses by the IPCC’, Climatic Change
92: 1–29.

26

Assessing an IPCC assessment

Methodology
2.1 Investigative approach

2

The two primary aims of the investigation were to assess, first of all, whether errors
would affect the content of the IPCC’s key statements on regional climate-change
impacts, and, second, whether these key statements would meet seven quality
criteria formulated by the PBL (see Section 2.2). These key statements were taken
from the Summary for Policymakers of the IPCC Working Group II Report (2007)
and, in part, presented in Table SPM.2 of the IPCC Synthesis Report (2007) (see
Annex A). Figure 2.1 offers a graphical explanation of the area of investigation,
Figure 2.1A places the statements on regional impacts of climate change in the
context of the whole IPCC Fourth Assessment Report, and Figure 2.1B represents
the approach to the chapter analysis. Table SPM.2, containing 32 bulleted
statements on the most important projected impacts (four per region), is located at
the top of a ‘pyramid’.
A secondary aim of our investigation was to identify errors and assess the quality
of statements on regional climate-change impacts at levels below the Summary for
Policymakers of the Working Group II Report.
In Figure 2.1B, the ‘pyramid’ shows the analysis levels that we used in our
investigation. The Working Group II 2007 Report itself contains four layers. Level
1 contains the main chapter texts, level 2 contains the Executive Summaries
of the individual chapter texts, level 3 contains the Technical Summary (giving
an extensive overview of the whole report), and level 4 is the Summary for
Policymakers (a short document, approved line-by-line by the governments
involved in the IPCC). Each chapter text refers to hundreds of literature sources on
which the assessment was founded (‘level 0’). In some cases, we also investigated a
few sources that were mentioned in the literature references (‘level −1’)
The Synthesis 2007 Report has only two layers, a main report, divided into six
topics, and a Summary for Policymakers. The information in the Synthesis Report
has a higher level of aggregation than that in the Working Group Reports (level 5).
The information in the Summary for Policymakers of the Synthesis Report has the
highest level of aggregation (level 6).
The heuristic approach that was followed by PBL policy analysts involved in this
study, was to start by reading the Executive Summaries (level 2) of the regional
chapters with a critical eye, and from there work upwards to level 6 (Summary for
Policymakers of the Synthesis Report) as well as downwards, to level −1. We made

Methodology

27

Figure 2.1

Stucture of the Fourth Assessment Report and regional chapter analysis
A. Structure of the Fourth Assessment Reports

Working Group I
report
The Physical
Science Basis

The AR4 Synthesis
Report

Working Group II
report
Impacts, Adaptation
and Vulnerability

Working Group III
report
Mitigation of
Climate Change

Table SPM.2

Example of some
projected regional
impacts
(see Annex A)

B. Regional chapter analysis pyramid
Level of aggregation
Level of
detail

6

5

4

3

2

1

0

-1

Summary for policy
makers (SPM)
Synthesis report (SYR)

Summary for policy makers (SPM) Synthesis report (SYR)
Table SPM.2, Examples of some projected regional
impacts – see Annex A

Synthesis report main text

Synthesis report 2007
Topic 3 (Climate Change and its impacts), section 3.2,
para 3.3.2, Impacts on Regions, p.50 and 52

Summary for policy makers
Working Group II

Summary for policy makers Working Group II 2007 report
(SPM WG II) p. 13 – 15.

Technical summary Working Group II
Executive summaries chapter texts
Working Group II

Regional chapter texts Working Group II

Literature references

Technical Summary Working Group II 2007 report
• Section 4.2, Regional Impacts, Adaptation and
Vulnerability p. 48 – 58
• Box TS.6, The main projected impacts for regions,
p. 59 – 63
• Tabel TS.4, Example of regional impacts, p.67
Regional chapter including Executive summaries
• Ch. 13 Latin America
• Ch. 9 Africa
• Ch. 14 North America
• Ch. 10 Asia
• Ch. 15 Polar regions
• Ch. 11 Australia –
(Arctic and Antarctic)
New Zealand
• Ch. 16 Small Islands
• Ch. 12 Europe

Underlying literature

Figure 2.1A shows the structure of the IPCC’s Fourth Assessment Report, 2007. The Working
Group II Report contains eight chapters (from a total of 20 chapters) describing the impacts
in eight world regions. The main conclusions of the reports by the three Working Groups
were summarised per report, and also included in the Fourth Assessment Synthesis Report.
The last contains examples of some projected regional impacts; see Table SPM.2 (Annex A).
This table was the starting point in our investigation. Figure 2.1 B shows the approach to
the regional chapter analysis. The statements from Table SPM.2, at the top of the ‘pyramid’,
were analysed by investigating their foundation in the five consecutive levels, and in the
relevant underlying literature.

28

Assessing an IPCC assessment

sure that we covered all the bulleted statements in Table SPM.2, looking for any
possible weaknesses, thus, effectively working from the top layer of the Synthesis
Report down to the more detailed regional chapter texts of the Working Group II
Report, and in many cases to the underlying key literature references. In our search
for the underpinning of statements, we also looked for evidence in the introductory
and sectoral chapters of the Working Group II Report and, when this was deemed
necessary, in the Working Group I Report.
 egistration website
R
The PBL launched a public website that would be available for the course of one
month, in order to give all experts in the Netherlands the opportunity to contribute
to this investigation. We asked for submissions of possible errors found in regional
chapters of the Working Group II Report. By the end of that month, the PBL had
registered 40 submissions; however, most of them were about issues related to the
Working Group I Report. Five submissions were related to the regional chapters of
the Working Group II Report; two of those five were included in our report, three
submissions did not qualify for inclusion. All submissions and PBL’s responses have
been published on the PBL website.
 rocess
P
We invited the IPCC authors of the regional chapters of the Working Group II
Report to comment on our draft findings, and in our communication we went
through several iterations. In some cases, we also contacted the authors of cited
references. The draft of our report was also reviewed by selected internal and
external, national and international experts. Moreover, this project has been
conducted under the independent supervision of the Royal Netherlands Academy
of Arts and Sciences (KNAW).

2.2 Typology of errors and qualifications
The typology of errors and qualifications (‘comments’) presented here was arrived
at through an inductive process – the results from which are outlined in this section.
A distinction was made between ‘major’ and ‘minor’ errors or comments. Issues
were labelled to be ‘major’ when we judged them to have a significant quantitative
impact on statements pertaining to food, water, ecosystems, coastal areas, and
health.
 rrors
E
We used a sharp definition: ‘errors’ were considered inaccuracies in the IPCC Fourth
Assessment Report that we believed to require repair through an erratum or a
redoing of the assessment of the specific issue at hand. Experts – in particular, PBL
policy analysts and IPCC authors – reached a high level of agreement on what they
considered to be an ‘error’ in this precise sense. Both the Himalayan glacier melting
by 2035 and the 55% of the Netherlands that was said to be below sea level, were
considered to be such ‘errors’.

Methodology

29

We distinguished two types of errors:
E1. Inaccurate statement
Statements should be an accurate and correct reflection of the material found
in literature, augmented by the expert judgment of a writing team. If a statement appeared to be factually incorrect, then we considered whether this was
a minor imprecision or inaccuracy or if a major mistake had been made that
could have led to misinterpretations by policymakers. We applied the following
subcategories to this type of error:
E1a) Inaccuracies that can be corrected by issuing an erratum – This would be
mistakes that could be simply corrected without new scientific assessment
considerations, such as: typographical errors, incorrect phrasing of part of a
sentence, wrong dimensions, and wrong reference years. The error on the
percentage of Dutch land area below sea level belongs to this subcategory (see
Annex C).
E1b) Errors that would in fact require a redoing the assessment of the specific issue
at hand, such as: establishing a new range of numbers by revised calculations
from the reference sources available during the assessment period, and/ or
rephrasing of the expert judgment including its uncertainty labelling.1) The
Himalayan glacier error belongs to this subcategory (see Annex B).
E2. Inaccurate referencing
References used in statements should be correct. Whenever we found a reference to a wrong source, or when the source was not correctly cited, we qualified this to be ‘inaccurate referencing’. In all cases, an erratum was proposed.
Qualifications
In addition to the errors in the Fourth Assessment Report that could be corrected.
either in an erratum or through reassessment, we distinguished between seven
types of qualifications or comments that would signify the quality of a statement.
C1. Insufficiently substantiated attribution
In some cases, observed or projected impacts on systems are described as
being ‘due to climate change’, ‘temperature increase’ or other climate parameters. Since, often, there are other drivers that could be involved in causing such
an impact (e.g. population growth, industrialisation, migration, and changes
in land use and land cover), care must be taken to provide the proper context
in IPCC statements about the effects of climate change, by referring to other
causes and/or stresses that may influence the impacts discussed. Statements
that, in our judgment, did not sufficiently signal the presence of multiple
stresses, or suggested a one-sided attribution of impacts to climate change
while other factors also would have been expected to play a critical role, were
qualified as ‘insufficiently substantiated attribution’.

1) This, of course, excludes new findings in literature published after completion of the assessment
reports.

30

Assessing an IPCC assessment

C2. Insufficiently founded generalisation
In some cases, the impacts, for instance, on certain types of livestock or
countries, were generalised or extrapolated to include other livestock or entire
regions. In such cases, we believe authors ought to make clear why they think
that the evidence in the references justifies such generalisation. Statements,
for which such argumentation was lacking, and for which such evidence could
not be found in the report or in the references, were qualified as ‘insufficiently
founded generalisation’.
C3. Insufficiently transparent expert judgment
Expert judgment on results that can be found in the scientific literature, is an
essential part of the IPCC assessment process; this includes giving an indication
of uncertainties. An assessment is thus not a simple summary of the literature.
The expertise of an international group of authors adds value by judging the
importance and relative weight of the information from the literature. For
the sake of transparency, we believe that the reasoning behind these judgments, including the reasoning behind their levels of likelihood and/or confidence2), should be accessible to a non-expert reviewer who attempts to trace
the travelling of facts from the referenced literature to the Summaries for
Policymakers.
If, in our opinion, this was not the case, and this fact was found to be problematic, we qualified the particular judgment as an ‘insufficiently transparent expert
judgment’. This does not imply that this IPCC expert judgment was wrong, since
the authors as a group may have had their reasons, and may have considered additional information or knowledge that was not explicitly referred to.
However, in such cases a convincing substantiation of the statement was
lacking in the report.
C4. Inconsistency of messages
When moving up from statements in the main chapter texts to statements in
the Summary for Policymakers of a Working Group Report or of the Synthesis
Report, the sentences were inevitably shortened and the wording may have
needed some adjustments. However, Summary for Policymaker texts need
to be consistent with the main text, as required by the IPCC procedures. This
holds both for the meaning of the message as well as for the level of certainty.
SPM approval session decisions include a list of issues that need to be checked
for consistency with the Technical Summary and the main chapters in preparation of the final publication. Whenever we found that message content and/
or confidence level had changed when it moved from main text to Summary

2) The IPCC authors were in fact requested to produce a ‘traceable account’ of their assessment
of uncertainty in expert judgments in the Fourth Assessment process (see Guidance Notes for Lead
Authors of the IPCC Fourth Assessment Report on Addressing Uncertainties, July 2005, http://www.
ipcc-wg1.unibe.ch/publications/supportingmaterial/uncertainty-guidance-note.pdf). For the Third
Assessment, a similar guidance was given (Uncertainties in the IPCC TAR, 2000, http://stephenschneider.stanford.edu/Publications/PDF_Papers/UncertaintiesGuidanceFinal2.pdf). It is our general impression that this part of the guidance has never been fully implemented in the assessment process. Also,
the option to have separate traceable accounts underlying the reports has not been pursued.

Methodology

31

for Policymakers and also if there was any inconsistency between chapters, we
qualified this as an ‘inconsistency of messages’.
C5. Untraceable reference
As stated before, references used in statements should be accurate. If we
could not find the reference at all, we qualified this as an ‘untraceable reference’.
C6. Unnecessary reliance on grey referencing
Grey literature was an indispensable part of the Working Group II assessment
(this also applies to Working Group III), since not all relevant literature had
been published in peer-reviewed scientific journals. The grey literature needed
to meet quality standards before it could be used. There are huge differences
in the types of literature, going from thoroughly reviewed scientific books to
much less reviewed – or even non-reviewed – publications. When a substantial scientific statement only referred to a grey publication, although strong
peer-reviewed journal references could have been referred to at the time, we
qualified this to be ‘unnecessary reliance on grey referencing’.
C7. Statement unavailable for review
In the preparation of a Final Draft Report, the results from government and
expert reviews were included. This could mean that new literature sources
would be added in order to properly address the Second Order Draft (SOD)
review comments. Since reviews of a SOD often asked for the addition of
sources, such additions had to be considered as acceptable revisions. However,
if a new statement contained new literature findings that were not clearly
derived from a content issue raised in the Second Order Draft review, then
this new material would have been kept out of the review process. This is not
supposed to happen; expert reviewers or governments must have access to
all findings that will be part of the published report. In the cases in which these
new literature findings were added as described above, we qualified them as a
‘statement unavailable for review’.
It is important to note that the assignments to these seven qualifications were not
necessarily agreed on by all the experts involved in our investigation, including
the IPCC authors. These assignments were based on the judgment of PBL policy
analysts who had to establish the reliability of IPCC reports as part of their work at
the science–policy interface. The aim of this exercise has been to identify potential
areas for improvement in future IPCC assessments. The underlying assumption
being that times have changed and that, consequently, quality and transparency
requirements for the IPCC have been raised.

32

Assessing an IPCC assessment

Results and
Discussion

3

This Chapter consists of four parts. First, it presents the appraisal of the statements
on regional climate-change impacts that appear in the Summary for Policymakers of
the Synthesis Report. Second, an overview is given of errors found in the Working
Group II Fourth Assessment Report. Third, it addresses the approach followed by
Working Group to arrive at its summary conclusions on the regional impacts of
climate change. And fourth, a thematic overview is given of our comments on the
IPCC report, aimed at further improving the IPCC process.
Our detailed analysis of the regional chapters and summaries can be found in Part II
of the present report.

3.1 Status of IPCC summary statements on regional climate-change impacts
In this investigation we have not encountered errors that undermine the summary
conclusions of the Fourth Assessment Report of the IPCC on the regional impacts
of climate change. The Working Group II Report presents ample evidence of
observations of climate-change responses by physical and biological systems.
Furthermore, the consequences of climate change generally impose a serious
risk given the projections on future climate change of Working Group I. However,
there is considerable room for scientific debate on statements that refer to
climate-change impacts on a regional scale. We have found that, the more specific
statements become in time and space, the larger the uncertainties. In addition, the
body of scientific publications is smaller, in particular where it concerns local and
regional changes occurring in developing countries.
Of all the summary statements on the regional impacts in the Working Group II
Report – a subset of which is included in the 32 statements in Table SPM.2 of the
Synthesis Report – only one was found to be affected by a minor error. This minor
error, in our view, requires a response and reassessment by the IPCC.
Besides checking the summary statements for possible errors, we also tested
the quality of these statements with respect to a range of issues: attribution,
generalisation, transparency of expert judgment, consistency of messages,
traceability of references, credibility of references, and availability of statements
for review (see Section 2.2). We have one or more comments to make on seven

Results and discussion

33

Table 3.1

Results from PBL analysis for 32 statements contained in Table SPM.2 of the
Synthesis Report
Region
Africa

Asia

Australia and
New Zealand

Europe

Latin America

North America

Polar regions

34

Topic (and system/sector)
Water stress
(freshwater resources)
Yields from rain-fed agriculture
(food, fibre and forest products)
Cost of adaptation to sea level rise
(coastal systems and low-lying areas)
Arid and semi-arid land
(ecosystems)
Freshwater availability
(freshwater resources)
Flooding from sea and rivers
(coastal systems and low-lying areas)
Natural resources and the environment
(ecosystems)
Diarrhoeal disease
(health)
Biodiversity loss
(ecosystems)
Water security
(freshwater resources)
Production from agriculture and forestry
(food, fibre and forest products)
Risks from sea level rise and increases in storms and coastal flooding
(coastal systems and low-lying areas)
Inland flash floods, coastal flooding and erosion
(industry, settlement and society; coastal systems and low-lying areas)
Mountainous areas
(ecosystems)
Southern Europe
(most systems/sectors)
Heat waves and forest fires
(health)
Disappearance of tropical forest
(ecosystems)
Biodiversity loss
(ecosystems)
Productivity in crops and livestock
(food, fibre and forest products)
Water availability
(freshwater resources)
Western mountains and water resources
(freshwater resources)
Rain-fed agriculture
(food, fibre and forest products)
Heat waves
(health)
Coastal communities and habitats
(coastal systems and low-lying areas)
Biophysical effects
(ecosystems)
Impacts for human communities
(industry, settlement and society)
Infrastructure and indigenous lifestyle
(industry, settlement and society)

Assessing an IPCC assessment

Error in statement? Comments?
Yes (minor)
No
No

Yes (major)

No

Yes (minor)

No

No

No

Yes (major)

No

No

No

No

No

No

No

Yes (minor)

No

No

No

No

No

No

No

No

No

Yes (minor)

No

No

No

No

No

No

No

No

No

Yes (major)

No

No

No

No

No

Yes (minor)

No

No

No

No

No

No

No

No

No

No

Region

Small islands

Topic (and system/sector)
Vulnerability of ecosystems and habitats
(ecosystems)
Sea level rise
(coastal systems and low-lying areas)
Coastal conditions
(coastal systems and low-lying areas)
Water resources
(freshwater resources)
Invasion by non-native species
(ecosystems)

Error in statement? Comments?
No
No
No

No

No

No

No

No

No

No

of the 32 statements contained in Table SPM.2 of the Synthesis Report. If the
criteria, as used in this investigation for assessing the quality of statements, would
be systematically applied in future IPCC assessments, it would become easier for
the users of these assessments, such as the PBL, to establish the reliability of IPCC
conclusions. Table 3.1 summarises the results for the statements contained in Table
SPM.2 of the Synthesis Report.

3.2 Errors
I naccurate statements (code: E1)
Table 3.2 presents all newly found errors in the regional chapters of the Fourth
Assessment Working Group II Report, as well as the error regarding the Himalayan
glacier melting and the error about the percentage of Dutch land area below sea
level. Of these errors, only one travelled up to the Summaries for Policymakers. The
range for the number of people projected to be exposed to increased water stress
due to climate change in Africa, by 2020, turned out to have been erroneously
calculated by adding up the ranges for African sub-regions that were given in
the underlying reference. The author of the original study – who had recently
discovered this error – performed a recalculation, which led to a slightly smaller
uncertainty range (90 to 220 million people, instead of 75 to 250 million). We
considered this a minor mistake, since the difference does not significantly change
the message and the original paper did not provide the details that would have
allowed the correct computation of this range. In our view, the IPCC should quickly
respond to and assess this slight inaccuracy, even though it is not significant given
the uncertainties surrounding such projections.
Only one major new error was found, in Chapter 9 on Africa. A projected decrease
by 50 to 60% in extreme wind and turbulence over fishing grounds was mistakenly
represented as a 50 to 60% decrease in productivity as a result of changes in wind
and turbulence. However, this error and the remaining five other errors that were
newly found did not travel to the level of the summaries, and therefore do not
affect the summary conclusions. This is also the case for the two errors that were
discovered before we began our investigation (concerning the melting rate of the
Himalayan ice and the percentage of Dutch land area below sea level).

Results and discussion

35

Table 3.2

Inaccurate statements identified by the PBL in the IPCC Working Group II Report
and the Synthesis Report of 2007a

Region
Africa

Asia

Australia and
New Zealand
Europe

Latin America

Major/
minor
error
Minor

Pages
Error
11 (SYR), Increased water stress by 2020
13, 435, 444 ‘between 75 and 250 million people are project to
be exposed’ should be ‘between 90 and 220 million
people are projected to be exposed’ (see Section 5.1)
435, 444
Minor
Increased water stress by 2050s
‘350-600 million people’ should be ‘350490 million people’ (see Section 5.2)
448
Productivity decrease in fisheries under doubling of CO2 Major
‘extreme wind and turbulence could decrease
productivity by 50-60%, while turbulence will
probably bring about a 10% decline in productivity in
the spawning grounds, and an increase of 3% in the
main feeding grounds’ should be ‘the frequency of
extreme wind and turbulence events could decrease
by 50-60%, while mean turbulence will probably
decline by 10% in the spawning grounds and increase
by 3% in the main feeding ground of anchovy (IPCC
proposed an alternative erratum; see Section 5.2)
493
Major
Melting of Himalayan glaciers
The second paragraph of Section 10.6.2 of the
Working Group II contribution and a repeat
of part of the paragraph in Box TS.6 refer to
poorly substantiated estimates on rate of
recession and date for the disappearance of
Himalayan glaciers (2035). (See Annex B)
50
Minor
Observed warming since 1950
‘0.3 to 0.7 °C’ should be ‘0.4 to 0.7 °C’ (see Section 7.2)
Major
547*
Percentage of Dutch land area below sea level
‘below sea level’ should be ‘at risk of
flooding’ (see Section 8.2)
555
Minor
Decreases in crop yields by 2050
‘legumes −30 to +5%’ should be ‘legumes −14 to +1%’;
‘sunflower −12 to +3%’ should be
‘sunflower −12 to +1%’; and
‘tuber crops −14 to +7%’ should be ‘tuber
crops −9 to +8%’ (see Section 8.2)
583
Minor
Frequency of hurricanes
‘frequency and intensity of hurricanes’
should be ‘frequency and intensity of
intense hurricanes’ (see Section 9.2)
598
Minor
People living in water-stressed watersheds
‘Net increases in the number of people living in
water-stressed watersheds’ should be ‘Number of
people living in water-stressed watersheds and net
increases due to climate change’ (see Section 9.2)

Simple
erratum/
response
needed
Response
needed

Effect on Summary
for Policymakers
WG II and
Synthesis Report
Yes

Response
needed

No

Erratum

No

Response
needed

No

Erratum

No

Erratum

No

Erratum

No

Erratum

No

Erratum

No

a
These statements pertain to regional impacts of climate change (addressed in Working Group
II Chapters 9–16), together with the two known inaccuracies. Unless stated otherwise, the page
numbers refer to the Working Group II Report.
* Erratum already published by IPCC (last checked June 25th).

Inaccurate referencing (code: E2)
Table 3.3 provides an overview of all newly found errors in referencing in the
regional chapters of the Fourth Assessment Working Group II Report. We found
four instances of inaccurate referencing, which could and should be repaired by
issuing errata. These inaccuracies have no effect on the summary conclusions.

36

Assessing an IPCC assessment

Table 3.3

Inaccurate references newly identified by the PBL in the IPCC Working Group II
Report of 2007, Chapters 9–16

Region
Asia

Page
59, 471

Australia and
New Zealand

518

Latin America

598

Small islands

697

Proposed erratum
Coral reefs
The reference to Wilkinson (2004) should be replaced by Wilkinson (2000) (see Section 6.2)
Great Barrier Reef
The references to Berkelmans et al. (2004), Crimp et al. (2004) and Jones et al. (2004b)
should be replaced by a reference to Sheehan et al. (2006) (see Section 7.1)
Water supply
The references to Vásquez (2004) and UNMSM (2004) should be interchanged (see Section 9.2)
Freshwater lens
The reference to World Bank (2000) after ‘would reduce the freshwater
lens by 65%’ should be replace by Falkland (1999) (see Section 12.2)

The errors traced in the investigation presented above do not undermine the
conclusions of the IPCC on the regional effects of climate change.

3.3 Risk-oriented approach
The authors of the Summary for Policymakers of the Working Group II Fourth
Assessment Report – and especially those of the Summary for Policymakers of the
Synthesis Report – during the Fourth Assessment were asked by the Working Group
II Co-Chairs to identify the conclusions in the Working Group II assessment as ‘key
findings’. These key findings would, in turn, be described in two ways: as textual
descriptions of ‘main projected impacts’, and as a tabulated range of possible
impacts scaled to climate change. With regard to the main projected impacts,
examples of those that travelled to the Synthesis Report, Co-Chair Dr. Parry,
explained the approach to us as follows:
‘In outline: i) we asked author teams to identify a) a tabulated range of impacts for which
uncertainty ranges were known and b) a description of the ‘main projected impacts’ (as well
as vulnerabilities and adaptation issues); ii) we made clear that by ‘main’ we meant the
most significant challenges and opportunities that need to be faced; iii) we asked that each
author team’s list should cover the main relevant impact fields (e.g. water, food, health,
coasts, ecosystems, etc.) but not exhaustively if some were not relevant; iii) reviewers and
review editors commented on the selection of main projected impacts, and the lists were
frequently revised; v) the order of listing is not a rank order; vi) there should be a line of sight
connection between the listings of conclusions, i.e. from executive summary of the chapter,
through the Technical Summary, the Summary for Policymakers of the Working Group II
Report and the Synthesis Report, but vii) the wording of the conclusions might need to vary
according to its context in these different places. . . .
[Thus] our conclusions list the main projected impacts. Most of these impacts involve
potential losses in human welfare and therefore are negative. This stems from two things:
i) because the balance of impacts is, indeed, negative in welfare terms and also ii) because
positive impacts can often be exploited by mechanisms other than policy, i.e. autonomous
adaptation.’ (Parry, personal communication, 20 June 2010)

Results and discussion

37

The approach of Working Group II towards the selection of specific conclusions
from the regional chapters has thus focused on which impacts would be the most
important for policymakers to respond to.
The Synthesis Report, on page 47 (Section 3.3 on ‘Impacts of future climate
change’), gives a brief explanation of the selection of the particular key findings:
‘The following is a selection of key findings14 regarding the impacts of climate change on
systems, sectors and regions, as well as some findings on vulnerability, for the range of
climate changes projected over the 21st century.’

Footnote 14 specifies the selection criteria:
‘Criteria of choice: magnitude and timing of impact, confidence in the assessment,
representative coverage of the system, sector and region.’

And in the Summary for Policymakers of the Synthesis Report, the note to Table
SPM.2 (‘Examples of some projected impacts’) states:
‘The magnitude and timing of impacts that will ultimately be realised will vary with the
amount and rate of climate change, emissions scenarios, development pathways and
adaptation.’

As a result of the approach that was followed, mostly negative impacts were
singled out (see Table SPM.2, reproduced in Annex A of the present report). In fact,
the reason why hardly any example of a positive impact of climate change can be
found in Table SPM.2 of the Synthesis Report is that, according to the IPCC authors,
most of the positive impacts were not of sufficient policy relevance. The Working
Group II Co-Chair further explained the following:
‘i) most main impacts are negative, because mankind’s activities are generally tuned to
current climate, not to something different from that; most change therefore implies a cost;
ii) in net welfare terms, as Working Group II found, aggregate impacts are indeed estimated
to be negative (see much of Chapter 20 of the Working Group II Report, summarised on
page 17 of the Summary for Policymakers); iii) autonomous adaptation is better at making
more of opportunities than protecting against the adverse, and iv) policy is experienced in
providing protection against the adverse.’ (Parry, personal communication, 20 June 2010)

We labelled this approach to summarising as a ‘risk-oriented’ approach – mainly,
because it implicitly guides assessors to focus more on risk than on opportunities.
We believe that this risk-oriented approach has been implicit in the process to
arrive at conclusions based on the chapters and that by accepting its outcomes
it has in effect been approved by all the governments. The consequence of this
approach for which facts did or did not travel to the highest levels of summary,
can be illustrated by several examples. For instance, the focus in Table SPM.2 of
the Synthesis Report is on crops for which yields are likely to be reduced in Africa,
but the table does not mention the crops for which yields are likely to increase due
to climate change. For Australia and New Zealand, Table SPM.2 shows many risks
but only one benefit of climate change (initial benefits for agricultural production

38

Assessing an IPCC assessment

in New Zealand), while Section 11.4 on key future impacts and vulnerabilities
concludes: ‘climate change is likely to give rise to six key risks in specific sectors.
[…]. There are also four key benefits for particular sectors.’ Furthermore, in Africa,
also decreases in the numbers of people suffering from water-stress due to climate
change are projected for some areas. Of course, these numbers cannot be simply
subtracted from increases in water stress elsewhere (other people are involved),
but they constitute potential positive effects of climate change. Finally, an example
of a selection against mentioning the positive effects of climate change was found
with respect to the likely positive effect of climate change on forest production in
North Asia (Russia), which some policymakers may find to be of relevance to their
decision making. This potentially positive effect did not reach the Summaries for
Policymakers, partly because the authors attached a ‘medium confidence’ to their
statement.
A second aspect of the risk-oriented approach that was followed by the Working
Group II Fourth Assessment Report, is that in summary statements on the impacts
of climate change, information rightfully was provided on the climate-change
risk. However, this information was often not contextualised in the summary
statements by also mentioning other impact factors. This was even the case when
these other factors were much larger than the impact that was attributable to
climate change. To give an example, Arnell (2004) showed that the number of
people living in water-stressed watersheds (defined as having less than 1,000 m3
of water per year, per person), even without climate change, would rise strongly
over time (a rise of 1.5 to 2 billion people by 2025, globally, compared to 1995, see
his Table 5), mainly due to population growth in already water-stressed areas. This
increase is much larger than the additional increase related to climate change.
Again, some policymakers may wish to see both numbers – that is, changes with
and without climate change – within the same context in a summary.
A third, and final, characteristic of the risk-oriented approach is that, often, the
upper ends of uncertainty ranges (the worst outcomes that are projected) were
highlighted. One example comes from the chapter on Australia and New Zealand,
in which 60% of the Great Barrier Reef was projected to regularly bleach, by 2020.
This percentage is at the upper end of the uncertainty range. Another example,
in the Synthesis Report, in Table SPM.2, is the illustration of the risk of species
loss in the European mountains, by the statement ‘in some areas up to 60% under
high emissions scenarios by 2080.’ This statement highlights the upper end of the
uncertainty range (the most sensitive situations and the scenario with the most
impact), and does not specify the whole uncertainty range; although the words ‘up
to’ signal that there is a range, only the upper end was specified, causing it to be
highlighted.
We believe it is appropriate – and even necessary – to apply a risk-oriented
approach to climate-change impact assessment. It is a common approach in the
fields of, for example, public health, public safety, and anti-terrorism. Nevertheless,
we note that the nature of such an approach, especially for the Summary for
Policymakers of the Synthesis Report, could easily go unrecognised by readers.
The legitimacy of the findings could be increased if the reader would be explicitly
informed about the fact that a particular approach has been employed. However,

Results and discussion

39

we feel that a risk-oriented approach is not enough. After all, some policymakers
may wish to also receive information on the positive regional consequences and
the full ranges of uncertainty at the highest level of summary. Therefore, a more
explicit discussion on this issue in the IPCC Plenary may be considered.

3.4 Comments
In this investigation, besides looking for errors, we have evaluated the quality of
statements on regional climate-change impacts in the IPCC Fourth Assessment
Report by using seven criteria (see Section 2.2). A full account of our findings is
given in Part II. In the section below, an overview is given of our main comments,
grouped along our seven criteria.1)
 ttribution (code: C1)
A
The regional chapters in the Working Group II Report are typically judicious in
separating out the impacts that they attribute to anthropogenic climate change. In
the text examined, only one instance was found of an ‘insufficiently substantiated
attribution’ (major comment), in which a future attribution is questionable and
more care should have been taken to specify what part of a projected change
depends on climate change as its driver.
In the Technical Summary, in Table TS.4, the number of additional heat-related
deaths per year is given for Australia, with the suggestion that all 3,000 to 5,000
additional heat-related deaths projected for 2050 would be dependent on an
increase in temperature. However, a large part of this increase is solely due
to changes in population size and age distribution (see Section 7.2). In future
assessments, even more care must be taken to prevent insufficiently substantiated
attributions.
Generalisation (code: C2)
In most cases, the generalisations contained in the regional chapters are wellunderpinned by the chapter material. In the text examined only two instances
were found where the generalisation is questionable (both major comments). Two
statements in Table SPM.2 of the Synthesis Report involve ‘insufficiently founded
generalisations’. First, it is questionable whether freshwater availability will
decrease in all parts of Asia as indicated (Central, South, East and South-East Asia)
(see Section 6.1). Second, it is questionable, on the basis of evidence presented in
the chapter, whether productivity will decrease also for livestock farming other
than cattle, and, more generally in Latin America, because the evidence provided
relates only to Bolivia and Argentina (see Section 9.1). In future assessments, more
care must be taken to prevent insufficiently founded generalisations.

1) Part II also presents five of our comments that do not fit our qualification categories. Among
these, there is one major comment on the ambiguity of the meaning of phrases such as ‘in the Sahelian region’ (see Section 5.3).

40

Assessing an IPCC assessment

Transparency of expert judgment (code: C3)
One of the main findings of our investigation is that, for six out of the 32 key
statements on climate-change impacts in different regions (Table SPM.2), the
regional chapters insufficiently convey the IPCC authors’ reasoning behind their
weighing of the evidence that was available from the literature (‘insufficiently
transparent expert judgment’). For other important statements in the regional
chapters, also, information on the reasoning behind them was lacking. Expert
judgment will always have to play a crucial role in IPCC assessments, or in fact in any
assessment. But in those cases where readers may wish to ascertain the reliability
of a statement, having the authors’ reasoning explained in the text would be of
great help in improving the credibility of the claims made.
Below, by way of example, is a list of six questions pertaining to statements in Table
SPM.2, which we were unable to find the answer to, on the basis of the regional
chapters and the references contained therein:
1. Why is a projected decline in cereal yields in dry years in Morocco, by 2020,
representative of an overall decline in yields in ‘rain-fed agriculture’ in ‘some
countries’ in Africa? (see Section 5.1) (major comment)
2. With respect to the cost of adaptation to the projected sea level rise in Africa
towards the end of the 21st century, we asked: how was the range of 0 to 156%
of GDP for different countries in the referenced study translated to a 5 to 10%
range for the whole continent? (see Section 5.1) (minor comment)
3. In which way is the information about changes in ‘freshwater availability’ in Asia
(which in the chapter is given in both absolute flows and per-capita availability)
combined to arrive at the assessment that freshwater availability is projected to
decrease in many parts of Asia by the 2050s? (see Section 6.1) (major comment)
4. The report provides information on the range of temperature change for which
a regular bleaching of 60% of the Great Barrier Reef is expected; how was this
temperature range converted into the statement that this level of bleaching
would be reached by the year 2020? (see Section 7.1) (minor comment)
5. What is the underpinning for the projected loss of up to 60% in plant and animal
species, in some European mountainous areas, under high emissions scenarios
by 2080 – given that the references named in the chapter only state this
percentage in relation to plant species? (see Section 8.1) (minor comment)
6. How was the large range (from negative to positive) of projected changes in
yields of rain-fed crops (varying over crops and locations) for North America
translated to a 5 to 20% increase in ‘aggregate yields’? (see Section 10.1) (minor
comment)
Relying on expert judgment is a necessary practice in situations where wellresearched findings are scarce and potential risks are great. In such situations,
the joint deliberation of experts may simply be the best available way to make an
assessment. Yet, the transparency of the reasoning in such a deliberation would
enhance the verifiability and hence the potential credibility of the assessments that
experts arrived at. This investigation ex post reconstructed some of the reasoning.
Through communication with the IPCC authors, a large part of the answers was
received (these answers have been included in Part II), which led to an increase
in our confidence about the statements. Although, for a significant part of the
cases, the IPCC authors, when asked, were able to reconstruct their reasonings

Results and discussion

41

and convey them to us, we are convinced that, especially for statements that are
included in the Summaries for Policymakers, it is necessary in future assessments to
have such reasoning available in the underlying material (chapter texts).
Consistency of messages (code: C4)
Our investigation revealed only two instances of inconsistencies (minor comments)
within the report. This concerned inconsistencies in the likelihood level of increases
in fires in Australia and New Zealand (‘likely’ according to the main text of Chapter
11 and ‘virtually certain’ according to the Executive Summary of the Chapter) and
inconsistencies in the confidence level about the relevance of past experience
with extreme events to policy-making on adaptation in Europe (‘high confidence’
in the Executive Summary of Chapter 12 and ‘very high confidence’ in the Technical
Summary and Summary for Policymakers). Such minor flaws could have been
prevented by even more careful cross-checking and editing than was done for the
Working Group II Fourth Assessment Report.
Another issue related to consistency of messages is the general lack of specification
in the summaries about baseline years or periods related to the statements. Part
II does not address this issue, since it deals with individual statements; here it is
flagged as a more general issue. Variable baselines (20th century; 1961-1990; 1990;
1995; ‘today’) are used in the regional chapters of the Working Group II Report (as
well as in the underlying references), but many statements in the summaries are
vague about which particular baseline is being assumed for the projected ‘changes’.
For qualitative accounts this is less important, but baselines need to be crystal clear
when numbers are involved, since these are sensitive to the choice of baseline.
Ideally, baselines should be the same throughout a report, but this is not always
possible.
Traceability of references (code: C5)
Of the references that we checked, only very few turned out to be untraceable.
For instance, the projected ‘up to 50%’ reduction in yields in rain-fed agriculture
for some countries in Africa, was ultimately based on an untraceable reference
(MATUHE, 2001). Given the high importance attached to the statement, this is
a major comment. On several other examples we had only minor comment to
make. A hyperlink turned out to be outdated; it probably has been functioning in
2006/2007, but it was not in 2010 (FAO, 2002, see Section 9.2). Yet another example
concerns a paper on a projected increase in coffee-leaf miners in Brazil, which was
listed in the references of Chapter 13 as ‘submitted’ to Climatic Change, but we
found that the paper had never been accepted by this particular journal, which
makes this reference untraceable. Although we found a paper with the same title
that was published in 2008, in Pesquisa Agropecuária Brasileira, Brasília, the paper
had not been accepted by any journal at the time the IPCC report was published. It
would be our advice that papers may only be used as literature reference in future
IPCC reports if they have been accepted for publication prior to the publication date
of the IPCC assessment report.
Credibility of references (code: C6)
Only two instances were found of statements relying on references to grey
literature, when literature published in peer-reviewed journals was also available to

42

Assessing an IPCC assessment

which referencing would have been more appropriate. For example, the statement
that up to 40% of the Amazon Rainforest could react drastically to even a slight
reduction in precipitation was underpinned by a reference to a peer-reviewed
WWF/IUCN report of 2000 on forest fires, while – in this case – also more relevant
high profile peer-reviewed journal articles had been published by Cox et al. (2000;
2004). In our opinion, this issue deserves more attention from IPCC authors in the
future. We consider it a minor comment.
Availability of statements for review (code: C7)
In one instance, we found that a new, quantitative conclusion was added after the
Second Order Draft had been completed, without there being sufficient grounds
from review comments for such an addition. This statement, on the ‘up to 50%’
reduction in yields from rain-fed agriculture for some countries in Africa, thus, had
been unavailable for review, while it was allowed to travel from an untraceable
source up to the highest level of summary (Table SPM.2 of the Synthesis Report).
This is a major comment. It is our conviction that such high-impact information
should have been made available for review, and that this is applied in future IPCC
reports.

3.5 Conclusion
 o significant errors found in summary conclusions
N
The foundations for thirty-two IPCC Fourth Assessment summary conclusions
on the regional impacts of climate change have been investigated. These
conclusions show examples of projections of climate-change impacts on food,
water, ecosystems, coastal regions and health, for all the earth’s continents. These
conclusions have not been undermined by errors, although one of the conclusions
contains a minor inaccuracy: in hindsight, not 75 to 250 million people, but 90 to 220
million people are projected to be exposed to increased water stress due to climate
change in Africa, by 2020.
The provenance of summary statements needs to become more transparent in
future reports
Seven of the investigated 32 conclusions on the regional impacts of climate change
contain information that we were unable to sufficiently trace to the underlying
chapters in the IPCC Working Group II Report or to the references therein. For
two of these conclusions, our critical comments pertain to insufficiently founded
generalisations from existing scientific research, in both cases from local to
regional scales, and for one of them also from one type of livestock to livestock in
general. While acknowledging the essential role of expert judgment in scientific
assessments, the PBL recommends to improve the transparency of these
judgments in future IPCC reports.
The regional chapters: one error and some comments
In the underlying regional chapters, in addition to the two already known errors
about the melting of the Himalayan glaciers and about the Dutch land area
below sea level, another significant error was found: a 50 to 60% decrease in
productivity in anchovy fisheries on the African west coast was projected on the

Results and discussion

43

basis of an erroneous interpretation of the literature references. We found certain
inaccuracies, ranging from (very) small errors in numbers to imprecise literature
references. In addition, we have some critical comments to make. One of them
relates to the fact that the report does not specify how many of the additional heatrelated deaths projected for Australian cities are actually attributable to climate
change – a sizeable fraction is due to demographic changes alone. However, these
shortcomings do not affect the investigated 32 summary conclusions or other parts
of the IPCC summaries.
Examples of negative impacts dominate at summary level
The IPCC Working Group II Report focuses on climate impacts, adaptation and
vulnerability. It was found that, in the IPCC’s highest level summary, the conclusions
that were derived from the regional chapters of the Working Group II contribution
to the Fourth Assessment Report single out projections of negative climate-change
impacts. Our investigation revealed that IPCC authors considered these to be
the most relevant to policymakers. The PBL has labelled this as a ‘risk-oriented’
approach, which had been implicitly endorsed by the governments that constitute
the IPCC (including that of the Netherlands). The PBL subscribes to the importance
of an approach that highlights what may go wrong under unmitigated climate
change, but the Fourth Assessment Report lacked a clear explanation of the
choice of approach and its consequences. Alternatively, it could be argued that
policymakers should be presented with a complete picture in the Summaries for
Policymakers, not just with negative examples (without suggesting that potential
positive effects cancel out potential negatives effects).
No consequences for overarching conclusions
Our findings do not contradict the main conclusions of the IPCC on impacts,
adaptation and vulnerability related to climate change. There is ample
observational evidence of natural systems being influenced by climate change on
regional levels. And the negative impacts under unmitigated climate change in the
future pose substantial risks to most parts of the world, with risks increasing at
higher global average temperatures.

44

Assessing an IPCC assessment

Recommendations

4

Based on our findings and conclusions in the previous chapters, we present
recommendations for the Fifth Assessment Report process. Again, there will be
four assessment reports (by Working Groups I, II, and III and a Synthesis Report),
due in 2013 and 2014. The report by Working Group II will consist of two separate
volumes; a sectoral part A and a regional part B (IPCC, 2009). Our recommendations
primarily apply to part B of the Fifth Assessment Report by Working Group
II, although most of our recommendations may also apply to the entire Fifth
Assessment Report.

4.1 Minimising the risk of errors
In comprehensive scientific assessments of thousands of pages that refer to a
large corpus of scientific work, errors are inevitable, especially when dealing
with quantitative information. The current IPCC review process is meant to be
instrumental in removing errors that creep into the texts during the writing of the
several drafts. If errors do occur in the main texts, they rarely propagate to the level
of the Summary for Policymakers, since this relatively short text of approximately 15
pages is reviewed and approved line by line, word by word by the IPCC Plenary. In
our investigation, we found only one minor error in the investigated 32 statements.
However, the full range of IPCC reports is supposed to be authoritative, also in the
underlying texts. Therefore, as much as possible, errors need to be detected and
removed. With regard to finding and correcting errors, we believe that an open,
pro-active and transparent policy, involving wide circles of the assessment report
users, would increase the reliability of IPCC, given its recent increased visibility and
public exposure. It is a complex and highly dynamic field. Small mistakes belong to
this process and should not be seen to jeopardise the overall credibility of science.
However, concerns about statements that are made in such elaborate assessments
are inevitable. By enabling constant updating of documents and by publishing
‘errata’ as an integral part of the process, the IPCC may strengthen the societal
legitimacy of its work.
In light of the above, we recommend:
 Create a public registration website for the submission of possible errors found in the
published report. Errors are inevitable, therefore, the IPCC should make provisions
to deal with these, immediately following publication of an IPCC report. Since
a few months, erratum pages are available on the IPCC website for the Fourth
Assessment Reports of 2007. For the Fifth Assessment Reports to be published

Recommendations

45

in 2013 and 2014, such erratum pages should be made available directly after
publication. In addition, we suggest the launch of a public registration website
for the reporting of possible errors. Subsequently, these errors (e.g. regarding
numbers or typographic errors), when found to be genuine, could be quickly corrected by publishing them on the erratum website. For more substantial errors,
additional comments and explanations from the IPCC should be added and also
posted on the website. From our own recent experience, we believe that such
a registration website, if designed properly, would be manageable. Experience
with this approach may be gained by setting up experimental registration websites for the two upcoming Special Reports in 2011.
 Check and double-check references. More checks should be done with regard
to accurate referencing to the scientific literature, which should always be
traceable and available for reviewers. With regard to the traceability, a userfriendly option would be to create hyperlinks within the text leading to the
original publications.
 Investigate possibilities of pre-publication ‘crowd sourcing’. Finding errors, notably
in numbers, is a time-consuming effort for authors, reviewers, Co-Chairs and
Technical Support Units, who already are limited in terms of time and resources.
The possibility of using ‘crowd sourcing’ technologies as part of the review
during the last stages of completion, therefore, should be investigated. Crowd
sourcing has the advantage of providing a large reservoir of voluntary resources
to be tapped. A disadvantage could be the amount of resources needed to
manage and moderate tasks that would be handed out to the public.

4.2 Investing in the improvement and transparency
of foundations of summary conclusions
The findings in Section 3.4, and the more detailed analysis in Part II of this report,
lead us to the following recommendations:
 Make sure that statements that attribute impacts to climate change are well
founded in scientific research, including systematic observations, modelling
and statistics. The climate-change component of impacts should be carefully
characterised.
 Make sure that expert judgments that are incorporated in summaries are made
transparent and plausible by explaining the line of reasoning behind them in the
main text. This applies especially to:
–– case-study generalisations to whole sectors, areas or regions; such generalisations should be well supported by scientific evidence;
–– choosing certain intervals of specific numbers from a wide range of figures.
This would be a great help to any institute that is asked to assess the plausibility
and reliability of statements, and could enhance the credibility of the assessment
process. Authors are faced with the dilemma that strict limitations on page
lengths are imposed by the Working Group management, limiting the amount of
information that can be included.
 Check and double-check consistency of messages and confidence intervals/
likelihood levels between chapters and different summaries.
 Grey literature should only be used when peer-reviewed scientific journal
literature on a certain subject is not available.

46

Assessing an IPCC assessment

4.3 Strengthening the quality control by the chapter teams
IPCC authors work on a voluntary basis, or their work is a part of their regular jobs
as scientists and, often, it is an addition to their regular workload. The budget for
the IPCC organisation is relatively small, and assessment time lines are kept very
strict. The high cost-effectiveness is both a strength and a weakness in the IPCC
process.
The amount of scientific literature on climate change is skyrocketing, policy
implications become more visible and, recently, the credibility of the IPCC has come
under scrutiny by the media. This implies a need for higher quality standards, next
to a future increase in the amount of assessment work. Moreover, Coordinating
Lead Authors have to manage their writing teams, and deal with large flows of
information within a limited amount of time. Therefore, we recommend that a
procedure is set up – within the shortest possible time frame, and which will be
implemented at the very beginning of the writing process – for appointing and
employing chapter assistants who will help the Coordinating Lead Authors with
quality-control issues, such as the checking of references (including basic checks
on quantitative information), the cross-checking between findings on regional and
sectoral levels, the cross-checking between statements concerning regional climate
projections in the Working Group II Report and those in the report by Working
Group I. In order to make all this possible, a serious increase in the contributions
from the governments that are involved in the IPCC would be necessary.

4.4 Strengthening the review process
We recommend that certain improvements are made in the review process, in
order to further minimise the risk of errors and to improve the foundation of the
summary conclusions:
 Ensure that all texts are fully covered by expert reviewers. The current review
process is generally rather passive — expert reviewers are free to comment
on those parts of the text that they are interested in, but there is no guarantee
that qualified expert reviewers will scrutinise all parts of all texts including the
references. A pro-active targeted expert review is therefore recommended.
The IPCC could establish a web-based nomination procedure, in which experts
subscribe to reviewing certain parts of a report. The Co-Chairs and Technical
Support Units could then select the experts and see to it that all parts of the
texts are covered by at least a few qualified reviewers.
 Do not allow new material to be introduced after the last review if this material is not
related to the review. After the Second Order Draft review, new material should
only be added if it is clearly in response to, or related to substantive issues raised
in review comments. Otherwise, newly introduced material would be excluded
from expert review. The IPCC procedures should stipulate more clearly how new
material should be dealt with.
 Do not allow referencing to scientific articles that have been submitted but not (yet)
accepted for publication by the time the IPCC assessment report is published. This
has already been good practice in many of the IPCC reports, but IPCC procedures
should provide some ruling on this issue.

Recommendations

47

 Review by governments of the foundation and provenance of statements in Summaries
for Policymakers. Governments have an important responsibility in the review
process and, therefore, should seriously invest in the government/expert
review of Second Order Drafts. Apart from the reviewing of issues of specific
national interest, governments, assisted by independent experts, could review
provenance and foundations for statements in the draft version of the Summary
for Policymakers. This report provides an example of how this could be done
methodologically. Since this would be a time-consuming effort that draws from
many resources, the IPCC could consider extending the expert/government
review period for the Fifth Assessment Report. Governments could work
together in this process and share resources. In addition, this would provide
them with a solid frame of reference for the line-by-line approval session of the
final Summary for Policymakers.

4.5 Timing of the assessments
The quality of regional chapters could be further improved by making better use of
information that is available in the sectoral chapters of Working Group II reports.
The best way to achieve this would be to finalise regional chapters not until sectoral
chapters have become final. This would help to integrate information at regional
levels and contribute to the transparency of the expert judgments. In particular,
this can help to provide stronger underpinning of generalisations of case studies to
entire regions or sectors, by also making use of regional modelling studies.

4.6 Balancing the assessment of climate-change impacts
The section on ‘risk-oriented approach’ (3.3) reveals that, at summary level,
the most negative impact projections have been highlighted. This approach is
understandable and justifiable, but it has not been made explicit. We believe that
such a risk-oriented approach, although essential, is also one-sided. It could be
argued that the Summary for Policymakers should provide policymakers with
a full picture; it should not only present the most negative examples (without
suggesting that potential positive effects cancel out potential negative effects). We
recommend that the Fifth Assessment Report explains which approach has been
taken.
In addition, we suggest that a Summary for Policymakers includes two separate
sections dealing with projected regional impacts on water, food, ecosystems,
coastal regions and health, divided as follows:
 One section that describes robust outcomes, with the full range of projections
of climate-change impacts, including not only the uncertainties, but also any
positive impacts, containing:
–– A description of uncertainties, as has been employed by Working Group III in
their report in the Fourth Assessment. Per statement, an indication could be
given of the amount of evidence (number and quality of independent sources)
and level of agreement (on a particular finding). At summary level, the first
choice would be to highlight statements that are based on a large number of

48

Assessing an IPCC assessment

publications all converging to the same conclusions. ‘Likelihood’ statements
could only be made if sufficient scientific evidence would be available, and
‘confidence’ statements would be expert judgments, which are subjective by
nature and, normally, are untraceable for the reader;
–– An indication of the relative importance of other drivers that affect food,
water, ecosystems, coastal areas, and health, compared to drivers of climate
change, such as population growth, industrialisation and land use. The impacts
that can be attributed to climate change should be separated, as clearly as
possible, from those that are caused by other factors.
 One section that explicitly describes the most important negative impacts, as
was done for the Fourth Assessment Report, but this time including a worstcase risk approach, based on clearly explicated risk assessment methodology.
This section could highlight future impacts that may be unlikely to occur,
but could have major consequences, and therefore would pose significant
risks. Such a section could also include more speculative projections of high
climate sensitivity, accelerated warming, and tipping points in ecosystems. An
exploration of such an approach has been presented in the PBL study ‘News in
Climate Science and Exploring the Boundaries’ (December, 2009).
Reasons to consider this line of action include:
 The outsiders of the scientific community that study impacts, adaptation and
vulnerability, will not automatically understand the ‘risk-oriented approach’,
as we have labelled the Working Group II approach that leads to a sample of
the most important negative impacts. Working Group I followed a different
approach, establishing the full range of possible projections including its
uncertainties. Without proper explanation, the results at the summary level of
Working Group II could easily be interpreted as being an alarmist view (as was
our interpretation at the outset of our analysis, before we received further
explanation from the Working Group II Co-Chair and authors).
 We believe that policymakers (and their analysts) need to see the complete
picture at summary level. Positive impacts are important, because they may also
become smaller when mitigation measures are introduced. There may be offsets
possible between positive and negative impacts of climate change, for instance,
in relation to agricultural yields. Although there are justifiable objections against
making a cost-benefit analysis of positive and negative impacts, to not mention
them would be ‘policy-pre-emptive’.
 In order to obtain a complete picture, we also believe that policymakers – in
both public and private sectors – would need to be informed about the worstcase risks, when they are developing risk management, disaster prevention and
insurance policies.
 Finally, some of the comments made and the errors found – including those
received through our registration website – were related to the issue of
accentuating or even heightening the severity of climate-change impacts. We
recommend that the authors of the Fifth Assessment Report (AR5) be made
specifically aware of this issue. Care should be taken with phrasing of statements
that could be perceived by readers as heightening the projected impacts of
climate change.

Recommendations

49

4.7 Investing in climate-change impact science
Climate-change impact science is relatively young, although the amount of
literature is expanding rapidly. Figure 1.8 in the Working Group II Report and Figure
SPM.1 in the Synthesis Report (see Figure 1.1 in this report) show that there are
many observational data available on Europe and the United States that indicate
physical and biological changes, over time, but there is only a small amount of
observational data available on such changes in developing countries, while
developing countries are more vulnerable to climate change. In addition, model
projections become more uncertain at geographically more detailed levels, while
policymakers ask for concrete information on their specific region or location.
This calls for intensifying investments in regional climate impact observation
systems, regional modelling and capacity building, particularly in developing
countries
References
IPCC, 2009. Chapter outline of the Working Group II contribution to the IPCC Fifth Assessment report
(AR5), http://www.ipcc.ch/meetings/session31/doc20-rev1.pdf,
PBL, 2009. News in Climate Science and Exploring the Boundaries. http://www.rivm.nl/bibliotheek/
rapporten/500114013A.pdf

50

Assessing an IPCC assessment

Part II – Detailed analysis of
regional chapters and
summaries
This Part presents:
 our findings referring to the foundation for the 32 summary conclusions on regional impacts in the Synthesis Report (see Annex A) – below, these conclusions
have been typeset as headers, followed by our findings;
 additional findings at lower levels, such as those referring to the foundations for
statements on regional impacts in the Technical Summary and Executive Summaries of the regional chapters, and also some findings on parts of the main text
that we came across while working on other findings – below, statements have
been typeset as headers, followed by our findings.
For our investigation, we used several criteria for assessing the quality of IPCC
statements. First of all, we employed a distinction between obvious factual ‘errors’
– which we believe require an erratum on the IPCC Fourth Assessment web pages
– and ‘comments’, the latter being critical remarks, made from our specific position
of having to assess these statements on behalf of Dutch policymakers. These criteria were developed ‘on the job’, while categorising our findings. In total, there are
nine criteria, and associated with these are two types of ‘errors’ (E1-E2) and seven
types of ‘comments’ (C1-C7):
 E1: Inaccuracy of a statement;
 E2: Inaccuracy of literature referencing;
 C1: Insufficiently substantiated attribution, when a certain impact had been attributed to climate change without convincing foundation;
 C2: Insufficiently founded generalisation, when case studies had been extrapolated
to whole regions/sectors without convincing foundation;
 C3: Insufficiently transparent expert judgment, when we could not trace the reasoning behind a statement from underlying texts or literature references given;
 C4: Inconsistency of messages, when found between the different layers in the
IPCC report;
 C5: Untraceable reference, when a literature reference could not be found;
 C6: Unnecessary reliance on grey referencing, when reference had been made
to grey literature (literature other than from peer-reviewed journals), while
adequate peer-reviewed scientific journal references would have been available;

Part II – Detailed analysis of regional chapters and summaries

51

 C7: Statement unavailable for review, when new information had been introduced
after the last review but was not clearly derived from a content issue raised in
this last review.
When a certain statement in the IPCC Fourth Assessment Report would meet all the
criteria, the statement was considered to be well founded and reliable.

52

Assessing an IPCC assessment

Africa
5.1 Analysis of statements in Table SPM.2 of the Synthesis
Report

5

 y 2020, between 75 and 250 million of people are projected to be exposed to
B
increased water stress due to climate change. (E1; minor)
This statement was based on Table 11 in a study by Arnell (2004), which contains
results on increased water stress by 2025 (instead of 2020) as computed by
several climate models for five sub-regions in Africa (and other continents). In
our communication with Nigel Arnell, concerning the addition of numbers in the
table (which gives a range of between 75 and 240 million people, consistent with
the rounded numbers in the statement), he informed us that he had recently
noticed that, in this case, the addition of ranges was an unreliable method for
determining the total range. For the HADCM3 model the table contains a range of
model outcomes from three model runs of the SRES A2 scenario and a range from
two model runs of the B2 scenario, but the low and high values did not originate
from the same model run. Simply adding up these extreme values from the model
outcomes for the sub-regions may thus overstate the range. It must be added here
that, although this type of error is well known to occur in scientific practice, the
IPCC authors, including Nigel Arnell who was a Lead Author of a sectoral chapter
(Chapter 3) of the Working Group II Report, considered the straightforward
addition of those extreme values appropriate at the time. With the benefit of
hindsight, a relatively simple reanalysis of the data would have brought this error
to light. According to Arnell (2010), the correct range that results from adding
the right numbers for Africa would be: 88 to 217 million people. This implies that
‘between 75 and 250 million’ should have read ‘between 90 and 220 million’ when
rounded to the nearest 10 (E1b). However, since the difference is only small and
uncertainties are considerable, and because the original paper did not provide the
details that would have allowed correct computing of this range, we consider this
to be a minor inaccuracy.
By 2020, in some countries, yields from rain-fed agriculture could be reduced by up
to 50%. (C3,C5,C7; major)
This statement is not directly a statement on climate change, but on climate
variability: in individual years, droughts can cause up to 50% in yield reductions.
The implicit message here is that when droughts would become more frequent
due to climate change, more years with up to 50% in yield reductions would occur.
The statement could easily mislead readers into thinking that average annual
yields could be reduced by up to 50% due to climate change. In the Summary for
Policymakers of the Working Group II Report, the paragraph that contains this

Africa

53

statement starts with a sentence introducing the notion of climate variability, which
puts the statement more into context. We have several major comments to make
about its provenance. The statement was based on a report by the International
Institute for Sustainable Development (IISD) (Agoumi, 2003) that referred to
studies performed within the framework of a UNEP–GEF project that we were
unable to trace (C5) and to the Initial National Communications (INCs) of Morocco,
Algeria and Tunisia. However, only the INC of Morocco (Kingdom of Morocco,
2001) reported a decline in yields for 2020: ‘a 50% reduction in cereal yields in
Morocco might occur in dry years and 10% in normal years’. The IPCC authors
explained to us that present-day climates and projected future climate change in
the three countries are very similar – with a reference to Figure 11.2 in the Working
Group I Report of AR4 – and also that only cereals are grown without irrigation.
Furthermore, using information from the European statistical information service
(EUROSTAT) the authors made plausible that, due to current climate variability, the
yields in Algeria, Morocco and Tunisia have been varying annually, including yield
reductions of nearly 70% in individual years, in the period between 2000 and 2006.
In hindsight, these additional explanations could have provided further foundations
for the statement, had they been included in Chapter 9 (C3).
The Initial National Communication of Morocco indicated that this statement was
based on ‘The Study of Morocco’s Vulnerability to CC Effects’ (MATUHE, 2001).
However, we were unable to trace this study, nor could it be provided by the IPCC
authors (C5). Although the IPCC does not formally require that authors check
references of references of references, it would have been very helpful in this case;
the IPCC requires that the chapter team reviews the quality and validity of nonpeer-reviewed sources (C5). As it stands, the statement cannot be traced back to
any original scientific research.
Finally, the statement seemed to have been added to the main text of Chapter 9
after the Second Order Draft had been reviewed, but we were unable to find the
particular substantive comment(s) made in this review that could have led to the
addition of this statement (C7).
Agricultural production, including access to food, in many African countries is
projected to be severely compromised. This would further adversely affect food
security and exacerbate malnutrition.
This statement – which together with the previous statement comprises the second
bulleted statement on Africa in Table SPM.2 (see Annex A) – is fully supported by
the underlying material.
Towards the end of the 21st century, projected sea level rise will affect low-lying
coastal areas with large populations. The cost of adaptation could amount to at least
5 to 10% of Gross Domestic Product (GDP). (C3; minor)
The first sentence of this statement is fully supported by the underlying material.
We have a comment to make about the second sentence of the statement, which
was based on a table in a study by Niang-Diop (2005). The text does not explicate
how the experts derived the 5 to 10% in adaptation costs from this study, since
it covered a range from 0.5 to 156.2% (C3). The IPCC authors explained to us that
they used 5 of 12 studies summarised in the table in Niang-Diop (2005). Those that

54

Assessing an IPCC assessment

scored below the range of 5 to 10% were either countries that did an assessment
not covering the entire country (Cameroon: 0.5%; Gambia: 1%), or those with a very
short coastline (Democratic Republic of the Congo: 4.1%), or relatively rich countries
(Nigeria: 2.2%). Those that scored well above this range were either very small island
states (Comoros: 49.7%) or countries for which the costs had been overestimated
(Mauritania: 156.2%). This answer explains the 5 to 10% range. We believe it would
have been helpful if this explanation had been provided in Chapter 9.
By 2080, an increase of 5 to 8% of arid and semi-arid land in Africa is projected
under a range of climate scenarios.
This statement is fully supported by the underlying material.

5.2 Additional findings
 he population at risk of increased water stress in Africa is projected to be between
T
75–250 million and 350–600 million people by the 2020s and 2050s, respectively
(E1; minor).
This statement is part of the Executive Summary and is also included in Table
TS.4 on page 67 of the Technical Summary. The range of 75 to 250 million people
is discussed above in Section 5.1 (first statement). From Nigel Arnell, the author
of the underlying 2004 study, we learned that also the second range (350 to 600
million people in the 2050s) was incorrectly computed from Table 11 in his study;
the extreme values for the sub-regions cannot be added together. This remained
unnoticed during the writing of the Working Group II Report. According to Arnell,
in hindsight, the correct range should have been 350 to 490 million (E1b). Given the
large uncertainties, we consider this a minor error. This inaccuracy has no impact on
the IPCC conclusions in the various Summaries for Policymakers.
In South Africa, fisheries could be affected […]. Recent simulations based on the
NCAR GCM under a doubling of carbon dioxide indicate that extreme wind and
turbulence could decrease productivity by 50-60%, while turbulence will probably
bring about a 10% decline in productivity in the spawning grounds and an increase
of 3% in the main feeding grounds. (E1; major)
This statement originates from Section 9.4.4 of the main text, and was based on
a study by Clark et al. (2003), which indicated that the changes in the frequency of
extreme wind and turbulence events is expected to decrease quite substantially, by
50 to 60%; it did not refer to a similar decrease in fishery productivity (E1a). Also, in
the latter part of the sentence it should have been the changes in mean turbulence
that decline by 10% and increase by 3%, and not the productivity. The IPCC authors
initially indicated that the reference to ‘productivity’ should have been deleted and
‘for anchovy’ should have been added at the end. In that case, the statement would
read as follows:
Recent simulations based on the NCAR GCM under a doubling of carbon dioxide indicate
that the frequency of extreme wind and turbulence events could decrease by 50 to 60%,
while mean turbulence will probably decline by 10% in the spawning grounds and increase
by 3% in the main feeding grounds for anchovy.

Africa

55

From the IPCC authors we received the following proposal for an erratum:
Recent simulations based on the NCAR GCM under a doubling of carbon dioxide indicate
that changes in extreme wind and turbulence could decrease potential spawning and
increase potential larval feeding of anchovy.
This major error has no impact on the IPCC conclusions in the various Summaries for
Policymakers.

5.3 Findings from the PBL registration website
 gricultural intensification and/or expansion into marginal lands can trigger
A
additional conflicts, cause crop failure, exacerbate environmental degradation (e.g.,
Olsson et al., 2005). (minor)
This statement can be found on page 441, Section 9.2.2.4, of the main text. One of
the reactions we received on the registration website indicated that the reference
to Olsson et al. (2005) in relation to this particular statement is not the most
relevant one, because the article does not focus primarily on the intensification
and/or expansion of agriculture and its consequences. Olsson et al. (2005) is about
changes in precipitation patterns that have resulted in the greening of the Sahel
over the past 10 years (5 years at the time of writing). We believe that other, more
appropriate references would have been available for this statement, but we have
not assigned this less appropriate referencing to one of our qualification categories.
It has no impact on the IPCC conclusions in the various Summaries for Policymakers.
In the Sahelian region of Africa, warmer and drier conditions have led to a reduced
length of growing season with detrimental effects on crops. (major)
This statement was taken from page 9 of the Summary for Policymakers of the
Working Group II Report. One of the reactions we received indicated that this
summary statement did not adequately convey the nuances contained in Chapter
9.1) In response to that reaction, we wish to make a major comment on this
statement. It was based on two sentences in Section 1.3.6.1 of the main text: ‘In
Sahelian countries, increasing temperature in combination with rainfall reduction has led
to a reduced length of vegetative period, no longer allowing present varieties to complete
their cycle (Ben Mohamed et al., 2002).’ and: ‘In the case of the Sahel region of Africa,
warmer and drier conditions have served as a catalyst for a number of other factors that
have accelerated a decline in groundnut production (Van Duivenbooden et al., 2002).’
Even though the statement does not directly refer to the regional chapters, it is a
statement in the Summary for Policymakers on a regional impact.
Although, the Summary for Policymakers’ text is consistent with the formulations
in the main text, we believe that the subject of both case studies, which deal with
millet, groundnut and cowpeas in Niger, cannot be generalised to describe ‘the
Sahelian region’ and to ‘effects on crops’. In reaction to this, the IPCC authors replied
that ‘the Sahelian region’ is not the same as ‘the entire Sahelian region’ and ‘effects on
1) This reaction and the previous one were received as one combined reaction; in our report we treat
them separately, since they pertain to different statements.

56

Assessing an IPCC assessment

crops’ is not the same as ‘effects on all crops’. In addition, the authors have pointed
to the heading of this section (p. 9, Working Group II Summary for Policymakers)
that says these are ‘examples’ and ‘have not yet become established trends’. The
authors further commented that these are obviously not universal statements. In
their opinion, it would be unnecessary for every statement on certain parts of a
place or region to say ‘parts of’, and similarly about statements on systems to say
‘some of’, because in their view this non-universality is implicit.
Indeed, the disclaimer in the header does explain that these are examples and not
established trends, but to us it is not obvious that the statement therefore refers
to some crops in a part of, for instance, the Sahelian region, and we believe it would
have been better if these specifications had been added. This major comment
has no consequences for the IPCC conclusions in the various Summaries for
Policymakers.
References
Agoumi, A., 2003. Vulnerability of North African countries to climatic changes: adaptation and
implementation strategies for climatic change. Developing Perspectives on Climate Change:
Issues and Analysis from Developing Countries and Countries with Economies in Transition. IISD/
Climate Change Knowledge Network, 14 pp. http://www.cckn.net//pdf/north_africa.pdf
Arnell, N.W., 2004. Climate change and global water resources: SRES emissions and socio-economic
scenarios. Global Environ. Change, 14, 31-52.
Arnell, N.W., 2010. Personal communication.
Ben Mohamed A., Duivenbooden, N. van, Abdoussalam, S., 2002, Impact of climate change on
agricultural production in the Sahel- part I. methodological approach and case study for millet in
Niger, Climatic Change, 54, pg 327-348.
Clark, B.M., S. Young and A. Richardson, 2003. Likely effects of global climate change on the purse
seine fishery for Cape anchovy Engraulis capensis off the west coast of Southern Africa (SE
Atlantic). http://swfsc.noaa.gov/uploadedFiles/Education/lasker_events.doc.
Van Drunen, M.A., R. Lasage and C. Dorland, Eds., 2005. Climate Change in Developing Countries: An
Overview of Study Results from the Netherlands Climate Change Studies Assistance Programme.
CABI Publishing,Amsterdam, 320 pp.
Duivenbooden, N. van, Abdoussalam, S., Ben Mohamed A., 2002. Impact of climate change on
agricultural production in the Sahel- part II> Cae study for groundnut and cowpea in Niger,
Climatic Change, 54, pg 349-368.
Kingdom of Morocco, 2001. Initial National Communication on the United Nations Framework
Convention on Climate Change, Ministry of Land-use Planning, Housing, and the Environment,
October. http://unfccc.int/resource/docs/natc/mornc1e.pdf
MATUHE, 2001. Etude de Vulnérabilité du Royaume du Maroc face aux impacts des changements
climatiques. Ministère chargé de l’Aménagement du Territoire, de l’Urbanisme, del’Habitat et de
l’Environnement, septembre 2001.
Niang-Diop, I., 2005. Impacts of climate change on the coastal zones of Africa. Coastal Zones in
Sub-Saharan Africa: A Scientific Review of the Priority Issues Influencing Sustainability and
Vulnerability in Coastal Communities, IOC, Ed., IOCWorkshop Report No. 186. ICAM Dossier No. 4,
27-33.
Olsson, L., Eklundh, L., Ardö, J., 2005. A recent greening of the Sahel—trends, patterns and potential
causes. Journal of Arid Environments, 63, pg 556–566.
Otter, L., Olago, D.O. and Niang, I. (eds), 2007. Global Change Processes and Impacts in Africa: A
Synthesis. START/East African Educational Publishers, Nairobi. 346 pages. September 2007.

Africa

57

58

Assessing an IPCC assessment

Asia

6

6.1 Analysis of statements in Table SPM.2 of the Synthesis Report

 y the 2050s, freshwater availability in Central, South, East and South-East Asia,
B
particularly in large river basins, is projected to decrease. (C2,C3; major)
This statement originates from the Executive Summary of Chapter 10 of the main
text. Section 10.4.2 of the main text does not clearly reveal projections indicating
an absolute decrease in freshwater availability in all indicated areas by the 2050s.
There is no indication of how freshwater availability should be interpreted in this
statement, that is, whether it concerns absolute flows or per-capita availability,
which makes it hard to establish the line of reasoning that was followed, since a
combination of evidence using different metrics has been cited in the chapter (C3).
In the main chapter text, there are various referenced projections of ‘water stress’
(availability relative to demand) for South and Southeast Asia by 2050, projections
of reduced run-off in mid-China by the end of the 21st century, and reductions in
the per-capita availability of fresh water in India by 2050. There is a reference to a
region in Central Asia (Hindu-Kush) and its dependence on meltwater from glaciers,
but no long-term projections are given. However, according to model calculations,
the runoff projections for 2100 in the sectoral Chapter 3 (‘Freshwater resources and
their management’) show an increase in annual runoff in South, East and Southeast
Asia (Figure 3.8, on page 201). This, of course, does not contradict possible severe
water stress during dry seasons, but neither does it support the generalisation that
led to the statement in the Summary for Policymakers (C2).
Coastal areas, especially heavily populated megadelta regions in South, East and
South-East Asia, will be at greatest risk due to increased flooding from the sea and,
in some megadeltas, flooding from the rivers.
This statement is fully supported by the underlying material.
Climate change is projected to compound the pressures on natural resources and
the environment associated with rapid urbanisation, industrialisation and economic
development.
This statement is fully supported by the underlying material.
Endemic morbidity and mortality due to diarrhoeal disease primarily associated with
floods and droughts are expected to rise in East, South and South-East Asia due to
projected changes in the hydrological cycle. (C3; minor)
This statement is fully supported by the underlying material.

Asia

59

6.2 Additional findings
The already discovered error on the melting rate of glaciers (Section 10.6.2 of the
main text) is treated in Annex B.
 he frequency and extent of forest fires in northern Asia is expected to increase in
T
the future due to climate change and extreme weather events that could likely limit
forest expansion. (C3,C6; major)
See the Executive Summary, page 471, the Technical Summary, pages 49, and Box
TS.6 on page 60. We have a comment to make on this statement. In the underlying
text, nuances and uncertainties are clearly described: ‘The observations in the
past 20 years show that the increasing intensity and spread of forest fires in North and
South-East Asia were largely related to rises in temperature and declines in precipitation
in combination with increasing intensity of land uses (see Section 10.2.4.4).Whether
this trend will persist in the future or not is difficult to ascertain in view of the limited
literature on how the frequency and severity of forest and brush fires will likely respond
to expected increase in temperature and precipitation in North and South-East Asia (see
Section 10.3.1).’ In the text, the expected future increase was solely underpinned
by a reference to a single conference paper, which was based on a speech by the
Vice-Minister of the Ministry of the Russian Federation for Civil Defence and did not
include any references (Vorobyov, 2004). Given the response by the authors that
the statement is an adequate reflection of other literature not listed in the chapter,
we concluded that the statement is an insufficiently transparent expert judgment
(C3) and that the referencing was unnecessarily relying on grey literature (C6). The
IPCC authors explained to us the policy relevance of including this statement in the
Executive Summary of the chapter and in the Technical Summary. Even though the
research in this area was in an early stage and the modelling of future impacts was
limited in 2006, the authors considered this subject to be sufficiently important
because of its potential scale of effect. The statement was awarded only medium
confidence (having an even chance of being correct), but given the use of the word
‘likely’, we think it could have been worded more carefully. These major comments
have no consequences for the IPCC conclusions in the various Summaries for
Policymakers.
Recent risk analysis of coral reef suggests that between 24% and 30% of the reefs
in Asia are likely to be lost during the next 10 years and 30 years, respectively. (E2;
minor)
For this statement, see the Executive Summary, page 471, and the Technical
Summary, Box TS.6 on page 59. We identified an inaccurate reference affecting this
statement. The statement cannot be deduced from the figures presented in the
included table (Table 10.6). Also, the underlying reference (Wilkinson, 2004) does
not provide information to support the statement. The IPCC authors indicated that
(Wilkinson, 2004) was cited incorrectly; this should have been (Wilkinson, 2000)
(E2). This minor inaccuracy has no impact on the IPCC conclusions in the various
Summaries for Policymakers.

60

Assessing an IPCC assessment

The Third Assessment Report (TAR) predicted that the area-averaged annual mean
warming would be about 3°C in the decade of the 2050s and about 5°C in the
decade of the 2080s over the land regions of Asia as a result of future increases in
atmospheric concentration of greenhouse gases (Lal et al., 2001a). (minor).
For this statement, see Section 10.1.1 of the main text, page 471. We have one
minor comment to make on this statement (which could not be placed in one of
the categories). In the First Order Draft, the statement used the word ‘reported’
instead of ‘predicted, and one of the comments in the review of the First Order
Draft was that ‘The TAR reported that...’ should be changed to ‘The TAR predicted
that...’ or ‘The TAR forecasted that...’. This comment was responded to, saying that
‘IPCC Assessment Reports do not predict or forecast – they only report’, however,
the sentence was changed nevertheless to ‘The TAR predicted that…’. It may
be useful to note that in the IPCC reports, normally, the verb ‘to project’ is used
instead of ‘to predict’ to denote the fact that a statement is made relative to a
particular scenario. A comment on the Second Order Draft, made by the Republic of
Korea, was to ‘specify the SRES scenario used for this assessment statement’. This
comment was accepted according to the authors’ response (‘Appropriate revisions
and editing made’), but no changes were implemented in the final texts. This minor
comment has no consequences for the IPCC conclusions in the various Summaries
for Policymakers.
References
Vorobyov,Y., 2004. Climate change and disasters in Russia. Proc. World Climate Change Conference,
Moscow, Y. Izrael, G. Gruza, S. Semenov and I. Nazarov, Eds., Institute of Global Climate and
Ecology, Moscow, 293-298.
Wilkinson, C., Ed., 2000. Status of Coral Reefs of the World: 2000.Australian Institute of Marine
Science, Townsville, 363 pp.
Wilkinson, C., Ed., 2004. Status of Coral Reefs of the World: 2004, Volume 1. Australian Institute of
Marine Science, Townsville, 302 pp.

Asia

61

62

Assessing an IPCC assessment

Australia and
New Zealand

7

7.1 Analysis of statements in Table SPM.2 of the Synthesis Report
 y 2020, significant loss of biodiversity is projected to occur in some ecologically
B
rich sites, including the Great Barrier Reef and Queensland Wet Tropics. (E2,C3;
minor)
In the text of Chapter 11 of the Working Group II Report, which underpins the
statement, we found an inaccurate reference. Biodiversity loss on the Great Barrier
Reef is covered by Table 11.6, in which is stated, for 2020: ‘Bleaching and damage
to the Great Barrier Reef equivalent to that in 1998 and 2002 in up to 50% of years.
(Berkelmans et al., 2004; Crimp et al., 2004) and 60% of the Great Barrier Reef regularly
bleached ( Jones, 2004b).’ The references for the first and second statement are
not correct (E2). The IPCC authors indicated that the correct reference for both
statements is Sheehan et al. (2006).
Furthermore, we were unable to derive the statements in Table 11.6 from the
provided, correct reference (Sheehan et al, 2006). The reference relates impacts
to temperature changes, not to specific years, which in this case would have been
2020. Implicitly, the IPCC authors converted warming into years by using Table 11.4,
that is, a near-coast warming in the range of 0.1 to 1.0 °C by 2020. As also confirmed
by the IPCC authors, this conversion should have been explicitly stated in the
caption of Table 11.6 (C3). We consider this to be a minor comment.
By 2030, water security problems are projected to intensify in southern and eastern
Australia and, in New Zealand, in Northland and some eastern regions.
This statement is fully supported by the underlying material.
By 2030, production from agriculture and forestry is projected to decline over much
of southern and eastern Australia, and over parts of eastern New Zealand, due to
increased drought and fire. However, in New Zealand, initial benefits are projected
in some other regions.
This statement is fully supported by the underlying material.
By 2050, ongoing coastal development and population growth in some areas of
Australia and New Zealand are projected to exacerbate risks from sea level rise and
increases in the severity and frequency of storms and coastal flooding.
This statement is fully supported by the underlying material.

Australia and New Zealand

63

7.2 Additional findings
3 ,000 to 5,000 more heat related deaths per year. (C1; major)
This example of regional impact was taken from Table TS.4 of the Technical
Summary. We have a major comment to make on this statement.
From Table TS.4 readers may mistakenly conclude that 3,000 to 5,000 more heatrelated deaths could be directly attributed to temperature change (as is the case
with the ‘additional people suffering from increased water stress’ also shown
in the table). However, the largest part of the 3,000–5,000 range is dependent
on expected changes in population sizes and age distributions within cities (C1).
With the other drivers remaining constant, temperature change would only be
responsible for 300 to 900 more heat-related deaths (this can be calculated
from Table 2 in McMichael et al., 2003). This is only a small part of the 3,000 to
5,000 more heat-related deaths mentioned in the table. The underpinning for the
statement in Table TS.4 is given in Section 11.4.11 on health. The relevant sentence
is: ‘Assuming no planned adaptation, the number of deaths is likely to rise from 1,115/
yr at present in Adelaide, Melbourne, Perth, Sydney and Brisbane to 2,300 to 2,500/yr
by 2020, and 4,300 to 6,300/yr by 2050, for all SRES scenarios, including demographic
change (McMichael et al., 2003)’. The stated numbers include both demographic and
temperature effects. In the chapter is not mentioned that the effect of temperature
change alone is relatively small, compared to the combined effect of temperature
change and changes in population sizes and age distributions.
We believe that the total number of heat-related deaths by 2050 (3,000–5,000) is
a number that IPCC should publish, as policymakers should be made aware of all
heat-related deaths, not just those due to climate change. However, it is also our
opinion that an IPCC report on climate-change impacts should show the different
components including the climate-change related component, if available or
traceable. It is no easy matter to establish a climate-change related component,
but one way of doing this would be to make a comparison between changes in
the demographic situation of 2050 with climate change and without. A simple
calculation performed by the PBL resulted in 1,000 to 3,000 additional heat-related
death per year, as a consequence of climate change, which is significantly less
than the 3,000 to 5,000 range given in Table TS.4. This major comment has no
consequences for the IPCC conclusions in the various Summaries for Policymakers.
Heat waves and fires are virtually certain to increase in intensity and frequency. (C4;
minor)
This statement originates from the Executive Summary. We have a minor comment
to make on this statement. In our opinion, there is an inconsistency between the
level of likelihood in the Executive Summary and in the chapter text. The relevant
chapter text in Section 11.3.1 repeatedly uses ‘likely’ for increases related to fires,
and it not clear why ‘likely’ was subsequently translated to ‘virtually certain’ in the
Executive Summary (C4). The IPCC authors have concluded, after consultation with
authors of relevant references, that the likelihoods in the chapter may have been
understated. This minor comment has no consequences for the IPCC conclusions in
the various Summaries for Policymakers.

64

Assessing an IPCC assessment

Since 1950 there has been a 0.3 to 0.7 °C warming in the region. (E1; minor)
This statement, taken from the Technical Summary on page 50, is different from
the Executive Summary, which states: ‘Since 1950, there has been 0.4 to 0.7°C
warming,…’. This is a minor inaccuracy (E1a). The authors confirmed this to be a
typographical error in the Technical Summary; 0.4 °C is the correct number. This
minor inaccuracy has no impact on the IPCC conclusions in the various Summaries
for Policymakers.
References
Berkelmans, R., G. De’ath, S. Kininmonth and W.J. Skirving, 2004. A comparison of the 1998 and 2002
coral bleaching events of the Great Barrier Reef: spatial correlation, patterns and predictions.
Coral Reefs, 23, 74-83.
Crimp, S., J. Balston, A. Ash, L. Anderson-Berry, T. Done, R. Greiner, D. Jones, R.N., 2004b. Managing
climate change risks. The Benefits of Climate Policies: Analytical and Framework Issues, J. Corfee
Morlot and S. Agrawala, Eds., OECD, 251-297.
http://www.oecd.org/document/35/0,2340,en_ 2649_34361_ 34086819_1_1_1_1,00.html.
McMichael, A., R. Woodruff, P. Whetton, K. Hennessy, N. Nicholls, S. Hales, A. Woodward and
T. Kjellstrom, 2003. Human Health and Climate Change in Oceania: A Risk Assessment 2002.
Commonwealth Department of Health and Ageing, 128 pp. http://www.health.gov.au/internet/
main/publishing.nsf/Content/2D4037B384BC05F6CA256F1900042840/$File/env_climate.pdf
Sheehan, P., R. Jones, A. Jolley, B.L. Preston, M. Clark, P.J. Durack, S. Islam, F.Sun, and P.H.
Whetton, 2006. Climate Change and the Global Knowledge Economy: An Immediate Challenge.
CSES Climate Change Working Paper No. 11. Victoria University Centre for Strategic Economic
Studies, Melbourne, 35 pp.
http://www.cfses.com/documents/climate/11_Sheehan_Jones_et_al_Climate_Change_.pdf

Australia and New Zealand

65

66

Assessing an IPCC assessment

Europe

8

8.1 Analysis of statements in Table SPM.2 of the Synthesis Report

 limate change is expected to magnify regional differences in Europe’s natural
C
resources and assets. Negative impacts will include increased risk of inland flash
floods and more frequent coastal flooding and increased erosion (due to storminess
and sea level rise).
This statement is fully supported by the underlying material.
Mountainous areas will face glacier retreat, reduced snow cover and winter tourism,
and extensive species losses (in some areas up to 60% under high emissions
scenarios by 2080). (C3; minor)
We have one minor comment to make on this statement (which is also included,
verbatim, in the Summary for Policymakers of the Working Group II Report).
The Executive Summary only addresses extensive species loss in ‘mountain
plant communities, which face a loss in species of up to 60% under high emission
scenarios’, but does not mention mountain fauna, nor does Thuiller et al. (2005),
the publication on which the example was based. Thus, the example given in the
statement in the Summary for Policymakers (‘in some areas up to 60%...’) was not
fully derived from the text in the Executive Summary or from the main text of
Chapter 12. In addition, it is not transparent how the authors have underpinned
the example (C3). After an additional literature search on our part, it appeared
that there was a reference that the authors could have used for underpinning the
example. A report by the Potsdam Institute for Climate Impact Research (PIK)
(Schröter et al., 2004) that was not included in the references of the chapter,
explicitly presented quantitative results which refer to species loss in mountain
flora and fauna of up to 60% (Figure 25 in Schröter et al., 2004). We judged this
omission of a reference and of a more traceable line of reasoning from the chapter
to the Summary for Policymakers to be of minor effect.
In southern Europe, climate change is projected to worsen conditions (high
temperatures and drought) in a region already vulnerable to climate variability, and
to reduce water availability, hydropower potential, summer tourism and, in general,
crop productivity.
This statement is fully supported by the underlying material.
Climate change is also projected to increase the health risks due to heat waves and
the frequency of wildfires.
This statement is fully supported by the underlying material.

Europe

67

8.2 Additional findings
The already discovered error on the Dutch land area below sea level (in the main
text of Chapter 12, on page 547) is discussed in Annex C.
 daptation to climate change is likely to benefit from experiences gained in
A
reactions to extreme climate events, by specifically implementing proactive climatechange risk management adaptation plans (very high confidence). (C4; minor)
This statement appears in the Technical Summary (page 53) and in the Summary for
Policymakers of the Working Group II Report (page 14). However, the exact same
statement received a lower confidence level (‘high confidence’) in the Executive
Summary (page 543) (C4). The IPCC authors replied that the correct level of
confidence should have been ‘very high confidence’ This was an editorial mistake;
IPCC has indicated that it will issue an erratum. This minor inconsistency has no
impact on the IPCC conclusions in the Summary for Policymakers of the Synthesis
Report.
In southern Europe, general decreases in yield (e.g., legumes -30 to +5%; sunflower
-12 to +3% and tuber crops -14 to +7% by 2050) and increases in water demand (e.g.,
for maize +2 to +4% and potato +6 to +10% by 2050) are expected for spring sown
crops (Giannokopoulos et al., 2005; Audsley et al., 2006). (E1; minor)
The statement, which was taken from the main text of Chapter 12 (page 555),
is slightly inaccurate (E1a). In Giannokopoulos et al. (2005), four Mediterranean
regions have been distinguished. Only two are located in Europe (those labelled
‘N-W’ and ‘N-E’). The numbers (ranges) presented in the IPCC Working Group II
Report refer to all four regions, including North Africa, Jordan, Libya and Egypt,
and should not have been reported in the chapter on Europe. The correct ranges
for the European regions should read: Legumes (beans/soybean) -14 % to +1 %; C3
(sunflower) -12% to + 1%; and Tubers (potato) -9% to +8 %). The IPCC authors have
told us that this finding will lead to an IPCC erratum. These minor inaccuracies have
no impact on the IPCC conclusions in the various Summaries for Policymakers.
Annual average runoff is projected to increase in northern Europe (north of 47°N)
by approximately 5 to 15% up to the 2020s and 9 to 22% up to the 2070s, for the
SRES A2 and B2 scenarios and climate scenarios from two different climate models
(Alcamo et al., 2007) (Figure 12.1). Meanwhile, in southern Europe (south of 47°N),
runoff decreases by 0 to 23% up to the 2020s and by 6 to 36% up to the 2070s (for the
same set of assumptions). (C3; minor)
We have a minor comment to make on this statement. The quantitative information
in this statement (Chapter 12, page 549), as well as in Figure 12.1, could not be
found in the specific paper cited (Alcamo et al., 2007) (C3). In a reaction, the IPCC
authors responded by saying that – for the accomplishment of the most up-to-date
review of scientific knowledge – they sometimes also used underlying data that had
formed the basis of the results in the peer-reviewed publications; they obtained
this background data by contacting the authors of these publications and asking for
specific figures. Our comment does not imply that such information should not be
used, but only that the provenance of this information could have been made more
transparent. This minor comment has no consequences for the IPCC conclusions in
the various Summaries for Policymakers.

68

Assessing an IPCC assessment

Climate-related increases in crop yields are expected mainly in northern Europe,
e.g., wheat: +2 to +9% by 2020, +8 to +25% by 2050, +10 to +30% by 2080
(Alexandrov et al., 2002; Ewert et al., 2005; Audsley et al., 2006; Olesen et al., 2007),
and sugar beet +14 to +20% until the 2050s in England and Wales (Richter and
Semenov, 2005),… The impacts on autumn sown crops are more geographically
variable; yield is expected to strongly decrease in most southern areas, and increase
in northern or cooler areas (e.g., wheat: +3 to +4% by 2020, -8 to +22% by 2050, -15
to +32% by 2080) (Santos et al., 2002; Giannakopoulos et al., 2005; Audsley et al.,
2006; Olesen et al., 2007). (C3; minor)
We have a minor comment to make on this statement. It is not transparent how
the quantitative information on wheat yield and sugar beet as presented in this
statement (Chapter 12, page 555) was derived from the numbers given in the
cited references (C3). The IPCC authors explained to us in more detail how these
numbers had been established on the basis of the cited references. Although we
have no reason to doubt the correct underpinning of these results, we do think that
the provenance of presented quantitative information in IPCC reports – if it cannot
be derived from the cited references – should be clearly explained for reasons of
transparency and traceability. This minor comment has no consequences for the
IPCC conclusions in the various Summaries for Policymakers.
However, on balance health risks are very likely to increase (C3; major)
We have a major comment to make on this statement, which was taken from the
Executive Summary (page 543) and summarises the IPCC authors’ expert judgment
on the balance between positive and negative effects of climate change on health.
Supportive material is presented in the main text (see e.g. Section 12.4.11 on pages
557–558; Figure 12.3 on page 558; Table 12.1 on page 546, and Table 12.4 on page
565), which also provides a more regionally differentiated picture. However, a
quantitative underpinning for the statement is lacking in the main text (C3).
The IPCC authors indicated to us that they weighed the positive and negative
effects and this was their final assessment. Pages 557 to 558 of Chapter 12 give a
detailed account of all impacts, and we found the negative impacts to be the norm,
not the positive ones. However, although the list of negative effects is longer,
no indication of their relative weights is given, which makes the expert judgment
about the ‘balance’ in our view insufficiently transparent.
Although the statement was not included in the various Summaries for
Policymakers, many readers may consider it to be highly policy relevant.
This major comment has no consequences for the IPCC conclusions in the various
Summaries for Policymakers.

Europe

69

8.3 Findings from the PBL registration website
 or the first time, wide-ranging impacts of changes in current climate have been
F
documented: retreating glaciers, longer growing seasons, shift of species ranges,
and health impacts due to a heat wave of unprecedented magnitude. (minor)
We received a reaction to this statement which criticised the presenting of the
2003 heat wave as a consequence of climate change. In response to that reaction,
we wish to make a minor comment on this statement (although it does not fit one
of our qualification categories). The Working Group II Summary for Policymakers
states, for Europe, on page 14: ‘For the first time, wide-ranging impacts of changes in
current climate have been documented: retreating glaciers, longer growing seasons, shift
of species ranges, and health impacts due to a heat wave of unprecedented magnitude.
The observed changes described above are consistent with those projected for future
climate change.’ This text, as well as its counterparts in Table TS4.2 of the Technical
Summary (page 51) and in the Executive Summary (page 543), present the health
impacts from the 2003 heat wave as an example of ‘wide-ranging impact of
changes in current climate’. Thus, the text implicitly suggests that the 2003 heat
wave is the result of recent climate change. However, one can never attribute a
specific extreme weather event of the past – such as that particular heat wave
–to changes in current climate. In fact, Schär and Jendritzky (2004) concluded:
‘The European heatwave of 2003: was it merely a rare meteorological event or a first
glimpse of climate change to come? Probably both.’ The IPCC authors have indicated
that the statement in the Summary for Policymakers does not contain a formal
attribution – otherwise, the second sentence ‘the observed changes described
above are consistent with those projected for future climate change’ would not
have been necessary; the authors’ reaction is factually correct, but the wording
remains suggestive in our view. Chapter 1 of the Working Group II Report contains
statements that are more careful, assessing the evidence of observed increases in
heat waves in Europe. This minor comment has no consequences for the Summary
for Policymakers of the Synthesis Report.
References
Alcamo, J., M. Floerke and M. Maerker, 2007. Future long-term changes in global water resources
driven by socio-economic and climatic changes. Hydrological Sciences, 52, 247-275.
Alexandrov, V., J. Eitzinger, V. Cajic and M. Oberforster, 2002. Potential impact of climate change on
selected agricultural crops in north-eastern Austria. Glob. Change Biol., 8, 372-389.
Audsley, E., K.R. Pearn, C. Simota, G. Cojocaru, E. Koutsidou, M.D.A. Rounsevell, M. Trnka and V.
Alexandrov, 2006. What can scenario modelling tell us about future European scale agricultural
land use, and what not? Environ. Sci. Pol., 9, 148-162.
Ewert, F., M.D.A. Rounsevell, I. Reginster, M.J. Metzger and R. Leemans, 2005. Future scenarios of
European agricultural land use I. Estimating changes in crop productivity. Agr. Ecosyst. Environ.,
107, 101-116.
Giannakopoulos, C., M. Bindi, M. Moriondo, P. LeSager and T. Tin, 2005. Climate Change Impacts
in the Mediterranean Resulting from a 2oC Global Temperature Rise. WWF report, Gland
Switzerland. Accessed 01.10.2006 at http://assets.panda.org/downloads/medreportfinal8july05.
pdf.

70

Assessing an IPCC assessment

Olesen, J.E., T.R. Carter, C.H. Díaz-Ambrona, S. Fronzek, T. Heidmann, T. Hickler, T. Holt, M.I.
Mínguez, P. Morales, J. Palutikhof, M. Quemada, M. Ruiz-Ramos, G. Rubaek, F. Sau, B. Smith and
M. Sykes, 2007: Uncertainties in projected impacts of climate change on European agriculture
and terrestrial ecosystems based on scenarios from regional climate models. Climatic Change, 81,
S123-S143.
Richter, G. and M. Semenov, 2005: Re-assessing drought risks for UK corps using UKCIP02 climate
change scenarios. Final report of Defra project CC0368.
Schär, C. and G. Jendritzky, 2004: Climate change: hot news from summer 2003. Nature, 432, 559560.
Schröter, D., et al., 2004. Advanced Terrestrial Ecosystem Analysis and Modelling (ATEAM): Final
Report 2004. Section 5 and 6 and Annex 1 to 6. Accessed 22-4-2010 at http://www.pik-potsdam.de/
ateam/ateam_final_report_sections_5_to_6.pdf.
Santos, F.D., K. Forbes, and R. Moita, Eds., 2002: Climate change in Portugal: scenarios, impacts and
adaptation measures. SIAM project report, Gradiva, Lisbon, 456 pp.
Thuiller, W., S. Lavorel, M.B. Araújo, M.T. Sykes and I.C. Prentice, 2005: Climate change threats plant
diversity in Europe. Proc. Natl. Acad. Sci. U.S.A., 102, 8245-8250.

Europe

71

72

Assessing an IPCC assessment

Latin America

9

9.1 Analysis of statements in Table SPM.2 of the Synthesis report

 y mid century, increases in temperature and associated decreases in soil water are
B
projected to lead to gradual replacement of tropical forest by savanna in eastern
Amazonia. Semi-arid vegetation will tend to be replaced by arid-land vegetation.
This statement is fully supported by the underlying material.
There is a risk of significant biodiversity loss through species extinction in many
areas of tropical Latin America.
This statement is fully supported by the underlying material.
Productivity of some important crops is projected to decrease (a) and livestock
productivity to decline (b), with adverse consequences for food security. In
temperate zones, soybean yields are projected to increase (c). Overall, the number
of people at risk of hunger is projected to increase (d). (C2; major)
We subdivided this statement by inserting (a) to (d) between issues. We have one
comment to make on part (b).
Part (a) (Productivity of some important crops is projected to decrease) This is fully
supported by the underlying material.
Part (b) (Livestock productivity is projected to decline). The ‘livestock’ appears to be
limited to ‘cattle’ and this part of the statement is underpinned by references to
studies on Bolivia and central Argentina. The generalisation to other livestock,
which would include pigs, poultry, small ruminants and other animals (together also
comprising a substantial fraction of livestock production in Latin America) and to
the whole of Latin America, is not underpinned in Chapter 13. We therefore believe
that this is an ‘insufficiently founded generalisation’ (C2).
Statement (c) (In temperate zones, soybean yields are projected to increase). This is fully
supported by the underlying material.
Statement (d) (The number of people at risk of hunger is projected to increase). This is
fully supported by the underlying material.
Changes in precipitation patterns and the disappearance of glaciers are projected to
significantly affect water availability for human consumption, agriculture and energy
generation.
This statement is fully supported by the underlying material.

Latin America

73

9.2 Additional findings
 able 13.6 Net increases in the number of people living in water-stressed
T
watersheds. (E1; minor)
Table 13.6 (on page 598 of Chapter 13) combines the absolute number of people
that live under water-stressed conditions (for the non-climate cases, columns 2, 3
and 5) with the number of people that experience an increase in water stress due to
climate change (Arnell, 2004). The table’s caption, however, suggests all numbers
are increases, but this is not correct (E1a). This issue also had been recognised by
the IPCC authors, who have indicated to us that they intend to have an erratum
issued. Note that we consider this to be a minor inaccuracy and that we have no
comment to make about the table’s content. This minor inaccuracy has no impact
on the IPCC conclusions in the various Summaries for Policymakers.
...the frequency of occurrence of weather and climate extremes is likely to increase
in the future; as is the frequency and intensity of hurricanes in the Caribbean Basin.
(E1; minor)
This statement was taken from the Executive Summary of Chapter 13, page 583,
right column, second paragraph. The statement contains a minor inaccuracy. The
Technical Summary of the Working Group I 2007 Report (p. 74) states: ‘A synthesis
of the model results to date indicates, for a warmer future climate, increased peak wind
intensities and increased mean and peak precipitation intensities in future tropical
cyclones, with the possibility of a decrease in the number of relatively weak hurricanes,
and increased numbers of intense hurricanes. However, the total number of tropical
cyclones globally is projected to decrease’. We conclude that ‘frequency and intensity
of hurricanes’ should have read ‘frequency and intensity of strong hurricanes’
(E1a). This minor inaccuracy has no impact on the IPCC conclusions in the various
Summaries for Policymakers.
Up to 40% of the Amazonian forests could react drastically to even a slight
reduction in precipitation; this means that the tropical vegetation, hydrology and
climate system in South America could change very rapidly to another steady state.
(C6; minor)
We have a minor comment to make on this statement, which originates from
Section 13.4.1 of Chapter 13 (page 596). The statement was based on Rowell
and Moore (2000), which is a peer-reviewed report by the World Wide Fund for
Nature and the International Union for Conservation of Nature (WWF/IUCN) on
a global review of forest fires, and not a study on changes in vegetation due to
climate change. That report, in turn, was mainly based on Nepstad et al. (1999)
(in Nature). In our opinion, both documents were not the most obvious choice of
reference in this case, as their focus is on forest fires (and logging). More adequate
peer-reviewed, scientific journal literature would have been available to support
this statement, such as Cox et al. (2000; 2004) (C6). This minor comment has no
consequences for the IPCC conclusions in the various Summaries for Policymakers.
...some plants will become locally extinct because the elevation range would not
permit natural adaptation to temperature increase (FAO, 2002). (C5; minor)
This statement originates from Section 13.4.1 of Chapter 13 (page 596, right
column). We were unable to trace FAO (2002) and the URL in the references does

74

Assessing an IPCC assessment

not provide a link to the FAO website (C5). This issue also has been recognised by
the IPCC authors, who have indicated that they intend to have an erratum issued.
This minor comment has no consequences for the IPCC conclusions in the various
Summaries for Policymakers.
...the incidence of the coffee leafminer (Perileucoptera coffeella) and the nematode
Meloidogyne incognita are likely to increase in future in Brazil’s production area.
The number of coffee leaf miner cycles could increase by 4%, 32% and 61% in 2020,
2050 and 2080, respectively, under the SRES A2 scenarios (Ghini, 2007). (C5; minor)
This statement originates from Section 13.4.2 of Chapter 13 (page 597, right
column). We have a minor comment to make on this statement. The reference to
the paper by Ghini et al. (2007) was listed as having been submitted to the journal
Climatic Change, but was never published in Climatic Change (C5). It was published
in Pesquisa Agropecuária Brasileira, Brasília 43 (2): 187-194, in 2008 (Ghini et al.,
2008). We are of the opinion that referring to submitted papers that have not been
accepted for publication is incorrect, since they have not yet completed the peerreview process. Moreover, the reference may end up to be untraceable, as was
the case here. In our opinion, papers should at least be accepted for publication,
or be in press, so that traceability is guaranteed. This minor comment has no
consequences for the IPCC conclusions in the various Summaries for Policymakers.
A highly stressed condition is projected between 2015 and 2025 in the water
availability in Colombia, affecting water supply and ecosystem functioning in the
páramos (IDEAM, 2004), and very probably impacting on the availability of water
supply for 60% of the population of Peru (Vásquez, 2004). The projected glacier
retreat would also affect hydroelectricity generation in some countries, such as
Colombia (IDEAM, 2004) and Peru; one of the more affected rivers would be the
Mantaro, where an hydroelectric plant generates 40% of Peru’s electricity and
provides the energy supply for 70% of the country’s industries, concentrated in Lima
(UNMSM, 2004). (E2; minor)
In this statement, taken from Section 13.4.3 of Chapter 13 (page 598, right column)
we found that the references to UNMSM and Vasquez had been mixed up. The
statement on the water supply in Peru is not supported by the Vásquez (2004)
paper, but by UNMSM (2004). and the statement on hydro-electricity in the
Mantaro river is supported by Vásquez (2004) (E2). This minor inaccuracy has no
impact on the IPCC conclusions in the various Summaries for Policymakers.
References
Arnell, N.W., 2004. Climate change and global water resources: SRES emissions and socio-economic
scenarios. Global Environ. Change, 14, 31-52.
Cox, P.M., R.A. Betts, C.D. Jones, S. Spall and I.J. Totterdell, 2000. Acceleration of global warming
due to carbon-cycle feedbacks in a coupled climate model. Nature, 408, 184-187.
Cox, P.M., R.A. Betts, M. Collins, P. Harris, C. Huntingford, C.D. Jones, 2004. Amazon dieback under
climate-carbon cycle projections for the 21st century. Theor. Appl. Climatol. 78, 137–156. (doi:10.
1007/s00704-004-0049-4)
FAO, 2002. El cambio climático y los bosques. Boletín electrónico Julio 2002, FAO, Rome. http://
www.ecosur.net/cambio_climatico_y_los_bosques.html.
Ghini, R., E. Hamada, M.J. Pedro Júnior and J.A. Marengo, 2007. Climate change and coffee pests in
Brazil. Climatic Change, submitted.

Latin America

75

Ghini, R., E. Hamada, M.J. Pedro Júnior and J.A. Marengo, 2008. Risk analysis of climate change on
coffee nematodes and leaf miner in Brazil. Pesquisa Agropecuária Brasileira, Brasília 43 (2): 187194.
Nepstad, D.C., A. Veríssimo, A. Alencar, C. Nobre, E. Lima, P. Lefebvre, P. Schlesinger, C. Potter,
P. Mountinho, E. Mendoza, M. Cochrane, V. Brooks, 1999. Larg e scale impoverishment of
Amazonian forests by logging and fire, Nature, Vol. 398, 8 April.
Rowell, A. and P.F. Moore, 2000. Global Review of Forest Fires. WWF/IUCN, Gland, Switzerland, 66
pp. http://data.iucn.org/dbtw-wpd/edocs/2000-047.pdf.
UNMSM, 2004. Calor intenso y largas sequías. Especials, Perú. http://www.
unmsm.edu.pe/Destacados/contenido.php?mver=11.
Vásquez, O.C., 2004. El Fenómeno El Niño en Perú y Bolivia: Experiencias en Participación Local.
Memoria del Encuentro Binacional Experiencias de prevención de desastres y manejo de
emergencias ante el Fenómeno El Niño, Chiclayo, Peru. ITDG, 209 pp.

76

Assessing an IPCC assessment

North America

10

10.1 Analysis of statements in Table SPM.2 of the Synthesis Report

 arming in western mountains is projected to cause decreased snowpack, more
W
winter flooding and reduced summer flows, exacerbating competition for overallocated water resources.
This statement is fully supported by the underlying material.
In the early decades of the century, moderate climate change is projected to
increase aggregate yields of rain-fed agriculture by 5 to 20%, but with important
variability among regions. Major challenges are projected for crops that are near the
warm end of their suitable range or which depend on highly utilised water resources.
(C3; minor)
From Chapter 14 and the references therein we could not derive the reasoning that
led to the 5 to 20% range of yield increases under moderate climate change. Of the
six references named in the chapter, Adams et al. (2003), appears to be the main
reference. Adams et al. report (in their Table I) changes in rain-fed yield, from –8% to
+41% for 2060 (with adaptation) (C3). Adams et al. and other references (Thomson
et al. 2005, Tsvetsinkaya et al., 2003) show there are regions with significant yield
decreases for specific rain-fed crops (5 to 25%), even if adaptation measures are
adopted. It could be argued that this observation is covered by the phrase ‘but with
important variability among regions’, but this could not be derived from the chapter
text. The IPCC authors provided us with the reasoning that had been followed: for
Adams et al. (2003), the projected yield changes for the four food crops considered
(corn, soy, wheat, sorghum) are +7, +17, +13, and +5% respectively, according to the
Regional Climate Model with adaptation. The whole idea of that paper was that
the Regional Climate Model gave more credible projections than the Global Climate
Model, so those are the numbers that the IPCC authors used. Furthermore, they
confirmed that they highlighted variability due to, for example, different varieties
of wheat with the statement that there is ‘important variability among regions’.
They found it hard to see how they could have been more precise without reporting
individual values from tables. We believe that the above explanation of the
reasoning behind the 5 to 20% range could have been made more transparent with
just a few extra lines of text. We wish to add here that the IPCC authors’ expert
judgment had been based on a consideration of a vast body of literature and we
have no reason to doubt the quality and correctness of the expert judgment.

North America

77

Cities that currently experience heat waves are expected to be further challenged by
an increased number, intensity and duration of heat waves during the course of the
century, with potential for adverse health impacts).
This statement is fully supported by the underlying material.
Coastal communities and habitats will be increasingly stressed by climate change
impacts interacting with development and pollution.
This statement is fully supported by the underlying material.

10.2 Additional findings
 igures TS.15 and 14.1 on the economic damages from hurricanes. (minor)
F
Figures TS.15 (Technical Summary, page 55) and 14.1 (Chapter 14, page 621) in
the Working Group II Report illustrate the statement in the Technical Summary
that reads ‘Over the past several decades, economic damage from hurricanes in
North America has increased over fourfold, due largely to an increase in the value of
infrastructure at risk’. We have a minor comment to make on the figures. Because
of a shorter averaging period at the end of the time series (6 years instead of
10), the most recent increase was higher, compared with the increase that would
have resulted from applying a uniform averaging period at the end of the series.
However, a different choice of statistical method would not have affected the
statement. Furthermore, we have found this statement to be fully supported by
the underlying material. This minor comment has no consequences for the IPCC
conclusions in the various Summaries for Policymakers.
References
Adams, R.M., B.A. McCarl and L.O. Mearns, 2003. The effects of spatial scale of climate scenarios on
economic assessments: An example from U.S. agriculture. Clim. Change, 60, 131-148.
Brown, T. J., B. L. Hall and A. L.Westerling, 2004. The impact of twenty-first century climate change
on wildland fire danger in the western United States:An applications perspective. Clim. Change,
62, 365-388.
Flannigan, M.D., K.A. Logan, B. D.Amiro,W. R. Skinner and B. J. Stocks, 2004. Future area burned in
Canada. Clim. Change, 72, 1-16.
Thomson, A.M., R.A. Brown, N.J. Rosenberg, R.C. Izaurralde and V. Benson, 2005. Climate change
impacts for the conterminous USA:An integrated assessment. Part 3:Dryland production of grain
and forage crops. Clim. Change, 69, 43-65.
Tsvetsinskaya, E.A., L.O. Mearns, T. Mavromatis, W. Gao, L. McDaniel and M.W. Downton, 2003. The
effect of spatial scale of climatic change scenarios on simulated maize, winter wheat, and rice
production in the southeastern United States. Clim. Change, 60, 37-72.

78

Assessing an IPCC assessment

Polar regions
(Arctic and Antarctic)

11

11.1 Analysis of statements in Table SPM.2 of the Synthesis Report
 he main projected biophysical effects are reductions in thickness and extent of
T
glaciers, ice sheets and sea ice, and changes in natural ecosystems with detrimental
effects on many organisms including migratory birds, mammals and higher
predators.
This statement is fully supported by the underlying material.
For human communities in the Arctic, impacts, particularly those resulting from
changing snow and ice conditions are projected to be mixed.
This statement is fully supported by the underlying material.
Detrimental impacts would include those on infrastructure and traditional
indigenous ways of life.
This statement is fully supported by the underlying material.
In both polar regions, specific ecosystems and habitats are projected to be
vulnerable, as climatic barriers to species invasions are lowered.
This statement is fully supported by the underlying material.

Polar regions (Arctic and Antarctic)

79

80

Assessing an IPCC assessment

Small islands

12

12.1 Analysis of statements in Table SPM.2 of the Synthesis Report

 ea level rise is expected to exacerbate inundation, storm surge, erosion and other
S
coastal hazards, thus threatening vital infrastructure, settlements and facilities that
support the livelihood of island communities.
This statement is fully supported by the underlying material.
Deterioration in coastal conditions, for example through erosion of beaches and
coral bleaching, is expected to affect local resources.
This statement is fully supported by the underlying material.
By mid-century, climate change is expected to reduce water resources in many
small islands, e.g. in the Caribbean and Pacific, to the point where they become
insufficient to meet demand during low-rainfall periods.
This statement is fully supported by the underlying material.
With higher temperatures, increased invasion by non-native species is expected to
occur, particularly on mid- and high-latitude islands.
This statement is fully supported by the underlying material.

12.2 Additional findings
L ower rainfall coupled with accelerated sea-level rise compounds the threat on
water resources; a 10% reduction in average rainfall by 2050 is likely to correspond
to a 20% reduction in the size of the freshwater lens on Tarawa Atoll, Kiribati. (C3;
minor)
We have a minor comment to make on this statement. The statement appears in
the Technical Summary (Table TS2, page 58; Box TS.6, page 63), in the Executive
Summary of Chapter 16 (page 689) and in the main chapter text (pages 696 and
697), and is indicated to have been based on a World Bank report (World Bank,
2000). However, this reference indicates a 14% reduction in the freshwater lens,
as a consequence of a 10% reduction in precipitation. Figures change when other
factors combine with changes in precipitation; −12% change in the freshwater lens
is projected for 0.4 m sea level rise, and −38% for 0.4 m sea level rise in combination
with reduced island size due to erosion. The World Bank report also notes that a
10% increase in evapotranspiration would lead to a 6% reduction in the freshwater
lens. The chapter text is not clear on why the effect of a projected 10% increase in
evapotranspiration should be added to the effect of a projected 10% reduction in

Small islands

81

precipitation. The Coordinating Lead Authors explained to us that these changes
are correlated and that, hence, the 6% effect from evapotranspiration change
should indeed be added to the 14% effect from changes in precipitation. The
statement is correct, but we believe that this last point should have been explained
in the text. This minor comment has no consequences for the IPCC conclusions in
the various Summaries for Policymakers.
Less rainfall coupled with accelerated sea-level rise would compound this threat.
Studies conducted on Bonriki Island in Tarawa, Kiribati, showed that a 50 cm rise in
sea level accompanied by a reduction in rainfall of 25% would reduce the freshwater
lens by 65% (World Bank, 2000). (E2; minor)
Another instance of inaccurate referencing was found in the main text of Chapter
16 (page 697). However, the World Bank study does not report a 50 cm sea level
rise. The reference should have been to a study by Falkland (1999) (E2). This minor
inaccuracy has no impact on the IPCC conclusions in the various Summaries for
Policymakers.
References
World Bank, 2000: Cities, Seas and Storms: Managing Change in Pacific Island. Economies. Vol. IV:
Adapting to Climate Change. World Bank, Washington, District of Columbia, 135 pp
Falkland, A., 1999: Impacts of climate change on water resources of Pacific islands. PACCLIM 19
Workshop, Modelling the Effects of Climate Change and Sea Level Rise in Pacific Island 20
Countries. Auckland, New Zealand, 33pp.

82

Assessing an IPCC assessment

Annex A Table SPM.2 of the
Synthesis Report: examples of
some projected regional impacts
This table including its footnote is a direct copy of the original table SPM.2 on page
11 of Climate Change 2007 – Synthesis Report, A report of the IPCC.

Annex A

83

Table SPM.2. Examples of some projected regional impacts {3.3.2}
Africa

Table A.1

• By 2020, between 75 and 250 million of people are projected to be
exposed to increased water stress due to climate change.
• By 2020, in some countries, yields from rain-fed agriculture could be reduced by up to 50%. Agricultural
production, including access to food, in many African countries is projected to be severely
compromised. This would further adversely affect food security and exacerbate malnutrition.
• Towards the end of the 21st century, projected sea level rise will affect low-lying coastal areas with large
populations. The cost of adaptation could amount to at least 5 to 10% of Gross Domestic Product (GDP).
• By 2080, an increase of 5 to 8% of arid and semi-arid land in Africa is
projected under a range of climate scenarios (TS).

Asia

• By the 2050s, freshwater availability in Central, South, East and South-East
Asia, particularly in large river basins, is projected to decrease.
• Coastal areas, especially heavily populated megadelta regions in South, East and South-East Asia, will be at
greatest risk due to increased flooding from the sea and, in some megadeltas, flooding from the rivers.
• Climate change is projected to compound the pressures on natural resources and the environment
associated with rapid urbanisation, industrialisation and economic development.
• Endemic morbidity and mortality due to diarrhoeal disease primarily associated with floods and droughts
are expected to rise in East, South and South-East Asia due to projected changes in the hydrological cycle.

Australia and
New Zealand

• By 2020, significant loss of biodiversity is projected to occur in some ecologically
rich sites, including the Great Barrier Reef and Queensland Wet Tropics.
• By 2030, water security problems are projected to intensify in southern and eastern
Australia and, in New Zealand, in Northland and some eastern regions.
• By 2030, production from agriculture and forestry is projected to decline over much of southern
and eastern Australia, and over parts of eastern New Zealand, due to increased drought and
fire. However, in New Zealand, initial benefits are projected in some other regions.
• By 2050, ongoing coastal development and population growth in some areas of
Australia and New Zealand are projected to exacerbate risks from sea level rise and
increases in the severity and frequency of storms and coastal flooding.

Europe

• Climate change is expected to magnify regional differences in Europe’s natural resources
and assets. Negative impacts will include increased risk of inland flash floods and more
frequent coastal flooding and increased erosion (due to storminess and sea level rise).
• Mountainous areas will face glacier retreat, reduced snow cover and winter tourism, and
extensive species losses (in some areas up to 60% under high emissions scenarios by 2080).
• In southern Europe, climate change is projected to worsen conditions (high temperatures
and drought) in a region already vulnerable to climate variability, and to reduce water
availability, hydropower potential, summer tourism and, in general, crop productivity.
• Climate change is also projected to increase the health risks due to heat waves and the frequency of wildfires.

Latin America

• By mid-century, increases in temperature and associated decreases in soil water are
projected to lead to gradual replacement of tropical forest by savanna in eastern
Amazonia. Semi-arid vegetation will tend to be replaced by arid-land vegetation.
• There is a risk of significant biodiversity loss through species extinction in many areas of tropical Latin America.
• Productivity of some important crops is projected to decrease and livestock productivity to decline, with
adverse consequences for food security. In temperate zones, soybean yields are projected to increase.
Overall, the number of people at risk of hunger is projected to increase (TS; medium confidence).
• Changes in precipitation patterns and the disappearance of glaciers are projected to significantly
affect water availability for human consumption, agriculture and energy generation.

84

Assessing an IPCC assessment

North America

• Warming in western mountains is projected to cause decreased snowpack, more winter flooding
and reduced summer flows, exacerbating competition for over-allocated water resources.
• In the early decades of the century, moderate climate change is projected to increase aggregate yields of rainfed agriculture by 5 to 20%, but with important variability among regions. Major challenges are projected for
crops that are near the warm end of their suitable range or which depend on highly utilized water resources.
• Cities that currently experience heat waves are expected to be further challenged
by an increased number, intensity and duration of heat waves during the
course of the century, with potential for adverse health impacts.
• Coastal communities and habitats will be increasingly stressed by climate
change impacts interacting with development and pollution.

Polar Regions

• The main projected biophysical effects are reductions in thickness and extent of glaciers,
ice sheets and sea ice, and changes in natural ecosystems with detrimental effects on
many organisms including migratory birds, mammals and higher predators.
• For human communities in the Arctic, impacts, particularly those resulting from
changing snow and ice conditions, are projected to be mixed.
• Detrimental impacts would include those on infrastructure and traditional indigenous ways of life.
• In both Polar Regions, specific ecosystems and habitats are projected to be
vulnerable, as climatic barriers to species invasions are lowered.

Small Islands

• Sea level rise is expected to exacerbate inundation, storm surge, erosion and other coastal hazards, thus
threatening vital infrastructure, settlements and facilities that support the livelihood of island communities.
• Deterioration in coastal conditions, for example through erosion of beaches
and coral bleaching, is expected to affect local resources.
• By mid-century, climate change is expected to reduce water resources in many small islands, e.g. in the
Caribbean and Pacific, to the point where they become insufficient to meet demand during low-rainfall periods.
• With higher temperatures, increased invasion by non-native species is
expected to occur, particularly on mid- and high-latitude islands.
Note:
Unless stated explicitly, all entries are taken from the text of the Working Group II Summary for Policymakers. They are either ‘very high confidence’ or ‘high confidence’ statements, related to different
sectors (agriculture, ecosystems, water, coasts, health, industry and settlements). The Working Group
II Summary for Policymakers refers to the source of the statements, timelines and temperatures. The
magnitude and timing of impacts that may ultimately occur will vary according to the amount and
rate of climate change, emissions scenarios, development pathways and adaptation.

Annex A

85

Annex B The error on the
melting of the Himalayan
glaciers
In the media, much attention was paid to this erroneous statement in the chapter
on Asia, page 493 of Section 10.6.2, which suggested the disappearance of
Himalayan glaciers by 2035:

Glaciers
in the Himalaya are receding faster than in any other part of the world [...]
the likelihood of them disappearing by the year 2035 and perhaps sooner is very
high if the Earth keeps warming at the current rate. Its total area will likely shrink
from the present 500,000 to 100,000 km2 by the year 2035 (E1; major)
Here, we provide an analysis of this error and some background information,
which also provides an update that includes literature that became available after
production and publication of the Fourth Assessment Report.
In the Synthesis Report (IPCC, 2007) and in the Summary for Policymakers (SPM),
and the Technical Summary (TS) of the Working Group I contribution to the IPCC
Fourth Assessment Report (Solomon et al., 2007), no explicit statements were
made on Himalayan glacier retreat. In the Summary for Policymakers of the
Working Group II Report (Parry et al., 2007), there was only one reference to the
Himalayas:
Glacier melt in the Himalayas is projected to increase flooding, and rock avalanches from
destabilised slopes, and to affect water resources within the next two to three decades. This will
be followed by decreased river flows as the glaciers recede.

This statement is rather general and correct, although the impact on water
resources due to glacier retreat is very likely to be rather limited in the next two to
three decades. However, in the Technical Summary, a major error occurred in Box
TS6, on page 59:
If current warming rates are maintained, Himalayan glaciers could decay at very rapid rates,
shrinking from the present 500,000 km2 to 100,000 km2 by the 2030s.

This text was based on page 493 of Section 10.6.2 of Working Group II Report,
which states:

86

Assessing an IPCC assessment

Glaciers in the Himalaya are receding faster than in any other part of the world [...] the
likelihood of them disappearing by the year 2035 and perhaps sooner is very high if the Earth
keeps warming at the current rate. Its total area will likely shrink from the present 500,000
to 100,000 km2 by the year 2035 (WWF 2005).

The first mention of ‘2035’ was taken from a publication by the World Wide Fund for
Nature (WWF, 2005), which cited a news story (Pearce, 1999) about an unpublished
study (Hasnain, 1999) that does not estimate a date for disappearance of Himalayan
glaciers.
The second mention of ‘2035’, again a major error, was not taken from the (WWF,
2005), but could be traced to a rough estimate (Kotlyakov, 1996) on the shrinkage
of all non-polar glaciers (excluding those in basins of internal drainage) between
the present day and the year 2350. Furthermore, the Himalayan glaciers do not
cover 500,000 km2, but about 30,000 km2, as was correctly indicated in the first
sentence of Section 10.6.2, which stated that: Himalayan glaciers cover about three
million hectares.
In conflict with knowledge of glacier–climate relationships, disappearance by 2035
would require a 25-fold acceleration of the loss rate during the 1999-2035 period,
compared to the estimated past loss rate between 1960 and 1999 (Dyurgerov and
Meier, 2005), a situation which is extremely unlikely to occur.
Here, we provide some additional background information, mainly based on a
study by Oerlemans et al. (2007) and on a recent American Geophysical Union
lecture by leading experts in this field (Kargel et al., 2010). The lecture was held
on 14 December 2009 in San Francisco, in response to media attention concerning
the claim in the IPCC Working Group II Report that the Himalayan glaciers might be
gone by 2035 (Parry et al., 2007).
Glacier retreat and water supply
The region-wide average glacier area retreat rate is probably between 0.1 and 0.5%
per year. Although the size of the glacier area will change substantially this century,
especially in the most vulnerable eastern zone of the Himalayas, glaciers (such as
Khumbu and Imja), will not disappear entirely, or even mostly, by 2035, as stated in
the Working Group II contribution to the Fourth Assessment Report of the IPCC.
All glaciers combined store roughly 3,400 km3 of fresh water. This represents
almost three years in combined water discharges from the Indus, the Ganges and
the Brahmaputra. The prevalent negative balances of Himalayan glaciers (meaning
the annual net withdrawals of ice) total between 0.4 and 1.2% of current river flows,
which is not very significant, basin-wide. So, although over half a billion people
draw water from these melting glaciers, this only represents between one and a
few per cent of their water supplies.
Increased melting may further increase water discharge by 1 to 2%, in the next few
decades. Within 50 to 100 years, the discharge could decrease by several per cent,
as the current negative balance contribution decreases. However, on the scale of
the subcontinent, hydrological problems are likely to stem more from population

Annex B

87

growth, inefficiency in water resource distribution and application, and from
changes in the Monsoon and the Westerlies, due to global warming and changes in
the Elevated Heat Pump (EHP). Changes in the Monsoon and the Westerlies already
appear to be substantial and are likely to be increasingly impactful.
On a local scale, the seasonal influence of glacier melt can be strong, and even
rivers as large as the Indus fluctuate widely throughout the year because of the
seasonality of water storage (mostly in the form of ice), and melting of both
glaciers and high altitude snow fields. The greatest vulnerability of large numbers of
people to glacier state and dynamics is to those in arid and semi-arid lands, such as
north-eastern Afghanistan, Uzbekistan, parts of Pakistan, and western China. This
concerns tens of millions of people.
Sea level rise
Since 1900, the majority of the observed glaciers around the world are retreating,
including those in central Asia. Based on Oerlemans et al. (2007), global glacial
contribution to sea level rise was 4.5±0.7 cm during the 1900-2000 period,
compared to a total sea level rise of ~19.5 cm in the 1870-2004 period. This estimate
is higher than the IPCC estimate of 3.0±1.0 cm for the 20th century, and better
explains the observed sea level rise as caused by thermal expansion and changes
in the cryosphere. Based on Oerlemans et al. (2007) and Kargel et al. (2010), it is
roughly estimated that the annual contribution from Himalayan glaciers to sea level
rise is 0.06±0.04 mm, or ~2% of the current annual sea level rise of 3.1±0.7 mm (Parry
et al., 2007). The rate of Himalayan glacier melting appears to have increased over
time, but so has sea level rise, and therefore the 2% contribution may still apply.
References
Dyurgerov, M.B. and M.F. Meier, 2005. Glaciers and the changing earth system, a 2004 snapshot,
Occasional Paper No. 58. Boulder, Institute of Arctic and Alpine Research, University of Colorado:
117
Hasnain, S.I., 1999. Report on Himalayan Glaciology. New Dehli, ICSI working group on Himalayan
Glaciology: 22.
IPCC, 2007. Climate Change 2007. Synthesis report. Contribution of Working Groups I, II and III to the
Fourth Assessment Report of the Intergovernmental Panel on Climate Change. Core writing team:
Pachauri, R.K. and Reisinger, A. (Eds.). Geneva, Switzerland.
Kargel, J.S., R. Armstrong et al., 2010. Satellite-era glacier changes in High Asia. Background support
presentation for NASA “Black Carbon and Aerosols” press conference associated with Fall AGU,
Dec. 14, 2009. Updated and expanded Feb. 17, 2010.
Kotlyakov, V.M., 1996. Variations of Snow and Ice in the past and at present on a Global and Regional
Scale. I. H. PROGRAMME. Paris, UNESCO.
Oerlemans, J., M. Dyurgerov et al., 2007. "Reconstructing the glacier contribution to sea-level rise
back to 1850." The Cryosphere 1: 59-65.
Parry, M.L., O.F. Canziani et al., 2007. Climate Change 2007: Impacts, Adaptation and Vulnerability.
Cambridge, Cambridge University Press.
Pearce, F., 1999. Flooded Out. New Scientist. 162.
Solomon, S.D., M.M. Qin et al., 2007. Climate Change 2007: The Physical Science Basis. Contribution
of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate
Change., Cambridge University Press, Cambridge, United Kingdom and New York, NY, USA.
WWF, 2005. An overview of Glaciers, glacier retreat and its subsequent impacts in Nepal, India and
China: 67.

88

Assessing an IPCC assessment

Annex C The error on
the percentage of Dutch
land area below sea level
In February 2010, an error was detected in a text supplied by the Netherlands
Environmental Assessment Agency (PBL) for Chapter 12 (on Europe), page 547,
Subsection 12.2.3, of the Working Group II contribution to the Fourth Assessment
Report, regarding the risks of flooding in the Netherlands:
 he Netherlands is an example of a country highly susceptible to both sea-level rise
T
and river flooding because 55% of its territory is below sea level where 60% of its
population lives and 65% of its Gross National Product (GNP) is produced. (E1;
major)
The error was discovered by a Dutch journalist of a weekly newspaper (Vrij
Nederland) who checked information about the Netherlands, triggered by the
publicity around the error on the Himalayan glaciers. On page 547 of the chapter
text, it says that 55% of the Netherlands is located below sea level. This should
have read that 55% of the Netherlands is at risk of flooding (E1); 26% of the country
is at risk because it is located below sea level, and another 29% is susceptible to
river flooding.1) Examples of the latter are the near floodings in the mid-1990s,
of areas along the rivers Meuse and Waal – areas that are well above sea level.
The Netherlands is sensitive to climate change. Sea level rise as well as peak river
discharges have been analysed frequently, and according to government policy this
requires precautionary measures.
The original text that included this error was submitted by the Netherlands
Environmental Assessment Agency (PBL) in mid 2005. A similar error also occurred
in publications by others. A commentary that was published in Nature in 2005
(Kabat et al., 2005) refers to 60% of the Dutch land area as being located below sea
level. In 2007, an English version of a note from the Dutch Ministry of Transport
also stated (Dutch Ministery of Transport, 2007) that about 60% of the country was
1) Here, the area of the Netherlands that is prone to flooding has been divided in a binary manner, starting with determining the part of the country that lies below NAP (a particular reference sea level) – which is 26% – the rest of the country at risk
of flooding is then categorised as being at risk of flooding by rivers; of course, more finegrained subdivisions could also have
been made, since obviously the part of the country that lies below sea level is also at risk of flooding by rivers, and parts of the
country that lie above NAP are also at risk of flooding by the sea.

Annex C

89

situated below sea level. It would have been correct to state that about 60% of
the Netherlands would be prone to flooding by sea if the sea level would rise by 1
m above the 2000 level and a severe storm would hit the coast during spring tide
– but this is not the same thing. In the review process of the IPCC Working Group
II Report, there were several comments and suggested changes relating to this
chapter, but this error remained unnoticed.
Immediately after detection, the PBL reported the error to the international
community. It was also reported to the IPCC, and on 27 April 2010 is was included in
a list of errata on IPCC reports. The incorrect wording in the IPCC report does not
affect the message of the conclusion. The error has no significance for the main
conclusions in the chapter, or for those on Europe, nor for the conclusions in the
various Summaries for Policymakers.
We acknowledge that this error was not the fault of the IPCC (Coordinating)
Lead Authors or Co-Chairs. The error was made by a Contributing Author from
the PBL, and the (Coordinating) Lead Authors are not to blame for relying on
Dutch information provided by a Dutch agency. The lesson to be learned for an
assessment agency such as ours, is that quality control is needed at the primary
level of a literature assessment. This should be done by checking basic data, to
the extent feasible, even when authoritative references are available. Of course,
there are limitations to checking data, given the vast amounts of specific regional
information.
References
Kabat, P., Vierssen, W. van, Veraart, J.A., Vellinga, P., Aerts, J., 2005. Climate proofing the
Netherlands. Nature, vol. 438, p. 283-284
Ministry of Transport, Public Works and Water Management, 2007. Flood maps in the Netherlands,
http://www.safecoast.org/editor/databank/File/folder%20engels%20def%201%20febr07.pdf

90

Assessing an IPCC assessment

Annex D Sea level rise:
consequences for the
Netherlands
 ummary
S
Errors in the IPCC report regarding the Himalayan glaciers and the Dutch land area
below sea level have triggered a discussion on whether the Dutch policies that deal
with sea level rise should be revised. The answer is clearly ‘no’: the contribution
from melting Himalayan glaciers to sea level rise is very limited (around 2%). Moreover, the policy concerning safety against flooding in the Netherlands, as formulated
in the National Water Policy Plan 2009-2015 (Ministry of Transport, Public Works and
Water Management, 2009) and elaborated in the Delta Programme, is well founded
on the available knowledge on climate change, sea level rise, river discharges and
uncertainties. The current discussions related to the IPCC have no effect on the
assumptions under the National Water Policy Plan and the Delta Programme.
There is scientific agreement that the main causes of mean global sea level rise
are thermal expansion of sea water and melting of land ice due to increasing
temperatures. However, future projections of sea level rise are uncertain because
of the large range of temperature projections and the limited knowledge about
the melting behaviour of especially the arctic ice shields (PBL2007, Katsman et al.,
2008).
Sea level rise: limited contribution from the Himalayan glaciers
Over the last century, the sea levels have risen, world wide, by 14 to 20 cm, corresponding with an average of 1.7±0.3 mm a year (Church et al., 2006). Along the
Dutch coast, the sea level rise was of the same order of magnitude: 19 cm with an
average of 1.8±0.2 mm a year (PBL, 2010). Satellite data indicate an accelerated
trend in global sea level rise of 3.1±0.7 mm a year, over the past 15 years (Church
and White, 2006; Rahmstorf et al., 2007). It is unclear whether this acceleration
will continue. The effect of glacier melting in the Himalayas on global sea level
rise is limited: based on Oerlemans et al. (2007) and Kargel et al. (2010), it is estimated that the annual contribution from the Himalayan glaciers to sea level rise
is 0.06±0.04 mm, that is, around 2% of the current annual sea level rise of 3.1±0.7
mm (Parry et al., 2007). Therefore can be concluded that the contribution from the
melting of the Himalayan glaciers to sea level rise is very limited.

Annex D

91

Estimates on future sea level rise for the Netherlands: large uncertainties
There is scientific agreement that the main causes of mean global sea level rise
are thermal expansion of sea water and the melting of land ice due to increasing
temperatures. However, future projections of sea level rise are uncertain because
of the large range of temperature projections and the limited knowledge about
the melting behaviour of especially the arctic ice shields (PBL, 2007; Katsman et
al., 2008). The range for temperature rise is due to i) natural variability, such as the
recent slowing down of the increase in average global temperature, ii) the climate
sensitivity of the atmosphere, iii) uncertainties in atmospheric processes, and iv)
uncertainties in future emission of greenhouse gases.
When considering the possible sea level rise for the Netherlands, the most recent
estimates on sea level rise for the Netherlands cover a range of 35 to 85 cm for 2100
(KNMI, 2006), or in case of high-end/worst-case estimates, the rise is between 130
cm (Deltacommissie, 2008) and 150 cm (PBL, 2007).
In 2007, in collaboration with the Royal Netherlands Meteorological Institute
(KNMI), the PBL explored plausible future extreme sea level rise scenarios for the
Netherlands and – based on paleoclimatological data – estimated a worst-case sea
level rise of 1.5 m/century (PBL, 2007). Given the technical adaptive capacity of the
Netherlands and the considered safety margins, the PBL concluded that, with the
available techniques, the delta region of the Netherlands could be kept safe even
in case of such an extreme sea level rise, but that in the long term, spatial measures
could be required (PBL, 2007). In 2008, the Dutch Delta committee (Deltacommissie) presented a high-end estimate on sea level rise of between 65 and 130 cm for
2100, based on a temperature-rise scenario of 6 degrees, and also concluded that
technically the delta region of the Netherlands could protect itself against flooding (Deltacommissie 2008). The maximum ranges for sea level rise given by the
IPCC (2007) and the KNMI (2006), were based on scenarios with a maximum of 4
degrees temperature rise. A 6-degree temperature rise thus exceeds the 4-degree
scenario of the IPCC, but still lies within the total range of uncertainty with respect
to temperature rise (e.g. PBL, 2009, News in Climate Science).
Consequences for policy on safety against flooding
Following the advice of the Delta committee in 2008, the policy on safety against
flooding and freshwater management will be revised, given the increases in population and economical values in the area that has been sensitive to flooding since
the 1960s, and the challenges imposed by climate change. The recent National
Water Policy Plan 2009-2015 (Dutch Ministry of Transport, Public Works and Water
Management, 2009) has announced that i) the national standards for the safety
against flooding will be reconsidered, and ii) a long-term adaptive strategy on
climate proofing of the Netherlands will be constructed, including an analysis of
possible consequences for spatial development. To effectuate these points, the
Delta Programme has been launched, together with a Delta Act, Delta Fund and
a Delta Commissioner responsible for the coordination of the programme. In this
Delta Programme, the four available KNMI scenarios (KNMI, 2006) are taken as the
plausible range for future uncertainties for the Netherlands, with respect to climate
change and sea level rise.

92

Assessing an IPCC assessment

Current investments in the protection against flooding in the Netherlands are
justified by the fact that a substantial number of dikes (24%) did not meet presentday safety standards because of insufficient maintenance and new hydrological
conditions. The investments that are required for the improvement of the dikes
are not fully known yet, because for 32% of the dikes, the quality status is currently
unknown (Dutch Ministry of Transport, Public Works and Water Management,
2006).
Given that sea level rise is a slow process and that the uncertainties in future projections are considerable, future additional investments in the safety of the Netherlands will depend on progressing scientific insights. Sea level rise is addressed in
the current policy on safety against flooding. For safety measures in coastal areas,
decisions with a short lead time (5 years) are based on the present sea level rise of
20 cm/century and – based on earlier KNMI scenarios – decisions with lead times of
50 to 100 years are based on 60 cm sea level rise. Spatial reservations for the long
term (100 to 200 years) are based on a high sea level rise of 85 cm. Possible impacts
of climate change from floods due to extreme river discharges have also been
incorporated in the management strategy ‘Room for the River’ through spatial
reservations in certain areas for which additional measures could be required in the
future. Finally, the process of regular evaluations of the quality of the dikes, every
six years, will include the advancing knowledge about climate change, sea level
rise and its consequences (Dutch Ministry of Transport, Public Works and Water
Management, 2009).
An additional, important factor that will determine river discharges in the Netherlands is the river and safety management upstream in the Rhine and Meuse river
basins. Since 2009, the EU High Water Directive has been operational. It provides a
legal basis for international appointments in flood risk management on the scale of
transboundary river basins.
Considering the slow process of sea level rise, the adaptive capacity of the Netherlands with respect to technical measures (e.g., strengthening of dikes, storm barriers, dunes) is thought to be sufficient – even in worst-case scenarios – provided
that the required budgets become available (PBL, 2007; Deltacommissie, 2008).
However, the study by the PBL (2007) and Ligtvoet et al. (2009) point out that, in
the long run, management of peak river discharges will be the main challenge for
the Netherlands, and that adaptive capacity concerning spatial developments is
limited, because of their relative slowness and irreversibility. Therefore, choices
that will be made in the coming decades, in urban development, nature development and configuration of the main water system in the Netherlands, will also
determine the future climate resilience of the Netherlands (PBL, 2009). It has been
shown that a combination of spatial and technical measures regarding river areas,
IJsselmeer area and the south-western delta, may contribute most to reducing the
vulnerability of the Netherlands, in both the short term and the long term (Ligtvoet
et al., 2009).
Conclusion
The current policy on safety against flooding in the Netherlands, as formulated
in the National Water Policy Plan and elaborated in the Delta Programme, is well

Annex D

93

founded on the available knowledge on climate change, sea level rise, river discharges and uncertainties. The current discussions about the IPCC have no effect on
the assumptions underlying the Dutch National Water Plan.
The required investments in the period up to 2020 (although not fully known yet)
are to ensure that the quality of the dikes will meet present-day standards (Water
Act). The required future maintenance and investment levels will depend on the
new safety standards and the measures that will be adopted within the framework of a long-term adaptive strategy. The recently launched Delta Programme
within the framework of the National Water Policy Plan 2009-2015 (Dutch Ministry
of Transport, Public Works and Water Management, 2009) contains an extensive
research agenda to focus future research on safety against flooding and on freshwater supply in the Netherlands. In addition, the agenda lists the drafting of a longterm adaptive strategy, addressing the still large uncertainties concerning climate
change. For 2015, further policy decisions are expected on extra measures and
investments for safety against flooding and for freshwater supply. On a political
level, it has been decided that within the context of the Delta Programme, by 2020,
financial reservations will be made for additional investments and maintenance.
References
Church, J.A., and N.J. White, 2006. A 20th century acceleration in global sea-level rise, Geophys.
Res. Lett., 33, L01602, doi:10.1029/2005GL024826. Update data to 2007 on PSMSL website (link to
data)
Deltacommissie, 2008. Samen werken aan water. Bevindingen van de Deltacommissie.
Deltacommissie, Den Haag, the Netherlands.
Kargel, J.S., R. Armstrong et al., 2010. Satellite-era glacier changes in High Asia.Background support
presentation for NASA “Black Carbon and Aerosols” press conference associated with Fall AGU,
Dec. 14, 2009. Updated and expanded Feb. 17, 2010.
Katsman, C.A., W.Hazeleger, S.S. Drijfhout, G.J. van Oldenborgh & G. Burgers, 2008. Climate
scenarios of sea level rise for the northeast Atlantic Ocean: a study including the effects of ocean
dynamics and gravity changes induced by ice melt, Climatic Change 91:351–374
KNMI, 2006. Climate change scenarios for the Netherlands. Scientific report WR 2006-01. Koninklijk
Nederlands Meteorologisch Institute, Bilthoven, the Netherlands.
Ligtvoet, W.J. Knoop, B. Strengers & A. Bouwman, 2009. Flood protection in the Netherlands:
framing long term challenges and options for a climate resilient delta. PBL publication number
500078004. Netherlands Environmental Assessment Agency – PBL, Bilthoven/Den Haag, the
Netherlands.
Ministry of Transport, Public Works, and Water management, 2006. Primaire Waterkeringen
getoetst, Landelijke rapportage Toetsing 2006. Ministry of Public Works, Transport and
Watermanagement, Den Haag, the Netherlands
Ministry of Transport, Public Works, and Water management, 2009. Nationaal Waterplan 2009-2015.
Ministry of Public Works, Transport and Water management, Den Haag, the Netherlands.
Oerlemans, J., M. Dyurgerov et al., 2007. "Reconstructing the glacier contribution to sea-level rise
back to 1850." The Cryosphere 1: 59-65.
Parry, M. L., O.F. Canziani et al., 2007. Climate Change 2007: Impacts, Adaptation and Vulnerability.
Cambridge, Cambridge University Press.
PBL, 2007. The Netherlands in the future. Second Sustainability Outlook - part Physical Environment
of the Netherlands. Netherlands Environmental Assessment Agency – PBL, Bilthoven/Den Haag,
The Netherlands

94

Assessing an IPCC assessment

PBL, 2009. Roadmap to a climate-proof Netherlands. PBL publication number 500078003.
Netherlands Environmental Assessment Agency – PBL, Bilthoven/Den Haag, The Netherlands
PBL, 2009. News in Climate Science. Netherlands Environmental Assessment Agency – PBL,
Bilthoven/Den Haag, The Netherlands.
PBL, 2010. Zeespiegelstand langs de Nederlandse kust en mondiaal, 1891-2008. Versie 7.
Compendium voor de Leefomgeving. http://www.compendiumvoordeleefomgeving.nl/
indicatoren/nl022907-Zeespiegelstand-Nederland.html?i=9-54. Netherlands Environmental
Assessment Agency – PBL, Bilthoven/Den Haag. Statistics Netherlands, Den Haag/Heerlen,
Wageningen UR, Wageningen, The Netherlands .
Rahmstorf, S., 2007. A semi-empirical approach to projecting future sea-level rise. Science315: 367370

Annex D

95

Annex E Abbreviations
ASTER

NCAR		

Advanced Spaceborne Thermal Emission and
Reflection Radiometer (a remote sensing
instrument)

National Center for Atmospheric Research
(Boulder, USA)

CLA		

Non Governmental Organisation

Coordinating Lead Author

CO 2		
Carbon dioxide

CRU		
Climate Research Unit (at the University of East
Anglia, United Kingdom)

EHP		
Elevated Heat Pump

FAO		
Food and Agriculture Organization

FOD		
First Order Draft

GCM		
Global Circulation Model (a type of climate
model)

GDP		
Gross Domestic Product

GNP		
Gross National Product

H-K		

SOD		
Second Order Draft

SPM		
Summary for Policymakers

SRES		
IPCC Special Report on Emission Scenarios

SYR		
IPCC Synthesis Report (of Working Groups I, II
and III reports)

TAR		
Third Assessment Report of the IPCC

TS
Technical Summary

UN
United Nations

UNFCCC
United Nations Framework Convention on
Climate Change

UNMSM

Himalaya-Karakoram

Universidad Nacional Mayor de San Marcos
(Lima, Peru)

IPCC		

URL

Intergovernmental Panel on Climate Change

Uniform Resource Locator (e.g. an internet
address)

IUCN		
International Union for Conservation of Nature

KNAW		
Royal Netherlands Academy of Arts and
Sciences

KNMI		
Royal Netherlands Meteorological Institute

NAP
Amsterdam Ordnance Datum

96

NGO		

Assessing an IPCC assessment

WG
Working Group

WG I
IPCC Working Group I (on the Science of Climate
Change)

WG II
IPCC Working Group II (on Impacts, Adaptation
and Vulnerability)

WG III
IPCC Working Group III (on Mitigation of Climate
Change)

WWF
World Wide Fund for Nature

Annex E

97

98

Assessing an IPCC assessment

Colophon
Responsibility
Netherlands Environmental Assessment Agency

Editors
L.A. Meyer, A.C. Petersen

Steering group
M.A. Hajer, J.A. Oude Lohuis, L.A. Meyer, A.C. Petersen, G. de Lange, J. Timmerhuis,
E. Buitelaar

Research
J.R.M. Alkemade, M. Bakkenes, G.J. van den Born, A.F. Bouwman, J.J.M. van Grinsven,
H.B.M. Hilderink, P.H.M. Janssen, W. Ligtvoet, A.J.G. Manders, J.G. van Minnen, B.J. Strengers,
H. Visser, M. Vonk, M. de Vos (PBL), R.J. Swart (WUR)

Translation and editing
A. Righart (PBL), C. Ettema (Shades of Green - Natural Translations)

Other contributors
F.S. de Blois, C. Campman, J.H.J. Dolmans, S.A. van Esch, M. Hartman, A.M. Oosterhuis,
J.F. de Ruiter, C. Spelbos

Design and layout
Uitgeverij RIVM

Corresponding Author
L. Meyer, leo.meyer@pbl.nl

Colophon

99

PBL-omslag zonder flappen
340 x 240 mm

Assessing an IPCC assessment
An analysis of statements on projected
regional impacts in the 2007 report

Assessing an IPCC assessment
The Netherlands Environmental Assessment Agency (PBL) has investigated the scientific foundations for the IPCC summary conclusions
of the Fourth Assessment Report of 2007 on projected regional
climate-change impacts, at the request of the Dutch Minister for the
Environment. Overall the summary conclusions are considered well
Hier
komt
bijwere
voorkeur
flaptekst
tesignificant
staan, in plaats
founded
and
none
foundeen
to contain
any
errors.
van deze algemene tekst
The Working
Group II contribution
to the Fourth
Assessment
Het Planbureau
voor de Leefomgeving
is hét
nationale Report
showsinstituut
ample observational
evidence
of regional climate-change
voor strategische
beleidsanalyses
op het gebied
impacts,
have
beenenprojected
vanwhich
milieu,
natuur
ruimte. to pose substantial risks to most
parts of the world, under increasing temperatures. However, in some
Het PBL draagt bij aan de kwaliteit van het strategiinstances the foundations for the summary statements should have
sche overheidsbeleid door een brug te vormen tussen
been made more transparent. While acknowledging the essential role
wetenschap en beleid en door gevraagd en ongevraagd,
of expert judgment in scientific assessments, the PBL recommends to
onafhankelijk en wetenschappelijk gefundeerd, verkenimprove the transparency of these judgments in future IPCC reports.
ningen, analyses en evaluaties te verrichten waarbij een
integrale benadering voorop staat.
In addition, the investigated summary conclusions tend to single out
the most
important
negative
impacts of climate change. Although
Planbureau
voor
de Leefomgeving
this approach
was agreed to by the IPCC governments for the Fourth
Locatie Bilthoven
Assessment
Report,
Postbus
303 the PBL recommends that the full spectrum of
regional
impacts
is summarised for the Fifth Assessment Report,
3720
AH Bilthoven
including
the274
uncertainties.
The PBL believes that the IPCC should
T: 030
274 5
invest F:
more
quality
030 in
274
4479control in order to prevent mistakes and shortcomings,
to the extent possible.
E: info@pbl.nl
www.pbl.nl

Netherlands
Planbureau
Environmental
voor de Leefomgeving,
Assessment Agency,
<maand>
July200X
2010

snijlijn

rug

snijlijn