Decisions taken by the Panel at its 32nd Session
With regards to the Recommendations resulting from the
Review of the IPCC Processes and Procedures by the InterAcademy Council (IAC)

Busan, Republic of Korea, 11-14 October 2010

Preamble:
The IPCC welcomes the IAC’s review. Its recommendations will be important to improve the way
the IPCC works and how it is governed on behalf of the thousands of scientists who conduct
careful, thorough assessments on all aspects of climate change and on behalf of the global
community that utilizes its work.
The IPCC is taking decisive action to respond to these recommendations in a way that is
transparent and open, and ensures that the highest quality assessments are produced and made
available to the international community.
At its 32nd Session, the Panel agreed to immediately implement many of the recommendations. On
others, the Panel has formed Task Groups to undertake further work with a view to completion at
its next Session, in line with guidance from the IAC.
The IAC review highlights the contribution the IPCC has made to improve the understanding of the
scientific, technical and socio-economic aspects of climate change, and the commitment of the
world’s leading scientists and other experts to a robust assessment process.
The work of preparing the Fifth Assessment Report (AR5) remains on course and will benefit from
the Panel’s decisions on the IAC recommendations.

IPCC Secretariat
c/o WMO • 7bis, Avenue de la Paix • C.P. 2300 • 1211 Geneva 2 • Switzerland
telephone : +41 (0) 22 730 8208 / 54 / 84 • fax : +41 (0) 22 730 8025 / 13 • email : IPCC-Sec@wmo.int • www.ipcc.ch

Contents

This document presents the Panel’s decisions relevant to the IAC’s Review of the Processes and
Procedures of the IPCC and terms of reference (TOR) and work plan for the Task Groups
established to consider issues further and prepare proposals for consideration and decision by the
Panel at its next Session (IPCC-XXXIII) scheduled to be held in the first half of 2011.

1

Decisions by the Panel on Procedures relevant to the IAC’s Review and their
recommendations in
•
•

Chapter 2 “Evaluation of IPCC’s Assessment Processes” and
Chapter 3 “IPCC’s Evaluation of Evidence and Treatment of Uncertainty”
-

2

3

Decisions by the Panel on Governance and Management relevant to the IAC’s
Review and their recommendations in Chapter 4 “Governance and Management”
-

TOR for the Task Group on Governance and Management

-

TOR for the Task Group on Conflict of Interest Policy

Decisions by the Panel on a Communications Strategy as relevant to the IAC’s
Review and their recommendations in Chapter 4 “Governance and Management”
-

4

TOR for the Task Group on Procedures

TOR for the Task Group on a Communications Strategy

The Task Groups and their Composition

Appendices
1.
2.
3.
4.

“General Guidance on the Use of Literature in IPCC Reports”
“General Guidance on the Role of Review Editors”
“Proposed IPCC Protocol for Addressing Errors in Previous Assessment Reports”
“Draft Guidance Note for Lead Authors of the Fifth Assessment Report on
Consistent Treatment of Uncertainties”

-2-

1

DECISIONS BY THE PANEL ON PROCEDURES

The Panel welcomed and acknowledged the recommendations and suggestions by the IAC on the
IPCC’s assessment process (Chapter 2 and 3 of the IAC Report), and made the following specific
decisions:
Scoping
The Panel noted that in its report the IAC has recommended:
“The IPCC should make the process and criteria for selecting participants for scoping meetings
more transparent.”
The Panel agreed with this recommendation
Implementation plan to be determined by the Task Group on Procedures with the view to make a
decision at its next Session (IPCC-XXXIII).
Author Selection
The Panel noted that in its report the IAC has recommended:
“The IPCC should establish a formal set of criteria and processes for selecting Coordinating Lead
Authors and Lead Authors.”
The Panel agreed with this recommendation
Formal criteria are included in the existing procedures. Enhanced implementation and
transparency as well as potential additional criteria and procedures to be considered by the
Task Group on Procedures with the view to make a decision at its next Session (IPCC-XXXIII) for
future work.
The Panel noted that in its report the IAC has recommended:
“The IPCC should make every effort to engage local experts on the author teams of the regional
chapters of the Working Group II report, but should also engage experts from countries outside of
the region when they can provide an essential contribution to the assessment.”
The Panel agreed with this recommendation
This is already implemented for AR5. Further implementation to be considered by the Task Group
on Procedures with the view to make a decision at its next Session (IPCC-XXXIII) for future work.
Sources of Data and Literature
The Panel noted that in its report the IAC has recommended:
“The IPCC should strengthen and enforce its procedure for the use of unpublished and non-peerreviewed literature, including providing more specific guidance on how to evaluate such
information, adding guidelines on what types of literature are unacceptable, and ensuring that
unpublished and non-peer-reviewed literature is appropriately flagged in the report.”
The Panel agreed with this recommendation
The Panel decided to strengthen the application of its procedures on the use of unpublished and
non-peer reviewed literature. It decided to implement this recommendation and further key
elements through its procedures and guidance notes. The Panel noted the General Guidance on
the Use of Literature in IPCC Reports (contained in IPCC-XXXII/INF.4) as revised in General
Guidance on the Use of Literature in IPCC Reports (Appendix 1) which addresses the related
aspects in the IAC recommendations and decided to endorse them as a Guidance Note. The Panel
-3-

urges the Co-Chairs of Working Group I, II, III and TFI to take any necessary steps to ensure that
this guidance note is applied in the development of IPCC reports.
Handling the Full Range of Views
The Panel noted that in its report the IAC has recommended:
“Lead Authors should explicitly document that a range of scientific viewpoints has been
considered, and Coordinating Lead Authors and Review Editors should satisfy themselves that due
consideration was given to properly documented alternative views.”
The Panel agreed with this recommendation
The Panel emphasizes that handling the full range of scientific views is a core principle of the IPCC.
Its procedures clearly require the representation of differing scientific viewpoints and encourages
rigorous adherence by the CLAs, LAs, and REs. The Panel urges the IPCC Chair, the Co-Chairs of
the Working Groups and TFI to take any necessary steps to ensure that this principle continues to
be applied in the development of IPCC reports. Further implementation to be considered by the
Task Group on Procedures with the view to make a decision at its next Session (IPCC-XXXIII).
Report Review
The Panel noted that in its report the IAC has recommended:
“The IPCC should adopt a more targeted and effective process for responding to reviewer
comments. In such a process, Review Editors would prepare a written summary of the most
significant issues raised by reviewers shortly after review comments have been received. Authors
would be required to provide detailed written responses to the most significant review issues
identified by the Review Editors, abbreviated responses to all non-editorial comments, and no
written responses to editorial comments.”
The Panel agreed with this recommendation in principle
Implementation options to be considered by the Task Group on Procedures with the view to make
a decision at its next Session (IPCC-XXXIII).
The Panel noted that in its report the IAC has recommended:
“The IPCC should encourage Review Editors to fully exercise their authority to ensure that
reviewers’ comments are adequately considered by the authors and that genuine controversies are
adequately reflected in the report.”
The Panel agreed with this recommendation
The Panel decided to strengthen its application of procedures, and amend them where necessary,
to enable Review Editors to fully exercise their role. The Panel noted the new Guidance Note on
the Role of Review Editors (Appendix 2) which addresses the related aspects in the IAC
recommendations. The Panel urges the Co-Chairs of Working Group I, II, III and TFI to take steps
to ensure that this guidance note is implemented in the development of its work.
Summary for Policymakers
The Panel noted that in its report the IAC has recommended:
“The IPCC should revise its process for the approval of the Summary for Policymakers so that
governments provide written comments prior to the Plenary.”
The Panel acknowledges the importance of both written comments and inputs from the floor, which
are current practice. No revision to the process is required.
-4-

Procedure for the handling of potential errors identified after approval of IPCC reports
IAC discussion and suggestion in the Box analyzing the Himalayan glacier error (IAC Report
page 22). Discussion of time required for a response on Himalayan glacier error (IAC Report
page 54).
The Panel agreed on the need to establish a process for evaluating, addressing and correcting, if
necessary, potential errors and further developing errata as appropriate.
The Panel noted the “Proposed IPCC Protocol for Addressing Errors in Previous Assessment
Reports” (Appendix 3) which describes a clear decision tree, based on the nature of the material
and the steps necessary to avoid bias, so that potential errors could be addressed as rapidly as
practical.
The Panel urges the IPCC Chair, the IPCC Vice-Chairs, the Co-Chairs of Working Group I, II, III
and TFI to take any necessary steps to ensure that this protocol is finalized and then used for
evaluation of potential errors and developing errata as appropriate. Further analysis to be
considered by the Task Group on Procedures with the view to submit a proposal for a decision at
the next Session (IPCC-XXXIII).
IPCC’s Evaluation of Evidence and Treatment of Uncertainty
The Panel noted that in its report the IAC has made several recommendations:
“All Working Groups should use the qualitative level-of-understanding scale in their Summary for
Policymakers and Technical Summary, as suggested in IPCC’s uncertainty guidance for the
Fourth Assessment Report. This scale may be supplemented by a quantitative probability scale, if
appropriate.”
“Chapter Lead Authors should provide a traceable account of how they arrived at their ratings for
level of scientific understanding and likelihood that an outcome will occur.”
“Quantitative probabilities (as in the likelihood scale) should be used to describe the probability of
well-defined outcomes only when there is sufficient evidence. Authors should indicate the basis for
assigning a probability to an outcome or event (e.g., based on measurement, expert judgment,
and/or model runs).”
“The confidence scale should not be used to assign subjective probabilities to ill-defined
outcomes.”
“The likelihood scale should be stated in terms of probabilities (numbers) in addition to words to
improve understanding of uncertainty.”
“Where practical, formal expert elicitation procedures should be used to obtain subjective
probabilities for key results.”
The Panel agreed with these recommendations
The Panel decided to improve the IPCC guidance on evaluation of evidence and treatment of
uncertainty. It is implementing the six recommendations in the IAC Review as part of a broader
package of updates to procedures and guidance notes. The Panel noted with appreciation the
Draft Guidance Note for Lead Authors of the Fifth Assessment Report on Consistent Treatment of
Uncertainties (Appendix 4) and requested the Co-Chairs of Workings Group I, II and III to present
the final document to the Panel at its next Session. The final document should provide more detail
on traceable accounts, the evolution of the guidance since AR4 and explain how each of the
six recommendations in the IAC review is addressed. The Panel urges the Co-Chairs to take any
necessary steps to ensure that the guidance note is implemented in the development of its work.
-5-

Terms of reference for a Task Group on Procedures
The Panel welcomed and acknowledged the recommendations and suggestions by the IAC on the
IPCC’s assessment process (Chapters 2 and 3 of the IAC Report) and decided to establish an
inter-sessional Task Group on Procedures to develop proposals on further implementation of the
recommendations. The Task Group is specifically requested to address, inter alia, the issues listed
in Annex I to this decision and propose amendments, including Appendix A to the Principles
Governing IPCC work and relevant Guidance Documents, if necessary, by 31 January 2011.
Governments will then be invited to provide comments on the proposals by 28 February 2011 to
allow preparation of a revised draft for consideration and decisions by the Panel at its next Session
(IPCC-XXXIII).
The Task Group on Procedures is open to participation by the members of the IPCC and consists
of Australia, Bolivia, Brazil, Canada, Chile, Germany, India, Iran, Maldives, Netherlands,
New Zealand, Niger, Norway, Peru, Saudi Arabia, South Africa, Swaziland, Switzerland, Thailand,
and USA. The Task Group will elect Co-Chairs to coordinate its work.
The Task Group will seek the advice of the IPCC Chair, the IPCC Vice-Chairs, Working Group and
TFI Co-Chairs and the Secretary. The duration of the Task Group is until the IPCC’s 33rd Session
unless decided otherwise.

-6-

Annex I
The Task Group should address the issues listed below as mentioned in the IAC recommendations
(Chapters 2 and 3), IPCC responses at its 32nd Session and IPCC-XXXII/Doc. 22. For each of the
issues the Task Group should establish a timetable for action, consider resource implications and
identify responsibilities for implementation. It should propose amendments to the Appendix A to the
Principles Governing IPCC Work and relevant guidance documents if needed taking into account
decisions made at IPCC-XXXII.
Chapter 2: Evaluation of IPCC’s Assessment Process
Scoping
1. Recommendation: The IPCC should make the process and criteria for selecting participants for
scoping meetings more transparent.
Author Selection
2. Recommendation: The IPCC should establish a formal set of criteria and processes for selecting
Coordinating Lead Authors and Lead Authors.
3. Recommendation: The IPCC should make every effort to engage local experts on the author
teams of the regional chapters of the Working Group II report, but should also engage experts from
countries outside of the region when they can provide an essential contribution to the assessment.
Sources of Data and Literature
4. Recommendation: The IPCC should strengthen and enforce its procedure for the use of
unpublished and non-peer-reviewed literature, including providing more specific guidance on how
to evaluate such information, adding guidelines on what types of literature are unacceptable, and
ensuring that unpublished and non-peer-reviewed literature is appropriately flagged in the report.
Handling the Full Range of Views
5. Recommendation: Lead Authors should explicitly document that a range of scientific viewpoints
has been considered, and Coordinating Lead Authors and Review Editors should satisfy
themselves that due consideration was given to properly documented alternative views.
Report Review
6. Recommendation: The IPCC should adopt a more targeted and effective process for responding
to reviewer comments. In such a process, Review Editors would prepare a written summary of the
most significant issues raised by reviewers shortly after review comments have been received.
Authors would be required to provide detailed written responses to the most significant review
issues identified by the Review Editors, abbreviated responses to all non-editorial comments, and
no written responses to editorial comments.
7. Recommendation: The IPCC should encourage Review Editors to fully exercise their authority to
ensure that reviewers’ comments are adequately considered by the authors and that genuine
controversies are adequately reflected in the report.
Summary for Policymakers
8. Recommendation: The IPCC should revise its process for the approval of the Summary for
Policymakers so that governments provide written comments prior to the Plenary.

-7-

Procedure for the handling of potential errors identified after approval of IPCC reports
IAC discussion and suggestion: Box analyzing of Himalayan glacier error (IAC Report page 22).
Discussion of time required for a response on Himalayan glacier error (IAC Report page 54).
Chapter 3: IPCC’s Evaluation of Evidence and Treatment of Uncertainty
9. Recommendation: All Working Groups should use the qualitative level-of-understanding scale in
their Summary for Policymakers and Technical Summary, as suggested in IPCC’s uncertainty
guidance for the Fourth Assessment Report. This scale may be supplemented by a quantitative
probability scale, if appropriate.
10. Recommendation: Chapter Lead Authors should provide a traceable account of how they
arrived at their ratings for level of scientific understanding and likelihood that an outcome will occur.
11. Recommendation: Quantitative probabilities (as in the likelihood scale) should be used to
describe the probability of well-defined outcomes only when there is sufficient evidence. Authors
should indicate the basis for assigning a probability to an outcome or event (e.g. based on
measurement, expert judgment, and/or model runs).
12. Recommendation: The confidence scale should not be used to assign subjective probabilities
to ill-defined outcomes.
13. Recommendation: The likelihood scale should be stated in terms of probabilities (numbers) in
addition to words to improve understanding of uncertainty.
14. Recommendation: Where practical, formal expert elicitation procedures should be used to
obtain subjective probabilities for key results.

-8-

2

DECISIONS BY THE PANEL ON GOVERNANCE AND MANAGEMENT

The Panel
The Panel noted that in its report the IAC has recommended:
“The IPCC should establish an Executive Committee to act on its behalf between Plenary
sessions. The membership of the Committee should include the IPCC Chair, the Working
Group Co-Chairs, the senior member of the Secretariat, and 3 independent members, including
some from outside of the climate community. Members would be elected by the Plenary and
serve until their successors are in place.”
The Panel at its 32nd Session:
I.

Agreed to work toward establishing a formal body to provide governance functions that
are necessary between sessions of the panel, strengthen coordination activities, and
have oversight of the organisation’s administration and communications; according to
the mandate to be agreed in the 33rd Session.

II.

The Task Group should consider options for the implementation of the decision
concerning the recommendation mentioning the establishment of an Executive
Committee. These options include those for the mandate, size, composition, functions
and reporting of the body referred to in this recommendation.

III.

The Task Group shall make recommendations on the options mentioned in decision II
to the 33rd Session of the Panel, with a view to taking a decision.

The Secretariat
The Panel noted that in its report the IAC has recommended:
“The IPCC should redefine the responsibilities of key Secretariat positions both to improve
efficiency and to allow for any future senior appointments.”
“The IPCC should elect an Executive Director to lead the Secretariat and handle day-to-day
operations of the organization. The term of this senior scientist should be limited to the
timeframe of one assessment.”
The Panel at its 32nd Session:
I.

Requested the Task Group to examine the role of the Secretariat in its relation with WMO,
UNEP, the IPCC-Chair, the Vice-Chairs, Co-Chairs of the WGs and the TFI, and Technical
Support Units. The Task Group is requested to review the responsibilities of key Secretariat
positions and consider the issues associated with it and to make recommendations to the
Panel at its 33rd Session. It is also requested to consider issues associated with the
potential creation of a new post of an “Executive Director” to lead the Secretariat.

The IPCC Chair; Working Group Co-Chairs
The Panel noted that in its report the IAC has recommended:
“The term of the IPCC Chair should be limited to the timeframe of one assessment.”
“The terms of the Working Group Co-Chairs should be limited to the timeframe of one
assessment.”

-9-

The Panel at its 32nd Session:
I.

Requested the Task Group to consider issues related to the IAC recommendations on the
term of the IPCC Chair and working group Co-Chairs, including continuity issues.

II. Noted that any amendments to the existing IPCC Rules of Procedure for Elections could be
applied only to subsequent elections.
III. Requested the Task Group to report their recommendations to the 33rd Session for
decision.
Conflict of Interest Policy
The Panel noted that in its report the IAC has recommended:
“The IPCC should develop and adopt a rigorous conflict of interest policy that applies to all
individuals directly involved in the preparation of IPCC reports, including senior IPCC
leadership (IPCC Chair and Vice Chairs), authors with responsibilities for report content
(i.e., Working Group Co-Chairs, Coordinating Lead Authors, and Lead Authors), Review
Editors, and technical staff directly involved in report preparation (e.g., staff of Technical
Support Units and the IPCC Secretariat).”
The Panel at its 32nd Session:
I.

Agreed with this IAC recommendation.

II. Decided to implement a rigorous conflict of interest policy, taking into consideration the
specific circumstances related to participation in IPCC activities.
III. Established a Task Group on Conflict of Interest Policy to propose options for such a policy,
consulting with relevant organisations, for its decision at the 33rd Session.
The IPCC Bureau
The Panel noted that in its report the IAC has recommended:
“The IPCC should develop and adopt formal qualifications and formally articulate the roles and
responsibilities for all Bureau members, including the IPCC Chair, to ensure that they have
both the highest scholarly qualifications and proven leadership skills.”
The Panel at its 32nd Session:
I.

Decided to refer this issue to the relevant Task Groups with a particular focus on roles and
responsibilities for all Bureau members, including the IPCC Chair.

II. The Task Group on Governance and Management should report back to the Panel at the
33rd Session.

- 10 -

Terms of reference for a Task Group on Governance and Management
The Panel welcomed and acknowledged the recommendations and suggestions by the IAC
on the IPCC’s governance and management (Chapter 4 of the IAC Report) and decided to
establish an inter-sessional Task Group on Governance and Management to develop
proposals related to the recommendations by the IAC and the decisions taken at the
32nd Session as listed above. The Task Group is specifically requested to address, inter alia,
the issues listed in Annex II to this decision and propose amendments, including to the
Principles Governing IPCC Work, and its Appendices, and other relevant documents, if
necessary, by 31 January 2011. Governments will then be invited to provide comments on
the proposals by 28 February 2011 to allow preparation of a revised draft for consideration
and decisions by the Panel at its next Session (IPCC-XXXIII). Matters related to conflict of
interest policy will be addressed by a dedicated “Task Group on Conflict of Interest Policy”
as described below.
The Task Group on Governance and Management (*) is open to participation by the
members of the IPCC and consists of Belgium, Canada, China, Cuba, France, Germany,
India, Iran, Korea, Lesotho, Maldives, Mali, New Zealand, Norway, Peru, Russia, Saudi
Arabia, Slovenia, South Africa, Spain, Sweden, UK, and USA. The Task Group will elect its
Co-Chairs to coordinate its work.
The Task Group will seek the advice of the IPCC Chair, the IPCC Vice-Chairs, Working
Group and TFI Co-Chairs, and the Secretary. The duration of the Task Group is until the
IPCC’s 33rd Session unless decided otherwise.

(*) Correction on 26.10.2010: Norway and Spain have been added as they indicated their willingness
to participate in the Task Group on Governance and Management (also corrected on Page 15, List of
Task Groups formed at IPCC-XXXII and Composition).

- 11 -

Annex II
The Task Group on Governance and Management should address the issues listed below
as mentioned in the IAC recommendations (Chapter 4 of the IAC Report), IPCC responses
at its 32nd Session and IPCC-XXXII/Doc. 22. For each of the issues the Task Group should
establish a timetable for action, consider resource implications and identify responsibilities
for implementation. It should propose amendments to the Principles Governing IPCC Work,
its Appendices, and other relevant documents if needed taking into account decisions made
at IPCC-XXXII.
Chapter 4: Governance and Management
The Panel
1.
Recommendation: The IPCC should establish an Executive Committee to act on its
behalf between Plenary sessions. The membership of the Committee should include the
IPCC Chair, the Working Group Co-chairs, the senior member of the Secretariat, and
3 independent members, including some from outside of the climate community. Members
would be elected by the Plenary and serve until their successors are in place.
The IPCC Chair
2.
Recommendation: The term of the IPCC Chair should be limited to the timeframe of
one assessment.
The IPCC Bureau
3.
Recommendation: The IPCC should develop and adopt formal qualifications and
formally articulate the roles and responsibilities for all Bureau members, including the IPCC
Chair, to ensure that they have both the highest scholarly qualifications and proven
leadership skills.
4.
Recommendation: The terms of the Working Group Co-chairs should be limited to the
timeframe of one assessment.
The Secretariat
5.
Recommendation: The IPCC should redefine the responsibilities of key Secretariat
positions both to improve efficiency and to allow for any future senior appointments.
6.
Recommendation: The IPCC should elect an Executive Director to lead the
Secretariat and handle day-to-day operations of the organization. The term of this senior
scientist should be limited to the timeframe of one assessment.

- 12 -

Terms of reference for a Task Group on Conflict of Interest Policy
The Panel welcomed and acknowledged the recommendations and suggestions by the IAC
on the IPCC’s conflict of interest policy (as discussed in Chapter 4 of the IAC Report) and
decided to establish an inter-sessional Task Group on Conflict of Interest Policy as
discussed in Chapter 4 of the IAC Report to develop proposals on further implementation of
the IAC recommendations and decision taken by the Panel at its 32nd Session. The Task
Group is specifically requested to address, inter alia, the issues listed in Annex III to this
decision and propose amendments, including to the Principles Governing IPCC Work and
relevant documents, if necessary, by 31 January 2011. Governments will then be invited to
provide comments on the proposals by 28 February 2011 to allow preparation of a revised
draft for consideration and decisions by the Panel at its next Session (IPCC-XXXIII).
The Task Group is open to participation by the members of the IPCC and consists of:
Bangladesh, China, Malaysia, New Zealand, Slovenia, Sudan, UK and USA. The Task
Group will elect its Co-Chairs to coordinate its work.
The Task Group will seek the advice of the IPCC Chair, the IPCC Vice-Chairs, Working
Group and TFI Co-Chairs and the Secretary. The duration of the Task Group is until the
IPCC’s 33rd Session unless decided otherwise.

Annex III
The Task Group should address the issues listed below as mentioned in the IAC
recommendations (Chapter 4), IPCC responses at its 32nd Session and IPCC-XXXII/Doc. 22.
For each of the issues the Task Group should establish a timetable for action, consider
resource implications and identify responsibilities for implementation. It should propose
amendments to the Principles Governing IPCC work and relevant documents if needed
taking into account decisions made at IPCC-XXXII.
Chapter 4: Governance and Management
Conflict of Interest Policy
1.
Recommendation: The IPCC should develop and adopt a rigorous conflict of interest
policy that applies to all individuals directly involved in the preparation of IPCC reports,
including senior IPCC leadership (IPCC Chair and Vice Chairs), authors with responsibilities
for report content (i.e., Working Group Co-Chairs, Coordinating Lead Authors, and Lead
Authors), Review Editors, and technical staff directly involved in report preparation (e.g., staff
of Technical Support Units and the IPCC Secretariat).

- 13 -

3

DECISIONS BY THE PANEL ON A COMMUNICATIONS STRATEGY

The Panel noted that in its report the IAC has recommended:
“The IPCC should complete and implement a communications strategy that emphasizes
transparency, rapid and thoughtful responses, and relevance to stakeholders, and which
includes guidelines about who can speak on behalf of IPCC and how to represent the
organization appropriately.”
The Panel accepts the recommendation to develop a communication strategy.
Taking into account the core products of the organization, the Strategy will clarify the scope
and objectives of IPCC communication, with clear guidelines on authority, representation
and identification of spokespeople.
The Panel decided to establish a Task Group to guide the development of the
Communications Strategy. The first draft should be presented to the IPCC Bureau at its next
Session with a view to adopting the Communication Strategy at the 33rd Session of the
Panel.
Terms of reference for a Task Group on a Communications Strategy
The Task Group on the IPCC Communications Strategy will, taking into account the core
scientific review and assessment role of the IPCC and its scientific and intergovernmental
nature, guide the development of a comprehensive and concise communications strategy
that:
−

Defines the scope of IPCC communications, including about (a) the results and
products of assessments, (b) errors, corrections and other issues arising from the
work of IPCC, and (c) improving understanding of the processes and governance of
IPCC;

−

Provides guidance regarding whether balanced communications materials derived
from IPCC products that have been approved or accepted by the Panel should be
developed, and under what circumstances;

−

Articulates a set of general objectives for IPCC communications, including its website,
emphasizing transparency, rapid and thoughtful responses, political neutrality, and
relevance to stakeholders;

−

Identifies targeted audiences and stakeholders, recognizing their diversity of
languages;

−

Includes guidelines on who can speak on behalf of IPCC and how and when
authorized spokespersons should represent the organization appropriately, as well as
how communication materials will be authorized; and

−

Addresses any potential conflicts of interest regarding communications.

The Task Group will seek the advice of the IPCC Chair, the IPCC Vice-Chairs, Working
Group and TFI Co-Chairs and the Secretary. The Task Group membership is open to
representatives of governments that are members of the IPCC. The Task Group consists of
Belgium, Canada, Gambia, Germany, Iran, Japan, Madagascar, Mexico, Netherlands,
Norway, Saudi Arabia, Senegal, UK, USA, and Zambia. The Task Group will elect its
Co-Chairs to coordinate its work.
The work of the Task Group will be supported by the Communications team within the
Secretariat.
The Task Group will produce a first draft of the Strategy for consideration at the first Bureau
meeting in 2011, with a view to the Panel adopting the Strategy at its 33rd Session.
- 14 -

4

THE TASK GROUPS

The Task Groups are open to participation by the members of the IPCC. Via a show of
hands at the closing Plenary Session, country participation of the four Task Groups dealing
with a) Procedures, b) Governance and Management, c) Conflict of Interest Policy and d)
Communications Strategy was agreed, and is provided below.
The Task Groups will elect their Co-Chairs in due-time.
To facilitate the work of the Task Groups, the Panel decided on the 14th of October, 2010
that 25 trips would be allocated from the IPCC Trust Fund budget for travel to any necessary
meetings for the four established groups.
The four Task Groups will report back to the Panel at its 33rd Session. The duration of the
Task Groups’ mandates is until the IPCC’s 33rd Session, unless decided otherwise.

List of Task Groups formed at IPCC-XXXII and Composition

Task Group on Procedures
Australia, Bolivia, Brazil, Canada, Chile, Germany, India, Iran, Maldives, Netherlands,
New Zealand, Niger, Norway, Peru, Saudi Arabia, South Africa, Swaziland, Switzerland,
Thailand, and USA
Task Group on Governance and Management (*)
Belgium, Canada, China, Cuba, France, Germany, India, Iran, Korea, Lesotho, Maldives,
Mali, New Zealand, Norway, Peru, Russia, Saudi Arabia, Slovenia, South Africa, Spain,
Sweden, UK, and USA
Task Group on Conflict of Interest Policy
Bangladesh, China, Malaysia, New Zealand, Slovenia, Sudan, UK and USA
Task Group on Communications Strategy
Belgium, Canada, Gambia, Germany, Iran, Japan, Madagascar, Mexico, Netherlands,
Norway, Saudi Arabia, Senegal, UK, USA, and Zambia

(*) Correction on 26.10.2010: Norway and Spain have been added as they indicated their willingness
to participate in the Task Group on Governance and Management (also corrected on Page 11, Terms
of reference for a Task Group on Governance and Management).

- 15 -

Appendix 1

General Guidance on the Use of Literature in IPCC Reports
Introduction
The Technical Support Units (TSUs) of the three IPCC Working Groups drafted this guidance document to recall the
Principles Governing IPCC Work, particularly the “Procedure for using non-published/non-peer-reviewed sources in IPCC
Reports”, and to enhance implementation of the underlying principles by posing questions whose answers will help
ensure that the requirements are met. Following these principles will ensure that all relevant statements and lines of
discussion are properly substantiated by adequate literature, and that all relevant text undergoes appropriate review.
These guidelines will be presented and discussed at Lead Author meetings and backed up by further training as needed.
Guidance on the use of non-published/non-peer-reviewed (“grey”) literature
1. Overview of current Principles Governing IPCC Work
Appendix A, Section 4.2.3, to the Principles Governing IPCC Work states
Contributions should be supported as far as possible with references from the peer-reviewed and
internationally available literature
Extract from Annex 2 of Appendix A to the Principles Governing IPCC Work:

Procedure for using non-published/non-peer-reviewed sources in IPCC Reports
Because it is increasingly apparent that materials relevant to IPCC Reports, in particular, information about the
experience and practice of the private sector in mitigation and adaptation activities, are found in sources that
have not been published or peer-reviewed (e.g., industry journals, internal organisational publications, nonpeer reviewed reports or working papers of research institutions, proceedings of workshops etc) the following
additional procedures are provided. These have been designed to make all references used in IPCC Reports
easily accessible and to ensure that the IPCC process remains open and transparent.
1. Responsibilities of Coordinating, Lead and Contributing Authors
Authors who wish to include information from a non-published/non-peer-reviewed source are requested to
a. Critically assess any source that they wish to include. This option may be used for instance to obtain case
study materials from private sector sources for assessment of adaptation and mitigation options. Each
chapter team should review the quality and validity of each source before incorporating results from the
source into an IPCC Report.
b. Send the following materials to the Working Group/Task Force Bureau Co-Chairs who are coordinating the
Report:
• One copy of each unpublished source to be used in the IPCC Report
•

(…)
October 2010

The following information for each source:
o Title
o Author(s)
o

Name of journal or other publication in which it appears, if applicable

o

Information on the availability of underlying data to the public

o

English-language executive summary or abstract, if the source is written in a non-English
language

o

Names and contact information for 1-2 people who can be contacted for more information about
the source.

Literature Guidance Note

5. Treatment in IPCC Reports
Non-peer-reviewed sources will be listed in the reference sections of IPCC Reports. These will be integrated
with references for the peer-reviewed sources.1 These will be integrated with references to the peer reviewed
sources stating how the material can be accessed, but will be followed by a statement that they are not
published.
The two distinct but related principles that are at the core of these procedures are (1) to ensure the quality, robustness
and validity of the information assessed and (2) to ensure the accessibility of the sources for reviewers of the report
drafts.

2. Questions to help determine the appropriateness of including a non-published/non-peer reviewed
reference
Non-published/non-peer-reviewed sources are often called grey literature. Although highly relevant information can be
contained in the grey literature, use of this literature brings with it an extra responsibility for the author teams to ensure
the quality and validity of cited sources and information. Authors need to be clear why a particular source is used and in
some circumstances may need to explain this in the text.
Considering the following questions will help ensure that the principles underlying the IPCC Rules and Procedures are
properly implemented.
a) Who (e.g., what organization) is the source of the grey literature citation?
b) What information does the citation add to the assessment?
c)

Is the information cited available from a peer-reviewed journal source? If yes, is the citation needed?

d) Are there lines of evidence from other (peer-reviewed or non-peer-reviewed) sources that support the citation or
reach different conclusions? If yes, is the citation needed?
e) What are the qualifications of the author(s) of the document?
f)

Was there any review of the material presented? If so, how wide or extensive was that review? How credible are
the reviewers?

g) Why was the document written? How was the research funded? Could the researcher and/or publisher of the
document be perceived as having a particular bias or agenda? If yes, what caveats are needed?
h) Why wasn’t the information published in a peer-reviewed journal?

3. Acceptability of sources in IPCC Reports
Since the development of the Principles Governing IPCC Work, there has been a rapid growth in new forms of
communication and media in which the public finds climate relevant information. The IPCC principles for use of nonpublished/non-peer reviewed sources do not change with this move to more electronic communication. Blogs, social
networking sites (e.g., Twitter, Facebook), and visual media do not currently meet the standards for use in scientific
assessments and developing key findings in IPCC Reports, and are therefore not acceptable for use. In the absence of
other sources, newspapers and magazines may provide limited ancillary information for an assessment, but not for key
findings.
Personal communications of scientific results are also not acceptable sources.

1 Non-published sources also will be listed in the reference sections of IPCC Reports.

October 2010

Page 2 of 3

Guidance on Use of Literature for IPCC AR5 Lead Authors

Literature Guidance Note

4. Accessibility of non-published/non-peer reviewed references
Non-published/non-peer-reviewed references need to be accessible by the reviewers at the time of the review. In order
to ensure a minimum level of accessibility of all sources used in the report, authors MUST provide a copy of each
source of information that is not publicly available (preferably as a non-editable electronic document) and the
additional information specified in the IPCC principles. These must be received by the TSU by the time that the First
Order Draft (FOD) and Second Order Draft (SOD) respectively are due to the TSU.

5. Guidance on the use of sources going through peer-review and literature cut off dates
After the distribution of the SOD, authors may only include additional literature that further supports statements that
have already been substantiated by one or more references. Authors may not introduce new information in the Final
Draft (FD) that substantially alters the content and conclusions of the report compared to the SOD.
In order to be included in the respective chapter drafts, literature must meet the following requirements:
 For inclusion in the First Order Draft (FOD): submitted for peer-review and a copy provided to the TSU prior
to the date when the FOD is due to the TSU;
 For inclusion in the Second Order Draft (SOD): submitted for peer-review and a copy provided to the TSU
prior to the date when the SOD is due to the TSU;
 For inclusion in the Final Draft (FD): accepted for publication and a copy provided to the TSU prior to the
date when the FD is due to the TSU. Acceptance for publication MUST be substantiated by (i) letter from
the editor, (ii) DOI-Nr., or (iii) published as accepted on the journal’s website.
The specific cutoff dates will be provided to the authors by the TSU early in the assessment cycle.
Any reference that does not fulfill these criteria will be removed from the draft contribution together with the
statement(s) that it supports if there are no other supporting references. It is therefore not advisable to base a line of
argument or conclusion on a single, not-yet accepted paper.

October 2010

Page 3 of 3

Guidance on Use of Literature for IPCC AR5 Lead Authors

Appendix 2

Role of Review Editors
Annex I of the IPCC Procedures (Appendix A to the Principles Governing IPCC Work) states that Review Editors (REs)
will:
• Assist the Working Group in identifying reviewers for the expert review process;
•

Ensure that all substantive expert and government review comments are afforded appropriate
consideration;

•

Advise lead authors on how to handle contentious/controversial issues;
Ensure genuine controversies are reflected adequately in the text of the Report; and

•
•

Submit a written report to the Working Group Session. REs also may be requested to attend Sessions
of the Working Group.

REs will need to ensure that where significant differences of opinion on scientific issues remain, such differences are
described in an annex to the Report.
REs are not actively engaged in drafting Reports.

Review Editors are critical for achieving the IPCC mandate of producing comprehensive and objective assessments in an
open and transparent manner. They also help ensure that IPCC reports are neutral with respect to policy. To achieve
these goals, REs undertake a number of activities:
Before the First Order Draft (for review by Experts) and the Second Order Draft (for review by Governments and Experts)
are sent for review, the REs should identify possible reviewers for the entire chapter and for sections that reach key
conclusions. The Technical Support Unit (TSU) will contact these reviewers. During the review period, the REs should
read their Chapter carefully to prepare for the review comments and to identify possible crosscutting issues that will
need to be addressed.
It is important to note that REs may not submit a review of their own Chapter but may do so on others. The REs do not
rewrite the text – this is the job of the authors.
When the review comments are received, they will be collated by the TSU and sent to the Chapter Coordinating Lead
Authors (CLAs) and REs. The REs should read all comments on their Chapter to identify critical issues that will likely
require discussion with the Chapter author team. The focus should be on sections where the review comments are
inconsistent or contradictory, where considerable rewriting will likely be needed, and where scientific controversies exist.
The REs should come prepared to the subsequent Lead Author Meeting to ensure that the author team fully and
appropriately addresses the review comments, and that the author team fairly represents the range of scientific opinion.
This process will be repeated for the Second Order Draft.
Important dates for REs include (i) when the review periods begin, i.e., when you get access to your chapter, (ii) when
comments are distributed to the CLAs and REs, and (iii) the two Lead Author meetings that you are expected to attend.
At the end of the process, the REs will provide a brief written report to the Working Group Co-Chairs. The exact format
of the report has yet to be determined. Where appropriate, REs will be requested to attend Sessions of the Working
Group and of the IPCC to communicate their findings from the review process and to assist in finalizing the Summary for
Policymakers (SPM).

October 2010

Guidance for IPCC AR5 Review Editors

Appendix 3

Proposed IPCC Protocol for Addressing Errors in Previous Assessment Reports
Draft
Note: The Panel urges the IPCC Chair, the IPCC Vice-Chairs, the Co-Chairs of the Working Group I, II, III and TFI to take any
necessary steps to ensure that this protocol is finalized and then used for evaluation of potential errors and developing
errata as appropriate. Further analysis to be considered by the Task Group on Procedures with the view to make a decision
at its next Session (IPCC-33).

At its 40th Session (May 2010) the IPCC Bureau discussed a protocol for correcting errors in reports already
released. The specific protocol below was prepared by Working Group II (WGII) Co-chairs to assist further
consideration of that matter.
This protocol is intended to be used only to correct errors that could have been avoided in the context of the
information available at the time the report was written. It should not be used to make changes that reflect new
knowledge or scientific information that became available only after the literature cut-off date for the report in
question. It should also not be invoked to reflect a change in opinion. Rather, it should be reserved for errors of
fact or accuracy in representing the underlying science.
This protocol is intended to address the full range of possible errors from simple typos through complicated
issues of sourcing, interpretation, analysis, or assessment. The protocol is intended to be invoked when at least
one Co-Chair of the report containing the putative error, one Coordinating Lead Author (CLA) of the chapter
containing the putative error, or one member of the current Bureau requests evaluation of a putative error.
The protocol is presented as a dichotomous tree (if condition n (where n is 1-9) is met), follow the tree below the
condition. If condition n is not met, go to option a, with the same number value. For example, if statement 1 is
correct, go to statement 2. If statement 2 is incorrect, go to statement 2a, etc.
Proposed protocol
1) If the putative error is in a working Group (WG) report, the CLAs of the relevant chapter are approached and
asked whether they agree that there is an error.
2) If the CLAs agree that there is an error, then they are asked to decide whether the fix requires a simple
erratum or a more thorough evaluation.
3) If the fix is a simple erratum, then one is constructed by the CLAs and submitted to the current WG
Bureau for approval.
3a) If a more complicated evaluation is required, then the current Chair appoints a Review Team
containing, as a minimum, two experts who were not involved in drafting the chapter, plus at least
one CLA or Lead Author (LA) from the chapter with the error, and charges that Review Team with
proposing, within one month’s time, a simple erratum statement. The Chair then submits this to
the relevant WG Bureau for approval.
2a) If the CLAs of the chapter containing the putative error feel there is no error, then the current Chair
appoints, within two weeks, an Initial Review Group of three Bureau members and/or CLAs/LAs from
the current assessment to analyze the text in question and see if they agree with the CLAs of the
chapter with the putative error. The response from the Initial Review Group is due in two weeks.
4) If the Initial Review Group agrees that there was no error, then the CLAs of the chapter with the
putative error are tasked with preparing, within two weeks, a brief document, to be posted on the
IPCC and WG web sites, explaining why the text in question was in fact not an error.

1

4a) If the Initial Review Group feels there is an error, they ask the current Chair to appoint, within one
month, an Independent Review Committee. This committee should consist of at least three experts
not involved in drafting the chapter with the putative error and not involved as a Bureau Member,
CLA, or LA on the assessment with the putative error or the current assessment.
5) If the Independent Review Committee decides there is no error, then the CLAs of the chapter with
the putative error are tasked with preparing, within two weeks, a brief document, to be posted
on the IPCC and WG web sites, explaining why the text in question was in fact not an error.
5a) If the Independent Review Committee decides that there is an error, they are tasked with
providing, within one month, an erratum statement. If the erratum is approved by the current
WG bureau, Co-chairs, and Chair, then the Chair makes a decision on whether the erratum
can be posted at that point or needs to be approved by the plenary.
1a) If the putative error is in a Synthesis Report, then the Chair and Co-chairs (from the assessment cycle with
the putative error) and CLAs of the chapter that was the source of the underlying information are
approached and asked whether they agree that there is an error.
6) If all agree that there is an error, then the Chair (from the assessment cycle with the putative error) is asked
to decide whether the fix requires a simple erratum or a more thorough evaluation
7) If the fix is a simple erratum, then one is constructed by the Chair (from the assessment cycle with the
putative error) and submitted to the current Bureau for approval.
7a) If a more complicated evaluation is required, then the current Chair appoints a Review Team
containing, as a minimum, two experts who were not involved in drafting the chapter, plus at least
one CLA or LA from the chapter with the underlying information, and charges that review team
with proposing, within one month’s time, a simple erratum statement. The current Chair then
submits this to the current Bureau for approval.
6a) If any of the Chair and Co-chairs (from the assessment cycle with the putative error), and the CLAs of the
chapter that was the source of the underlying information feel there is no error, then the current Chair
appoints, within two weeks, an Initial Review Group of three Bureau members and/or CLAs/LAs from
the current assessment to analyze the text in question and see if they agree that there was no error.
The response from the Initial Review Group is due in two weeks.
8) If the Initial Review Group agrees that there was no error, then the Chair (from the assessment cycle
with the putative error) is tasked with preparing, within two weeks, a brief document, to be posted
on the IPCC and WG web sites, explaining why the text in question was in fact not an error.
8a) If the Initial Review Group feels there is an error, they ask the current Chair to appoint, within one
month, an Independent Review Committee. This committee should consist of at least three experts
not involved in drafting the chapter with the putative error and not involved as a Bureau Member,
CLA, or LA on the assessment with the putative error or the current assessment.
9) If the Independent Review Committee decides there is no error, then the Chair (from the
assessment cycle with the putative error) is tasked with preparing, within two weeks, a brief
document, to be posted on the IPCC and WG web sites, explaining why the text in question
was in fact not an error.
9a) If the Independent Review Committee decides that there is an error, they are tasked with
providing, within one month, an erratum statement. If the erratum is approved by the current
WG bureau, Co-chairs, and Chair, then the current Chair makes a decision on whether the
erratum can be posted at that point or needs to be approved by the plenary.
If any of the individuals identified as playing leading roles on behalf of author teams of previous reports are not
available, then the current Chair will identify an individual or individuals best qualified to play those roles.

2

Appendix 4

2

Guidance Notes for Lead Authors of the
IPCC Fifth Assessment Report on Consistent Treatment of Uncertainties

3

Draft

4

The Panel requested the Co‐Chairs of Working Group I, II and III to present the final document to the Panel at its next Session.

5

These guidance notes are intended to assist Lead Authors of the Fifth Assessment Report (AR5)
in the consistent treatment of uncertainties across all three Working Groups. These notes define a
common approach and calibrated language that can be used broadly for developing expert
judgments and for evaluating and communicating the degree of certainty in findings of the
assessment process. These notes refine background material provided to support the Third and
Fourth Assessment Reports [1] and [2]; they represent the results of discussions at a CrossWorking Group Meeting on Consistent Treatment of Uncertainties convened in July of 2010 [3].
They also address key elements of the recommendations made by the 2010 independent review
of the IPCC by the InterAcademy Council [4]. Alternative approaches in the literature can be
used, but should be related to the approach outlined here. Each Working Group will supplement
these notes with more specific guidance on particular issues consistent with the common
approach given here.

1

6
7
8
9
10
11
12
13
14
15
16
17
18

The AR5 will rely on two metrics for communicating the degree of certainty in key findings:

19
20

•

21
22
23
24

•

Confidence in the validity of a finding, based on the type, amount, quality, and
consistency of evidence (e.g., mechanistic understanding, theory, data, models, expert
judgment), and the degree of agreement. Confidence is expressed qualitatively.
Quantified measures of uncertainty in a finding expressed probabilistically (based on
statistical analysis of observations or model results, or expert judgment).

25
26
27
28
29
30
31
32
33
34
35

In order to develop their key findings, author teams should evaluate the associated evidence and
agreement. Depending on the nature of the evidence evaluated, teams have the option to quantify
the uncertainty in the finding probabilistically. In most cases, author teams will present either a
quantified measure of uncertainty or an assigned level of confidence. It is important for author
teams to develop findings that are general enough to reflect the underlying evidence but not so
general that they lose substantive meaning. For findings (effects) that are conditional on other
findings (causes), consider independently evaluating the degrees of certainty in both causes and
effects, with the understanding that the degree of certainty in the causes may be low. In
particular, this approach may be appropriate for high-consequence conditional outcomes with a
high degree of certainty.

36
37
38
39
40
41
42
43

Sound decision making that anticipates, prepares for, and responds to climate change depends on
information about the full range of possible consequences and associated probabilities. Such
decisions often include a risk management perspective. Because risk is a function of probability
and consequence, information on the tails of the distribution of outcomes can be especially
important. Low-probability outcomes can have significant impacts, particularly when
characterized by large magnitude, long persistence, broad prevalence, and/or irreversibility.
Author teams are therefore encouraged to provide information on the tails of distributions of key

1

1
2

variables, reporting quantitative estimates when possible and supplying qualitative assessments
and evaluations when appropriate.

3
4
5

Treat issues of uncertainty

6
7
8
9
10
11
12
13

1. At an early stage, consider approaches to communicating the degree of certainty in key
findings in your chapter. Identify key findings as they emerge and give attention to
evaluating confidence and quantifying uncertainties in them. Determine the areas in your
chapter where a range of views may need to be described, and those where the author team
may need to develop a finding representing a collective view. Agree on a carefully
moderated and balanced process for doing this well in advance of actually confronting these
issues in a specific context.

14
15
16
17
18
19
20

2. Be prepared to make expert judgments in developing key findings, and explain those
judgments by providing a traceable account by describing in the chapter text your evaluation
of relevant evidence and agreement. Such a description may include standards of evidence
applied, approaches to combining or reconciling multiple lines of evidence, conditional
assumptions, and explanation of critical factors. When appropriate, consider using formal
elicitation methods to organize and quantify these judgments [5].

21
22
23
24
25
26
27
28
29
30

3. Be aware of a tendency for a group to converge on an expressed view and become
overconfident in it [6]. Views and estimates can also become anchored on previous versions
or values to a greater extent than is justified. One possible way to avoid this would be to ask
each member of the author team to write down his or her individual assessments of the level
of uncertainty before entering into a group discussion. If this is not done before group
discussion, important views may be inadequately discussed and assessed ranges of
uncertainty may be overly narrow [7]. Recognize when individual views are adjusting as a
result of group interactions and allow adequate time for such changes in viewpoint to be
reviewed.

31
32
33
34
35

4. Be aware that the way in which a statement is framed will have an effect on how it is
interpreted [8]. (A 10% chance of dying is interpreted more negatively than a 90% chance of
surviving.) Avoid value-laden statements, and consider complementary statements (e.g.,
chances of dying and of surviving).

36
37
38
39

5. Consider that, in some cases, it may be appropriate to describe findings for which the
evidence and understanding are overwhelming as statements of fact without using
uncertainty qualifiers.

40
41
42

Review the information available

43
44
45
46

6. Consider all plausible sources of uncertainty. Experts tend to underestimate structural
uncertainty arising from incomplete understanding of or competing conceptual frameworks
for relevant systems and processes [6]. Consider previous estimates of ranges, distributions,
2

1
2

or other measures of uncertainty, their evolution, and the extent to which they cover all
plausible sources of uncertainty.

3
4
5
6
7

7. Assess issues of uncertainty and risk to the extent possible. When probabilistic approaches
are available, consider ranges of outcomes and their associated probabilities with attention to
outcomes of potential high consequence. Additional value can come from information that
supports robust decisions for a wide range of climate and socioeconomic futures [9].

8
9
10

Evaluate and communicate at the appropriate level of precision

11
12
13
14
15
16

The following process and language should be applied to evaluate and communicate the degree
of certainty in key findings. Paragraph 8 explains the basis of confidence in terms of level of
evidence and degree of agreement. Paragraph 9 defines the confidence scale. Paragraph 10
discusses quantified measures of uncertainty. Finally, paragraph 11 provides criteria for
communication of uncertainty at different levels of precision.

17
18
19
20
21
22
23

8. Consider the following dimensions for evaluating the validity of a finding: the type, amount,
quality, and consistency of evidence (summary terms: “limited,” “medium,” or “robust”), and
the degree of agreement (summary terms: “low,” “medium,” or “high”). Generally, evidence
is most robust when there are multiple, consistent independent lines of high-quality evidence.
Provide a traceable account describing your evaluation of evidence and agreement in the text
of your chapter.

24
25

•

For findings with high agreement and robust evidence, present a level of confidence or a
quantified measure of uncertainty.

•

For findings with high agreement or robust evidence, but not both, assign confidence or
quantify uncertainty when possible. Otherwise, assign summary terms for your evaluation
of evidence and agreement.

•

For findings with low agreement and limited evidence, assign summary terms for your
evaluation of evidence and agreement.

•

In any of these cases, the degree of certainty in findings that are conditional on other
findings should be evaluated and reported independently.

26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

9. A level of confidence is expressed using five qualifiers “very low,” “low,” “medium,”
“high,” and “very high.” It is used to synthesize author teams’ judgments about the validity
of findings as determined through evaluation of evidence and agreement. Figure 1 depicts
summary statements for evidence and agreement and their relationship to confidence. There
is flexibility in this relationship; for a given evidence and agreement statement, different
confidence levels could be assigned, but increasing evidence and agreement is correlated
with increasing confidence. Confidence cannot necessarily be assigned for all combinations
of evidence and agreement in Figure 1 (see paragraph 8). Presentation of findings with “low”
and “very low” confidence should be reserved for areas of major concern, and the reasons for
3

1
2
3
4

their presentation should be carefully explained. Confidence should not be interpreted
probabilistically, and it is distinct from “statistical confidence.” Additionally, a finding that
includes a probabilistic measure of uncertainty does not require explicit mention of the level
of confidence associated with that finding if the level of confidence is “high” or “very high.”

5
6
7

8
9
10
11

Figure 1: A depiction of evidence and agreement statements and their relationship to confidence.
Confidence increases towards the top-right corner as suggested by the increasing strength of shading.

12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

10. Likelihood, as defined in Table 1, provides one method of describing quantified uncertainty
with calibrated language. It can be used to express a probabilistic estimate of the occurrence
of a single event or of an outcome, e.g., a climate parameter, observed trend, or projected
change lying in a given range. Likelihood may be based on statistical or modeling analyses,
elicitation of expert views, or other quantitative analyses. The categories defined in this table
can be considered to have “fuzzy” boundaries. A statement that an outcome is “likely” means
that the probability of this outcome can range from ~66% (fuzzy boundaries implied) to
100% probability. This implies that all alternative outcomes are “unlikely” (0%-33%
probability). When there is sufficient information, it is preferable to specify the full
probability distribution or a probability range (e.g. 90-95%) without using the terms in Table
1. “About as likely as not” should not be used to express a lack of knowledge – see paragraph
8 for that situation. Additionally, there is evidence that readers may adjust their interpretation
of this likelihood language according to the magnitude of perceived potential consequences
[10].

28

Table 1. Likelihood Scale.

29

Term
Virtually certain
Very likely
Likely
About as likely as not
Unlikely
Very unlikely
Exceptionally unlikely

Likelihood of the outcome
99-100% probability
90-100% probability
66-100% probability
33 to 66% probability
0-33% probability
0-10% probability
0-1% probability
4

1
2
3
4
5
6
7

11. Characterize key findings using calibrated uncertainty language that conveys the most
information to the reader, based on the criteria (A-F) below [11]. These criteria provide
guidance for selecting among different alternatives for presenting uncertainty, recognizing
that in all cases it is important to include a traceable account of relevant evidence and
agreement in your chapter text. The criteria given below describe how to report an assessed
finding regarding a variable (e.g., a measured, simulated, or derived quantity or its change).

8
9
10
11
12
13

A. A variable is ambiguous, or the processes determining it are poorly known or not
amenable to measurement: Confidence should not be assigned; assign summary terms for
evidence and agreement (see paragraph 8). Explain the governing factors, key indicators,
and relationships. If a variable could be either positive or negative, describe the preconditions or evidence for each.

14
15
16
17
18

B. The sign of a variable can be identified but the magnitude is poorly known: Assign
confidence when possible; otherwise assign summary terms for evidence and agreement
(see paragraphs 8 and 9). Explain the basis for this confidence evaluation and the extent
to which opposite changes would not be expected.

19
20
21
22
23
24

C. An order of magnitude can be given for a variable: Assign confidence when possible;
otherwise assign summary terms for evidence and agreement (see paragraphs 8 and 9).
Explain the basis for estimates and confidence evaluations made, and indicate any
assumptions. If the evaluation is particularly sensitive to specific assumptions, then
evaluate confidence in those assumptions.

25
26
27
28
29
30
31

D. A range can be given for a variable, based on quantitative analysis or expert judgment:
Assign likelihood or probability for that range when possible; otherwise only assign
confidence (see paragraphs 8-10). Explain the basis for the range given, noting factors
that determine the outer bounds. State any assumptions made and estimate the role of
structural uncertainties. Report likelihood or probability for values or changes outside
the range, if appropriate.

32
33
34
35
36
37
38
39

E. A likelihood or probability can be determined for a variable, for the occurrence of an
event, or for a range of outcomes, e.g., based on multiple observations, model ensemble
runs, or expert judgment: Assign a likelihood for the event or outcomes, for which
confidence should be “high” or “very high” (see paragraphs 8-10). In this case, the level
of confidence need not be explicitly stated. State any assumptions made and estimate the
role of structural uncertainties. Consider characterizing the likelihood or probability of
other events or outcomes within the full set of alternatives.

40
41
42
43
44
45

F. A probability distribution or a set of distributions can be determined for the variable
either through statistical analysis or through use of a formal quantitative survey of expert
views: Present the probability distribution(s) graphically and/or provide a range of
percentiles of the distribution(s), for which confidence should be “high” or “very high”
(see paragraphs 8-10). In this case, the level of confidence need not be explicitly stated.

5

1
2

Explain the method used to produce the probability distribution(s) and any assumptions
made, and estimate the role of structural uncertainties.

3
4
5
6

In summary, communicate uncertainty carefully, using calibrated language for key
findings, and provide traceable accounts describing your evaluations of evidence and
agreement in your chapter.

7
8
9
10

References

11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

1. Moss, R., and S. Schneider. 2000. Uncertainties, in Guidance Papers on the Cross Cutting Issues
of the Third Assessment Report of the IPCC, edited by R. Pachauri, T. Taniguchi, and K. Tanaka,
Intergovernmental Panel on Climate Change (IPCC), Geneva.
2. IPCC, “Guidance Notes for Lead Authors of the IPCC Fourth Assessment Report on Addressing
Uncertainties,” available at:
http://www.ipcc.ch/publications_and_data/publications_and_data_supporting_material.htm
3. Manning, M.R., M. Petit, D. Easterling, J. Murphy, A. Patwardhan, H-H. Rogner, R. Swart, and
G. Yohe (Eds). 2004. IPCC Workshop on Describing Scientific Uncertainties in Climate Change
to Support Analysis of Risk and of Options: Workshop report. Intergovernmental Panel on
Climate Change (IPCC), Geneva.
4. InterAcademy Council. 2010. Climate Change Assessments, Review of the Processes and
Procedures of the IPCC, available at: http://reviewipcc.interacademycouncil.net/
5. Morgan, M.G., H. Dowlatabadi, M. Henrion, D. Keith, R. Lempert, S. McBride, M. Small and T.
Wilbanks. 2009. Best Practice Approaches for Characterizing, Communicating, and Incorporating
Scientific Uncertainty in Climate Decision Making. U.S. Climate Change Science Program.
Synthesis and Assessment Product 5.2.
6. Morgan, M.G., and M. Henrion. 1990. Uncertainty: A Guide to Dealing with Uncertainty in
Quantitative Risk and Policy Analysis, Cambridge University Press, Cambridge, UK. (See
particularly chapter 6 “Human judgment about and with uncertainty”.)
7. Straus, S. G., A.M. Parker, J. B. Bruce and J. W. Dembosky. 2009. The group matters: A review
of the effect of group interaction on processes and outcomes in analytic teams. RAND Working
Paper WR-580-USG.
8. Kahneman, D. and A. Tversky. 1979. Prospect theory: an analysis of decision under risk.
Econometrica 47, 263-291.
9. Lempert, R. J., S.W. Popper, and S.C. Bankes. 2003. Shaping the Next One Hundred Years: New
Methods for Quantitative Long-Term Policy Analysis. RAND Corporation; and Lempert, R. J.
and M. E. Schlesinger. 2000. Robust strategies for abating climate change. Climatic Change 45,
387-401.
10. Patt, A. G. and Schrag, D. 2003. Using specific language to describe risk and probability.
Climatic Change 61, 17-30 (2003); and Patt, A. G. and S. Dessai. 2004. Communicating
uncertainty: lessons learned and suggestions for climate change assessment. Comptes Rendu
Geosciences 337, 425-441.
11. Kandlikar, M., J. Risbey, and S. Dessai. 2005. Representing and Communicating Deep
Uncertainty in Climate Change Assessments, Comptes Rendu Geosciences 337, 443-451.

46

6